Senior Data Engineer
Firmus Technologies
Sydney
6 days ago
Senior Data Engineer
Firmus Technologies is seeking a Senior Data Engineer to lead the design, build, and operation of a self-hosted data platform for AI infrastructure. The role involves data ingestion, storage, quality, governance, and deployment on Kubernetes. Requires expertise in Python, SQL, Kafka, Airflow, dbt, and hybrid infrastructure experience.
On-site
Full-time
Senior
Python
SQL
Salary
Not specified
Core Qualifications
Technical (Must-have)
PythonSQLKafkaAirflowdbtKubernetesHelmCI/CDSnowflakeBigQuery
Soft Skills
communication
Tools (Must-have)
KubernetesHelmAirflowdbtSnowflakeBigQueryDatabricks
Key Responsibilities
- Design and operate streaming and batch ingestion pipelines from GPU cluster telemetry, cooling sensors, power grid, and incident data.
- Design and implement storage solutions across columnar, time-series, relational, object, and vector store patterns.
- Implement data quality validation across ingestion, transformation, and serving layers.
- Build the transformation layer that produces clean, well-modelled data from raw ingested data.
- Implement data catalogue, lineage tracking, access control, and data contracts across teams.
- Deploy and operate all data platform components via infrastructure-as-code on Kubernetes.
Senior Data EngineerData EngineeringPythonSQLKafkaKubernetesAirflowdbtAI InfrastructureSydney