Lead Data Engineer

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field; equivalent practical experience considered.

• 6+ years building and operating production data pipelines and consumer-facing data models, from source ingestion through to published data products.

• Strong hands-on Python and SQL, with the credibility to make design calls and the willingness to still write and review code.

• Hands-on experience with Databricks or a comparable Spark-based lakehouse, including Delta Lake, MERGE, incremental processing, and performance tuning.

• Deep dimensional modeling experience — grain, keys, slowly changing dimensions, facts and dimensions, conformed dimensions.

• Demonstrated technical leadership: setting standards, leading design, and raising other engineers’ work, whether or not the role carried a lead title.

• Experience owning data that other teams depend on, including handling breaking changes and production data incidents.

• Experience with orchestration (Dagster, Databricks Workflows, Airflow, or similar), Git-based collaborative development, code review, and CI/CD — Azure DevOps or comparable.

• Experience setting observability and SLA/SLO expectations for data other teams depend on, and running the incident and communication path when they are missed.

• Ability to explain trade-offs clearly to engineers, product owners, and business stakeholders, and to say no to a design that will not hold.

• Databricks certification (Data Engineer Professional or equivalent demonstrated depth).

• Unity Catalog at multi-team scale: catalogs, schemas, external locations, permissions, and lineage.

• dbt at scale on Databricks, and Python-based modeling frameworks over Delta Lake.

• Dagster and Dagster Cloud, including assets, asset checks, and branch deployments.

• Practical experience with data contracts, ODCS, or data-mesh style data product ownership.

• Experience with a declarative Python ingestion framework such as dlt (dltHub) or comparable.

• Data quality and observability tooling such as Great Expectations, Monte Carlo, or similar.

• Familiarity with MLOps practice — MLflow, model registries, and model serving — sufficient to design data products that ML systems can depend on.

• Azure and Azure DevOps.

• Financial services, insurance, or another regulated industry, including data access, lineage, and audit expectations.

• Experience introducing AI-assisted development into a team’s normal workflow in a disciplined way.

If you require an accommodation to complete the application and recruitment process due to a disability, please email . This information will be held in confidence and used only to determine an appropriate accommodation for the application and recruitment process.

Please note that the above email is solely for individuals with disabilities requesting an accommodation. General employment questions should not be sent through this process.

We are proud to be an equal opportunity employer committed to being inclusive and attracting, retaining, and growing an inclusive workforce.

Originally posted on Himalayas