Technical Lead - Data Engineering

Technical Lead - Data Engineering

The Role

Technical Lead - Data Engineering and play a critical role in driving our enterprise data platform strategy. You will lead the design, development, and delivery of modern data solutions using Snowflake, DBT, Azure/AWS, Python, Airflow, and CI/CD technologies. This role combines deep technical expertise with leadership responsibilities, guiding engineering teams, defining best practices, and ensuring the successful delivery of scalable, secure, and high-performing data platforms.

Your responsibilities:

• Lead the design and implementation of scalable and secure data platforms using Snowflake, DBT, Airflow, Python, and Azure/AWS services.
• Define technical architecture, coding standards, engineering best practices, and development frameworks for the data engineering team.
• Drive the adoption of CI/CD pipelines using GitHub Actions, Azure DevOps, Jenkins, or similar tools to automate build, test, deployment, and release management processes.
• Lead the development, optimization, and maintenance of complex ETL/ELT pipelines for large-scale data processing.
• Establish DevOps and DataOps practices to improve deployment efficiency, reliability, and operational excellence.
• Mentor and coach data engineers through technical guidance, code reviews, architecture reviews, and knowledge-sharing sessions.
• Collaborate with enterprise architects and business stakeholders to translate business requirements into scalable technical solutions.
• Own platform reliability, monitoring, performance tuning, and troubleshooting of production data pipelines.
• Implement Infrastructure as Code (IaC) using Terraform/Terragrunt to automate cloud resource provisioning.
• Drive data quality, governance, security, and compliance standards across the data ecosystem.
• Lead technical discussions, solution design workshops, and project planning activities.
• Evaluate emerging technologies and recommend innovative approaches to improve data engineering capabilities and delivery processes.
• Support Agile delivery and provide technical leadership throughout the project life cycle.

Desirable Skills/Knowledge/Experience

• Experience with Generative AI and AI-powered data engineering solutions.
• Experience with Power BI, MicroStrategy, or other BI tools.
• Knowledge of Kubernetes, Docker, and containerized deployments.
• Experience with Databricks and modern lakehouse architectures.
• Azure Data Factory, Synapse Analytics, or AWS Glue experience.
• Experience implementing DataOps frameworks and observability platforms.
• Exposure to enterprise architecture and governance frameworks.

Languages: Python (primary), SQL, Bash
Cloud: Azure, AWS
Tools: Airflow, DBT
Data: Snowflake, Delta Lake, Redis, Azure Data Lake
Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Azure Monitor

The Offer

• Day Rate: £475/day (Inside IR35)
• Contract Length: 6 months Initial (Extendable)
• Location: Central London
• Model: Hybrid (2x week on site)