SC Cleared Data Engineer

Job Title: Data Engineer - Active SC Clearance Required

Location: Leeds, 1x per week

Duration: 3 months with strong chance of extension

Contract: Outside IR35

*Current active SC Clearance is essential. Applications from candidates who are only eligible for SC Clearance cannot be considered.*

This is a great opportunity for an experienced Data Engineer to support the delivery of a greenfield Azure cloud data platform for a major transformation programme. This contract will see you building a modern Azure data platform that brings together complex data from multiple systems, improving data quality, automation and reporting while helping create a single, trusted view of customer information.

Key Responsibilities

• Design, build and optimise Azure data pipelines using Azure Data Factory and Azure Databricks
• Develop data ingestion solutions using Change Data Capture and Managed File Transfer
• Build canonical data models, conformed dimensions and fact tables
• Deliver curated Gold-layer datasets for reporting, analytics and AI use cases
• Automate data validation and quality assurance processes, replacing manual checks
• Configure Unity Catalog, clusters, pipelines and data contracts
• Contribute to data governance, security, lineage and platform design
• Collaborate on solution design, code reviews, testing and defect resolution

Essential Skills & Experience

• Current active SC Clearance (mandatory)
• Strong commercial experience with Azure Databricks, Azure Data Factory and Azure data engineering
• Advanced Python (including Pandas) and SQL
• Hands-on experience with Unity Catalog
• Experience designing data warehouses, including slowly changing dimensions (SCD) and dimensional modelling
• Knowledge of CDC, data ingestion patterns and governed data platforms
• Understanding of data governance, security and lineage
• Experience within regulated or public sector environments is advantageous
• Familiarity with Power BI is beneficial but not essential