PySpark Developer (Microsoft Fabric) - 6-Month Contract - London

PySpark Developer (Microsoft Fabric) - 6-Month Contract - London

Contracting role based in London (Hybrid, 1-3 days in office) for a 6-month engagement. You will design, build, and maintain data pipelines powering an end-to-end mastering flow for a leading global financial markets infrastructure and data provider, working across PySpark, Azure Cosmos DB, and Microsoft Fabric.

Key Responsibilities

• Build and maintain PySpark pipelines for ingestion, normalisation, bi-temporal processing, and publication of financial instrument data
• Develop and optimise Azure Cosmos DB read/write patterns and integrate with external APIs for Minting & Matching
• Implement data quality checks, comprehensive unit/integration tests, and CI/CD pipeline management via GitLab CI
• Monitor and troubleshoot production pipelines using Eventstream telemetry, KQL, and DataDog
• Translate business requirements into technical solutions, maintaining documentation and knowledge repositories

What You Will Ideally Bring

• Strong proficiency in Python and PySpark, with hands-on experience building streaming mastering pipelines
• Solid Azure experience - Cosmos DB (NoSQL), ADLS/OneLake, and Key Vault
• Strong understanding of temporal data modelling, data quality frameworks, and CI/CD practices
• Experience with lakehouses, dataflows, and semantic models within Microsoft Fabric - preferred
• Domain knowledge of financial instruments and reference data - advantageous

Contract Details

• Duration: 6 months (with potential extension)
• Day Rate: Up to £450 per day (Inside IR35)
• Location: - Hybrid London
• Start Date: ASAP