Azure Databricks Solution Architect

The contract will be based in London and require onsite attendance one day per week.

Duration is 6 months and this will be inside IR35

This role will require full SC Clearance

About the Role

Are you an expert in cloud data architecture with a passion for building secure, scalable, and cutting-edge data platforms?

We are seeking an Azure Databricks Solution Architect to spearhead the design, implementation, and enablement of our secure Azure Databricks platform. In this role, you won't just build a platform; you will shape how our organization leverages data. You will align the platform with our enterprise architecture, security standards, and operating model, establishing the core reference patterns, governance controls, and onboarding frameworks that empower decentralized teams to innovate safely.

Key Responsibilities

• Platform Design & Deployment: Design and deploy Azure Databricks workspaces, clusters, and policies using secure, enterprise-grade patterns aligned with cost and security guardrails.
• Governance & Security: Implement Unity Catalog (metastores, access connectors, external locations, fine-grained RBAC) and enforce a strict least-privilege access model across multiple teams.
• Architecture Assurance: Support security and Technical Design Authority (TDA) assurance by producing high-quality architecture diagrams, control mappings, and risk mitigations.
• Data Enablement: Define reference patterns for data ingestion, transformation, and data product creation to support analytics, data engineering, and machine learning use cases.
• Standards & Lineage: Establish organizational standards for data isolation, delta sharing, and data lineage.
• Onboarding & Knowledge Transfer: Create "how-to" and "how-not-to" playbooks, define guardrails for internal teams, and provide mentoring and knowledge transfer to internal engineers and architects.
• Collaboration: Act as a key bridge between Cloud Architecture, Security, the Central Data Platform (CDP) teams, and the TDA.

Essential Skills & Experience

• Enterprise Delivery: Proven track record of designing and delivering Azure Databricks platforms within large enterprise or public sector environments.
• Deep Technical Mastery: Hands-on experience with:
• Unity Catalog (governance, catalog structure, and permissions)
• Azure Networking (VNets, Private Link, NSGs, UDRs)
• Storage & Identity: ADLS Gen2 integration and access control, alongside Entra ID (Azure AD) integration.
• Governance & Compliance: Experience navigating formal security assurance processes and architecture governance frameworks.
• Modern Data Strategy: A strong understanding of modern data platform operating models, data mesh principles, and data product concepts.

Key Deliverables (Your First Focus)

• A fully implemented, secure Databricks platform in Azure.
• Comprehensive reference architecture and detailed design documentation.
• Robust governance and RBAC models tailored for multi-team use.
• Practical onboarding guides and operational playbooks for internal teams.
• A final evaluation findings report and strategic adoption recommendation.