LEAD AWS DATA ARCHITECT

1 hour ago

Contract

London

United Kingdom

Role: LEAD AWS DATA ARCHITECT

Location: London

Hybrid

Contract (Inside IR35)

The Role

AWS Data Hands-On Architect to build and evolve our Payments Data Platform-ingesting ISO 20022 events into an AWS Lakehouse to deliver best-in-class governance, observability, and cost optimisation, while owning the data product end-to-end across CX, Payments, and CPO, managing the backlog, SLAs, and contracts, and delivering runbooks, SLOs, and quarterly cost/quality reports.

Your responsibilities:

Data products (To-Be): Channel Ops Warehouse (~30-day high-perf layer) and Channel Analytics Lake (7+ years). Expose status and statements APIs with clear SLAs.
Platform architecture: S3/Glue/Athena/Iceberg Lakehouse, Redshift for BI/ops. QuickSight for PO/ops dashboards. Lambda/Step Functions for stream processing orchestration.
Streaming & ingest: Kafka (K4/K5/Confluent) and AWS MSK/Kinesis; connectors/CDC to DW/Lake. Partitioning, retention, replay, idempotency. EventBridge for AWS-native event routing.
Event contracts: Avro/Protobuf, Schema Registry, compatibility rules, versioning strategy.
As-Is - To-Be: Inventory APIs/File/SWIFT feeds and stores (Aurora Postgres, Kafka). Define migration waves, cutover runbooks.
Governance & quality: Data-as-a-product ownership, lineage, access controls, quality rules, retention.
Observability & FinOps: Grafana/Prometheus/CloudWatch for TPS, success rate, lag, spend per 1M events. Runbooks + actionable alerts.
Scale & resilience: Tens of millions of payments/day, multi-AZ/region patterns, pragmatic RPO/RTO.
Security: Data classification, KMS encryption, tokenization where needed, least-privilege IAM, immutable audit.
Hands-on build: Python/Scala/SQL; Spark/Glue; Step Functions/Lambda; IaC (Terraform); CI/CD (GitLab/Jenkins); automated tests.

Your Profile

Essential skills/knowledge/experience:

Streaming & EDA: Kafka (Confluent) and AWS MSK/Kinesis; Kinesis Firehose; ordering, replay, exactly-at-least-once semantics; EventBridge for event routing and filtering.
Schema management: Avro/Protobuf + Schema Registry (compatibility, subject strategy, evolution).
AWS data stack: S3/Glue/Athena, Redshift, Step Functions, Lambda; Kinesis & S3-Glue streaming pipelines; Glue Streaming; DLQ patterns.
Payments & ISO 20022: PAIN/PACS/CAMT, life cycle modelling, reconstruction, SWIFT/file channel knowledge.
Governance: Data-mass mindset, ownership, quality SLAs, access, retention, lineage.
Observability & FinOps: Build dashboards, alerts, cost KPIs; troubleshoot low throughput at scale.
Delivery: Production code, performance profiling, code reviews, automated tests, secure-by-design.

Data Architecture Fundamentals (Must-Have)

Logical data modelling: Entity-relationship diagrams, normalization (1NF through Boyce-Codd/BCNF), denormalization trade-offs; functional dependencies & anomalies.
Physical data modelling: Table design, partitioning strategies, indexes; storage patterns for OLTP vs analytics.
Normalization & design: Normalize to 3NF/BCNF for OLTP; understand when to denormalize for queries; Data Vault, star schemas.
CQRS: Read/write segregation; event sourcing; reconstruction, when CQRS is justified vs overkill.
Event-Driven Architecture (EDA): Event-first design; aggregate boundaries; pub/sub patterns; orchestration; idempotency; at-least-once delivery.
Bounded contexts & domain modelling: Anti-corruption layers, shared Kernel, published language, ubiquitous language.
Entities, value objects & repositories: Domain entity identity; immutability; repository abstraction over persistence; temporal/versioned records.
Domain events & contracts: Schema versioning (Avro/Protobuf); backward/forward compatibility; event replay; mapping domain events to Kafka topics and Aurora tables.

More jobs