Senior Data & DevOps Engineer

Drest • TRAFFIC • Not Specified, Not Specified, United Kingdom • 7h ago

The Senior Data and DevOps Engineer is responsible for the performance, scalability, and reliability of Drest's data infrastructure and pipelines. This role ensures the efficient ingestion, transformation, and delivery of high-volume data, while also maintaining robust, cost-effective cloud operations. As a hands-on engineer, the Senior Data and DevOps Engineer is actively involved in building pipelines, defining infrastructure as code, and supporting critical systems. Reporting to the Devops Lead Engineer, they work closely with data science, backend, and the platform team to enable high-quality analytics and drive technical excellence across data and infrastructure.

What you will be accountable for:

Design, build, and maintain robust data pipelines capable of handling tens of millions of events per day in both batch and real-time processing contexts.
Manage and optimise AWS cloud infrastructure, ensuring high availability, performance, cost-efficiency, and security.
Develop infrastructure-as-code using Terraform, supporting scalable and maintainable infrastructure deployments.
Build and monitor data warehouse solutions (e.g. Redshift), ensuring data is accessible, clean, and well-modelled for analytics and product teams.
Drive system performance and operational excellence by improving observability, uptime, and deployment processes across data and platform systems.

What the Senior Data and Devops Engineerwill be responsible for:

Design, implement and maintain scalable and reliable data pipelines
Build, optimize and monitor data warehousing solutions with Redshift and other columnar data stores
Develop infrastructure-as-code using Terraform to provision and manage cloud infrastructure
Own and manage AWS-based systems, ensuring cost-effective, secure and high-performance operations
Support and enhance the data and platform stack to ensure uptime, observability, and recoverability of key systems
Collaborate with engineering and product teams to ensure data needs are met and infrastructure bottlenecks are identified early
Support analytics and reporting workflows by making data accessible, clean, and well-modeled
Implement system performance improvements and improve alerting, monitoring and release processes

5+ years of experience in data engineering, including multiple end-to-end pipeline builds
Solid experience with AWS cloud services, including S3, EC2, Lambda, RDS, Redshift, Glue, and Kinesis, Mongodb, Postgresql
Proven expertise in Terraform and infrastructure-as-code practices
Strong SQL and data modeling skills, and experience with both SQL and NoSQL data stores
Strong understanding of dbt (or equivalent) and Tableau
Hands-on experience with Python for data processing and automation tasks
A background working in environments with high throughput data (millions of events per hour)
Understanding of best practices around security, scalability, and maintainability in cloud-native systems
Comfort working independently in a fast-paced, highly collaborative environment
Great communication skills, ability to explain complex systems clearly to technical and non-technical stakeholders
Familiarity with Docker, Kubernetes, or other container orchestration platforms
A willingness to be available outside standard working hours when needed to support critical issues or key deliveries.

It would be a bonus if you have

Experience in the gaming industry
Experience with event-driven applications
Exposure to CI/CD pipelines and automated testing for data infrastructure
Experience with additional data warehouses (BigQuery, Snowflake, etc.)

Related Jobs

Network Engineer CCNP Fortinet

Access Computer Consulting Plc • APPLICATION • London, Not Specified, United Kingdom • 38min ago

Engineer

Network Engineers Inside IR35 Fulltime onsite Heathrow Area 24/7 shift pattern Require solid Network Engineers with experience including Fortinet Firewall, Fortinet SDWAN, Cisco LAN and WIFI, Cisco Nexus, general routing...

38min ago

Apply

NPPV3 Cleared Field Engineer

Specialist Computer Centres PLC • APPLICATION • Somerset, Not Specified, United Kingdom • £230 / * • 1h ago

Engineer

NPPV3 Cleared Field Engineer - 2 months Contract - £230/day via umbrella company - onsite Somerset - SCC FLEX CONTRACT We are looking for a dedicated and hands-on Field Engineer to join our team in executing critical har...

1h ago

Apply

Graph Engineer

Harvey Nash Group • TRAFFIC • London, Not Specified, United Kingdom • 1h ago

Engineer

Contract Graph Engineer - SC Cleared Location: UK-based (Remote/Hybrid depending on project phase) Type: Contract (Inside IR35) Duration: 6 months initially, with strong potential for extension Clearance: Active SC Clear...

1h ago

Apply

Azure Data Engineer - Local Government / Public Sector

Salt Digital Recruitment • TRAFFIC • London, Not Specified, United Kingdom • 1h ago

Engineer

Job Title: Data Engineer - Azure Local Government Inside IR35 Location: Central London (Hybrid - 1-2 days on-site per week) Rate: £500/day (Inside IR35) Contract: 6 months (initial) Start: ASAP/on notice Salt's Public Se...

1h ago

Apply

PKI Engineer

Experis - ManpowerGroup • TRAFFIC • London, Not Specified, United Kingdom • 1h ago

Engineer

PKI Engineer Remote 5 months - extensions £720 per day inside ir35 - umbrella only Role overview: Lead the architecture, design, and operations of Public Key Infrastructure (PKI) solutions. Manage Hardware Security Modul...

1h ago

Apply

2nd Line IT Support Engineer - Local Government

Salt Digital Recruitment • TRAFFIC • London, Not Specified, United Kingdom • 1h ago

Engineer

Job Title: 2nd Line IT Support Engineer - Local Government Location: South-West London Contract Duration: Initial 2 months Day Rate: £250 per day (Inside IR35) Start Date: 17th March 2025 Work Arrangement: Office-based (...

1h ago

Apply

Kubernetes Engineer

Wenham Carter Ltd • APPLICATION • Not Specified, Not Specified, United Kingdom • 2h ago

Engineer

We're supporting a major IT consultancy on a search for a Kubernetes Engineer, they are engaged with a global bank on a large-scale migration project, shifting from an on-prem Hadoop-based data platform to a Kubernetes-b...

2h ago

Apply

DV Cleared Network Engineer

Damia Group Ltd • APPLICATION • London, Not Specified, United Kingdom • 2h ago

Engineer

DV Cleared Network Architect - 6 months+ - £700-£720pd Inside IR35- Hybrid working of 2/3 days on site in Vauxhall or Stratford My client are looking for a Network Engineer who will work on: . Strategic network delivery...

2h ago

Apply

Senior Delivery Engineer - Arista Spine & Leaf (Inside IR35)

WNTD • APPLICATION • London, Not Specified, United Kingdom • 2h ago

Engineer

Job Title: Senior Delivery Engineer - Arista Spine & Leaf (Inside IR35) Location: Ideally 1 day per week onsite in London Contract Rate: £490 per day (Inside IR35) Contract Type: 6months + (likely extension) Role Ove...

2h ago

Apply

Senior Golang Engineer

Salt • APPLICATION • London, Not Specified, United Kingdom • £330 - £350 / * • 2h ago

Engineer

Senior Golang Software Engineer Rate: £330-£350 per day (Inside IR35) Contract: 6-12 months Location: Hybrid - 3 days per week onsite in London or Burgess Hill We're hiring a Senior Golang Software Engineer to join a glo...

2h ago

Apply