AWS Data Engineer - AirFlow

Square One Resources • TRAFFIC • Manchester, City, United Kingdom • £400 / * • 14h ago

Job Title: Airflow/AWS Data Engineer
Location: Manchester Area (3 days per week in the office)
Rate: Up to 400 per day inside IR35
Start Date: 03/11/2025
Contract Length: Until 31st December 2025
Job Type: Contract

Company Introduction:

An exciting opportunity has become available with one of our sector-leading financial services clients. They are seeking a talented AWS DevOps/Data Engineer to join their growing data engineering function. This role will play a key part in designing, deploying, and maintaining modern cloud infrastructure and data pipelines, with a focus on Airflow, AWS, and data platform automation.

Key Responsibilities:

Deploy and manage cloud infrastructure across Astronomer Airflow and AccelData environments.
Facilitate integration between vendor products and core systems, including data lakes, storage, and compute services.
Establish and enforce best practices for cloud security, scalability, and performance.
Configure and maintain vendor product deployments, ensuring reliability and optimized performance.
Ensure high availability and fault tolerance for Airflow clusters.
Implement and manage monitoring, alerting, and logging solutions for Airflow and related components.
Perform upgrades, patches, and version management for platform components.
Oversee capacity planning and resource optimization for Airflow workers and AWS resources.
Manage integrations with source control systems (GitHub, GitLab) and CI/CD pipelines.
Collaborate with AWS teams and internal stakeholders for pipeline scalability and optimization.
Design and implement process improvements, including automation, data delivery optimization, and infrastructure re-design.
Develop ETL pipelines and data workflows using AWS and SQL technologies.
Partner with cross-functional teams (product, design, and leadership) to resolve technical issues and enhance platform capabilities.
Build analytical tools and dashboards to leverage data pipelines for actionable business insights.

Key Requirements:

Proven experience as an AWS DevOps Engineer or Data Engineer in complex cloud environments.
Strong hands-on expertise with AWS services (EC2, S3, Lambda, RDS, IAM, CloudWatch, etc.).
Demonstrated experience with Airflow (Astronomer) setup, orchestration, and optimization.
Proficiency in infrastructure as code (IaC) tools such as Terraform or CloudFormation.
Experience with CI/CD pipelines and tools like Jenkins, GitHub Actions, or GitLab CI.
Solid understanding of containerization technologies (Docker, Kubernetes).
Working knowledge of Python and SQL for automation and data pipeline development.
Familiarity with monitoring and observability tools (Grafana, Prometheus, CloudWatch).
Strong grasp of data architecture principles and ETL design patterns.
Financial services or regulated industry experience (desirable).

Related Jobs

DevOps Engineer - Government Department - Inside IR35 - AZURE - SC CLEARED

Tecknuovo Ltd • Manchester, Lancashire, United Kingdom • 17h ago

Engineer

DevOps Engineer - Government Department - Inside IR35 - AZURE - SC CLEARED Tecknuovo is a digital technology consultancy with a Deliberately Different approach. This means we focus on people, not technology. We move at p...

17h ago

Apply

Azure Sentinel Cyber Security Engineer

Alexander Mann Solutions - Public Sector Resourcing • ATS • Cheshire, Warrington, United Kingdom • 21h ago

Engineer Outside IR35

On behalf of Sellafield, we are looking for an Azure Sentinel Cyber Security Engineer (outside IR35) for a 12 month contract based hybrid in Warrington - 2/3 days per week. SC Clearance is an essential requirement for th...

21h ago

Apply

SC Cleared AWS DevOps Engineer - Inside IR35

Opus Recruitment Solutions • TRAFFIC • Manchester, City, United Kingdom • £500 / * • 22h ago

Engineer

I am working with a consultancy feeding into the public sector looking for an SC cleared AWS DevOps engineer. Inside IR35 500 per day End date - 31st March 2026 Active SC clearance Onsite travel to Leeds/ Newcastle/ Manc...

22h ago

Apply

SC Cleared AWS DevOps Engineer - Inside IR35

Opus RS • Manchester, United Kingdom • £50 - £62 / hour • 1d ago

Engineer

I am working with a consultancy feeding into the public sector looking for an SC cleared AWS DevOps engineer. Inside IR35 £500 per day End date - 31st March 2026 Active SC clearance Onsite travel to Leeds/ Newcastle/ M...

1d ago

Apply

Senior GCP Cloud Infra Engineer

Stackstudio Digital Ltd. • TRAFFIC • Manchester, Not Specified, United Kingdom • £500 - £525 / * • 1d ago

Engineer

Job Title: Senior GCP Cloud Infra Engineer Location: Bristol, Leeds, Halifax, Manchester, Edinburgh, London (last preference) (Hybrid 2 days mandatory in office per week) Job Type: Contracting (Inside IR35) Duration: 6 m...

1d ago

Apply

Electronics Engineer

Certain Advantage • Manchester, Greater Manchester, GB • £40 / hour • 3d ago

Engineer Umbrella

Electronics Engineer 12 Month Contract Greater Manager £40 Umbrella Certain Advantage are working with a globally known manufacturing company, are actively seeking an Electronics Engineer to work standard days on a Sub...

3d ago

Apply

Senior GCP Cloud Infra Engineer

Stackstudio Digital Ltd. • TRAFFIC • Yorkshire, Halifax, United Kingdom • £500 - £525 / * • 5d ago

Engineer

5d ago

Apply

Software Engineer - 5-month contract: Manchester/Hybrid

Adecco • City, Manchester, United Kingdom • 5d ago

Engineer Outside IR35

Software Engineer - 5-month contract: Manchester/Hybrid Interviewing now for a Software Engineer for a 5-month project (outside IR35.) You will play an integral part of an agile team to enhance, build, and deliver produc...

5d ago

Apply

Software Engineer - 5-month contract: Manchester/Hybrid

Adecco • TRAFFIC • Manchester, City, United Kingdom • 5d ago

Engineer Outside IR35

5d ago

Apply

Embedded Software Engineer

Vantage Consulting • City, Manchester, United Kingdom • 5d ago

Engineer

Electronics Design Engineer - Contract (Manchester) An opportunity for a proven Electronics Design Engineer to contribute to advanced product development within embedded and power electronics. Project Scope: Design and t...

5d ago

Apply