Data Engineer

Behavioural Insights Team • TRAFFIC • Lancashire, Manchester, United Kingdom • 8h ago

Available on a full time or part-time basis (39 hours or 32 hours per week/ 4-5 days) Flexible working options include flexible start and finish times and some home working. Starting salary: from £42,000 per annum FTE (depending on experience), plus benefits and bonus. Based in Central London (Blackfriars) and Manchester. We use a hybrid model with 2 days per week minimum in the office.

Are you looking for a new challenge and want to join a successful and rapidly growing organisation? Do you have a passion for enabling people to perform at their best? If so, we want to hear from you.

We are looking for a Data Engineer (Python) to join our Data Science and Technology team at BIT. As a Data Engineer you will be responsible for the technical delivery of our digital products and client solutions, working closely with our Group Product Lead and Full Stack Developer to ensure we are building and maintaining robust, scalable technology. You will play a crucial role in maintaining and enhancing Predictiv - our online research platform running at production scale - while delivering innovative data engineering solutions for client-facing projects.

This is an exciting opportunity to use your technical skills while contributing to meaningful, high-impact projects. You will gain hands-on experience with modern data engineering tools, cloud infrastructure, and collaborative data science workflows, all while working in a fast paced environment focussed on driving social impact.

About the Behavioural Insights Team (BIT)

Behavioural Insights Team (BIT) is a social purpose company. It exists to improve people's lives and communities. It works in partnership with national, regional and local governments, businesses, foundations and charities, to tackle major policy problems, often through testing and implementing simple yet powerful changes.

It has 250 staff based in 8 countries across Europe, the Americas and Asia Pacific, and has run more than 1000 projects to date, including over 700 randomised controlled trials in dozens of countries. In addition to its own interventions, it openly shares expertise and helps others build capacity and skills to apply behavioural science for social good.

What does a Data Engineer at BIT do?

Contribute to forward development and lead maintenance of Predictiv - our industry-best research platform.
Lead on the development and maintenance of Python data pipelines to collect, clean, and process data at scale, ensuring robustness, reliability, and efficiency.
Enable efficient model training and deployment.
Collaborate across teams to deliver outcome-focused, data-driven solutions and products that generate revenue and deliver social impact.
Help to build applied research infrastructure and tooling that allows our teams to perform scalable and robust research activities.
Steward our data governance processes, enabling our mission teams to make impactful use of our data assets.

What makes a great Data Engineer at BIT?

Demonstrated practical application of Python in professional settings - using the language consistently in data-centric projects.
Experience in developing and maintaining pipelines and infrastructure for products or applications, at scale.
Strong knowledge of the Python Data Science stack (e.g., pandas/polars, scikit-learn).
Ability to independently develop and maintain robust Python-based ETL/ELT data pipelines.
Ability to independently develop LLM-based tools/products (e.g., RAG workflows).
Familiarity with version control tools such as Git/GitHub.

In addition, we are particularly interested to hear from candidates who have:

Experience setting up cloud platforms (e.g., AWS, GCP, or Azure).
An understanding of DevOps concepts such as Infrastructure as Code, CI/CD, or MLOps.
Knowledge of Python-based data pipeline frameworks (e.g. Airflow, Metaflow) and Python-based product development packages (e.g. FastAPI, Streamlit).
An interest in developing or improving full stack product development skills.

Who are we and what can we offer you?

The Behavioural Insights Team is the world leader in applying behavioural science to public policy challenges in order to strengthen communities and improve lives.

We design and rigorously test new ideas across a range of areas such as health, crime and education, to establish what works and what doesn't. We have helped Tuberculosis patients take their medicine; reduced police susceptibility to cyberattacks; and helped organisations close their gender pay gap.

BIT really is a great place to work:

Generous pension scheme with 7% employer contribution
A flexible benefit pot of £1,100 to spend on our flexible benefits programme which currently includes things like the ability to buy extra annual leave at the start of the year, gym membership, cycle to work scheme etc or just take as cash.
25 days of annual leave (in addition to public holidays) for full-time employees, rising to 27 days after 2 years continuous service
We offer enhanced parental leave payments
We encourage flexible working and have a culture which enables this; for example flexible start and finish times and an IT set up which allows for easy working from home options
We have regular team lunches with inspirational guest speakers.
We're a social bunch and have a variety of interest groups including Film Club, BIT Kids for our parents & 5-a-side team to name a few.
We have a great office, which has a fabulous roof terrace overlooking the Thames river, an amazing coffee machine and fridges stocked full of soft drinks!

How to apply

We use a great platform called Applied to reduce bias in the recruitment process. You will need to answer a few work-based questions which will be anonymously marked by members of our team. Our selection is based on performance in these questions, as well as reviewing your CV to understand your experience.

Candidates with the top scoring answers will be invited to a structured interview where you get to meet some of the team. At the interview stage you will be asked questions that reflect the role and also to complete a work-based task, so you have the chance to demonstrate your skills. It's at this stage we first see your CV. We always give all applicants feedback, no matter what stage your application reaches.

Please note:

You will need to have the right to work in the UK in order to be able to undertake this role. We will ask all applicants short-listed for an interview to provide evidence of their right to work during the recruitment process. If you do not have the right to work for BIT but are the most suitable person for the role we will explore with you whether sponsorship may be an option. However, this may not be possible for all roles.

Some of the personal data you provide to Applied when you apply for a role with The Behavioural Insights Team will be passed on to The Behavioural Insights Team for the purposes of (a) reviewing your application; (b) inviting you for interview; and/or (c) offering you a job and getting you set up as an employee/contractor.For full details on how BIT collects and uses your personal data please visit our website -

If your application is not successful, any personal data provided to us by Applied (including any personal data relating to equal opportunities monitoring as set out above) will be retained by The Behavioural Insights Team for no longer than 6 months after the application deadline has passed (other than in exceptional circumstances where we may need to retain your personal data for longer), at which point your personal data will be securely deleted. We only keep your information for that period in order to allow us to comply with relevant legal obligations.

Related Jobs

AWS Cloud Engineer

e-Careers Limited • TRAFFIC • Lancashire, Manchester, United Kingdom • 3d ago

Engineer

Take your first steps towards a new and exciting career in Cloud Computing. e-Careers is a dynamic and forward-thinking training company that is dedicated to changing lives through innovative training solutions. The mark...

3d ago

Apply

Lead Software Engineer

Alexander Mann Solutions - Public Sector Resourcing • ATS • Lancashire, Manchester, United Kingdom • 3d ago

Engineer

On behalf of DWP, we are looking for a Lead Software Engineer for a 12 month (Inside IR35) contract based Hybrid in either Manchester, Newcastle, Leeds or Birmingham. The Department for Work and Pensions (DWP) is respons...

3d ago

Apply

Senior Software Engineer - Croydon, England, United Kingdom; Manchester, England, United Kingdom

Jane's Group • TRAFFIC • Lancashire, Manchester, United Kingdom • 4d ago

Engineer

Croydon, England, United Kingdom; Manchester, England, United Kingdom Janes empowers military, government, and defence leaders to act with confidence in an increasingly complex world. Our trusted defence, security, and g...

4d ago

Apply

Traffic CAD Engineer

Service Care Solutions Ltd • TRAFFIC • Lancashire, Manchester, United Kingdom • 6d ago

Engineer

A client within the Public Sector based in Greater Manchester is currently recruiting for an CAD Engineer to join their Highways Engineering Design team as soon as possible. The client is offering a full time, temporary...

6d ago

Apply

Principal Highway Structures Engineer

Wilde Consulting Engineers • TRAFFIC • Lancashire, Manchester, United Kingdom • 2w ago

Engineer

Opportunity: Principal Highway Structures Engineer Location: Manchester (Hybrid) Rate: Negotiable and Competitive Duration: 12 months IR35: Outside We require the services of a Principal Highway Structures Engineer to su...

2w ago

Apply

Remote Senior Software Engineer - Disney+

WorksHub • TRAFFIC • Lancashire, Manchester, United Kingdom • 3w ago

Engineer

At Disney Streaming we're responsible for building what aims to be the largest media streaming platform in the world, serving not just Disney+, but ESPN+, Star+, and Hulu. We foster innovation, we challenge each other, a...

3w ago

Apply

Employment Tax Director

PRO-TAX RECRUITMENT LIMITED • TRAFFIC • Lancashire, Manchester, United Kingdom • 2h ago

Employment Tax Director Manchester City Centre Salary: £80,000 - £110,000 + benefits Do you want to take your next step? Are you currently working employment tax and want to have more autonomy? Enjoy working in Mancheste...

2h ago

Apply

Volunteer - Software Developer - Virtual Reality & 3D Printing

SkilledUp Life Ltd • TRAFFIC • Lancashire, Manchester, United Kingdom • 2h ago

Developer

Volunteer - Software Developer - Virtual Reality & 3D Printing Important Disclosure Check regularly your 'Other' inbox, 'Spam' folder, and SkilledUp.Life message tab to not miss important emails following your applic...

2h ago

Apply

Interim Financial Controller

Simpson Judge Ltd • TRAFFIC • Lancashire, Manchester, United Kingdom • 8h ago

6 month FTC Central Manchester offices Hybrid working; 3 days in office and 2 days WFH per week ACCA or ACA qualification is essential Some international travel may be required Salary circa. £65,000 p.a. At least busines...

8h ago

Apply

UK - Programmatic Associate

Incubeta • TRAFFIC • Lancashire, Manchester, United Kingdom • 2d ago

Programmatic Associate About You A desire to transition to an advisory role having previously developed knowledge of DV360 in a hands-on activity management role. Prior familiarity with (or a desire to learn) CM360 as a...

2d ago

Apply