Principal Data Engineer (IC5/IC4)

Yahoo Mail is the ultimate consumer inbox with hundreds of millions of users. It's the best way to access your email and stay organized from a computer, phone or tablet. With its beautiful design and lightning fast speed, Yahoo Mail makes reading, organizing, and sending emails easier than ever. Location: Dublin, Ireland (Hybrid - 2 days/week on-site). Applicants must have the legal right to work in Ireland at the time of application.

A little about Us - As a global email provider, Yahoo Mail is the ultimate Consumer Inbox with over 220 million monthly active users. We create technology that changes the internet while handling billions of inbound connections per day to manage trillions of messages requiring petabytes of efficient storage. Our vision is to be the best consumer email platform helping users manage the business of their lives. We are currently 100% committed to a native public cloud architecture.

About the team

The Mail Intelligence Org develops intelligent capabilities at scale to uncover interests, reveal habits, and personalize user journeys across the Yahoo ecosystem. We manage billions of messages using advanced backend systems and algorithms, including NLP, GenAI, and ML. Our infrastructure extracts information and integrates diverse data sources to amplify key insights. We seek innovative, entrepreneurial engineers who value high standards, code quality, and world class architectural expertise.

A lot about YOU

AI Forward Mindset: You view AI as an essential partner in increasing your impact, with a demonstrated commitment to experimenting with emerging tools to accelerate delivery.

Architectural Depth: You possess a strong foundation in computer science and data engineering, with a proven track record of managing complex cloud native architectures at scale.

Technical Mastery: You are highly proficient in one object oriented programming language (Python / Java preferred), SQL, Spark (PySpark / Java Spark preferred). Architecting, designing and building batch and streaming data pipelines. Hadoop is a big plus.

Data Integrity Focus: You maintain uncompromising standards for data quality, built data quality frameworks, architect systems that prevent failures before they occur.

Collaborative Architect Mindset: You excel in working across global time zones, providing thought leadership and technical guidance to multidisciplinary project teams.

Your Responsibilities

Make an impact on Y! Mail Intelligence mission: lead the design, build and evolution of data solutions and pipelines that are foundational to AI/ML powered personalization and assistive intelligence, scale to 220+ million monthly active users, are robust, self healing, and leverage AI driven diagnostics to minimize manual recovery efforts.

Automated Data Governance: Establish and implement automated frameworks for data quality and compliance, ensuring integrity across petabyte scale datasets.

Technical Leadership and Collaboration: Primary technical liaison between Dublin and US based cross functional teams to align data strategies and architectural roadmaps.

AI Driven Efficiency: Proactively identify and automate repetitive engineering tasks by integrating AI tools into the development lifecycle, replacing manual workflows with intelligent automation.

Advanced Performance Optimization: Lead the continuous improvement of data workflows, balancing cost, quality, and speed through automated testing and performance tuning.

Strategic Tradeoff Management: Exercise sophisticated judgment in managing system complexity, exercising an AI first mindset to solve large scale data challenges.

Qualifications

• Educational Requirement: Bachelor's degree in Computer Science, Data Engineering, or a related field (must have).
• Experience: 8+ years of professional experience in data engineering or big data processing.
• Cloud Expertise: Deep proficiency with public cloud platform (bonus points for Google Cloud Platform - GCP proficiency).
• Big Data Stack: Expert level knowledge of stream/batch processing frameworks and data orchestration (Composer/Airflow).
• AI Proficiency: Demonstrated ability to leverage AI coding assistants (e.g., GitHub Copilot, Cursor) and LLMs (e.g., Claude, Gemini) to accelerate development velocity and optimize system architecture.
• Problem Solving: Proven ability to solve complex, multifaceted problems with innovative, scalable solutions.
• Communication: Excellent verbal and written communication skills, with the ability to influence technical direction across international teams.

Preferred Qualifications

• Prior experience with email systems or high volume messaging platforms.
• Advanced experience with DataProc, Dataflow, Composer and Looker Studio for intelligent data visualization.
• Master's degree in a technical field.

Yahoo is proud to be an equal opportunity workplace. All qualified applicants will receive consideration for employment without regard to age, race, gender, color, religion, national origin, sexual orientation, gender identity, veteran status, disability or any other protected category. Yahoo will consider qualified applicants with criminal histories in a manner consistent with applicable law.

Yahoo is dedicated to providing an accessible environment for all candidates during the application process and for employees during their employment.