Azure Databricks ArchitectUS-CA-San Diego
Job ID: Type: Regular Full-Time
# of Openings: 1
Category: Accounting/Finance
Corporate Office
Overview An Azure Databricks Architect is a technical role focused on designing and implementing scalable data solutions using Azure Databricks, acting as a subject matter expert in the platform to create efficient data pipelines, manage data lakes, and ensure data quality while leveraging the full capabilities of the Azure cloud ecosystem; essentially, they are responsible for architecting robust data solutions on Azure Databricks, considering data ingestion, processing, storage, and analysis needs within the context of an organization's business requirements.
Responsibilities Solution Design: Develop and implement data architecture using Azure Databricks, including data ingestion pipelines, data transformation logic, and data warehousing strategies. Building and optimizing ETL pipelines, particularly in Databricks using PySpark, Scala, or Spark SQL. Ability to create and manage Databricks notebooks for data exploration, pipeline development, and data visualization. Familiarity with scheduling and automating Databricks jobs.
Big Data Processing: Design and optimize ETL/ELT pipelines using Apache Spark, Python, Scala, or SQL. Experience building ETL/ELT pipelines, data cleansing, and data transformation processes.
Data Technologies: Proficiency in setting up, configuring, and managing Azure Databricks clusters, as well as optimizing performance and costs. Understanding of how to manage, process, and store big data in Azure Data Lake or other storage solutions.
Technical Leadership: Guide development teams, data engineers, and business analysts on best practices for data modeling, data processing, and optimization within the Databricks platform.
Data Integration: Design and build data integration solutions to connect diverse data sources, including on-premises systems, cloud-based applications, and APIs.
Performance Optimization: Monitor and optimize data processing performance on Databricks clusters to ensure efficient data processing at scale.
Security and Governance: Implement data security measures and access controls to protect sensitive data within the Azure Databricks environment. Knowledge of networking concepts within Azure, such as virtual networks, firewalls, private endpoints, and network security groups. Ensure compliance with security and governance best practices, including RBAC, encryption, and data masking.
Collaboration: Work closely with data engineers, data scientists, business intelligence analysts, and business stakeholders to understand requirements and translate them into technical solutions.
Compliance
Adhere to company policy and procedure, complete annual compliance training, and comply with business continuity and vendor management policy. These are crucial to the success of the organization and have regulatory compliance and/or safety and soundness for the sustainability of the credit union.
Comply with all Anti Money Laundering laws such as the Bank Secrecy Act, OFAC and USA PATRIOT Act and all related policies and procedures including accurate Member/Customer Identification Program, member due diligence, accurate OFAC match processing, and reporting unusual activities and suspected fraud.
SUPERVISORY RESPONSIBILITIES
Effectively lead data teams, manage complex data projects, and ensure proper implementation of architectural solutions.
Qualifications EDUCATION and/or EXPERIENCE
Bachelors degree and 8+ years related experience required, or equivalent combination of education and experience. Credit union or financial institution experience highly preferred.
8+ years of experience in big data architecture and cloud solutions, preferably on Azure
Experience using Databricks and expertise in Azure services such as Azure Data Factory, Azure Data Lake, Azure Synapse, Azure SQL, Azure Functions, and Apache Spark.
Dax Programming for Power BI, CSharp.net preferred.
Experience in a financial institution is highly preferred.
Experience using Correlation Keystone is preferred, but not necessary.
Proficient in Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Aure SQL Database, Azure Data Lake Storage (ADLS), and Azure Blob Storage, Databricks Notebooks, Spark SQL, PySpark, and Scala, Databricks Workspaces, Unity Catalog, and role-based access (RBAC)
Expertise in Azure data services like Azure Data Lake Storage, Azure Cosmos DB, Azure Synapse Analytics, and Azure Key Vault.
Data Modeling - Strong skills in data modeling, including star/snowflake schemas, normalizations, denormalizations, and working with NoSQL and relational databases.
Programming Languages - Proficiency in Python, SQL, Scala, or other relevant languages for data manipulation and analysis.
OTHER QUALIFICATIONS
Written and verbal communication skills which support effective communication of information to groups of clients, customers, and the general public. Ability to effectively present information and respond to questions from managers and staff. Excellent verbal and written communication skills required.
Leverage interpersonal skills to communicate with non-technical personnel at various levels within the credit union. Ability to work in a collaborative environment and establish relationships with team members, department managers, executive management, and key stakeholders.
Must be able to maintain records to document data program development and revisions. Identify and work with appropriate staff to resolve data and technical issues. Contribute to projects, which include verbal and written participation.
Must possess strong creative skills, good writing skills and a high degree of personal initiative; excellent communication and interpersonal skills, including writing reports and presentations; group facilitation skills; ability to develop professional relationships; as well as networking internally and externally. Must have good phone and e-mail etiquette, a professional presentation, and a high degree of personal initiative.
Must be organized, self-disciplined and detail-oriented with the ability to manage and juggle multiple projects. Persistent and dependable, especially in meeting deadlines; able to manage multiple competing priorities.
Must have advanced leadership skills with the ability to develop staff, provide ongoing coaching and grow individuals. Ability to develop processes and to organize information.
Proven ability to partner effectively across all levels of the organization and develop positive working relationships. Must be able to adapt well to change and be passionate about finding new ways to solve evolving business needs.
Must act as a team player with various departments throughout the organization. Ability to remain flexible and supportive as departmental needs arise and be willing to adjust priorities to meet changing demands and needs in the marketplace. Create and maintain a team atmosphere within the branch.
Must be able to work during regular business hours and occasional nights or weekends as needed to meet and support business partner and community/business networking or events. This position requires the employee to travel occasionally to Southern and Northern California Branches as well as to corporate office.
BluPeak Credit Union is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex including sexual orientation and gender identity, national origin, disability, protected Veteran Status, or any other characteristic protected by applicable federal, state, or local law.
Employment may be contingent upon BluPeak Credit Unions receipt of an acceptable and job-related background check, drug screen and credit check, as applicable and permissible by law. BluPeak Credit Union is committed to working and providing reasonable accommodation to applicants with physical, mental disabilities and sincerely held religious beliefs. For more information, contact Human Resources.
PM17
Compensation details: 00 Yearly Salary
PI6306e0865adf-5217
