Job description

Overview We are seeking an experienced Azure Databricks Solution Architect who is passionate about designing and implementing scalable data solutions. In this role, you will leverage your expertise in Azure Databricks to create efficient data pipelines, manage data lakes, and ensure high data quality, aligning with our organizational business requirements. Key Responsibilities • Solution Design: Develop and implement robust data architecture using Azure Databricks, including data ingestion and transformation pipelines, while optimizing ETL processes utilizing PySpark, Scala, or Spark SQL. • Big Data Processing: Design and enhance ETL/ELT pipelines using Apache Spark, Python, Scala, and SQL, with an emphasis on data cleansing and transformation. • Data Technologies: Configure, manage, and optimize Azure Databricks clusters for performance and cost efficiency; adeptly handle big data in Azure Data Lake and other storage solutions. • Technical Leadership: Mentor development teams, data engineers, and analysts on best practices for data modeling and optimization within the Databricks ecosystem. • Data Integration: Build data integration solutions to connect various data sources, both on-premises and cloud-based. • Performance Optimization: Monitor and enhance data processing performance on Databricks clusters, ensuring scalability and efficiency. • Security and Governance: Implement security measures and access controls within Azure Databricks to safeguard sensitive data, while ensuring compliance with governance best practices. • Collaboration: Liaise with cross-functional teams to translate business requirements into effective technical solutions. • Compliance: Adhere to company policies and regulatory frameworks, particularly regarding Anti-Money Laundering laws and compliance-related mandates. Supervisory Responsibilities: Lead data teams and manage complex projects to ensure successful implementation of architectural solutions. Qualifications • Bachelor's degree and 8+ years of relevant experience, with a strong preference for credit union or financial institution backgrounds. • Extensive experience in big data architecture and cloud solutions, primarily within Azure. • Proficient in Azure services including Azure Data Factory, Azure Data Lake, Azure Synapse, Azure SQL, and Apache Spark. • Expertise in programming with Databricks, Spark SQL, PySpark, Scala, and familiarity with DAX programming for Power BI. • Solid understanding of data modeling principles and experience with both NoSQL and relational databases. Other Qualifications: • Strong verbal and written communication skills for effective collaboration with clients and colleagues. • Ability to manage multiple projects and deliver results while maintaining a positive team atmosphere. • Leadership skills to mentor staff and drive engagement within the organization. • Willingness to accommodate flexible work hours and occasional travel to our Southern and Northern California branches and corporate office. At BluPeak Credit Union, we pride ourselves on being an Equal Opportunity Employer. All qualified applicants are encouraged to apply for this exciting opportunity. Employment may be contingent upon an acceptable job-related background check. Please refer to Human Resources for reasonable accommodation details. Location: San Diego, CA

Requirements