Job description

Test Manager / Test Architect with Strong Azure Databricks Exp Eden Prairie, MN (Onsite – Hybrid Model) 6 months+ Job Title: Test Manager / Test Architect – Azure Databricks, Python, PySpark – ETL and Migration Specialist Job Summary: We are seeking a highly skilled and experienced Test Architect / Test Manager to drive data engineering and migration initiatives using Azure Databricks, Python, and PySpark. The ideal candidate will lead the team in testing of scalable data pipelines and oversee the migration of legacy systems to modern cloud-based architectures. Key Responsibilities: • Manage Testing of end-to-end data pipelines using Azure Databricks, Python, and PySpark. • Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements. • Ensure data quality, integrity, and governance across all migration and transformation processes. • Mentor junior engineers and provide technical leadership within the team. • Develop reusable frameworks and automation tools to streamline migration and data processing tasks. • Stay updated with the latest trends and best practices in cloud data engineering and migration. Required Skills & Qualifications: • Bachelor’s or Master’s degree in Computer Science, Engineering, or related field. • 12+ years of experience in data engineering with expertise in Python and PySpark. • 3+ years of experience working with Azure Databricks and other Azure data services (e.g., Azure Data Lake, Azure Synapse). • Proven experience in ETL and data migration projects, including strategy, execution, and validation. • Strong understanding of distributed computing, data modeling, and ETL/ELT processes. • Experience with CI/CD pipelines and version control tools (e.g., Git, Azure DevOps). • Excellent problem-solving and communication skills. • Ability to lead and manage multiple projects simultaneously. Preferred Qualifications: • Azure certifications (e.g., Azure Data Engineer Associate). • Experience with other cloud platforms (AWS, GCP) is a plus. • Familiarity with Delta Lake, MLflow, and other Databricks ecosystem tools. • Experience in Agile/Scrum environments.

Requirements