Job description

Overview We are seeking a skilled and experienced Cloud Azure Databricks Architect to join our dynamic team. The ideal candidate will have extensive experience in building, managing, and optimizing data pipelines and solutions on Azure Databricks. They should also possess strong expertise in PySpark, Python, SQL, Unity Catalog, and excellent communication skills. Key Responsibilities Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. Implement and manage Unity Catalog for data governance, access control, and metadata management. Create naming standards, guidelines, practical design patterns and frameworks for data processing. Optimize data pipelines for performance, scalability, and cost-effectiveness. Create documentation for data workflows, processes, and solutions. Communicate effectively with stakeholders to provide project updates and insights. Microservices & API Design: Design scalable, loosely coupled architectures. Data & AI Strategy: Integrate data management and AI/ML components into architectural designs. Required Skills and Qualifications 10+ years of experience in data engineering or a related field. Strong hands-on experience with Azure Databricks and other Azure native services including Purview, Synapse and Power BI. Proficiency in PySpark and Python for large-scale data processing. Experience with Unity Catalog for managing data governance and access control. Understanding of data modelling concepts and ETL/ELT workflows. Excellent problem-solving and analytical skills. Strong communication skills, both written and verbal, to collaborate with technical and non-technical stakeholders. Seniority level Mid-Senior level Employment type Full-time Job function Engineering and Information Technology Industries Software Development •

Requirements