Job description

Required Skills • Azure cloud expert • Databricks expert o Cluster management o Security (roles and permissions) • Experience with security / networking assessments and reports • Machine learning experience • Medallion architecture • Snowflake experience w/ Databricks • Terraform with Azure Databricks • Cluster/query optimization • Azure Storage Additional Skills Job Description Databricks Platform Lead Engineer Location: Onsite in Newport Beach, CA, or Columbus, Ohio, preferred 4 days/week Contract: long-term contract with potential to extend or convert Department: Data & Analytics Reports To: Director of Data Engineering / Platform Engineering Lead Overview Need Databricks Platform Lead Engineer to transform our current data science environment into a resilient, enterprise-grade platform. This role will lead the modernization of platform operations by automating manual processes, integrating services, and establishing best practices for Databricks administration and platform engineering. You’ll work closely with Machine Learning Engineers (MLEs), platform engineers, and data scientists to ensure seamless, scalable, and secure platform capabilities. Key Responsibilities • Databricks Leadership & Enablement • Guide engineering and data science teams through Databricks best practices. • Deliver comprehensive knowledge transfer (KT) to internal platform engineers. • Enable self-service capabilities for Data Science teams. • Platform Engineering & Automation • Automate service principal management and security access workflows. • Establish alerting mechanisms for system failures and anomalies. • Optimize and clean up the existing Databricks environment for scalability and maintainability. • Integration & Configuration • Connect Databricks with other tools (Azure, H2O, Snowflake) to ensure seamless interoperability. • Standardize configuration workflows and governance policies to improve reliability. • Troubleshooting & Support • Troubleshoot and resolve failures in Databricks and Azure pipelines. • Implement updates across Data Science-owned accounts and services. • Collaboration & Coaching • Collaborate with MLEs (e.g., Roxxanne and Chris) and platform engineers (e.g., Koda). • Provide hands-on support and mentorship to team members unfamiliar with Databricks. Required Qualifications • 5+ years of experience in platform engineering, cloud administration, or ML engineering. • Deep expertise in Databricks administration, architecture, and optimization. • Strong experience with Azure, Snowflake, and H2O.ai integrations. • Proven ability to automate infrastructure and security workflows. • Excellent communication and documentation skills. • Self-starter with a proactive mindset and strong problem-solving abilities. Preferred Qualifications • Experience in enterprise data platform transformation. • Familiarity with CI/CD pipelines and DevOps practices. • Prior experience in coaching or mentoring technical teams. Notes to Vendor Candidates need all of the required skills Candidates must be able to go on-site in Columbus, Ohio, or Newport Beach, CA

Requirements