Job description

Databricks architect/admin/infrastructure expert Location: 100% remote (must be in the greater Seattle area) 3-6 month contract (will likely extend) This role is for someone that has depth of experience setting up Databricks instances, sizing clusters, performance tuning, data sharing between Databricks accounts, sharing Unity Catalog across Databricks instances/accounts, and other things related to AZURE infrastructure to support Databricks. So, not a Data Engineer for building data pipelines; more of a Databricks architect/admin/infrastructure expert. DevOps/Infrastructure architect type with strong Databricks experience. Need for a Databricks data platform engineer/architect who has experience with IaC (Terraform) and CICD (GitHub Actions) starting asap for 3-6 months for building our EDP platform post EDP POC . Below are the requirements: 1) Architect, design and build Enterprise Data Platform, the technology stack includes – Azure Infrastructure, ADLS Gen 2, Databricks, ADF, Collibra, PowerBI and Snowflake using best practices and compliant with Gates Foundation guidelines and policies. 1b) Serve as Databricks administrator and subject matter expert; scale and tune Databricks across the enterprise; create shareable Unity Catalog bringing together disparate Databricks solutions. 2) Develop automation for provisioning of platform components using Infrastructure as Code (IaC). 3) Create process for sizing, provisioning based on persona (for example: Data Scientist, Data Engineers, Analysts, Researchers etc.), teams or projects including managing and governance of non-production and production environments. 4) Establish CICD best practices, automation, and pipelines for code deployments. 5) Collaborate with foundation architects, engineers, cloud enablement and network teams 6) Build the data platform for data sharing with internal and external users/partners. 7) Establish platform level SLA's, backup, and DR strategy. 8) Adhere to Azure policies, create, modify as applicable based on best practices. 9) Leverage the work that has been done in the EDP POC and build out the platform in non-prod and prod environments.

Requirements