Job description

Job title: Databricks SME with Data Modeling & AI Agents Expertise Location: Dallas, TX Duration: 12+ months contract with possible extension/ Fulltime Job Description: We are seeking a highly skilled Databricks Subject Matter Expert with deep expertise in data modeling and AI agents to lead our data and AI initiatives. This role combines technical leadership, hands-on development, and strategic guidance to drive our organization's data-driven transformation. Key Responsibilities: Databricks Platform Leadership • Design and implement scalable data lakehouse architectures using Databricks • Optimize Delta Lake performance and manage data lifecycle strategies • Lead Unity Catalog implementation for enterprise data governance • Develop and maintain complex ETL/ELT pipelines using Spark SQL and PySpark • Establish best practices for cluster management and cost optimization • Implement CI/CD workflows for data and ML pipelines Data Modeling & Architecture: • Design dimensional models (star/snowflake schemas) for analytics workloads • Implement medallion architecture (bronze/silver/gold) data layers • Develop data vault models for enterprise data warehousing • Create and maintain data quality frameworks and validation processes • Establish master data management strategies • Design real-time streaming data architectures AI Agents Development: • Build intelligent agents using large language models (LLMs) • Implement RAG (Retrieval Augmented Generation) systems • Design multi-agent workflows and orchestration frameworks • Develop function calling and tool integration capabilities • Create conversational AI interfaces for business users • Implement vector databases and semantic search solutions Technical Leadership: • Mentor junior data engineers and data scientists • Conduct technical reviews and architecture assessments • Establish coding standards and development best practices • Lead proof-of-concept initiatives for emerging technologies • Collaborate with stakeholders to translate business requirements into technical solutions Required Qualifications Technical Skills: • Databricks Expertise: 3+ years hands-on experience with Databricks platform • Data Engineering: Advanced Spark SQL, PySpark, and Delta Lake proficiency • Cloud Platforms: Experience with AWS/Azure/Google Cloud Platform data services • Programming: Python, SQL, Scala (preferred) • ML Frameworks: MLflow, scikit-learn, TensorFlow/PyTorch • Data Modeling: Dimensional modeling, data vault, modern data architecture patterns • AI/ML: LLMs, vector databases, RAG implementations, agent frameworks About the Company: dhanutek

Requirements