Job description

• Position Type: Contract (4-6 Months, likely to extend or convert to a perm. position). • Work Location: Hybrid - Columbus, OH (Local or willing to relocate Day 1). • Work Authorization: U.S. Citizenship Required • Industry: Research and Advanced Technology. We are seeking an experienced Data Engineer to design and maintain scalable data pipelines supporting advanced AI and analytics initiatives. This role will focus on implementing the Medallion architecture within **Azure DataBricks**, optimizing data quality, performance, and readiness for machine learning and analytics use cases. Key Responsibilities • Design, develop, and maintain scalable data pipelines using Azure DataBricks. • Implement and optimize the Medallion architecture (bronze, silver, gold layers) for efficient data processing. • Collaborate with data scientists, analysts, and business stakeholders to understand and deliver on data requirements. • Ensure data quality, consistency, and governance across all layers of the data architecture. • Automate data ingestion, transformation, and enrichment processes to support AI/ML readiness. • Monitor, troubleshoot, and optimize data workflows for performance and scalability. • Document engineering workflows, configurations, and best practices for team enablement. Required Skills & Qualifications • Proven experience building data pipelines with Azure DataBricks and implementing Medallion architecture. • Strong proficiency in Python, SQL, and Spark for ETL/ELT development. • Experience preparing data for AI/ML models, including feature engineering and data normalization. • Familiarity with Delta Lake, Azure Synapse, and Azure Data Factory. • Understanding of data governance, data security, and compliance best practices. • Strong problem-solving, documentation, and communication skills. • Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience). Preferred Qualifications • Experience with MLflow, Unity Catalog, or other MLOps tools. • Knowledge of CI/CD pipelines for data engineering and deployment. • Familiarity with real-time data processing and streaming systems (e.g., Kafka, Event Hubs).

Requirements