Job description

Senior Data Engineer – Azure Databricks Lakehouse - Chicago (Hybrid) We are hiring a Senior Data Engineer to design and implement enterprise-grade data solutions leveraging lakehouse and cloud technologies. This senior-level, hands-on role offers autonomy, collaboration, and leadership opportunities in developing a high-performing data ecosystem. Responsibilities: • Architect & Design scalable data solutions on the Databricks Lakehouse Platform in Azure. • Develop & Maintain production-grade ELT/ETL pipelines using Spark (PySpark or Scala), SQL, notebooks, and Delta Live Tables (DLT). • Optimize performance and cost-efficiency through Delta Lake tuning, partitioning, and compute usage enhancements. • Model data employing medallion architecture, star-schema, and other analytics frameworks. • Collaborate with data scientists, analysts, and engineers to deliver quality data products. • Mentor & Guide junior and mid-level engineers, fostering a culture of excellence. • Standardize & Govern data engineering practices with documentation and best practices. • Stay Updated on emerging cloud data architecture technologies and trends. Qualifications: • 6+ years of experience in building modern data solutions at scale. • Deep expertise in Spark (PySpark/Scala), SQL, and data processing in Databricks. • Proven success in implementing Lakehouse solutions using medallion architecture in Azure. • Experience creating metadata-driven ELT/ETL pipelines. • Knowledge of Azure Data Factory for orchestration. • Familiarity with Lambda architectures (batch + streaming). • Background in agile methodologies and CI/CD frameworks. • Bachelor's degree in CS, Engineering, or related field, or equivalent experience. • Excellent communication and documentation skills, with a collaborative approach. Preferred: • Experience in insurance or finance domains. • BI and data analysis skills supporting end-to-end platform development. • Experience with offshore or hybrid team coordination. Benefits & Culture: • Hybrid work model (1-2 days/week in Downtown Chicago). • Modern, cloud-first tech stack. • Culture emphasizing trust, collaboration, mentorship, and ownership. • Opportunities to lead initiatives and make significant impacts. If you're passionate about building innovative data platforms and value autonomy and continuous learning, we want to hear from you. #J-18808-Ljbffr

Requirements