Job description

Summary: We are seeking an experienced data engineer to join our team on a part-time basis to work on integrating and optimizing data systems using Azure, PySpark, SQL, Databricks, and Unity Catalog. The ideal candidate will have a strong background in cloud-based data engineering and be able to manage complex data workflows efficiently, with a focus on supply chain data. The project aims to modernize the data infrastructure, improve scalability, and optimize performance by leveraging Databricks’ unified analytics capabilities. Responsibilities • Collaborate with product teams to design end-to-end, reliable, and scalable data solutions using modern cloud technologies. • Build data acquisition pipelines from sources such as Oracle, SAP, and SQL databases. • Optimize data storage, compute performance, and cost efficiency during and after migration. • Document architecture, design decisions, and operational procedures. Must-Have Skills • Azure Databricks /Spark optimization • Python • Medallion Architecture • Azure Synapse • oracle/ Delta Lake • Azure / Cloud Infrastructure

Requirements