Job description

Job Description The Data Engineer Lead will play a pivotal role in driving data and transformation strategies, enhancing operational deliverables. Key Responsibilities • Gather, document, and promote data engineering best practices to ensure high-quality data standards. • Design and manage algorithms and pipelines in collaboration with diverse teams to address business needs. • Conduct optimizations and root cause analysis for continuous improvement, partnering with business and IT stakeholders to resolve technical issues. • Collaborate with the business community to identify and prioritize data requirements, supporting strategic initiatives and operational goals. Requirements • Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or a related quantitative field (certification in Databricks Data Engineering is an advantage). • At least 8 years of experience/knowledge in designing, building, and optimizing data pipelines and datasets. • At least 3 years of experience working in Azure and Databricks technology. • Working knowledge of SQL, relational databases, query authoring, and performing root cause analysis on internal and external data processing issues. • Understanding of processes supporting data transformation, data structures, metadata, dependency, and workload management. • Familiarity with big data tools: Hadoop, Spark, etc., relational SQL, and NoSQL databases. • Knowledge of object-oriented/object function scripting languages: Python, Java, etc.

Requirements