Job description

The Sr. Data Engineer will be responsible for designing, developing, and managing ETL processes for Program Integrity data marts in Snowflake and SQL Server. This role involves building and maintaining data pipelines using Azure Data Factory (ADF), implementing DevOps practices, ensuring data accuracy, and collaborating with cross-functional teams. The position requires advanced cloud-based data engineering expertise, strong problem-solving skills, and the ability to work independently on complex technical issues. The engineer will also ensure adherence to security, compliance, and industry standards. Key Responsibilities Lead the development and management of ETL processes for Program Integrity data marts in Snowflake and SQL Server. Design, build, and maintain data pipelines using Azure Data Factory (ADF) to support ETL operations and Peer Group Profiling outputs. Implement and manage DevOps practices to ensure efficient data loading and workload balancing. Collaborate with cross-functional teams for seamless data integration and consistent data flow. Monitor, optimize, and troubleshoot data pipelines and workflows. Independently identify and resolve complex technical issues. Communicate effectively with both technical and non-technical stakeholders. Ensure data accuracy, quality, and compliance with industry standards and regulations. Lead cloud-based project delivery within defined timelines, scope, and performance benchmarks. Required Qualifications Over 5 years of hands-on expertise in cloud-based data engineering and data warehousing, with a focus on building scalable and efficient data solutions. More than 2 years of focused experience in Snowflake, including automated data ingestion workflows using Snowflake SQL procedures and Python scripting. Practical experience with Azure Data Factory (ADF), SQL Server, Blob Storage, and Snowflake. Skilled in managing various data formats (CSV, JSON, VARIANT) and performing data loading/exporting tasks using SnowSQL with ADF orchestration. Proficient with Snowpark, Apache Spark, pandas, NumPy, and Scikit-learn for data processing and modeling. Strong experience with Python and other scripting languages for automation. Proven ability to develop cloud-native ETL processes and pipelines using modern Azure and Snowflake technologies. Strong analytical thinking and independent problem-solving skills. Preferred Qualifications Certification in Azure or Snowflake. Experience with data modeling and database design. Knowledge of data governance and data quality best practices. Familiarity with other cloud platforms (AWS, Google Cloud). Certifications Email ID * This field is required Please enter valid emailId. Cell phone * This field is required Please enter valid cell phone. First Name * This field is required Please enter valid first name. Last Name * This field is required Please enter valid last name. #J-18808-Ljbffr Compunnel, Inc.

Requirements