Job description

Data Engineer Role Overview We are seeking a skilled Data Engineer to join our team at Ztek Consulting. The ideal candidate will have expertise in Azure services, data management, and storage. Job Description The successful candidate will be responsible for designing and implementing scalable data storage systems using Azure services like Azure SQL Database, Azure Data Lake Storage, and Azure Synapse Analytics. This role involves ensuring the performance, cost-effectiveness, and scalability of these systems. Additionally, the candidate will develop and implement data integration processes using Azure Data Factory. Extracting data from various sources, transforming it, and loading it into data warehouses or data lakes is also a key responsibility. In this role, you will utilize big data technologies such as Apache Spark to create data processing workflows and pipelines that support data analytics and machine learning applications. You will build and maintain new and existing applications in preparation for large-scale architectural migrations within an Agile function. Maintaining the performance of data pipelines and databases to ensure efficient data processing is crucial. This role requires monitoring and optimizing data pipelines and database performance. Experience in building data and analytics platforms focused on Azure data and analytics solutions is highly valued. A strong understanding of software development and scripting languages, ETL tools, database management, and knowledge of data modeling and warehousing concepts is essential. Excellent problem-solving and troubleshooting abilities, attention to detail, and commitment to data accuracy are required. Working experience with cloud-based data migration, analytical skills, and ability to conduct root cause analysis is necessary. A background in Scrum Master/Product owner and ability to deliver results in an Agile environment is also expected. The ideal candidate will possess 5+ years of experience in the field and 3+ years of programming experience in Python/Pyspark. Knowledge of Jira, Confluence, SAFe development methodology, and DevOps is beneficial.

Requirements