Job description

The Core Responsibilities! - Gather technical requirements, assess compatibility, and analyze findings to provide appropriate solutions. - Design, develop, and maintain scalable data pipelines (batch and real-time) for data ingestion, processing, and transformation. - Develop and maintain data storage solutions, ensuring efficient data retrieval and storage in data Lakehouse and Warehouses. - Implement and maintain API integrations, including REST APIs, for seamless data flow between systems. - Implement and ensure data security and privacy measures, complying with industry standards and regulations. - Monitor and optimize the performance of systems and pipelines. - Improve the scalability, efficiency, and cost-effectiveness of data pipelines. - Troubleshoot and resolve data-related issues in a timely manner. Let's Talk About You! - Bachelor's degree or equivalent in the field of Computer Science or Engineering. - 5+ years of proven experience as a Data Engineer, specifically working with Microsoft Azure technologies. - Highly proficient in T-SQL, Pythonand Spark programming language. - Hands-on experience with Microsoft cloud applications (ie. Azure, Synapse, Fabric, Event Hub) for Data Engineering. - Solid understanding of data lakehouse and warehousing concepts and best practices. - Experience with data modeling and reporting. - Experience working with structured, semi-structured, and unstructured data. - Excellent written and verbal communication skills. - Excellent multi-tasking and organizational skills. Nice to Have! - Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect Expert. - Experience in DevOps pipelines and CI/CD. - Experience in writing data quality routines for cleansing data. - Experience working in the financial industry. - Experience with Agile and Waterfall SDLC methodologies. - Experience with Atlassian (Confluence and JIRA). Job Complexities! - The role of Senior Data Engineer is inherently dynamic and challenging, with each project presenting a unique set of complexities. Success in this role depends on the ability to navigate the varied data challenges and design scalable solutions. From adopting new technologies and tools to coming up with innovative solutions, each project is an adventure, highlighting how the job is always changing. Being flexible and open-minded is crucial for understanding the complexities of today’s data engineering.

Requirements