Job description

Title: Senior Software Engineer – Cloud & Data Architecture (Snowflake | Azure | Python) Location: NYC, NY(Hybrid) Duration: 12-24+ Months Key Responsibilities Primary duties are working in a team of data engineers and data architects to acquire data from internal and external sources, ingest into our repositories, and optimize it for data consumptions by end-users. • Implementing methods to improve data reliability and quality. • Combining raw information from different sources to create consistent formats. • Developing and testing architectures for data extraction and transformation. • Building and maintaining optimal data pipeline architectures. • Assembling large and complex data sets, as well as Big Data technologies. • Ensuring that data is readily available to data scientists, analysts, and other users. Key Skills: • 8+ years of experience in developing data acquisition, data ingestion, data transformation, file handling, encryption, sFTP protocol, ETL, ELT workloads on-prem and in the cloud. • 8+ years of experience in writing SQL queries, database warehousing: normalization, query optimization, store procedures, views, etc. • 8+ years of experience in Python, Java or .Net or C++ to build scripts, data movement applications and integration APIs. Desired Qualifications: • Experience in using: • Snowflake. • APIs, containerization, and Kubernetes. • DevOps: Gitlab CI/ CD pipelines (both YAML and UI based), Gitlab runners, Terraform for CICD. • Snaplogic. • Perl/XML. • DBT, Data Build Tool. • REST APIs, GraphQL, RPC, gRPC, and Webhooks protocols. • Linux and shell scripting. • Designing and implementing event-driven architectures, messaging middleware, and microservices. • Azure cloud services for APIs, including Azure Functions, Azure API Management, and Azure App Service. • Containerization technologies, such as Docker and Kubernetes, and their deployments options on Azure: Azure Container App, Azure App Service, Azure Container Instances, and Azure Kubernetes Service. • Best practices for code quality, testing, and performance optimization. • NoSQL databases. • AI / ML concepts and implementations. • Gitlab Duo.

Requirements