Job description

Position: Senior DataStage and Azure Data Factory Developer Location: Hybrid - Birmingham, AL, Memphis, TN, Raleigh, NC, Charlotte, NC Job Description: Join our dynamic team as a Senior DataStage and Azure Data Factory Developer, where you will leverage your extensive experience in data engineering to lead projects that empower data-driven decisions. Key Responsibilities: • Lead data engineering projects and manage teams with 6+ years of industry experience, including at least 2 years in a technical lead role. • Utilize 3+ years of hands-on expertise with Azure Data Factory, Azure SQL Database, and a variety of Azure data services. • Drive the migration process of ETL workflows from DataStage to Azure Data Factory. • Design, develop, and maintain ETL pipelines for extracting, transforming, and loading data into Databricks. • Implement best practices for data quality checks, lineage, and governance within the Databricks environment. • Employ Azure DevOps and CI/CD methodologies for automating deployments and managing data pipelines and infrastructure. • Utilize programming languages such as Python and SQL effectively. • Manage SQL and NoSQL databases, especially those within the Azure ecosystem. • Apply your knowledge of data warehousing concepts with tools like Azure Synapse Analytics. • Ensure compliance with Azure data governance, security, and best practices. • Maintain a deep understanding of the Azure Cloud Platform and engineering infrastructure setup. Preferred Qualifications: • Hands-on experience with Python and PySpark, particularly in Databricks Notebooks and Workspace. • Capable of architecting and designing Databricks deployment strategies along with experience in data processing, transformation, and integration. • Strong grasp of distributed computing principles and data processing frameworks. • Comprehensive knowledge of Databricks tools, including Delta Tables, Delta Lake Tables, Unity Catalog, and serverless execution setups. We invite you to bring your skills and join us in developing innovative data solutions. Apply now!

Requirements