Centurion Consulting Group is seeking an Azure Data Factory Engineer to ensure data integrity during migration processes. The role involves data profiling, SQL query writing, and configuring data pipelines in Azure Data Factory.
Centurion Consulting Group is seeking an Azure Data Factory Engineer to collaborate with business analysts and stakeholders to ensure data integrity throughout the migration lifecycle. The role involves data profiling, cleansing, and validation, as well as writing SQL queries and configuring data pipelines in Azure Data Factory. Responsibilities • Collaborate with business analysts, data architects, and stakeholders to gather requirements and ensure data integrity throughout the migration lifecycle. • Ability to create Data mapping documents (Source to Target) by understanding business rules. • Perform data profiling, cleansing, and quality checks to ensure accuracy and completeness of migrated data. • Analyze and map source Oracle database schemas to target Dataverse tables/entities, defining transformation and cleansing logic as required. • Write and optimize SQL queries, stored procedures, and scripts for Oracle databases to support data migration and validation. • Implement data transformation processes to ensure data compatibility with Dataverse data types, relationships, and constraints. • Experience with Dataverse data types and CRUD operations. • Experience configuring data pipelines in ADF (to extract, transform, load). Experience using .csv files as sources of data for the ADF pipelines. • Experience using Oracle CDCs as sources of data for the ADF pipelines. Experience logging errors (audit framework) from an ADF pipeline to a Dataverse table. • Experience pushing data to Dataverse table(s) including necessary transformations to match target field types (i.e. Dataverse lookups, choices, date/time, etc.). • Experience configuring setting for pipelines to improve concurrency/performance (i.e., ADF cores, memory, etc.). • Experience with additional Azure services (e.g., Logic Apps, Data Lake, Azure Functions, Azure SQL). • Experience/exposure in python/shell scripting is a plus. • Ability to generate a reconcile report of the migrated data using SQL or scripts. • Must be familiar with Git or source code repository & CI/CD process. Skills • Collaborate with business analysts, data architects, and stakeholders to gather requirements and ensure data integrity throughout the migration lifecycle. • Ability to create Data mapping documents (Source to Target) by understanding business rules. • Perform data profiling, cleansing, and quality checks to ensure accuracy and completeness of migrated data. • Analyze and map source Oracle database schemas to target Dataverse tables/entities, defining transformation and cleansing logic as required. • Write and optimize SQL queries, stored procedures, and scripts for Oracle databases to support data migration and validation. • Implement data transformation processes to ensure data compatibility with Dataverse data types, relationships, and constraints. • Experience with Dataverse data types and CRUD operations • Experience configuring data pipelines in ADF (to extract, transform, load). Experience using .csv files as sources of data for the ADF pipelines • Experience using Oracle CDCs as sources of data for the ADF pipelines. Experience logging errors (audit framework) from an ADF pipeline to a Dataverse table • Experience pushing data to Dataverse table(s) including necessary transformations to match target field types (i.e. Dataverse lookups, choices, date/time, etc.) • Experience configuring setting for pipelines to improve concurrency/performance (i.e., ADF cores, memory, etc.) • Experience with additional Azure services (e.g., Logic Apps, Data Lake, Azure Functions, Azure SQL). • Ability to generate a reconcile report of the migrated data using SQL or scripts • Must be familiar with Git or source code repository & CI/CD process. • Experience/exposure in python/shell scripting is a plus Company Overview • Centurion Consulting Group provides expert marketing communication, market research, strategy and marketing, data analysis services, etc. It was founded in 1995, and is headquartered in Los Angeles, California, USA, with a workforce of 2-10 employees. Its website is http://www.centurionconsulting.com.
Centurion Consulting Group is seeking an Azure Data Factory Engineer to ensure data integrity during migration processes. The role involves data profiling, SQL query writing, and configuring data pipelines in Azure Data Factory.
Diverse Lynx is seeking an Azure Data Factory specialist to design and implement Infrastructure as Code (IaC) using Terraform for Azure resources. The role involves automating Azure services and integrating CI/CD pipelines while ensuring security and compliance.
Procentrix, Inc. is seeking a Data Engineer specializing in Azure Data Factory to modernize their data ecosystem by migrating on-premise data to cloud environments. The role involves designing and maintaining data pipelines while ensuring data quality and compliance.
Intellect Solutions LLC is seeking an Azure Data Factory (ADF) Engineer with an Active Secret clearance to support data migration and transformation processes. The role involves collaborating with stakeholders, optimizing SQL queries, and configuring data pipelines in Azure.
Tech Cratic is seeking a Data Engineer with expertise in ETL processes and data warehousing, particularly in the healthcare sector. The role involves developing and managing ETL pipelines using Azure Data Factory and Snowflake.
Procentrix, Inc. is seeking a Data Engineer with expertise in Azure Data Factory to modernize their data ecosystem by migrating on-premise data to the cloud. The role involves designing and maintaining data pipelines while ensuring data quality and compliance.
Centurion Consulting Group is seeking an Azure Data Factory Engineer to ensure data integrity during migration processes. The role involves data profiling, SQL query writing, and configuring data pipelines in Azure Data Factory.
Diverse Lynx is seeking an Azure Data Factory specialist to design and implement Infrastructure as Code (IaC) using Terraform for Azure resources. The role involves automating Azure services and integrating CI/CD pipelines while ensuring security and compliance.
Procentrix, Inc. is seeking a Data Engineer specializing in Azure Data Factory to modernize their data ecosystem by migrating on-premise data to cloud environments. The role involves designing and maintaining data pipelines while ensuring data quality and compliance.
Intellect Solutions LLC is seeking an Azure Data Factory (ADF) Engineer with an Active Secret clearance to support data migration and transformation processes. The role involves collaborating with stakeholders, optimizing SQL queries, and configuring data pipelines in Azure.
Tech Cratic is seeking a Data Engineer with expertise in ETL processes and data warehousing, particularly in the healthcare sector. The role involves developing and managing ETL pipelines using Azure Data Factory and Snowflake.
Procentrix, Inc. is seeking a Data Engineer with expertise in Azure Data Factory to modernize their data ecosystem by migrating on-premise data to the cloud. The role involves designing and maintaining data pipelines while ensuring data quality and compliance.
Centurion Consulting Group is seeking an Azure Data Factory Engineer to ensure data integrity during migration processes. The role involves data profiling, SQL query writing, and configuring data pipelines in Azure Data Factory.
Diverse Lynx is seeking an Azure Data Factory specialist to design and implement Infrastructure as Code (IaC) using Terraform for Azure resources. The role involves automating Azure services and integrating CI/CD pipelines while ensuring security and compliance.
Centurion Consulting Group is seeking an Azure Data Factory Engineer to ensure data integrity during migration processes. The role involves data profiling, SQL query writing, and configuring data pipelines in Azure Data Factory.