Job description

We are seeking an experienced and highly skilled QA Engineer with expertise in Azure and Databricks to join our data and reporting team. In this role, you will be responsible for ensuring the quality, consistency, and accuracy of data across the enterprise analytics ecosystem. This role focuses on validating data pipelines and transformations in Databricks, testing semantic models, ensuring metadata accuracy, and automating quality checks for enterprise reporting solutions like Power BI and downstream data consumption layers. JOIB DESCRIPTION: · Validate data ingestion, transformation, and load (ETL/ELT) processes built on Databricks. · Design and execute test cases to verify data accuracy, completeness, and lineage across medallion layers (Bronze, Silver, Gold). · Monitor pipeline executions and identify data drift, schema changes, or transformation issues. · Test semantic data models supporting Power BI, Tableau, or other BI tools for accuracy and alignment with business definitions. · Validate measures, KPIs, and calculated fields within the reporting layer for correctness. · Conduct end-to-end testing from source data to enterprise reports ensuring consistency of business logic. · Validate metadata management processes and ensure alignment with enterprise data catalog or governance platforms. · Implement automated checks for data quality dimensions like completeness, validity, consistency, and timeliness. · Develop and maintain automated testing frameworks for data validation using Python, PySpark, or Databricks notebooks. · Integrate automated tests into CI/CD pipelines (Azure DevOps, GitHub Actions, etc.). · Build reusable data validation scripts leveraging Databricks and other data testing tools. · Work closely with data engineers, data modelers, BI developers, and governance teams to define testing standards. · Participate in code reviews and data quality reviews to ensure compliance with QA standards. · Document test cases, results, and maintain traceability between business requirements and validation outcomes. SCOPE: · Strong expertise in Databricks, including Spark (PySpark/Scala). · Hands-on experience working with cloud platforms (AWS, Azure, or Google Cloud). · Experience with CI/CD tools like Azure DevOps, GitHub Actions, or Terraform for infrastructure-as-code. · Familiarity with data modeling, data warehousing, and database design. · Strong understanding of data formats such as Parquet, ORC, JSON, and Avro. · Ability to work in an agile environment and adapt to changing requirements. · Strong analytical and problem-solving skills with a focus on optimizing data engineering solutions. · Study, analyze and understand business requirements in context to business intelligence. · Spot key performance indicators with apt objectives · Make essential technical and strategic changes to improvise present business intelligence systems · SQL querying for better results CREDENTIALS: Bachelor’s degree in computer science, Engineering, or a related field (or equivalent experience). Proven experience working with Databricks for large-scale data processing. Strong proficiency in SQL and experience with relational databases. 5+ years of data engineering experience Proficient in SQL writing skills Experience in restaurant / gaming industry REQUIRED SKILLS: · Minimum 6 years of experience of working on Data platforms. · Strong SQL skills for data validation and test case development. · Hands-on experience with Databricks, Spark, and Delta Lake environments. · Experience testing data pipelines and transformations in Azure or cloud-based data platforms. · Knowledge about database management, SQL querying, data modeling, and Online Analytical Processing. · Additional consideration given for experience scripting and programming language such as Python · Ability to mentor junior team members and contribute to technical leadership

Requirements