JD: Candidate to build a Delta Lake. An understanding on Medallion architecture is important. Designed Batch and Real time data ingestion end to end. Data Lineage using Data Catalog - maintained that and used the tool to discover data sources and targets; Designed Data Quality Strategies identify CDE -critical data elements, assigning data stewards from business side and coded the DQ metrics generation in curated zone/silver layer. Designed CDC/incremental loads, full load strategies - using hash keys, key columns and merge designs. Designed Data Security - PII/NPI, RBAC- role based access control. Azure Data factory pipeline creation Azure DevOps Designed using Azure Data factory pipeline and Databricks - writing out to azure blob and then finally to snowflake as consumption were from Power BI /snowflake. Initially worked with client team and did extensive data analysis to create the model, data delivery mechanism, cloud storage, retention requirement, security, RBAC [role based access control]...etc Streaming datasets/file- were captured using AUTOLOADER delta live tables into raw and designed to be updated into silver, gold. Provide guidance on Azure cost management strategies, including resource tagging, budgeting, and monitoring. Conduct Azure security assessments, vulnerability scanning, and threat modeling to enhance cloud security. Discussing potential projects and their parameters with clients, executives, and software developers. Planning out the blueprints for software projects, including defining the scope, allocating resources, setting deadlines, laying out communication strategies, and indicating tests and maintenance. Assembling and leading the project team. Participating in and supervising each stage of the project. Ensuring each project stays on schedule and adheres to the deadlines. Determining and overseeing consistent testing, evaluation, and troubleshooting of all products in all stages of completion. Tracking milestones, deliverables, and change requests. Serving as a liaison to communicate information regarding changes, milestones reached, and other pertinent information. Preparing project proposals and discussing potential projects with clients and stakeholders Facilitating project initiation by defining project scope and requirements, and preparing the necessary documents and requirements Developing project plans and timelines to ensure the timely submission of project deliverables Managing project budgets and resources to ensure the timely completion of milestones Identifying and managing project risks Facilitating team meetings and collaboration Liaising for changes and negotiations with relevant stakeholders Ensuring software quality standards are met and requirements are submitted within budget and on time Closing the project and ensuring proper documentation"
Job Type
Fulltime role
Skills required
No particular skills mentioned.
Location
null, Minnesota
Salary
No salary information was found.
Date Posted
April 8, 2025
The Azure Databricks Architect at Interon IT Solutions will design and implement data ingestion strategies using Azure technologies while ensuring data quality and security. The role involves leading projects, collaborating with stakeholders, and managing resources to deliver effective data solutions.