Job description

Job Title: Lead Databricks Data Architect Location: Atlanta, GA (Onsite/Hybrid/Remote as per requirement) Experience: 8 – 10 Years Job Type: Full Time Job Overview We are seeking an experienced Lead Databricks Data Architect to design and implement robust, scalable, and secure cloud-based data solutions. This role involves working with Azure cloud services, leading development of data ingestion pipelines, and optimizing data architecture to meet evolving business needs. Key Responsibilities • Architect and build end-to-end data solutions using Azure Databricks, Azure Data Factory, and Azure SQL Database. • Lead the development of real-time and batch data pipelines using Delta Live Tables (DLT) and Structured Streaming. • Collaborate with stakeholders to translate business requirements into scalable data warehouse solutions on Azure. • Provide strategic leadership for large-scale data programs and ensure alignment with organizational goals. • Drive performance optimization of Spark jobs and Databricks workflows through advanced tuning techniques. • Build and maintain metadata-driven ingestion frameworks to support automation and scalability. • Work cross-functionally with data scientists, analysts, and other stakeholders to deliver business-aligned data solutions. • Implement data governance, quality checks, and security measures to maintain data integrity and compliance. • Monitor and enhance cloud-based data infrastructure for performance and cost-efficiency. • Lead or support migration and upgrade efforts of legacy applications (e.g., .NET Framework) to cloud platforms. • Utilize Azure DevOps tools including Git, pipelines, and work item management. • Troubleshoot issues on Windows Server environments (e.g., IIS, Event Viewer, Performance Monitor). • Participate in Agile/Scrum ceremonies including sprint planning, retrospectives, and daily standups. Technical Skills Required • Strong proficiency in: • Azure Databricks, ADF, Azure SQL • Apache Spark (job tuning and internals) • Structured Streaming, Delta Live Tables • Experience in: • .NET Frameworks 4.8.2 and Core 6/8 • ASP.NET MVC, Blazor, WebForms • Web services & API integrations • Microsoft SQL Server (DDL, DML, stored procedures) • Nota Fiscal integration (Preferred in finance/logistics/related domains) • Familiarity with: • Azure authentication and key vault • Agile/Lean methodologies Preferred Qualifications • Proven experience designing and delivering enterprise-level cloud data platforms. • Ability to lead cross-functional teams and mentor junior developers. • Excellent communication, documentation, and stakeholder management skills. Familiarity with CI/CD and infrastructure-as-code in cloud environments.

Requirements