Gov Services Hub is seeking a Senior ADB Developer with expertise in Azure Databricks, Python, and Spark Streaming to develop large-scale data pipelines. This contract position is based in Pleasanton, California, offering a competitive hourly rate.
Job Summary Job Description: Must Have Skills – Skill 1 – Yrs of Exp – Azure Databricks Skill 2 – Yrs of Exp – Python Skill 3 – Yrs of Exp – Spark Streaming Good To have Skills – Skill 1 – Yrs of Exp – Java or Scala Skill 2 – Yrs of Exp – CI/CD pipelines Skill 3 – Yrs of Exp –GitHub workflows _____________________________________ About the Role: We are looking for a highly skilled Senior Azure Databricks (ADB) Developer to join our Data Engineering team. This role involves developing large-scale batch and streaming data pipelines on Azure Cloud. The ideal candidate will have strong expertise in Python, Databricks Notebooks, Apache Spark (including Structured Streaming), and real-time integration with Kafka. You will work with both relational databases like DB2 and NoSQL systems such as MongoDB, focusing on performance optimization and scalable architecture. Key Responsibilities: • Design and Develop: Create real-time and batch data pipelines using Azure Databricks, Apache Spark, and Structured Streaming. • Data Processing: Write efficient ETL scripts and automate workflows using Python. • Data Integration: Integrate with various data sources and destinations, including DB2, MongoDB, and other enterprise-grade data systems. • Performance Optimization: Tune Spark jobs for optimal performance and cost-effective compute usage on Azure. • Collaboration: Work with platform and architecture teams to ensure secure, scalable, and maintainable cloud data infrastructure. • CI/CD Support: Implement CI/CD for Databricks pipelines and notebooks using tools like GitHub and Azure DevOps. • Stakeholder Communication: Interface with product owners, data scientists, and business analysts to translate data requirements into production-ready pipelines. Required Skills: • 10+ years of experience in data engineering • Python Proficiency: • Data Manipulation: Using libraries like Pandas and NumPy for data manipulation and analysis. • Data Processing: Writing efficient ETL scripts. • Automation: Automating repetitive tasks and workflows. • Debugging: Strong debugging skills to troubleshoot and optimize code. • Database Management: • SQL: Advanced SQL skills for querying and managing relational databases. • NoSQL: Experience with NoSQL databases like MongoDB or Cassandra. • Data Warehousing: Knowledge of data warehousing solutions like Google BigQuery or Snowflake. • Big Data Technologies: • Kafka: Knowledge of data streaming platforms like Apache Kafka. • Version Control: • Git: Using version control systems for collaborative development. • Data Modeling: • Schema Design: Designing efficient and scalable database schemas. • Data Governance: Ensuring data quality, security, and compliance. • Database Management: • DB2: Understanding of DB2 architecture, SQL queries, and database management. • MongoDB: Knowledge of MongoDB schema design, indexing, and query optimization. • Programming Skills: • Proficiency in languages such as Java, Python, or JavaScript to write scripts for data extraction and transformation. • Experience with BSON (Binary JSON) for data conversion. • Cloud Services: • Experience with cloud platforms like AWS or Azure for deploying and managing databases. Preferred Skills: • Experience with Java or Scala in Spark streaming. • Familiarity with Azure services like Data Lake, Data Factory, Synapse, and Event Hubs. • Background in building data platforms in regulated or large-scale enterprise environments. Job Type: Contract Pay: $62.22 - $65.27 per hour Expected hours: 40 per week Work Location: Hybrid remote in Pleasanton, CA 94566
Gov Services Hub is seeking a Senior ADB Developer with expertise in Azure Databricks, Python, and Spark Streaming to develop large-scale data pipelines. This contract position is based in Pleasanton, California, offering a competitive hourly rate.
Exusia, Inc. is seeking a Senior Data Engineer with expertise in Azure Databricks, Python, and Pyspark to join their global delivery team. The role involves designing, developing, and maintaining large-scale data engineering solutions while collaborating with business stakeholders.
Exusia, Inc. is seeking a Senior Data Engineer with expertise in Azure Databricks, Python, and Pyspark to design and maintain large-scale data solutions. This full-time position is remote and requires 6-10 years of experience in Data Management and Engineering.
Shrive Technologies is seeking a Technical Lead with expertise in Python, PySpark, Snowflake, Databricks, and Azure. The role requires strong leadership skills and extensive experience in software development and cloud platforms.
Realign LLC is seeking an experienced Azure Databricks Developer proficient in Python, PySpark, and SQL for a long-term contract role in Pittsburgh, PA. The candidate will focus on designing and optimizing data pipelines in Azure cloud environments.
REALIGN LLC is seeking an experienced Azure Databricks Developer proficient in Python, PySpark, and SQL for a long-term contract in Pittsburgh, PA. The role involves designing and optimizing data pipelines in Azure cloud environments.
Gov Services Hub is seeking a Senior ADB Developer with expertise in Azure Databricks, Python, and Spark Streaming to develop large-scale data pipelines. This contract position is based in Pleasanton, California, offering a competitive hourly rate.
Exusia, Inc. is seeking a Senior Data Engineer with expertise in Azure Databricks, Python, and Pyspark to join their global delivery team. The role involves designing, developing, and maintaining large-scale data engineering solutions while collaborating with business stakeholders.
Exusia, Inc. is seeking a Senior Data Engineer with expertise in Azure Databricks, Python, and Pyspark to design and maintain large-scale data solutions. This full-time position is remote and requires 6-10 years of experience in Data Management and Engineering.
Shrive Technologies is seeking a Technical Lead with expertise in Python, PySpark, Snowflake, Databricks, and Azure. The role requires strong leadership skills and extensive experience in software development and cloud platforms.
Realign LLC is seeking an experienced Azure Databricks Developer proficient in Python, PySpark, and SQL for a long-term contract role in Pittsburgh, PA. The candidate will focus on designing and optimizing data pipelines in Azure cloud environments.
REALIGN LLC is seeking an experienced Azure Databricks Developer proficient in Python, PySpark, and SQL for a long-term contract in Pittsburgh, PA. The role involves designing and optimizing data pipelines in Azure cloud environments.
Gov Services Hub is seeking a Senior ADB Developer with expertise in Azure Databricks, Python, and Spark Streaming to develop large-scale data pipelines. This contract position is based in Pleasanton, California, offering a competitive hourly rate.
Exusia, Inc. is seeking a Senior Data Engineer with expertise in Azure Databricks, Python, and Pyspark to join their global delivery team. The role involves designing, developing, and maintaining large-scale data engineering solutions while collaborating with business stakeholders.
Gov Services Hub is seeking a Senior ADB Developer with expertise in Azure Databricks, Python, and Spark Streaming to develop large-scale data pipelines. This contract position is based in Pleasanton, California, offering a competitive hourly rate.