Job description

Direct message the job poster from GBIT (Global Bridge InfoTech Inc) Sr. US IT Recruiter/Bench Sales at Global Bridge InfoTech Inc Location: Remote ||SFO, CA preferred. Contract: Long term Job Summary: We are seeking a highly skilled Azure Data Engineer to join our data engineering team. The ideal candidate will have strong experience in designing, building, and maintaining scalable data pipelines and infrastructure on cloud platforms like Azure and AWS. You will be responsible for developing robust ETL processes, working with big data technologies such as Hadoop and Spark, and contributing to system design for real-time analytics and fault-tolerant architectures. Key Responsibilities: • Design, build, and maintain ETL/ELT pipelines using modern orchestration tools like Luigi , Azure Data Factory , or similar. • Develop scalable data lakes and data ingestion frameworks leveraging Azure Data Lake Storage , Databricks , or AWS S3 . • Work closely with cross-functional teams to support real-time data processing and streaming analytics solutions. • Implement and optimize big data solutions using technologies like Hadoop , Apache Spark , and Delta Lake . • Design and implement secure, scalable cloud infrastructure using Azure (preferred) and AWS services for data storage and processing. • Ensure robust access control and security policies across cloud resources using RBAC , IAM , network security groups , etc. • Participate in system architecture discussions focused on fault tolerance , high availability , and performance optimization . • Automate infrastructure and deployments using CI/CD practices and Infrastructure as Code (IaC) tools (e.g., Terraform, ARM templates). • Monitor and troubleshoot pipeline and infrastructure issues to ensure data reliability and integrity. Required Qualifications: • Bachelors or Masters degree in Computer Science, Information Systems, Engineering, or related field. • 4+ years of experience in data engineering , with at least 2+ years on Azure and/or AWS . • Strong programming skills in Python , Scala , or Java for building ETL workflows. • Hands-on experience with Luigi , Airflow , or other pipeline orchestration tools. • Solid understanding of cloud access controls , identity management , and security best practices . • Deep knowledge of big data technologies Hadoop, Spark, Hive, and related ecosystems. • Experience in real-time analytics and streaming data processing using Kafka, Spark Streaming, or similar. • Proven track record of contributing to the design of scalable, fault-tolerant architectures in cloud environments. Preferred Skills: • Experience with data modeling , data governance , and metadata management . • Familiarity with containerization (Docker, Kubernetes) and DevOps practices. • Exposure to machine learning pipelines or analytics platforms is a plus. Seniority level • Seniority level Mid-Senior level Employment type • Employment type Contract Job function • Job function Information Technology • Industries IT Services and IT Consulting Referrals increase your chances of interviewing at GBIT (Global Bridge InfoTech Inc) by 2x Inferred from the description for this job Medical insurance 401(k) Vision insurance Paid maternity leave Get notified when a new job is posted. Sign in to set job alerts for Data Engineer roles. San Francisco, CA $160,000.00-$180,000.00 2 weeks ago Foster City, CA $192,000.00-$260,000.00 1 week ago San Francisco, CA $182,000.00-$232,000.00 1 day ago San Francisco, CA $207,000.00-$279,000.00 6 days ago San Francisco, CA $149,998.00-$250,000.00 5 days ago San Francisco, CA $112,597.33-$152,810.66 5 days ago San Mateo, CA $120,000.00-$160,000.00 2 weeks ago San Francisco, CA $145,000.00-$165,000.00 2 weeks ago San Francisco, CA $255,000.00-$405,000.00 2 days ago San Francisco, CA $130,000.00-$238,000.00 2 weeks ago San Francisco, CA $150,000.00-$200,000.00 2 weeks ago San Francisco, CA $140,000.00-$180,000.00 3 days ago San Francisco, CA $120,000.00-$150,000.00 2 months ago San Francisco, CA $180,000.00-$220,000.00 2 weeks ago San Francisco, CA $177,500.00-$210,000.00 2 months ago San Francisco, CA $90,000.00-$150,000.00 1 month ago San Francisco, CA $150,000.00-$260,000.00 5 months ago San Francisco, CA $125,640.00-$174,500.00 5 days ago Were unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr

Requirements