Job description

About the position Responsibilities • Design and implement highly scalable OLTP systems for real-time workloads and OLAP systems for complex analytical queries on massive datasets. • Build, optimize, and maintain large-scale batch and streaming pipelines using frameworks such as Apache Spark, Flink, Presto/Trino, or Kafka Streams. • Optimize systems for low-latency queries, high-throughput ingestion, and interactive analytics, ensuring seamless performance as data volumes scale to petabytes. • Develop and integrate with modern storage and processing systems (e.g., Snowflake, BigQuery, Redshift, Cassandra, HDFS, Delta Lake, Iceberg) to support hybrid analytical/transactional workloads. • Ensure high availability, reliability, and monitoring across large compute and storage clusters with automated failover and recovery. • Partner with data scientists, ML engineers, and product teams to build unified, secure, and cost-efficient data platforms. Requirements • Strong proficiency in Java, Scala, Python, or Go, with proven experience building distributed back-end systems. • Deep understanding of database internals, query optimization, indexing, and ACID vs. eventual consistency trade-offs. • Hands-on experience with big data frameworks (Spark, Flink, Kafka) and distributed SQL engines (Presto, Trino, Hive, Impala). • Expertise in designing OLAP/OLTP architectures for scale and high concurrency. • Solid grounding in distributed systems, concurrency, parallelism, and caching techniques. Nice-to-haves • Experience with HTAP (Hybrid Transactional/Analytical Processing) systems or real-time analytics platforms. • Familiarity with data lakehouse architectures and formats like Parquet, ORC, Delta, Iceberg, Hudi. • Knowledge of containerized deployments (Docker, Kubernetes) and cloud-native data architectures (AWS Redshift, GCP BigQuery, Azure Synapse). • Background in query engine development or contributing to open-source OLAP/OLTP frameworks. Benefits • Healthcare • PTO • Eligible for stock options

Requirements