Real-time OLAP databases usually trade performance for cost when moving from local storage to cloud object storage. This talk shows how we extended Apache Pinot to use cloud storage while still achieving sub-second P99 latencies. We’ll cover the abstraction that makes Pinot location-agnostic, strategies like pipelining, prefetching, and selective block fetches, and how to balance local and cloud storage for both cost efficiency and speed.
Neha Pawar is a Founding Engineer, and Head of Data Infra at StarTree (https://www.startree.ai/), which aims to democratize data for all users by providing real-time, user-facing analytics. Prior to this, she was part of LinkedIn's Data Analytics Infrastructure org for 5 years, working on Apache Pinot & ThirdEye. She is passionate about big data technologies and real-time analytics databases. Neha is an Apache Pinot PMC and Committer. She has made numerous impactful contributions to Apache Pinot, with a focus on realtime streaming engine and ingestion. At StarTree, she led initiatives such as Tiered Storage and StarTree Serverless. She actively fosters the growing Apache Pinot and StarTree community & loves to evangelize Pinot by making entertaining video tutorials & illustrations, writing blogs and delivering tech talks. When not sipping Pinot, you can find Neha jamming with her husband, painting or hiking with her dogs.
