Scaling Apache Pulsar to 10 Petabytes/Day

Pulsar is used by a portfolio of products at Splunk for stream processing of different types of data, including metrics and logs. In this talk, Karthik Ramasamy will share how Splunk helped a flagship customer scale a Pulsar deployment to handle 10 PB/day in a single cluster. He will talk about the journey, the challenges faced, and the trade-offs made to scale Pulsar and operate it reliably and stably in Google Cloud Platform (GCP).

17 Minutes
Karthik Ramasamy

Karthik Ramasamy, Senior Director of Engineering at Splunk

Karthik Ramasamy is a Senior Director of Engineering at Splunk. Before Splunk, he was co-founder and CEO of Streamlio where he focused on building the next generation real time processing engine based on Apache Pulsar. He previously served as the engineering manager and technical lead for real-time analytics at Twitter where he co-created Twitter Heron. He has two decades of experience working in parallel databases, big data infrastructure, and networking. Karthik is the author of several publications, patents, and "Network Routing: Algorithms, Protocols and Architectures". He has a Ph.D. in computer science from the University of Wisconsin, Madison with a focus on big data and databases.

Register for 2 days of keynotes & interactive tech talks on all things P99.

Virtual Event

October 19-20, 2022

Share on social with #p99conf and a link to for a chance to win $500.