Participate in the State of Distributed Systems Survey | Take Survey

Scaling Apache Pulsar to 10 Petabytes/Day

Pulsar is used by a portfolio of products at Splunk for stream processing of different types of data, including metrics and logs. In this talk, Karthik Ramasamy will share how Splunk helped a flagship customer scale a Pulsar deployment to handle 10 PB/day in a single cluster. He will talk about the journey, the challenges faced, and the trade-offs made to scale Pulsar and operate it reliably and stably in Google Cloud Platform (GCP).

17 Minutes
Share on facebook
Share on twitter
Share on email
Share on linkedin
Share on facebook
Share on twitter
Share on email
Share on linkedin
Karthik Ramasamy

Karthik Ramasamy, Senior Director of Engineering at Splunk

Karthik Ramasamy is a Senior Director of Engineering at Splunk. Before Splunk, he was co-founder and CEO of Streamlio where he focused on building the next generation real time processing engine based on Apache Pulsar. He previously served as the engineering manager and technical lead for real-time analytics at Twitter where he co-created Twitter Heron. He has two decades of experience working in parallel databases, big data infrastructure, and networking. Karthik is the author of several publications, patents, and "Network Routing: Algorithms, Protocols and Architectures". He has a Ph.D. in computer science from the University of Wisconsin, Madison with a focus on big data and databases.

P99 CONF OCT. 6-7, 2021

REGISTER FOR YOUR FREE TICKET