Segment-Based Storage vs. Partition-Based Storage: Which is Better for Real-Time Data Streaming?

Storage is a critical component of any real-time data streaming system, and the choice of storage model can significantly affect the system’s performance, scalability, and reliability. Two popular storage models for real-time data streaming systems are segment- and partition-based storage.

In this talk, we will start by explaining what segment-based and partition-based storage means and how they work. We will explore the differences between the two storage models, including how data is organized, how data is stored, and how data is accessed.

We will discuss how a segment-based storage model provides better scalability, performance, and reliability than the partition-based model and how segment-based storage solves some deficiencies of the partition-based model, including the need to re-partition topics just to increase the storage capacity of a topic.

Attendees will leave this talk with a clear understanding of the differences between segment- and partition-based storage and how they affect real-time data streaming systems’ performance, scalability, and resiliency.

28 minutes
Register now to access all 50+ P99 CONF videos and slide decks.
Watch this session from the P99 CONF livestream, plus get instant access to all of the P99 CONF sessions and decks.

David Kjerrumgaard, Developer Advocate at StreamNative

David is a committer on the Apache Pulsar project, the author of "Pulsar in Action" and co-author of "Practical Hive." He currently serves as a Developer Advocate for StreamNative, focusing on strengthening the Apache Pulsar community through education and evangelization. Before that, he was a principal software engineer on the messaging team at Splunk and Director of Solutions for two Big Data startups, Streamlio and Hortonworks.

P99 CONF OCT. 23 + 24, 2024

Register for Your Free Ticket