Storage is a critical component of any real-time data streaming system, and the choice of storage model can significantly affect the system’s performance, scalability, and reliability. Two popular storage models for real-time data streaming systems are segment- and partition-based storage.

In this talk, we will start by explaining what segment-based and partition-based storage means and how they work. We will explore the differences between the two storage models, including how data is organized, how data is stored, and how data is accessed.

We will discuss how a segment-based storage model provides better scalability, performance, and reliability than the partition-based model and how segment-based storage solves some deficiencies of the partition-based model, including the need to re-partition topics just to increase the storage capacity of a topic.

Attendees will leave this talk with a clear understanding of the differences between segment- and partition-based storage and how they affect real-time data streaming systems’ performance, scalability, and resiliency.