4 eBPF Tech Talks at P99 CONF: Optimization on the Fly

Share This Post

Not (yet) an expert on BPF? No worries! We have you covered at P99 CONF, with a range of presentations that reveal just how powerful this technology is. The real beauty of the extended Berkeley Packet Filter lies in its ability to add custom functionality to the Linux kernel without modifying source code or introducing additional overheads. In other words, eBPF gives you the ability to monitor, observe and even optimize performance of systems while they are running.

Watch P99 CONF Talks On Demand

Let’s take a look at four talks, each of which explores innovative ways of applying eBPF to different system challenges. Starting with Liz Rice with a follow up presentation from last year, this year we will hear about …

Zero-overhead container networking with eBPF and Netkit

Liz Rice (Isovalent at Cisco)

Netkit is a new enhancement to eBPF that replaces the virtual Ethernet (veth) connections that previously connected containers to the network namespace of their host. Until now, the overhead of veth connections meant that containerized applications could not communicate as quickly as if they were running directly on the host. In this talk you’ll learn how Netkit and other eBPF-enabled capabilities now allow container networking to run as fast as host networking.

Bonus reading: Get the complete O’Reilly Learning eBPF book that Liz wrote; it’s free, compliments of Isolvalent.

Scheduler Tracing With ftrace + eBPF

Jason Rahman (Microsoft)

Understanding application latency requires understanding the underlying layers of the system. The operating system scheduler is one of those layers which impacts application latency and performance. Here, we’ll learn how to leverage both ftrace and eBPF (along with Perfetto for visualization) to capture the runtime behavior of the Linux scheduler. Along the way we’ll explore some interesting quirks (arguably bugs) in the existing CFS scheduler, and also begin exploring the new EEVDF scheduler appearing in recent Linux kernels.

Bonus reading: Read Jason’s blog “Intel CPU Die Topology,” where he looks at a pair of Intel CPUs and observes a few key details on their on-die interconnect topology and impact on cross-core communication patterns.

Using eBPF Off-CPU Sampling to See What Your DBs are Really Waiting For

Tanel Poder

At P99 CONF 23, I introduced the general concept of using eBPF-populated Task State Arrays to keep track of all Linux applications’ (including database engines) thread states and activity without relying on the built-in instrumentation of the application. For example, the “wait events” built into database engines are not perfect; some voluntary waits (system calls) are not properly instrumented in all database engines. There are also other involuntary waits caused by OS-level issues, like memory allocation stalls, CPU queuing, and task scheduler glitches. This year, I will show the latest eBPF-based “xcapture” tool in practical use, measuring where MySQL, Postgres, and DuckDB really spend their time, both when on CPU and sleeping. All this can be done without having to change any source code of the database engine or applications running on it.

Bonus reading: Browse Tanel’s 0x.tools site and see the discussion it sparked on Hacker News.

Noisy Neighbor Detection with eBPF

Jose Fernandez (Netflix)

In multi-tenant environments, performance issues often arise from the “noisy neighbor” problem, where one container’s excessive CPU usage degrades the performance of adjacent containers. At Netflix, we’ve developed a low-overhead solution leveraging eBPF to continuously instrument the Linux scheduler and detect these issues in real time. This talk will explore how eBPF is used to monitor run queue latency, associate process IDs with cgroup IDs, and emit actionable metrics. Attendees will gain insights into the implementation details, optimization techniques for eBPF code, and how this approach ensures high performance and reliability in a shared compute environment, offering practical guidance for similar applications in their infrastructures.

Bonus reading: Read the Netflix blog that Jose recently co-authored on “Noisy Neighbor Detection with eBPF” and review the Hacker News discussion.

More To Explore

P99 CONF Speaker Spotlight: Piotr Sarna

Get to know Piotr Sarna, Founding Engineer at poolside, in anticipation of his P99 CONF talk: “Database Drivers: Performance Perspectives”