- people - netdevconf.info

0x17: Talk, Nanotube: Turning XDP Code into Custom Hardware Pipelines for FPGA SmartNICs
by Jamal Hadi Salim 07 Oct '23

07 Oct '23

Many modern NICs have built-in packet processing capabilities that can effectively perform the type of use case that XDP is often used for. Stephan Diestelhorst, Neil Turton, Kieran Mansley, Kimon Karras, Rip Sohan , Thomas Calvert and Steven Pope say that being able to seamlessly offload this XDP code to the NIC – under the control of the Linux networking subsystem and tooling – would increase server efficiency and performance without affecting the user experience. Stephan Diestelhorst et al will discuss Nanotube: an open-source compiler that consumes standard XDP code and produces a tailor-made, application-specific dataflow pipeline suitable for execution on FPGA NIC platforms. They have demonstrated generated Katran on AMD/Xilinx FPGA-based NICs to offload with throughput at line rate (100 Gbps) on AMD NICs! In this talk they will describe (i) the structure of the compiler; (ii) performance of typical workloads; and (iii) how map accesses by host software are supported using specially annotated packets over the normal host data path. And last but not least they will discuss requirements to allow better integration and support by the Linux kernel, in particular the validator, map handling, and vendor-neutral support requirements. cheers, jamal

1 0

0x17: Talk, P4 Compatible Control Plane Driver for Linux Kernel
by Jamal Hadi Salim 07 Oct '23

07 Oct '23

Our friends at Intel and AMD will share their thoughts on how to integrate P4TC[1] into their drivers. In this talk Anjali Singhai, Neha Singh, James Choi, Namrata Limaye, Sridhar Samudrala, Mahesh Shirshyad and Vipin Jain will discuss enhancements to existing TC offload interfaces to accommodate multiple P4 pipeline and runtime control. They will elaborate on the association of the s/w and h/w datapaths, hardware P4 dataplane loading and initialization as well as runtime control (using the classical skip_sw/hw flagging). These details will include describing the kernel infrastructures and any extension to current TC offload. Anjali et all will discuss potential enable shimming of legacy kernel offloads such as tc-flower on to P4TC interfaces. Additionally, they will talk about interfaces to support legacy hooks (switchdev ops, FDB ops) and coexistence of these with P4TC compatible driver. They will make the infrastructure code at the conference. cheers, jamal [1] https://netdevconf.info/0x17/sessions/talk/integrating-ebpf-into-the-p4tc-d…

1 0

0x17: Talk, SO_TIMESTAMPING: powering fleetwide RPC monitoring
by Jamal Hadi Salim 07 Oct '23

07 Oct '23

Willem de Bruijn says timestamping, via SO_TIMESTAMPING, is key to debugging network stack latency. Instead of gut-feel finger pointing between network and kernel tribes we can just get down to the facts of where the latency really is. SO_TIMESTAMPING can isolate transmission, reception and even scheduling sources. Capturing connection state along with timestamps further enables root cause discovery, such as TCP receive window size. Capturing timestamps at more points, such as traffic shaping and NIC hardware, expands visibility to tough issues like incast. SO_TIMESTAMPING has seen iterative development to enable fleetwide RPC monitoring at Google. They presented details of this infra called "Fathom" at SIGCOMM 2023[1]. In this talk Willem will start us beyond where that SIGCOMM paper ends. He will take a deep dive on the Linux kernel infrastructure that makes fleetwide continuous latency analysis and attribution possible. API extensions include covering TCP bytestreams, capturing transport protocol state along with events (OPT_STAT), and supporting selective sampling (OPT_CMSG). The talk reviews the core SO_TIMESTAMPING API, discusses non-obvious extensions (MSG_EOR, SO_RCVLOWAT), summarizes gotchas from the field (OPT_ID_TCP), and explains how all this combines to enable robust continuous RPC monitoring. It touches on clock synchronization and precision. Finally, it compares this UAPI to dynamic tracing with uprobes, kprobes, tracepoints and BPF. Should be fun, cant wait to attend this talk! [1] search for "fathom" at https://conferences.sigcomm.org/sigcomm/2023/program.html cheers, jamal

1 0

Assessing the impact of Linux networking on CPU consumption
by Jamal Hadi Salim 07 Oct '23

07 Oct '23

You may be a guru at reading perf output and then with a handwave, swish, you can whip your bash script into a frenzy to create flame graphs... but you would still need to dig deeper and isolate where your bottlenecks in the network are. Davide Miola, Fulvio Risso and Federico Parola took a different approach by creating a tool using eBPF to instrument strategic functions in the network RX and TX entry-points. They crafted a novel tool that they call "Netto"[1] which gives you a focussed view of only the network infra's contribution. Netto can give you answers to coarse grained details of "how much is the rx softirq chewing vs RX syscall" all the way to more fine-grained rx softirq details like the driver cost, GRO, NF, etc. And then do all this in real time with low cpu overhead... In this talk, Davide et al will go into details of the design and functionality of Netto as well as describe the different kernel hook points. They will then illustrate how one would be able to use this tool to get insights into the network infra's overheads for a specific workload. [1]https://github.com/miolad/netto cheers, jamal

1 0

0x17: Talk, Scripting the Linux Routing Table with Lua
by Jamal Hadi Salim 06 Oct '23

06 Oct '23

You can run Lua scripts inside the kernel using the Lunatik project! Lunatik has been previously used for scripting Netfilter and XDP subsystems, allowing users to create complex network filters using Lua. For instance, it has been used for filtering L7 traffic such as HTTP and HTTPS inside the kernel. In this talk, Lourival Vieira Neto, Marcel Moura, Ana Lúcia de Moura and Roberto Ierusalimschy present the evolution of this framework and illustrate a sample use case implementing a kernel-level adaptive routing service. This service allows the system to monitor the reliability of a network route and adjust its routing table dynamically in case of failure. For instance, a home router might use such service to fall back to the cellular network using a mobile phone as a gateway. Lourival et al also spice this service up by implementing a protocol that allows nodes to advertise themselves as alternative gateways and for the router to notify nodes when elected as the new gateway. Lourival et al will describe the kernel details and associate infrastructure interaction ( netdevice notification chain, kthread, socket, RCU and FIB, etc). cheers, jamal

1 0

0x17: Talk, eBPF Qdisc: a generic building block for traffic control
by Jamal Hadi Salim 06 Oct '23

06 Oct '23

Adding a new packet scheduling algorithm in the kernel requires implementing a Qdisc kernel module - which could be cumbersome. Writing a custom scheduling algorithm which may not make sense for the rest of the community is frowned upon. On top of Qdiscs, different classifiers and rules can be applied to packets as well, which further complicates the combination of choices. Amery Hung and Cong Wang introduce eBPF Qdisc, a fully programmable Qdisc with eBPF. Amery and Cong believe that the approach they are taking will provide developers not only a more flexible and safer way to implement packet schedulers, but also orchestrating packets in coordination with other networking components. In this talk, they first show that eBPF Qdisc is capable of realizing a sophisticated Qdisc by implementing fair queueing (fq) Qdisc. Then, they evaluate the performance of the fq against the existing fq in the Linux kernel. Finally, they will show how eBPF Qdisc can help developers implement Qdisc's with diverse functionalities through two use cases: 1) we build a more robust earliest departure time (EDT) rate limiter by coordinating with a fq Qdisc through eBPF maps and 2) show eBPF Qdisc as a flexible way to create network emulators. cheers, jamal

1 0

0x17 Talk, Leveraging Homa: Enhancing Datacenter RPC Transport Protocols
by Jamal Hadi Salim 06 Oct '23

06 Oct '23

Hyper-scale data centers, such as Bytedance, have a strong need for optimal RPC performance. Homa is purpose-built for RPCs making it ideal for such environments. In this talk, Xiaochun Lu and Zijian Zhang do a comparative study of gRPC's performance on Homa versus over TCP. They observe clear performance advantages in favor of Homa, especially for RPC messages under 50KB, in terms of both latency and throughput. Like John's talk[1], they optimize longer RPC messages and improve bidirectional messaging. Xiaochun and Zijian will further provide performance insights, describe their mods to the Homa kernel code, and guide participants through the tangible benefits of integrating Homa into their RPC transport framework. [1]https://netdevconf.info/0x17/sessions/talk/kernel-managed-user-buffers-in-homa.html cheers, jamal

1 0

0x17: Talk, Kernel-Managed User Buffers in Homa
by Jamal Hadi Salim 06 Oct '23

06 Oct '23

Reminder: One more day to go for early bird registration.. Homa was first introduced to us in John Ousterhout's exciting 0x16 keynote[1]. John argued in the keynote that it was time to replace TCP with Homa to avoid what he labelled as the "datacentre tax". For the uninformed: Homa is message-oriented like UDP or SCTP as opposed to TCP being stream-oriented. There is no notion of connection in Homa, only request/response RPC messaging. A single Homa socket can send/receive RPCs to many peers. One advantage that TCP has by virtue of using the streaming approach is it can pipeline very large messages (think 500KB across MTU sized packets on the wire) - whereas a message oriented approach will have to copy bytes to/from user space and DMA-able packet buffers in parrallel. In his experiments with an mlx5 based 25G NIC, Dr. Ousterhout found that TCP infact outperformed Homa with large messages. At 500KB message size TCP was able to achieve 17-19 Gbps throughput, Homa was only able to hit 10-11Gbps. How dare TCP beat Homa in any metric! So John set out to fix this situation ;->. When he was done with the preliminary work Homa was able to hit the same rate as TCP (17-19Gbps). In this talk, he will go over the approach he took in the kernel code to allow the kernel to manage user mmaped buffers, changes made to both recvmsg() and sendmsg() to make this happen. 1. https://netdevconf.info/0x16/sessions/keynote/keynote-ousterhout.html cheers, jamal

1 0

0x17: Talk, Zero Copy Receive using io_uring
by Jamal Hadi Salim 05 Oct '23

05 Oct '23

The war on memory bottlenecks[1][2] continues with this talk from David Wei and Pavel Begunkov. Socket recv() data is first copied/DMAed into the kernel memory and then again into user space memory - adding pressure to overall memory bandwidth and of course comes with a CPU cost. While there are other approaches to avoid the second copy on recv() such as DPDK, RDMA etc, David and Pavel argue that all of them have downsides - ranging from being proprietary, custom patched drivers, and difficult to debug and worse requiring a rewrite of the applications... Our esteemed speakers instead opt for continuing to use the network stack as is and io_uring as the user facing API. So what do we need to get this working? As in [2] that the NIC supports header splitting and RSS flow steering. On incoming data the headers traverse the TCP stack while data is DMA'ed not to the kernel memory but into user space... David and Pavel will discuss the overall approach they took and describe in some detail the kernel infra as well as what uapi would look like. They will further review kernel-existing zero copy approaches and how they plan to coexist with them. Last but not least they will dig into limitations and challenges of zero copy receive and how they overcome them to facilitate a real deployment. [1] https://netdevconf.info/0x17/sessions/talk/congestion-control-architecture-… [2] https://netdevconf.info/0x17/sessions/talk/device-memory-tcp.html cheers, jamal Reminder: 2 more days to go for early bird registration

1 0

0x17: Talk, Device Memory TCP
by Jamal Hadi Salim 05 Oct '23

05 Oct '23

My apologies for putting some marketing at the top of this email: Reminder, Early bird ticketing is open for 2 more days; the sooner you register the better it is for our planning... And now to tell you about one of my favorite talks this time.. ML/AI continues to be an inspiration for new Linux kernel infrastructure and a theme at netdev conf 0x17! There are a couple of patterns in ML traffic that inspire this work from Mina Almasry, Willem de Bruijn, Eric Dumazet and Kaiyuan Zhang: 1) There is _a lot of freaking data_ being transferred across the network - and it continues to increase dramatically (ex: to accommodate such large data transfers NICs with 800 Gps ports are starting to appear) and 2) The vast majority of this data is typically from a device(eg storage) to another device (eg GPU). For context, assume a scenario where data transfers from SSDs on machineA cross the network to GPUs on machineB; and on machine B: In the rx direction incoming data into the NIC goes to host memory, then from host memory to the GPU. In the tx direction outgoing data goes from GPU to host then out to the NIC. This data movement approach, with multiple round trips across the memory and PCIe busses, is no longer tenable at such large amounts of data quantity. Current host hardware is just not equipped to deal with this. While such bulk transfers amount to low packets/sec rates, you are simply not going to be able to achieve wire speed because of the PCIe and memory bandwith abuse from the multiple round trips (see talk [1] which delves into host limitations). Almasry et al introduce Device Memory TCP to solve these challenges. They implement socket API to enable a user to send directly from device memory (GPU in machineB above example or SSD in machineA) across the network - as well as be able to place incoming data directly in the device memory (GPU in machineB example). How does this magic happen? Well, we need some support from the NIC to split the header on rx such that the TCP headers go to the host and the rest of the data goes directly to the device. The reverse is true for tx direction. Packet headers get to go to stack and get processed in the standard (TCP) code path. Memory bandwidth use is cut dramatically and because NIC to device transfers now happen at the lowest level of the PCIe tree hierarchy PCIe bandwidth use is also cut down dramatically. With devmem TCP, Almasry et al with a setup of 4 machines of GPU+100Gbps NICs - and data sent and received directly from/to device memory - were able to reach ~96.6% line rate speeds. In the talk they will go over details of the kernel/uapi changes to achieve devmem TCP. cheers, jamal [1] https://netdevconf.info/0x17/sessions/talk/congestion-control-architecture-…

1 0