Introducing ‘ixy’, a simple user-space network driver written in high-level languages like Rust, Go, and C#, among others

Researchers Paul Emmerich et al have developed a new simple user space network driver called ixy. According to the researchers, ixy is an educational user space network driver for the Intel ixgbe family of 10 Gbit/s NICs. Its goal is to show that writing a super-fast network driver can be surprisingly simple in high-level languages like Rust, Go, Java and C# among others. Ixy has no dependencies, high speed, and a simple-to-use interface for applications to be built on it. The researchers have published their findings in a paper titled The Case for Writing Network Drivers in High-Level Programming Languages.

Initially, the researchers implemented ixy in C and then successfully implemented the same driver in other high-level languages such as Rust, Go, C#, Java, OCaml, Haskell, Swift, Javascript, and Python. The researchers have found that the Rust driver executes 63% more instructions per packet but is only 4% slower than a reference C implementation. Go’s garbage collector keeps latencies below 100 µs even under heavy load.

Network drivers written in C are vulnerable to security issues

Drivers written in C are usually implemented in production-grade server, desktop, and mobile operating systems. Though C has features required for low-level systems programming and fine-grained control over the hardware, they have vulnerabilities for security as “they are exposed to the external world or serve as a barrier isolating untrusted virtual machines”.

The paper states that the C code “accounts for 66% of the code in Linux, but 39 out of 40 security bugs related to memory safety found in Linux in 2017 are located in drivers. These bugs could have been prevented by using high-level languages for drivers.”

Implementing Rust, Go and other high level languages in ixy network driver

Rust: A lightweight Rust struct is allocated for each packet that contains metadata and owns the raw memory. The compiler enforces that the object has a single owner and only the owner can access the object. This prevents use-after-free bugs despite using a completely custom allocator. Rust is the only language evaluated in the case study that protects against use-after-free bugs and data races in memory buffers.

Go: It has an external memory that is wrapped in slices to provide bounds checks. The atomic package in Go also indirectly provides memory barriers and volatile semantics thus offering stronger guarantees.

C#: The researchers have implemented two external memories out of the many available. It offers a more direct way to work with raw memory by offering full support for pointers with no bounds checks and volatile memory access semantics.

Java: The researchers have targeted OpenJDK 12 which offers a non-standard way to handle external memory via the sun.misc.Unsafe object that provides functions to read and write memory with volatile access semantics.

OCaml: OCaml Bigarrays backed by external memory is used for DMA buffers and PCIe resources, the allocation is done via C helper functions. The Cstruct library from the OCaml allowed researchers to access data in the arrays in a structured way by parsing definitions similar to C struct definitions and generating code for the necessary accessor functions.

Haskell: It is a compiled functional language with garbage collection. The necessary low-level memory access functions are available via the Foreign package. Memory allocation and mapping is available via System.Posix.Memory.

Swift: Its memory is managed via automatic reference counting, i.e., the runtime keeps a reference count for each object and frees the object once it is no longer in use. It also offers all the features necessary to implement drivers.

JavaScript: ArrayBuffers is used to wrap external memory in a safe way, these arrays can then be accessed as different integer types using TypedArrays, circumventing JavaScript’s restriction to floating-point numbers. Memory allocation and physical address translation is handled via a Node.js module in C.

Python: For this driver, the implementation was not explicitly optimized for performance and meant as a simple prototyping environment for PCIe drivers and as an educational tool. The researchers have provided primitives for PCIe driver development in Python.

Rust is found to be the prime candidate for safer network drivers

After implementing the network driver ixy in all high-level languages, the researchers conclude that Rust is the prime candidate for safer drivers. The paper states, “Rust’s ownership based memory management provides more safety features than languages based on garbage collection here and it does so without affecting latency.”

Other languages like Go and C# are also a suitable language if the system can cope with sub-millisecond latency spikes due to garbage collection. Other languages like Haskell and OCaml will also be more useful if their performance is less critical than having a safe and correct system.

Though Rust performs better than C, it is 4% slower than the C driver. The reason behind is that Rust applies bounds checks while C does not. Another reason is that C does not require a wrapper object for DMA buffers.

introducing-ixy-a-simple-user-space-network-driver-written-in-high-level-languages-like-rust-go-and-c-among-others-img-0

Image Source: Research paper

Users have found the result of this high-level language implementation of network drivers quite interesting.

https://twitter.com/matthewwarren/status/1172094036297048068

A Redditor comments, “Wow, Rust and Go performed quite well. Maybe writing drivers in them isn't that crazy”

Many developers are also surprised to see the results of this case study, especially the performance of Go and Swift.

A comment on Hacker News says, “The fact that Go is slower than C# really amazes me! Not long ago I switched from C# to Go on a project for performance reasons, but maybe I need to go back.”

Another Redditor says, “Surprise me a bit that Swift implementation is well below expected. Being Swift a compiled native ARC language, I consider the code must be revised.”

Interested readers can watch a video presentation by Paul Emmerich on ‘How to write PCIe drivers in Rust, go, C#, Swift, Haskell, and OCaml’. Also, you can find more implementation details in the research paper.