Multikernel

Motivations

We argue that the challenge of future multicore hardware is best met by embracing the networked nature of the machine, rethinking OS architecture using ideas from distributed systems. We investigate a new OS structure, the multikernel, that treats the machine as a network of independent cores, assumes no inter-core sharing at the lowest level, and moves traditional OS functionality to a distributed system of processes that communicate via message-passing.

The model we develop, called a multikernel (Figure 1), is not only a better match to the underlying hardware (which is net- worked, heterogeneous, and dynamic), but allows us to apply in- sights from distributed systems to the problems of scale, adaptivity, and diversity in operating systems for future hardware.

  • Systems are increasingly diverse

    A general-purpose OS today must perform well on an increasingly diverse range of system designs, each with different performance characteristics. Current OS designs optimize for the common hardware case; this becomes less and less efficient as hardware becomes more diverse. Worse, when hardware vendors introduce a design that offers a new opportunity for optimization, or creates a new bottleneck for existing designs, adapting the OS to the new environment can be prohibitively difficult.

  • Cores are increasingly diverse

    Diversity is not merely a challenge across the range of commodity machines. Within a single machine, cores can vary, and the trend is toward a mix of different cores. Some will have the same instruction set architecture (ISA) but different performance characteristics [34,59], since a processor with large cores will be inefficient for readily parallelized programs, but one using only small cores will perform poorly on the sequential parts of a program [31, 42]. Other cores have different ISAs for specialized functions [29], and many peripherals (GPUs, network interfaces, and other, often FPGAbased, specialized processors) are increasingly programmable.

    Current OS designs draw a distinction between general-purpose cores, which are assumed to be homogeneous and run a single, shared instance of a kernel, and peripheral devices accessed through a narrow driver interface. However, we are not the only researchers to see an increasing need for OSes to manage the software running on such cores much as they manage CPUs today [55]. Moreover, core heterogeneity means cores can no longer share a single OS kernel instance, either because the performance tradeoffs vary, or because the ISA is simply different.

  • The inter-core interconnect matters

    Even for contemporary cache-coherent multiprocessors, message- passing hardware has replaced the single shared interconnect for scalability reasons. Hardware thus resembles a message-passing network. While on most current hardware the cache-coherence protocol between CPUs ensures that the OS can continue to safely assume a single shared memory, networking problems like routing and congestion are well-known concerns on large-scale multiprocessors are now issues in commodity intra-machine interconnects. The implication is that system software will have to adapt to the inter-core topology.

  • Messages cost less than shared memory;

  • cache coherence is not a panacea.

    As the number of cores and the subsequent complexity of the inter-connect grows, hardware cache-coherence protocols will become increasingly expensive. As a result, it is a distinct possibility that future operating systems will have to handle non-coherent memory, or will be able to realize substantial performance gains by bypassing the cache-coherence protocol. The overhead of cache coherence restricts the ability to scale up to even 80 cores.

THE MULTIKERNEL MODEL

In this section we present our OS architecture for heterogeneous multicore machines, which we call the multikernel model. In a nutshell, we structure the OS as a distributed system of cores that communicate using messages and share no memory (Figure 1). The multikernel model is guided by three design principles:

  1. Make all inter-core communication explicit.
  2. Make OS structure hardware-neutral.
  3. View state as replicated instead of shared. These principles allow the OS to benefit from the distributed systems approach to gain improved performance, natural support for hardware heterogeneity, greater modularity, and the ability to reuse algorithms developed for distributed systems.

Barrelfish

Barrelfish is a new research operating system being built from scratch and released by ETH Zurich in Switzerland, with assistance from Microsoft Research. We are exploring how to structure an OS for future multi- and many-core systems. We are motivated by two closely related trends in hardware design: first, the rapidly growing number of cores, which leads to a scalability challenge, and second, the increasing diversity in computer hardware, requiring the OS to manage and exploit heterogeneous hardware resources.