Overview

Linux is an operating system that is widely used in embedded systems such as consumer electronics, networking equipment, machine control, industrial, automation and so on and so forth. However, all systems do not have the same requirements in term of determinism and sometimes determinism, the ability to schedule the highest priority tasks in a consistent and predictable way, really matters. This is the case for Financial Services, Networking (QoS), Robots, Air Traffic Control Systems...

Compared to other real-time operating systems, Linux has the advantage to be open source with great hardware support. Yet it was not designed to be real-time. It was originally designed to be a time-sharing system where the goal is to give the best throughput from the hardware using all the resources at the maximum. This is the opposite requirement of the real-time constraints that needs determinism even at a low global throughput.

Throughout the years, different approaches have emerged to overcome this problem. The first approach is to modify the Linux Kernel itself in order to get the required latencies or the real-time APIs. This approach is covered by the project PREEMPT_RT led by the Linux kernel developers Ingo Molnar, Thomas Gleixner and Steven Rostedt.

The second approach is to add a layer between the Hardware and the Linux kernel to handle the real-time requirements so that the Linux kernel behaviour can stay as it is. This approach has been taken into account by different project like RTLinux, RTAI and Xenomai. Since only the last one is maintained actively on ARM, we will only talk about it.

The latency

This is the time between a given event and the desired effect. Real-time process is often associated with physical event like interrupt from a physical button. Real-time must guarantee worst case latencies under a given reaction time. The source of latency can be wide. One application can blocks or prompts another one or locks a resource, if the kernel runs the application stop, etc... In other word, the kernel latency is the addition of the interrupt latency, the handler duration, the scheduler latency and the scheduler duration.

The preemption

The Linux kernel is a preemptive operating system. This means that when a task is running in user space mode, it can get interrupted and the interrupt handler wakes up another task and this task is scheduled as soon as the interrupt handler returns. However, when a task is already executing in kernel mode, the system call has to be finished before another task can be scheduled. This is because the Linux kernel does not do kernel preemption by default. And in another way, the time to call the scheduler could be (theoretically) unlimited!

The PREEMPT_RT project

The PREEMPT_RT project is a long-term project developed originally by kernel developer Ingo Molnar, Thomas Gleixner and Steven Rostedt. The goal is building “blocks” that would enable the implementation of real-time in the current mainline Linux kernel. So far, matures features have been merged that benefits both RT and upstream. At the end, the long-term branch of RT should disappear and get totally merged into the Linux Kernel. Since the beginning of the Linux kernel version 2.6, the project has added into mainline:

  • The O scheduler
  • The Kernel Preemption
  • Better POSIX real-time API support
  • Priority inheritance support for mutexes
  • High-resolution timer
  • Threaded interrupts
  • Spinlock annotations

Sleeping Spinlocks

The PREEMPT_RT has some specific features when set. When you select the CONFIG_PREMPT_RT in your RT kernel, all kernel spinlocks are replaced by mutexes (also called sleeping spinlocks). They are just normal locks that works well with threaded interrupts because threads can block while interrupt handlers not. In other words, all kernel code can (virtually) be preemptible.

Threaded interrupts

The other specificity from PREEMPT_RT is the threaded interrupts. In this mode, all interrupts handler are converted to threaded interrupts.

The PREEMPT_RT is available as a set of patch against mainline Linux Kernel. This means that it is better to start with a system that is already fully supported with mainline. The patches set are available at this address: http://www.kernel.org/pub/linux/kernel/projects/rt/ . There are two things not to forget before compiling the kernel, that’s setting CONFIG_PREEMPT_RT and setting the high-resolution timers.

The Real-time extension Xenomai

Instead of modifying the Linux kernel to fit the real-time constraint, the project Xenomai add an extra layer between the hardware and the Linux kernel in order to deal with the Real-time tasks separately. The Xenomai project started in 2001 and had the goal to emulate a traditional RTOS. The idea was to facilitate the porting of programs to Linux by reproducing an API of traditional RTOS such as VxWorks as well as the POSIX API. This extension will never be merged into mainline kernel.

Each Xenomai branch has a stable user/kernel ABI and support is as much as possible independent from the Linux kernel version and that allows backward and forward compatibility. It supports hard real-time in user space thanks to the skins implementation of the real-time core. It uses high-resolution timers as Timer system and the time base may be periodic for each skin. The RTDM skin allows writing real-time drivers. Xenomai is supported on several architectures including ARM on both 32/64bits.

The threads in the user-space have two modes. They can be handling by the Xenomai Scheduler in primary mode or handling by the Linux scheduler in secondary mode. Everything is serviced by the Adeos I-pipe which is the interrupt pipeline. Its role is to propagate interrupts through the different domains running on the hardware. Thread can migrate from primary mode to secondary mode depending on the system call to issue.

Xenomai applications are started like a normal Linux application. They have access to all Linux services. After their initialization, they can declare themselves as real-time and switch to primary mode. In this mode, they lose access of the Linux services but gain real-time properties offer by the Xenomai scheduler. They also only use the device driver implemented in Xenomai and not the Linux kernel one anymore. If they ask for any Linux services, they will automatically goes to the second mode and loose all real-time properties. This implies two things: the application needs to be split into non-real-time and real-time parts and real-time device driver need to be implemented.There are currently several notable drivers model available such as Serial driver, RT UDP/IP stack, RT CAN driver, etc...