Edoardo Barbieri
on 24 February 2023
Welcome to this three-part blog series on real-time Linux.
Throughout the series, we will assess the key features of a real-time system. We will understand how a real-time capable Linux kernel differs from mainline, and touch upon the performance trade-offs you should consider when choosing real-time versus a low-latency kernel, for instance.
Although you don’t need prerequisite knowledge to follow this series, we recommend you have a basic understanding of the role played by Linux in the embedded portion of the compute spectrum. If you are new to the space and just getting started from scratch, get an intro to embedded Linux or delve into the official guide to Linux for embedded applications.
What is real-time Linux?
According to IDC’s forecasts, nearly 30% of the world’s data will need real-time processing by 2025 [1]. While workloads demand varying levels of real-time performance and desired cycle times differ depending on specific system configurations, analysts expect market adoption of real-time computing to continue and accelerate in the years ahead.
Demand for industrial PCs that drive control systems, industrial edge servers, PLCs, robots and drones with real-time capabilities is rising. On the software side, real-time computing via the Linux kernel is emerging as a valuable solution thanks to the rich support for hardware devices and peripherals. Among various approaches to bring real-time capabilities to the kernel, the PREEMPT_RT patch aims to increase predictability and reduce latencies by modifying the existing kernel code.
This three-part blog series aims to explain what real-time Linux is. Unfortunately, the topic is controversial, and there doesn’t seem to be 100% agreement over the terminology. In Part I, we will look at two definitions of what exactly is meant by real-time. As you will soon learn, a real-time system is time-bound and has well-defined, fixed time constraints.
However, there are several other real-time definitions, and some are contradictory. Because of the confusion, we will bring clarity by looking at some of the most common misconceptions around real-time Linux first. Later in this blog, we will provide a high-level overview of some verticals where real-time may be needed to further clarify its use cases.
Part II will look under the hood of a real-time Linux kernel. We will learn about preemption and the difference between user space to kernel space transition in a Linux kernel with and without real-time compute capabilities. Finally, in Part III we will introduce PREEMPT_RT and study the mechanisms the patchset implements to lower kernel latencies.
Without further ado, let’s get started.
Real-time Linux misconceptions
A real-time system only needs a real-time kernel
Before delving deeper into what constitutes real-time Linux, it is vital to understand that a real-time Linux kernel on its own will not necessarily make a system real-time. A kernel is only one component of a real-time system, and even the most efficient real-time operating system (RTOS) can be useless in the presence of other latency sinks. Even with a real-time capable kernel, each system usually requires specific tuning. Meeting the demands of real-time computing takes a careful understanding of the overall stack, from the underlying silicon to the operating system, the networking layer and applications.
Real-time = optimised performance
Another common misconception is that real-time results in so-called optimised performance. The misunderstanding often arises from video applications described as real-time because of the lack of perceived latency. These are usually just best-effort systems that are performant enough to remove any human notice of deadline failures. A real-time Linux kernel does not result in optimised performance. We are looking at a deterministic response to an external event, aiming to minimise response latency rather than optimised throughput. Indeed, a real-time Linux kernel will almost certainly perform worse than CFS or other schedulers in anything but task schedule response.
Always necessary
Also, a real-time OS is not always necessary. It sounds good for something to be ‘real time’, usually because of the performance connotation, but one must look at the actual consequences of a missed deadline and whether they warrant real-time requirements. For instance, if a deadline is on the order of seconds, a multi-GHz CPU with proper tuning will likely not miss it.
When to use real-time Linux
With the above clarifications in mind, when should one rely on a real-time Linux kernel?
According to a well-known definition, in a real-time system, the correctness of a computation depends not only upon the logical correctness of the result but also upon the time at which it produces it. If the timing constraints of the system are not met, system failure is said to have occurred [2]. The POSIX Standard 1003.1 has a similar definition. This standard states that real timeliness is the ability of an operating system to provide a required level of service in a bounded response time [3].
In short, a real-time system is best suited for extreme latency-dependent use cases, where a missed deadline results in failure, not just degradation of the system. Where do we typically see these scenarios?
Real-time Linux in target verticals and applications
Healthcare
Think of dedicated devices like life-support medical equipment, where operators expect the computing systems to fulfil tasks within a given deadline with catastrophic consequences in case of a missed deadline.
Factories
Besides medical robots in the healthcare industry, real-time capabilities play a role across various use cases and verticals. Among the lead market segments, we see process and discrete automation in the industrial sector. Determinism and real-time capabilities are vital on the factory floor. For instance, PLCs in an assembly line must deliver and process data in real-time to maintain system integrity and continued production, as a missed deadline could compromise the entire production line.
Telco
Implementing a virtualised, Open RAN architecture allows mobile operators to deploy specific network functions, allowing for greater flexibility and efficient utilisation of network resources. Running real-time Linux in an Open RAN architecture, operators can build modern, scalable, power-efficient and cost-effective 5G networks, ensure 6G readiness, and enable the deployment of innovative new services for subscribers.
Automotive
Real-time systems are also present in the automotive industry when the latency needs are really tight, and a missed deadline can be catastrophic, such as car brakes failing to prevent a crash. RTOS have generally been deployed in automotive to meet the required latencies. Nowadays, however, original equipment manufacturers and their Tier 1 suppliers increasingly rely on the Linux kernel for applications like infotainment systems and Human-Machine Interfaces.
Summary
Real-time Linux has much broader market applications than the mentioned ones, from product quality assurance and safety to oil and gas and transportation with strict precision requirements for automation systems.
How do you know if a low-latency OS is acceptable over a real-time Linux kernel? To answer this question, you must look at the latency requirements and the consequences of missing a deadline. An informed decision can only come from that data. If your latency needs are really tight and a missed deadline results in system failure, real-time Linux may be necessary. On the other hand, if the latency needs are a bit less stringent and the consequences are less dire, then a low-latency kernel may be enough.
Further reading
We covered what real-time is and the common misconceptions that arise when discussing real-time Linux. A real-time system responds to events within a specified deadline. If the timing constraints of the system are unmet, failure occurs. As mentioned above, there are broad market applications for real-time Linux.
Stay tuned for Part II of this mini-series, where we will discuss Linux kernel preemption, which preemption modes are available in mainline, and the role of the PREEMPT_RT patches in the Linux kernel.
