Real-Time Robotics: Before you start

This article is part of a mini-series on real-time robotics. Be sure to also checkout Part 2: The Kalman Filter and Part 3: The Extended Kalman Filter !

robot in need of a real-time controller

Robotics is a rapidly evolving field, and with the development of more advanced robotics systems and components, there is an increasing need for accuracy and reliability. Both of these can only be achieved if the robot’s sensors follow suit, but sensors being noisy by nature, the need for sophisticated filtering algorithms to process sensor data becomes even more important. One of the most effective and widely used real-time filtering algorithms is the Kalman Filter, but does it apply to Robotics ? In this mini series, we'll explore the Kalman Filter and if it can be used in real-time robotics applications. But first the basics - reading your sensors correctly.

Slip, Drift, Noise, and …

Robotics systems rely heavily on sensors to provide accurate information about the environment and the robot itself. However, sensors are not perfect and often suffer from inaccuracies due to various factors. These inaccuracies can be divided into three main categories: slip, drift and noise.

walking robot slipping over wet floor
  • Slip refers to the difference between the measured (or expected) position or orientation of a robot and its actual position or orientation. Slip can be caused by a variety of factors, but most commonly internal to the robot due to wear on mechanical components and external to the robot due to reduced friction in grasping an object or driving a wheel.
  • Drift refers to a gradual, persistent change in the output of a sensor over time. Drift can be caused by a variety of factors, such as temperature changes, aging of the sensor, or in case of sensor fusion, a bad calibration.
  • Noise refers to random, unwanted variations in the output of a sensor. Noise can be caused by a variety of factors, such as electrical interference, mechanical vibrations, or random fluctuations in the sensor's environment.

I wish that was it, but there’s a fourth factor that we first need to control before we can address the above three correctly…

Jitter ?!

Yes. Let us explain: Jitter refers to variations in the time at which a sensor reading is taken, rather than the value of the reading itself. Take some time to study the graph below: slip, drift and noise will distort our observation of the actual value, but jitter will distort our estimate of when the change happened.

jitter explained in a simple time diagram

In this context, jitter is a problem for every real-time control algorithm that relies on accurate timing information (hence the term ‘real-time’), as jitter will introduce errors into the control signals that are generated, no matter how good their quality !

basketball playing robot

Timing is everything !

Example: Imagine you are dribbling a basket ball. Each time the ball is about to hit its peak, you need to push it down again. The timing of pushing down depends heavily on your eyes seeing and hands feeling the ball.

Imagine now that your hands and eyes sense a different time for the basketball’s peak. Even worse, sometimes your eyes see the peak first and then the hands feel it, and next the other way around. Since you can’t get the timing right, the ball will quickly loose height and come down to rest.

These jumps in timing is what we call jitter, and even with perfect eyes and perfect touch, jitter will make you loose the ball. Exactly the same will happen in real-time control systems, the controller will fail to keep the system running.

Addressing Jitter First

synchronized clocks

To address jitter directly, we use synchronization techniques to align the timing of sensor readings with a common reference signal. The most common synchronization techniques in robotics are:

  1. Common clock signals: In this technique, multiple sensors sample their inputs at regular intervals, using a common electrical pulse signal to coordinate their operation. So you are guaranteed that every measurement was done at the same time.
  2. Precision timing protocols: Protocols such as IEEE 1588 (Precision Time Protocol) can be used to synchronize the operation of multiple sensors and the control system with high accuracy, up to microseconds, using a combination of hardware and software techniques.

If jitter can not be addressed directly, so measurements are made out-of-cadance, one can revert to measuring the time at which a sensor reading was taken. This requires a setup with three important features:

  1. Taking Timestamps: Many sensors have built-in real-time clocks or timestamps that can be used to record the time at which a reading was taken.
  2. Synchronizing clocks: An external clock signal, such as a GPS signal or the IEEE 1588 protocol above, can be used to synchronize the clocks of the sensors and the control system, allowing to record the exact time at which each reading was taken.
  3. Re-alignment: A means to re-align the measurements to a single time instant, by interpolating or extrapolating each sensor reading to that time instant.

Eliminating jitter does not eliminate slip, drift and noise in a robotic control system. It only allows us to start working on the real-time data readings. Now let’s explore the open source libraries available to implement real-time data readings.

Eliminating Jitter in C/C++

There are several open-source C/C++ libraries available that you can use to implement a real-time sensor reading. Python is generally not considered a suitable language for implementing real-time control systems, due to the way it is designed and implemented.

Some well known libaries are:

    • Xenomai: is a real-time operating system (RTOS) that provides a set of libraries and tools for building real-time applications on Linux. It is designed to support a wide range of real-time applications, including control systems, data acquisition systems, and embedded systems.
    • Orocos Real-Time Toolkit (RTT): is a C++ library for building real-time control systems on Linux. It is part of the Orocos Project, which is a set of tools and libraries for building robotics applications.
    • Rock Robotics : integrates with the Orocos RTT and ROS and contains a Stream Aligner component to align data streams based on time stamps.
    • Real-Time Linux (PREEMPT_RT): A library specifically designed for implementing EKF-based simultaneous localization and mapping (SLAM) algorithms.
    • Simulink Coder: generates and executes C and C++ code from Simulink® models, Stateflow® charts, and MATLAB® functions. The generated source code can be used for real-time and nonreal-time applications, including simulation acceleration, rapid prototyping, and hardware-in-the-loop testing


This article touched on the basics of robotics: proper jitter-free real-time sensing. A next step is to process these measurements into a meaningful state, for example the position or velocity of your robot. That’s what we’ll cover in Part 2 !

Continue to Part 2

Discover Our Expertise

Our Expertise in Real-time Motion Control

Our OROCOS development work of the Real-Time Toolkit (RTT) gave us the leading expertise in Real-time Motion Control for both ROS and OROCOS.

Read Our Expertise in Real-time Motion Control

Inverse and Forward Kinematics

Intermodalics has been at the core development of the Kinematics and Dynamics Library (KDL).

Read Inverse and Forward Kinematics

Robot Operating System ROS

We are a ROS-native company, standardising all our tooling on the ROS ecosystem. We offer ROS Consulting and best practices for standard ROS paradigms and insights into ROS design decisions its shortcomings.

Read Robot Operating System ROS