Message Synchronization

Sensors run at different rates and their messages arrive with different delays. To combine a camera image with a LiDAR scan, you must pair messages taken at the same moment, by their timestamps, not by when they arrived. ROS 2's message_filters offer exact-time and approximate-time synchronizers for this.

Pairing policy Allowed time difference (approximate) Extra LiDAR delay Car speed

Pairing by timestamp

Every ROS 2 message carries a header stamp, the time the data was captured. An exact time synchronizer only pairs messages whose stamps are identical, which works for hardware-triggered sensors. An approximate time synchronizer pairs messages whose stamps are within an allowed difference, picking the closest. A moving object at speed $v$ is misplaced by

$$\Delta x = v\,\lvert t_{\text{camera}} - t_{\text{lidar}} \rvert$$

What to notice

Pairing by arrival time grabs whatever came last, so with LiDAR delay, its points land behind the car in the image. Exact time finds almost no pairs, since the two sensors are not triggered together. Approximate time pairs messages captured close together and the LiDAR points line up with the car, even with a long delay, because it uses capture time, not arrival time. Too small an allowed difference finds few pairs, and too large allows poor pairs. Hardware triggering, or at least synchronized clocks, makes all of this work better.