1. Introduction
A multi-sensor system refers to a system that obtains and combines data from multiple sensors, with data fusion being a pivotal aspect in such systems. In the context of Simultaneous Localization and Mapping (SLAM) systems, multi-sensor fusion is a critical issue, as it involves integrating data from different sensors to enhance the robustness, accuracy, and reliability of the system [
1,
2,
3]. However, the sampling frequencies and timestamps of different sensors may vary, leading to data asynchronicity. Hence, a high-precision time synchronization mechanism is required to align data temporally [
4,
5].
Light Detecting and Ranging (LiDAR), as an important SLAM method that uses laser beams to measure distance, velocity, and other properties, has found widespread applications in various fields including autonomous driving [
6,
7], environmental monitoring [
8], geological exploration [
9], and robot navigation [
10,
11,
12]. However, LiDAR shares the same multi-sensor data fusion challenges as other sensing technologies. A typical LiDAR system consists of multiple sensor units that output data at different frequencies, including laser trigger pulses, positioning and attitude data from the Positioning and Orientation System (POS), rotation angles, the number of revolutions, and the rotational speed of the laser scanner. These data require precise time synchronization during the generation of LiDAR point cloud data. The accuracy and stability of internal multi-sensor time synchronization play a crucial role in ensuring data consistency, target tracking precision, and system efficiency improvement [
13,
14,
15]. More precise time-synchronized multi-sensor data enhances the quality of inputs for sensor fusion machine learning (ML) models, thereby advancing the performance of artificial intelligence (AI) applications [
16,
17].
There are many factors that affect the accuracy of multi-sensor time synchronization, namely, time reference error, signal transmission delay, system response error, laser jitter error, and timestamping error. The former four errors are tied to hardware or system design. For instance, the reference error refers to the inherent error in the pulse signal that serves as the benchmark. If the benchmark experiences jitter or loss, it can directly affect the accuracy of time synchronization. System transmission error refers to the unexpected delays or loss of synchronization data between data acquisition time and synchronization time, caused by interference or inherent issues in external data acquisition devices such as POS, which lead to data transmission delays or data loss. System response error refers to the delay error that occurs during the acquisition, time synchronization, and storage of external data due to insufficient real-time performance of the Micro-Controller Unit (MCU) within the control system. Laser jitter refers to the temporal fluctuation between the laser trigger signal and the actual emission of the laser beam, which is inherent to the characteristics of the laser device. If the laser trigger signal is used as the synchronization signal for the laser echo, it will inevitably introduce laser jitter error [
18,
19].
In addition to the previously mentioned factors influencing the time synchronization precision of multi-sensor systems, the operational and environmental conditions of the system itself also have an impact on multi-sensor time synchronization. Such factors include conditions of the workplace, carrier platform vibrations and shocks, electromagnetic environment, and lighting conditions. More specifically, positioning and precise timing services cannot be provided in Global Navigation Satellite System (GNSS)-denied environments such as indoors and caves. Variations in temperature and humidity have the potential to induce drifts in sensor operating characteristics, particularly in clock circuits where temperature-induced drift effects may occur. Vibrations and shocks from the carrier platform may disrupt the operation of the POS system, thereby compromising the accuracy of its positioning and attitude measurements. Exposure of the multi-sensor system to intense electromagnetic interference can negatively impact internal circuitry, leading to noise generation or even system malfunction. Furthermore, excessively intense ambient light has the capacity to disrupt the photoelectric system, possibly resulting in the blinding of photoelectric devices.
Timestamping error is linked to the selection of time synchronization methods and software algorithms. Implementing an effective time synchronization principle along with suitable software algorithms can significantly minimize timestamping error. Common methods for internal multi-sensor time synchronization in LiDAR systems include the following categories, and their pros and cons are listed in
Table 1.
(1) Hardware synchronization method: This method employs hardware circuits [
14,
20], Field-Programmable Gate Arrays (FPGAs) [
21], and other kinds of specified hardware [
22,
23] to synchronize the clocks of multiple sensors, ensuring that all sensors share the same time reference. This approach is direct and simple, avoiding the need for additional communication and protocols, resulting in lower latency. However, it requires specialized hardware design and implementation, which may increase costs and complexities.
(2) Software synchronization method: This method utilizes software algorithms to synchronize the clocks of multiple sensors. It can be implemented through the Precision Time Protocol (PTP) [
24], or other custom protocols [
25,
26]. This approach is flexible and does not require additional hardware, but it may introduce some delays and complexities in communication and protocol.
In recent years, within the SLAM systems for robotics or autonomous driving, there has been a surge of research adopting continuous-time trajectory estimation [
27,
28], extended Kalman filters [
29], or deep-learning-based frameworks [
30] to achieve LiDAR–Inertial Measurement Unit (IMU)–Camera time synchronization. Although these methods provide high synchronization accuracy, they require significant computational resources, making it challenging to achieve real-time online time synchronization in high-frequency, high-precision SLAM applications.
(3) Hybrid synchronization method: This method combines hardware and software synchronization techniques to achieve high-precision time synchronization. It can be implemented through a combination of GNSS [
14] or other hardware circuits and software algorithms [
4]. This approach can achieve high precision and low latency, but it requires additional hardware and complex software design.
The aforementioned time synchronization methods all use a precise third-party time reference, either internal or external traditional time synchronization (TRTS), to align various frequency data within the LiDAR system. This approach ensures high synchronization accuracy and minimizes timestamping errors. However, the following issues still exist: First, when assigning timestamps to LiDAR data from distinct sensors, a timestamping error of time stepping arises for each data. This error compounds to a timestamping error of time stepping among the different frequency data of the LiDAR during point cloud processing. Secondly, laser pulses exhibit time jitter, leading to discrepancies between the laser range data and interpolated data derived from other low-frequency sensors during point cloud generation.
To address the issues associated with external time synchronization methods like unreliable or missing external clocks, and internal hardware synchronization methods such as increased hardware costs and reduced reliability due to additional hardware components, we propose an internal time synchronization method, i.e., laser pulse-driven time synchronization (LPTS), in our custom-developed LiDAR system. To generate a time reference for other low-frequency data within the LiDAR system, electrical pulses synchronized with laser beams are utilized to drive the MCU’s high-precision internal timer. The LPTS method can decrease the timestamping error between the different frequency data of the LiDAR. Furthermore, the LPTS method adopts an electrical signal synchronized with the laser as the time synchronization reference signal, thereby circumventing the issue of laser timing jitter.
The results demonstrate that the LPTS method effectively reduces the impact of laser jitter and achieves better synchronization accuracy compared to the TRTS method, without requiring additional hardware such as an FPGA. Instead, it relies solely on the peripherals integrated within the MCU, thereby delivering enhanced performance at a lower hardware cost, making it well-suited for time synchronization in multi-sensor systems.
3. Experimental Setup
To discuss the experimental setup for implementing our proposed LPTS time synchronization method, the hardware and software of our custom-developed LiDAR system will be introduced in this section.
3.1. System Hardware
The block diagram for implementing the LPTS time synchronization method in our custom-developed LiDAR system is illustrated in
Figure 2, comprising a control unit, laser unit, POS unit, laser scanner unit, data acquisition and storage unit, and Human–Machine Interface (HMI). The blue solid lines, red dashed lines, and green dotted lines signify the data connection channels, time synchronization connections, and laser beams among the units, respectively.
The core module of the LiDAR system is the control unit, which utilizes a high-performance MCU running an embedded real-time operating system to initialize, control start/stop functions, detect status, and manage errors of the system’s units. Data links are employed by the main control unit for communication and control with other units. The time synchronization signal within the LiDAR system is a Transistor–Transistor Logic (TTL) pulse signal synchronized with the laser beam from the laser unit. The control unit records and stores this synchronization pulse signal, which is then used to timestamp the scanning angle provided by the laser scanner unit and the positional, angular, and other data from the POS unit. The timestamped data frames are subsequently stored in the data acquisition and storage unit. After being collimated by the lens, the laser beam is directed by the laser scanner unit onto the target object. Subsequently, the reflected laser beam is once again directed by the laser scanner unit, focused by the lens, and finally illuminated onto the Avalanche Photo Diode (APD) within the data acquisition and storage unit, where it is converted into an electrical signal. The electrical signal is digitized and stored in the data acquisition and storage unit.
The key performances of the custom-developed LiDAR system are presented in
Table 2.
3.1.1. Control Unit
In our custom-developed LiDAR system, the control unit is tasked with initiating each unit, commencing and ceasing system operations, coordinating time synchronization, engaging with users, managing errors, and gathering data. Time synchronization plays a critical role as it impacts the integration of position, attitude, distance, angle, and other data produced by the LiDAR system, ultimately influencing the precision of SLAM.
Within the LiDAR system, the control unit is equipped with an STM32H743 MCU from STMicroelectronics, featuring a clock speed of 400 MHz, 2 MB of Flash memory, 1 MB of RAM, and 168 I/Os supporting up to 35 communication ports. This MCU is externally expanded with 32 MB of Quad Serial Peripheral Interface (QSPI) Flash and 32 MB of Synchronous Dynamic Random Access Memory (SDRAM), enhancing the control unit’s real-time performance and scalability in the LiDAR system. The embedded software operates on the FreeRTOS real-time operating system with a thread scheduling frequency of 1000 Hz and prioritizes interrupt responses to ensure timely system reactions. Furthermore, the embedded software leverages the TouchGFX HMI framework to deliver a seamless HMI and robust interaction capabilities, simplifying the configuration of LiDAR system parameters.
3.1.2. Laser Unit
The laser unit comprises a semiconductor-pumped pulsed laser emitting at a wavelength of 532 nm, with a repetition frequency of 2 kHz, a pulse width of 3 ns, and a single pulse peak power of 200 kW. This laser can produce a TTL electrical pulsed signal in sync with the laser beam, serving as a reliable time reference for the LPTS method.
A lens is integrated at the outlet of the laser unit to collimate the emitted laser, ensuring that the spot diameter is less than 2 mm at a distance of 50 m.
The control unit interfaces with the laser unit via the Universal Asynchronous Receiver–Transmitter (UART) port, enabling functions such as laser initialization, parameter adjustment, control activation, status queries, and more.
3.1.3. POS Unit
The POS unit utilizes the INS-DH-OEM by Inertial Labs, capable of reaching a maximum data update rate of 100 Hz via UART. It delivers horizontal and vertical positioning accuracies of 0.01 m and 0.02 m, respectively, in Real-Time Kinematic (RTK) mode. The unit provides a heading accuracy of 0.05 deg and a pitch and roll accuracy of 0.015 deg.
The HG4930 IMU built into the NS-DH-OEM has a vibration tolerance range within the frequency of 10–200 Hz, with a maximum vibration amplitude of 20 G (196 ).
3.1.4. Laser Scanner Unit
The laser scanner unit employs a servo motor to maneuver a wedge reflector, with the servo motor attaining a maximum speed of 2000 rpm and offering a scanning angle accuracy of 0.072 deg. The servo motor driver connects to the control unit through an RS485 to the UART bus and communicates with the MCU through the Modbus protocol. The control unit can initiate the servo motor upon system startup, regulate its speed, and retrieve data on the motor’s angle and laps.
After being reflected and deflected by the laser scanner unit, the laser performs a circular scan on the target. The laser reflected back from the target is focused by the laser scanner unit and sent to the data acquisition and storage unit.
3.1.5. Data Acquisition and Storage Unit
The LiDAR system captures and records the full waveform of the laser echo signal. The data acquisition and storage unit is equipped with two APD modules. These modules serve to convert the laser beam emitted by the laser device and the laser beam reflected from the target, respectively. During the subsequent point cloud data processing phase, the distance to the target is determined by calculating the time difference between the peaks of the two laser waveforms. Due to the considerable attenuation of the reflected laser beam and its potential coupling with noise, we have implemented filtering and amplification circuits preceding the APDs. This ensures that the amplitude of the converted electrical signal falls within the ±2.5 V range. Additionally, this unit incorporates four channels of 12-bit high-speed Analog-to-Digital Converters (ADCs), operating at a sampling rate of 2 Giga-Samples Per Second (GSPS), and offering 2 terabytes of data storage. The control unit configures the operational mode and parameters of the data acquisition and storage unit via Ethernet using the Reduced Media Independent Interface (RMII), transmitting timestamped valid data extracted from the POS unit, servo motor speed, rotation angle, and accumulated lap information to the storage unit at regular intervals. In the initial phase of LiDAR data post-processing, the full-waveform echo data is integrated with the information provided by the control unit while utilizing timestamp synchronization.
3.1.6. UAV Platform
The unmanned aerial vehicle (UAV) platform used in the experiment is a hexacopter with a payload capacity of 25 kg, capable of carrying a LiDAR system weighing 18 kg. The battery pack carried by the unmanned aerial vehicle (UAV) can sustain a flight time of 2 h and provide a maximum power output of 500 W, fully meeting the 200 W power demand of the LiDAR system. Furthermore, the UAV can withstand winds of up to force three (speeds < 6 m/s). Flexible materials are used for connecting when installing the LiDAR system onto the UAV, aiming to reduce vibrations and thus minimize the impact on the operation of the IMU in the POS system.
The LiDAR system operates within a temperature range of 0–40 °C and functions optimally during the daytime, provided there is no direct sunlight exposure to the lens, as well as at night. The flight parameters adopted in the experiment are listed in
Table 2.
After selecting and designing the various hardware components of the LiDAR system, it is necessary to consider the system’s requirements for shock and vibration resistance when operating on a UAV platform. When designing and assembling the various units of the LiDAR system, fixed connections should be utilized to ensure a consistent working environment for all components within the LiDAR system. At the same time, to guarantee the reliability of electronic components on the Printed Circuit Board (PCB), it is imperative to prioritize welding quality and spray a three-proof paint to fulfil insulation, moisture resistance, and corrosion resistance requirements.
The experimental setup of our custom-developed LiDAR system is shown in
Figure 3. The LiDAR system is mounted on a UAV platform, and the laser beam is directed towards the target object. The POS unit is assembled on the UAV platform to provide real-time positioning and attitude data.
3.2. System Software
When designing system software, the primary considerations are software robustness and real-time responsiveness. Therefore, we chose a Real-Time Operating System (RTOS) and related software frameworks for the software design. Tasks with low real-time requirements are handled through task switching; tasks with high real-time requirements are responded through interrupts. Simultaneously, we adopted a mature Graphical User Interface (GUI) framework to design the HMI. These measures enable us to successfully integrate the LPTS method proposed in this paper into the LiDAR system, achieving high-precision time synchronization for multiple sensors.
The software flowchart of the LiDAR system, with the LPTS method integrated, is described in
Figure 4. Upon powering on the LiDAR system, the MCU within the control unit initiates the execution of specialized code. The initial step entails the control unit initializing various components, such as MCU peripherals, external units, GUI, and FreeRTOS. Upon completion of the initialization process, the entire LiDAR system is ready, triggering the start of thread scheduling by the RTOS.
Furthermore, the RTOS switches between threads, specifically the GUI thread, system status thread, and hardware control thread, in accordance with their respective priorities. The GUI thread is responsible for handling touch interactions and screen updates, and it facilitates an HMI. Simultaneously, the system status thread assesses system conditions, monitors errors, and administers error processing. The hardware control thread, on the other hand, manages communication with external hardware components including the POS unit, laser unit, laser scanner unit, and data acquisition and storage unit. This thread controls start/stop operations and monitors the status of these components.
Finally, to enhance the real-time performance of the LiDAR system, interrupts are utilized to respond to critical events like laser pulse counting, POS data frame timestamp, and laser scanner data frame timestamp. The system prioritizes interrupt responses to ensure timely system reactions. Specifically, the laser pulse capture interrupt is assigned the highest priority level in the Nested Vector Interrupt Controller (NVIC) to guarantee zero data loss during high-load operations.
A global variable,
, which tracks and stores the number of laser pulses, is the
x in (
9). Upon receiving a TTL pulse from the laser unit, the MCU triggers an interrupt, initiating the laser pulse interrupt service program. The timer counting register dedicated to precise time synchronization is reset, followed by incrementing the global variable
by 1. Additionally, another global variable,
, representing the timestamp of the full-wave echo signal, synchronizes with the
. Subsequently, when data frames from the POS unit or the laser scanner unit are transmitted to the MCU, the corresponding interrupt is triggered. The timestamping process updates the
and
variables with the contemporaneous value of the
, which are
m and
n in (
10) and (
11), respectively. The
and
variables are refreshed based on the present value of the timer’s counting register, which represent the
and
in Equations (10) and 11, respectively.
Given the recorded parameters
x,
m,
n,
, and
, the timestamps of the POS and laser scanner data can be calculated using the LPTS method through (
9)–(
11). Additionally, the interpolated timestamp of the POS data, aligned with the laser pulse, can be determined using (
12). Similarly, the interpolated timestamp of the laser scanner data can be obtained through a comparable process.
5. Conclusions
This study implements a laser pulse-driven multi-sensor time synchronization method through a custom-developed LiDAR system. We systematically analyze the timestamping precision discrepancies between LPTS and TRTS methods, with experimental validation confirming their performance characteristics. The principal conclusions derived from the experimental data are as follows:
(1) Theoretical analysis reveals that the LPTS method exhibits a timestamping error of ±1, contrasting with the TRTS method’s twofold greater error margin (±2). This conclusion aligns with experimental data in
Figure 8, where LPTS demonstrates better error distribution characteristics, with timestamping error deviations predominantly clustered around zero compared to TRTS.
(2) The responding delay of the control system contributes a positive timestamping error for both the LPTS and TRTS methods.
Through integrated theoretical modeling and experimental validation, our findings demonstrate that the adoption of an event-driven architecture effectively decouples internal synchronization precision from the stability of external GNSS signals. Specifically, the LPTS method exhibits a timestamping error of ±1, enabling 0.1 s level time synchronization accuracy in our custom-developed LiDAR system.
However, it is important to note that the achievement of high-precision time synchronization, whether using LPTS or TRTS, relies on high-precision internal timers of the MCU and the MCU’s rapid response to interrupts. To pursue even higher time synchronization accuracy, it is advisable to select an MCU with stronger real-time performance and a more precise clock source when designing the LiDAR system. Furthermore, it is imperative to enhance the response to interrupt events during the design phase of the control system software.