Movement Direction Classification Using Low-Resolution ToF Sensor and LSTM-Based Neural Network

Abstract

This study proposes an effective method for identifying human movement direction in indoor environments by leveraging a low-resolution time-of-flight (ToF) sensor and a long short-term memory (LSTM) neural network model. While previous studies have employed camera-based or high-resolution ToF-based sensors, we utilize an 8 × 8 array ToF sensor, which is neither expensive nor related to any privacy issues. Furthermore, in contrast to the conventional rule-based algorithm, the proposed method employs the LSTM model to effectively handle the sequential time-series data. Experimental evaluations, including both basic single-person scenarios and complex multi-user challenge scenarios, confirm that the proposed LSTM-based approach achieves outstanding accuracy of 98% in identifying human entry and exit movements.

1. Introduction

The advancement of sensor-based automation technologies and the development of the internet of things (IoT) have not only enhanced the convenience of daily life but also contributed significantly to improving energy efficiency in building management systems [1,2,3,4]. In particular, public institutions and commercial buildings increasingly require automated systems capable of accurately detecting indoor occupancy to optimize energy consumption related to heating, ventilation, air conditioning (HVAC) and lighting systems [5,6]. As a result, extensive research has been performed on developing energy optimization systems based on human detection and movement analysis technologies [7,8,9,10,11].

Traditional approaches have mainly relied on camera-based sensors or high-resolution time-of-flight (ToF) sensors, each with inherent limitations. Camera-based systems raise concerns about personal privacy, as they expose facial features and movement trajectories, making them unsuitable for privacy-sensitive environments, such as restrooms [12,13]. On the other hand, although the high-resolution ToF sensors do not raise any privacy issues, these sensors are expensive, so there are limitations for large-scale deployment and maintenance [14,15,16]. For this reason, several studies have been performed on occupancy detection and movement estimation using low-resolution ToF sensors. However, existing methods often rely on rule-based algorithms, such as blob detection and random forest regression models, which may demonstrate acceptable performance in controlled settings but lack adaptability in complex environments involving multiple users and dynamic conditions [17].

To overcome these issues, this study proposes a novel approach that combines a low-resolution ToF sensor with a long short-term memory (LSTM) neural network capable of learning from sequential time-series data. The proposed method utilizes an 8 × 8 low-resolution ToF sensor, effectively resolving the privacy issues associated with camera-based systems while offering greater cost efficiency compared to high-resolution ToF sensors. Although low-resolution ToF sensors are inherently limited in distinguishing fine-grained human actions or resolving interactions in densely populated environments, the objective of this study is confined to recognizing human presence and movement direction in low-traffic, structured indoor spaces, such as doorways and corridors. In such contexts, detailed spatial resolution is not essential, whereas privacy preservation and cost efficiency are of primary concern. Accordingly, the adoption of an 8 × 8 ToF sensor array is considered both sufficient and appropriate for the intended application domain. Furthermore, to overcome the inflexibility and performance limitations often observed in conventional rule-based algorithms, the proposed method employs a long short-term memory (LSTM) neural network, which enhances adaptability to dynamic indoor environments, including those involving multiple users. By directly learning human movement patterns to analyze the sequence data acquired by the sensors, the proposed LSTM model can classify the direction of entry and exit with high accuracy even in complex indoor environments. The experimental results demonstrate that the proposed method achieves a high average accuracy of 98% across various scenarios, including single- and multiple-user cases, and maintains robust performance under unseen user conditions, such as different walking patterns and speeds.

2. Background

2.1. Research on People Counting

Occupancy counting plays a critical role in building energy management and automation systems, particularly in reducing energy consumption through the optimization of lighting and HVAC operations. The core objective of this technology is to accurately detect the number and density of people entering and occupying indoor spaces, and a variety of sensor-based approaches have been studied to achieve this.

One widely adopted commercial method utilizes passive infrared (PIR) sensors [18,19,20]. PIR sensors detect changes in infrared radiation caused by human motion using a pyroelectric sensor and a Fresnel lens. They are typically mounted on ceilings or walls and offer a wide field of view (FoV) for motion detection. However, PIR sensors cannot determine movement direction, and they are susceptible to false positives from environmental temperature changes. Considering these challenges, active sensing technologies such as low-resolution ToF sensors present a meaningful alternative to conventional passive systems, especially in scenarios where determining the direction of human movement is required. Meanwhile, high-resolution camera-based systems have also been extensively researched for occupancy detection [21,22,23]. These systems can achieve high accuracy by integrating object detection algorithms, such as YOLO and Faster R-CNN. Nonetheless, camera-based approaches raise significant privacy concerns, especially in sensitive environments, such as hospitals and restrooms. Although some studies have attempted to address privacy issues through anonymization techniques (e.g., face blurring or low-resolution imaging), these solutions often increase computational overhead and decrease detection accuracy while also being sensitive to environmental changes [24,25,26].

Other studies have explored the use of radio frequency (RF) sensors, including Wi-Fi, Bluetooth and ultra-wideband (UWB), to estimate occupancy [27]. These techniques infer the number of users by measuring signal strength from mobile devices within a space. However, they require users to carry mobile devices, which limits their applicability in situations where users do not have smartphones or where signal quality is poor. Even though low-resolution thermal cameras have also been investigated for presence detection, distinguishing between human heat sources and environmental thermal noise nevertheless remains a major challenge [28,29].

Meanwhile, recent studies have also explored the use of low-resolution ToF sensors, which are inherently free from privacy concerns and offer significantly lower costs compared to both camera systems and high-resolution ToF sensors. These characteristics make them a promising alternative for smart building applications. For instance, one prior study combined a random forest regression model with clustering techniques and the Hungarian algorithm to estimate human movement direction using low-resolution ToF data [17]. However, this approach exhibited significant performance degradation in densely populated environments due to the structural limitations of clustering algorithms. Specifically, when two individuals moved within close distance (less than 1 m), the error rate increased by up to 20.5%, limiting the system’s applicability in multi-user indoor environments. Moreover, due to constraints imposed by the COVID-19 pandemic, this study generated approximately 900 datasets in a simulated environment, which may have affected the reliability and generalizability of the results in real-world scenarios.

To overcome these challenges, the present study proposes a new approach that integrates a low-resolution ToF sensor with an LSTM neural network, which is highly effective in learning from sequential data. Unlike rule-based algorithms, the LSTM model can capture long-term temporal dependencies and maintain robust performance in dynamic and complex multi-user environments. Additionally, the proposed model can be easily adapted to new environments with minimal additional training, setting it apart from previous studies.

2.2. Overview of Recurrent Neural Network and LSTM

Time-series data are characterized by the dependency of the current state on its previous states. To effectively process such sequential data, the recurrent neural network (RNN) architecture was introduced in [30]. Unlike conventional feedforward neural networks, RNNs maintain and reuse information from previous time steps through hidden layers, enabling the learning of continuous patterns over time, as seen in Figure 1. This capability has led to their wide application in various time-series analysis tasks, including natural language processing, stock market prediction and human behavior analysis.

However, traditional RNNs suffer from the vanishing gradient problem when dealing with long sequences. During backpropagation, the gradients used to update weights can diminish exponentially over time steps, particularly when activation functions, such as sigmoid or hyperbolic tangent, are applied. This phenomenon hinders the network’s ability to retain information from earlier time steps, thereby limiting its capacity to model long-term temporal dependencies. This issue is especially problematic in movement direction classification, where the task requires integrating positional changes across multiple sequential frames. If the initial motion information is not preserved, the network may fail to capture the continuous trajectory of movement, such as a gradual transition from one direction to another. Consequently, the model may exhibit reduced sensitivity to direction changes that occur progressively over time, leading to inaccurate classification in real-world scenarios.

To overcome this limitation, the LSTM model was proposed [31]. LSTM enhances the traditional RNN structure by introducing a cell state mechanism that allows important information to be preserved over long periods, while irrelevant data are selectively discarded. This is achieved through three key gating mechanisms represented by the following equations:

$$f_t=\sigma \left(W_f\left[x_t,h_{t-1}\right]+b_f\right),$$

(1)

$$i_t=\sigma \left(W_i\left[x_t,h_{t-1}\right]+b_i\right),$$

(2)

$$o_t=\sigma \left(W_o\left[x_t,h_{t-1}\right]+b_o\right).$$

(3)

Here, $\sigma$ denotes the sigmoid activation function, and $W$ and $b$ represent the weights and biases, respectively. Equation (1) defines the forget gate, which determines the parts of the previous cell state that should be discarded. Values closer to 1 retain the information, while values near 0 remove it. Equation (2) is the input gate, which selectively updates the cell state with new information. Equation (3) represents the output gate, which decides how much of the cell state should influence the current hidden state. The cell state $C_t$ and hidden state $h_t$ are updated as follows:

$$C_t=f_t\cdot C_{t-1}+i_t\cdot {\tilde{C}}_t,$$

(4)

$${\tilde{C}}_t=\tanh \left(W_C\left[x_t,h_{t-1}\right]+b_C\right),$$

(5)

$$h_t=o_t\cdot \tanh \left(C_t\right).$$

(6)

In Equation (5), ${\tilde{C}}_t$ represents the candidate cell state, which is generated by combining the current input and the previous hidden state through the hyperbolic tangent activation. Equation (4) calculates the updated cell state by combining the retained information from the past and the newly selected information. Equation (6) computes the hidden state by applying the output gate to the updated cell state. Through this gating mechanism, LSTM networks can effectively retain and learn relevant features from time-series data while also mitigating the vanishing gradient issue, enabling stable training over long sequences.

In this study, we process time-series data collected from the low-resolution ToF sensor using the LSTM model. The ToF sensor operates at 15 frames per second, capturing sequential movement patterns during human motion. Unlike rule-based approaches or random forest regression models, which rely on predefined heuristics and struggle to adapt to dynamic environments, LSTM networks are capable of learning continuous motion patterns and maintaining long-term temporal information via cell states. This allows the model to generalize well even in multi-user settings and under varying environmental conditions. By utilizing the input and forget gates, LSTM effectively filters out irrelevant information while retaining essential details, such as entry direction and movement patterns. Based on these advantages, the proposed approach enables accurate detection of human presence and movement direction in complex indoor environments using low-resolution ToF sensor data.

3. Materials and Methods

3.1. ToF Sensor

The ToF sensor is operated by measuring the round-trip time of flight to calculate the distance to an object. Compared to high-resolution camera sensors, ToF sensors offer advantages such as lower cost, enhanced privacy protection and robustness to environmental changes.

In this study, we utilize the VL53L8CX ToF sensor (STMicroelectronics, Geneva, Switzerland), shown in Figure 2a,b, to detect human movement in indoor environments. The VL53L8CX is a cost-effective ToF module based on vertical-cavity surface-emitting laser technology. It incorporates an 8 × 8 array of single-photon avalanche diode receivers and can detect objects up to 4 m away. As illustrated in Figure 2c, the sensor features a maximum diagonal FoV of 65°, with each individual detection zone covering approximately 5.625 × 5.625°, forming a grid for distance measurements, as shown in Figure 2d. Compared to conventional high-resolution camera-based systems, ToF sensors ensure better privacy and reduced system costs. Specifically, the VL53L8CX sensor used in this study, with its 8 × 8 low-resolution array structure, enables the differentiation of movement directions while minimizing data processing overhead relative to high-resolution alternatives.

3.2. Data Preprocessing

The ToF sensor measures the distance to objects and provides output in the form of the 8 × 8 array, denoted as $D_t$, as shown in Figure 3. Since the sensor is mounted on the ceiling and performs downward measurements, it does not directly measure the height of the person from the floor (denoted as $H_t$ in Figure 3). Therefore, to effectively identify human presence and distinguish moving objects from static background noise, such as furniture near the floor, the raw data must be converted into relative height values with respect to the floor.

To achieve this, the system first measures the average distance to the floor during an initialization phase. This reference distance, $d_{floor}$, is calculated using the central 4 × 4 region of the $D_t$ array to efficiently calculate the vertical distance between the sensor and the floor. Subsequently, each measured frame is converted into a relative height map $H_t$ by subtracting the current $D_t$ array from the floor reference:

$$H_t=d_{floor}-D_t.$$

(7)

This transformation yields relative height data $H_t$, effectively removing the floor from consideration and allowing the system to more clearly capture human movement patterns within the sensor’s field of view.

The data acquisition process is illustrated in Figure 4. An embedded board equipped with the ToF sensor transmits distance measurements from each receiver module to the PC via I2C communication. Along with the distance values, the data stream may include “X” entries, which indicate diodes that failed to receive the reflected signals from an object. The transmitted data are used as input for training the neural network model.

However, temporary delays or transmission errors in the I2C communication process may introduce noise into the measured data. For example, some parts of the raw distance data may be lost, or the next frame’s data may be sent prematurely. In some cases, characters intended to follow an “X” value may incorrectly merge with the “X” entry, resulting in corrupted data formats like “X1000”. Such noisy data can degrade the performance of the neural network, both during training and in real-time inference.

To ensure data quality, the noise detection and removal process is implemented, as shown in Figure 5. The sensor board includes the header string “START” at the beginning of each data packet to indicate the start of a new frame. The PC uses this marker to verify whether the complete set of 64 distance values corresponding to the 8 × 8 array is present. Any value marked as “X” (indicating missing reflections) is replaced with zero. If the frame contains invalid or incomplete data like “X1000”, it is discarded to maintain the integrity of the dataset. This process helps minimize noise and ensures that the neural network is trained with reliable and high-quality data.

3.3. Dataset Collection

For data collection, the VL53L8CX ToF sensor is mounted on the ceiling at a height of 2.45 m, as shown in Figure 6. The sensor operates at a sampling rate of 15 frames per second (fps) and is configured to record continuous 8 × 8 array distance data as the person passes through its detection area within the FoV. Each data measurement takes about 3–4 s, and the size of the sequence data follows a distribution of about 90–120 frames, with an average size of 105 frames. The experimental environment is designed to allow individuals to naturally walk across the sensor’s FoV, and data are collected for both leftward and rightward movement directions within the detection zone. To maintain consistency regardless of sensor orientation, movements from left to right are labeled as “rightward” and those from right to left as “leftward”. These direction-based labels are used to train the LSTM model for classifying movement direction.

To ensure diversity in the dataset, the participants included four females with heights ranging from 155 cm to 162 cm and four males with heights ranging from 170 cm to 182 cm. Each participant performed various movement patterns within the experimental space, and the collected data were used as input for training the neural network model. To reflect diverse movement scenarios, each participant was given autonomy over their walking behavior, including variations in walking speed, irregular speed patterns and upper limb movements during walking. The dataset constructed for this study consisted of two main scenario types: Basic Scenarios and Challenge Scenarios. Figure 7a illustrates representative examples of these basic movement patterns. The Basic Scenarios include standard movement patterns where a single person walks through the detection area. These are further categorized into three labels: “rightward”, “leftward” and “turn”, the latter referring to a directional change during movement.

Additionally, to account for various real-world variables in indoor environments, scenarios where users carry an object (e.g., a box) while walking are also included in the Basic Scenario dataset. As shown in Figure 7 and Figure 8, carrying an object alters the typical shape of the ToF data. In normal walking, reflections are typically received from the head, shoulders and torso. However, when a user carries an object, some of the reflected signals are added or altered by the object, resulting in localized dips and rises in the height map. In rule-based systems, such anomalies may cause the data to be misclassified—either as non-human or as two separate entities. To enable the neural network to learn such variations, these object-carrying movement patterns are also included in the training dataset.

The Challenge Scenarios involve complex situations where two individuals simultaneously walk through the sensor’s detection area. As illustrated in Figure 9, conventional rule-based algorithms using clustering techniques struggle to resolve such multi-object conditions. While sequential entries can still be separated and learned individually by neural networks, simultaneous entries present additional challenges that require further training. To improve the robustness of the model in these complex conditions, we introduce additional training data for the Challenge Scenarios, with two labeling categories—“rightward pair” and “leftward pair”—representing simultaneous two-person movement in each direction.

In total, the dataset comprises 450 samples from Basic Scenarios and 200 samples from Challenge Scenarios. These data are used for training the proposed LSTM-based neural network model.

3.4. Neural Network Training Parameters

To analyze the preprocessed time-series data $H_t$ obtained from the ToF sensor and infer human movement, we designed the LSTM-based neural network model using the TensorFlow Keras library.

$$H_t=\left[h_{t,1},h_{t,2},h_{t,3},\cdot \cdot \cdot ,h_{t,64}\right].$$

(8)

Since the total number of time steps $T_{total}$ in the time-series data collected from the ToF sensor may vary across samples, it is necessary to unify the sequence lengths to construct a single input tensor for the LSTM model. This is accomplished by padding shorter sequences with zero values at the end. However, since these padded zeros are not part of the actual data, they must be excluded during the training process. Additionally, as the ToF sensor captures frames even before and after the person enters or leaves the FoV, it is crucial to filter out irrelevant frames that do not contain meaningful movement.

Although a dedicated detection module could be developed to identify when the person enters or exits the sensor’s range, this study aims to shift the entire perception and inference process to the neural network, avoiding rule-based modules. To achieve this, we implement a custom masking layer that enables the model to ignore time steps filled with zeros, whether due to padding or non-movement periods before or after the person appears. As a result, the LSTM network processes only meaningful time steps, effectively skipping those where all feature values are zero. The masked time-series data, excluding irrelevant time steps, are expressed as follows:

$$\alpha =\min \left\{t\left|\sum _{d=1}^{64}{h}_{t,d}\ne 0\right.\right\},$$

(9)

$$X_t=H_{\alpha +t-1}\forall t\in \left[1,T\right].$$

(10)

Here, $\alpha$ represents the time index at which meaningful movement information first appears in the sequence. The effective input to the network, $X_t$, excludes all masked time steps and contains only the meaningful portions of the original data. Since each sample may have a different number of meaningful time steps, the time length $T$ used in training also varies across the dataset. This masking technique enhances model performance by ensuring the network focuses on actual motion-related data and avoids learning from irrelevant information, thereby reducing potential training errors.

The LSTM-based neural network architecture used in this study is illustrated in Figure 10. It consists of two stacked LSTM layers to capture deeper temporal patterns. The first LSTM layer has 64 output units and produces output at all time steps, allowing the second LSTM layer to learn comprehensive sequential features. The second LSTM layer has 32 output units and compresses the sequence into a single vector using the masked final time step. A fully connected layer with 16 units and the rectified linear unit (ReLU) activation function is then applied to introduce non-linearity. Finally, a SoftMax-activated dense layer with five output units is used to classify the input sequence into one of five movement patterns.

The model is trained using the Adam optimizer to optimize weights, and the categorical cross-entropy loss function, commonly used in multi-class classification problems, is employed:

$$L\left(y,\widehat{y}\right)=-\sum _{i=1}^N{y}_i\log \left({\widehat{y}}_i\right).$$

(11)

Here, $N$ is the number of classes; $y_i$ is the one-hot encoded true label; and ${\widehat{y}}_i$ is the predicted probability for each class. The model is trained with a batch size of 16 and for 20 epochs, using an 80:20 split between the training and validation datasets.

4. Results

To evaluate the performance of the proposed LSTM-based model for detecting human entry and exit, we analyzed both training and testing outcomes. The training dataset consisted of 330 Basic Scenario samples and 140 Challenge Scenario samples. In addition, we used 180 test samples to evaluate the generalization ability of the model. As shown in

Table 1. Dataset label distribution.

Label	Train	Validation	Test
Rightward	100	10	40
Leftward	100	10	40
Turn	100	10	40
Rightward pair	65	5	30
Leftward pair	65	5	30

, the training samples were categorized into datasets for training and datasets for validation.

4.1. Neural Network Training

The training accuracy and loss over epochs are shown in Figure 11. The model’s training accuracy converged to 1.0 after nearly the fifth epoch, while the loss steadily decreased, indicating that the model was trained stably and efficiently. This rapid convergence can be attributed to the characteristics of the low-resolution ToF sensor data, which enables the model to effectively learn patterns even with a relatively small number of epochs.

The proposed 8 × 8 array-based low-resolution ToF sensor is designed to convey motion-related information rather than detailed physical characteristics, such as height or body shape. In contrast, models based on high-resolution cameras or ToF sensors often exhibit variations in performance depending on the physical characteristics of individuals. However, the low-resolution sensor used in this study shows less sensitivity to such variations, allowing the model to maintain consistent learning performance across a variety of user conditions. This robustness highlights the model’s efficiency and suitability for deployment in diverse environments.

4.2. Test Results

To evaluate the test accuracy of the proposed LSTM-based neural network model, the test dataset consists of 40 samples for each label in the baseline scenario and 30 samples for each label in the challenge scenario. The model is assessed based on its ability to correctly predict the labels of five movement categories: rightward, leftward, turn, rightward pair and leftward pair. A prediction is considered successful if the model’s output matches the ground truth label. This evaluation approach aligns with the characteristics of LSTM-based neural networks, which are designed to learn temporal dependencies across entire sequences. Rather than focusing on frame-level decisions, the model captures motion patterns as they unfold over time, allowing it to infer movement direction from the sequential flow of sensor data. This perspective emphasizes behavior recognition at the sequence level, which naturally reflects the operational principle of time-series learning models.

Table 2. Test results using the trained model.

Trial	Leftward	Rightward	Turn	Rightward Pair	Leftward Pair	Success Rate
1	40/40	40/40	39/40	30/30	30/30	99.44%
2	39/40	39/40	40/40	30/30	30/30	98.89%
3	39/40	40/40	36/40	29/30	30/30	96.67%
4	40/40	40/40	38/40	28/30	30/30	97.78%
5	39/40	39/40	38/40	30/30	30/30	97.78%
Total	197/200	198/200	191/200	147/150	150/150	98.11%

presents the evaluation results using the test dataset. For each label, we recorded the number of successes on the test dataset and calculated the success rate for each trial and the average success rate for all trials. The model achieved an average accuracy of over 98% across five test runs. Notably, the model demonstrated excellent classification performance not only for single-user movements but also in more complex multi-user scenarios where two individuals moved simultaneously. These results validate the effectiveness of the proposed LSTM-based neural network in learning human movement patterns using low-resolution ToF sensor data. While traditional rule-based algorithms often suffer from reduced accuracy in multi-user environments due to the limitations of clustering techniques, our model successfully leverages temporal sequence learning to maintain robust and generalized performance even under variable and dynamic conditions.

5. Discussion

This study proposed a method for estimating human movement trajectories in indoor environments by combining the low-resolution ToF sensor with the LSTM-based neural network model. Traditional approaches using camera-based sensors often face limitations in indoor applications due to privacy concerns, while the use of high-resolution ToF sensors introduces significant cost-related challenges. To address these issues, our approach leverages an 8 × 8 low-resolution ToF sensor to collect data and employs an LSTM model to learn movement patterns, thereby providing a high-performance yet cost-effective solution.

The experimental results demonstrated that the proposed model achieved 98% accuracy on the test dataset. By using the neural network, the model effectively replaces traditional, complex rule-based algorithms and successfully learns and predicts movement trajectories. Moreover, the model was shown to maintain robust performance even in multi-user environments, highlighting its practical applicability in real-world scenarios. These findings suggest that the proposed system holds significant potential for a variety of applications, including smart building automation, indoor occupancy monitoring in public facilities and people-counting systems in privacy-sensitive areas. In particular, it offers a promising alternative to camera-based systems in environments such as hospitals, restrooms and meeting rooms.

Future work will investigate the model’s generalization capability in more diverse indoor environments, including spatial structures commonly found in offices, hallways and public facility entrances. Additional experiments will be conducted under conditions with partial occlusions or moving obstacles to assess robustness in more realistic scenarios. Furthermore, the system’s applicability will be examined when the sensor is installed above actual entryways, enabling a more direct evaluation of performance in real-world deployment settings. These efforts will aim to enhance the domain adaptability and practical usability of the proposed method.