Video Based Accurate Step Counting for Treadmills ^†

Abstract

The number of steps can be used to measure the distance, intensity, frequency, and oxygen consumption of the human activities indirectly, it has great significance to evaluate the amount of human movement. Different from traditional methods which based on wearable sensors, video based methods have the advantages of non-contact. In this paper, we propose a novel infrared video based accurate step counting algorithm for treadmills. Our approach is mainly divided into two parts, e.g., shoulder tracking and step counting. Shoulder movement is less disturbed and can accurately reflect the gait information than other body parts in the processing of running. By tracking the position of the shoulder in the image sequence, and analyzing the peaks and troughs of the trajectory, we can calculate the number of steps accurately in real time. The experiments are conducted on our own datasets, and the result achieved clearly demonstrate the effectiveness of the proposed method.

1. Introduction

Lack of activity is the cause of many diseases, it is an important factor to non- communicable diseases in countries of high-income, and increasingly in both low and middle income countries. Epidemiological research has demonstrated the effect of lacking physical activities and risk for several chronic diseases, including stroke, coronary heart disease, hypertension, osteoporosis, colon cancer, etc. Measuring physical activities and counting steps is an effective method to diagnose and treat some diseases, and it can also be an effective way to encourage people to increase their physical activity.

The objective quantification of physical activity is a challenge to those involve in research and practice. Traditionally, the quantification of physical activities has been assessed by means of questionnaires, however, the accuracy depends on the memory ability of the subjects. The invention of pedometers started the use of objective monitors to record physical activity. Pedometers are typically worn on the belt or waistband and respond to accelerations in some particular directions. With the rapid development and popularity of wearable devices, it is convenient to use wearable devices for steps counting. Lu et al. [1] made the sensor socks, which is unobtrusive and convenient, for measuring physical activities and counting steps. Their approach shows a high accuracy of the classification of physical activities and counting steps in a home or community environment. Wang et al. [2] designed a series of experiments testing step counting performance under different conditions to evaluate seven popular wristband activity monitors with nine subjects. The experiments involved various walking conditions which frequently happen in daily environment, including walking at different speed, walking with or without arm swing, walking along a winding path, walking on a treadmill, walking up stairs and down stairs. Mean absolute percentage error is adopted to analyze and compare the performance among the monitors, and the final experimental results demonstrate that the accuracy among different brands of monitors vary greatly.

Different from traditional sensor based methods, video based steps counting approaches have the advantage of non-contact. In this paper, we propose an infrared video based accurate step counting algorithm for treadmills. The remainder of the paper is organized as follows. Section 2 introduces the existing approaches to steps counting. Section 3 describes the proposed approaches, involving shoulder tracking and steps counting. The experimental results achieved are shown and discussed in Section 4. Section 5 concludes the paper with perspectives.

2. Related Work

In recent years, it has been intensively investigated how to accurately estimate daily step count. Existing approaches in step counting can be roughly categorized into sensor-based and video-based ones.

The former make use of a set of sensors placing at different parts, the data acquired from integrated inertial measurement unit (IMU) can provide step evaluation. Daniele et al. [3] measured steps by means of a foot pressure sensor, and applied it to remote therapy. Kinh et al. [4] proposed a new method for detect user’s step while walking via iPhone’s accelerometer. At first, discrete Kalman filter is explored to reduce noise and flatten the acceleration signals. Then a new filter method is used to recognize and mark up steps for signals to count steps. Tang et al. [5] introduced self-adaptive stepping to improve the accuracy of steeping under unrestricted walking and running models on smartphones. They proposed self-adaptive versions of two existing step counting methods, as well as a new stepping cycle recognition (SCR) method that performs well in their evaluations. Gu et al. [6] presented an accurate and robust step counting algorithm which is based on peak detection and analysis of step features, including periodicity, similarity, and continuity. The proposed method focuses on solving the over-counting problem caused by false walking, and the experimental results show that the proposed algorithm outperforms the commonly-used peak detection-based method and can improve the accuracy greatly. Pan et al. [7] proposed a novel step approach for smartphones users. The proposed algorithm is composed of two phases. The first phase collects linear acceleration and gravity values from the phone’s accelerometer. The second phase adopts the concept of correlation coefficients to identify whether the collected sensing measurements exhibit similar tendencies and calculates step counts. Video-based approaches have the advantage of non-contact, and especially suitable for treadmills. By means of video data, the moving parts of human body are tracked and analyzed. With the rapid development of computer vision technology, video based methods are expected to achieve high accuracy step counting in the real time.

3. Shoulder Tracking and Step Counting

The shoulder movement reflect the gait information more robust than other parts on the treadmill. Tracking the shoulder movement and analyzing its trajectory peaks and troughs provide a wealth of motion information for step counting. At present, there are many algorithm can be used to track human motion accurately in real time. In order to accurate and robust scale estimation for real-time visual tracking, Martin et al. [8] proposed a novel scale adaptive tracking approach by learning separate discriminative correlation filters for translation and scale estimation, which can accurately tracking the position of shoulder in the processing of running on the treadmill. In this paper, we adopt DSST algorithm for robust shoulder tracking.

By selecting the shoulder position manually (or detect the shoulder automatically) in the first frame and tracking it by DSST [8] in the following image sequence. The change of horizontal displacement of the body reflects the gait information, as shown in Figure 1. The horizontal indicates the video sequence and the vertical axis denotes the body displacement.

Figure 2 shows the trajectory of body horizontal position in the processing of running on the treadmill, and each peak represents a gait cycle. By calculating the number of peaks in the trajectory, the step counts on the treadmill can be obtained. It can be implemented by the ‘findpeaks’ command in Matlab.

4. Experimental Results

To validate the proposed method of steps counting, we conducted extensive experiments on our own infrared video database. The results are shown in the following sections. For a given image sequence, shoulder position is initialized manually for tracking, the centers of horizontal position of the body during the run are recorded to form the trajectory of the movement.

4.1. Database

The CISS-Infrared video database, is under construction, contains 20 people and the scale will increase rapidly in the next few years. In the construction process, it takes full account of the influence of age, gender, speed and time etc. The infrared camera is placed in the front of the treadmill to capture the upper motion data during running. All individuals are asked to running on the treadmill at different speeds, the camera capture the data at a rate of 25 frames per second and the whole process lasts about 20 min. Some examples in our infrared video datasets are shown in Figure 3.

4.2. Discussion

The proposed approach mainly includes shoulder tracking and steps counting. These techniques are mainly implemented in the Matlab platform, and tested on a PC with 8 cores of i7-6770 and 64 GB RAM.

Table 1. Computational cost of each step in the proposed approach.

Method	Consumed Time (ms)
Shoulder Tracking	2.5
Steps Counting	0.8

below illustrates the computational cost of the major step in the proposed method. From Table 1 we can see that the proposed algorithm can obtain the steps count in real time.

Table 2. The accuracy of our result compare to different bands of sport ring.

Method	Accuracy(%)
Fitbit	91.1 ± 3.7
Mambo	94.6 ± 2.5
Amazfit	93.3 ± 5.9
Xiaomi	97.6 ± 3.2
Honor	96.2 ± 4.8
HuaWei	99.3 ± 5.4
Ours Algorithm	99.5 ± 0.4

presents the results of the proposed method, we can see from the table that our approach can accurately obtain the number of steps in the process of running. Given a full image sequence, we tested our approach and achieved a state-of-art accuracy, 99.5%, and the variance is 0.4%. We compare the accuracy of the proposed algorithm with different brands of sports ring, e.g., Fitbit, Mambo, Amazfit, Xiaomi, Honor, Huawei, and the results are shown in Table 2.

From Table 2 we can see that our proposed algorithm is more accuracy and robust to other sports ring based step counting methods. Our final result are 99.5%, is 8 percentage points higher than the famous Fitbit sports ring, and the variant 0.4 is more robust than 3.7. The experiments are carried on the professional treadmill with the speed of 5 km/h, the final results demonstrate that our algorithm can be used to record the true number steps of the runners.

5. Conclusions

This paper proposes a novel approach to steps counting on the treadmill. Our approach is mainly divided into two parts, e.g., shoulder tracking and step counting. Shoulder movement is less disturbed and can accurately reflect the gait information than other body parts in the processing of running. By tracking the position of the shoulder in the image sequence, and analyzing the peaks and troughs of the trajectory, we can calculate the number of steps accurately in real time. The experiments are conducted on our own datasets, and the result achieved clearly demonstrate the effectiveness of the proposed method.