Physical activity can improve health and well-being, reduce the risk of many diseases, and improve the quality of life [1
]. However, a large number of people suffer injuries during exercise [2
]. A major contributing factor of exercise injuries is fatigue [2
]. Fatigue caused by repeated movement accumulates over time and may exceed the muscle tissues’ tolerance, contributing to musculoskeletal disorders (MSDs) [2
]. Thus, monitoring and predicting fatigue are important to reduce the risk of injuries. In the context of sports training, fatigue estimation can be used by coaches and physical therapists to avoid high levels of fatigue, which may adversely impact training and hinder performance in competition. In the context of rehabilitation, many patients are instructed to perform rehabilitation exercises at home by themselves. Without the therapist’s instructions and feedback on their movements, there are greater risks of secondary injury. Therefore, actively monitoring the onset of fatigue could provide important feedback in sports training, competition, and rehabilitation [4
Human activity recognition (HAR) is a broad research field that involves the identification of various human activities or gestures and more detailed knowledge about human activities (e.g., the quality of motions, emotions, and gender) based on sensor data [5
]. Recently, many novel data processing and analysis methods have been applied to HAR due to the introduction of wearable, low cost, low power sensors and live streaming of data [5
]. Meanwhile, advances in computer vision, machine learning, and artificial intelligence have enabled HAR to be widely used in athletic competition, healthcare, and elderly care applications [5
]. Even though much research has been conducted on human movement analysis and action recognition [5
], the study of automatic fatigue prediction or estimation is somewhat limited.
Typically, biomechanical variables are measured using motion capture for the kinematics of body segments, electromyography for muscle activity, and plantar pressure measurements for detecting stepping [11
]. Common techniques for detecting fatigue are to measure muscle activity, e.g., surface electromyogram (sEMG) [12
], and the kinematics of joint angles, e.g., optical motion capture [1
]. However, sEMG has limitations, e.g., the sensors may lose contact over time, particularly during vigorous exercise, and can only measure the activity of the particular muscle to which the sensor is attached. For optical motion capture, limited capture area and occlusion are key issues because reflective markers can be hidden from the camera and additional performers will increase occlusion in team sports. Even though there are some technologies that use structural lighting [13
] and multi-camera motion capture systems [14
] that can exclude blind spots, they are difficult to apply to outdoor activities. The multi-camera system usually takes longer to set up due to the large amount of equipment and is less flexible. On the contrary, wearable IMUs are small, lightweight, and robust to occlusions and interference. They do not restrict body movements and allow a participant to perform various tasks in arbitrary environments. On the other hand, force plates (FPs) are easy to use and do not require any equipment on the body, therefore saving much time in the setup of experiments. They also help to record the motion abnormalities of lower body segments during specific activities [16
]. Foot plantar sensors are also seldom discussed in the application of fatigue estimation. Therefore, force plates and IMUs are selected to measure exercise-induced fatigue, and their performances are compared.
A number of studies [17
] have been conducted to detect binary fatigue (fatigue vs. non-fatigue) by using various invasive or noninvasive devices including IMUs and sEMG. However, these models can only represent a simplistic process of fatigue development, and an essential early intervention is impossible before the athlete is deemed to be fatigued. Limited research [1
] has been conducted to monitor gradual and continuous changes in fatigue levels. The change from low levels of fatigue to high levels of fatigue is a continuous process, which may take place gradually, and the variations of human motion due to fatigue are fairly small during exercise, making it difficult to detect continuous changes in fatigue [17
]. Moreover, the associated literature [17
] only focuses on one task or exercise type (running only or jumping only) and lacks the ability to generalize the fatigue models to different exercises.
The onset of muscle fatigue is complex and may depend on personal fitness level, health conditions, types of exercise, and gender [8
]. The observed changes in movement or muscle electrical activity may not be consistent among all individuals. In other words, the expression of fatigue is more likely to be person-dependent.
In this work, a data-driven approach is investigated to estimate the onset of fatigue using force plate or IMU measurements in three different exercises (squat, high knee jack, and corkscrew toe-touch). The proposed framework is the first of its kind to model the continuous increase in fatigue based on each single repetition (rep) of exercise and the first to be evaluated on multiple exercises, showing its potential generalizability. Unlike most previous studies, which focused on extreme fatigue and non-fatigue detection, here, the approach is to monitor the fatigue accumulated status with continuous feedback. In the field of human motion recognition, the deep learning based framework makes a great contribution to understanding the quality of exercise for rehabilitation in a continuous process, automatically, without specific domain knowledge.
2. Related Work
In this section, learning approaches for fatigue estimation are discussed.
Various research has been conducted to evaluate fatigue through data from different sensors and different methodologies, and the choice of sensors depends on the purpose or objectives of each study. Most studies (e.g., [12
]) were conducted using surface electromyogram sensors (sEMG) to determine the physiological status of a muscle due to the activity. For example, Chattopadhyay et al. proposed an exhaustive set of features from the sEMG signals and analyzed the variability between subjects and between trials [25
]. Dong et al. proposed a method to evaluate the overall fatigue of human body movement based on combined sEMG and accelerometer signals and introduced a “forgetting factor” and fatigue level fusion coefficient to combine different localized muscle fatigue estimates with the overall fatigue level [4
]. However, sEMG may lose contact over time, particularly for dynamic exercise, and can only measure the particular muscle to which a sensor is attached. It cannot be used in the real world.
Inertial measurement units (IMUs) have also been used to analyze fatigue based on kinematic movement, especially in gait [18
], running [19
], and sprinting [29
]. Those results indicated the capability of IMUs to provide reliable and accurate measurements of temporal parameters during exercise. The most relevant studies to our research are [19
]. Buckley et al. predicted subject-dependent and subject-independent fatigue levels (non-fatigued and fatigue status) through data from a single IMU [19
]. In the experiment, running four-hundred meters at a natural pace was considered as a non-fatigue status, and a subsequent beep test (also known as the “multi-stage fitness test”) was used to induce fatigue. The results showed that a single IMU on the right shank had better performance than on the lumbar spine when assessing subject-independent fatigue estimation, and the subject-dependent classifier had higher accuracy than the subject-independent classifier. Stohrmann et al. [20
] monitored human fatigue by extracting kinematic parameters from wearable sensor data and investigated the kinematic changes evoked by fatigue during running. Twenty-one runners of different skill levels performed experiments on a treadmill and conventional outdoor track. Their findings showed that kinematic changes were related to fatigue for all runners, and fatigue was dependent on participants’ running technique [20
]. To date, there are only a few studies focusing on continuous movement changes induced by fatigue, e.g., [1
]. Ramos et al. presented a machine learning system to evaluate fatigue using electromyographic (EMG) and heart rate variability (HRV) measurements [21
]. The approach showed a potential to implement a combination of a dimensionless (0–1) global fatigue descriptor to reflect the onset of fatigue.
Additionally, in the past research, a force plate was generally used to measure the postural stability performance [30
], and some studies assessed counter-movement jump performance through the investigation of the ground reaction force-time profile [32
]. The study [34
] showed that fatigue due to the exercise of the calf-muscles of one leg could influence the body balance in the short term, and this can be measured by a force plate and an accelerator, which indicates the potential to distinguish human motion status through these two tools. In view of automatic recognition technology, different classifiers have been applied, such as the fuzzy logic (FL) classifier, the random forest classifier [19
], the support vector machine (SVM) classifier [12
], and the hidden Markov model (HMM) classifier [35
]. However, these currently applied algorithms can only classify fatigue and non-fatigued status, and it it challenging to monitor a gradual and continuous changing of fatigue level. The methods also vary according to various exercises and muscle types. Different exercises may cause fatigue in different human body parts, and the measures of fatigue highly rely on the types of sensor and sensor positions. Thus, this paper aims to develop related algorithms to detect continuous fatigue changes and generalize fatigue models for various exercises.
The aim of the present work was to evaluate the methodologies for monitoring continuous fatigue levels automatically during exercise. The performances of the random forest and CNN classifiers using force plate and IMU data were evaluated in this study. By comparing the correlation coefficient in the RF model, the displacements of COP were found to be most highly associated with fatigue during exercise than other commonly used features of COP including standard variance, variance, skew, kurtosis, area of ellipse, and the first derivative and second derivative of COP positions.
By applying the selected feature set of the force plate and IMU in a participant-specific prediction model for continuous fatigue, the results showed that motion data from the FP had similar performance for fatigue prediction as the IMU in terms of the average Pearson correlation coefficient. Up to an 89%, 93%, and 94% Pearson coefficient for the squat, jack, and corkscrew exercises, respectively, could be achieved for a subject-dependent regression model for continuous fatigue prediction with motion data from the force plate. The proposed model translated the participants’ perceived exertion into numerical scores ranging from zero to 10, contributing to quantifying the movement performance via supervised learning. In addition, for most participants, the CNN performed better than the RF regression model. As expected, since the CNN is capable of extracting features automatically and no specific domain knowledge is required, the results were more accurate than those of random forest. Participants 2, 5, and 13 were found to consistently achieve strong correlation between the estimated fatigue level and their self-reported fatigue level in the squat exercise. The same conclusion can be drawn for Participants 5, 11, 13, and 14 in the high knee jack exercise and for Participants 2, 5, 11, and 13 in the corkscrew exercise. Considering the data and demographics (Table 3
), the prediction performance was highly related to the number of sets that participants performed. The larger the number of repeated exercises per subject, the higher the performance of the prediction was, while the participants who performed a few reps of exercises had weaker correlations in the predicted model. This can be explained as large training data are expected to improve the model performance of the prediction. A further analysis with the same amount data for each participant will be conducted to examine the predictions in future work. Furthermore, when the experiments last for a long time, participants may lose interest and feel bored rather than really feel tired. In order to complete the experiments quicker, they may exaggerate fatigue levels, leading to unreliable results and low accuracy [50
]. Therefore, it may be necessary to consider this respondent fatigue during complex and long surveys or experiments.
Unlike most previous work, which has been limited to studies on fatigue during one type of exercise [1
], the present study was conducted on three different exercises. The test Pearson correlation coefficients in the RF and CNN models with FP data or IMU data demonstrated a good generalization ability of the proposed methods for different exercises. Since fatigue takes place gradually in a continuous fashion, the present work addressed the limitations associated with assuming only a two-class fatigue level prediction [17
], thereby capturing the buildup of fatigue during the progress of exercise, which is important for preventing over-training at the early stage of fatigue, contributing to reducing the risk of injuries. Our model not only can predict the onset of fatigue, but also provide insight into the gradual change of exercise performance due to fatigue.
The most related to our study is the study [1
], which conducted experiments to quantify the person-dependent continuous increase of fatigue over time, based on a squat dataset recorded with optical motion capture. It predicted fatigue for sets of squats assuming the foregoing squats were performed with the same fatigue level as the last squat within a set, which fit well with the subjective fatigue ratings since the ratings were given after sets of squats. In contrast, here, predicted fatigue was for every single repetition of squats without a prior assumption. In terms of accuracy, the predicted fatigue level in our study was also close to the real level based on the B&A analysis. The deviation of the regression line of the difference was small, which shows that the model can achieve good prediction whatever the level. In addition, optical motion capture was used in [1
], which would pose challenges in real-world applications, since the capture space is limited and it is difficult to apply to outdoor activities. The proposed approach requires manual segmentation of the motion repetitions. For online deployment, automated segmentation needs to be implemented [51
]. Alternatively, the approach could be modified to consider fixed-size windows, so that segmentation is not required [53
]. Based on the successful findings of this study, there is potential to implement a real-time application for monitoring continuous fatigue-induced changes of motion based on data from wearable sensors or wireless insoles. In the future, the authors will explore the influence of sensor positions on movement detection across multiple exercises and a participant-independent approach for fatigue estimation by collecting a large-scale dataset through simulation techniques.