Detecting Fatigue during Exoskeleton-Assisted Trunk Flexion Tasks: A Machine Learning Approach

Abstract

Back-Support Industrial Exoskeletons (BSIEs) can be beneficial in reducing the risk of injury due to overexertion during trunk flexion tasks. Most real-world tasks include complex body movements, leading to mixed outcomes that necessitate field-based methods for detecting overall physical demands. Monitoring fatigue can be beneficial in this regard to ensure that benefits of BSIEs are translated to the real world. Our experiment included 14 participants, who performed 30 repetitions of 45° trunk-flexion while assisted by a BSIE, first without fatigue and then at medium-high back fatigue (7/10 in the Borg scale). We extracted 135 features from recorded muscle activity, trunk motion, and whole-body stability across bending, transition, and retraction portions of each trunk-flexion cycle. Four classification algorithms, namely Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), and XGBoost (XGB), were implemented to assess fatigue prediction. XGB (Accuracy: 86.1%, Recall: 86%, Specificity: 86.3%) was effective in classifying fatigue with data obtained from a single EMG sensor located on the lower back (erector spinae) muscle. Meanwhile, stability measures showed high predictability with both RF (92.9%, 91.9%, 94.1%) and XGB (93.5, 94.1%, 93.1%). Findings demonstrate the success of force plates, and when replaced by pressure insoles, they can facilitate real-world fatigue detection during BSIE-assisted trunk-flexion tasks.

1. Introduction

Wearable assistive devices such as exoskeletons have been developed to address discomfort, pain, and injuries resulting from repetitive manual tasks. The prevalence of musculoskeletal injuries from overexertion and bodily reactions has surged in recent years. For instance, in the U.S. alone, there were approximately one million reported cases of overexertion injuries in 2021–2022 [1]. Although traditional ergonomic controls such as workforce training, workstation redesign, safety protocols, and use of material handling equipment (cranes, lifts, etc.) may be beneficial in reducing injury rates, they are often expensive, require company-wide implementation, and offer less flexibility with variation in tasks [2,3]. In most cases, despite implementing traditional ergonomic controls, workers continue to face escalating demands driven by ever-increasing consumer needs [4]. In response to these challenges, wearable assistive devices, such as Exoskeletons (EXOs), have emerged as promising solutions to augment human capabilities and mitigate the risk of injury. EXOs are typically categorized according to the body region they support, i.e., the upper body (e.g., arm/shoulder/back) or lower body (knee/ankle) [5,6]. Of all body regions, the lumbar region of the lower back is particularly susceptible to injury, with the highest reported injury rate of approximately 17% [7]. Back-Support Industrial Exoskeletons (BSIEs) offer support to the wearer’s torso during tasks requiring trunk flexion, effectively reducing the strain on lower back muscles and potentially lowering the risk of injury in this vulnerable area [8,9,10]. Studies have demonstrated significant reductions in lower back muscle activity when using BSIEs during static posture maintenance tasks involving trunk bending [9,11,12] and dynamic lifting tasks [13,14,15,16,17,18]. However, evaluations conducted in field environments have yielded mixed results regarding the effects of BSIEs on the human body, as outlined in our previous research [6].

Fatigue, often stemming from repetitive muscle activation, renders workers more susceptible to injury, as it compromises the neuromusculoskeletal systems [19]. When the demands placed on the body exceed its capacity to generate essential forces, it can result in failure, diminished work quality, performance errors, and increased injury risk [20]. Consequently, estimating fatigue levels during task performance becomes crucial for designing safer and more efficient work environments. Traditionally, fatigue has been assessed subjectively using scales [21], and objectively by monitoring changes in muscle force generation capacity. It is defined as “the inability of muscles to sustain force generation over time” in the [22]. Engaging in activities while fatigued can lead to detrimental effects, such as impaired balance and reduced control over body movements, thereby escalating the risk of falls [23,24]. While Back-Support Industrial Exoskeletons (BSIEs) may offer benefits in reducing the rate of muscle fatigue, they might also increase users’ susceptibility to falls [25], particularly with additional weight (2.2–4.5 kg), and the assistive torque provided could impact the wearer’s stability [23]. These concerns are exacerbated during dynamic tasks with increased inertial forces and in awkward or asymmetric postures [18,26,27,28]. To ensure the safety and efficiency of EXOs, it is crucial to integrate advanced technologies through which EXOs can provide optimum assistance to wearers, improving their wearer’s experience and reducing the risk of injuries.

Machine learning algorithms, particularly classification methods, have emerged as powerful tools across domains of manufacturing [29], construction [30], and healthcare [31] to analyze complex data and for making predictions. In healthcare, these algorithms offer advantages by improving disease diagnosis, assessing risks, and tailoring treatment plans based on learned data patterns [32]. Specifically, mathematical models are used to identify pertinent features from provided datasets, which are then employed for prediction of future states, or classification of the existing data into different groups. In the context of predicting differences in physiological conditions, machine learning classification algorithms play a pivotal role [33,34]. These include (a) decision trees, which partition the feature space into distinct regions based on simple decision rules, are interpretable, and are suitable for tasks with categorical outcomes [35], (b) Support Vector Machines (SVM), which aim to find the hyperplane that best separates different classes while maximizing the margin between them, making them effective for binary classification tasks with high-dimensional feature spaces [36], and (c) ensemble methods such as Random Forests and Gradient Boosting, which combine multiple classifiers to improve predictive performance and robustness [37,38,39]. Previous studies have demonstrated the utility of machine learning techniques in detecting fatigue during tasks such as walking, thereby reducing the risk of injury due to overexertion. By analyzing various physiological signals, including body movement [40,41,42,43,44], electromyography (EMG) signals [45], stability [46], and heart rate [40,41], machine learning models can accurately classify individuals into fatigued and non-fatigued states. Moreover, incorporating wearable sensors and advanced signal processing techniques enables real-time monitoring of fatigue levels, allowing for timely interventions to prevent overexertion-related injuries. Overall, machine learning-based fatigue detection systems can be helpful in enhancing workplace safety.

Past studies on fatigue detection conducted controlled laboratory experiments comprising fatiguing tasks, with fatigue progression tracked using both subjective and objective measurements. Subjective scales, such as the Borg scale, have been widely used to obtain a measure of global fatigue, which has been utilized as a reference for objective data while training fatigue detection models [47,48]. Thus, using objective data obtained from sensors, such models can predict perceived exertion levels. Meanwhile, one of the most common methods for quantifying localized fatigue is by recording muscle activity using surface electromyography (EMG) [43,49]. In a recent study, Root-Mean-Square envelopes of raw EMG signals were used among the feature set to detect fatigue level during walking tasks [43]. Fatigue is also known to influence postural control and movement coordination, specifically leading to altered movement kinematics (e.g., range of motion, velocity, and jerk) [40,48,50]. These changes in kinematic parameters with fatigue progression have been tracked using motion capture systems, including wearable sensors and infrared camera systems [43,48,51,52,53]. For instance, changes in gait kinematics can serve as reliable indicators of fatigue during walking [42,54]. Likewise, features such as joint angles, velocity, and acceleration extracted from position data (of 39 markers) obtained from infrared cameras were utilized to predict fatigue during repetitive manual tasks [53]. Alterations in motor control and their resulting variations in body movement due to fatigue are also observed in whole-body stability measures. For instance, deviations in the center of pressure (COP) measured using force plates have been reported to be correlated with fatigue [51,55,56,57]. A combination of multiple sensor systems has also been utilized to improve prediction performance, such as obtaining features from both EMG and motion capture systems [43]. Overall, efforts demonstrate efficacy of biomechanical data acquisition systems in developing objective approaches of fatigue detection.

Monitoring overall demands using multiple sensors may be challenging given the current limitations of sensor technologies as well as their intrusive nature, and intricacies of evaluating EXOs in real-world industrial scenarios [6]. Meanwhile, subjective ratings of fatigue may be biased. Additionally, most industrial tasks typically incorporate designated rest breaks to facilitate recovery. Since EXOs are still in early implementation stages, much remains unknown about their effects, including changes in optimal work–rest ratios, which are valuable in designing task parameters. Given the limitations of subjective ratings and the challenges of monitoring fatigue using multiple sensors, there is a pressing need to develop objective approaches for detecting fatigue while using EXOs. Such objective approaches can provide more accurate and unbiased fatigue detection, ensuring the safety and well-being of workers. The novelty of our study lies in the fact that this is the first study to develop an EXO-specific fatigue detection model to facilitate an accessible objective assessment of the devices in field environments. This study aims to (a) develop a model to predict fatigued states using objective biomechanical data and (b) determine specific measures (such as muscle activity, motion, and balance) that contribute to the highest predictability. Prior to conducting the study, we hypothesized that muscle activity measures would be most effective in classifying states of fatigue. For developing the fatigue detection model, we conducted a controlled experiment wherein study participants performed repetitive trunk flexion tasks. A detailed account of our experimental design, physiological data collection and feature engineering has been presented in Section 2. Subsequently, model development using common classification algorithms and their performance has been provided (Section 3). Lastly, we relate findings from our study to real-world scenarios (Section 4) and propose a novel design for portable EXO-specific fatigue detection system. Overall efforts presented in this study may be beneficial in developing approaches for monitoring benefits/limitations of exoskeleton-assisted industrial activities.

2. Materials and Methods

2.1. Participants

Fourteen young male adults from college population were recruited per the inclusion criteria of: (a) anthropometric measurements (height: 5–6 ft., weight: 120–200 lbs.), (b) exercise frequency of twice/week, and (c) no incidents of back/lower body musculoskeletal disorder in <6 months.

Table 1. Demographics of participant pool showing the Mean, standard deviation (SD), and the ranges (expressed as Maximum–Minimum) for anthropometric dimensions of participant age, height, weight, body mass index, chest/hip circumference.

Demographic Information	Mean (SD)	Range of Values
Age (yrs.)	20.21 (2.6)	18–28
Height (cm)	179.1 (3.7)	172–186
Weight (kg)	72.9 (6.2)	65.68–82.44
Body mass Index (kg/m2)	22.7 (2.4)	19.42–27.56
Chest Circumference (cm)	89.5 (3.9)	84–96
Hip Circumference (cm)	86.6 (6.0)	80–98

shows body measurements of study participants, as recorded on the first experimental session. We obtained written informed consent from all participants prior to data collection, as approved by the Rochester Institute of Technology’s Institutional Review Board, Human Subjects Research Office (HSRO#01113021) in accordance with the tenets of the Declaration of Helsinki.

2.2. Approach

2.2.1. Experimental Tasks

A wire grasping task was selected by grasping two wire connectors placed ~10 inches apart with their fingers. This ensured consistent hand movement across experimental trials. The wiring setup included a portable adjustable stand in front of the participant such that the subjects could bend at a ~45° sagittal flexion angle. Tasks included performing trunk flexion tasks in two ways: (a) repetitive bending and (b) intermittent bending. Thirty cycles of repetitive bending were performed once without fatigue, and then again at medium-high level of fatigue. To simulate fatigue, we incorporated a task cycle with 30-s sustained trunk flexion that was performed intermittently in addition to standing still tasks (15 s) before and after the sustained portion. For intermittent task cycles, a work–rest ratio of 60:15 sec was selected. This task was selected to simulate realistic task cycles as in industrial work. Durations, work–rest ratios, and number of cycles for the tasks were set based on a pilot study. The same tasks were also performed in asymmetric postures but were not included as a part of dataset in this study.

2.2.2. Exoskeleton Device

Trunk flexion tasks were performed while wearing a BSIE. Among the several types, passive ones are affordable, compact, and are more likely to be adopted by the workforce. Thus, we used a passive rigid BSIE, specifically the BackX Model AC (SuitX, Emeryville, CA, USA), with actuators located on each side of the hip. The BSIE selected for our study, as shown in Figure 1, consisted of a chest pad to support the trunk when performing trunk flexion, specifically offloading upper-body weight to the front region of the thigh using mechanical linkages. Among the three provided support levels, we selected the device at medium (~25 lbs.), which was kept consistent across participants.

2.2.3. Data Acquisition

Measures of interest in this study included muscle demands in lower back and leg regions, trunk movement, and whole-body stability. Both lower back and leg regions were selected as participants in our pilot study reported prominent presence of fatigue in legs. Thus, the muscle groups of interest included the left/right erector spinae longissimus (LES/RES) and the left/right biceps femoris muscles (LBF/RBF). Both muscle groups are known to contract the most during trunk flexion as these muscle groups are responsible for pulling the weight of the upper torso and torso to ensure a stable posture. We implemented electromyography (EMG) to record muscle activity during the repetitive bending task using four sensors of the Trigno Wireless EMG system (Delsys, Natick, MA, USA, 1200 Hz). We followed the protocols provided by SENIAM [58] for placing the sensors on respective muscle groups using double-sided tapes. To segment the data into different portions as well as to detect trunk movement, an optoelectronic motion capture system (VICON, Hauppauge, NY, USA, 100 Hz) was used. This included 20 reflective markers placed on the upper body (three on the upper back, two on the middle back, and three on the hip) and lower body (three on each leg and three on each foot) of each participant (Figure 1). Marker placement locations were determined based on guidelines provided by VICON [59]. For determining impacts on balance, two force plates were used (AMTI, Advanced Mechanical Technology, Watertown, MA, USA), where participant stood upon floor-embedded force plates (AMTI OR6-6 platform, MA, USA, 1000 Hz). All three systems were time-synced using the VICON NEXUS system.

Fatigue level in back and legs was obtained subjectively and was used to drive the duration of tasks. Perceived ratings of exertion (RPE) were obtained using the Borg RPE CR-10 scale [60,61]. The scale consists of levels denoted by numbers ranging from 0–10, where levels 0, 1, 2, 3, 4, 5, 7, 10 represent ‘no exertion’, ‘very slight’, ‘slight’, ‘moderate’, ‘somewhat severe’, ‘severe’, ‘very severe’, and ‘maximal’ levels of fatigue, respectively.

2.3. Experimental Design and Procedure

Due to inclusion of fatigue, multiple sessions were scheduled on separate days per participant, to avoid potential carry-over effects. The overall study comprised three separate sessions (training/session 1, session 2, session 3) with a ~48 h gap between each session for muscle recovery. We trained and familiarized the participants with study protocol and the BSIE. Upon signing the consent form, participants first performed a wall-sit task for self-calibration of the Borg RPE CR-10 scale. This was followed by attachment of electrodes and sensors, and MVCs from all four muscles (LES, RES, LBF, and RBF) were recorded by manually restricting body movement while instructing participants to exert maximal effort in the opposite direction. The first session concluded with participants performing two repetitions of each experimental task with/without assistance, and then familiarizing themselves with the BSIE. Adjustments were made according to the manufacturer provided instruction manual. Proper fit was subjectively confirmed with the participants and the adjustments were recorded for subsequent sessions. Each of the two subsequent experimental sessions included performing bending tasks with/without the BSIE. While this study only considered dataset for symmetrical bending, asymmetric postures were also performed with a fifteen-minute break between the posture conditions.

Protocol for experimental tasks in each condition consisted of performing 30 cycles of repetitive bending at the start (RPE in back: 0 (no exertion)) and at the end (RPE in back: 7 (medium-high exertion)). Task cycles with 30-s sustained bending, and two 15-s standing still activities were performed with 15-s intermittent breaks until participants reached a medium-high fatigue level. We decided these durations based on our pilot study. RPE ratings in the back region were obtained from participants on the BORG CR-10 scale after each task cycle. After performing 30 repetitive bending cycles at the end, the experimental condition was concluded. The second session ended when two/four conditions were completed, and participants were recalled for performing the third session after a minimum period of 48 h to allow complete muscle recovery.

2.4. Data Pre-Processing and Feature Engineering

The Nexus 1.7.1 (VICON, Hauppauge, NY, USA) software was used to export data from each sensor/marker of the EMG, Force Plate, and motion capture systems in a single (.csv) file that represented 30 cycles of repetitive bending. We developed a custom MATLAB code to import data from the excel file. Data obtained from the force plate and the motion capture system will be filtered using a second-order lowpass digital Butterworth filter with a normalized cutoff frequency of 10 Hz. Meanwhile, the EMG was filtered using a Butterworth filter in the 30 Hz and 300 Hz band [62,63]. To evaluate variations across the bending/retraction cycles, we segmented the activity using the position in the z direction of the upper back marker by detecting bending start/end and retraction start/end portions. To ensure consistency, the middle 40% of the range of movement, scaled from 0–1, was selected as shown in Figure 2. Each repetitive bending task was divided into 30 distinct bending (BD), retraction (RT), and transition (TS) portions. The TS portion was defined as the spatial movement from the detected end of bending to the detected start of retraction, which represents the portion where participants switched from bending to retraction.

After segmenting the repetitive bending cycles into distinct portions based on spatial location of the trunk, we calculated measures from the EMG, force plate, and motion data. As the raw EMG data cannot be directly used for processing, correlation, or comparison, we calculated the Root-Mean-Square (RMS) of the signal. Using the RMS, we determined peak amplitude of muscle activity in each of the four muscle groups (LES, RES, LBF, and RBF) for each portion (bending/transition/retraction) of each trunk flexion cycle. Normalization was performed using peak values for each EMG sensor on the back and legs obtained from MVC trials. Time-series data were converted to the frequency domain using the Fast Fourier transform (FFT) to calculate the median frequency of the signal. Values for the first trunk flexion cycle were taken as a reference to calculate change in normalized peak amplitude and median frequency for each subsequent trunk flexion cycle. This led to a total of 48 features that represented muscle activity.

For the trunk movement measure, a total of 54 features were extracted from 3 trunk markers located on the center of upper back, lower back, and hip. First, the norm of velocity and acceleration was calculated based on position of each of the three markers. Using the segmented portions, maximum, mean, and variance were calculated for both norm of velocity and acceleration for each portion of each trunk flexion cycle. Lastly, stability measures included features extracted from the Ground Reaction Forces (GRF) and Center of Pressure (COP) location. Specifically, exported files from the VICON Nexus software included COP locations and GRF for each of the two force plates, as well as combined COP and GRF values. Using COP coordinates, we calculated the maximum deviation from the mean values over each portion of each trunk flexion cycle, as well as velocity (based on position data in the horizontal plane). Mean, peak, standard deviation, and variation in the velocity of COP and GRF were then calculated per portion of each of the 30 cycles for all trials performed by all study participants. Obtained results for all three measures were exported as a separate Excel (.csv) file with each column representing a feature, and rows representing each trunk flexion cycle.

Table 2. Features per portion of trunk flexion cycle (bending, transition, and retraction) extracted from the measures of trunk movement, muscle activity, and whole-body stability.

Measure (Sensor)	Total Sensors	Total Features	Sensor Location	Features per Portion
Trunk Movement (Motion Capture System)	3	54	Upper back, lower back, hip	Maximum of norm of velocity, mean of norm of velocity, variance in the norm of velocity. (per marker)
Muscle Activity (Electromyography)	4	48	Left and right erector spinae, left and right biceps femoris	Peak amplitude of RMS of signal, median frequency, change in peak amplitude of RMS of signal, change in median frequency. (per sensor)
Whole-Body Stability (Force Plates)	2	33	Floor-embedded, one force plate underneath each foot	Maximum distance of COP, maximum GRF at left/right foot, maximum combined GRF, mean of combined GRF, standard deviation in GRF, peak COP velocity, mean COP velocity, standard deviation of COP velocity in sagittal and coronal plane, variance in COP velocity, standard deviation of combined GRF.

shows the total number of features extracted from measures of trunk movement, muscle activity, and whole-body stability.

2.5. Classification Algorithms and Model Development

We developed custom-tailored code in Python to train and test machine learning models. Pre-labeled results of feature engineering were imported as a table, followed by separation based on type of measure. Our dataset consisted of feature groups for the following scenarios: (a) trunk motion measures, (b) muscle activity features from all four sensors (LES, RES, LBF, and RBF), (c) features from lower back sensors, (d) features from LES, € features from RES, and (f) whole-body stability features. Four machine learning classification algorithms, namely Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), and XGBoost (XGB), were applied to determine fatigued state over a single bending/retraction cycle. As per the literature, these approaches are widely utilized and have demonstrated strong performance in the development of human-movement-based models [37,40,64]. Feature normalization through standardization was utilized in the implementation of the SVM and LR methods. A train–test split ratio of 0.8 to 0.2 was employed on the selected datasets to partition the dataset into training and testing subsets for model training and evaluation, respectively. Lastly, we implemented k-fold cross-validation with five folds to ensure robustness in our model evaluation process. The performance of each algorithm was evaluated based on the metrics described in the subsequent section.

2.6. Performance Evaluation

The performance of all four machine learning models was assessed by comparing the predicted fatigue labels with the ground truth data using measures of accuracy (A), sensitivity/recall (R), specificity (S), Precision (P), F1-score (F1), and G-index (G), as shown in

Table 3. Measures for performance evaluation of classification algorithms. TP: trunk bending activity labeled as fatigued correctly, TN: trunk bending activity labeled as not fatigued correctly, FP: trunk bending activity labeled as fatigued incorrectly, and FN: trunk bending activity labeled as not fatigued incorrectly.

Performance Measure	Acronym	Relation to Study Aims	Formulation
Accuracy	A	Ability of the model to detect fatigued state correctly.	$A=\left(\frac{TP+TN}{TP+FP+TN+FN}\right)\times 100\mathrm{\%}$
Sensitivity/Recall	R	Ability of the model to correctly identify all instances of the fatigued state.	$R=\left(\frac{TP}{TP+FN}\right)\times 100\mathrm{\%}$
Specificity	S	Ability of the model to correctly identify non-fatigued states.	$S=\left(\frac{TN}{TN+FP}\right)\times 100\mathrm{\%}$
Precision	P	Reliability of the model’s predictions of the fatigued state.	$P=\left(\frac{TP}{TP+FP}\right)\times 100\mathrm{\%}$
F1-score	F1	A metric to evaluate the overall effectiveness of the model in detecting fatigued states.	$F1=2\times \left(\frac{R\times P}{R+S}\right)\times 100\mathrm{\%}$
G-index	GI	Provides an overall performance of the model in detecting fatigued states.	$GI=\sqrt{{\left(1-R\right)}^2+{\left(1-P\right)}^2}$

. For this study, true positive (TP) denotes the trunk bending activity labeled as fatigued correctly, true negative (TN) is the trunk bending activity labeled as not fatigued correctly, false positive (FP) is the trunk bending activity labeled as fatigued incorrectly, and false negative (FN) is the trunk bending activity labeled as not fatigued incorrectly. Accuracy indicated the ability of the model to detect fatigued state of the trunk bending activity correctly. Sensitivity and specificity are the ability of the model to recognize the two states, namely performing bending while fatigued and without fatigue. Precision is the reliability of the model’s predictions of the fatigued state. The F1-score is the harmonic mean of sensitivity and precision, while the G-index is defined as the Euclidean distance between the point that associates with the best classifier on the receiver operating characteristic (ROC) curve with the point associated with our model. For assessing the model based on G-index, we used: (a) optimum if GI ≤ 0.25; (b) good if 0.25 < GI < 0.70; (c) random for GI = 0.70; and (d) bad for GI > 0.70. Feature importance values provide insights into the relative significance of each feature in predicting the target variable, with higher values indicating greater importance. Importantly, the sum of all feature importance values equals 1, offering a normalized measure of their collective relevance. As such, feature importance values were calculated for high performing algorithms to identify contributions of specific features towards overall model performance.

3. Results

3.1. Differences across Classification Algorithms and Measures

Table 4. Outcomes showing the values of accuracy (A), sensitivity/recall (R), specificity (S), precision (P), F1 score (F1), and G-index (GI) with different classification algorithms categorized according to the type of measures of trunk motion (UB: upper back, LB: lower back, Hip: hip), muscle activity (LES: left erector spinae, RES: right erector spinae, LBF: left biceps femoris, RBF: right biceps femoris), and whole-body stability along with their respective features.

Measures	Number of Features	Classification Algorithm	A	R	S	P	F1	GI
Trunk Motion (UB, LB, Hip)	54	SVM	62.5	67.5	57.3	-	-	-
		LR	67.8	69.5	66.2	-	-	-
		RF	79.6	77.4	81.7	82.1	0.80	0.28
		XGBoost	80	79.7	80.3	80.5	0.79	0.28
Muscle Activity (LES, RES, LBF, RBF)	48	SVM	74.4	69.2	79.9	77.6	0.72	0.38
		LR	76	75.4	77	76.6	0.75	0.34
		RF	94.5	93.6	95.5	93.7	93.6	0.09
		XGBoost	94.6	94.9	94.5	94.5	94.6	0.08
Muscle Activity (LES, RES)	24	SVM	75.3	72.5	78.4	77.2	0.74	0.36
		LR	69.4	66.8	72.3	70.9	0.68	0.44
		RF	90.5	88.7	92.5	90.6	0.89	0.14
		XGBoost	91.3	91.5	91.2	91.2	0.91	0.12
Muscle Activity (LES)	12	SVM	75.7	74.7	77.1	76.6	0.75	0.35
		LR	66.9	64.7	69.5	68.2	0.65	0.48
		RF	84.7	84.7	85	82.4	0.81	0.26
		XGBoost	85.1	85.4	84.9	85.2	85.2	0.21
Muscle Activity (RES)	12	SVM	64.7	62.5	67.5	66.47	0.63	0.50
		LR	61.7	58.7	65.3	63.1	0.60	0.55
		RF	84.6	84.1	85.2	86.1	0.84	0.22
		XGBoost	87.1	86.7	87.7	87.6	0.87	0.18
Muscle Activity—single sensor (avg(LES, RES))	12	SVM	64.7	68.6	72.3	71.5	0.69	0.42
		LR	64.3	61.7	67.4	65.6	0.62	0.51
		RF	84.6	84.4	85.1	84.2	0.82	0.24
		XGBoost	86.1	86	86.3	86.4	43.1	0.19
Whole-Body Stability (Force Plate L/R foot)	33	SVM	61.1	49.2	73.1	64.4	0.55	0.61
		LR	52.9	59.5	46.7	53.2	0.56	0.62
		RF	92.9	91.9	94.1	94.5	0.94	0.09
		XGBoost	93.5	94.1	93.1	93.3	0.94	0.10

shows the values of accuracy, specificity, and sensitivity with different classification algorithms categorized according to the type of measures and their respective features. The outcome of our analysis showed that the classification algorithms RF and XFB led to higher accuracy (A), recall (R), and specificity (S) than the SVM and LR algorithms for all three types of measures. Highest values were seen for the measures of muscle activity with RF (A: 94.5%, R: 93.6%, S: 95.5%) and XGB (A: 94.6%, R: 94.9%, S: 94.5%) algorithms with 48 features obtained from all four EMG sensors. This was followed by whole-body stability and the least predictability was seen for trunk motion measure. Values reduced with reduction in the number of sensors (as well as features), yet the good predictability was obtained using RF and XGB algorithms. For instance, XGB (A: 86.1%, R: 86%, S: 86.3%) was effective in classifying fatigue with data obtained from a single EMG sensor located on the lower back (erector spinae) muscle. On the other hand, using the data obtained from the force plate with RF (A: 92.9%, R: 91.9%, S: 94.1%) and XGH (93.5, 94.1%, 93.1%) led to high predictability. Overall outcomes show that floor-embedded force plates, and EMG sensors can be more effective in detecting fatigue compared to trunk movement.

3.2. Feature Importance Values

Outcomes of performance evaluation showed high performance of RF and XGB classification algorithms. To identify specific contributions of features, feature importance values for Random Forest (RF) and XGBoost (XGB) algorithms were analyzed across measures of muscle activity (four sensors and single-sensor contributions), trunk motion, and whole-body stability. Looking at trunk motion as shown in Figure 3, acceleration parameters for the hip marker during the retraction portion emerged as the top contributors for RF (0.03–0.04), followed closely by parameters for the transition phase. Particularly, ‘Max_normAccHIP1_R’, ‘Mean_normAccHIP1_R’, ‘var_normAccHIP1_R’, which denotes the maximum value, mean value, and variance of the norm of acceleration of the hip marker during retraction portion showed the highest importance for RF. Meanwhile, XGB showed the ‘Mean_normVelUB1_I’ feature, denoting the average of the norm of velocity of the upper back marker during transition as the highest contributor. Overall, the features related to bending exhibited lower importance values (<0.02) across both RF and XGB algorithms.

In the analysis of muscle activity (Figure 4), ‘delta’ parameters, representing changes in muscle activity over time, were identified as the most influential features for both RF and XGB algorithms. Specifically, the ‘delta_MF_s4_I’ feature, denoting the change in median frequency of the EMG sensor placed on the right leg, showed the highest importance for RF, while ‘delta_EMG_s1_I’, which denotes the change in amplitude of sensor on the left lower back, displayed highest importance with XGB. Notably, features from sensors placed on the left (Figure 5) and right erector spinae (Figure 6) exhibited distinct patterns of importance, with peak activity and ‘delta’ parameters demonstrating high relevance during specific phases of movement. When only the features from EMG sensor placed on the left lower back were considered, ‘delta_EMG_s1_R’ and ‘delta_EMG_s1_I’ features representing the change in amplitude during the retraction and transition portions of trunk flexion cycles were most prominent with both RF and XGB algorithms. On the other hand, considering the sensor on right lower back, ‘nPeak_RES_I’, which denotes the normalized peak amplitude during transition portion, displayed the highest importance for both the RF and XGB algorithms.

Figure 3. Feature importance values for measures of trunk motion for Random Forest (RF) and XGBoost (XGB) classifiers (note: features are displayed in the format ‘value/featurename_portion’, with portions as B: bending, I: transition, R: retraction).

Figure 3. Feature importance values for measures of trunk motion for Random Forest (RF) and XGBoost (XGB) classifiers (note: features are displayed in the format ‘value/featurename_portion’, with portions as B: bending, I: transition, R: retraction).

Looking at the features obtained from whole-body stability (depicted in Figure 7), ‘MeanCOPvel_I’, which represented the mean velocity of the Center of Pressure (COP) during the transition phase emerged as the most significant feature for both RF (0.07) and XGB (>0.1). Similar to the measures of muscle activity and trunk movement, top features from the stability measures were also associated with the transition and retraction phases, emphasizing the importance of these phases in detecting fatigue.

4. Discussion

This study focuses on developing an exoskeleton-specific fatigue detection model through a controlled laboratory experiment where study participants performed 30 repetitive trunk flexion tasks with/without medium-high back fatigue while assisted by a BSIE. Our detailed analysis included segmenting each bending cycle into three portions based on the trunk movement as bending, transitioning from bending to retraction, and retraction. This led to 135 features from measures of muscle activity in the lower back and legs, trunk motion, and whole-body stability during each trunk flexion and return cycle. While prior studies have developed similar fatigue detection models by recording physiological signals [40,41,42,43,45,46,54], this study is the first to develop a task- and exoskeleton-specific fatigue detection model. The fatiguing task selected for this study included intermittent trunk flexion task cycles with sustained bending and short duration relaxation breaks (15 s intervals), as most real-world tasks are defined as cycles with a diverse set of static, sustained, and dynamic activities. Between intermittent task cycles and repetitive tasks, we performed feature analysis on repetitive tasks, as they provided higher consistency between no fatigue and medium-high fatigue levels, as well as a higher amount of datapoints, which are crucial factors to develop efficient machine learning models.

We assessed the performance of four classification algorithms (SVM, LR, RF, and XGB) in detecting fatigue while performing BSIE-assisted trunk flexion tasks. These algorithms were primarily selected based on previous studies that developed similar models to detect/classify fatigue levels [43,45]. The results of our analysis demonstrated that the Random Forest (RF) and XGBoost (XGB) algorithms consistently outperformed the Support Vector Machine (SVM) and Logistic Regression (LR) algorithms in terms of accuracy (A), recall (R), and specificity (S) across all three types of measures—muscle activity, whole-body stability, and trunk motion. Similar outcomes have been reported in past studies, where RF and Gradient Boosted Decision Tree algorithms showed better performance over SVM [35,38,43]. Particularly noteworthy were the high values obtained for measures of muscle activity, with the RF (A: 94.5%, R: 93.6%, S: 95.5%) and XGB (A: 94.6%, R: 94.9%, S: 94.5%) algorithms utilizing 48 features from all four EMG sensors. High accuracy from EMG measures was expected due to the correlation of muscle fatigue and global fatigue [36,65,66]. However, placing four EMG sensors in real-world scenarios may be infeasible due to challenges in field evaluations, especially while wearing EXOs [6]. Subsequent analyses revealed a reduction in values with a decrease in the number of sensors (and features), yet RF and XGB algorithms consistently demonstrated good predictability. For instance, XGB achieved good performance (A: 86.1%, R: 86%, S: 86.3%) when utilizing data from a single EMG sensor located on the lower back (erector spinae) muscle. Interestingly, employing data obtained from the force plate yielded high predictability with RF (A: 92.9%, R: 91.9%, S: 94.1%) and XGB (A: 93.5%, R: 94.1%, S: 93.1%) algorithms. This could have been due to a combination of kinematic (movement in COP) and kinetic (ground reaction forces/GRF) features provided by force plates, as opposed to motion or muscle activation alone. These findings suggest that floor-embedded force plates may offer greater effectiveness in detecting fatigue.

The dominance of ‘delta’ parameters in both RF and XGB algorithms (Figure 4) suggests that changes in muscle activity over time play a crucial role in predicting fatigue during trunk bending tasks. Therefore, when using EMG systems, multiple repetitions of trunk flexion tasks may be required to accurately classify fatigue level. Meanwhile, looking at trunk motion, it was expected that movement differences between with vs. without fatigue could be clearly seen towards the end-effector (upper back) rather than the hip region. For instance, the upper-body region was found to be most effective in predicting fatigue during sit-to-stand activities [41]. However, the higher importance attributed to acceleration parameters for the hip marker during the retraction phase (RF algorithm) underscores the significance of this phase in trunk motion analysis and its potential correlation with fatigue. Meanwhile, the lower importance values associated with bending parameters suggest that while trunk bending is a critical component of the activity, other phases such as retraction and transition may have greater predictive power for fatigue. This could be because transition and retraction phases require higher effort to work against gravitational forces, especially given the added weight of the BSIE. Looking at stability, the prominence of features related to the mean velocity of the Center of Pressure (COP) during the transition phase suggests that maintaining stability during transitions between different phases of trunk bending is crucial for fatigue prediction (Figure 7). The consistency of top features being associated with the transition and retraction phases across measures of whole-body stability, muscle activity, and trunk motion indicates the importance of these phases in assessing overall stability and fatigue risk during trunk bending activities. Overall, these inferences underscore the multidimensional nature of fatigue prediction during trunk bending activities, emphasizing the importance of exploring diversity in physiological measures and their corresponding features in developing accurate predictive models.

Outcomes from this study may be used to develop a portable wearable system, or even a module attachment for BSIEs to indicate fatigued states in their wearers. One example is shown in Figure 8, similar to previously proposed mobile monitoring devices [67,68]. Our study shows the success of force plate and single-sensor EMG systems for classifying fatigued states during trunk bending tasks assisted by a BSIE. Among these two sensor systems, EMG requires skin preparation, attachments/taping, and may not be optimal for continuous monitoring of workers. On the other hand, force plates can be embedded underneath the floor. While EMG-derived features may offer advantages in detecting muscle fatigue in specific muscle groups, force plate system can provide a comprehensive assessment of overall body movement, reflecting a global measure of fatigue. This shows potential opportunities to replace force-plates with insole pressure sensors that provide similar measures, specifically acceleration, plantar pressure, COP, and GRF at each foot [35,46]. Our study employed two force plates, each supporting one foot, potentially facilitating a seamless transition to a pressure insole system. This transition could pave the way for more accessible and minimally invasive fatigue detection among BSIE users.

The literature shows several studies focused on developing fatigue detection models using machine learning methods. These studies utilize sensor outputs to predict perceived fatigue levels [53,69]. It is important to acknowledge that such models are trained on datasets generated from experiments, with predicted fatigue labels reflecting respective experimental conditions. Consequently, the model’s ability to provide reliable predictions in real-world settings depends on the similarity between the task and the training dataset [41,43]. This study introduces an EXO-specific model for fatigue detection. Specifically, the machine learning model demonstrated in this article was trained on a dataset derived from a controlled laboratory experiment where participants performed repetitive trunk flexion tasks while wearing a BSIE. Objective data, obtained through motion capture, EMG, and force plate systems, were used to extract features, which were then labeled based on each participant’s perceived exertion for each trunk flexion cycle. Notably, reported perceived fatigue levels may be influenced by the perception of task demands, which is potentially impacted by wearing an EXO. Wearers of EXOs might report lower RPE values due to the assistance provided by the BSIE. For instance, wearing BSIEs has been shown to enhance maximum acceptable limits by 6% [70]. However, it is also plausible that EXO wearers may feel more empowered (even with actuator assistance turned off), and thus, inclined to report lower RPE values than they would without the EXO. A lack of evidence exists regarding how wearing an EXO might influence the perception of fatigue. Conversely, objective measurements derived from sensor readings may also vary. For instance, studies have reported lower peak amplitude and median frequency of EMG signals in the lower back during initial and fatigued states when assisted with EXOs [71,72]. Similarly, the structural components of the device can impact natural body movement [73], thereby affecting kinematics and stability measures. Consequently, a model developed using training datasets without consideration of EXOs may not be suitable for fatigue detection during EXO-assisted tasks. By developing an EXO-specific model, our study aims to address the potential discrepancies in fatigue level prediction, ensuring that predicted outcomes are more aligned with realistic scenarios.

Although our article demonstrates high performance in detecting fatigue, our study is subject to a few limitations. The participants in this study included average-sized adult males, and generalizing findings from this study to a wider population would require recruitment of participants of varying gender, anthropometric measurements, and demographics. To confirm whether outcomes obtained from laboratory-based equipment (e.g., force plate) provide similar results as their portable counterparts, future endeavors will involve obtaining similar measures as force plates, but with a pressure insole system. This would also include assessing the performance of insole systems during real-world trunk bending tasks outside the laboratory settings. The presented findings in this work can be helpful to ergonomists in implementing BSIEs in industrial scenarios. The dataset in this study consisted of 30 cycles of repetitive trunk flexion performed with/without medium-high fatigue. Our next step would include conducting a more comprehensive study by increasing the number of experimental cycles and recruiting a much larger sample size. Future work could also delve into field evaluations, focusing on muscle demands in trunk musculature, such as the trapezius and oblique muscles, especially relevant for tasks involving asymmetric postures. Additionally, exploring lower body kinematics, stability measures based on foot contact area derived from marker positions, and a comparative analysis between the demands imposed on wearers of BSIE across newer and different types of these wearable assistive devices.

5. Conclusions

Monitoring fatigue levels can be crucial to ensure the safe and effective use of EXOs in real-world scenarios. Using an experiment involving fourteen participants who performed repetitive trunk flexion tasks with/without fatigue, we extracted features from muscle activity, trunk motion, and whole-body stability. Our analysis employed four classification algorithms, i.e., SVM, LR, RF, and XGB, to predict fatigue levels. Notably, XGB demonstrated high accuracy, recall, and specificity in classifying fatigue using data from a single EMG sensor placed on the lower back (erector spinae) muscle. Additionally, stability measures showed promising predictability, specifically when using the RF (A: 92.9%, R: 91.9%, S: 94.1%) and XGB (A: 93.5%, R: 94.1%, S: 93.1%) algorithms. A subsequent feature importance analysis demonstrated prominence of transition portion between bending/retraction portions, and retraction portions of the trunk flexion cycle. These findings underscore the potential of utilizing force plates, with the prospect of replacing them with pressure insoles, to facilitate real-world fatigue detection during BSIE-assisted trunk flexion tasks. Overall, efforts presented in this study demonstrate the potential benefits of integrating technologies with EXOs by utilizing machine learning algorithms to enhance workplace safety and optimize the utilization of assistive devices in industrial settings. Further research can delve into developing accurate portable sensing technologies to enable efficient evaluation of EXOs in real-world industrial scenarios.