Next Article in Journal
Segmentation of Aorta 3D CT Images Based on 2D Convolutional Neural Networks
Next Article in Special Issue
Challenging Ergonomics Risks with Smart Wearable Extension Sensors
Previous Article in Journal
Selective Disinfection Based on Directional Ultraviolet Irradiation and Artificial Intelligence
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Automated Workers’ Ergonomic Risk Assessment in Manual Material Handling Using sEMG Wearable Sensors and Machine Learning

1
School of Engineering, California State University East Bay, 25800 Carlos Bee Blvd., Hayward, CA 94542, USA
2
Department of Civil and Environmental Engineering, Hole School of Construction Engineering, University of Alberta, 9211 116 St. NW, Edmonton, AB T6G 1H9, Canada
3
Department of Civil, Construction, and Environmental Engineering, San Diego State University, 5500 Campanile Dr., San Diego, CA 92182, USA
*
Author to whom correspondence should be addressed.
Electronics 2021, 10(20), 2558; https://doi.org/10.3390/electronics10202558
Submission received: 22 September 2021 / Revised: 10 October 2021 / Accepted: 14 October 2021 / Published: 19 October 2021

Abstract

:
Manual material handling tasks have the potential to be highly unsafe from an ergonomic viewpoint. Safety inspections to monitor body postures can help mitigate ergonomic risks of material handling. However, the real effect of awkward muscle movements, strains, and excessive forces that may result in an injury may not be identified by external cues. This paper evaluates the ability of surface electromyogram (EMG)-based systems together with machine learning algorithms to automatically detect body movements that may harm muscles in material handling. The analysis utilized a lifting equation developed by the U.S. National Institute for Occupational Safety and Health (NIOSH). This equation determines a Recommended Weight Limit, which suggests the maximum acceptable weight that a healthy worker can lift and carry, as well as a Lifting Index value to assess the risk extent. Four different machine learning models, namely Decision Tree, Support Vector Machine, K-Nearest Neighbor, and Random Forest are developed to classify the risk assessments calculated based on the NIOSH lifting equation. The sensitivity of the models to various parameters is also evaluated to find the best performance using each algorithm. Results indicate that Decision Tree models have the potential to predict the risk level with close to 99.35% accuracy.

1. Introduction

While the introduction of automation and robotics alleviated the frequency and intensity of manual work, many industries such as construction still rely on labor manual work as the main source of production. Work-related musculoskeletal disorders (WMSDs) are among the most reported jobsite injuries [1,2]. Labor manual activities at construction sites offer a high risk of exposure to vibration, repetitive movements, forceful exertions, non-neutral trunk postures, and awkward motions for extended durations which increase the risk of WMSDs. Inability to prevent such insecure postures and movements can translate to escalated risk of injuries while deteriorating workers’ physical health, morale, and the quality of life, as well as the employer’s safety records and productivity. Prevention is undoubtedly more desirable than cure, and ergonomic risks can be successfully eliminated by raising workers’ self-awareness of hazardous postures and body movements.
The advancement of technology in miniaturized electronics and powerful cloud-based software has enabled successful applications of sensor-based Internet-of-Thing (IoT) platforms in construction jobsites [3,4]. Some of the recent applications of such IoT systems enabled quantifying workers body motions using wearable sensors. Many of those past studies used Inertial Measurement Units (IMUs) and vision-based systems to examine construction resources’ activities [5]. While those data collection techniques proved successful to detect workers movements, certain risks cannot be identified unless direct measurement of muscle activities are performed. This is because while an activity may not seem harmful, the real effect of awkward muscle movements, strains, and forces may result in hazardous ergonomic situations that cannot be identified using IMUs or vision-based systems. This paper presents the results of a series of empirical analyses that leverage surface electromyography (sEMG) sensors to directly measure the human muscle electrical impulses during activities that involve lifting objects. The hypothesis of the research is that sEMG can characterize ergonomically hazardous body postures and muscle extremes. To evaluate the hypothesis, the experiments results have been evaluated in the context of the National Institute for Occupational Safety and Health (NIOSH) Lifting Equation [6].

2. Literature Review

2.1. Workers’ Ergonomic Risk Assessment

Understanding the nature and preventing exacerbation of musculoskeletal disorders (MSDs) as a result of work-related injuries have been subject to many past studies. MSDs are referred to as the situations involving the nerves, tendons, muscles, and supporting structures of the body are injured. In particular, low back pain, shoulder injuries, and distal upper extremity disorders, including tendonitis, epicondylitis, and carpal tunnel syndrome, can be named as such disorders [7]. Work-related MSDs, or WMSDs, are among the most reported jobsite injuries that can negatively affect workers’ health, well-being, and productivity [1,2,8]. This has been identified by many previous past studies conducted in collaboration with national and international safety organization. For example, in a research study in 2004, Waters [7] summarized the findings of two research agendas to increase our understanding of the approaches that can prevent this kind of disorders. The first agenda was based on the data gathered from several hundred practitioners and safety and health specialists representing industry, labor, and academia, and developed by the National Institute for Occupational Safety and Health’s (NIOSH) National Occupational Research Agenda (NORA) MSD team. In the second agenda, which was developed by the National Research Council (NRC) and the Institute of Medicine’s (IOM) National Panel on MSDs and the Workplace, data had been collected from leading researchers in the fields of medicine, information science, and ergonomics. The results clearly showed that work-related injuries and illnesses represent a considerable health problem for the U.S. industrial labor force [7]. In another study by Pascual and Naqvi [9], the ergonomics risk assessment methods used by Joint Health and Safety Committees (JHSCs) have been investigated. It is concluded that most JHSC curricula have minimal ergonomics scope; thus, JHSCs rely mostly on injury reports and worker complaints to assess ergonomics risk, and indeed most ergonomics analysis tools available require some ergonomics knowledge.
There are research studies on proactive monitoring of industrial workers’ awkward postures and non-ergonomic motions that date back to the 1990s. Schoenmarklin et al. [10] monitored the acceleration in the flexion/extension as the best kinematic parameter for evaluation of the low and high incident rates of hand/wrist Cumulative Trauma Disorders (CTDs). However, recent studies focused more on using automated methodologies. Such approaches often take advantage of the advancements in wearable sensors or vision-based monitoring methods to detect such hazardous situations [11,12]. Recently, wearable sensing technologies have provided opportunities to gather near real-time data to analyze workers’ safety and health situations [13]. These approaches are often characterized by features such as being low-cost, easy to use, highly accuracy, and non-intrusive [14]. In a study aiming at reviewing MSDs in the construction industry, Valero et al. [15] indicated that the subjectivity and lack of accuracy of visual assessment demand replacing such observations with more accurate and precise posture measurement devices and methods. In some recent studies, in order to identify jobsite workers activity, accelerometers embedded in smart mobile phones have been used [12,13,14,16,17,18,19]. An artificial neural network (ANN)-based models have been developed for identifying falls and manual material handling activities with a high accuracy using the smartphone installed on workers’ bodies by Akhavian and Behzadan [16]. Yang et al. [19] presented a method based on smartphone sensor data acquisition and the concept of labor intensity to evaluate construction workers’ workloads. A sensor application based on the smartphone platform was created to effectively measure labor intense activities so that the application could track construction workers’ movement data unobtrusively.
Recently, IMU sensor-based activity identification models have been explored for a diverse set of applications in the construction industry such as work sampling and fall detection to address practical implementation issues with smartphones [20,21,22,23]. Furthermore, the IMU sensor-based models have been vastly utilized towards enhancing ergonomic and safety aspects of construction activities [12,16,24,25]. Yan et al. [25] designed a motion warning system a real-time motion to detect predefined thresholds of hazardous ergonomic postures and alert the worker’s smartphones. Even though this IMU system recognized movement directions, angles, and rotation, it failed to recognize the real muscle stress and power. In order to validate the system, robust clinical motion data output and sufficient alarm ringing were utilized in both laboratory and field experiments on a construction site located in Hong Kong. The results show that the proposed system assists construction workers to prevent WMSDs without disturbance and interruption during the operations [25]. Jahanbanifar and Akhavian [24] modeled body movements and physical activities, including pushing and pulling, in laboratory-scale experiments. The performed force was measured by a work simulator tool, and a smartphone sensor installed on the active arm collected accelerometer data. An ANN was trained with the accelerometer data and the force levels. The testing data results reveal that the trained model can predict the force level with over 87.5% accuracy [24]. Generally, such methods use IMUs as a wearable tool that detects acceleration produced due to performing a specific activity. The produced data are used to train machine learning algorithms that enable detecting the identified hazardous activities in examples not used during the training of the model. Such models are then evaluated for their generalization, and their performance is enhanced through the use of more training examples and sophisticated prediction algorithms.
Despite the fact that sensors provide detailed information, not all sensors can be utilized in the construction industry due to the dynamic nature of construction activities [26]. Based on previous research, the ideal sensors for construction applications should have some unique characteristics such as being simple and easy to wear, unobtrusive, affordable, and wireless. Moreover, the sensor should provide valid data and import minimal or no preprocessing for noise cancellation. Thus, identifying a reliable sensor for classifying and monitoring construction activities is crucial, and it can help develop health monitoring systems to prevent WMSDs. In recent studies, attaching sensors using armbands and wristbands have been identified as an affordable, non-invasive, lightweight, and wireless wearable data collection method that is suitable to gather workers’ forearm sEMG and inertial measurement unit (IMU) data [26].

2.2. Research Motivation

Many researchers have employed these data collection methods for different applications in various fields; nevertheless, the use of sEMG sensors has not been fully investigated for the ergonomic risk evaluation in the context of construction workers’ safety and health. Neck disorders have been the subject of a study where sternocleidomastoid and the upper trapezius muscle activities were monitored using sEMG sensors [27]. The experiments, however, were intrusive, and the data collection probes were not wearable. In another study, sEMG systems were operated to examine the lower back movements in prefabricated panels erection [8]. The predictive ability of the developed model was somewhat limited due to limitations, such as using only one motion segment. Matsumura et al. experimented with a wrist-band type sEMG sensor to analyze and recognize sEMG data using a Neural Networks (NN) model. This research was limited only to identify a few wrist movements and did not address the construction industries’ safety requirement [28]. In a recent study, researchers have assessed construction workers’ fatigue level using sEMG systems [29]. Results has shown a considerable difference in sEMG parameters while subjects represented different fatigue levels. Results also proved the workability of the wearable sEMG to evaluate workers’ muscle fatigue as a means for assessing their physical stress on construction sites [29]. Bangaru et al. [14] evaluated the data quality and reliability of forearm sEMG and IMU data from a wearable sensor for jobsite activity classification. In order to achieve this goal, seven experiments have been conducted. The experiments’ results show that the arm-band sensor data quality is close to the conventional sEMG and IMU sensors, with perfect relative and absolute reliability between trials for all the considered activities.
Despite the potential of wearable sEMG systems to identify different activities risk, the feasibility of a wearable sEMG system to identify heavy weightlifting activities has received very little attention. To address this issue, and in order to identify high-risk lifting activities in jobsites, the present study aimed to propose a machine learning modeling approach where an sEMG-based system is used in a total of 54 experiments to collect data from closely monitored lifting activities. Traditional Machine learning algorithms were implemented to classify the sEMG signal data collected from the lifting activities. The outcomes of this study are expected to identify light classification models that do not require extensive computational burdens and have the potential to be deployed on the edge and in the field (e.g., job sites).

2.3. Contributions

This study makes several contributions to the body of knowledge in worker safety in material handling tasks. First, the current study empirically identifies a muscle that can present changes in the sEMG signals depending on the weight of the material lifted. Second, a widely acceptable standard, namely NIOSH lifting equation was successfully used as a benchmark for qualitative and quantitative risk evaluation, which shows its potential for other modeling approaches in the domain of workers’ ergonomic risk assessment. Third, by implementing multiple machine learning algorithms, this study also helps identify models for successful classification of material handling risk using sEMG data as well as models that might not be appropriate for analyzing sEMG signals. Fourth, the results of this study are expected to help alleviate the issues with current manual material handling processes and provide researchers and practitioners with an automated sensor- and machine learning-based decision-support tool to assess workers ergonomic risks. The following sections delineate the developed methodology.

3. Research Methodology

In the presented study, a series of experiments were conducted in a controlled environment to measure the muscle activity while performing manual material handling. A sEMG system was used to detect muscle movements in static lifting tasks, which frequently occur as part of material handling. The risk assessment was conducted using the NIOSH lifting equation. In this section, first, a brief overview of the EMG system used and the NIOSH lifting equation are provided, and then the experiments are described in detail.

3.1. EMG System

EMG sensors detect micro-electrical signals generated by muscles once muscle-cells are neurologically activated by brain commands. The superior data quality made EMG systems increasingly popular in industries where significant human movement is expected. Most importantly, its potential to measure internal muscle movements and strains as well as to identify extremes make it an ideal tool to assess postural hazards. EMG sensors are categorized into two types and the first type is known as ‘insertion-electrodes’. By inserting a needle-like-electrode into a deep, targeted muscle, precise muscle EMG movements can be measured. Since this procedure always requires a medical expert for the insertion and it has some risk of bleeding, infections, and nerve injuries, it is not suitable for everyday use by workers. Therefore, the less expensive and safer second type, ‘Surface-type-EMG’ or sEMG sensors, with more accurate ‘wet-type’ electrodes, were selected for the experiment. When a muscle requires more force, additional motor units will be added, and then sEMG signal becomes denser visually with increased amplitude. A weighted sum of the action potential is therefore recorded as the sEMG value, since the muscle fibers are located deep and far from the surface.
The proposed system consists of an input stage, a signal processing stage, and an evaluation stage. The real-time micro-electrical signals from sEMG electrodes are transferred to the receiver via Bluetooth and micro-voltages of data will be amplified and converted to Digital from Analog. The software algorithm processes captured muscle motion data and translated muscle movements into meaningful parameters in real-time. The Noraxon Mini DTS system with 2 wireless sEMG muscle sensors was used for this experiment along with Myo-Muscle software version MR3 3.14.28. The software can handle kinesiological data captured with the Noraxon Mini DTS system and provide thorough insights for any performance improvement, injury prevention, and specific feedback from neuromuscular sensors. Some features of Myo-Muscle version MR3 3.14.28 include quickly assessing sEMG noise and frequency contents, evaluating the quality of skin preparation using a built-in impedance checker, configuring multiple signal data for subsequent analyses, enabling IMU data for specific sensors, and storing and recovering collected signal data. A webcam was also used to record tests for later synchronization of the collected data for analysis.

3.2. NIOSH Lifting Equation

The top five ergonomic issues in industries with heavy manual work include working over a long period of time at the same position, awkward twisting or bending, working at cramped positions, handling heavy tools, equipment, or materials in an unsafe manner, and working with an injury. Major hazardous ergonomic postures, longer holding times, and repetitive work in a similar posture for an extended duration can clearly increase the risks of future WMSD. All such instances are reflected in an equation devised by NIOSH to prescribe the allowable limit for lifting weights. The NIOSH lifting equation entails a series of variables and multipliers to consider the relative position of a person to the object they lift as well as the quality of the workers grip, frequency and duration of lifting activity, and twisting requirement [6]. Using NIOSH lifting equation, the Recommended Weight Limit (RWL), which prescribes the maximum acceptable weight that a healthy employee could lift, can be determined (see Equation (1)).
RWL = LC (51) × HM × VM × DM × AM × FM × CM
where LC is a load constant equal to 51 lbs. and the rest of the parameters are multipliers (M), including H as the horizontal location of the object relative to the body, V as the vertical location of the object relative to the floor, D as the distance the object is moved vertically, A as the asymmetry angle or twisting requirement, F as the frequency and duration of lifting activity, and C as the coupling or quality of the workers grip on the object. Table 1 and Figure 1 adopted from [6] describes the variables used in this equation. Horizontal Multiplier (or Horizontal Location of Hands) (HM): During the experiment horizontal location of hands at the start was 15 inches and the end was 24 inches approximately to the mid-point of ankle bones. Vertical Multiplier (or Vertical Location of the Hands) (VM): During the experiment, vertical location of hands at the origin was 14 inches and the destination was 32 inches approximately against the floor. Distance Multiplier (or Vertical Travel Distance) (DM): During the experiment, vertical travel distance of the lifting task between origin and the destination was 18 inches approximately. Asymmetric Multiplier (or Asymmetric Angle) (AM): During the experiment, asymmetric angle was zero due to no rotation or twisting of trunk and shoulders. Linear bending forward, up, or down only considered and twisting was restricted due to the time availability of the research for additional muscle measurements. Frequency Multiplier (FM): In this experiment, 10 lifts per minute were concerned to obtain desired results based on available resources. Coupling Multiplier (CM): During the experiment, lifting objects with Good Couplings were used. CM values for Good Coupling are 1, Fair coupling is 2, and Poor Coupling is 3.
A Lifting Index (LI) value is also calculated by dividing the actual weight to be lifted by the RWL (see Equation (2)).
LI = Weight ÷ RWL
Therefore, an LI value below 1.0 does not pose a significant risk to a healthy employee. A value exceeding 1.0 means an increased risk with the task and an LI value above 3.0 states that the task as a high-risk to a healthy worker. As the LI increases, the level of upper back injury risk and associated MSD risk increases correspondingly.

3.3. Experiments

In order to determine the most sensitive muscles to the lifting tasks, simple and straight body movements related to hands, forearms, arms, legs, thighs, and back were performed to evaluate the change in the signal received from sEMG sensors connected to those muscles. Figure 2a shows the two points on both the Thoracic (TH) and Multifidus (MU) muscles where the sEMG sensors were attached. The ideal location of sEMG placements to obtain the optimum signal strength was identified by moving sensors along the back muscles and simulating the lifting task prior to the actual experiment. While the research team member was lifting the weight, the sEMG patterns of TH (upper-back) and MU (lower-back) muscles were observed. Additionally, Figure 2b shows the research team member lifting the weights in different postures. A member of the research team performed a few pre-defined safe lifting activities according to NIOSH lifting requirements. Sensors connected to muscles other than the lower back and upper back produced false positives, false negative, and nondiscriminatory and inconsistent results. As such, the upper back (i.e., Thoracic or TH) muscles were chosen for data collection.
All the lifting tasks were performed by a member of the research team, a male at age 40, height 5′8″, and weight 180 lbs. lifting multiple weights. A dumbbell set with adjustable weights from 10 lbs. to 35 lbs. and a size of approximately 14″ × 9″ × 9″ was used as the ‘Weight’. During the entire experiment, the lifting speed and frequency were kept constant with some pre-training to minimize the impact. The total number of 54 tests have been done using weights of 10 lbs., 15 lbs., 20 lbs., 30 lbs., and 35 lbs. and with H values equal to 15 and 17. Each test was repeated 10 times at 10 lifts per minute except for tests with 35 lbs. and H17, which were done only 5 times to limit the potential of any injury. In addition, one experiment with the 10 lbs. weight was removed due to incomplete data collection. Repeating each weight lift several times was done to collect more data with higher generalizability potential of the machine leering models to be developed. The overall experiment was completed within 1 h including the rest breaks. Table 2 describes the lifting experiment variables. All the variables were assessed under various experiments to calculate variables in the NIOSH equation to determine the RWL. The horizontal distance between ankles to the mid-point of the hands grasping the object (H) at origin and destination kept unchanged during the entire research. Vertical travel distance (V) of the lifting task between origin and the destination also kept unchanged during the entire research. Twisting was restricted and linear bending forward, up, or down was only allowed. Therefore, Asymmetric Angle was zero due to no rotation or twisting of trunk and shoulders. All the variables except for the H in the last set of experiments were kept constant. A slightly higher H value of 17″ was used in the last set of experiments.
According to the Equation (1) and based on the multipliers provided in Table 2, the RWL is calculated as 12.4 for the first five sets of experiments. Therefore, the values of LI can be calculated from Equation (2) and are shown for each experiment in Table 3.
As shown in Table 3, a total of 54 experiments were conducted to collect sEMG data from closely monitored lifting activities. Using the NIOSH lifting equation, the experiments were placed under three risk categories using LIs from 0.8 to 3.2. First, 19 tests were designed to have ‘Nominal Risk’ with an LI value below or equal to 1.2. The next 20 tests were designed as ‘Increased Risk’ activities by increasing the LI value from 1.2 to 2.4. The last 15 activities were ‘High Risk’ lifting activities with an LI value above 2.8.

4. Data Analysis

Experiments 1 to 9 were used as a proof-of-concept to observe potential changes in the received signal from the upper back sensor as a result of weightlifting activities. The lifter started by lifting a dumbbell-set weighted 10 lbs. located on a bench and placed it on a higher level. Since only the magnitude of the received signal is sufficient to evaluate the effect on muscles, absolute values of the collected sEMG data are used. The experiments were carried out to recognize such errors and to identify precautions. To minimize such errors more sEMG data were collected from multiple attempts. Averaged representations of multiple datasets are expected to be more reliable than individual bursts of activities. Average peak absolute value equal to 150 µV of the signals received was used for calculations of the first 9 experiments. Additionally, filtering or averaging the signal over time can suppress high-frequency deflections of the sEMG data to create a smoother signal. Fast Fourier transform (FFT) was used to smooth out the data and represent them in the frequency domain. FFT is also used to observe the effect of varying weights on the signal frequency domain in addition to the raw data in time domain. The median of FFT signals was used in the subsequent machine learning models. Furthermore, built-in hardware and software filtrations optimized the EMG signal quality. Similar to the first 9 experiments, the following five sets of experiments were conducted with a fixed weight of 15, 20, 25, 30, and 35 lbs. s.

4.1. Machine Learning Classification Algorithms

In order to establish a model to classify ergonomic risk based on the lifting activities, the performance of four machine learning classification algorithms, including Decision Tree, Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Random Forest, have been examined. This study utilized R programming language via RStudio version 1.3.959 and its associated function packages in machine learning, including psych, e1071, ggplot2, dplyr, caret, class, randomForest, and rpart, to train and test the developed machine learning classification models.

4.1.1. Decision Tree

Decision Tree, a supervised learning algorithm, can be used for both classification and regression analyses. In other words, it can handle both categorical and continuous input and output variables. The main goal of Decision Tree in this study is to classify data into a reasonable outcome based on a set of features. Some advantages of using Decision Tree include fast computing speed, easy interpretation, identification of important features, and ability to accommodate missing data. However, outcomes from Decision Tree models can have high variability in performance. Decision Tree is also considered as a fundamental component of Random Forest which is described later in this section. Configurations of Decision Tree were set in R programming language as “classification tree” option and “prediction” function. The calculated classification error rate represents the testing accuracy of the developed Decision Tree model in this study.

4.1.2. Support Vector Machine (SVM)

SVM is another type of supervised learning algorithm commonly used to classify features in a dataset into categories that use either a linear or non-linear model. SVM can be used to generate multiple separating classes where data points are split into particular segments that include only one type of data. SVM works by drawing a decision boundary to classify data into reasonable categories. In this study, SVM model was created by using a commonly used Caret package in R programming language. Configurations of SVM in this study include “type” that was set as “c-classification” and the “kernel function” for the linearity of the classifier that was set as “linear” for the prediction function.

4.1.3. K-Nearest Neighbor (KNN)

KNN, a supervised non-linear classification algorithm (i.e., no assumption about distributions of data), is used to classify data into a number of classes. The use of KNN algorithm heavily depends on the determination of the k-value. The KNN works by (1) defining the number K of classes by the user, (2) identifying the K-nearest neighbor of unknown data point based on the distance, (3) determining the number of data points in each class, and (4) assigning the data points to a class that has the most neighbors. Configurations of KNN in this study include the use of Euclidean Distance, the pre-defined k-value, and classification type. In the specific case of KNN, and in order to select the best value of k (i.e., the number of data points nearest to each data point used for majority voting), a comparison between twenty-seven outcomes of K values was conducted. As a result, K = 1 could provide the most accurate classification outcome, as shown in Figure 3.

4.1.4. Random Forest

Random Forest is defined as an average of multiple deep Decision Trees in order to overcome the over-fitting problem of individual Decision Tree. It can be used for both classification and regression models based on the development of various Decision Trees. Despite the fact that Random Forest can address the shortcoming of overfitting in Decision Tree models, it shows a difficulty in interpretability. Random Forest works by aggregating classification outcomes/predictions made by multiple Decision Trees. Each tree in the forest is trained on a subset of the bootstrapped dataset. Configurations of Random Forest in this study include classification type and 500 trees with 2 variables tried at each split.

4.2. Model Development

In this study, weight, and H as well as the maximum value, minimum, mean, median of modified FFT, and standard deviation for each test were initially selected as the features for training the machine learning classification models based on the state-of-the-art research in this area. To avoid having correlations among the selected features, Pearson correlation tests were performed for all three time-segmentations. Consequently, the maximum and mean features were found to be correlated with other features in a statistically significant way as shown in Figure 4. Specifically, the mean feature was statistically correlated with the median, standard deviation, and maximum values with the correlation coefficients of 0.99, 0.98, and 0.90, respectively, at the 95% confidence level. The maximum value was also statistically correlated with the standard deviation, mean, and median with the correlation coefficients of 0.95, 0.90, and 0.85, respectively, at the 95% confidence level. Therefore, a conservative approach was taken to remove the two features and finalize the features used for machine learning classification models as weight, H, minimum, median, and standard deviation.
The study was also designed to examine the sensitivity of the models to various window sizes for feature extraction. In particular, three classification scenarios were conducted with 1-s window sizes segmentation (total data points n = 425), 0.5-s window sizes segmentation (n = 845), and 0.25-s window sizes segmentation (n = 1688) with n is the number of data points in each window size choice. For example, in the 1-s segmentation, the data were extracted every 1 s, resulting in a total of 425 data points whereas, in the 0.5-s segmentation, the data were extracted every 0.5 s, resulting in 845 data points. In addition, this study utilized a 50% overlap between adjacent time segmentations to ensure the consideration of all value-adding data points that might be split into two consecutive segmentations, as suggested by [5]. Specifically, the second half of a time segmentation is overlapped with the first half of its successive time segmentation. For instance, there were 213 data points from the 1-s segmentation overlapped with the data in the 0.5-s segmentation. In this design, three risk classes, including “Nominal Risk”, “Increased Risk”, and “High Risk”, have been considered as the output or labels for the classification models.

4.3. Results

To ensure the generalization of the developed machine learning models, the sEMG signal data were split into training (75%) and testing sets (25%). The training dataset was used to formulate parameters (or features) of the machine learning models (i.e., design the classification engines). The testing dataset was used to calculate the accuracy rate of correctly classified data points that were kept out during training (i.e., assess reliability of the classified results). To further enhance generalizability of the developed models and avoid overfitting, a 10-fold cross-validation was used in this study to provide validation outcomes of 10 replications of the training and testing datasets. The cross-validation approach enhances generalizability since 10 combinations of each dataset were run and replaced iteratively with a low variation of 2% between 10 runs. Furthermore, prior to the classification process, analysis of correlations between model features was conducted to ensure that the choice of the features in subsequent machine learning classification is valid. As a result, the sensitivity to noise of the developed machine learning models become less. After 10 iterations of both datasets, the average classification accuracies were recorded in Table 4.
Considering the results in Table 4, the Decision Tree algorithms manifested the best performance for classifying lifting activities based on the risk assessment using the NIOSH equations across two time-segmentations: 1-s (i.e., 99.05% accuracy), 0.5-s (i.e., 99.98% accuracy), and 0.25-s (i.e., 99.96% accuracy). Table 5 provides the confusion matrix of the classification results using the Decision Tree across three time-segmentations. The only one misclassification found was “Increased Risk” to be “High Risk” across the three window sizes. This might be due to the high LI values (from 1.2 to 2.4) in the definition of the “Increased Risk” category compared with the “High Risk” category including LI values above 2.8. In other time segmentations (0.5- and 0.25-s), no classification error was found, indicating the effectiveness of using Decision Tree in analyzing sEMG signal data.
In the 0.5-s segmentation, KNN provided the least accurate classification result (99.05%) compared to the other three methods: Decision Tree (99.98%), SVM (99.97%), and Random Forest (99.97%). The “Increased Risk” class had the highest misclassification rate compared to the “Nominal Risk” and “High Risk” classes in all three window size segmentation cases.

5. Discussion

Results indicate that the level of physical stress and WMSD risks represented by the NIOSH Lifting Index can be detected using machine learning models trained with sEMG sensor data. LI value below 1.2 as characterized in the first 19 experiments correspond to an average peak value of around 150 µV and 165 µV (time domain) and 20 µV and 21 µV (frequency domain). The higher values of weight and correspondingly LI are associated with higher values of average peak with the highest being 250 µV (time domain) and 33.56 µV (frequency domain). Therefore, this correlation can be leveraged to determine the risk level of lifting task according to the NIOSH lifting equation and using machine learning models. This relationship is depicted in the graph shown in Figure 5, where the sEMG values of the TH are plotted against the LI and risk levels are color-coded.
The results of machine learning classification show that the decision tree model tends to provide the most accurate classification results of the collected EMG signal data using the NIOSH lifting equation. In addition, the “Increased Risk” class has the highest misclassification error with the “High Risk” class. While the classification testing accuracy at 0.5-s segment size reaches its highest value, the 0.25- and 1-s ones show relatively comparable results. The choice of the segmentation size dictates the level of information packed in one window but also affects the number of data points used to train machine learning models. Across three time-segmentations, the results using KNN model showed the highest misclassification error compared the other three models (i.e., lowest overall accuracy rate). The testing accuracies of KNN model were recorded from 99.05% (in 1-s segmentation) to 98.58% (in 0.25-s segmentation). To rule out the possibility of overfitting in the case of KNN with K = 1, another test with K = 5 (which showed the second highest accuracy in Figure 3) was conducted. According to Table 6, KNN with K = 5 accuracies fail to outperform those of the model with K = 1, so it is highly unlikely that overfitting has been an issue in this case.

6. Conclusions

Appropriate body posture in heavy work, particularly during material handling activities, can prevent the risk of WMSDs, minimize days away from work, and consequently improve the job productivity and safety records. In this study, the surface electromyogram (sEMG) sensors are employed to enable the automatic detection of harmful lifting activities using NIOSH lifting equation. A total number of 54 simple lifting tests have been conducted using adjustable weight in six categories with various three risk levels defined based on the NIOSH lifting equation. Results show that the level of risk characterized by the NIOSH Lifting Index and measured by the pressure on the upper back (i.e., Thoracic or TH) muscles using the sEMG sensors can be determined using machine learning models. Towards this goal, four machine learning classification models (i.e., SVM, KNN, Decision Tree, and Random Forest) have been trained using the sEMG data points and risk levels. Decision Tree algorithm resulted in the highest test accuracy level of 99.05%, 99.98%, and 99.96% across three time-segmentations of 1-s, 0.5-s, and 0.25-s, respectively. The level of physical stress and WMSD risks associated with lifting tasks are represented by the NIOSH Lifting Index. LI value below 1.0 denotes a Normal Risk to a healthy employee. As the LI increases, the level of associated risk increases correspondingly, and once the LI value exceeds 3.0 it represents a higher risk associated with the task. sEMG amplitude difference related to a single muscle signifies the varying level of muscle activities.
This study contributes to the body of knowledge and practice in industries such as construction where lifting is a major part of manual material handling. It investigates ergonomic postural hazards using sEMG sensors and enables classification of risk based on the lifting equation developed by NIOSH using four different machine learning methods. The classification results confirmed that there are distinguishable patterns in the sEMG signal data based on the weight in a lifting activity, which proves useful to provide insights regarding recognition of ergonomically hazardous body postures. This study, however, is limited since the experiments were conducted by only one person. Future work should increase the number of participants from which data is collected and conduct subject-dependent and subject-independent tests to evaluate generalizability of the results. Models that are more generic in their function can then be obtained to develop applications that are ready to deploy in the field. Deep learning models will be implemented using a larger dataset of sEMG signals to leverage the outcomes reported from the developed machine learning models and evaluate more sophisticated algorithms.

Author Contributions

Conceptualization, R.A. and S.E.M.; methodology, R.A. and S.E.M.; software, P.H.D.N., M.S.R.; validation, M.S.R. and P.H.D.N.; formal analysis, P.H.D.N.; investigation, S.E.M. and M.S.R.; resources, R.A.; data curation, P.H.D.N.; writing—original draft preparation, S.E.M. and R.A.; writing—review and editing, P.H.D.N. and R.A.; visualization, S.E.M. and P.H.D.N.; supervision, R.A.; project administration, R.A.; funding acquisition, R.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Ethical review and approval were waived for this study, due to the absence of Personal Identifiable Information (PII) in data, collected from a member of the research team.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy issue.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Armstrong, T.J.; Buckle, P.; Fine, L.J.; Hagberg, M.; Jonsson, B.; Kilbom, A.; Kuorinka, I.A.; Silverstein, B.A.; Sjogaard, G.; Viikari-Juntura, E.R. A conceptual model for work-related neck and upper-limb musculoskeletal disorders. Scand. J. Work. Environ. Health 1993, 19, 73–84. [Google Scholar] [CrossRef] [PubMed]
  2. Alireza Ahankoob, A.A. Mitigating Ergonomic Injuries in Construction Industry. IOSR J. Mech. Civ. Eng. 2013, 6, 36–42. [Google Scholar] [CrossRef]
  3. Akhavian, R.; Behzadan, A.H. Simulation-based evaluation of fuel consumption in heavy construction projects by monitoring equipment idle times. In Proceedings of the 2013 Winter Simulations Conference (WSC), Washington, DC, USA, 8–11 December 2013; pp. 3098–3108. [Google Scholar]
  4. Mansouri, S.; Castronovo, F.; Akhavian, R. Analysis of the synergistic effect of data analytics and technology trends in the AEC/FM industry. J. Constr. Eng. Manag. 2020, 146, 04019113. [Google Scholar] [CrossRef]
  5. Akhavian, R.; Brito, L.; Behzadan, A. Integrated mobile sensor-based activity recognition of construction equipment and human crews. In Proceedings of the 2015 Conference on Autonomous and Robotic Construction of Infrastructure, Ames, IA, USA, 2–3 June 2015; pp. 1–20. [Google Scholar]
  6. Waters, T.R.; Putz-Anderson, V.; Garg, A. Application Manual for the Revised NIOSH Equation; Department of Health and Human Services: Cincinnati, OH, USA, 1994.
  7. Waters, T.R. National efforts to identify research issues related to prevention of work-related musculoskeletal disorders. J. Electromyogr. Kinesiol. 2004, 14, 7–12. [Google Scholar] [CrossRef]
  8. Jia, B.; Kim, S.; Nussbaum, M.A. An EMG-based model to estimate lumbar muscle forces and spinal loads during complex, high-effort tasks: Development and application to residential construction using prefabricated walls. Int. J. Ind. Ergon. 2011, 41, 437–446. [Google Scholar] [CrossRef]
  9. Pascual, S.A.; Naqvi, S. An Investigation of Ergonomics Analysis Tools Used in Industry in the Identification of Work-Related Musculoskeletal Disorders. Int. J. Occup. Saf. Ergon. 2008, 14, 237–245. [Google Scholar] [CrossRef] [Green Version]
  10. Schoenmarklin, R.W.; Marras, W.S.; Leurgans, S.E. Rapid Communication Industrial wrist motions and incidence of hand/wrist cumulative trauma disorders. Ergonomics 1994, 37, 1449–1459. [Google Scholar] [CrossRef]
  11. Roberts, D.; Torres Calderon, W.; Tang, S.; Golparvar-Fard, M. Vision-Based Construction Worker Activity Analysis Informed by Body Posture. J. Comput. Civ. Eng. 2020, 34, 04020017. [Google Scholar] [CrossRef]
  12. Nath, N.D.; Akhavian, R.; Behzadan, A.H. Ergonomic analysis of construction worker’s body postures using wearable mobile sensors. Appl. Ergon. 2017, 62, 107–117. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Ahn, C.R.; Lee, S.; Sun, C.; Jebelli, H.; Yang, K.; Choi, B. Wearable Sensing Technology Applications in Construction Safety and Health. J. Constr. Eng. Manag. 2019, 145, 03119007. [Google Scholar] [CrossRef]
  14. Bangaru, S.S.; Wang, C.; Aghazadeh, F. Data Quality and Reliability Assessment of Wearable EMG and IMU Sensor for Construction Activity Recognition. Sensors 2020, 20, 5264. [Google Scholar] [CrossRef]
  15. Valero, E.; Sivanathan, A.; Bosché, F.; Abdel-Wahab, M. Musculoskeletal disorders in construction: A review and a novel system for activity tracking with body area network. Appl. Ergon. 2016, 54, 120–130. [Google Scholar] [CrossRef] [PubMed]
  16. Akhavian, R.; Behzadan, A.H. Smartphone-based construction workers’ activity recognition and classification. Autom. Constr. 2016, 71, 198–209. [Google Scholar] [CrossRef]
  17. Bayat, A.; Pomplun, M.; Tran, D.A. A Study on Human Activity Recognition Using Accelerometer Data from Smartphones. Procedia Comput. Sci. 2014, 34, 450–457. [Google Scholar] [CrossRef] [Green Version]
  18. Sherafat, B.; Ahn, C.R.; Akhavian, R.; Behzadan, A.H.; Golparvar-Fard, M.; Kim, H.; Lee, Y.-C.; Rashidi, A.; Azar, E.R. Automated Methods for Activity Recognition of Construction Workers and Equipment: State-of-the-Art Review. J. Constr. Eng. Manag. 2020, 146, 03120002. [Google Scholar] [CrossRef]
  19. Yang, Z.; Yuan, Y.; Zhang, M.; Zhao, X.; Tian, B. Assessment of Construction Workers’ Labor Intensity Based on Wearable Smartphone System. J. Constr. Eng. Manag. 2019, 145, 04019039. [Google Scholar] [CrossRef]
  20. Joshua, L.; Varghese, K. Automated recognition of construction labour activity using accelerometers in field situations. Int. J. Prod. Perform. Manag. 2014, 63, 841–862. [Google Scholar] [CrossRef]
  21. Khan, S.H.; Sohail, M. Activity monitoring of workers using single wearable inertial sensor. In Proceedings of the 2013 International Conference on Open Source Systems and Technologies, Lahore, Pakistan, 16–18 December 2013; pp. 60–67. [Google Scholar] [CrossRef]
  22. Ryu, J.; Seo, J.; Jebelli, H.; Lee, S. Automated Action Recognition Using an Accelerometer-Embedded Wristband-Type Activity Tracker. J. Constr. Eng. Manag. 2019, 145, 04018114. [Google Scholar] [CrossRef]
  23. Yang, K.; Aria, S.; Ahn, C.R.; Stentz, T.L. Automated Detection of Near-miss Fall Incidents in Iron Workers Using Inertial Measurement Units. In Proceedings of the Construction Research Congress 2014, Atlanta, GA, USA, 19–21 May 2014; American Society of Civil Engineers: Reston, VA, USA, 2014; pp. 935–944. [Google Scholar] [CrossRef]
  24. Jahanbanifar, S.; Akhavian, R. Evaluation of wearable sensors to quantify construction workers muscle force: An ergonomic analysis. In Proceedings of the 2018 Winter Simulation Conference (WSC), Gothenburg, Sweden, 9–12 December 2018; pp. 3921–3929. [Google Scholar] [CrossRef]
  25. Yan, X.; Li, H.; Li, A.R.; Zhang, H. Wearable IMU-based real-time motion warning system for construction workers’ musculoskeletal disorders prevention. Autom. Constr. 2017, 74, 2–11. [Google Scholar] [CrossRef]
  26. Awolusi, I.; Marks, E.; Hallowell, M. Wearable technology for personalized construction safety monitoring and trending: Review of applicable devices. Autom. Constr. 2018, 85, 96–106. [Google Scholar] [CrossRef]
  27. Nimbarte, A.D.; Aghazadeh, F.; Ikuma, L.H.; Harvey, C.M. Neck Disorders among Construction Workers: Understanding the Physical Loads on the Cervical Spine during Static Lifting Tasks. Ind. Health 2010, 48, 145–153. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Matsumura, Y.; Mitsukura, Y.; Fukumi, M.; Akamatsu, N.; Yamamoto, Y.; Nakaura, K. Recognition of EMG signal patterns by neural networks. In Proceedings of the 9th International Conference on Neural Information Processing (ICONIP ’02), Singapore, 18–22 November 2002; Volume 2, pp. 750–754. [Google Scholar] [CrossRef]
  29. Jebelli, H.; Lee, S. Feasibility of Wearable Electromyography (EMG) to Assess Construction Workers’ Muscle Fatigue. In Advances in Informatics and Computing in Civil and Construction Engineering; Springer International Publishing: Cham, Switzerland, 2019; pp. 181–187. [Google Scholar] [CrossRef]
Figure 1. Description of the variables used to calculate NIOSH lifting equation multipliers.
Figure 1. Description of the variables used to calculate NIOSH lifting equation multipliers.
Electronics 10 02558 g001
Figure 2. (a) Thoracic (upper-back) and Multifidus (lower-back) EMG-sensor positions. (b) The research team member lifting the weights.
Figure 2. (a) Thoracic (upper-back) and Multifidus (lower-back) EMG-sensor positions. (b) The research team member lifting the weights.
Electronics 10 02558 g002
Figure 3. Selection of appropriate k value for KNN classification.
Figure 3. Selection of appropriate k value for KNN classification.
Electronics 10 02558 g003
Figure 4. Results of Pearson correlation test among model features.
Figure 4. Results of Pearson correlation test among model features.
Electronics 10 02558 g004
Figure 5. Fourier transformed averages of peak values (µV) on horizontal axis versus the LI on vertical axis.
Figure 5. Fourier transformed averages of peak values (µV) on horizontal axis versus the LI on vertical axis.
Electronics 10 02558 g005
Table 1. NIOSH Lifting Equation Multipliers.
Table 1. NIOSH Lifting Equation Multipliers.
UnitLoad Constant (LC)Horizontal Multiplier (HM)Vertical Multiplier (VM)Distance Multiplier (DM)Asymmetric Multiplier (AM)Frequency Multiplier (FM)Coupling Multiplier (CM)
Metric23 kg(25/H)1 − (0.003|V − 75|)0.82 + (4.5/D)1 − (0.0032A)12
U.S. FPS 51 lb(10/H)1 − (0.0075|V − 30|)0.82 + (1.8/D)1 − (0.0032A)12
1 determined based on the frequency of lifts and the vertical distance (V). 2 determined based on the coupling quality and the vertical distance (V).
Table 2. NIOSH Lifting Equation Multipliers According to Table 1 and Figure 1.
Table 2. NIOSH Lifting Equation Multipliers According to Table 1 and Figure 1.
Origin—Lifting the weightHVDACFDuration
15″14″18″0Good10/min1 h/day
HMVMDMAMCMFM
0.670.880.92110.45
Destination—Placing the weightHVDACFDuration
24″32″18″0Good10/min1 h/day
HMVMDMAMCMFM
0.420.990.92110.45
Table 3. Variable Weights used in the Experiments to Calculate the RWL and LI.
Table 3. Variable Weights used in the Experiments to Calculate the RWL and LI.
ExperimentsLoad (LBS)HRWLLIRisk
1–9101512.400.8Nominal Risk
10–19151512.401.2Nominal Risk
20–29201512.401.6Increased Risk
30–39301512.402.4Increased Risk
40–49351512.402.8High Risk
50–54351711.003.2High Risk
Table 4. Classification testing accuracy (%) of machine learning models.
Table 4. Classification testing accuracy (%) of machine learning models.
Time SegmentationDecision TreeSVMKNNRandom Forest
1-s (n = 425)99.0597.1799.0597.07
0.5-s (n = 845)99.9899.9799.0599.97
0.25-s (n = 1688)99.9699.1098.5899.95
Table 5. Confusion matrix of classification results from Decision Tree.
Table 5. Confusion matrix of classification results from Decision Tree.
Risk Class1-S Segmentation0.5-S Segmentation0.25-S Segmentation
NRIRHRNRIRHRNRIRHR
Nominal Risk (NR)3400680013700
Increased Risk (IR)021104210801
High Risk (HR)00500010000204
Table 6. Classification testing accuracy (%) for KNN with K = 5.
Table 6. Classification testing accuracy (%) for KNN with K = 5.
Time SegmentationKNN
1-s (n = 425)97.16
0.5-s (n = 845)97.63
0.25-s (n = 1688)98.56
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Mudiyanselage, S.E.; Nguyen, P.H.D.; Rajabi, M.S.; Akhavian, R. Automated Workers’ Ergonomic Risk Assessment in Manual Material Handling Using sEMG Wearable Sensors and Machine Learning. Electronics 2021, 10, 2558. https://doi.org/10.3390/electronics10202558

AMA Style

Mudiyanselage SE, Nguyen PHD, Rajabi MS, Akhavian R. Automated Workers’ Ergonomic Risk Assessment in Manual Material Handling Using sEMG Wearable Sensors and Machine Learning. Electronics. 2021; 10(20):2558. https://doi.org/10.3390/electronics10202558

Chicago/Turabian Style

Mudiyanselage, Srimantha E., Phuong Hoang Dat Nguyen, Mohammad Sadra Rajabi, and Reza Akhavian. 2021. "Automated Workers’ Ergonomic Risk Assessment in Manual Material Handling Using sEMG Wearable Sensors and Machine Learning" Electronics 10, no. 20: 2558. https://doi.org/10.3390/electronics10202558

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop