# A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- Multi-modal wireless wearable sensors are used for digitally computing RULA scores.
- A robust kinematic model has been developed for upper limbs motion tracking.
- Digital implementation of the new method has been validated by comparison with an online RULA calculator.

## 2. DULA Method

#### 2.1. RULA

#### 2.2. Kinematics for DULA

#### 2.2.1. Reference Coordinate Setup

- Stretch the arm forward while the palm is facing downwards. This is considered the right arm base posture.
- Record the gravity vector g at base posture, upon which the $\hat{{z}_{{F}^{*}}}$ for the forearm frame is determined.$$\hat{{z}_{{F}^{*}}}=-\frac{g}{\parallel g\parallel}.$$
- Perform supination motion and record angular velocity $\omega $ during the motion. The $\hat{{y}_{{F}^{*}}}$ for the forearm frame is hence obtained as:$$\hat{{y}_{{F}^{*}}}=-\frac{\omega}{\parallel \omega \parallel}.$$
- The cross product of $\hat{{y}_{{F}^{*}}}$ and $\hat{{z}_{{F}^{*}}}$ results in ${x}_{{F}^{*}}$ for the forearm frame. However, it is not humanly possible to have $\hat{{y}_{{F}^{*}}}$ and $\hat{{z}_{{F}^{*}}}$ exactly perpendicular to each other, hence, ${x}_{{F}^{*}}$ is normalized.$${x}_{{F}^{*}}=\hat{{y}_{{F}^{*}}}\times \hat{{z}_{{F}^{*}}},$$$$\hat{{x}_{{F}^{*}}}=\frac{{x}_{{F}^{*}}}{\parallel {x}_{{F}^{*}}\parallel}.$$
- Finally, $\hat{{z}_{{F}^{*}}}$ is obtained to complete right hand coordinate frame convention.$$\hat{{z}_{{F}^{*}}}=\hat{{x}_{{F}^{*}}}\times \hat{{y}_{{F}^{*}}}.$$
- Thus, the computed rotation matrix belongs to a special orthogonal group $SO\left(3\right)$. The forearm reference frame in the IMU frame has the following form:$${}_{I}^{{F}^{*}}\mathbf{R}=\left[\begin{array}{ccc}\hat{{x}_{{F}^{*}}}& \hat{{y}_{{F}^{*}}}& \hat{{z}_{{F}^{*}}}\end{array}\right].$$

#### 2.2.2. Frame Transformations

- Record IMU orientation ${}_{G}^{I}\mathbf{R}$ when the arm is in the base posture.
- ${}_{I}^{{F}^{*}}\mathbf{R}$ is computed according the method in Section 2.2.1.
- ${}_{G}^{F}\mathbf{R}$ is continuously acquired from the IMU sensor in real-time.
- To find the forearm orientation with respect to IMU frame, i.e, ${}_{I}^{F}\mathbf{R}$, the following equations are presented:$${}_{G}^{F}\mathbf{R}={\phantom{\rule{0.277778em}{0ex}}}_{G}^{I}\mathbf{R}{\phantom{\rule{0.277778em}{0ex}}}_{I}^{F}\mathbf{R},$$$${}_{I}^{F}\mathbf{R}=\phantom{\rule{0.277778em}{0ex}}{\left({}_{G}^{I}\mathbf{R}\right)}^{T}{\phantom{\rule{0.277778em}{0ex}}}_{G}^{F}\mathbf{R}.$$
- Finally, we find the orientation of the forearm with respect to the forearm reference frame $\left\{{F}^{*}\right\}$,$${}_{{F}^{*}}^{F}\mathbf{R}=\phantom{\rule{0.277778em}{0ex}}{\left({}_{I}^{{F}^{*}}\mathbf{R}\right)}^{T}{\phantom{\rule{0.277778em}{0ex}}}_{I}^{F}\mathbf{R}{\phantom{\rule{0.277778em}{0ex}}}_{I}^{{F}^{*}}\mathbf{R}.$$

#### 2.2.3. Upper Arm Orientation

#### 2.2.4. Forearm Orientation

- Find the orientations of upper arm base frame $\left\{{U}^{*}\right\}$ and forearm base frame $\left\{{F}^{*}\right\}$ with respect to global reference frame $\left\{G\right\}$ as:$${}_{G}^{{F}^{*}}\mathbf{R}={\phantom{\rule{0.277778em}{0ex}}}_{G}^{{I}_{f}}\mathbf{R}{\phantom{\rule{0.277778em}{0ex}}}_{{I}_{f}}^{{F}^{*}}\mathbf{R},$$$${}_{G}^{{U}^{*}}\mathbf{R}={\phantom{\rule{0.277778em}{0ex}}}_{G}^{{I}_{u}}\mathbf{R}{\phantom{\rule{0.277778em}{0ex}}}_{{I}_{u}}^{{U}^{*}}\mathbf{R},$$
- Compute the relative rotation ${\mathbf{R}}_{1}$ between both frames $\left\{{F}^{*}\right\}$ and $\left\{{G}^{*}\right\}$:$${}_{G}^{{F}^{*}}\mathbf{R}\phantom{\rule{0.277778em}{0ex}}{\mathbf{R}}_{1}={\phantom{\rule{0.277778em}{0ex}}}_{G}^{{U}^{*}}\mathbf{R},$$$${\mathbf{R}}_{1}={\left({}_{G}^{{F}^{*}}\mathbf{R}\right)}^{T}{\phantom{\rule{0.277778em}{0ex}}}_{G}^{{U}^{*}}\mathbf{R}.$$
- The forearm orientation $\left\{F\right\}$ defined in $\left\{{F}^{*}\right\}$ is mapped in the upper arm reference frame $\left\{{U}^{*}\right\}$ using similarity transformation technique:$${}_{{U}^{*}}^{F}\mathbf{R}={\mathbf{R}}_{1}^{T}{\phantom{\rule{0.277778em}{0ex}}}_{{F}^{*}}^{F}\mathbf{R}\phantom{\rule{0.277778em}{0ex}}{\mathbf{R}}_{1}.$$
- Since both forearm and upper arm orientations are now established in the same coordinate system defined by $\left\{{U}^{*}\right\}$, the relative rotation matrix ${\mathbf{R}}_{2}$ is found as:$${}_{{U}^{*}}^{U}\mathbf{R}\phantom{\rule{0.277778em}{0ex}}{\mathbf{R}}_{2}{=}_{{U}^{*}}^{F}\mathbf{R},$$$${\mathbf{R}}_{2}={\left({}_{{U}^{*}}^{U}\mathbf{R}\right)}^{T}{\phantom{\rule{0.277778em}{0ex}}}_{{U}^{*}}^{F}\mathbf{R}.$$
- The forearm orientation with respect to the upper arm is considered to be formed by the combination of two rotations, the forearm extension/flexion ${\alpha}_{f}$ about the current Y, and the forearm pronation/supination ${\beta}_{f}$ about the current Z axis.$${\mathbf{R}}_{2}={\mathbf{R}}_{{YZ}^{\prime}}=\phantom{\rule{0.277778em}{0ex}}{\mathbf{R}}_{Y}\left({\alpha}_{f}\right)\phantom{\rule{0.277778em}{0ex}}{\mathbf{R}}_{{Z}^{\prime}}\left({\beta}_{f}\right)=\left[\begin{array}{ccc}{s}_{11}& {s}_{12}& {s}_{13}\\ {s}_{21}& {s}_{22}& {s}_{23}\\ {s}_{31}& {s}_{32}& {s}_{33}\end{array}\right].$$$${\alpha}_{f}=arctan\left({s}_{21},{s}_{22}\right),$$$${\beta}_{f}=arctan\left({s}_{13},{s}_{33}\right).$$

#### 2.3. Load Identification

## 3. DULA System Development

#### 3.1. Hardware

#### 3.2. Software

## 4. Results

## 5. Discussion

## 6. Conclusions

## Author Contributions

## Funding

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

MSD | Musculoskeletal disorders |

RULA | Rapid upper limb assessment |

DULA | Digital upper limb assessment |

IMU | Inertial measurement unit |

F or f | Forearm |

U or f | Upper arm |

## References

- Winkel, J.; Mathiassen, S.E. Assessment of physical work load in epidemiologic studies: Concepts, issues and operational considerations. Ergonomics
**1994**, 37, 979–988. [Google Scholar] [CrossRef] - Dockrell, S.; O’Grady, E.; Bennett, K.; Mullarkey, C.; Mc Connell, R.; Ruddy, R.; Twomey, S.; Flannery, C. An investigation of the reliability of Rapid Upper Limb Assessment (RULA) as a method of assessment of children’s computing posture. Appl. Ergon.
**2012**, 43, 632–636. [Google Scholar] [CrossRef] [PubMed] - Lei, L.; Dempsey, P.G.; Xu, J.G.; Ge, L.N.; Liang, Y.X. Risk factors for the prevalence of musculoskeletal disorders among Chinese foundry workers. Int. J. Ind. Ergon.
**2005**, 35, 197–204. [Google Scholar] [CrossRef] - Parent-Thirion, A.; Biletta, I.; Cabrita, J.; Vargas Llave, O.; Vermeylen, G.; Wilczyńska, A.; Wilkens, M. Sixth European Working Conditions Survey–Overview report. Eurofound
**2016**. [Google Scholar] [CrossRef] - Hignett, S.; McAtamney, L. Rapid entire body assessment (REBA). Appl. Ergon.
**2000**, 31, 201–205. [Google Scholar] [CrossRef] - Sanchez-Lite, A.; Garcia, M.; Domingo, R.; Angel Sebastian, M. Novel ergonomic postural assessment method (NERPA) using product-process computer aided engineering for ergonomic workplace design. PLoS ONE
**2013**, 8, e72703. [Google Scholar] [CrossRef] - Moore, J.S.; Vos, G.A. The strain index. In Handbook of Human Factors and Ergonomics Methods; CRC Press: Boca Raton, FL, USA, 2004; pp. 109–114. [Google Scholar]
- Karhu, O.; Kansi, P.; Kuorinka, I. Correcting working postures in industry: A practical method for analysis. Appl. Ergon.
**1977**, 8, 199–201. [Google Scholar] [CrossRef] - McAtamney, L.; Corlett, E.N. RULA: A survey method for the investigation of work-related upper limb disorders. Appl. Ergon.
**1993**, 24, 91–99. [Google Scholar] [CrossRef] - Basahel, A.M. Investigation of work-related musculoskeletal disorders (MSDs) in warehouse workers in Saudi Arabia. Procedia Manuf.
**2015**, 3, 4643–4649. [Google Scholar] [CrossRef] - Bao, S.; Howard, N.; Spielholz, P.; Silverstein, B. Two posture analysis approaches and their application in a modified rapid upper limb assessment evaluation. Ergonomics
**2007**, 50, 2118–2136. [Google Scholar] [CrossRef] - Intranuovo, G.; De Maria, L.; Facchini, F.; Giustiniano, A.; Caputi, A.; Birtolo, F.; Vimercati, L. Risk assessment of upper limbs repetitive movements in a fish industry. BMC Res. Notes
**2019**, 12, 1–7. [Google Scholar] [CrossRef] [PubMed] - Valentim, D.P.; de Oliveira Sato, T.; Comper, M.L.C.; da Silva, A.M.; Boas, C.V.; Padula, R.S. Reliability, construct validity and interpretability of the Brazilian version of the Rapid Upper Limb Assessment (RULA) and Strain Index (SI). Braz. J. Phys. Ther.
**2018**, 22, 198–204. [Google Scholar] [CrossRef] [PubMed] - Balasubramanian, V.; Swami Prasad, G. An EMG-based ergonomic evaluation of manual bar bending. Int. J. Ind. Syst. Eng.
**2007**, 2, 299–310. [Google Scholar] [CrossRef] - González, A.; Barrios-Muriel, J.; Romero-Sánchez, F.; Salgado, D.; Alonso, F. Ergonomic assessment of a new hand tool design for laparoscopic surgery based on surgeons’ muscular activity. Appl. Ergon.
**2020**, 88, 103161. [Google Scholar] [CrossRef] - Kong, Y.K.; Choi, K.H.; Cho, M.U.; Kim, S.Y.; Kim, M.J.; Shim, J.W.; Park, S.S.; Kim, K.R.; Seo, M.T.; Chae, H.S.; et al. Ergonomic assessment of a lower-limb exoskeleton through electromyography and Anybody modeling system. Int. J. Environ. Res. Public Health
**2022**, 19, 8088. [Google Scholar] [CrossRef] - Huang, C.; Kim, W.; Zhang, Y.; Xiong, S. Development and validation of a wearable inertial sensors-based automated system for assessing work-related musculoskeletal disorders in the workspace. Int. J. Environ. Res. Public Health
**2020**, 17, 6050. [Google Scholar] [CrossRef] - Moore, J.; Stuart, S.; McMeekin, P.; Walker, R.; Celik, Y.; Pointon, M.; Godfrey, A. Enhancing free-living fall risk assessment: Contextualizing mobility based IMU data. Sensors
**2023**, 23, 891. [Google Scholar] [CrossRef] - Akhmad, S.; Arendra, A.; Findiastuti, W.; Lumintu, I.; Pramudita, Y.D. Wearable IMU Wireless Sensors Network for Smart Instrument of Ergonomic Risk Assessment. In Proceedings of the 6th Information Technology International Seminar (ITIS), Surabaya, Indonesia, 14–16 October 2020; pp. 213–218. [Google Scholar]
- Mohamad, D.; Md Deros, B.; Ismail, A.R.; Daruis, D.D.I.; Sukadarin, E.H. RULA analysis of work-related disorder among packaging industry worker using digital human modeling (DHM). In Advanced Engineering Forum; Trans Tech Publications Ltd.: Kuala Lumpur, Malaysia, 2013; Volume 10, pp. 9–15. [Google Scholar]
- Hussain, M.M.; Qutubuddin, S.; Kumar, K.P.R.; Reddy, C.K. Digital human modeling in ergonomic risk assessment of working postures using RULA. In Proceedings of the International Conference on Industrial Engineering and Operations Management, Bangkok, Thailand, 5–7 March 2019; pp. 5–7. [Google Scholar]
- Hu, B.; Ma, L.; Zhang, W.; Salvendy, G.; Chablat, D.; Bennis, F. Predicting real-world ergonomic measurements by simulation in a virtual environment. Int. J. Ind. Ergon.
**2011**, 41, 64–71. [Google Scholar] [CrossRef] - Cao, W.; Zhong, J.; Cao, G.; He, Z. Physiological Function Assessment Based on Kinect V2. IEEE Access
**2019**, 7, 105638–105651. [Google Scholar] [CrossRef] - Yu, Y.; Yang, X.; Li, H.; Luo, X.; Guo, H.; Fang, Q. Joint-level vision-based ergonomic assessment tool for construction workers. J. Constr. Eng. Manag.
**2019**, 145, 04019025. [Google Scholar] [CrossRef] - Forsman, M.; Hansson, G.Å.; Medbo, L.; Asterland, P.; Engström, T. A method for evaluation of manual work using synchronised video recordings and physiological measurements. Appl. Ergon.
**2002**, 33, 533–540. [Google Scholar] [CrossRef] [PubMed] - David, G.C. Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occup. Med.
**2005**, 55, 190–199. [Google Scholar] [CrossRef] [PubMed] - Cheng, H.Y.K.; Cheng, C.Y.; Ju, Y.Y. Work-related musculoskeletal disorders and ergonomic risk factors in early intervention educators. Appl. Ergon.
**2013**, 44, 134–141. [Google Scholar] [CrossRef] [PubMed] - McDevitt, S.; Hernandez, H.; Hicks, J.; Lowell, R.; Bentahaikt, H.; Burch, R.; Ball, J.; Chander, H.; Freeman, C.; Taylor, C.; et al. Wearables for biomechanical performance optimization and risk assessment in industrial and sports applications. Bioengineering
**2022**, 9, 33. [Google Scholar] [CrossRef] - Padilla, B.E.O.; Glushkova, A.; Menychtas, D.; Manitsaris, S. Designing a web-based automatic ergonomic assessment using motion data. In Proceedings of the 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments, Rhodes, Greece, 5–7 June 2019; pp. 528–534. [Google Scholar]
- Balser, F.; Desai, R.; Ekizoglou, A.; Bai, S. A novel passive shoulder exoskeleton designed with variable stiffness mechanism. IEEE Robot. Autom. Lett.
**2022**, 7, 2748–2754. [Google Scholar] [CrossRef] - Islam, M.; Bai, S. Payload estimation using forcemyography sensors for control of upper-body exoskeleton in load carrying assistance. Model. Identif. Control (Online)
**2019**, 40, 189–198. [Google Scholar] [CrossRef] - Tahir, A.; An, Z.; Bai, S.; Shen, M. Robust payload recognition based on sensor-over-muscle-independence deep learning for the control of exoskeletons. IEEE Trans. Circuits Syst. II Express Briefs
**2023**. [Google Scholar] [CrossRef] - Ergonomics, O. RULA—Rapid Upper Limb Assessment. 2019. Available online: https://www.rula.co.uk/assessment.html (accessed on 12 May 2023).

**Figure 1.**DULA diagram for ergonomic assessment. (

**a**) Upper arm postures and corresponding scores, (

**b**) forearm postures and corresponding scores, (

**c**) load being lifted and corresponding scores, (

**d**) wearable sensor band.

**Figure 2.**Reference coordinate frames setup for the forearm $\left\{{F}^{*}\right\}$ and the upper arm $\left\{{U}^{*}\right\}$.

**Figure 3.**Frame setup for the kinematics of arm motion. The coordinate frames are shown in curly braces, whereas rotation matrices adjacent to arrows connecting two frames represent their transformation. $\left\{G\right\}$ is the global reference frame. $\left\{{F}^{*}\right\}$ and $\left\{I\right\}$ are the forearm reference frame and IMU frame, respectively, and they are attached to the initial configuration of the forearm. Frame $\left\{F\right\}$ moves with the forearm.

**Figure 4.**Moving from the upper arm frame to the forearm frame, when forearm orientation is already defined with respect to frame $\left\{{U}^{*}\right\}$.

Reference | Assessment Tool | Data Acquisition Method | Wearable | Non-Obstructive | Load Identification |
---|---|---|---|---|---|

[10,11,12,13,23,24] | RULA/Modified RULA/SI | Optical/Self-report | No | No | No |

[14,15,16] | Muscle activity level | EMG/AnyBody software | Yes | Yes | Yes |

[20,21,22] | RULA | DHM/VR/CATIA software | Yes | No | No |

[17,19] | RULA/REBA/NIOSH | Inertial | Yes | Yes | No |

[18] | Artificial Intelligence based | Inertial and optical | Yes | No | No |

Our work | RULA | Inertial | Yes | Yes | Yes |

RULA Score | Action Level | Action |
---|---|---|

1–2 | 1 | Acceptable working pattern; no changes are required. |

3–4 | 2 | Working pattern may be changed; hence, further investigations are suggested. |

5–6 | 3 | Working pattern will soon require changes; therefore, further investigations are needed. |

>7 | 4 | Working pattern highly risks health and should be immediately changed. |

Posture | DULA | RULA | Posture | DULA | RULA |
---|---|---|---|---|---|

1 | 6 | 6 | 6 | 6 | 6 |

2 | 5 | 5 | 7 | 7 | 6 |

3 | 5 | 4 | 8 | 5 | 5 |

4 | 4 | 4 | 9 | 6 | 6 |

5 | 6 | 5 | 10 | 7 | 7 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Tahir, A.; Bai, S.; Shen, M.
A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation. *Sensors* **2023**, *23*, 4863.
https://doi.org/10.3390/s23104863

**AMA Style**

Tahir A, Bai S, Shen M.
A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation. *Sensors*. 2023; 23(10):4863.
https://doi.org/10.3390/s23104863

**Chicago/Turabian Style**

Tahir, Abdullah, Shaoping Bai, and Ming Shen.
2023. "A Wearable Multi-Modal Digital Upper Limb Assessment System for Automatic Musculoskeletal Risk Evaluation" *Sensors* 23, no. 10: 4863.
https://doi.org/10.3390/s23104863