An Internet of Medical Things-Based Smart Electromyogram Device for Monitoring of Musculoskeletal Disorders ^†

Abstract

Electromyography (EMG) is a technique that measures the electrical activity of the muscles, and it has been used extensively in the field of physiotherapy to assess muscle function and activity. Grading muscle power is an important aspect of assessing muscle function, as it provides information about the strength and endurance of muscles. Presently, physiotherapists use manual muscle testing (MMT) to grade muscle power; however, it requires a therapist with good expertise. In this work, a Smart EMG device based on the Internet of Medical Things (IoMT) is designed and developed for monitoring patients suffering from abnormal musculoskeletal health conditions. EMG signals are acquired from normal, healthy individuals as well as from patients with abnormal health conditions. Muscle power grading is used to grade the EMG signals and a convolutional neural network-based (CNN) deep learning algorithm is utilized to visualize the progress of the course of treatment provided to patients with musculoskeletal problems, such as stroke and spinal cord injuries, among other issues. The entire analysis is performed using the Google Colaboratory IoT cloud platform, and the algorithms are coded using the Python programming language. The results demonstrate that the proposed IoMT-based smart device can predict different muscle power with an average accuracy of 97.5%, which proves the effectiveness of the device. This work appears to be of high clinical relevance, since the proposed device is capable of providing valuable information about muscle function and enables physiotherapists to design personalized treatment plans for patients with musculoskeletal disorders.

1. Introduction

Physiotherapy is one of the treatment procedures for various musculoskeletal disorders and it helps the patients to retain their muscular ability. The physiotherapy process helps to improve mobility of the muscle function as well as to reduce muscle pain. The assessment of muscle function is very tedious by manual muscle testing (MMT), which affects the effectiveness of the physiotherapy process [1]. MMT is a traditional and commonly followed method for grading the power of the muscle in the therapy process, and this muscle power grading depends on the skillset of the therapist. However, a lack of expertise and variability between different physiotherapist leads to an inaccurate assessment of the patient progress [2,3].

Electromyography is a technique that measures the electrical activity of the muscles, and the electrical signals acquired using the electromyography technique are called electromyogram (EMG) signals [4]. EMG gains more significance in various fields, such as precision medicine, telesurgery, and rehabilitation medicine [4,5,6,7,8,9]. In general, two different methods are commonly followed, namely invasive and non-invasive EMG signal acquisition. An invasive method for acquiring EMG requires invasive procedures, such as needle electrodes or catheters, which produces discomfort to the patients; whereas non-invasive methods are less expensive and highly reliable.

The Industrial Internet of Things (IIoT) has made a footprint on various fields, such as smart cities, telemedicine, and product industries. Several core technologies are involved, and remarkable growth has been achieved in healthcare applications [10,11,12]. In addition, huge volumes of IIoT data are used to train various artificial intelligence (AI) models to predicts various events. Furthermore, the hardware graphical processing unit (GPU) from various vendors, such as Neural Compute Stick from Intel and Jetson Nano/Xavier from Nvidia, which enables the capability of the GPU to the edge devices. In addition, the Google and Amazon Web Service enable the capability of the GPU over cloud platform, whereas without owning any hardware one can deploy deep learning/machine learning algorithms with the help of the cloud platform [10].

Recent developments in the field of artificial intelligence enables the diagnosis of various musculoskeletal problems, such as spinal cord injuries and stroke, among other issues. Several researchers have proposed various deep learning and machine learning techniques for disease diagnosis [13,14,15,16,17,18,19,20,21]. Ref. [13] developed a hand/finger gesture classifier based on EMG signals using artificial neural networks (ANNs). The authors made various personalized classifiers other than the CNN, namely support vector machine (SVM), random forest (RF), and logistic regression (LR). The authors concluded that the CNN exhibits more accuracy than SVM, RF, and LR. Ref. [20] proposed a real time system for the identification of plant disease using various deep learning algorithms, namely a CNN, a polynomial deep belief network (PDBN), and a linear deep belief network (LDPN). The authors concluded that the CNN exhibited higher accuracy when compared to other two DBN-based classifiers. Basak et al. (2021) [21] utilized an SVM and an ANN for the classification EMG signals using principal component analysis (PCA) and linear discriminant analysis (LDA). While seeing various studies in the literature, it can be seen that the CNN is good at classifying bio-images and signals, especially EMG.

The objective of this work is to devise an IoMT-based smart EMG device which assists physiotherapists to adjust treatment plans in real-time and provide immediate feedback to patients. This work is arranged as follows: Section 2 presents the materials and methods; Section 3 demonstrates the results and conclusions; and Section 4 presents the conclusion.

2. Methodology

In this study, an IoMT-based EMG wearable device is designed and developed for the monitoring of motor recovery function. In hospitals, the proposed IoMT-based EMG wearable device can be attached to patients undergoing physiotherapy for musculoskeletal problems. The EMG signals from different patients are collected and computed using the IoT cloud platform which satisfies the functionality of the IIoT. For this, the proposed system can be categorized into two different parts, namely an IoMT-based EMG acquisition device and an IoT cloud computing platform.

An IoMT-based EMG acquisition device is wearable hardware which acquires the EMG signals from patients subjected to physiotherapy treatment and uploads the EMG signal to an IoT cloud platform. The IoT cloud platform then performs the EMG signal processing, classifying the EMG signals according to muscle power grading, stores the classification output day-wise for future reference, and sends the recovery progress of the patient through email/short messaging service (SMS) to the physician and the patient.

2.1. IoMT-Based EMG Acquisition Device

Figure 1 shows the overall block diagram of the proposed IoMT-based EMG device. The proposed device consists of an EMG sensor module, an IoT controller, and a power supply battery unit. The proposed device is a wearable IoMT-based EMG acquisition device. Additionally, the components of the proposed device, such as the EMG sensor module, the IoT controller, and the battery unit, are enclosed with the help of an acrylic box to ensure physical protection. The three electrodes of the EMG sensor module shall be placed to the brachioradialis, which is the forearm muscles, and the box shall be tied to the arm with the help of a Velcro strap. Since the proposed device is connected to a Wi-Fi connection and is powered by the battery unit, the device records the EMG of individuals with normal/abnormal muscles, and the in-built Wi-Fi module of the proposed device helps to transmit the recorded EMG signal to a Google spreadsheet via hypertext transfer protocol secure (HTTPS).

2.1.1. EMG Sensor Module

In general, the muscle activity can be monitored using a surface EMG sensor, namely an electromyography sensor. In this work, a MyoWare brand sensor with the product number (SEN-13723) is utilized which is manufactured by Advancer Technologies. The sensor is very compact and operates on the supply voltage ranges from +2.9 V to +5.7 V. The sensor provides output in two different modes, namely raw EMG and EMG envelope. In addition, the sensor module has polarity reversal protection, adjustable gain, and is compatible with various market microcontrollers.

The Ag/AgCl non-invasive electrodes can be fixed directly to the sensor module and the module can be directly placed to the outer skin of the humans. Furthermore, the sensor module has an inbuilt instrumentation amplifier module and a filter to remove unwanted noise present in the acquired EMG signal. The instrumentation amplifier amplifies the acquired analog EMG signals which can be given to the analog pins of the advanced microcontrollers.

2.1.2. Node Microcontroller Unit

The node microcontroller unit (NodeMCU) ESP8266 Wi-Fi module with 802.11 b/g/n is utilized in this work, which is versatile, highly affordable, and flexible.

The NodeMCU has a wide range of built-in libraries, such as MQTT, Wi-Fi, and HTTP, and it acts as an easy platform which supports various development frameworks, namely Node-Red and Platform IO, for connected devices. Figure 2 shows the function of the NodeMCU microcontroller in the proposed IoMT-based EMG device. In this work, the amplified analog EMG signals from the EMG sensor module is fed to the NodeMCU microcontroller. Initially, the microcontroller set up a Wi-Fi connection and it starts acquiring the EMG signal. The acquired EMG signal is then converted from analog to digital using the in-built analog-to-digital converter (ADC). The digitally converted EMG signal is then stored in a buffer and, once the buffer reaches a certain size or a specified time interval elapse, the microcontroller transmits the acquired EMG signal in a secured way to the Google spreadsheet via hypertext transfer protocol secure (HTTPS).

2.2. IoT Cloud Platform

Figure 3 shows the architectural overview of the proposed IoT cloud platform for the monitoring of motor recovery function. Firstly, the EMG signals acquired from different patients are stored in individual Google spreadsheets in the Google drive. Then, the computation is performed with the help of Google Colaboratory, which acts as an IoT/IIoT cloud platform. Once the EMG signal values are stored in the Google spreadsheets, the signal processing algorithm process the raw EMG signal values.

2.2.1. EMG Signal Processing

In this work, a pre-processing of the EMG signal is performed by two different methods, namely the Hilbert–Huang transform (HHT) and a scalogram. The HHT is adopted along with the fast Fourier transform (FFT) for the removal of noise from the acquired raw EMG signals [14], and the scalogram is used to convert one-dimensional EMG signals into two-dimensional time frequency signals which is given as an input to the CNN classifier. The HHT is a combination of the empirical mode decomposition (EMD) and the Hilbert transform (HT). In addition, the EMD preserves the characteristics of the varying frequencies by decomposing the EMG signal into different mono component signals called intrinsic mode functions (IMFs); i.e., it is composed of a narrow band of frequencies. The EMD can be used for any complicated dataset by decomposing it into a finite small number of components. The EMD algorithm applied on the EMG signal y[n] is represented as follows [16,17]:

$$\mathrm{y}[\mathrm{n}] = {\sum }_{i=1}^k{IMF}_i\left[n\right]+z_k\left[n\right]$$

(1)

where IMF_i[n] is the ith IMF, z_k[n] is the residue, and k represents the total number of IMFs. The HT method is used to demodulate the IMFs into amplitude and frequency modulation signals. Once the different IMFs are obtained, the frequency of each IMF can be calculated using the fast Fourier transform (FFT). In general, the signals from any domain can be represented in frequency domain using an FFT analysis; the FFT extracts the various frequency components present in the given signal over a period of time by a sampling process [15]. Identifying noise in terms of IMFs helps to remove the appropriate IMFs and to concatenate the remaining IMF results in the suppression of noise. The continuous wavelet transform (WT) of an EMG signal is depicted as a 2D time frequency plot, called a scalogram [18,19]. The time parameter is denoted in the x-axis and the scale parameter, which is inversely proportional to the frequency of the signal, is denoted in the y-axis. The absolute value of the WT is proportional to the color intensity of the pixel of the scalogram plot.

2.2.2. Deep Learning Algorithm

In the proposed work, a convolutional neural network (CNN) is adopted to classify muscle power using EMG signals. In physiotherapy treatments, muscle power is always graded using manual muscle testing (MMT).

Therapy is performed by subjecting patients to various exercises. In the MMT process, resistance is applied to the muscle of the patient and grading is performed while the muscle is contracted. Similarly, the grading is derived for various actions, which is shown in Figure 4. The architecture of a CNN consists of one input/output layer, one or more pooling and convolution layers, and several fully connected layers [18,19,20,21]. In general, the training process of the CNN is performed with the help of two different stages, namely a forward and a backward propagation stage. The spatial information is extracted from the given 2D image using the CNN in the forward propagation stage, while the internal parameters are updated throughout the network to optimize the objective function. There is no constraint in setting the number of layers in the CNN. So, the total number of layers can be selected according to the train–test approach. The architecture of a CNN is shown in Figure 5.

Three different layers are present in the forward propagation, namely the convolution layer, the pooling layer, and the fully connected layer. Firstly, the possible features from the given input image are learned to recollect the bonding between the pixels of the input image using the convolution layer. The biases and an activation function are used to enhance the learning process, and the process is given by the following equation [18]:

$$y_k^j=f\left({\sum }_{i\in S_n}{y}_i^{j-1}\ast w_{in}^j+b_k^j\right)$$

(2)

where $y_k^j$ is the ith component of the layer k, S_n is the nth region of the convolution layer with feature map of j − 1 layer, $w_{in}^j$ and $b_k^j$ are the weight matrix and the added bias, respectively. After the convolution layers (CLs), the pooling layer is present, which is used to eliminate or reduce the redundant features extracted by the CLs. The max pooling concept is utilized in this work as a polling layer whose output $y_k^j$ is given as follows [18]:

$$y_k^j=f\left(w_k^j\ast \max \left(y_k^{j-1}\right)+b_k^j\right)$$

(3)

where down sampling is performed for the output $y_k^j$ of the convolution layer, $w_k^j$ represents the weight matrix, and $b_k^j$ denotes the bias matrix, while the max pooling function is represented by $\max \left(y_k^{j-1}\right)$, which is used to reduce the dimensions of the feature maps sourced by the convolution layer. Several convolution and pooling layers are combined together which increase the network depth of the CNN architecture. In general, the final output of the fully connected layers (FCLs) is achieved by organizing it layer-by-layer. The final FCL gives the output feature, which is expressed as follows [18]:

$$x^z=f(w^a{y}^{a-1}+b^a)$$

(4)

where the order of the CNN network architecture is denoted as $a$, the output of the FCL is represented by $x^z$, and the activation function is denoted as $f$.

3. Results and Discussion

According to the standard electrode placement protocol, the MyoWare EMG sensor was placed at the brachioradialis, which is the forearm muscle [13], and the EMG signals were acquired from various healthy individuals and various individuals suffering from musculoskeletal disorders under the supervision of a physiotherapist. Proper consent was received from the both healthy and injured individuals, and the EMG signals were acquired. Generally, two different phases are involved in the classification process, namely the training phase and the testing phase. For the training phase, 80% of the acquired EMG signals from healthy and injured individuals were utilized and the knowledge model was created. For the testing phase, 20% of acquired EMG signals from healthy and injured individuals were used. Four different performance metrics, namely accuracy, precision, recall, and F1-score, were performed [22] to evaluate the performance of the adopted CNN classifier; the results are presented in

Table 1. Performance Metrics of Multiclass CNN Classifier.

Class	Accuracy %	Precision	Recall	F1-Score	Class
Grade 0	97.6	0.94	0.92	0.93	Grade 0
Grade 1	97.1	0.92	0.91	0.91	Grade 1
Grade 2	97.8	0.92	0.95	0.93	Grade 2
Grade 3	98.3	0.94	0.96	0.95	Grade 3
Grade 4	97.1	0.92	0.91	0.91	Grade 4
Grade 5	97.1	0.92	0.91	0.91	Grade 5

. It can be seen that the performance metrics for all six classes were evaluated. The average accuracy of the multiclass CNN classifier is 97.5%, which indicates that the proposed device is more effective for identifying the course of the physiotherapy treatment process. The average precision, recall, and F1-score of the adopted multiclass CNN classifier is 92.6%, 92.6%, and 92.3%, respectively.

For data transmission over the internet, two different library packages were utilized, namely smtplib and mime, which can be installed with Python software version 3.12.7 using commands such as pip install smtplib and pip install mime. The purpose of using the Multipurpose Internet Mail Extension (MIME) and the Simple Mail Transfer Protocol (SMTP) is shown in Figure 6. It is observed that the MIME allows the user to exchange data files, such as images, audio, and video, over email, whereas the SMTP helps the user to send email to any machine which is connected to internet.

Figure 7 shows the decision support of the IoT cloud platform for the physician which was sent through email. It is observed that the suspicion about the patient health condition, along with patient ID, EMG signal, and graph of day-wise muscle power grade was sent to the physician. Generally, the Google Colaboratory requires a user account to perform computation, and it is seen that the sender email ID is same as the email ID of the user account of the Google Colaboratory. Also, it is clearly stated that the subject and content, “Status of patient (ID:001)” and “The patient (ID:001) has a progressive improvement, and the patient’s current power grade is Grade 2—Poor|Active movement but not against gravity. The EMG and day-wise muscle power grade are attached here for your reference”, respectively, was sent to the physician’s email, which enables the physician to ensure the progress of the physiotherapy process.

Figure 8 shows the day-wise muscle power grade values as plotted in the form of graph for a duration of 30 days, which helps the physician to identify the progress of the course of therapy in an easy manner. The duration of the day-wise plot can be changed to any desired number of days, and it is demonstrated that the proposed work can store the grade values for a maximum of 40 days. The x-axis denotes the number of days, and the y-axis denotes the frequency which, in turn, the grade was predicted using the CNN. So, it is evident that the proposed IoMT-based smart EMD device helps the physician/therapist to track the course of the therapy process and improves the patient’s quality of life.

4. Conclusions

Physiotherapy requires the continuous monitoring of muscle activity to track progress and adjust treatment plans accordingly. Many patients across the globe have limited access to physiotherapy services due to geographical or financial constraints. Traditional methods for monitoring muscle activity are very expensive and require specialized equipment. In this study, a cost effective IoMT-based smart EMG device was devised which can be easily deployed in a clinical setting or in a patient’s residence. Various performance metrics, such as accuracy, precision, recall, and F1-score, for the utilized multiclass CNN classifier were evaluated. The results demonstrate that the proposed device enables physiotherapists to provide services to patients who may not have access to traditional physiotherapy services. It was observed that the accuracy, precision, recall and F1-score of the utilized CNN classifier is 97.5%, 92.6%, 92.6%, and 92.3%, respectively. It was further demonstrated that any deviations in the treatment process were identified and notification was sent to the physician with the help of the IoT cloud platform. Overall, the proposed IoMT-based smart EMG device for physiotherapy applications is highly suitable for real-time and remote monitoring of muscle activity, which leads to improved patient outcomes and better access to physiotherapy services.