Machine Learning Classification for a Second Opinion System in the Selection of Assistive Technology in Post-Stroke Patients

Featured Application

The results of this work can be used to support future choices regarding machine learning algorithms in various application domains.

Abstract

It is increasingly important to provide post-stroke patients with rapid access to patient-tailored assistive technologies to increase independence, mobility, and participation. Automating the selection of assistive devices based on artificial intelligence could speed up the process and improve accuracy. It would also relieve the burden on diagnosticians and therapists and speed up the introduction of new ranges by automating databases. This article compares selected machine learning classification methods in the area of post-stroke rehabilitation device selection. The article covers the specifics of the selection, the choice of classification methods, and the identification of the best one, as well as the experimental part, the description of the results, the comparison process, and directions for further research. The novelty lies both in the topic, as the choice of classification method has an impact on the accuracy of classification in the selection of medical materials, and in the manner of the comprehensive approach. The possible contribution is of great scientific and clinical relevance, but above all, it has economic and social importance, enabling post-stroke individuals to return more quickly to the community, learning, and work, and relieving the burden on the health care system.

1. Introduction

According to the World Health Organization (WHO), stroke is the second most common cause of death worldwide and mostly affects people over 40 years of age. With an ageing population and unfavorable lifestyle changes, an increase in the number of strokes is to be expected. Therefore, it is increasingly important to ensure that assistive devices are adequately provided to increase independence, mobility, and participation in the local community for people after stroke [1,2,3,4,5,6].

Stroke is defined as focal, sudden, vascular-induced damage to the central nervous system (retina, spinal cord, or brain), the diagnosis of which requires confirmation of the presence of a stroke focus on neuroimaging studies or the persistence of focal symptoms for more than 24 h while excluding other sources of neurological impairment. We distinguish between:

• Ischaemic stroke (80–90% of cases);
• Haemorrhagic stroke (10–20% of cases) [1,2,3,4,5,6].

Ischaemic stroke refers to an episode of focal damage to the nervous system, whereas haemorrhagic stroke refers to rupture of a vessel and extravasation of blood into the brain or subarachnoid space. The sequelae of a stroke are various neurological losses, primarily paresis, balance problems, and cognitive impairment. It is also estimated that 25 to 60 per cent of stroke survivors suffer from depression. In 25 to 50% of patients, stroke leads to permanent disability. Communication difficulties are also one of the possible consequences, leading to social exclusion. A decrease in life satisfaction is observed in 83% of stroke survivors. One in five people who have experienced stroke cannot move without assistance, and one in four is dependent whileperforming activities duringdaily life. Up to one in two patients (25–50%) experience reduced mobility. Brain damage also leads to disorders of aperson’s intellectual capacity, consciousness, emotions, and overall personality structure. Systemic complications (dependent on care and early rehabilitation) such as bedsores, pneumonia, urinary tract infections, or thromboembolic complications may also occur. Epileptic seizures, manic states, anxiety attacks, and compulsive laughing or crying also occur. The aforementioned sequelae also lead to economic consequences such as inability to work or medical costs [1,2,3,4,5,6].

Task-specific training with assistive devices is more effective in reducing disability compared to usual task-specific care in the subacute phase after stroke, but such devicesare equally effective in the chronic phase of stroke [7,8]. Although common features and effects of living with limited mobility are apparent, individual stroke patients have personalised requirements that should be considered in the design of assistive technology devices. Thus, the development of artificially intelligent systems to support the selection of assistive devices makes significant sense, increasing the speed of selection and accuracy of decisions. This allows the diagnostician to focus on the details without having to go through the entire selection algorithm [9,10]. Patients have little influence over the choice of assistive devices; despite this, patients are willing to use assistive devices as intended. It should be mentioned that maintenance and inspection procedures vary depending on the manufacturer, etc. [11].The daily use of assistive technology acts as an additional facilitator to increase activity and participation of people with stroke, encompassing not only the use of limbs in activities of daily living, but also much more broad uses in terms of health-related quality of life (HRQoL), balance, prevention of adverse events, or development of exercise capacity [12].The most commonly used assistive technologies in the rehabilitation of post-stroke patients are shown in

Table 1. The most commonly used assistive technologies in the rehabilitation of post-stroke patients (own study).

Name	Description
wheelchair	Wheelchairs are vehicles that provide permanent or temporary mobility for immobile people in their daily living space. Wheelchairs are assistive equipment when there are mobility difficulties, limited motor coordination, and reduced lower limb function. They are usually the first choice when these types of problems are noticed.
walker	Walkers are designed to help disabled people get around. Most models of walkers have functional wheels to increase the patient’s mobility and a shelf on which to place a bag, groceries, or a meal, or even on which to sit down.
foot drop orthose	Foot drop orthoses help to restore lost function by actively assisting the movement of lifting the foot in people with extensor muscle paresis.
knee brace	A knee brace is an orthopaedic device used to brace and stabilise this part of the lower limb. In some situations, it replaces a plaster bandage that immobilises the knee joint or restricts its mobility.
elbow crutch	Elbow crutches allow movement even when the lower part of the body is immobilised. By distributing the body’s weight and transferring it partly to the arms, they relieve pressure on the legs and protect them from reinjury.
parapodium	The purpose of the parapodium is to set the patientupright, stabilise the lower limbs and trunk, and free the upper limbs from their supporting function.
shell	The shell stabilises and immobilises the body part in question.
forearm stabiliser	The forearm stabiliseris designed to stabilise the radius and elbow bones in an orthopaedic setting. Two cast aluminium splints, dorsal and palmar, hold the wrist in a functional position.
tracheostomy tube	A tracheostomy tube is a special tube placed in the trachea and attached to the neck with ribbons. Tracheostomy tubes can be of different designs, such as tricuspid or with a sealing balloon. Tracheostomy tubes are used to provide the patient with an open airway and adequate controlled ventilation.
four-legged cane	The four-legged cane is designed for people with limited mobility who need stable support and assistance when walking. The four-point support design provides significantly more stability than conventional canes.
prosthesis	A prosthesis is an artificial replacement for a missing body part or organ.
sling	A sling is a type of stabiliser that is used to stabilise and immobilise the upper limb.
decubitus cushion	A decubitus cushion reduces the risk of pressure sores (decubitus ulcers).
lift with an upright function	A lift with an upright function is a device that allows people to be lifted comfortably into a semisitting position both from a lying position (e.g., from a bed or even from the floor) and from a sitting position (e.g., from a wheelchair or chair).
orthopaedic hamstring	An orthopaedic hamstring is a type of special orthosis that is mainly worn on the lumbosacral region of the spine (e.g., a lumbar orthosis).
brace	The function of the brace is to stabilise the skeletal system and restrict mobility beyond the physiological range of motion of the human body.
orthopaedic waistcoat	The orthopaedic waistcoat supports the treatment of not only injuries to the shoulder itself, but also injuries to the shoulder girdle, shoulder joint, and elbow.
hemilflex	The hemiflex relieves pressure on the shoulder joint and stabilises the affected limb by supporting the forearm and fixing it to the brace rail.

.

Articles in the field of machine learning were analysed, with a particular focus on articles involving the algorithms that were used in the experimental part. Careful review of six main databases showed 316 publications concerning selection of assistive devices (published 1952–2023 and none of them using machine learning) but only 18 concerning selection of assistive devices for post-stroke patients (published 1992–2023 and none of them using machine learning). The analysis conducted aimed to better understand the scope of the use of machine learning in a scientific context. It provided a better understanding of this scope and confirmed that machine learning is also applicable in the medical sphere.

The aim of the study was to compare selected machine learning classification methods using an app in the area of orthopaedic supply item selection after stroke.

The main criteria were the selected item, the accuracy of prediction, and the speed of learning. Thus, detailed tasks within the study were as follows:

• Analysis and selection of appropriate classification methods in the area of post-stroke assistive device item selection;
• Creation of an application to compare the selected classification methods for the selection of a medical supply item.

2. Material and Methods

2.1. Material

The dataset used consisted of three inputs and one output. The input data were:

• Patient gender;
• Age of patient;
• Number of weeks that hadelapsed since the stroke.

The output data were:

• Item frompost-stroke assistive devices presented in numerical form.

The entire dataset consisted of 1350 records and had 49 unique outputs.

We based our study on actual anonymized data from Polish patients in clinical practice, selecting assistive technologies forpost-stroke patients. This made it possible to directly address the actual characteristics of the patient sample under study, thereby increasing the suitability of the solution under study for clinical applications, without the need for additional validation on many different patient groups. This partly solved the problem of validation, as our results could be compared with the choices made forthe same patients by actual specialists.

2.2. Methods

The code was intended to be as simple as possible, functional, without unnecessary elements, and readable. The code was to be written with an eye on time efficiency, functionality, and resource consumption. The user interface was to be as simple as possible, without unnecessary elements focusing on the goal. It was also written using version control.

In the process of selecting the algorithms, an analysis was made as to whether to enter the attribute in symbolic or numeric form in the output. The results for the numerical form gave more qualitative results. These trials were carried out several times, each time presenting a similar result. After a preliminary analysis of the classification algorithms in Statistica 14.0.1 (StatSoft Inc., Tulsa, OK, USA) and Weka 3.9.6 (Waikato Environment for Knowledge Analysis, Hamilton, New Zealand), the three algorithms with the best performance were selected. The selected algorithms were:

• Decision tree;
• Random forest;
• MultiLayer Perceptron (MLP) neural network.

Decision trees are distinguished by the hierarchical nature of decision-making; hence, the selection of the order of features according to which the set of objects will be subdivided at each stage is crucial. A tree-graph consists of roots, nodes, edges, and leaves. The root of the tree corresponds to the selected attribute, the branches represent the values of this attribute, and the leaves are the nodes from which no edges emerge.

Random forests use the construction of multiple decision trees during learning and then generate a class being:

• For classification: the dominant of the classes;
• For regression: the predicted average of the individual trees.

MLP is a fully connected feedforward artificial neural network. It consists of at least 3 layers: input, hidden, and output and provides quick learning for most of the popular classification and prediction problems.

The three aforementioned algorithms achieved up to 80–90% prediction accuracy at the selection stage.

We dealt with overfitting by providing a relatively large data set, avoiding noisy or imprecise data sets, and reducing the complexity of the model.

Python (3.11.2, Python Software Foundation) is the most suitable language for exploring machine learning. Off-the-shelf libraries allow both easy implementation of machine learning algorithms and easy exploration of algorithm attributes for a dataset that we need in order to compare them. They also allow implementation of an intuitive graphical interface.

Training models using machine learning algorithms are performed using the sklearn library. Scikit-learn (sklearn) is one of the most popular libraries for classical machine learning algorithms. It is built on top of two core Python libraries: NumPy and SciPy. Scikit-learn supports most supervised and unsupervised learning algorithms. Scikit-learn can also be used for data mining and analysis, making it an excellent tool for machine learning beginners.

3. Results

The app was to allow three input attributes: gender, age, and weeks post-stroke. At the click of a button, the app was to process these inputs according to the models implemented in the app and then show the prediction results of the three algorithms, the accuracy of the algorithms’ predictions, and the training time. The application had to be as simple as possible for the user.

An application written in Python processed a csv file with a learning dataset, and then three different models were trained using libraries based on the learning file:the random forest model, the decision tree (C&RT) model, and the MLP model. Learning times and accuracy of these algorithms were recorded with each of the three models. The models were recorded using the pickle module. After collecting data from the user via a graphical interface on gender, age, and weeks, the data were processed to produce three results using the three different models.

These results areshown in the graphical interface along with the accuracy and training time of each method (Figure 1).

The application wasdivided into three main parts written in Python, the part responsible for training the algorithms and loading the data, the part for predicting and showing the results, and the part responsible for the graphical interface.

The first key element wasto load the data from the csv file and then split it into input and output data, and then further split it into training and test data, with test data accounting for 25% of the total data. This used the pandas library to load the data and the sklearn library to split the data into training and test data. The input columns were loaded into the X value, and the output columns were loaded into the y value. At the beginning of the code snippet were the imports of the individual libraries, followed by the initialisation of the three global variables. First, data from a csv file named ‘MagDaneLicz.csv’wereloaded into the pandas data frame, and then random samples were displayed from the data frame using the sample() method. Input variables and output variables were then extracted from the data frame. The output variables were stored in the y variable, and the input variables were stored in the X variable. Finally, the data weresplit into a training set and a test set using the train_test_split() function of the scikit-learn library where the size of the test set was 25%.

The next key element was training the machine learning algorithms, calculating accuracy, and measuring training time. Each algorithm was imported, respectively, from the sklearn library. The function responsible for calculating the accuracy of the algorithms was also imported. Using the fit function, models were trained, based on the training output and input values. A prediction was then made based on the test input values, and accuracy was calculated using the test output values and the predicted data.

The code snippet started by importing a random forest classifier from the scikit-learn library and a function to calculate accuracy from the scikit-learn library. A random forest classifier object was then created, and the fit() method was called on the object to train the model on the training data. The time.time() function was called before and after the fit() method was called to determine how long it takes to train the model. Class labels for the test data were predicted using the predict() function. The accuracy of the model was calculated using the accuracy_score() function, displayed in the console, and also saved to a variable. The same process wasthen repeated for the decision tree classifier and the neural network. In both cases, the relevant classifiers and functions for calculating accuracy were imported. Next, classifier objects were created and trained on the training data, and then a class label was predicted for the test data. Finally, the accuracy of the model was calculated, displayed in the console, and saved to a file.

For both decision tree and random forest model training, the settings were chosen by selecting the most accurate settings using the ‘grid search’ function, which did not make it into the final version of the application due to optimisation considerations. The ‘n_estimators’ argument specified the number of decision trees; in this case it was 10. The ‘max_depth’ argument specified the depth of the forest; in the case of the forest it was unlimited depth, and in the case of the tree it was a maximum of 20 levels. The ‘min_samples_leaf’ argument specified the number of examples per leaf; for a random forest it was one, while for a tree it was four examples. The ‘min_samples_split’ argument meant how many examples were needed for a node split; for a forest it was two examples, while for a tree it was six examples. The ‘criterion’ argument indicated which criterion for the quality of the split was used, and in this case it was the Gini criterion. The Neural Network, on the other hand, was trained according to parameters extracted through analysis in Statistica. The ‘hidden_layer_sizes’ argument specified the number of neurons in each hidden layer of the network. In this case, the hidden network layers had 22 and 49 neurons, respectively. The ‘solver’ argument specified the optimisation algorithm that was used to optimise the network weights. In this case, the optimisation algorithm “lbfgs” was used. The ‘max_iter’ argument specified the maximum number of iterations of the optimisation algorithm that were performed by the model. In this case, the algorithm performed a maximum of 148 iterations. The ‘activation’ argument specified the activation function used by the model. In this case, the activation function ‘tanh’ was used. The ‘random_state’ argument was used to ensure reproducibility of the results. This ensured that when the same random state was used, exactly the same model was obtained.

The predict function was also a key part. It took input values, and then, using these and the models, predicted the medical supply items. These items were stored as numbers in the csv file, so the number_to_text function converted the numbers into appropriate text describing what eachnumber meant. All results were then displayed on the graphical interface. This function took three parameters: gender, age, and number of weeks. It then performed the following actions: It assigned values to the variables plec, age, and weeks, which werereturned by the functions set_plec(), set_age(), and set_weeks(). These functions returned the values selected in the GUI. A one-dimensional array called input_query was created, which contained the values plec, age, and weeks. The predict() method was called on the model, model2, and model3 objects, which were models of random forest, decision tree, and neural network, respectively. The returned results were assigned to the variables result, result2, and result3. Results were converted from numeric values to text using the number_to_text() function. The times were then shown in the graphical interface to 5 decimal places. The times were followed by accuracy results converted to a percentage in the graphical interface. The predict function then displayed the predicted result on the GUI.

The last key section was the graphical interface, which was divided into two parts. The part responsible for the input data started with the initialisation of the ‘customkinter’ library and the creation of the frame. Next, the application name information at the top of the application was created. The next element was the option field responsible for selecting the gender. The set_plec function was also written for this element, which tookthe selected value and then turned it into a number depending on the selected gender. Then, there were two fields to enter the age and the weeks after the stroke and functions for them to access this data. The last element from this section was the button responsible for the predict function. The second part of the interface was the one responsible for displaying the results. It contained labels with information about the name of the classifier, the training time of this classifier, its accuracy, and what the prediction result was.

Training data representing 75% of the total data wereused to train the models withmachine learning algorithms.

The main performance evaluation criteria were training times and accuracy of data prediction. The algorithms were also tested manually by entering the input data and comparing the results obtained to the output in the dataset.

Below are training times and accuracy of the selected algorithms and the predicted rounded results for the given dataset:

• For the random forest algorithm, the training time was 0.015 s, and the accuracy was 92%.
• For the decision tree algorithm, the training time was 0.001 s, and the accuracy was 92%.
• For the MLP neural network algorithm, the training time was 0.676 s, and the accuracy was 91%.

For the random forest (Figure 2), the results of the analysis were different from those for the random trees (Figure 3); the number of selectable windows and the possible options in the basic window decreased.

Each training time was satisfactory, and the shortest training time was achieved usingthe decision tree algorithm. The accuracy of the algorithms was satisfactory in terms of predicting results. All achieved similar results in terms of accuracy.

The best results were achieved using MLP 4-70-48 (

Table 2. Results for the best MLP neural networks (best result in bold).

Network Structure	Quality (Training)	Quality (Testing)	Quality (Validation)	Algorithm
MLP 4-70-48	91.124413	89.231722	91.675412	BFGS
MLP 4-71-48	85.095137	83.663366	87.128713	BFGS
MLP 4-75-48	82.241015	76.732673	77.722772	BFGS
MLP 4-41-48	80.338266	77.227723	81.188119	BFGS
MLP 4-37-48	76.321353	67.821782	74.752475	BFGS

).

The first experiment involved predicting the results by manually entering data from the dataset and comparing the results to the output. The results matched the dataset, and all three algorithms correctly predicted the output. All subsequent trials produced the same results. An experiment was then conducted to manually input data similar to thosein the dataset. All algorithms gave correct results up to a certain range of closeness to the data in the dataset. For the rest of the entered data outside this range, the results of the algorithms overlapped to some extent.

As a result of the duration of the study, a comparison was made between a dataset in two versions, one having text in the output and the other having numbers in the output. The dataset having numbers in the output presented noticeably better results than those having text in the output.

4. Discussion

Summarizing the advantages of the random forest model developed, it can be said that, despite its simplicity, it provides the highest accuracy among the solutions studied and seems sufficient for the problem we have posed. Formulating quantitative conclusions, it can be said that it provides a high and sufficient accuracy of 92.01% for our application with a relatively short training time of 0.01297s.

4.1. Related Work

Machine learning systems to support the selection of assistive devices, especially in patients with neurological deficits (such as post-stroke patients), are rare. They currently fulfil the role of second opinion systems. We intend to develop them, including further types of input to increase accuracy, but the final decision is always made by a human; hence, a functional test is usually carried out. Nevertheless, systems such as ours, by prompting the specialist decisionmaker, promote the codification of knowledge and experience and speed up decision-making and the provision of the patient with the necessary assistive devices. Among other things, this allows less experienced therapists to make fewer mistakes, reducing their burden.

A review of the databases showed that there is no comparable software; hence, it is difficult to compare our results with those of other authors. A review of four key databases, PubMed, WoS, Scopus, and Google Scholar, returned only four scientific articles in English with the keywords ‘assistive device’, ‘stroke’, and ‘machine learning’ [13,14,15,16]. The introduction of a second opinion or expert system into the selection process of assistive technologies, not only in post-stroke patients, seems to be a necessity, as this has been a research gap thusfar. The novelty of the proposed solution lies in the combination of a comprehensive, population-typical dataset and the selection of a classification method in terms of classification accuracy in the selection of assistive technologies. An important novelty is the use of three different artificial intelligence solutions for this purpose and the direct comparison of their results for the same dataset. This will allow us to better plan both the replication of our research and the further development of similar solutions. This is important because the number of people in need of assistive technologies, including the elderly, will grow, and some of the assistive devices, along with changes in people’shealth status, will need to be periodically checked and readjusted.

The number of possible assistive technologies solutions is also growing, and soon it will take effort just to know all the appropriate solutions for different levels and types of deficits. Furthermore, with the availability of 3D printing, it may be worth considering bespoke printing of personalised assistive technology devices based on AI-based eHealth system proposals and human-diagnostician decisions.

4.2. Limitations of Our Study

Key limitations, often encountered in clinical settings, include the lack of integration and comprehensiveness of the data collected and the data’sauditing for fitness for use by AI systems.

The main limitation is the selection of a convenience sample, i.e., a raw database, and reliance on extraneous evaluation. The number of characteristics that could be considered is also a limitation.

4.3. Directions for Further Research

The primary direction for development is to extend the functionality of the graphical interface by adding the ability to select which algorithms we want to compare, and to add more test parameters. The above direction assumes the addition of possibilities to select algorithms, by implementing more algorithms, and the possibility to select them from the graphical interface. It is also assumed that additional test parameters will be added to the graphical interface. Personalization of assistive technology becomes a challenge, as the movements exercised with the devices (also as part of performing activities of daily living) are often predefined and similar for different people. To ensure optimal exercise, these should be the result of individualizing the choice of training process (range of movement, support strength, pace, type, and degree of change) based on the person’s degree, type, and deficit profile. Currently, standardized clinical measures for profiling movement deficits in post-stroke patients are subjective and imprecise [11]. The future of research in the area of assistive technologies design lies in the increased use of robotic devices, personalised by 3D printing; Internet of Things wearable sensors for personalised training of limb use, gait and balance, and activities of daily living and health monitoring [17,18,19,20,21]; and even novel brain–computer interfaces [22,23] and associated computational models [24]. Further, more advanced studies may additionally use data from postural and gait analysis [25,26]. We will also draw inspiration as to the direction of further research from the work of [27,28,29].

Big clinical data requires data integration and analysis by connecting medical devices using an intelligent and distributed platform of sensors, effectors through (IoT) to local or cloud-based artificial intelligence/machine learning (AI/ML) systems. This approach allows the integration of data from different sources to provide earlier and more accurate diagnosis and prognosis of a patient’s condition and take appropriate preventive and therapeutic actions [30,31,32] (Figure 4 and Figure 5).

Preventive medicine is based onsystems already in place that could change the face of medicine in the future, focusing on the prevention of disease and secondary changes and maintaining the best possible health (including the functional state of the user) instead of responding to a decline in health (functional state) in the form of therapeutic intervention [33]. The use of artificial intelligence has already repeatedly proven its usefulness in large, complex systems, including monitoring and diagnostics [34,35,36]. Hence, the proposed solution can fit into the development of new and upgraded existing systems using artificial intelligence for neurology [37,38,39], neurorehabilitation [40,41,42], and neurological physiotherapy [43,44,45,46].

5. Conclusions

Using Statistica 14.0.1 and Weka 3.9.6 software, appropriate classification methods were selected in the area of post-stroke assistive device selection. These were the random forest, decision tree (C&RT), and MLP neural network methods. An application was created to compare the selected classification methods, in Python language, in the area of post-stroke assistive device selection. The application allows input values to be entered and then predicts, using three different models, three different results regarding the selection of an assistive device, along with showing the accuracy and training time of these models. For the data set concerning the selection of an assistive device after a stroke, the random forest method was the most appropriate.