Next Article in Journal
Preclinical Experimental Study on New Cervical Implant Design to Improve Peri-Implant Tissue Healing
Previous Article in Journal
Machine Learning-Driven GLCM Analysis of Structural MRI for Alzheimer’s Disease Diagnosis
Previous Article in Special Issue
Enhancing Fermentation Process Monitoring through Data-Driven Modeling and Synthetic Time Series Generation
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Bioengineering
Volume 11
Issue 11
10.3390/bioengineering11111154
bioengineering-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Introduction of AI Technology for Objective Physical Function Assessment

by
Nobuji Kouno
1,2,3,*,
Satoshi Takahashi
1,2,
Masaaki Komatsu
1,2,
Yusuke Sakaguchi
1,2,4,
Naoaki Ishiguro
2,
Katsuji Takeda
2,
Kyoko Fujioka
1,
Ayumu Matsuoka
5,
Maiko Fujimori
5 and
Ryuji Hamamoto
1,2,*
1
Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo 104-0045, Japan
2
Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan
3
Department of Surgery, Graduate School of Medicine, Kyoto University, Yoshida-konoe-cho, Sakyo-ku, Kyoto 606-8507, Japan
4
Department of Neurosurgery, Faculty of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8655, Japan
5
Division of Survivorship Research, National Cancer Center Institute for Cancer Control, 5-1-1 Tsukiji, Chuo-ku, Tokyo 104-0045, Japan
*
Authors to whom correspondence should be addressed.
Bioengineering 2024, 11(11), 1154; https://doi.org/10.3390/bioengineering11111154
Submission received: 24 September 2024 / Revised: 9 November 2024 / Accepted: 14 November 2024 / Published: 16 November 2024
(This article belongs to the Special Issue ML and AI for Augmented Biosensing Applications)
Browse Figures
Versions Notes

Abstract

:
Objective physical function assessment is crucial for determining patient eligibility for treatment and adjusting the treatment intensity. Existing assessments, such as performance status, are not well standardized, despite their frequent use in daily clinical practice. This paper explored how artificial intelligence (AI) could predict physical function scores from various patient data sources and reviewed methods to measure objective physical function using this technology. This review included relevant articles published in English that were retrieved from PubMed. These studies utilized AI technology to predict physical function indices from patient data extracted from videos, sensors, or electronic health records, thereby eliminating manual measurements. Studies that used AI technology solely to automate traditional evaluations were excluded. These technologies are recommended for future clinical systems that perform repeated objective physical function assessments in all patients without requiring extra time, personnel, or resources. This enables the detection of minimal changes in a patient’s condition, enabling early intervention and enhanced outcomes.

1. Introduction

Physical function assessment is crucial for clinical decision-making and for guiding treatment strategies across various medical specialties. Surgical treatment, as well as medical treatments, such as dialysis and respiratory management, have an invasive aspect, and physical function is an extremely important indicator determining whether the patient can overcome such invasiveness and accept the therapeutic benefits [1]. Medical professionals use one or more physical function assessment tools, medical history, and hospital impressions to determine treatment strategies. Even among patients of the same age and disease status, variations in physical functions can lead to different treatment approaches and intensities [2].
The importance of physical function assessment in the practice for patients with cancer is no exception. Cancer rates are rising; 35 million are estimated to be newly diagnosed in 2024, an increase from 20 million in 2022 [3]. This trend applies not only to older people but also to younger people [4,5]. Although various guidelines suggest standard treatments according to the cancer stage, they cannot be applied uniformly because patients vary in their ability to accept invasion. Furthermore, due to global aging trends, cancer treatment decisions and clinical trial eligibility can no longer be based solely on chronological age [6]. Failure to adequately assess tolerability in patients can lead to adverse events, reduced quality of daily life after treatment, death, and decreased satisfaction among patients and their families. Moreover, inappropriate aggressive and invasive cancer treatments can cause economic losses [7]. Thus, effectively determining each patient’s ability to tolerate cancer treatment is critical; physical function is especially important.
The Eastern Cooperative Oncology Group Performance Status (ECOG PS) is the most commonly used physical function assessment tool in oncological practice [8]. ECOG PS is used to determine indications for treatment and as an eligibility criterion for clinical trials [9]. It is an easy-to-understand 5-point scale (0–4) [8] but largely depends on subjective judgment [9]. For decades, discrepancies in evaluations between those conducted by oncologists and physicians, physicians and nurses, and physicians and patients have been noted [10,11,12]. This interrater variability in ECOG PS prevents the medical team from sharing a unified treatment plan. In addition, despite the simplicity of the ECOG PS assessment, its documentation rate in electronic health records is approximately 50% for patients with colorectal cancer [13]. This percentage was markedly lower than expected.
Additionally, several measuring tools in oncological practice have been proposed for physical function assessment: the Short Physical Performance Battery (SPPB), Timed Up and Go (TUG) test, grip strength, and walking speed (4 m/6 m/8 m/10 m walking time, or 6 min walking time) [14]. These are called objective physical function assessments, which are quantitative and do not rely on the individual judgment of assessors [9,14]. SPPB is rated on a 12-point scale across three domains: 4 points for the ability to maintain closed-leg standing, semi-tandem, and tandem for 10 s; 4 points for the time to walk 4 m; and 4 points for the time required to stand and sit five times [15]. The TUG test measures the total time required to stand up from a chair, walk back and forth a distance of 3 m, and sit down [16]. Several measurement methods have been proposed for walking speed based on distance or time [14]. Grip strength was measured using a dynamometer [14].
Associations between objective physical function assessments and clinical outcomes in cancer patients have been reported. Some systematic reviews have highlighted the correlation between SPPB, TUG, walking speed, grip strength, and all-cause mortality [17,18]. In the surgical field, the SPPB is a predictor of prolonged hospital stay [19] and hospitalization-associated disability [20]. TUG is associated with severe postoperative complications and 90-day [21] or 5-year postoperative mortality [22]. In the field of chemotherapy, SPPB and grip strength predict treatment plan modifications or dose reduction [23,24,25]. The nutritional status of patients with cancer has also been associated with SPPB and grip strength [26,27]. Repeated objective physical function assessment enables the monitoring of the impact of treatment and determination of the effectiveness of rehabilitation interventions [28,29,30]. Furthermore, a decline in gait speed by 0.1 m/s results in an 8% increased risk of cardiovascular diseases [31]. Objective physical function assessment is also related to frailty and has attracted increased attention due to global aging trends [32]. Objective physical function assessment tools, such as SPPB, TUG, and walking speed, especially the 10-Meter Walk Test, are listed in the physical function domain according to the National Comprehensive Cancer Network Guidelines for Older Adult Oncology [33].
Thus, objective physical function assessments are highly informative due to the association between their results and various clinical outcomes. Conversely, they have several limitations. Although implementing physical function assessment may seem simple, the assessor must consider the possibility of the patient falling during the evaluation. Additionally, constraints related to personnel, time, and space limit the incorporation of these assessments into hospital care, especially in outpatient settings, which hinders their implementation in daily clinical practice [1]. As mentioned earlier, changes in physical function can be monitored through repeated assessments over time [28,29,30]; however, a single assessment at one point during each clinical course may not be adequately conducted owing to these barriers (Figure 1). Consequently, objective physical function assessment tools have not been widely adopted in clinical practice. The recent COVID-19 pandemic has further strained healthcare resources, leading to a shortage of therapists, particularly those conducting objective physical function assessments [34]. Moreover, the uneven distribution of therapists across regions has resulted in healthcare disparities [35].
To address these limitations, several studies have simplified the measurement of SPPB and TUG using Doppler sensors [36], one-dimensional light detection and ranging [37], automated multiple cameras [38], and instrumented inertial measurement units [39]. While these technologies automate the process of objective physical function assessment, they do not alleviate the constraints of space, personnel, or additional equipment. Furthermore, the cameras and other tools used may not be suitable for actual clinical use in terms of functionality.
Artificial intelligence (AI) technology has garnered attention in recent years owing to advances in machine learning (ML) with the advent of deep learning; advances in information infrastructure technology, such as the emergence of inexpensive and high-performance graphics processing units; and the ease of utilizing large-scale data, such as the expansion of public databases with the advent of the big data era [40,41]. The medical field is no exception, with active medical research utilizing AI technology in various forms, including medical image analysis [42,43,44,45], omics analysis [46,47,48], and natural language processing, with a focus on electronic medical records (EHRs) [49,50]. Importantly, medical devices utilizing AI technology are being promoted in rapid succession. According to the latest U.S. FDA database, more than 800 AI-powered medical devices have been approved [51]. AI technology is also being studied for objective physical function assessment, where AI is being used to provide promising options for estimating established physical function indicators from videos of various movements, sensor data, and EHR table data without the need to perform special movements. In fact, we are also currently developing an AI model to assess a patient’s physical function from routine clinical activities, such as entering and exiting the consultation room or sitting and standing from a chair. Research on this concept is novel, making it essential to explore which types of data—whether video, images, sensor data, or clinical information—can effectively predict objective physical function using AI. Therefore, in this review, we will discuss the effectiveness and issues of using AI to evaluate physical function by comprehensively surveying related papers published to date, assuming that AI technology will be used to objectively, accurately, and easily evaluate physical function. In particular, since ML is the fundamental technology behind current AI research [52], we focus on and discuss objective physical function assessments using ML technology.

2. Methods

2.1. Search Strategy

PubMed was searched in April 2024 for relevant studies using the following search strategy: (“Short Physical Performance Battery” OR “Timed up and Go” OR “grip strength” OR “walking speed” OR “gait speed” OR “fall risk”) AND (“artificial intelligence” OR “machine learning” OR “deep learning”). The titles and abstracts of the selected papers were searched.

2.2. Selection Criteria

The inclusion criteria for selecting the papers were that they should have been written in English. The publication period was not restricted. Notably, we determined that, in light of patients’ quality of life and the current situation in clinical practice, there is a need to develop a system that uses ML to predict the results of objective physical function assessments from videos of patients’ movements in hospitals during routine medical examinations, without conducting additional assessments. Thus, we reviewed techniques that can be adopted in clinical practice without requiring specific conditions solely for assessing physical function. Specifically, this review included studies where SPPB, TUG, walking speed, and grip strength were predicted using ML from videos, sensor data, or clinical information from EHR without manual measurement or additional clinical resources. Conversely, we excluded studies that utilized ML solely to automate the evaluation process, such as those that performed TUG in front of a camera without a measurer or conducted TUG or SPPB at home with attached sensors. Similarly, studies that used inertial measurement units (IMUs) or Doppler technology to automate the assessment of SPPB and TUG to reduce human resources were excluded. The same applied to walking speed, where the goal was to simplify the measurement of the 6 min walk or 10 m walk. From another perspective, several studies used ML to segment successive movements (such as standing up, walking, and turning) and extract gait parameters from IMU data or videos related to SPPB, TUG, and walking speed. Additionally, studies that used grip strength as an explanatory variable to predict SPPB scores or past TUG test scores and to predict future TUG test scores were included in the review. Only recent studies by the same authors were included, while studies by the same authors with different concepts were also included. Predicting gait speed posed challenges owing to the different movements involved. Therefore, we selected studies proposing systems for future use in clinical practice.

3. Results

3.1. Overall Trends in AI-Based Objective Physical Function Assessment Research

The initial search yielded approximately 370 articles with the following distribution: SPPB, 11; TUG, 68; fall risk, 132; walking or gait speed, 116; and grip strength, 40. In most studies, SPPB, TUG, and walking speed/gait speed were considered explanatory variables for predicting clinical outcomes such as survival and complications using ML. From these results, we identified 26 articles that fit our objective physical concept without performing the assessment process itself. A summary of these results is presented in Table 1. The predicted labels and number of studies were as follows: 9 papers on TUG or SPPB, 2 papers on grip strength, and 15 papers on walking speed. Over half of these studies used sensors such as IMUs, smartphones, smart home sensors, smartwatches, global navigation satellite systems (GNSSs), and Kinect V2 as input data. Most selected studies employed internal validation methods like k-fold cross-validation (CV), leave-one-out CV, train–test split, or train–val–test split. To evaluate the classification model, metrics such as accuracy, F1-score, and area under the ROC curve (AUC) were used. Common metrics for evaluating the regression model included errors and correlation indices. We summarized the results in the following sections, focusing first on the labels and then on the input data.

3.2. Characteristics of Studies Setting TUG/SPPB as an Output Label

Of the studies that used ML technology to objectively evaluate physical function, nine used TUG or SPPB as the output label (Table 1) [53,54,55,56,57,58,59,60,61]. Five of these studies used sensor data as input data: IMU, smartphone as an inertial sensor, and Kinect V2 as the depth sensor. In contrast, one study used videos of walking back and forth in front of a stereo camera as input data [61]. All labeled data were measured manually. Of the eight studies that used the TUG test as the label, five used classification tasks, and three used regression tasks. However, of the two papers that used the SPPB as the label, one used regression, and the other used classification. For the classification of SPPB and TUG, the cutoff values were set independently for each study: TUG, 10/13.5/14/20 and SPPB, six (male)/nine (female). As a distinctive label setting, Bloomfield et al. performed a classification for patients undergoing total knee arthroplasty (TKA) with a TUG test preoperatively and postoperatively and determined as a label whether there was an improvement of 2.27 s [55].
The ML techniques employed in most studies include orthodox techniques, such as support vector machine and elastic net, and ensemble methods, such as AdaBoost and XGBoost. Friedrich et al. used long short-term memory (LSTM) and a convolutional neural network (CNN) with sensor data [54]. Hasegawa et al. used commercial machine learning software with no coding required, such as Prediction One version 3.0.1.3 (Sony Corporation, Tokyo, Japan) [58].
The characteristics of the dataset cohort included preoperative orthopedic patients, older patients in institutions or receiving outpatient care, patients with cancer, and patients with neurological diseases such as facioscapulohumeral muscular dystrophy. Friedrich et al. used the OTAGO dataset of older adults with frailty, which is an existing open-source dataset [54,79].

3.3. Characteristics of Studies Setting Grip Strength an Output Label

Two studies on grip strength used regression analyses [62,63]. Hwang et al. used variables from physical profiles and body part measurements as input data for 164 healthy young volunteers [62]. In this study, various combinations of variables, including demographic and anthropometric information and posture, were tested and compared for each participant, and an attempt was made to propose the model with the highest predictive power. The methods used were MLP regression and three different polynomial regressions, and the results of comparing the performance of the regressions showed that including all the variables performed better than other combinations of variables. In addition, MLP regression showed higher performance than polynomial regression, and MLP regression that considers all variables, achieved the best performance in grip strength prediction. Bae et al. used national big data from billions of people aged over 65 years, called the Korean National Fitness Award data [63]. The aim was to determine the best ML regression model for predicting grip strength in adults aged 65 years and over using various independent variables, such as body composition, blood pressure, and physical ability. The dependent variable was grip strength, and it was shown that the CatBoost Regressor had the lowest mean square error and the highest R2 value among the seven prediction models tested.
These results show that a regression model based on ML can accurately predict the grip strength of older adults and may be useful for reducing the risk of musculoskeletal disorders of the upper limbs.

3.4. Characteristics of Studies Setting Walking Speed an Output Label

Similarly to studies that set the SPPB/TUG as an output label, the most common types of input data were recorded by the IMU, IMU with GNSS, smartphone as an inertial sensor, smartwatch, or 3D optical motion capture. However, two studies employed images, such as plantar pressure images and silhouettes of walking individuals [75,76], and the same number of studies used videos that tracked participants’ walking as input data [77,78]. For measuring walking speed as label data, Soltani et al. used GNSS measurements [65]; Kidzinski et al. used the Vicon (OMG plc, Oxford, UK) system [77,80], an optical motion capture system; and Lonini et al. used the GAITRite system, a gait analysis system with an electronic walking mat [78]. Other studies used manual measurements or treadmill gait settings. Canonical measures of walking speed included the 25-foot walking test [64], 10 m walking test [70], and 6 min walking test [68,71]. Eleven studies predicted walking speed as a regression task, while four studies approached it as a classification task. As a distinctive label setting, Lee et al. established the cutoff point for the classification task based on the difference between the preoperative and postoperative walking speeds for patients undergoing TKA; the difference was categorized as either an increase of over 10%, a decrease of over 10%, or neither [73]. Davis et al. performed a regression using the gait speed reserve of adults aged > 50 years in Ireland, which was defined as the maximum gait speed minus the usual gait speed [74].
For studies that obtained input data from sensors, ML techniques were similar to the general types used in the SPPB/TUG studies. However, for studies using images as input data, Sikandar et al. employed Bidirectional LSTM [75], while Chen et al. utilized CNN [76]. Both studies that used videos as inputs relied on pose estimation models. Kidzinski et al. compared the accuracy of walking speed estimation with CNN, random forest (RF), and Ridge, using features extracted via OpenPose [77]. Lonini et al. implemented DeepLabCut, an open-source pose estimation model with a graphical user interface [78].
The most common diseases in the dataset cohort were neurological disorders such as stroke [66,78], multiple sclerosis (MS) [64,68,70], and cerebral palsy [77]. Approximately half of the participants were healthy. Davis et al. used the large dataset from TILDA Wave 3 [74,81], while Sikandar et al. utilized the gait dataset from the Osaka University Institute of Scientific and Industrial Research dataset A [75,82].

3.5. Summary of Sensor Data Acquisition

Many studies predicting SPPB, TUG, and walking speed, but not grip strength, have used sensor data, mainly IMUs, as inputs. Smartphones and smartwatches were also used as IMUs. In studies in which the GNSS and IMU are used together, the GNSS is not counted as an IMU. The number of attached IMU sensors ranged from one to a maximum of five.
In the case of two or more IMUs, the attachment points are symmetrical. Polus et al. performed the TUG test for patients undergoing total hip arthroplasty (THA) with sensors above and below the knee bilaterally 2 weeks before and 2 weeks after surgery, classifying the TUG scores (more than 14 s or not) at 6 weeks postoperatively using 55 spatiotemporal and joint-specific metrics extracted from a series of sensor data [53]. Bloomfield et al. were part of the same research group as Polus et al., and the sensor attachment sites and extracted variables were the same. By contrast, Bloomfield et al. aimed to predict postoperative TUG improvements based on preoperative TUG tests [55]. McGinnis et al. had participants walk bilaterally on a treadmill with sensors placed above and below the knees. Seven patterns were set up with combinations of sensor positions, and a linear model trained on data from 17 healthy participants was used to estimate the 6 min walk of 30 patients with MS on the ground. The best model (root mean square error = 0.12) was developed using a single sensor on the sacrum [68].
In the case of single sensors, the site of fixation was either a shoe or a pelvic area such as the waist, sacrum, or lower back. When a smartwatch is used as the sensor, it may be fixed to the wrist [72], but Juen et al. used a smartphone fixed with a belt at the L3 level to collect data [71]. Further, Zhuparris et al. used participants’ smartphones as accelerometers but instructed them to handle them as usual without any specific fixation [56].
Only one study used optical sensor data as the input. Lee et al. predicted an improvement in walking speed after TKA using clinical information from the EHR and gait parameters extracted from walking at the Human Motion Analysis Lab [73]. When smart home sensors and Kinect V2 were installed in facilities and residences, the data collection period ranged from 8 h to 7 days [57,83]. In some cases, the collection lasted for several weeks (Figure 2) [84].

3.6. Studies with Image Input Data

Of the papers reviewed, two studies predicted walking speed using images as input data, whereas no studies predicting SPPB, TUG, or grip strength utilized images as input data. Sikandar et al. extracted five features (full-body height, full-body width, mid-body width, lower-body width, full-body area, apparent body area, and area between two legs) from a series of silhouette images at three different walking speeds. These five features were engineered into time series data, and a walking speed classification task was performed using bidirectional long short-term memory [75]. Chen et al. also established a region of interest for plantar foot pressure images, including the first toe, first metatarsal head, second metatarsal head, and heel. The images were classified into three levels of walking speed and two levels of walking duration using a CNN with each image as the input (Figure 3A) [76].

3.7. Studies with Video Input Data

Of the papers reviewed, one study predicted TUG and one study predicted walking speed using video as input data. Yuan et al. recorded facility residents walking in front of a stereo camera, extracted gait parameters using a mask R-CNN, and obtained gait speed and step length as the core features. They developed an original multiple regression model with various variables, including these core features, to estimate TUG scores [61]. Kidzinski et al. tracked and filmed the gait of 1026 pediatric patients with cerebral palsy using a single camera and obtained 1792 videos [77]. Key points were extracted using OpenPose, a common video pose estimation model, and the data for each coordinate were input as time series data to the CNN, along with summary statistics, such as the mean and percentiles for the RF/ridge regression. Gait parameters, such as walking speed and indices, were predicted. The OpenPose + CNN strategy yielded the best metrics for all prediction labels.
Lonini et al. employed DeepLabCut, a ResNet-based pose estimation model, to predict the walking speed of patients with stroke. Only the lower-body movements of the participants were recorded in the gait videos. In addition, the walking parameters measured using the GAITRite system were used as ground truth data (Figure 3B).

3.8. Studies with Tabular Input Data

Some studies employed tabular data as inputs. These mainly included clinical information extracted from EHRs and other sources or tabular datasets of existing datasets, including body part measurements or other objective measures of physical function as explanatory variables. Hasegawa et al. included calf circumference and grip strength as explanatory variables and conducted a binary classification task for the SPPB score with a cutoff value [58]. Kraus et al. also included the grip strength of the dominant and non-dominant hands as explanatory variables and conducted a regression analysis for the TUG score [59]. Hwang et al. conducted a grip strength regression task with variables including hand width and length in addition to age, height, and weight. Using data from the Korean National Physical Fitness Award, Bae et al. predicted grip strength using several variables, including TUG scores and walking speed (Figure 3C) [63].

4. Discussion

Research papers predicting the results of objective physical function assessments (TUG, SPPB, grip strength, and walking speed) using sensor data, videos, images, questionnaires, EHR items, and big data were reviewed. Most of the articles were related to walking speed and were published earlier than those related to other objective physical function assessments. Compared to SPPB, TUG, and walking speed, grip strength does not require much effort to measure if only a meter is available, which may explain why fewer studies have predicted grip strength using the ML method. However, fewer studies on the SPPB and TUG than on walking speed may be attributed to the components of the SPPB other than walking, such as maintaining balance and standing up. The TUG test includes getting up, transitioning to walking, and changing directions, making it a more challenging task to predict.
The types of objective physical function assessments and task types varied across the reviewed studies. Walking speed has been shown to correlate with disease when changes of 0.1 m/s occur [31], while grip strength changes of 5–6.5 kg may be considered clinically significant [85]. SPPB scores of 0–3 vs. 10–12, 4–6 vs. 10–12, and 7–9 vs. 10–12 are associated with differences in all-cause mortality [86]. For TUG, studies indicate that categorizing results as slow (≥15 s), intermediate (11–14 s), and fast (≤10 s) correlates with increased rates of postoperative complications and mortality temporally [87]. While it is challenging to establish a clear standard for best metrics among the reviewed studies, prediction at a finer resolution is essential.
Several studies employed IMUs and other sensors as modalities for extracting input data. Although the use of smartwatches and smartphones as sensors is reasonable, the impact of COVID-19 may present challenges in attaching things, bringing things into the living environment, and installing things touched by an unspecified number of people. In addition, smartphones have evolved remarkably in recent years; however, the challenge for older adults may still be high, and having to instruct them on the use of the application is a burden. Even if patients are instructed to wear these devices at home, the quality and quantity of the data can be unassured. However, the need for additional sensor equipment poses a barrier; using a single sensor, such as a smartphone, smartwatch, or IMU, may reduce the psychological burden on participants by minimizing the effort required for device attachment or set up.
Some studies have also used tabular data from EHRs and other sources, and these datasets include measurements of body parts or other types of objective functional assessment results (e.g., walking speed is used to predict grip strength, and conversely, grip strength is used to predict SPPB). Thus, despite efforts to resolve these challenges, other physical function assessment procedures in clinical practice may require an additional burden. In this respect, video, which is non-contact and can obtain data at a fixed point, has an advantage. However, there are challenges in clinical implementation, such as video shooting conditions that serve as inputs and the need for a video camera operator to track the walker. A video-based model that replaces the existing physical function evaluation process and results by extracting features of walking, standing, and sitting in a clinical setting would be beneficial. This would reduce the burden on medical professionals and enable more detailed observation of physical functions.
Aside from grip strength, the task of predicting physical function measures was either a classification or regression task in the reviewed studies. While classification tasks are useful for screening and determining eligibility or ineligibility for treatment by setting certain cutoff values, they fall short in expressing the degree of deviation from these cutoffs and cannot track changes in physical function over time. On the other hand, for measures like the TUG and walking speed, the amount of change—whether improvement or worsening—has been reported to correlate with clinical outcomes [88] and the risk of vascular diseases [31]. Therefore, regression tasks may be more useful than classification tasks. Even multilevel classification tasks with finer categories are preferred.
Furthermore, the distribution of the label data and the size of the dataset are very important for training ML models. Several studies have predicted walking speed using datasets from younger age groups, whereas studies predicting the SPPB/TUG have been conducted on older individuals, leading to imbalanced data. To develop a regression model that effectively screens and tracks score trends over time, careful consideration of the cohort size and participant background during research design is crucial.
Lastly, none of the reviewed studies were validated using external data. To address the limitations of the dataset size, several studies have employed k-fold CV and leave-one-out CV to evaluate model performance. Only a few studies used large datasets from self-administered centers. Most studies were based on datasets with fewer than 100 participants. In the future, it will also be important to create an environment for external validation using large-scale datasets by preparing public datasets that include tabular data, sensors, and video data related to objective physical function evaluation.

5. Conclusions

We reviewed the literature on the prediction of objective physical function indices such as the SPPB, TUG, grip strength, and walking speed using AI technology without performing the evaluation. If objective physical function assessment can be easily and repeatedly performed using such technologies, it will be possible not only to determine eligibility at the start of treatment but also to detect minimal changes over time, which may contribute to personalized medicine. In particular, the number of elderly cancer patients is increasing, and to determine the optimal treatment plan based on each patient’s physical function, it is crucial to develop an AI system that can accurately predict the results of objective physical function assessments from in-hospital motion videos when considering the QOL of elderly cancer patients.

Author Contributions

Conceptualization, N.K., S.T. and R.H.; investigation, N.K. and S.T.; writing—original draft preparation, N.K. and S.T.; writing—review and editing, N.K., S.T., M.K., Y.S., N.I., K.T., K.F., A.M., M.F. and R.H. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Cabinet Office BRIDGE (programs for bridging the gap between R&D and the ideal society (Society 5.0) and generating economic and social value) and the RIKEN Center for the Advanced Intelligence Project.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No new data were generated for this study.

Acknowledgments

We would like to thank all members of the Hamamoto Laboratory, who provided valuable advice and a comfortable research environment.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Bakker, E.A.; Zoccali, C.; Dekker, F.W.; Eijsvogels, T.M.H.; Jager, K.J. Assessing physical activity and function in patients with chronic kidney disease: A narrative review. Clin. Kidney J. 2021, 14, 768–779. [Google Scholar] [CrossRef]
  2. Toyoshima, K.; Seino, S.; Tamura, Y.; Ishikawa, J.; Chiba, Y.; Ishizaki, T.; Fujiwara, Y.; Shinkai, S.; Kitamura, A.; Araki, A. Difference between "Physical Fitness Age" Based on Physical Function and Chronological Age Is Associated with Obesity, Hyperglycemia, Depressive Symptoms, and Low Serum Albumin. J. Nutr. Health Aging 2022, 26, 501–509. [Google Scholar] [CrossRef]
  3. World Health Organization. WHO Global Survey on the Inclusion of Cancer Care in Health-Benefit Packages, 2020–2021; World Health Organization: Geneva, Switzerland, 2024.
  4. Longevity, T.L.H. Older patients with cancer: Evidence-based care needs evidence. Policy Rev. 2021, 2, e678. [Google Scholar]
  5. Zhao, J.; Xu, L.; Sun, J.; Song, M.; Wang, L.; Yuan, S.; Zhu, Y.; Wan, Z.; Larsson, S.C.; Tsilidis, K.K. The global trends in incidence, death, burden and risk factors of early-onset cancer from 1990 to 2019. BMJ Oncol. 2023, 2, e000049. [Google Scholar] [CrossRef]
  6. Given, B.; Given, C.W. Older adults and cancer treatment. Cancer Interdiscip. Int. J. Am. Cancer Soc. 2008, 113, 3505–3511. [Google Scholar] [CrossRef]
  7. Cheung, M.C.; Earle, C.C.; Rangrej, J.; Ho, T.H.; Liu, N.; Barbera, L.; Saskin, R.; Porter, J.; Seung, S.J.; Mittmann, N. Impact of aggressive management and palliative care on cancer costs in the final month of life. Cancer 2015, 121, 3307–3315. [Google Scholar] [CrossRef]
  8. West, H.J.; Jin, J.O. JAMA Oncology Patient Page. Performance Status in Patients With Cancer. JAMA Oncol. 2015, 1, 998. [Google Scholar] [CrossRef]
  9. Scott, J.M.; Stene, G.; Edvardsen, E.; Jones, L.W. Performance status in cancer: Not broken, but time for an upgrade? J. Clin. Oncol. 2020, 38, 2824. [Google Scholar] [CrossRef]
  10. Kim, Y.J.; Hui, D.; Zhang, Y.; Park, J.C.; Chisholm, G.; Williams, J.; Bruera, E. Differences in Performance Status Assessment Among Palliative Care Specialists, Nurses, and Medical Oncologists. J. Pain Symptom Manag. 2015, 49, 1050–1058.e2. [Google Scholar] [CrossRef]
  11. Neeman, E.; Gresham, G.; Ovasapians, N.; Hendifar, A.; Tuli, R.; Figlin, R.; Shinde, A. Comparing Physician and Nurse Eastern Cooperative Oncology Group Performance Status (ECOG-PS) Ratings as Predictors of Clinical Outcomes in Patients with Cancer. Oncologist 2019, 24, e1460–e1466. [Google Scholar] [CrossRef]
  12. Schnadig, I.D.; Fromme, E.K.; Loprinzi, C.L.; Sloan, J.A.; Mori, M.; Li, H.; Beer, T.M. Patient-physician disagreement regarding performance status is associated with worse survivorship in patients with advanced cancer. Cancer 2008, 113, 2205–2214. [Google Scholar] [CrossRef]
  13. Agaronnik, N.; Lindvall, C.; El-Jawahri, A.; He, W.; Iezzoni, L. Use of natural language processing to assess frequency of functional status documentation for patients newly diagnosed with colorectal cancer. JAMA Oncol. 2020, 6, 1628–1630. [Google Scholar] [CrossRef]
  14. Muhandiramge, J.; Orchard, S.G.; Warner, E.T.; van Londen, G.J.; Zalcberg, J.R. Functional decline in the cancer patient: A review. Cancers 2022, 14, 1368. [Google Scholar] [CrossRef]
  15. Guralnik, J.M.; Ferrucci, L.; Simonsick, E.M.; Salive, M.E.; Wallace, R.B. Lower-extremity function in persons over the age of 70 years as a predictor of subsequent disability. N. Engl. J. Med. 1995, 332, 556–562. [Google Scholar] [CrossRef]
  16. Podsiadlo, D.; Richardson, S. The timed “Up & Go”: A test of basic functional mobility for frail elderly persons. J. Am. Geriatr. Soc. 1991, 39, 142–148. [Google Scholar]
  17. Ezzatvar, Y.; Ramírez-Vélez, R.; Sáez de Asteasu, M.L.; Martínez-Velilla, N.; Zambom-Ferraresi, F.; Izquierdo, M.; García-Hermoso, A. Physical function and all-cause mortality in older adults diagnosed with cancer: A systematic review and meta-analysis. J. Gerontol. Ser. A 2021, 76, 1447–1453. [Google Scholar] [CrossRef]
  18. Nakano, J.; Fukushima, T.; Tanaka, T.; Fu, J.B.; Morishita, S. Physical function predicts mortality in patients with cancer: A systematic review and meta-analysis of observational studies. Support. Care Cancer 2021, 29, 5623–5634. [Google Scholar] [CrossRef]
  19. Gill, T.M.; Becher, R.D.; Murphy, T.E.; Gahbauer, E.A.; Leo-Summers, L.; Han, L. Factors associated with days away from home in the year after major surgery among community-living older persons. Ann. Surg. 2023, 278, e13–e19. [Google Scholar] [CrossRef]
  20. Hori, K.; Usuba, K.; Sakuyama, A.; Adachi, Y.; Hirakawa, K.; Nakayama, A.; Nagayama, M.; Shimokawa, T.; Takanashi, S.; Isobe, M. Hospitalization-Associated Disability After Cardiac Surgery in Elderly Patients―Exploring the Risk Factors Using Machine Learning Algorithms―. Circ. Rep. 2021, 3, 423–430. [Google Scholar] [CrossRef]
  21. Sliwinski, S.; Faqar-Uz-Zaman, S.F.; Heil, J.; Mohr, L.; Detemble, C.; Dreilich, J.; Zmuc, D.; Bechstein, W.O.; Becker, S.; Chun, F. Predictive value of a novel digital risk calculator to determine early patient outcomes after major surgery: A proof-of-concept pilot study. Patient Saf. Surg. 2024, 18, 13. [Google Scholar] [CrossRef]
  22. Hendriks, S.; Huisman, M.G.; Ghignone, F.; Vigano, A.; de Liguori Carino, N.; Farinella, E.; Girocchi, R.; Audisio, R.A.; van Munster, B.; de Bock, G.H. Timed up and go test and long-term survival in older adults after oncologic surgery. BMC Geriatr. 2022, 22, 934. [Google Scholar] [CrossRef]
  23. Almugbel, F.A.; Timilshina, N.; Papadopoulos, E.; Al-Showbaki, L.; Alibhai, S.M. The role of grip strength and short physical performance battery test in predicting chemotherapy-related outcomes in older adults with cancer. J. Geriatr. Oncol. 2022, 13, 318–324. [Google Scholar] [CrossRef]
  24. Papadopoulos, E.; Helal, A.A.; Jin, R.; Monginot, S.; Berger, A.; Romanovsky, L.; Alibhai, S.M. The impact of pre-treatment muscle strength and physical performance on treatment modification in older adults with cancer following comprehensive geriatric assessment. Age Ageing 2022, 51, afac152. [Google Scholar] [CrossRef]
  25. Sourdet, S.; Brechemier, D.; Steinmeyer, Z.; Gerard, S.; Balardy, L. Impact of the comprehensive geriatric assessment on treatment decision in geriatric oncology. BMC Cancer 2020, 20, 1–9. [Google Scholar] [CrossRef]
  26. Ai, D.; Ding, N.; Wu, H. The impact of sarcopenia on nutritional status in elderly patients with gastrointestinal tumors. Sci. Rep. 2023, 13, 10308. [Google Scholar] [CrossRef]
  27. Tanaka, K.; Taoda, A.; Kashiwagi, H. The associations between nutritional status, physical function and skeletal muscle mass of geriatric patients with colorectal cancer. Clin. Nutr. ESPEN 2021, 41, 318–324. [Google Scholar] [CrossRef]
  28. Farrugia, M.; Erickson, K.; Wendel, E.; Platek, M.E.; Ji, W.; Attwood, K.; Ma, S.J.; Gu, F.; Singh, A.K.; Ray, A.D. Change in physical performance correlates with decline in quality of life and frailty status in head and neck cancer patients undergoing radiation with and without chemotherapy. Cancers 2021, 13, 1638. [Google Scholar] [CrossRef]
  29. Wooten, S.V.; Fleming, R.D.; Wolf, J.S., Jr.; Stray-Gundersen, S.; Bartholomew, J.B.; Mendoza, D.; Stanforth, P.R.; Stanforth, D.; Hernandez, L.M.; Tanaka, H. Prehabilitation program composed of blood flow restriction training and sports nutrition improves physical functions in abdominal cancer patients awaiting surgery. Eur. J. Surg. Oncol. 2021, 47, 2952–2958. [Google Scholar] [CrossRef]
  30. Lafaro, K.J.; Raz, D.J.; Kim, J.Y.; Hite, S.; Ruel, N.; Varatkar, G.; Erhunmwunsee, L.; Melstrom, L.; Lee, B.; Singh, G. Pilot study of a telehealth perioperative physical activity intervention for older adults with cancer and their caregivers. Support. Care Cancer 2020, 28, 3867–3876. [Google Scholar] [CrossRef]
  31. Veronese, N.; Stubbs, B.; Volpato, S.; Zuliani, G.; Maggi, S.; Cesari, M.; Lipnicki, D.M.; Smith, L.; Schofield, P.; Firth, J. Association between gait speed with mortality, cardiovascular disease and cancer: A systematic review and meta-analysis of prospective cohort studies. J. Am. Med. Dir. Assoc. 2018, 19, 981–988.e7. [Google Scholar] [CrossRef]
  32. Perracini, M.R.; Mello, M.; de Oliveira Máximo, R.; Bilton, T.L.; Ferriolli, E.; Lustosa, L.P.; da Silva Alexandre, T. Diagnostic accuracy of the short physical performance battery for detecting frailty in older people. Phys. Ther. 2020, 100, 90–98. [Google Scholar] [CrossRef] [PubMed]
  33. Dotan, E.; Walter, L.C.; Browner, I.S.; Clifton, K.; Cohen, H.J.; Extermann, M.; Gross, C.; Gupta, S.; Hollis, G.; Hubbard, J. NCCN guidelines® insights: Older adult oncology, version 1.2021: Featured updates to the NCCN guidelines. J. Natl. Compr. Cancer Netw. 2021, 19, 1006–1019. [Google Scholar] [CrossRef] [PubMed]
  34. Kelly, A.A.; Lewis, C.A.; Escalon, M.X. Inpatient Rehabilitation Issues Related to COVID-19. Phys. Med. Rehabil. Clin. N. Am. 2023, 34, 513–522. [Google Scholar] [CrossRef]
  35. Wilson, R.D.; Lewis, S.A.; Murray, P.K. Trends in the rehabilitation therapist workforce in underserved areas: 1980–2000. J. Rural. Health 2009, 25, 26–32. [Google Scholar] [CrossRef] [PubMed]
  36. Soubra, R.; Mourad-Chehade, F.; Chkeir, A. Automation of the timed up and go test using a doppler radar system for gait and balance analysis in elderly people. J. Healthc. Eng. 2023, 2023, 2016262. [Google Scholar] [CrossRef] [PubMed]
  37. Hernandez, H.H.C.; Ong, E.H.; Heyzer, L.; Tan, C.N.; Ghazali, F.; Yang, D.Z.; Jung, H.-W.; Ismail, N.H.; Lim, W.S. Validation of a multi-sensor-based kiosk in the use of the Short Physical Performance Battery in older adults attending a fall and balance clinic. Ann. Geriatr. Med. Res. 2022, 26, 125. [Google Scholar] [CrossRef]
  38. Duncan, L.; Gulati, P.; Giri, S.; Ostadabbas, S.; Mirbozorgi, S.A. Camera-based human gait speed monitoring and tracking for performance assessment of elderly patients with cancer. In Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Guadalajara, Mexico, 1–5 November 2021; IEEE: New York, NY, USA, 2021; pp. 3522–3525. [Google Scholar]
  39. Ponciano, V.; Pires, I.M.; Ribeiro, F.R.; Marques, G.; Garcia, N.M.; Pombo, N.; Spinsante, S.; Zdravevski, E. Is The Timed-Up and Go Test Feasible in Mobile Devices? A Systematic Review. Electronics 2020, 9, 528. [Google Scholar] [CrossRef]
  40. Hamamoto, R.; Suvarna, K.; Yamada, M.; Kobayashi, K.; Shinkai, N.; Miyake, M.; Takahashi, M.; Jinnai, S.; Shimoyama, R.; Sakai, A.; et al. Application of Artificial Intelligence Technology in Oncology: Towards the Establishment of Precision Medicine. Cancers 2020, 12, 3532. [Google Scholar] [CrossRef]
  41. Hamamoto, R.; Komatsu, M.; Takasawa, K.; Asada, K.; Kaneko, S. Epigenetics Analysis and Integrated Analysis of Multiomics Data, Including Epigenetic Data, Using Artificial Intelligence in the Era of Precision Medicine. Biomolecules 2020, 10, 62. [Google Scholar] [CrossRef]
  42. Yamada, M.; Saito, Y.; Imaoka, H.; Saiko, M.; Yamada, S.; Kondo, H.; Takamaru, H.; Sakamoto, T.; Sese, J.; Kuchiba, A.; et al. Development of a real-time endoscopic image diagnosis support system using deep learning technology in colonoscopy. Sci. Rep. 2019, 9, 14465. [Google Scholar] [CrossRef]
  43. Jinnai, S.; Yamazaki, N.; Hirano, Y.; Sugawara, Y.; Ohe, Y.; Hamamoto, R. The Development of a Skin Cancer Classification System for Pigmented Skin Lesions Using Deep Learning. Biomolecules 2020, 10, 1123. [Google Scholar] [CrossRef] [PubMed]
  44. Komatsu, M.; Sakai, A.; Komatsu, R.; Matsuoka, R.; Yasutomi, S.; Shozu, K.; Dozen, A.; Machino, H.; Hidaka, H.; Arakaki, T.; et al. Detection of Cardiac Structural Abnormalities in Fetal Ultrasound Videos Using Deep Learning. Appl. Sci. 2021, 11, 371. [Google Scholar] [CrossRef]
  45. Takahashi, S.; Takahashi, M.; Kinoshita, M.; Miyake, M.; Kawaguchi, R.; Shinojima, N.; Mukasa, A.; Saito, K.; Nagane, M.; Otani, R.; et al. Fine-Tuning Approach for Segmentation of Gliomas in Brain Magnetic Resonance Images with a Machine Learning Method to Normalize Image Differences among Facilities. Cancers 2021, 13, 1415. [Google Scholar] [CrossRef] [PubMed]
  46. Asada, K.; Kobayashi, K.; Joutard, S.; Tubaki, M.; Takahashi, S.; Takasawa, K.; Komatsu, M.; Kaneko, S.; Sese, J.; Hamamoto, R. Uncovering Prognosis-Related Genes and Pathways by Multi-Omics Analysis in Lung Cancer. Biomolecules 2020, 10, 524. [Google Scholar] [CrossRef]
  47. Kobayashi, K.; Bolatkan, A.; Shiina, S.; Hamamoto, R. Fully-Connected Neural Networks with Reduced Parameterization for Predicting Histological Types of Lung Cancer from Somatic Mutations. Biomolecules 2020, 10, 1249. [Google Scholar] [CrossRef]
  48. Takahashi, S.; Asada, K.; Takasawa, K.; Shimoyama, R.; Sakai, A.; Bolatkan, A.; Shinkai, N.; Kobayashi, K.; Komatsu, M.; Kaneko, S.; et al. Predicting Deep Learning Based Multi-Omics Parallel Integration Survival Subtypes in Lung Cancer Using Reverse Phase Protein Array Data. Biomolecules 2020, 10, 1460. [Google Scholar] [CrossRef]
  49. Hamamoto, R. Application of Artificial Intelligence for Medical Research. Biomolecules 2021, 11, 90. [Google Scholar] [CrossRef]
  50. Hamamoto, R.; Koyama, T.; Kouno, N.; Yasuda, T.; Yui, S.; Sudo, K.; Hirata, M.; Sunami, K.; Kubo, T.; Takasawa, K.; et al. Introducing AI to the molecular tumor board: One direction toward the establishment of precision medicine using large-scale cancer clinical and biological information. Exp. Hematol. Oncol. 2022, 11, 82. [Google Scholar] [CrossRef]
  51. FDA, Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices. Available online: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices (accessed on 13 May 2024).
  52. Hamamoto, R.; Takasawa, K.; Machino, H.; Kobayashi, K.; Takahashi, S.; Bolatkan, A.; Shinkai, N.; Sakai, A.; Aoyama, R.; Yamada, M.; et al. Application of non-negative matrix factorization in oncology: One approach for establishing precision medicine. Brief. Bioinform. 2022, 23, bbac246. [Google Scholar] [CrossRef]
  53. Polus, J.S.; Bloomfield, R.A.; Vasarhelyi, E.M.; Lanting, B.A.; Teeter, M.G. Machine learning predicts the fall risk of total hip arthroplasty patients based on wearable sensor instrumented performance tests. J. Arthroplast. 2021, 36, 573–578. [Google Scholar] [CrossRef]
  54. Friedrich, B.; Lau, S.; Elgert, L.; Bauer, J.M.; Hein, A. A deep learning approach for TUG and SPPB score prediction of (pre-) frail older adults on real-life IMU data. Healthcare 2021, 9, 149. [Google Scholar] [CrossRef] [PubMed]
  55. Bloomfield, R.A.; Broberg, J.S.; Williams, H.A.; Lanting, B.A.; McIsaac, K.A.; Teeter, M.G. Machine learning and wearable sensors at preoperative assessments: Functional recovery prediction to set realistic expectations for knee replacements. Med. Eng. Phys. 2021, 89, 14–21. [Google Scholar] [CrossRef] [PubMed]
  56. Zhuparris, A.; Maleki, G.; Koopmans, I.; Doll, R.J.; Voet, N.; Kraaij, W.; Cohen, A.; van Brummelen, E.; De Maeyer, J.H.; Groeneveld, G.J. Smartphone and Wearable Sensors for the Estimation of Facioscapulohumeral Muscular Dystrophy Disease Severity: Cross-sectional Study. JMIR Form. Res. 2023, 7, e41178. [Google Scholar] [CrossRef] [PubMed]
  57. Dubois, A.; Bihl, T.; Bresciani, J.-P. Identifying fall risk predictors by monitoring daily activities at home using a depth sensor coupled to machine learning algorithms. Sensors 2021, 21, 1957. [Google Scholar] [CrossRef] [PubMed]
  58. Hasegawa, S.; Mizokami, F.; Kameya, Y.; Hayakawa, Y.; Watanabe, T.; Matsui, Y. Machine learning versus binomial logistic regression analysis for fall risk based on SPPB scores in older adult outpatients. Digit. Health 2023, 9, 20552076231219438. [Google Scholar] [CrossRef]
  59. Kraus, M.; Stumpf, U.C.; Keppler, A.M.; Neuerburg, C.; Böcker, W.; Wackerhage, H.; Baumbach, S.F.; Saller, M.M. Development of a Machine Learning-Based Model to Predict Timed-Up-and-Go Test in Older Adults. Geriatrics 2023, 8, 99. [Google Scholar] [CrossRef]
  60. Sasani, K.; Catanese, H.N.; Ghods, A.; Rokni, S.A.; Ghasemzadeh, H.; Downey, R.J.; Shahrokni, A. Gait speed and survival of older surgical patient with cancer: Prediction after machine learning. J. Geriatr. Oncol. 2019, 10, 120–125. [Google Scholar] [CrossRef]
  61. Li, Y.; Zhang, P.; Zhang, Y.; Miyazaki, K. Gait analysis using stereo camera in daily environment. Berlin, Germany, 23–27 July 2019. IEEE: New York City, NY, USA, 2019; pp. 1471–1475. [Google Scholar]
  62. Hwang, J.; Lee, J.; Lee, K.-S. A deep learning-based method for grip strength prediction: Comparison of multilayer perceptron and polynomial regression approaches. PLoS ONE 2021, 16, e0246870. [Google Scholar] [CrossRef]
  63. Bae, J.-H.; Li, X.; Kim, T.; Bang, H.-S.; Lee, S.; Seo, D.Y. Prediction models of grip strength in adults above 65 years using Korean National Physical Fitness Award Data from 2009 to 2019. Eur. Geriatr. Med. 2023, 14, 1059–1064. [Google Scholar] [CrossRef]
  64. Supratak, A.; Datta, G.; Gafson, A.R.; Nicholas, R.; Guo, Y.; Matthews, P.M. Remote monitoring in the home validates clinical gait measures for multiple sclerosis. Front. Neurol. 2018, 9, 399303. [Google Scholar] [CrossRef]
  65. Soltani, A.; Dejnabadi, H.; Savary, M.; Aminian, K. Real-world gait speed estimation using wrist sensor: A personalized approach. IEEE J. Biomed. Health Inform. 2019, 24, 658–668. [Google Scholar] [CrossRef] [PubMed]
  66. Dobkin, B.H.; Xu, X.; Batalin, M.; Thomas, S.; Kaiser, W. Reliability and validity of bilateral ankle accelerometer algorithms for activity recognition and walking speed after stroke. Stroke 2011, 42, 2246–2250. [Google Scholar] [CrossRef] [PubMed]
  67. Mannini, A.; Sabatini, A.M. Walking speed estimation using foot-mounted inertial sensors: Comparing machine learning and strap-down integration methods. Med. Eng. Phys. 2014, 36, 1312–1321. [Google Scholar] [CrossRef] [PubMed]
  68. McGinnis, R.S.; Mahadevan, N.; Moon, Y.; Seagers, K.; Sheth, N.; Wright, J.A., Jr.; DiCristofaro, S.; Silva, I.; Jortberg, E.; Ceruolo, M. A machine learning approach for gait speed estimation using skin-mounted wearable sensors: From healthy controls to individuals with multiple sclerosis. PLoS ONE 2017, 12, e0178366. [Google Scholar] [CrossRef]
  69. Aziz, W.; Hussain, L.; Khan, I.R.; Alowibdi, J.S.; Alkinani, M.H. Machine learning based classification of normal, slow and fast walking by extracting multimodal features from stride interval time series. Math. Biosci. Eng. 2021, 18, 495–517. [Google Scholar] [CrossRef]
  70. Atrsaei, A.; Dadashi, F.; Mariani, B.; Gonzenbach, R.; Aminian, K. Toward a remote assessment of walking bout and speed: Application in patients with multiple sclerosis. IEEE J. Biomed. Health Inform. 2021, 25, 4217–4228. [Google Scholar] [CrossRef]
  71. Juen, J.; Cheng, Q.; Schatz, B. A natural walking monitor for pulmonary patients using mobile phones. IEEE J. Biomed. Health Inform. 2015, 19, 1399–1405. [Google Scholar] [CrossRef]
  72. Aziz, O.; Zihajehzadeh, S.; Park, A.; Tae, C.-G.; Park, E.J. Improving energy expenditure estimation through activity classification and walking speed estimation using a smartwatch. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; IEEE: New York, NY, USA, 2020; pp. 3940–3944. [Google Scholar]
  73. Lee, D.W.; Han, H.-S.; Lee, M.C.; Ro, D.H. Prediction of Postoperative Gait Speed Change after Bilateral Primary Total Knee Arthroplasty in Female Patients Using a Machine Learning Algorithm. Orthop. Traumatol. Surg. Res. 2024, 110, 103842. [Google Scholar] [CrossRef]
  74. Davis, J.R.C.; Knight, S.P.; Donoghue, O.A.; Hernández, B.; Rizzo, R.; Kenny, R.A.; Romero-Ortuno, R. Comparison of gait speed reserve, usual gait speed, and maximum gait speed of adults aged 50+ in Ireland using explainable machine learning. Front. Netw. Physiol. 2021, 1, 754477. [Google Scholar] [CrossRef]
  75. Sikandar, T.; Rabbi, M.F.; Ghazali, K.H.; Altwijri, O.; Alqahtani, M.; Almijalli, M.; Altayyar, S.; Ahamed, N.U. Using a deep learning method and data from two-dimensional (2D) marker-less video-based images for walking speed classification. Sensors 2021, 21, 2836. [Google Scholar] [CrossRef]
  76. Chen, H.-C.; Sunardi; Liau, B.-Y.; Lin, C.-Y.; Akbari, V.B.H.; Lung, C.-W.; Jan, Y.-K. Estimation of various walking intensities based on wearable plantar pressure sensors using artificial neural networks. Sensors 2021, 21, 6513. [Google Scholar] [CrossRef] [PubMed]
  77. Kidziński, Ł.; Yang, B.; Hicks, J.L.; Rajagopal, A.; Delp, S.L.; Schwartz, M.H. Deep neural networks enable quantitative movement analysis using single-camera videos. Nat. Commun. 2020, 11, 4054. [Google Scholar] [CrossRef] [PubMed]
  78. Lonini, L.; Moon, Y.; Embry, K.; Cotton, R.J.; McKenzie, K.; Jenz, S.; Jayaraman, A. Video-based pose estimation for gait analysis in stroke survivors during clinical assessments: A proof-of-concept study. Digit. Biomark. 2022, 6, 9–18. [Google Scholar] [CrossRef] [PubMed]
  79. Carl von Ossietzky Universität Oldenburg, O.T.A.G.O. Available online: https://uol.de/en/amt/research/projects/otago (accessed on 5 September 2024).
  80. Merriaux, P.; Dupuis, Y.; Boutteau, R.; Vasseur, P.; Savatier, X. A study of vicon system positioning performance. Sensors 2017, 17, 1591. [Google Scholar] [CrossRef]
  81. Donoghue, O.A.; McGarrigle, C.A.; Foley, M.; Fagan, A.; Meaney, J.; Kenny, R.A. Cohort profile update: The Irish longitudinal study on ageing (TILDA). Int. J. Epidemiol. 2018, 47, 1398–1398l. [Google Scholar] [CrossRef]
  82. Makihara, Y.; Mannami, H.; Tsuji, A.; Hossain, M.A.; Sugiura, K.; Mori, A.; Yagi, Y. The OU-ISIR gait database comprising the treadmill dataset. IPSJ Trans. Comput. Vis. Appl. 2012, 4, 53–62. [Google Scholar] [CrossRef]
  83. Stone, E.; Skubic, M.; Rantz, M.; Abbott, C.; Miller, S. Average in-home gait speed: Investigation of a new metric for mobility and fall risk assessment of elders. Gait Posture 2015, 41, 57–62. [Google Scholar] [CrossRef]
  84. Friedrich, B.; Steen, E.-E.; Fudickar, S.; Hein, A. Analysing the Correlation of Geriatric Assessment Scores and Activity in Smart Homes. arXiv preprint 2021, arXiv:2103.05971. [Google Scholar] [CrossRef]
  85. Bohannon, R.W. Minimal clinically important difference for grip strength: A systematic review. J. Phys. Ther. Sci. 2019, 31, 75–78. [Google Scholar] [CrossRef]
  86. Pavasini, R.; Guralnik, J.; Brown, J.C.; di Bari, M.; Cesari, M.; Landi, F.; Vaes, B.; Legrand, D.; Verghese, J.; Wang, C.; et al. Short Physical Performance Battery and all-cause mortality: Systematic review and meta-analysis. BMC Med. 2016, 14, 215. [Google Scholar] [CrossRef]
  87. Robinson, T.N.; Wu, D.S.; Sauaia, A.; Dunn, C.L.; Stevens-Lapsley, J.E.; Moss, M.; Stiegmann, G.V.; Gajdos, C.; Cleveland, J.C., Jr.; Inouye, S.K. Slower walking speed forecasts increased postoperative morbidity and 1-year mortality across surgical specialties. Ann. Surg. 2013, 258, 582–588; discussion 588–590. [Google Scholar] [CrossRef] [PubMed]
  88. Wada, Y.; Shojima, K.; Tamaki, K.; Mori, T.; Kusunoki, H.; Onishi, M.; Tsuji, S.; Matsuzawa, R.; Nagai, K.; Sano, K. Association Between Timed Up-and-Go Test and Future Changes in the Frailty Status in a Longitudinal Study of Japanese Community-Dwelling Older Adults. Clin. Interv. Aging 2023, 18, 1191–1200. [Google Scholar] [CrossRef] [PubMed]
Bioengineering 11 01154 g001
Figure 1. Challenges of objective physical function assessments in clinical settings. Key obstacles include: (A) space constraints—limited room for assessments; (B) time demands—assessments require significant time; (C) human resources—shortage of trained personnel; (D) specialized instruments—need for specific equipment; and (E) fall risk—ensuring patient safety and managing fall risk. These barriers make single assessments difficult during clinical courses. Additionally, the manual nature of these evaluations can lead to interrater variability, complicating their broader implementation.
Figure 1. Challenges of objective physical function assessments in clinical settings. Key obstacles include: (A) space constraints—limited room for assessments; (B) time demands—assessments require significant time; (C) human resources—shortage of trained personnel; (D) specialized instruments—need for specific equipment; and (E) fall risk—ensuring patient safety and managing fall risk. These barriers make single assessments difficult during clinical courses. Additionally, the manual nature of these evaluations can lead to interrater variability, complicating their broader implementation.
Bioengineering 11 01154 g001
Bioengineering 11 01154 g002
Figure 2. Utilization of sensor data to assess physical function. (A) IMU or GNSS: used for TUG classification, SPPB regression, and walking speed regression/classification. (B) Smartphone: employed for TUG and walking speed regression. (C) Smartwatch: applied for walking speed regression. (D) Smart home sensor: utilized for TUG classification and walking speed regression data collection in facilities and residences spanning from 8 h to several weeks for continuous monitoring. (E) Optical motion sensor: used for walking speed classification.
Figure 2. Utilization of sensor data to assess physical function. (A) IMU or GNSS: used for TUG classification, SPPB regression, and walking speed regression/classification. (B) Smartphone: employed for TUG and walking speed regression. (C) Smartwatch: applied for walking speed regression. (D) Smart home sensor: utilized for TUG classification and walking speed regression data collection in facilities and residences spanning from 8 h to several weeks for continuous monitoring. (E) Optical motion sensor: used for walking speed classification.
Bioengineering 11 01154 g002
Bioengineering 11 01154 g003
Figure 3. Utilization of images, videos, and tabular data in assessing physical function. (A) Image: walking silhouettes are used for walking speed classification. These images are from the OU-ISIR Dataset A. Plantar pressure is also used for walking speed classification. Plantar foot pressure images, including the regions of interest (T1, M1, M2, and HL), were classified using a convolutional neural network (CNN) at different walking speeds and times. (B) Video: walking in front of the camera, utilized for TUG and walking speed regression. Common pose estimation models, such as OpenPose and Mask R-CNN, were used for keypoint extraction. (C) Tabular data from big data and EHR, among others; used for TUG classification, grip strength regression, and walking speed regression.
Figure 3. Utilization of images, videos, and tabular data in assessing physical function. (A) Image: walking silhouettes are used for walking speed classification. These images are from the OU-ISIR Dataset A. Plantar pressure is also used for walking speed classification. Plantar foot pressure images, including the regions of interest (T1, M1, M2, and HL), were classified using a convolutional neural network (CNN) at different walking speeds and times. (B) Video: walking in front of the camera, utilized for TUG and walking speed regression. Common pose estimation models, such as OpenPose and Mask R-CNN, were used for keypoint extraction. (C) Tabular data from big data and EHR, among others; used for TUG classification, grip strength regression, and walking speed regression.
Bioengineering 11 01154 g003
Table 1. Objective physical function assessment with ML technology.
Table 1. Objective physical function assessment with ML technology.
ReferenceMain Device to Obtain Input DataDetails of Input Variable or DeviceLabel Setting/Label Measurement MethodOutput Label/Sort of TaskML TechnologyValidation MethodMetrics from Best ModelBaseline CharacteristicsConcept
Polus et al. [53] IMU4 sensors during TUG: above and below each knee before and 2 weeks after THATUG > 14(6 weeks after THA)TUG/classificationLDA, SVM10-fold CVLDA: Accuracy 0.8772 patients undergoing THAPreventing falls by predicting their risk based on TUG
Friedrich et al. [54]IMUSingle sensor on the right side of the hipSPPB: score itself
TUG: <10, 11–19, 20–29		SPPB/regression
TUG/classification		LSTM+CNNTrain–val–testAccuracy (TUG) 95.9%
Accuracy (SPPB) 94.3%		20 older patients (OTAGO study)Predicting TUG on real-life IMU data
Bloomfield et al. [55]IMU+EHR·4 sensors: above and below each knee during TUG
·Clinical information
·Patient-reported subjective measures		(Preoperative TUG—postoperative)
>2.27		TUG/classificationSVM, NB, RF, 10-fold CVRF: Accuracy 0.8082 patients undergoing TKAPredicting functional recovery for appropriately adjusting patient expectations
Zhuparris et al. [56]Smartphone·Health-related data from smartphone
·Sensor in smartphone		TUG score itselfTUG/regressionElastic Net, RF, xgBoost5-fold CVElastic Net:
R2 0.59 		38 patients with FSHDQuantifying FSHD progression with TUG
Dubois et al. [57]Depth sensorKinect V2 placed in each room of the rehabilitation centerTUG ≥ 13.5 sTUG/classificationAdaBoost, NB, KNN, SVM, RF, NNLeave-one-out CVKNN, NN: Accuracy 1.030 older patients in a rehabilitation centerPreventing fall with home-sensor data
Hasegawa et al. [58]EHR·Clinical information mainly from EHR
·Physical measurements		SPPB ≤ 6(men)/≤9(women) as fall riskSPPB/classificationPrediction One. Ver3.0.1.3 (SONY)
BLRA		Train–test splitPrediction One:
Accuracy 0.74 		797 older patients at frailty outpatient serviceComparing model performance of predicting fall risk based on SPPB
Kraus et al. [59]EHRClinical information from HERTUG score itselfTUG/regressionGLM, SVM, RF, xgBoost5-fold CVRF: MAE 2.7103 orthogeriatric patientsPredicting TUG without mobility data
Sasani et al. [60]Tabular dataComponents of GATUG < 10 s, TUG ≥ 10 s, uncertainTUG/classificationDecision Tree ClassifierNoneDecision Tree Classifier: Accuracy 78%1901 old patients undergoing cancer surgeryPredicting accurately TUG score with ML
Li et al. [61]VideoStereo cameraTUG score itselfTUG/regressionMask R-CNN+
polynomial regression		NoneRE <0.1 (20 participants in 40)40 older adults in a daycare facilityAssessing the health status of the older patients with TUG
Hwang et al. [62]Tabular dataVariables from physical profile and body part measurements (not from EHR)Grip strength score itselfGrip strength/regressionMLP regression and different polynomial regressions K-fold CVMLP regression: correlation 0.88164 healthy young volunteersPredicting grip strength accurately to reduce the risk of upper extremity disorder
Bae et al. [63]Big DataTabular data from Korean National Fitness Award Data from 2009 to 2019Grip strength score itselfGrip strength/regressionLR, LASSO, Ridge, RF, xGBoost, Light GBM, CatBoost5-fold CVCatBoost: MSE 16.6107,290 participants aged over 65Predicting grip strength without measuring
Supratak et al. [64]IMUSingle sensor on the lower back 25-foot walking test in clinicWalking speed/
regression		SVRCorrelationCorrelation 0.9832 young patients with MSValidating gait speed at home against a 25-foot walking test
Soltani et al. [65]IMU+GNSS2 sensors: on each wristWalking speed measured by GNSSWalking speed/
regression		LASSO (feature extraction)CVRMSE 0.0540 healthy young volunteersEstimating walking speed with personalization
Dobkin et al. [66]IMU2 sensors: above each ankleWalking speed measured by stopwatchWalking speed/
regression		Sensor system (Medical Daily Activity Wireless Network algorithm)CorrelationCorrelation 0.9812 patients with stroke
6 healthy participants		Acquiring quantitative data on daily performance
Mannini et al. [67]IMUSingle sensor on the right shoeWalking speed manually measured Walking speed/
regression		·Hidden Markov model
·Strap-down integration
·LR		Leave-one-out CVR2 0.9623 healthy adultsExploring the ML method to predict walking speed
McGinnis et al. [68]IMU5 sensors: on sacrum, bilateral thigh, and bilateral shank6 min walking test on a treadmillWalking speed/
regression		SVRLeave-one-out CVRMSE 0.12 (patients with MS)17 healthy participants
30 patients with MS 		Resolving the hurdle of assessing walking speed
Aziz et al. [69]IMUSingle sensor inside one shoeSlow/normal/fast speedWalking speed/
classification		RF, xgBoost, SVMTrain–test splitRF: Accuracy 1.010 healthy menAnalyzing gait patterns of aged people
Atrsaei et al. [70]IMUSingle sensor on the waist10 m walk testWalking speed/
regression		GPRLeave-one-out CVRMSE 1.1035 participants with MSPredicting walking speed at home with IMU
Juen et al. [71]SmartphoneSmartphone in waist belt at L36 min walking testWalking speed/
regression		SVM, GPRLeave-one-out CVSVM: Error 3.2328 patients with pulmonary disease
10 healthy participants		Monitoring individual health status continuously
Aziz et al. [72]Smartwatch Smartwatch on the right wristSpeed during treadmill walking: 0.5, 0.75, 1.0, 1.25, 1.5, 1.75 m/sWalking speed/
regression		GPRNoneMAPE 4% (best, 1.0 m/s)10 healthy young adultsAssessing walking speed for preventing chronic diseases
Lee et al. [73]Optical motion capture+ EHR·Clinical information from EHR
·Variables extracted from optical motion capture		The difference between post/pre-operative gait speedWalking speed/
classification		GBM10-fold CVAUC 0.86128 female patients undergoing bilateral TKAPredicting postoperative walking speed by preoperative clinical variable
Davis et al. [74]Big DataTabular data GSR = MGS—UGSWalking speed/
regression		HGBR5-fold CVR2 0.213925 participants from TILDA wave3Predicting gait speed from population statistical data
Sikandar et al. [75]Image5 ratio-based body measurement from marker free video imagesSlow (2 to 3 km/h), normal (4 to 5 km/h), and fast (6 to 7 km/h)Walking speed/
classification		BiLSTM17-fold CVAccuracy 92.79%34 participants (OU-ISIR dataset A)Classifying walking speed with body measurements
Chen et al. [76]ImagePlantar region pressure images(0.8, 1.6, 2.4 m/s) and (10, 20 min)Walking speed/
classification		ROI+CNNTrain–test splitF1-score: 1.00 (first toe, 2.4 m/s for 10 min)12 healthy young participantsDetecting appropriate exercise intensity
Kidzinski et al. [77]VideoTimeline keypoint data derived from OpenPoseWalking speed measured by the VICON systemWalking speed/
regression		OpenPose+ (CNN/RF/Ridge)Train–val–testOpenPose+CNN: Correlation 0.731026 pediatric patients with cerebral palsySimplifying the quantitative gait assessment
Lonini et al. [78]VideoBelow-waist videos of patients recorded by normal cameraWalking speed measured by GAITRiteWalking speed/
regression		DeepLabCut(ResNet based)Leave-one-out CVCorrelation 0.92eight patients with stroke Predicting the walking speed of patients with stroke without expensive instrument
Abbreviations: ML, machine learning; IMU, inertial measurement unit; EHR, electronic health record: GA, geriatric assessment; TUG, Timed Up and Go test; SPPB, Short Physical Performance Battery; LDA, linear discriminant analysis classifier; SVM, support vector machine; LSTM, long short-term memory; BiLSTM, bidirectional long short-term memory; CNN, convolutional neural network; NB, naive Bayes classifier; RF, random forest; xgBoost, eXtreme gradient boosting; AdaBoost, adaptive boosting; KNN, k-nearest neighbors; NN, neural network; BLRA, binomial logistic regression analysis; GLM, generalized linear model; MLP, multilayer perceptron regression; LR, linear regression; GBM, gradient boosting machine; CatBoost, categorical boosting; SVR, support vector regression; GPR, gaussian process regression; HGBR, histogram gradient boosting regression; CV, cross validation; ROI, region of interest; MAE, mean absolute error; MSE, mean squared error; RMSE, root mean square error; RE, relative error; MAPE, mean absolute percentage error; AUC, area under the curve; R2, R-squared value; ICC, intraclass correlation coefficient; GSR, gait speed reserve; MGS, maximum gait speed; UGS, usual gait speed; GNSS, global navigation satellite systems; TKA, total knee arthroplasty; THA, total hip arthroplasty; FSHD, facioscapulohumeral muscular dystrophy; MS, multiple sclerosis; OU-ISIR, Osaka University Institute of Scientific and Industrial Research; TILDA, The Irish Longitudinal Study on Aging.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kouno, N.; Takahashi, S.; Komatsu, M.; Sakaguchi, Y.; Ishiguro, N.; Takeda, K.; Fujioka, K.; Matsuoka, A.; Fujimori, M.; Hamamoto, R. Introduction of AI Technology for Objective Physical Function Assessment. Bioengineering 2024, 11, 1154. https://doi.org/10.3390/bioengineering11111154

AMA Style

Kouno N, Takahashi S, Komatsu M, Sakaguchi Y, Ishiguro N, Takeda K, Fujioka K, Matsuoka A, Fujimori M, Hamamoto R. Introduction of AI Technology for Objective Physical Function Assessment. Bioengineering. 2024; 11(11):1154. https://doi.org/10.3390/bioengineering11111154

Chicago/Turabian Style

Kouno, Nobuji, Satoshi Takahashi, Masaaki Komatsu, Yusuke Sakaguchi, Naoaki Ishiguro, Katsuji Takeda, Kyoko Fujioka, Ayumu Matsuoka, Maiko Fujimori, and Ryuji Hamamoto. 2024. "Introduction of AI Technology for Objective Physical Function Assessment" Bioengineering 11, no. 11: 1154. https://doi.org/10.3390/bioengineering11111154

APA Style

Kouno, N., Takahashi, S., Komatsu, M., Sakaguchi, Y., Ishiguro, N., Takeda, K., Fujioka, K., Matsuoka, A., Fujimori, M., & Hamamoto, R. (2024). Introduction of AI Technology for Objective Physical Function Assessment. Bioengineering, 11(11), 1154. https://doi.org/10.3390/bioengineering11111154

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop