Questioning Domain Adaptation in Myoelectric Hand Prostheses Control: An Inter- and Intra-Subject Study
2. Materials and Methods
2.1. Domain Adaptation and Transfer Learning Algorithms
2.1.1. Source, Target and Test Sets
2.1.2. Domain Adaptation Algorithms
- Structure Expansion/Reduction (SER): Given a random forest induced using the source data , each decision tree (DT) is processed independently by the SER algorithm. First of all, the set of all labeled points in the target data that reach the node v is computed. Then, in the expansion phase, a full tree expands from each leaf v with respect to . Lastly, with a bottom-up approach, the algorithm performs a reduction of the structure for each internal node v.This reduction is determined by two kinds of errors with respect to :
The subtree error is the empirical error of the subtree of which the root is v. The leaf error is defined to be the empirical error on v if it were to be pruned into a leaf. If the following condition holds:
- Subtree error ;
- Leaf error .
- Structure Transfer (STRUT): While the SER algorithm acts on size of DTs inside , the Structure Transfer algorithm changes the threshold. Since decision trees show similarity for similar problems, the STRUT algorithm exploits a top-down approach, adapting a DT trained on the source samples to the target samples by discarding all numeric threshold values in the tree. The values of the numeric thresholds are substituted by new thresholds for a node v using the subset of target examples that reach v.If is empty for a node v, v is pruned because it cannot be reached in the target domain. At each leaf, the final decision value is computed on the target training data.To perform threshold selection for feature , STRUT uses two parameters:
DG determines the distributional similarity, while IG quantifies the informative value of the threshold. The similarity is related only to those thresholds x whose IG is larger than the IG of any other in the -neighborhood of x for any sufficiently small . The STRUT algorithm searches for a threshold that gives a high similarity between the induced and the original distributions during the tree induction stage .The selection of the threshold can be considered as an optimization problem:
- Divergence gain DG;
- Information gain IG.
- MIX: Once both SER and STRUT are applied, we obtain two distinct forests as a result. MIX is a combination of the two previous algorithms. This is a simple majority voting ensemble applied to all decision trees of both forests generated by STRUT and SER. As can be seen from the results, MIX does not simply average the results of the previous algorithms but often outperforms both of its constituents and thus is the second best solution. An intuitive explanation described in  about this result is given in Results.
2.2. Experimental Setup
- NinaPro DB2 and DB3-Single acquisitions of intact subjects and transradial amputees: The two datasets include 40 intact subjects and 11 amputees. Three amputees are excluded because their data acquisitions are not complete. Each subject executed 40 movements 6 times. Each movement repetition lasted 5 s and was followed by 3 s of rest. Twelve Delsys Trigno Wireless electrodes were used to record the sEMG data from the forearm of the subjects. Following a recently employed approach , a sliding window of 200 ms and an increment of 10 ms were used for the extraction of signal features. Therefore, the time windows were split into train and test sets for the classifiers, considering repetitions (1,3,4,6) for the training and repetitions 2 and 5 for test. A factor of 10 at regular intervals was used to subsample the training set in order to reduce the computational demands.
- DB6-Intact subjects acquired for 5 consecutive days: This dataset is composed of 10 intact subjects and targets the analysis of data acquisition repeatability in the same subjects. Each subject executed 7 hand grasps 12 times, two times per day, for 5 consecutive days. Each grasp is followed with a few seconds of rest. During the acquisition of the movements, fourteen electrodes recorded sEMG data. Eight electrodes are positioned as the first eight electrodes in NinaPro DB2 and DB3 (i.e., equally spaced around the forearm at the height of the radio-humeral joint). The windowing procedure follows the same approach described for the previous datasets. For each session, repetitions (1,3,4,6,7,9,10,12) were dedicated to training, while repetitions (2,5,8,11) were used as test. In this case, the training set was also subsampled by a factor of 10 at regular intervals.
2.2.2. Experiment Settings
- Source only;
- Target only;
- Voting ensemble (SER, Target Only);
- Voting ensemble (STRUT, Target Only);
- Voting ensemble (MIX, Target Only).
- DB2 and DB3: The first experiments replicate the experiments previously cited:
In the training set, the subsets from 1 to 4 repetitions were taken into account for training. In each case, the k-fold cross validation was used for the optimization of the target model, with each fold corresponding to samples of one repetition. The source models, instead, were trained using all repetitions.
- Intact–Intact: the classification of each subject from DB2 exploits prior knowledge of remaining subjects of DB2.
- Amputee–Intact: the classification of each subject from DB3 exploits prior knowledge of remaining subjects of DB3 plus all subject of DB2.
- Amputee–Amputee: the classification of each subject from DB3 exploits prior knowledge of remaining subjects of DB3.
- DB6: Due to the very high number of repetitions for each subject (120), two simplified experimental settings were chosen:
- Intra-subject: each subject of DB6 exploits prior knowledge of the remaining repetitions of the same subject.
- Inter-subject: each subject of DB6 exploits prior knowledge of the remaining repetitions of the same subject plus all remaining subjects of DB6.
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
- Young, A.J.; Hargrove, L.J.; Kuiken, T.A. The effects of electrode size and orientation on the sensitivity of myoelectric pattern recognition systems to electrode shift. IEEE Trans. Biomed. Eng. 2011, 58, 2537–2544. [Google Scholar] [CrossRef][Green Version]
- Waris, A.; Niazi, I.K.; Jamil, M.; Gilani, O.; Englehart, K.; Jensen, W.; Shafique, M.; Kamavuako, E.N. The effect of time on EMG classification of hand motions in able-bodied and transradial amputees. J. Electromyogr. Kinesiol. 2018, 40, 72–80. [Google Scholar] [CrossRef][Green Version]
- Raichle, K.A.; Hanley, M.A.; Molton, I.; Kadel, N.J.; Campbell, K.; Phelps, E.; Ehde, D.; Smith, D.G. Prosthesis use in persons with lower-and upper-limb amputation. J. Rehabil. Res. Dev. 2008, 45, 961. [Google Scholar] [CrossRef]
- Atzori, M.; Rosanda, E.; Pajardi, G.; Bassetto, F.; Müller, H. Effect of movement type on the classification of electromyography data for the control of dexterous prosthetic hands. In Proceedings of the 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), New York, NY, USA, 29 November–1 December 2020; pp. 746–751. [Google Scholar]
- Yang, D.; Gu, Y.; Thakor, N.V.; Liu, H. Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration. Exp. Brain Res. 2019, 237, 291–311. [Google Scholar] [CrossRef]
- Parker, P.; Englehart, K.; Hudgins, B. Myoelectric signal processing for control of powered limb prostheses. J. Electromyogr. Kinesiol. 2006, 16, 541–548. [Google Scholar] [CrossRef]
- Englehart, K.; Hudgins, B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 2003, 50, 848–854. [Google Scholar] [CrossRef] [PubMed]
- Salminger, S.; Stino, H.; Pichler, L.H.; Gstoettner, C.; Sturma, A.; Mayer, J.A.; Szivak, M.; Aszmann, O.C. Current rates of prosthetic usage in upper-limb amputees–have innovations had an impact on device acceptance? Disabil. Rehabil. 2020, 1–12. [Google Scholar] [CrossRef] [PubMed]
- Farina, D.; Jiang, N.; Rehbaum, H.; Holobar, A.; Graimann, B.; Dietl, H.; Aszmann, O.C. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: Emerging avenues and challenges. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 797–809. [Google Scholar] [CrossRef]
- Kyrarini, M.; Lygerakis, F.; Rajavenkatanarayanan, A.; Sevastopoulos, C.; Nambiappan, H.R.; Chaitanya, K.K.; Babu, A.R.; Mathew, J.; Makedon, F. A survey of robots in healthcare. Technologies 2021, 9, 8. [Google Scholar] [CrossRef]
- Roche, A.D.; Rehbaum, H.; Farina, D.; Aszmann, O.C. Prosthetic myoelectric control strategies: A clinical perspective. Curr. Surg. Rep. 2014, 2, 1–11. [Google Scholar] [CrossRef]
- Rong, G.; Mendez, A.; Assi, E.B.; Zhao, B.; Sawan, M. Artificial intelligence in healthcare: Review and prediction case studies. Engineering 2020, 6, 291–301. [Google Scholar] [CrossRef]
- Atzori, M.; Müller, H. Control capabilities of myoelectric robotic prostheses by hand amputees: A scientific research and market overview. Front. Syst. Neurosci. 2015, 9, 162. [Google Scholar] [CrossRef] [PubMed]
- Oskoei, M.A.; Hu, H. Myoelectric control systems—A survey. Biomed. Signal Process. Control 2007, 2, 275–294. [Google Scholar] [CrossRef]
- Pale, U.; Atzori, M.; Müller, H.; Scano, A. Variability of muscle synergies in hand grasps: Analysis of intra-and inter-session data. Sensors 2020, 20, 4297. [Google Scholar] [CrossRef]
- Young, A.J.; Hargrove, L.J.; Kuiken, T.A. Improving myoelectric pattern recognition robustness to electrode shift by changing interelectrode distance and electrode configuration. IEEE Trans. Biomed. Eng. 2011, 59, 645–652. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Pan, S.J.; Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 2009, 22, 1345–1359. [Google Scholar] [CrossRef]
- Ben-David, S.; Blitzer, J.; Crammer, K.; Kulesza, A.; Pereira, F.; Vaughan, J.W. A theory of learning from different domains. Mach. Learn. 2010, 79, 151–175. [Google Scholar] [CrossRef][Green Version]
- Côté-Allard, U.; Fall, C.L.; Drouin, A.; Campeau-Lecours, A.; Gosselin, C.; Glette, K.; Laviolette, F.; Gosselin, B. Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 760–771. [Google Scholar] [CrossRef][Green Version]
- Zhao, H.; Zhang, S.; Wu, G.; Costeira, J.P.; Moura, J.M.F.; Gordon, G.J. Adversarial multiple source domain adaptation. Adv. Neural Inf. Process. Syst. 2018, 31, 8559–8570. [Google Scholar]
- Castellini, C.; Fiorilla, A.E.; Sandini, G. Multi-subject/daily-life activity EMG-based control of mechanical hands. J. Neuroeng. Rehabil. 2009, 6, 1–11. [Google Scholar] [CrossRef][Green Version]
- Sensinger, J.W.; Lock, B.A.; Kuiken, T.A. Adaptive pattern recognition of myoelectric signals: Exploration of conceptual framework and practical algorithms. IEEE Trans. Neural Syst. Rehabil. Eng. 2009, 17, 270–278. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Kanoga, S.; Kanemura, A. Transfer learning over time and position in wearable myoelectric control systems. In Proceedings of the 2018 IEEE International Conference on Industrial Technology (ICIT), Honolulu, HI, USA, 18–21 July 2018; pp. 1478–1483. [Google Scholar]
- Kanoga, S.; Matsuoka, M.; Kanemura, A. Assessing the effect of transfer learning on myoelectric control systems with three electrode positions. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Lyon, France, 20–22 February 2018; pp. 1–4. [Google Scholar]
- Ameri, A.; Akhaee, M.A.; Scheme, E.; Englehart, K. A deep transfer learning approach to reducing the effect of electrode shift in EMG pattern recognition-based control. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 28, 370–379. [Google Scholar] [CrossRef] [PubMed]
- Wang, W.; Chen, B.; Xia, P.; Hu, J.; Peng, Y. Sensor fusion for myoelectric control based on deep learning with recurrent convolutional neural networks. Artif. Organs 2018, 42, E272–E282. [Google Scholar] [CrossRef] [PubMed]
- Liu, J.; Sheng, X.; Zhang, D.; He, J.; Zhu, X. Reduced daily recalibration of myoelectric prosthesis classifiers based on domain adaptation. IEEE J. Biomed. Health Inform. 2014, 20, 166–176. [Google Scholar] [CrossRef]
- Prahm, C.; Schulz, A.; Paaßen, B.; Schoisswohl, J.; Kaniusas, E.; Dorffner, G.; Hammer, B.; Aszmann, O. Counteracting electrode shifts in upper-limb prosthesis control via transfer learning. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 956–962. [Google Scholar] [CrossRef][Green Version]
- Segev, N.; Harel, M.; Mannor, S.; Crammer, K.; El-Yaniv, R. Learn on source, refine on target: A model transfer learning framework with random forests. IEEE Trans. Pattern Anal. Mach. Intell. 2016, 39, 1811–1824. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Patricia, N.; Tommasit, T.; Caputo, B. Multi-source adaptive learning for fast control of prosthetics hand. In Proceedings of the 2014 22nd International Conference on Pattern Recognition, Stockholm, Sweden, 24–28 August 2014; pp. 2769–2774. [Google Scholar]
- Atzori, M.; Gijsberts, A.; Castellini, C.; Caputo, B.; Hager, A.M.; Elsig, S.; Giatsidis, G.; Bassetto, F.; Müller, H. Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 2014, 1, 1–13. [Google Scholar] [CrossRef][Green Version]
- Gregori, V.; Gijsberts, A.; Caputo, B. Adaptive learning to speed-up control of prosthetic hands: A few things everybody should know. In Proceedings of the 2017 International Conference on Rehabilitation Robotics (ICORR), London, UK, 17–20 July 2017; pp. 1130–1135. [Google Scholar]
- Liarokapis, M.V.; Artemiadis, P.K.; Kyriakopoulos, K.J.; Manolakos, E.S. A learning scheme for reach to grasp movements: On EMG-based interfaces using task specific motion decoding models. IEEE J. Biomed. Health Inform. 2013, 17, 915–921. [Google Scholar] [CrossRef]
- Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef][Green Version]
- Hunt, E.B.; Marin, J.; Stone, P.J. Experiments in Induction; Academic Press: Cambridge, MA, USA, 1966. [Google Scholar]
- Palermo, F.; Cognolato, M.; Gijsberts, A.; Müller, H.; Caputo, B.; Atzori, M. Repeatability of grasp recognition for robotic hand prosthesis control based on sEMG data. In Proceedings of the 2017 International Conference on Rehabilitation Robotics (ICORR), London, UK, 17–20 July 2017; pp. 1154–1159. [Google Scholar]
- Kuzborskij, I.; Gijsberts, A.; Caputo, B. On the challenge of classifying 52 hand movements from surface electromyography. In Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA, 28 August–1 September 2012; pp. 4931–4937. [Google Scholar]
- Kim, M.; Kim, K.; Chung, W.K. Simple and fast compensation of sEMG interface rotation for robust hand motion recognition. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 2397–2406. [Google Scholar] [CrossRef]
|DB||N° Sensors||N° Gestures||Repetitions|
|DB2||Delsys Trigno sEMG × 12||40||6|
|DB3||Delsys Trigno sEMG × 12||40||6|
|DB6||Delsys Trigno sEMG × 14(8)||7||12 × 2 × 5|
|DB2 and DB3|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Marano, G.; Brambilla, C.; Mira, R.M.; Scano, A.; Müller, H.; Atzori, M. Questioning Domain Adaptation in Myoelectric Hand Prostheses Control: An Inter- and Intra-Subject Study. Sensors 2021, 21, 7500. https://doi.org/10.3390/s21227500
Marano G, Brambilla C, Mira RM, Scano A, Müller H, Atzori M. Questioning Domain Adaptation in Myoelectric Hand Prostheses Control: An Inter- and Intra-Subject Study. Sensors. 2021; 21(22):7500. https://doi.org/10.3390/s21227500Chicago/Turabian Style
Marano, Giulio, Cristina Brambilla, Robert Mihai Mira, Alessandro Scano, Henning Müller, and Manfredo Atzori. 2021. "Questioning Domain Adaptation in Myoelectric Hand Prostheses Control: An Inter- and Intra-Subject Study" Sensors 21, no. 22: 7500. https://doi.org/10.3390/s21227500