Analysis of Man-Machine Interfaces in Upper-Limb Prosthesis: A Review

Abstract

This paper compiles and analyzes some of the most current works related to upper limb prosthesis with emphasis on man-machine interfaces. A brief introduction of the basic subjects is given to explain what a prosthesis is, what types of prostheses exist, what they serve for, how they communicate with the user (control and feedback), and what technologies are involved. The method used in this review is also discussed, as well as the cataloging process and analysis of articles for the composition of this review. Each article is analyzed individually and its results are presented in a succinct way, in order to facilitate future research and serve as a source for professionals related to the area of prosthesis, such as doctors, engineers, researchers, and anyone interested in this subject. Finally, the needs and difficulties of the current prostheses, as well as the negative and positive points in the results are analyzed, and the progress achieved so far is discussed.

1. Introduction

Orthopedic prostheses, which were initially purely aesthetic, have over time gained more and more functionality with technological advancement. They have become very useful tools for people who are amputees or have some congenital limb defect. Consequently, prostheses today resemble the replaced limb more and more and improve the lives of these people.

What may be the oldest prosthesis in the world, was found on the foot of the mummy of an Egyptian woman from 3000 years ago. It was a prosthesis of a toe made of wood and served only for aesthetic purposes [1]. Even today there are prostheses for these same purposes; however, with modern technology, prostheses are gaining more and more functionality. The improvements in the design, control, and sensorial feedback in the state of the art prostheses are approaching the aesthetics and functionalities of the lost limb, and closing the gap to meet the needs of users.

Although such prostheses offer exciting possibilities for amputees, there are still technological challenges as well as difficulties imposed on such tools by the users themselves. A long training period is required for some prostheses and there is a high rate of rejection due to the limitations that this technology still has, while other prostheses need to be implanted surgically, which generates fear due to the inherent risk of the surgical procedure. However, the acceptance rates of prostheses are increasing as they increasingly resemble the amputated or missing part of the human body [2].

One of the major challenges encountered today in the area of prostheses is to find a solution to the closed-loop control that is performed by signals sent from the brain to the limb and then back to the brain from the limb. This communication generates two-way information of extreme importance, such as precise commands of which muscles to use or sensation of pressure and pain. Without this feedback, the user cannot control the prosthesis without being able to see exactly what he is doing. Moreover as the user only has a visual response he loses the ability to effectively control the force applied or the speed performed in a movement. Another point to be improved is the capture of signals for the activation of the prosthesis, which at the moment is part of the reasons for rejection by users [2].

The use of each prosthesis will depend on the level of amputation. The levels are defined by the position where the limb was amputated. The closer the amputation is to the shoulder, the more sophisticated the prosthesis will have to be for satisfactory performance and acceptance by the user. The level of amputation will also dictate how difficult the implantation of the prosthesis will be, since some technologies rely on the remaining limb muscles for more intuitive control.

The classification of prostheses is discussed in Section 2, as well as the techniques and technologies used today, their positive and negative points, how they can help and if they are really feasible, what areas have had to advance in order for these technologies to be executed and how each one works. In Section 3 the methodology used, the search engine and the inclusion and exclusion criteria are described. The selected papers are analyzed in Section 4, where the results are presented. In Section 5 the results are discussed, as well as the problems pertinent to this area of research and the advances that have been made. Section 6 presents the conclusions and gives a brief analysis of the article as a whole.

The Stephens-Fripp et al. [3] approach is related to prostheses that do not need surgery and the main disadvantage of this work is that it does not discuss sensory feedback (that requires surgery). Atzori and Müller [4] make a brief and objective review on the advances in both the commercial and scientific areas of myoelectric robotic prostheses for hand amputees. Still on hand prostheses, Maat et al. [5] presented several works related to passive hand prostheses and demonstrated the direction in which the technology in this area is evolving and Schofield et al. [6] highlighted and compared methods of sensory feedback in motorized upper extremity prosthesis. Meanwhile Carey et al. [7] compared myoelectric and body-powered upper-limb prostheses considering the following aspects: control, feedback, rejection of the body to the prosthesis, cosmesis, and functionality. Finally, Jackson and Bolger [8], in turn, proposed a review in order to provide a deep understanding about electroencephalography signals and their measurements. They discussed mechanistic explanations of EEG more clearly so that researchers, in general, understand the subject without having a strong background in physics and neurophysiology.

This review aims to present an analysis of the most up-to-date articles related to man-machine interfaces in upper limb prosthesis, to facilitate future research in this area and to serve as a source for professionals related to the area of prosthesis, such as doctors, engineers, researchers, and anyone interested in this subject. The myoelectric prosthesis group is emphasized (Section 2.1.3) as this group is the most up-to-date type of prosthesis.

2. State of the Art

Prostheses are medical devices used to replace specific limbs of the human body, whether upper limbs or lower limbs. The most common prostheses are arms with hands and legs with feet, initially designed for aesthetic reasons. However, with technological advances, it has become possible to create articulated and controllable prostheses. Thus, considering the interaction between the user and these devices, prostheses can be divided into two types: passive and active prostheses.

Passive prostheses can be subdivided into aesthetic and functional. Passive prostheses of the upper limbs, even if designed for aesthetic purposes, end up serving the healthy limb in tasks requiring two hands, while passive prostheses that are designed to be functional have a part intended to perform a specific task, such as a holder for cutlery, toothbrush or as a universal holder, where the user can fit various tools [5].

Active prostheses, in turn, can be controlled mechanically by the body (body activation), as in the case of a prosthesis that needs to be flexed so that a cable pulls and opens the hand, allowing the user to grasp objects, or by activation such as a battery (electrical activation), for example, that drives the engines which, in turn, do the mechanical work.

Electrical activation prostheses can also be divided, based on the type of control, into three groups:

• Myoelectric prosthetics, which are electrically activated prostheses controlled by the electromyographic signals captured in the stump muscles or any other part of the body through electrodes positioned on the skin just above the muscle in question [9];
• Prosthetics controlled by buttons, where these buttons can be activated manually by the healthy limb, by muscles in the back, or by muscles remaining in the stump [9];
• Hybrid prostheses, which are prostheses that combine different techniques of data capture for a more precise control of the prosthesis [10].

Figure 1 represents a diagram of the different types of prostheses based on their activation and control, and is based on the classifications presented earlier in this section (state of the art). In the passive prostheses, the man-machine interface is the simplest possible, and is only a fitting on the amputated limb without any direct information path. Any intention of movement that the user has, is sent only to the stump, which may or may not mechanically influence the prosthesis. In the active prostheses, due to the greater complexity of their interfaces, their aspects are discussed separately in Section 2.1. In order to facilitate the understanding of the subject matter discussed in this study, and due to the great variety of concepts, methods and approaches related to the implantation and control prostheses, a Section 2.2 was created, which presents, for example, types of feedback and brain-machine interface.

2.1. Man-Machine Interface in Active Prostheses

In this type of prostheses, the man-machine interface depends on the type of activation, because it is directly linked to the way the user’s intention is captured. They are divided into body activation prostheses, button controlled prostheses, myoelectric prostheses and hybrid prostheses.

2.1.1. Body Activation Prostheses

The user’s communication with body-activated prostheses, or body-powered, as it is also known, is done mechanically by means of cables attached to the patient’s shoulder or torso. The prosthesis is designed so that certain movements that the user wishes to make with the remaining limb can be translated directly into it [7]. An example of this type of interface is where arm elongation allows the user to open or close the claw at the end of the prosthesis. One of the advantages of this type of communication is that it presents a simple and intuitive control.

2.1.2. Controlled by Buttons

This type of prosthesis requires external power to drive the various motors involved in joint movement. The more mobile joints, the more degrees of freedom for the user, which controls each motor individually via buttons, which can be activated manually if the user has lost only one of the upper limbs, or pressure sensors positioned on the muscles that will trigger them. This type of communication has as an advantage concerning the degree of control that the user has over the prosthesis, however, the amount of training that is required to operate the individual motors is very long and exhaustive, and most electrical prostheses cannot manage multiple actuations, increasing user frustration with this technology [11].

2.1.3. Myoelectric Prostheses

The basis of this interface is the capture of electromyographic signals (EMGs) from electrical activities of the excitable cells of the muscles through electrodes on the skin (non-invasive method) or implanted directly into the muscles (invasive method). Whenever the brain sends a signal to a muscle, this electrical activity increases [12]. The prosthesis has software responsible for treating these signals and using them to drive the motors of the various joints.

A positive point of this technology is the possibility of implanting the electrodes in any muscle of the body, but for a greater similarity with the real member, muscles are usually chosen near the stump. There is an interface that acts like the limb’s natural control mode, even if there is only a one-way information path (data goes from the user to the prosthesis).

The problem with the non-invasive method is that the EMG signal collection can be drastically affected by several factors such as the position of the electrode, the movement of the area where the electrodes are placed, sweating and even the noise generated by the motors. The difficulty of training is also one of the negative points of this interface, and is responsible for the high rate of abandonment of this method by unilateral amputees (amputees of only one of the upper limbs). In the case of bilateral amputees (amputees of the two upper limbs), the acceptance is greater. This interface allows a patient with shoulder disarticulation to perform movements that would be impossible with prostheses using simpler interfaces.

The invasive method promises to improve the uptake and stability of these signals, since it is receiving the information directly in the muscle in question [13]. Among the invasive methods, it is important to highlight the recent use of epimysial electrodes to EMG recordings [14]. However, it also has some of the disadvantages of the non-invasive method, such as noise, for example, and this technology cannot be applied when the muscles, from which EMG signals are drawn, are very close.

2.1.4. Hybrid Prostheses

It is called a hybrid because it unites different user-acquired data acquisition techniques, such as myoelectric signals, electrical activities of the cerebral cortex (Section 2.2.2), osseointegration, epimysial electrodes, and pattern recognition. The combination of different techniques increases the user’s ability to control the prosthesis, facilitating the training to use it, and making the control more intuitive [15].

It is important to mention that osseointegrated prostheses (OIP) techniques have been developed and applied as an alternative to conventional socket-type prostheses [16].

2.2. Other Concepts and Techniques Involved

There are a great variety of techniques used with the current active prostheses ranging from the implantation of these prostheses to the types of feedback provided from them to the user. These techniques belong to areas such as medicine, computing, and electronics which are involved in their application, and are associated with data capture, signal processing and sensory feedback.

The process of implantation of the prosthesis begins by connecting the prosthetic device to the patient’s nerves or muscles. Subsequently, the signals that are emitted from the prosthesis (feedback by mechanical stimulation or other channels) are analyzed and reach the user through electrical impulses. After the prosthesis is connected to the patient’s body and it receives and interprets feedback signals, such as how much force is applied or how much, for example, a prosthetic hand is opened, a brain-machine interface is required to read the signal coming from the brain, interpret this signal and then actually control the prosthesis.

Since the passive prostheses are basically aesthetic or with a specific function, they are not controlled by the user. Thus, the connection of this type of prosthesis to the individual’s body is basically through tissue structures that tie the device to the patient. Thus, the methods presented in this section are related to active prostheses.

2.2.1. Methods of Connecting the Prosthesis to the User

The connection of the prosthesis to the patients’ bodies can be through surgery. In this case, one of the most common methods is target muscle reinnervation (TMR).

After an amputation of any degree, the muscles of the amputated region are lost, but the nerve endings attached to those muscles are not. The TMR surgical technique consists of reusing these nerve endings in healthy muscles as signals in myoelectric uptake. First, the muscle that serves as the source for myoelectric signals is denervated, and then it receives the nerve related to the desired movement [17]. This technique allied to the EMG-based man-machine interface is a powerful ally, since the use of a nerve previously used to close the hand, for example, can now also be used to close the prosthetic hand, providing a control much more intuitive to the user.

2.2.2. Brain-Machine Interfaces (BMIs)

Brain-machine interfaces are physical structures through which it is possible to pick up the electrical signals coming from the brain and send them to the prosthesis. The most common are electroencephalography (EEG) and electrocorticography (ECoG).

Electroencephalography is a non-invasive technique that allows the analysis of the electrical activity of the brain through electrodes positioned on the individual’s scalp [8], while electrocorticography is a very similar technique, but the electrodes are placed in the individual’s brain, directly on the cerebral cortex and thus it is an invasive method [18]. The signals captured by these techniques can be used in the direct control of prosthetics with a brain-machine interface, similar to those with myoelectric signals, but without the need of the muscle as an intermediate.

The signals captured from the muscles or the brain present a high level of complexity, and are often mixed with other unwanted signals, making it difficult to use them to control the prosthesis. In order for these signals to be harnessed, it is necessary to use pattern recognition techniques, which are algebraic and computational techniques applied to the data obtained to separate the useful information from the expendable. It is possible to create algorithms that will learn to use this data through training, improving the control of such prosthesis [19,20].

2.2.3. Types of Prosthesis Feedback for the User

In addition to receiving electrical impulses from the brain, prosthetics need to inform the user whether the command issued has been executed or to what extent it was executed. This return information (feedback) can be vibrational, tactile, electrical, or somatosensory.

• Vibrational: this technique was developed to close the sensory loop in a man-machine interface. Basically it can be applied to any prostheses, but it was developed to improve the control in prosthesis controlled by electromyographic signals [21]. In order for the user to feel what he touches, for example, sensors are placed on the prosthetic fingertips that, when activated to the touch, trigger vibrational devices installed on the surface of the patient’s skin. With this, the patient may indirectly feel that he is “touching” something.
• Tactile: it is very similar to the vibrational, with a single difference, the device related to the touch information does not vibrate, but rather causes a slight pressure on the user’s skin.
• Electrical: low-level electrical current pulses can be modulated to resemble the sensation of touch when discharged onto the skin [22]. This type of feedback is an ally of intuitive learning in prosthesis, and although it may present some degree of interference with myoelectric signal readings in EMG prostheses, there are techniques to filter these signals and allow this feedback to be used safely.
• Somatosensory: it is an electrical stimulus directly applied to the nervous system of the individual. This stimulus may be related to strength and position data, for example, from artificial sensors, in an attempt to mimic the natural sensitivity of the lost limb [6]. It is the most current form of feedback applied to prostheses, but it still lacks further studies because these stimuli are not easily controlled and need more accurate mapping for the correct areas of the nervous system to be activated.

3. Methodology

Initially a survey was carried out to find the possible keywords to be used in the research platforms (PubMed, Google Scholar, Web of Science, Science Direct and IEEE Xplore). This list of keywords was modified as the files found presented new keywords also useful for the search.

The search syntax of the different platforms was analyzed for a more refined search and to obtain articles that better fit the requirements for this review. Thus, the syntax used was: (“Limb” OR “Upper limb” OR “arm” OR “hand”) AND (“prosthesis”) AND (“mechatronic” OR “biomechatronic” OR “robotic” OR “bionic”).

This type of syntax allows the platform to find articles related to these words in a corresponding way, making it easier to select the files found.

The syntax could also be modified by adding new keywords, for a specific search of articles related to a certain subject within the general theme of this review, such as prostheses controlled by myoelectric signals for example.

Other search criteria were also applied, depending on the platform, such as the year of publication of articles and level of relevance.

With a better choice of keywords to refine the syntax and search criteria, it was possible to decrease the number of articles found on one of the search platforms from 1405 to 232. As the criteria increased, the number of articles decreased, while the number of articles within the required subjects increased in percentage terms. In the initial selection, out of more than 300 articles surveyed, 104 were chosen for analysis and filtering with the inclusion criteria.

The inclusion criteria, which served as a way to filter the searched articles and to select only the most current and relevant ones to be included in this review, was as follows:

• Articles must be linked directly or indirectly with the subject man-machine interface in upper limb prostheses;
• Articles must have been published within the last seven years;
• All articles must present relevant results for the study area;
• There can be no duplicates.

These criteria were applied manually to the 104 articles selected. After analysis and application of the criteria, 27 articles were chosen to be presented in Section 4 and 17 to be used as references in the Section 1 and Section 2.

Figure 2 is based on the articles used in this review. All keywords of the articles have been cataloged and organized as to their frequency of use, and are arranged graphically in descending order. The purpose of Figure 2 is to facilitate future searches for studies related to upper limb prostheses.

4. Results

This section presents the 27 articles selected for this review. Each article has its objectives, tests, and results described objectively, and

Table 1. Main techniques applied to man-machine interfaces.

Authors	Year	Number of Individuals Who Participated in the Tests	Condition of Individuals	Signal Pickup Type	Task Type	Prosthesis	Feedback	Hit Rate/Improvement
Dalley; Varol; Goldfarb	2012	5	healthy	sEMG	sequence of postures	virtual	visual	92.2%
Gonzalez; Castellini	2013	10	healthy	ultrasound	force test	-	visual	85–90%
Ninu et al.	2013	13	9 healthy and 4 transradial amputees	sEMG	grasp	Otto Bock Sensor Hand	vibrotactile	60%
Mcmullen et al.	2014	2	epileptics	ECoG	sequential movements	MPL	visual	70–100%
Smith; Kuiken; Hargrove	2014	5	healthy	iEMG	movements in 3 dimensions	virtual	visual	63–98.1%
Witteveen; Rietman; Veltink	2014	10	amputees	sEMG	grasp and lift	virtual	visual e vibrotactile	35–90%
Fifer et al.	2014	2	epileptics	ECoG	reach and grasp	MPL	visual	82–96%
Jorgovanovic et al.	2014	10	healthy	sEMG	handling objects	virtual	electrotactile	72%
Young et al.	2014	4	amputees	iEMG e TMR	simultaneous movements	EMG	visual	64–78%
Cipriani et al.	2014	4	healthy	iEMG	virtual posture	iEMG	visual	79–90%
Raspopovic et al.	2014	1	transradial amputees	sEMG	force test	EMG	somatosensory	>90%
Memberg et al.	2014	2	quadriplegics	sEMG e iEMG	daily tasks	-	somatosensory	-
Hartmann et al.	2015	7	healthy	sEMG	data analysis	-	electrotactile	100%
Hasson; Manczurowsky	2015	32	healthy	sEMG	“slice” movement 1	virtual	vibrotactile	-
Clemente et al.	2015	5	transradial amputees	Several	daily tasks	commercial prosthesis	vibrotactile	-
Dosen et al.	2015	12	10 healthy e 2 amputees	sEMG	force test	EMG	visual	various
Akhlaghi et al.	2015	6	healthy	Ultrasound	standardized movements	virtual	-	91–92%
Ma; Thakor; Matsuno	2015	13	healthy	sEMG	specific movements 2	EMG	-	-
Pasquina et al.	2015	2	amputees	iEMG	daily tasks	iEMG	-	-
Vidovic et al.	2015	11	7 healthy e 4 amputees	sEMG	specific movements 3	-	-	75–92%
Guo et al.	2016	16	13 healthy e 3 amputees	sEMG e Infra-Red Spectroscopy (NIRS)	specific movements 4	virtual	-	various
Osborn et al.	2016	12	10 healthy e 2 amputees	sEMG	hold objects	EMG	force (loop between sensor and prosthesis)	10–50% of improvement
Schiefer et al.	2016	2	amputees	sEMG	feel, reach and move objects	EMG	somatosensory	89–96%
Schweisfurth et al.	2016	12	11 healthy e 1 amputee	sEMG	grasp	Michelangelo	electrotactile	12–36% of improvement
Prahm et al.	2016	17	healthy	sEMG	specific movments 5	EMG	-	98.7%
Controzzi et al.	2017	1	amputee	sEMG	daily tasks	own development(SSSA-MYHAND)	position and force (loop between sensor and prosthesis)	-
Mastinu et al.	2017	1	amputee	iEMG	daily tasks	iEMG	somatosensory	98%

¹ Arm moved in two directions to a waypoint, then return to starting position; ² Rest, open hand, close hand, pronate, supinate, pronate open, pronate close, supinate open, supinate close; ³ Wrist pronation, wrist supination, wrist extension, wrist flexion, hand opening, fine pinch, key grip, and no movement; ⁴ Wrist flexion, wrist extension, radial deviation, ulnar deviation, pronation, supination, fist, hand open, index point, fine pinch, tripod grasp, ball grasp, and rest; ⁵ Rest, hand open/close, hand flexion/extension, wrist pronation/supination, and their simultaneous combination.

gives complementary information to these analyzes. To help clarify the results shown in this section, it presents technical data complementary to the articles analyzed, which are presented in chronological order. This table has data related to the number of participants in the tests of each article, the physical condition of these participants, the type of signal captured for the control of the prosthesis, the tasks performed in the tests, the prosthesis used, the sensory feedback of the prosthesis, and the rate achieved or improved.

Among the parameters presented in Table 1, the type of signal captured for the control of the prosthesis and the type of feedback stand out, as they directly influence the efficiency of the prosthesis, as discussed below.

The type of input signal can be corrupted by noise and, consequently, the prosthesis does not perform or performs the movement incorrectly. Characteristics, such as the source, amplitude, and frequency of the signal are directly associated with these noises.

Depending on how feedback is sent from the prosthesis to the user, there are significant influences in the execution of the command given to the prosthesis, especially in activities of grasping and holding objects, in which visual feedback stands out.

4.1. Type of Input Signal

4.1.1. Ultrasound

González and Castellini [23] emphasized that medical ultrasonography is fast, harmless to the patient and provides high temporal/spatial resolution. They used ultrasound images of the forearm to predict, in addition to the kinematic configuration of the hand, the strength of the fingers. The tests were performed with 10 healthy participants and a rapid analysis of on-off (activation and non-activation of muscles) showed that it was possible to train a system to predict, with low error (10 to 15%), the intermediate values of force of the fingers.

Akhlaghi et al. [24] used the analysis of ultrasound images taken from the forearm muscles of six healthy individuals to predict the possible movements of the hand. All the individuals made several standardized movements, during the training phase, so that the images pertaining to each movement supplied a database that would be used in a classification algorithm. In the test phase the algorithm was used to classify and predict the movements based on the new image readings and to control a virtual hand. The accuracy level of the control was 92%, which suggests that the technique has potential.

4.1.2. Electromyographic Surface Signals (sEMG)

EMG presents good temporal resolution [25] and sEMG is non-invasive, but this technique has several disadvantages. It is strongly affected by external noise, as discussed in Section 2.1.3. Guo et al. [25] performed an analysis of two techniques for the control of prostheses: control by sEMG, and near infrared spectroscopy (NIRS). For this analysis, the two techniques were tested in three ways with 13 healthy subjects and three amputees: only sEMG, only NIRS, and the two techniques combined. The accuracy of the classification in offline training and the online performance of the three configurations were evaluated. The best results were achieved by combining the two techniques, sEMG, and NIRS, without any greater complexity of the system, suggesting that this approach is feasible for future developments.

Dalley et al. [26] demonstrated the control of a virtual prosthesis through myoelectric signals with the use of two surface electrodes. The test was performed with five healthy individuals, where they captured the myoelectric signals of the forearm of these individuals to control several states of the prosthetic hand. For comparative purposes, the subjects performed a second test with a virtual glove (physical glove that transmits movement data to the computer). The two forms of interaction presented good levels of control and with similar performances.

Vidovic et al. [27] proposed a computational solution to improve the use of EMG prostheses. His study used pattern analysis with adaptive classifiers to optimize the myoelectric signals captured in the user’s muscles. The study was done with seven healthy people and four transradial amputees. The methodology involved the collection of myoelectrical data and the use of two classifiers, such as LDA (linear discriminant analysis) and QDA (quadratic discriminant analysis), in a first offline test. The results were below what were expected due to the scenario often referred to as covariate shift and to overcome this problem, the adaptation of the trained classifier was performed considering only a small calibration set containing the nonstationarities. This solution generated a significant improvement in the results. Online tests have also been made to test the effectiveness of the algorithms. The results showed that the adaptation of the classifiers improved the prediction of movements using the myoelectric signals, but the low number of amputated limbs compromised the results of the online tests, and the performance of the amputees was lower than that of the healthy individuals.

Prahm et al. [28] also addressed a computational solution for the recognition of myoelectric patterns applied to control prosthesis. However, in order to mitigate the difficulties of analysis of previous works, created by the diversity of data that each work used individually, they used tools of recognition of present patterns on open platforms such as BioPatRec and Netlab. The signals collected from the electrodes positioned on the forearms of 17 healthy individuals and the data are available on the BioPatRec platform. The final objective was to evaluate the best way to obtain the highest accuracy of the predictions based on the myoelectric data collected and the lowest computational complexity. The performance of three classifiers, a multi-layer perceptron (MLP) and two linear models, linear discriminant analysis (LDA), and generalized linear models (GLM), were compared. Algorithms created using the Netlab tools performed better than BioPatRec, but only offline analyses were done. The authors suggest integrating the Netlab recognition algorithms into the BioPatRec platform.

Ma et al. [29] studied muscle synergy, in the form of a matrix of signals, for the control of multiple degrees of freedom in a hand prosthesis. They applied a non-negative matrix factorization (NMF) method for prosthesis control. Two tests were performed with healthy individuals, one offline with four participants, for data analysis and parameter adjustment, and one online with 10 participants (one of them participated in the offline experiment), to evaluate the performance of the algorithm and the proposed control. The conclusion was that applications of weak muscle strength interfered negatively in the control of the prosthesis and that medium and high strengths would always be necessary for good control and stability. Although the participants were able to complete the control tests, the extensive use of the prostheses caused the displacement of the sensors on the skin, which resulted in a loss of control over the prosthesis, and the attempt to control multiple joints using multiple signals also meant an increase in noise, which generated involuntary movements.

Controzzi et al. [30] also opted for EMG signals to control their prosthesis. They developed a closed-loop myoelectric prosthesis (SSSA-MYHAND) in the prosthesis itself, where position and force sensors send data to the controller that adjusts the grasping movement. The prosthesis was tested with an individual with transradial amputation and was developed to present the same performance in daily tasks as conventional prostheses on the market but with lower weight and cost.

Ninu et al. [31] highlighted that myoelectrically-controlled transradial prostheses have been used for a long time as a relatively robust and intuitive approach to control hand prosthetic devices. In their study, 13 volunteers, four transradial and nine healthy amputees, participated in tests using the Otto Bock Sensor Hand commercial prosthesis to evaluate the importance of vibrotactile feedback as an indication of strength. The results showed that force feedback could replace the visual, but it was not essential for the task.

Schiefer et al. [32] presented an experiment carried out with two users of myoelectric prostheses, where pressure sensors and prosthetic hand opening were added to the prosthesis and connected directly to their peripheral nervous systems using the remaining nerves of the stump. The wave types altered the individual’s form of perception and they were able to feel the phantom limb. The tests, which consisted primarily of feeling, finding and moving objects, were performed with blindfolded individuals and had the following variations: no feedback, pressure feedback, hand-opening feedback, and both feedbacks together. Both individuals obtained better results using both feedbacks. One of the subjects stated that he/she were able to feel the weight of the object using the somatosensory feedback, which was proven through tests. The confidence and acceptance of the prosthesis as well as the individual sense that the prosthesis is part of the body increased after several uses with feedbacks. This work was also based on the work of Raspopovic et al. [33].

Memberg et al. [34] approached the man-machine interface in a different way. They used the interface doubly to have commands that were sent from the body to the machine, and then back to the human body instead of the prosthesis. In this work, the use of common myoelectric signals was associated with an action implemented directly on the nerves of a desired muscle through a neuroprostheses system which involved the implantation of two stimulators (IST-12), each with 12 stimulation channels and two EMG recording and telemetry channels. Two individuals with high-level quadriplegia had four myoelectric receptor channels (structures that pick up myoelectric signs from muscles and send them to an external control unit) implanted directly in muscles that were still in voluntary motion and another 24 stimulation electrodes were implanted in inactive muscles of the arm, chest, hand, and other parts of the body. The results showed that the implants allowed one of the individuals to recover the movements of the shoulder, arm and hand, and the other to recover the movements of the arm and hand. Even though they could not handle arm weight with stimulation alone, both individuals were able to perform simple daily tasks, and the implants were still fully functional after two and a half years.

4.1.3. iEMG

Smith et al. [35] aimed at evaluating the three degrees of freedom (DOFs) for the control of a virtual hand prosthesis using intramuscular electromyographic signals (iEMG). The tests were performed using five healthy individuals, where the parallel and sequential controls were evaluated. The results showed that, in the tasks that required several DOFs, the parallel control was significantly superior to the sequential control.

Young et al. [36] compared the use of three different control strategies: conventional myoelectric control, sequential control by pattern recognition (only one degree of freedom), and control by simultaneous pattern recognition. The tests were performed with four transradial amputees who had already undergone TMR. It was found that the two techniques of pattern recognition were superior to conventional control, however, the users chose to use the conventional technique when they had to use only one degree of freedom, and when they had to use several degrees of freedom, they opted in most cases to use the technique of simultaneous movements.

Cipriani et al. [37] conducted a study with four healthy individuals where electromyographic capture electrodes are implanted in forearm muscles to control four degrees of freedom of the fingers of a prosthetic hand. All individuals were able to control the prosthesis intuitively and without training. Although non-amputated individuals leaves the results biased, this study showed easier control with intramuscular electrodes than with conventional surface electrodes.

Pasquina et al. [38] demonstrated the first prosthesis with myoelectric sensors surgically implanted in transradial amputees. The prosthesis has three degrees of freedom and three pairs of implantable sensors were needed, one for each degree of freedom. Sensors capture the myoelectric signals and perform wireless communication with the prosthesis, in addition, they connect directly with the muscles related to the desired movements, unlike the superficial myoelectric signals, which, besides having skin interference, are not positioned close enough to the muscles. The test was performed with two individuals, but as one was still in the preoperative period, the results can only be related to the other. During the training period, involuntary movements of one joint appeared in the activation of another, this was due to the degree of training of the individual, and not to errors in reading the sensors. As the level of training increased, the user was able to control the three degrees of freedom separately or simultaneously with much more fluidity and without undesired movements, which makes the result of this article very promising.

Mastinu et al. [39] presented a blend of the most up-to-date technologies in the implementation of a myoelectric prosthesis that benefits from a neuroskeletal muscle interface (man-machine interface surgically installed in the stump bone, capable of receiving the EMG signals of the desired muscles and interconnected with the nervous system peripheral for somatosensory feedback) to perform the communications between the individual and the prosthesis. In addition to direct myoelectric control (a signal, a degree of freedom), two robust pattern recognition algorithms were implemented. The prosthesis was tested on a single individual and showed promising real-time control in daily tasks, as well as satisfactory sensory feedback, allowing the user to pick up delicate objects even when deprived of other senses other than from the prosthesis. Moreover, it still serves as a platform for future research.

4.1.4. ECoG

Fifer et al. [40] entered the area of ECoG where this capture is done directly from the brain. The study was implemented to the Modular Prosthetic Limb (MPL) of the Laboratory of Applied Physics of Johns Hopkins University. After encoding the brain signals corresponding to certain types of movement in two individuals while they used the arm to pick up objects, it was possible to identify potential signals to be used in the brain-machine interface for MPL control. The electroencephalographic signals captured and treated allowed the two individuals to control the prosthesis in the simultaneous movements of reaching and grasping, using thought.

McMullen et al. [15] also used ECoG signals. They developed a proprietary system for training of prostheses control through brainwaves called HARMONIE, a virtual environment integrated with neural control and augmented reality. Even taking 12 s to complete the tasks, two individuals with ECoG implants were able to control, only with visual thinking and feedback with computational vision, a prosthesis, performing the actions of reaching, grasping and releasing objects.

4.2. Feedback Comparison

Osborn et al. [41] added the feedback functionality to the myoelectric prostheses. In this work, force sensors were added to the surface of the prostheses fingers, and information from these sensors is sent back to the myoelectric prosthesis, closing a finger loop with the prosthesis force controller. This neuromimetic feedback is used to control the strength and stability of the prosthetic hand grip. The system was evaluated with 10 healthy subjects and two transradial amputees, one of them bilateral. In order to perform the grasp test with the prosthesis, only one pair of sensors was used. Individuals performed a series of tests where they had to grab objects without breaking or knocking them down and in the presence of feedback the number of successes was considerably greater, making this loop a good candidate to integrate future prostheses controls.

Clemente et al. [42] added vibrotactile feedback to existing prostheses on the market. Sensors were connected to the digits of each prosthetic finger and incorporated into vibrational stimulators in the forearm of five transradial amputees, who used the adaptation to their prostheses for one month, proving the effectiveness of the new system with promising results.

Witteveen et al. [43] performed a study with ten upper limb amputees. The amputees were asked to perform tasks of grabbing and lifting objects of different weights with a prosthetic hand, all in a virtual environment. The tests were performed with vibrotactile feedback and visual feedback. The results were better with the vibrotactile feedback than without any feedback, but the results with only visual feedback surpassed all, suggesting that the vibrotactile feedback is useful in the control of the prosthesis, but other studies must be carried out to know its degree of benefit in daily tasks.

While the other works on vibrotactile feedback have indicated that this type of feedback is similar to visual, Hasson and Manczurowsky [44] presented a study where the presence of this information made the control results worse. The tests were performed in two different experiments using a virtual myoelectric prosthesis, the first with twenty-seven individuals and the second with five individuals, all healthy. The presence of vibrational feedback did not increase the users’ ability to control the prosthesis and even negatively affected the performance of some.

Jorgovanovic et al. [45] discussed the study of the impact of electrotactile feedback on prosthesis control. The study was performed with 10 healthy individuals in a virtual environment where each individual used the benefit of feedback to handle objects. The ability of individuals to extend training to objects that were not present in the training was also evaluated. The study presented promising results where the performance of the tasks, although increased at runtime, was improved by the presence of feedback.

Hartmann et al. [46] analyzed the noise problem that electrotactile feedback causes and proposed a computational solution, analyzed entirely by computer. Since myoelectric prostheses use electrical signals from the muscles, electrotactile feedback can be a problem in acquiring this data because its signal can be picked up by the EMG electrodes as noise. In this work a blanking process was applied to the feedback, where the signal was identified, and replaced by another value when read by the EMG electrodes. The blanking technique worked perfectly and can be used as a simple solution for the implementation of electrotactile feedback in myoelectric prostheses.

Dosen et al. [47] explored a more objective side of feedback by bringing an improved form of visual feedback. A force bar was displayed on a screen where the user (10 healthy individuals and two amputees) received the visual indication and proportional of the force being applied to the myoelectric prostheses. This study was compared to the conventional forms of force feedback (as vibrational and electric) where control requires greater training because it is purely intuitive. All the individuals were able to control the prosthesis

In the work of Schweisfurth et al. [48] the communication loop was closed between the prosthesis and the user himself using electrotactile feedback and then compared to the vibrational feedback of the previous works. The tests were performed using this feedback on a Michelangelo commercial hand prosthesis with 11 healthy subjects and a transradial amputee. The control loop was programmed using Matlab Simulink and Real Time Windows Target. Electrotactile feedback has 8 possible responses (4 response electrodes, each with two possible frequencies) for different strength levels and was applied to the skin of the non-dominant forearm. The use of electrotactile feedback improved the control of the myoelectric prostheses in relation to the vibrational feedback, allowing a predictive control by the users.

In the study by Raspopovic et al. [33] the sensory feedback used in the man-machine interfaces of prostheses goes a step further. When the somatosensory feedback was introduced the sensory stimuli were sent directly to the peripheral nervous system. This study was performed with an individual who had undergone transradial amputation 10 years prior to the publication of the article. Traverse intrafascicular multichannel electrodes (TIMEs), which received signals from sensors on the fingertips of a myoelectric prosthesis, were directly connected to the individual’s median and ulnar nerves with the intention of delivering signals directly to the nervous system that mimic the physiological ones of the lost limb. During all the tests the individual remained deprived of visual and auditory feedback. Initially the individual was able to reproduce three different force levels with only artificial somatosensory feedback, taking an average of two seconds to adjust the force, and after seven days of training he could reproduce the three strength levels quickly more than 90% of the time. This result demonstrates that the similarity to human physiology has reduced the training necessary to control a myoelectric prosthesis from months to days, in addition to returning the sensation of touch to the amputated individual, which is a significant step.

5. Discussions

The research and analysis of the articles addressed in this review revealed that most of the research into man-machine interfaces for upper limb prosthesis is still at a very early stage. Although some of them are based on the lines of thought of previous ones, there is no standardization among them, which makes it difficult to analyze and quantify the results. As an example, it is possible to see a strong discrepancy in the results of Clemente et al. [42] and Hasson and Manczurowsky [44], while Witteveen, Rietman, and Veltink [43] do not present conclusive results.

Another tendency in the trails is the low number of amputees used in the tests. Although data extracted from healthy people are of extreme importance for the evolution of this area, a greater participation of amputees would be able to show even more clearly what is missing in order to develop prosthesis to become more like real limbs.

There is a strong tendency in the search for an interface that communicates directly with the nervous system, but there are not many recent publications on the subject, which makes most of the more current prosthetic interfaces limited to EMG signal acquisition. However, a brain-machine interface (BMI) was represented in the works of Fifer et al. [40] and McMullen et al. [15], where electrical signals are used directly from the human brain, and in Schiefer et al. [32] and Mastinu et al. [39] where the closest brain-machine interaction is present when environmental information is sent directly to the peripheral nervous system of the individual through sensors in the prosthesis, generating real physiological sensations.

Clearly, all areas involved in prostheses development will not evolve at similar speeds. While communication with the nervous system is still in the research phase, significant advances can be seen in the computational area of the interface. Numerous approaches have been made to improve signal acquisition, processing, and development of algorithms to make prostheses smarter, and to address user needs, thus facilitating daily tasks. The use of virtual platforms allied to the development of prostheses, such as was presented in the works of Dalley et al. [26] and Smith et al. [35] is also a strong ally in the evolution of this technology. Other computational approaches can be seen in Vidovic et al. [27] and Prahm et al. [28], where we have the processing of signals and classifiers working together to optimize the prediction of the desired movements.

The predominance of electromyographic signal acquisition in the most current ventures indicates that the man-machine interfaces converge to this area, but as research related to the peripheral nervous system advances, it is expected that this type of control will be replaced by one more similar to normal limb control (capture of signals directly from the peripheral nervous system).

Although there are still some challenges to be overcome, such as the failure to acquire signals at the EMG interfaces, the acquisition of motor signals directly from the peripheral nervous system and the mapping of movement intentions and responses to stimuli, the results of the analyzed works show a promising development for existing man-machine interfaces. It is also evident from the diversity of the lines of thought seen in the research that new solutions are being sought, as well as new approaches and improvements to existing solutions, making the idea of a prosthesis that can replace the real human limb and present very similar characteristics, something that is getting ever closer.

6. Conclusions

This article analyzes some of the most current works referring to man-machine interfaces applied to upper limb prosthesis and describes the techniques involved, as well as the results found in the more prominent articles. The use of a search plan and application of the inclusion criterion allowed the search for these works to be done in an efficient and iterative way in platforms, such as PubMed and Google Scholar, and programs such as Mendeley software, version 1.19 (Elsevier, London, UK) and Microsoft Excel Professional Plus 2016 (Microsoft Software, Redmond, WA, USA) played essential roles in the analysis of each article and organization of ideas, as well as in the construction of this review.

The result is a compilation of recent works that may be an initial source for researchers in the field, as well as independent researchers who want to know more about the subject. Making it clear that the work covered in this review is only part of the effort employed by students and researchers that contributed in some way to the evolution of prostheses as we know them.

This work presents powerful tools for the improvement of current prostheses and serves as a basis for future developments, as well as presenting a series of needs that will serve as a compass so that other researchers can discover new approaches and solutions.

The man-machine interface in the prosthesis is still far from perfect, but considerable progress has already been made in comparison with earlier works. Where there was a wooden hand for aesthetic purposes only, one now has a prosthetic hand, able to perform movements directly and intuitively controlled by the user and sends physiological sensations back, imitating the loop that the real hand had and improving control of the new hand. As the technologies involved evolve, the quality of life of amputees, people with congenital malformations of limbs, and even quadriplegics will improve proportionally.