In Vivo Usability Test of Vascular Intervention Robotic System Controlled by Two Types of Master Devices

Conventional vascular intervention (VI) procedures are typically performed manually under exposure to X-rays, whereby several problems are presented that need to be addressed owing to the patients and doctors being exposed to large amounts of radiation. In such cases, employing radiation protection units is not a long-term solution to avoid physical damage. Therefore, to overcome these issues, we propose a robotic VI system in this study. Moreover, we compare the extent of radiation exposure in the case of the conventional manual VI procedure with that in the case of the robotic procedure. The radiation exposure is then analyzed from the perspective of the doctor. Subsequently, the results of usability tests for two proposed master devices are presented in terms of the NASA task load index (NASA-TLX) and the system usability scale (SUS) score. To verify the effectiveness of the robotic VI system, animal experiments are conducted using a pig model. Among the two types of master devices tested with the proposed robotic VI system, the ergonomically designed 2-degree-of-freedom master device is found to be more effective than the joystick-type device in terms of the usability test scores. Hence, the proposed robotic VI procedure is shown to be advantageous in terms of reducing radiation exposure and improving usability.


Introduction
Conventional vascular intervention (VI) procedures are used to diagnose and treat diseases of the vascular system with the aid of catheters, guidewires, and various interventional devices, such as balloons and stents, with minimal incisions. These procedures require the use of imaging guidance methods, such as fluoroscopy, computed tomography (CT), magnetic resonance imaging (MRI), and other imaging modalities. Angiography, embolization, and angioplasty are the most frequently performed vascular interventional procedures worldwide [1][2][3][4][5][6]. During conventional VI procedures, both patients and doctors are exposed to large amounts of radiation from CT or X-rays. Such exposure has a detrimental effect on human health. For example, radiation exposure can cause skin damage, which can potentially lead to carcinogenesis as the radiation damage from repeated VI procedures accumulates [7].
Radiation dose limits for individuals subjected to occupational exposure are regulated by the international commission on radiation protection (ICRP). According to the ICRP 103 recommendations [8], the effective dose limit for radiation workers is 50 mSv per year; moreover, an additional recommendation is an effective dose limit of less than 20 mSv per year on average over five years. To prevent occupational radiation exposure, radiation workers wear radiation protectors in the form of aprons, collars, glasses, and gloves. However, they may become physically fatigued owing to the weight of the apron. Moreover, as radiation protection gloves are used minimally during surgical procedures in the operating room, exposure doses on the hands are still high [2]. To overcome these shortcomings, remote operations using master/slave robotic systems are needed for VI procedures to minimize radiation exposure.
Many research groups have developed VI robotic systems, such as the remote-controlled robotic system [9], VI surgical robot [10], CorPath GRX robotic system [11], Magellan robotic system [12], stereotaxis system, sensei robotic system [13], and COAST guidewire robot [14]. The effectiveness and feasibility of these developed VI robotic systems have been verified through numerous clinical trials for various VI procedures.
The main goal of developing the VI robotic system is to reduce the radiation exposure that occurs in the VI procedure, control the precise movement of the surgical tool, and reduce the physical fatigue of the doctor. Zhou et al. [9] and Yu et al. [10] demonstrate in vitro experiments, both systems have elaborated manipulating surgical tool using force feedback.
In the clinical trials of the CorPath GRX robotic system, percutaneous coronary intervention with manipulation of coronary guidewires was conducted on eight patients. The performance of the robotic system was evaluated and reported to be equal to or better than that in the case of the manual procedure [15].
Since the release of the first clinical report of the Hansen Sensei Robotic system in endovascular abdominal aortic aneurysm repair [16], the Magellan robotic system with six French catheter systems has been used to conduct more complicated procedures such as fenestrated endovascular aneurysm repair [17] or thoracic endovascular aortic repair.
Clinical trials for thoracic aortic repair using the Magellan robotic system have shown significantly less cerebral embolization when compared to that in the case of manual techniques in 11 patients [18]. The study concluded that due to its active maneuverability, control, and stability, the robotic system is likely to reduce contact with the atheromatous aortic arch wall, thereby resulting in less cerebral embolization.
However, these robots face limitations when operating with only catheters and guidewires. Robots with catheter and guidewire drives are suitable for cardiovascular interventions targeting large vessels. However, it is difficult to use these robots for procedures targeting microvascular lines. For example, the treatment of the transarterial embolization of liver tumor requires the operation of microcatheters and micro guide wires.
The microbot medical Inc. have developed the LIBERTY system to overcome these challenges. LIBERTY is a fully disposable robot system that can drive micro guide wire and microcatheter. The LIBERTY system showed the possibility of microvascular treatment of the robotic system by conducting coil deploy experiment on micro vessels through animal experiments [19]. The LIBERTY system also has the merit of reducing radiation exposure of doctors because it can be operated remotely.
In this study, we compared the radiation exposure dose that the doctor received during the robotic and conventional VI procedures. In addition, based on the system usability scale (SUS) score and the NASA task load index (NASA-TLX), a VI robotic system usability test was conducted to compare a 2 DOFs ergonomically designed master device and a joystick-type master device. Subsequently, in vivo experiments were conducted using a pig model to measure the radiation exposure dose received by the doctor and analyze the results of the usability tests of the two types of master devices.

7 DOFs VI Slave Robot
In this study, a 7 DOFs VI slave robot system was developed to control the motions of the catheter, guidewire, microcatheter, and micro guide wire. Figure 1a shows the 7 DOFs VI slave robot proposed in this study, which is a modified version of that developed by Cha et al. [20][21][22]. Cha et al. developed a 4 and 5 DOFs robotic systems for vascular intervention. The 4 DOFs robotic system controls the guidewire and the catheter, and the 5 DOFs robotic system includes steerable catheter. The unique feature of the 7 DOFs VI slave robot is that it employs a two-way design to minimize the total length of the robotic system. The front part of 7 DOFs VI slave robot is used to drive the guidewire and the catheter, and the longitudinal part is used to control the motion of the micro guidewire and the microcatheter. The 7 DOFs VI slave robot employs a telescopic mechanism in the central axis to control the catheter, as shown in Figure 1b. The guidewire moves sideways through the y-connector; this is done to enable the installation of up to four intervention tools on the 7 DOFs VI slave robot at once before conduction of the procedure so that the operator does not need to intervene during the VI procedure. The 7 DOFs VI slave robot can implement catheter rotation/insertion, guidewire rotation/insertion, microcatheter insertion, and micro guide wire rotation/insertion motions, as shown in Figure 1b.
To use the VI robotic system in a real operating room, it is necessary to consider the installation of the VI robotic system. During the VI procedure, the position of the bed and the C-arm are required to be freely adjustable. Therefore, the proposed VI slave robot was developed in the form of a bed mounting to maintain the robot's position with respect to the patient. In addition, a 5 DOFs robot arm is used to adjust the position of the VI slave robot such that it could approach the patient's treatment area in the right direction. The movement of the 5 DOFs robotic arm is shown in Figure 2a. For user convenience, a motor and a brake were installed at each joint of the 5 DOFs arm. In the case of normal operation, when power is supplied to the 5 DOF robot arm, the brake on the robot arm is released so that the robot arm can be properly positioned. Conversely, in the event of an emergency when the power supply is not stable, the brake on the 5 DOF robot arm is locked to ensure patient safety. The position of the VI robot was adjusted by operating the handle. The force/torque sensor (ROBOTUS, RFT40) installed on the handle of the 5 DOFs robot arm is employed to position the handle easily using an admittance control algorithm. The 7 DOFs VI slave robot thus mounted on the side of the bed with the 5 DOFs robot arm is shown in Figure 2b.
When the user operates the handle, the measured force and torque f are converted to a position command u arm of the 5 DOFs robot arm through admittance control according to the expression where [A] ∈ R 5×5 is the admittance gain matrix.

Master Devices of AI Robotic System
In this study, two types of master devices were tested in the in vivo experiments. The first master device has 2 DOFs, as shown in Figure 3. The motion of this device was inspired by the movement of the operator's hand during the VI procedure. In the conventional VI procedure, the catheter and guidewire are manually operated. Thus, the ergonomically designed 2 DOFs master device was implemented to involve the same motions as those in the conventional VI procedure. Upon the activating button being pressed, the command from the master device is transferred to the VI slave robot. Further, a mode switch button is used to change the catheter control mode to the guidewire control mode and vice versa. This master device adopts the position-position control method, where the displacement of the VI slave robot is proportional to that of the master device.
The advantage of the ergonomically designed 2 DOFs master device is that it mimics the movement of the operator's hand. Therefore, any operator can easily and intuitively use it. Conversely, the disadvantage of this device is that the operator has to perform repetitive motions to send the catheter and the guidewire to the target vessel. Using a motion scale, the operator can easily manipulate the catheter or the guidewire. However, the operator may feel physical burden/fatigue when operating this master device for a long duration.
The second master device also has 2 DOFs, as shown in Figure 4a and a joystick-type structure comprising two hemisphere-type linkages. These linkages were coupled to the joystick handle at the center of the device so that insertion and rotation of either the catheter or the guidewire could be achieved using the two rotational degrees of the joystick. Two buttons were placed on the right side of the device. Similar to the ergonomically designed master device, the slave robot could move only when the button was pressed. Further, a mode switch button was used to change the catheter control mode to guidewire control mode and vice versa. The joystick-type master device adopts the position-velocity control strategy, where the velocity of the VI slave robot is proportional to the position command of the master device. In the 2 DOFs joystick-type device, the motion speed of the VI slave robot increased as it moved farther away from the center of the joystick structure. The merit of the 2 DOFs joystick-type master device with position-velocity control is that the operator does not have to perform repetitive motions, which can save time and physical effort of the operator. The operation of the master device is similar to that of gaming joystick-type device. The disadvantage of the 2 DOFs joystick device, however, is that the operator requires more time to get used to the 2 DOFs joystick master device because the control modes of the conventional VI procedure and 2 DOFs joystick-type device are different.

7 DOFs VI Slave Robot Control for the Two Master Devices
As explained in Sections 2.1 and 2.2, the master-slave VI robotic system is composed of a 2 DOFs master device and 7 DOFs slave robot. To control the slave system, which has more DOFs than the master device does, the mode switching method was applied to select one of the four VI tools (catheter, guidewire, microcatheter, and micro guide wire). The mode switch consists of two buttons: a mode switch button on the master device and a virtual button on the programming software for the 7 DOFs VI slave robot. The mode switch button on the master device was used to change the mode between the catheter and guidewire modules. The same button also changed the mode between the microcatheter and micro guide wire modules. The software button for the 7 DOFs VI slave robot control software was used to change between the catheter-guidewire and microcatheter-guidewire modes. The four modes of the 7 DOFs VI slave robot using the two mode switching buttons are described in Figure 5. In this work, two different 2 DOFs master devices were used to control the 7 DOFs VI slave robot. The ergonomically designed 2 DOFs master device adopted the positionposition control method, as illustrated in Figure 6. When the operation shown in Figure 6a occurred in the ergonomically designed 2 DOFs master device, the insertion motions were transmitted to the 7 DOFs VI slave robot. In addition, when the motion shown in Figure 6b occurred, the rotation motion was transmitted to the slave robot. The movements of the slave robot according to the inputs of the ergonomically designed master device are shown in Figure 6c.
The expressions representing the movements of the 7 DOFs VI slave robot according to the motions of the ergonomically designed 2 DOFs master device are as follows: where θ cMI and θ pMI denote the current and previous positions of the master device handle for insert operation. θ cMR and θ pMR denote the current and previous positions of the master device handle for rotate operation. (θ SI ) mode and (θ SR ) mode are the insertion and rotation motions of the VI slave robot position, respectively. Each module has a different control gain λ ERGO−SI mode and λ ERGO−SR mode . The 2 DOFs joystick-type master device adopts the position-velocity control method, as shown in Figure 7. When the operation shown in Figure 7a occurred, the insertion and rotation motions were transmitted to the slave robot. The movements of the slave robot according to the inputs of the joystick-type master device are shown in Figure 7b. The expressions representing the movements of the slave robot according to the motions of the joystick-type master device are as follows: The expressions representing the movements of the slave robot according to the motions of the joystick-type master device are as follows: .
where θ cMI and θ center denote the current and center positions of the joystick-type master device handle for insert operation. θ cMR and θ center denote the current and previous positions of the master device handle for rotate operation ( . θ SI ) mode and ( . θ SR ) mode represent the insertion and rotation of the slave robot velocity, respectively. Each module has a different control gain λ JOY−SI mode and λ JOY−SR mode , similar to that of the ergonomically designed 2 DOFs master device.
The schematic of the robot control system is presented in Figure 8. The motion data from the master devices were transmitted to a PC via RS232 communication. The motion command was transmitted to the motor controller of the 5 DOFs robot arm and 7 DOFs VI slave robot via controller area network (CAN) communication.

Experimental Conditions for Measurement of Radiation Exposure
Herein, in vivo experiments were conducted to evaluate the performance of the 7 DOFs VI robotic system. To verify the effectiveness of the system, animal experiments were conducted using a pig model. The Animal Care Committee of the Department of Laboratory Animal Resources of Yonsei University College of Medicine approved this study protocol (Permit number: 2018-0224). In the experiments, the radiation exposure doses were measured for both the conventional and robot-based VI procedures.
In the conventional procedure, the doctor manipulates the catheter (Sungwon medical, Seoul, Korea) and guidewire (Terumo, Tokyo, Japan) manually; therefore, the doctor has to be located close to the X-ray source, as shown in Figure 9a. In the experiments described in this paper, the console of the master device of the robotic VI system was located inside the procedure room, taking into account emergency situations in the case of human procedures. The distance between the radiation source and the master device was about 5 m, and a lead window shield was placed between them. In this case, the doctor only needs to manipulate the master device to control the catheter and guidewire, and therefore does not have to be present close to the X-ray source, as shown in Figure 9b. The radiation dose was measured using a real-time radiation dosimeter (RaySafe i2, RaySafe, Sweden) at the anterior chest position of the doctor's apron during the conventional and robotic VI procedures. Realtime radiation dosimetry allows for the measurement of the radiation dose at a certain point in time as well as the accumulated radiation dose for each procedure. A total of 10 doctors participated in the radiation exposure measurement tests for procedures involving a selection of the pig's hepatic, splenic, and right renal arteries. Ten doctors participated in the VI robot system usability test. Among them, six doctors had never used the VI robot system used in this study before, three doctors had 2 or 3 prior experiences with the VI robot, and one doctor had operated VI robot more than 30 times. In addition, the expert level of the doctors participating in the experiment is diversely distributed from 1 year to over 20 years in the department of radiology.
Appl. Sci. 2021, 11, 5453 10 of 17 Ten doctors participated in the VI robot system usability test. Among them, six doctors had never used the VI robot system used in this study before, three doctors had 2 or 3 prior experiences with the VI robot, and one doctor had operated VI robot more than 30 times. In addition, the expert level of the doctors participating in the experiment is diversely distributed from 1 year to over 20 years in the department of radiology.

Usability Tests of Two Different Master Devices for Robotic VI System
In this study, we analyze the usability of the two master devices. The same targets were employed for the measurement of the radiation exposure. Two different target points were each selected for the hepatic and right renal arteries to analyze the usability

Usability Tests of Two Different Master Devices for Robotic VI System
In this study, we analyze the usability of the two master devices. The same targets were employed for the measurement of the radiation exposure. Two different target points were each selected for the hepatic and right renal arteries to analyze the usability of the two master devices. The completion times were measured when the catheter and guidewire reached the target points of the in vivo vessels. The NASA-TLX [23] was used as an additional index for comparing the two devices. The NASA-TLX questionnaire was used to measure the workload. Further, the SUS score [24] was measured.
The NASA-TLX is a widely used, multidimensional assessment tool that rates perceived workload to assess a task, system, or team's effectiveness or other aspects of performance. It was developed by the Human Performance Group at NASA's Ames Research Center. The NASA-TLX has six evaluation criteria: mental demand, physical demand, temporal demand, performance level, effort level, and frustration level. The NASA-TLX criteria are evaluated for each task within a range of 100 points in 5-point steps. For example, if a task does not require significant mental effort to complete, a low score is assigned to the mental demands question, and vice versa. In conclusion, the lower the test subject's NASA-TLX score, the lower the mental and physical demands required to complete the task.
The SUS score was originally created by John Brooke in 1986. It was used to evaluate a wide variety of products and services, including hardware, software, and applications. Each item of the SUS score has five response options for respondents, ranging from strongly agree to strongly disagree. The SUS score involves 10 items that evaluate the level of consent and aims to assess the system by combining the positive and negative aspects of its performance. The SUS score generally provides a high level of subjective perspective on the usefulness of the system. As a method of analyzing the SUS score, Bangor et al. [25] developed the evaluation criteria for the SUS scores. Bangor et al. assigned terms of what is "acceptable" or "not acceptable" for the SUS score. The acceptable range corresponds to scores of above 70 and the unacceptable range corresponds to scores below 50. Additionally, Bangor et al. designated the range between 50 and 70 as "marginally acceptable". For the VI robotic system usability test, we asked the subjects to fill out the NASA-TLX and SUS score questions after completing the task for each device.

Pig Hepatic and Splenic Arteries
To measure the operator radiation exposure and the usability of the two master devices for the robotic systems, the starting and target points (target 1 = hepatic artery, target 2 = splenic artery) were set as shown in Figure 10. The experiments for the hepatic and splenic arteries were conducted three times per target per master device per operator.  Figure 11 shows the results of the radiation exposure measurements of the operator during the conventional and robotic procedures. For the conventional procedure for target 1, the radiation exposure was measured to be 4.38 ± 1.95 µSv, whereas that for the robotic procedure for target 1 was measured to be 0.11 ± 0.04 µSv. For target 2, the radiation exposure doses were measured to be 6.16 ± 1.86 µSv and 0.12 ± 0.06 µSv for the conventional and robotic VI procedures, respectively. In the hepatic and splenic artery experiments, it was observed that the robotic procedure reduced radiation exposure by about 97% compared to the conventional procedure. For the pig hepatic and splenic artery experiments, the NASA-TLX and SUS scores were used to evaluate the usability. The results of the overall workload score based on the NASA-TLX and the system integration score evaluation based on the SUS score are summarized in Table 1. The values indicate the mean and standard deviation of the usability scores for the two master devices for each target. The operators reported higher NASA-TLX scores for both target 1 and target 2 when using the 2 DOFs joystick-type master device, implying that this device is physically and mentally harder to operate. In addition, higher SUS scores were reported for both target 1 and target 2 when using the 2 DOFs ergonomically designed master device. In the case of the 2 DOFs joystick-type master device, the score was about 55 points for both target 1 and target 2. According to the SUS score evaluation criterion, the 2 DOFs joystick-type master device belongs to the "marginal low" category among the acceptability ranges. However, the ergonomically designed 2 DOFs master device received approximately 73 points for the SUS score for the hepatic artery target 1 and target 2. This score is considered "acceptable". Thus, the 2 DOFs ergonomically designed master device outperformed the 2 DOFs joysticktype device in terms of user work intensity and system integration.
In addition, Table 2 shows the average of moving distances of the two master devices and the number of retractions of surgical instruments during the process of selecting blood vessels to reach the targets. The lesser the moving distance and the number of retractions, the lesser energy the master device consumes. For both target 1 and target 2, the moving distance of the joystick-type master device was measured to be greater than that of the 2 DOFs ergonomically designed master device. Moreover, the number of retractions of surgical instruments for the 2 DOFs joystick-type master device was measured to be higher than that for the 2 DOFs ergonomically designed master device. In terms of these two quantitative measurements, the 2 DOFs ergonomically designed master device outperformed the 2 DOFs joystick-type device for both the hepatic artery and splenic artery.

Pig Right Renal Artery
In the case of the right renal artery selection experiment, the starting and target points were set as shown in Figure 12. The selection of the right renal artery was also performed three times for each target and for each master device per operator. Compared to the hepatic and splenic artery experiments, it was easier to approach the target points in the right renal artery experiment. The results of the measurement of the radiation exposure to operators during the conventional and robotic VI procedures for the right renal artery are shown in Figure 13. In the conventional procedure for the right renal artery target 1, the radiation exposure was measured as 2.84 ± 0.91 µSv, whereas that for the robotic procedure was 0.08 ± 0.04 µSv. Further, for the right renal artery target 2, the radiation exposure doses of the operator were measured as 2.61 ± 0.87 µSv and 0.06 ± 0.02 µSv for the conventional and robotic VI procedures, respectively. In the right renal artery target 1 and target 2 experiments, it was observed that the robotic VI procedure reduced the radiation exposure by about 97% compared to the conventional procedure. For the usability tests, we employed two measures in this study: NASA-TLX and SUS score. The results of the overall workload scores based on the NASA-TLX and the system integration score evaluations based on the SUS score are summarized in Table 3. Table 3. Overall workload and system integration scores for pig right renal artery experiments (mean and standard deviation).

Master Device
Usability Index Target  The operators reported higher NASA-TLX scores for both target 1 and target 2 of the right renal artery when using the 2 DOFs joystick-type master device, implying that this device is physically and mentally harder to operate. Further, higher SUS scores were reported for target 1 and target 2 of the right renal artery when using the 2 DOFs ergonomically designed master device. However, the differences in the SUS scores of the right renal artery experiments were smaller than those of the SUS scores in the hepatic artery experiments. Fewer differences were observed between the NASA-TLX score of each target on the right renal artery. In the case of the 2 DOFs joystick-type master device, the mean score was about 72 points for target 1 and target 2. In the SUS score evaluation criterion, the 2 DOFs joystick-type master device was scored as "acceptable". In addition, in the case of the 2 DOFs ergonomically designed master device, SUS scores of about 76 points were obtained for both target 1 and target 2 of the right renal artery. Thus, the scores of the 2 DOFs ergonomically designed master device were also "acceptable". Thus, the 2 DOFs ergonomically designed master device outperformed the 2 DOFs joystick-type device in terms of user work intensity and system integration.
In addition, Table 4 shows the average of moving distances of the two master devices and the number of retractions of surgical instruments during the process of selecting blood vessels to reach the targets. For both target 1 and target 2, the moving distance of the joystick-type master device was measured to be greater than that of the 2 DOFs ergonomically designed master device. Moreover, the number of retractions of surgical instruments for 2 DOFs joystick-type master device was measured to be higher than that of the 2 DOFs ergonomically designed master device. In terms of quantitative measurement, the 2 DOFs ergonomically designed master device outperformed the 2 DOFs joystick-type device for the right renal artery.

Discussion
In the animal experiments with the pig model, the robotic VI procedure reduced the radiation exposure to the operator by about 97% when compared to the conventional procedure. The smaller amount of radiation exposure observed during the robotic VI procedure is due to the use of the radiation shield and the considerable distance of the doctor from the X-ray source. If the operating room of the robotic VI procedure and the master console room are separated, then it is expected that there would be no radiation exposure to the doctor. The effectiveness of reducing radiation exposure using the robotic VI procedure can thus prove highly beneficial to radiation workers as it can reduce unwanted potential side effects during surgical operations.
It is obvious that robotic intervention could reduce the radiation exposure to the operator or assistants. However, it is more important to improve the benefits to the patient with robotic intervention. In terms of the radiation dose exposure to the patient, it has been reported that robotic coronary artery intervention could reduce patient radiation exposure by up to 20%. The authors provided two potential mechanisms for radiation dose reduction for patients: improved visualization from closer monitor positioning during the robotic intervention and maximal angio unit table height, which is available only with robotic intervention [26]. They also reported no increase in fluoroscopy time and contrast media usage.
The 2 DOFs ergonomically designed master device outperformed the joystick-type device in terms of the NASA-TLX and SUS scores. Moreover, it was shown that the 2 DOFs ergonomically designed master device could reduce the mental burden on the operator as the system integration evaluation was better during the robotic VI procedure. This is because the position-position-based 2 DOFs master device was similar to the conventional procedure with respect to operator motions despite the repetitive movements. The selection of the target artery was more difficult in the hepatic artery experiments than in the right renal artery experiments. Hence, the usability test scores of the 2 DOFs joystick-type master device were poor. The 2 DOFs joystick-type master device may be suitable for robotic VI procedures targeting easy blood vessels, but it is unsuitable for procedures targeting difficult or tortuous blood vessels. These results also showed that the 2 DOFs ergonomically designed master device was better than the 2 DOFs joystick-type device in terms of quantitative measurement.
The next important step for improving the master device is the installation of a haptic function that could assure a safer procedure if accurate force feedback of the catheter and guidewire resistances against the vessel wall are delivered to the operator through the master device. Beginners would find it easier to use the ergonomic-type haptic device as it is more intuitive. However, it is possible that the joystick-type device may be more comfortable for those familiar with robotic procedures. Therefore, further testing is needed. Moreover, the ergonomic-type haptic device is safer to handle, because the range of motion is limited according to the working distance of the master stick. However, further considerations of the master and haptic device designs and tests are needed to assure patient safety during robotic interventions. In addition, the present research has a few limitations as well. Firstly, the number of participants in the experimental trials were limited. More meaningful conclusions may be obtained by increasing the number of trials by the participants. Secondly, the patient perspective was absent when reducing the radiation exposure. In our future work, we plan to integrate VI navigation and robotic VI systems to reduce the amount of radiation exposure to the patient.

Conclusions
We presented herein the results of the usability tests for two different master robotic devices and assessed the reduction in radiation exposure during the robotic VI procedure from the operator perspective. In vivo experiments were conducted to validate the effectiveness of the 7 DOFs VI slave robot and two different master devices. The in vivo experimental results demonstrated that the robotic VI procedure with the 2 DOFs ergonomically designed master device performed better than the 2 DOFs joystick-type master device did in terms of the operator workload and system integration evaluation. The various expert levels of the experiment participants and the different number of experiences with the VI robotic system did not affect the results. This means that the developed VI robot has its advantage with respect to usability. Further, the robotic VI procedure was shown to reduce radiation exposure to the operator by about 97% relative to the conventional procedure.