Eye-Hand Coordination Patterns of Intermediate and Novice Surgeons in a Simulation-Based Endoscopic Surgery Training Environment

Abstract

Endoscopic surgery procedures require specific skills, such as eye-hand coordination to be developed. Current education programs are facing with problems to provide appropriate skill improvement and assessment methods in this field. This study aims to propose objective metrics for hand-movement skills and assess eye-hand coordination. An experimental study is conducted with 15 surgical residents to test the newly proposed measures. Two computer-based both-handed endoscopic surgery practice scenarios are developed in a simulation environment to gather the participants’ eye-gaze data with the help of an eye tracker as well as the related hand movement data through haptic interfaces. Additionally, participants’ eye-hand coordination skills are analyzed. The results indicate higher correlations in the intermediates’ eye-hand movements compared to the novices. An increase in intermediates’ visual concentration leads to smoother hand movements. Similarly, the novices’ hand movements are shown to remain at a standstill. After the first round of practice, all participants’ eye-hand coordination skills are improved on the specific task targeted in this study. According to these results, it can be concluded that the proposed metrics can potentially provide some additional insights about trainees’ eye-hand coordination skills and help instructional system designers to better address training requirements.

Introduction

Minimal invasive surgery (MIS) is preferred by surgeons and patients as it has several benefits for the patients, causing less infection risk and shorter operation time (Lanfranco, Castellanos, Desai, & Meyers, 2004). However, MIS education for surgeon faces many challenges, such as difficulty to learn, while more than 30 procedures for the learning curves have been reported (Dankelman, Grimbergen, & Stassen, 2007; Moore & Bennett, 1995). In this respect, depending on the type of the operation, the number of such procedures can be increased up to 100 (Lehmann et al., 2005).

Surgeons involved in these types of operations need to develop specific psychomotor skills, such as eye-hand coordination, and depth perception to mention a few (Hermens, Flin, & Ahmed, 2013; Hernandez et al., 2004). As the location of the operation site can only be observed through a monitor in MIS, mislocation (e.g., in case that the display in the monitor through the endoscope and hands controlling the operational tool are at different locations) can make it impossible for the surgeon to follow his/her hands as well as the operative scene simultaneously (Batmaz, de Mathelin, & Dresp-Langley, 2017; Wentink, 2001). This mislocation problem can be more critical in complex surgery procedures, such that surgeons might even require assistance to control the endoscope.

Hence, surgical education programs today require very important knowledge and skills to be gained. In traditional training programs, these skills are gained in the operating theatre and upon practice on patients. According to earlier studies, critical errors (Gordon, Wilkerson, Shaffer, & Armstrong, 2001) can occur when these skills are not developed properly or urgently put in practice (Berkenstadt et al., 2003). In current education programs, novice surgeons should improve their skills and practical abilities on real patients supervised by experts (Evgeniou & Loizou, 2013), which is an expensive and very risky method. Apart from these factors, assessment of surgical skills is carried out by training specialists through structured schemes, such as OSATS (Objective structured assessment of technical skill), which is a method for testing specific operative skills in surgical trainees (Cagiltay, Ozcelik, Sengul, & Berker, 2017; Martin et al., 1997). However, as also reported by Moorthy et al. (2003) there are certain constraints in this method, such as resources and time to find supervising surgeons to observe and evaluate the performance of trainees (Moorthy, Munz, Sarker, & Darzi, 2003). Naturally, since these operations directly affect human safety, ethical issues may arise. As a result, in order to guarantee patient safety, trainees should be educated by other means than with actual patients.

In addition, the use of movement based measures has been studied extensively, and has been suggested as an effective method for monitoring surgery training (Stylopoulos & Vosburgh, 2007). It has been reported that motion analysis devices are useful tools to assess performance compared to merely relying on OSATS and time (Hernandez et al., 2004). Tracking hand and instrument movements using markers, known as ‘motion analysis’, has been suggested by earlier studies as an alternative method to OSATS in assessing the related skills by measuring the economy of movement (Datta, Chang, Mackay, & Darzi, 2002). Latko et al. (1997) studied videotaped and documented hand activities rated from 0 to 10 through observations, and provided some definitions for hand movements. According to them, when no regular exertions are detected, the hand activity is considered as ‘idle’, and when there is infrequent motion, it is considered as ‘steady motion’. Based on the frequency of the motion, they also propose ‘consistent conspicuous’ (long pauses or slow motions), ‘slow steady motion’, and ‘rapid steady motion’ (Latko et al., 1997).

Besides, several hand-movement metrics have been proposed, such as path length, motion smoothness, depth perception (as the total distance traveled by the instrument along the instruments’ axis), response orientation and grasping (Stylopoulos & Vosburgh, 2007). In this vein, some studies have been carried out on motor behaviors in surgical skills based mainly on the path-length, the amount of time to complete a procedure, and idle time, which topic needs to be considered but has remained rather neglected (D’Angelo et al., 2015). Oropesa et al. (2011) define ‘idle time’ as lack of movement in both hands representing the delay in motor planning or decision making (Oropesa et al., 2011). One example of motion metrics is the smoothness of hand function. Oropesa et al. (2013) define ‘motion’ as abrupt changes in acceleration resulting in jerky movements of the instrument (Oropesa, Chmarra, et al., 2013; Oropesa et al., 2011). Another proposed metric, ‘working space’, is suggested for the economy of the area and economy of volume efficiency in MIS (Oropesa, Chmarra, et al., 2013), and is determined by using an electromagnetic sensor to track the participants’ hand movements and the summation of distances from the sensor’s average spatial location (Mohamadipanah et al., 2016). However, further research has been reported to be necessary in a recent study to better understand the role and usage of psychomotor metrics, such as smoothness, to assess performance during certain medical procedures (Mohamadipanah et al., 2016).

Detecting the location of a given object in a precise manner is, such as a tool or hand, important for each of these metrics (Helsen, Elliott, Starkes, & Ricker, 2000; Oropesa, Chmarra, et al., 2013). In the literature, video-processing methods (Jiang, Zheng, & Atkins, 2015; Oropesa, Sánchez-González, et al., 2013 and motion-tracking systems (Oropesa et al., 2011) have been proposed to detect the position of the tool in a precise way, giving rise to other practical concerns.

Despite the presence of evidence showing a correlation between motor skills and eye events (Bishop, Kuhn, & Maton, 2014; Jiang, Zheng, Bednarik, & Atkins, 2015; Johnson, Lum, Rinehart, & Fielding, 2016; Parr, Vine, Harrison, & Wood, 2018; Schmitt, Cook, Sweeney, & Mosconi, 2014), there are very limited studies conducted to improve the understanding and objective measuring of surgical residents’ eye-hand coordination skills, while there is hardly any standardized measure in this regard, thus limiting the interpretation and generalization of related results (Wang, Howe, Lin, & Hsu, 2014). Hence, despite the fact that there exist some metrics providing insights into the relationship between the eye and the hand as well as their coordination, still there is a need to extend our knowledge of how hand movements are guided and controlled by vision (Wilson et al., 2010). Objective methods for analyzing surgeons’ hand movement patterns have not thoroughly led to devising fully satisfactory methods of assessment (Reiley, Lin, Yuh, & Hager, 2011); whereas such metrics are necessary in order to provide proper feedback and continuous analysis of psychomotor skills in MIS (Cagiltay et al., 2017; Oropesa et al., 2014).

Today, computer-based simulation environments offer many benefits since they are cheaper, provide more time for practice and can be easily modified for different rare cases (Kirk, 1996). Apart from such environments, in the literature, there are several studies showing that eye-tracking technology provides beneficial insights for surgical training purposes. In a review study, Tien et al. (2014) concluded that eye tracking offers reliable quantitative data for objective assessment purposes to improve performance in surgical training (T. Tien et al., 2014). Several studies have been conducted to better understand the eye-movement behaviors of surgical residents for improved skill assessment purposes. Research suggests that recording these eye movements may be beneficial both for skill assessment and training in the field of surgery (Hermens et al., 2013). Today, computer-based simulation environments provide several objective assessment capabilities, which can be attained continuously during the training period and without expert supervision (Ayodeji, Schijven, Jakimowicz, & Greve, 2007). However, the calculation methods for many of these metrics are implicit and within the scope of commercial simulators, making them difficult to manipulate.

In light of all these shortcomings and needs within the field of surgery, the present study first attempts to adapt the eye-movement event analysis approaches to the hand motion and propose new additional objective metrics for the hand-movement events. Additionally, through haptic devices the data pertaining to the hand locations of the surgical residents are collected along with their eye-movements while performing surgical tasks in a virtual reality environment. By analyzing both the eye and hand behaviors of the surgical residents, this study aims to understand the eye-hand coordination skills of the surgical residents in a more elaborate way.

Methods

The aim of this study is to assess the relationship between eye-gaze and hand-motion metrics to understand the eye-hand coordination behavior differences of intermediate and novice surgeons in a simulation-based endoscopic surgery environment.

The findings of the literature imply that there are several different algorithms which use constant threshold values to classify eye events into fixations and saccades. However, it is also reported that an ideal algorithm should automatically identify the threshold values, without requiring any parameter setting from the user (Andersson, Larsson, Holmqvist, Stridh, & Nyström, 2017). For instance, the BIT (Binocular-Individual Threshold) algorithm is a fully automatic, velocity-based algorithm to determine fixations (i.e., fixation duration and fixation number) and saccades from the eye data, using task- and individual-specific thresholds (van der Lans, Wedel, & Pieters, 2011). The algorithm is regarded as ‘machine and sampling frequency independent’ (van der Lans et al., 2011).

Due to the ability of automatically defining task-specific thresholds, this algorithm is more suitable for skill-based studies. Hence, in this study, the BIT algorithm is used to identify the fixation duration, fixation number and the saccades of the eye-gaze data. BIT is also used to classify hand movement events using the data collected within a surgical simulation environment. Since the eye and hand can move at different velocities, the threshold values should be determined automatically for each case with this algorithm. The source code of the algorithm is available on the authors’ website, using MATLAB (van der Lans et al., 2011).

Participants

A total of 15 surgical residents (ten surgeons and five interns) from the Department of Neurosurgery (six participants) and Otolaryngology (ENT) (four participants) from Hacettepe Medical School in Ankara, Turkey participated in this study. There were two skill level groups of participants, intermediate and novice, based on the categorization presented in Silvennoinen et al.’s study (Silvennoinen, Mecklin, Saariluoma, & Antikainen, 2009). In that concern, those who have operated at least one endoscopic surgery are considered as ‘intermediate’, whereas others who have only observed and assisted in endoscopic operations, but have not performed any surgeries by themselves, are considered as ‘novice’. As shown in

Table 1. Information about Participants.

Skill Level	Age	Department		Gender
		NRS	ENT	F	M
Intermediate	28.4	1	5	1	4
Novice	25.6	4	5	1	9
NRS: Neurosurgery ENT: Ear Nose Throat

, most of the participants were male (86.66%).

Detailed information about these participants according to their endoscopic surgical expertise (average number of operations observed, assisted and performed) is given in

Table 2. Participants’ Surgical Experience.

Skill Level	Average number of Endoscopic Surgery
	Observed	Assisted	Performed
Intermediate	52.0	39.6	23.8
Novice	8.2	1.0	0.0

.

Apparatus

Haptic devices can be integrated into training simulations for MIS procedures (Basdogan et al., 2004) in order to provide similar and real-world practice as well as a realistic sense of touch. Accordingly, in this study, a midrange professional ‘Geomagic Touch’ haptic device is used to perform the tasks in the scenarios. The simulation software recorded a hundred data points per second as the hand coordinates for both hands with the help of these haptic devices. Additionally, the eye movement data is gathered by using a 60 Hz eye-tracking device, the Eye Tribe (The Eye Tribe, 2014), which is easy to set up, transportable and reported as appropriate for use in scientific research (Ooms, Dupont, Lapon, & Popelka, 2015). This tool is used to track the user’s eye movements and calculate the on-screen gaze coordinates with an accuracy of 0.5°-0.7°. During the experiment, eye-gaze coordinates were gathered in a top-left oriented 2D coordinate system. The screen resolution is 1920 x 1080 pixels; in other words, the horizontal field of view (x-coordinate) is 1920 pixels, whereas the vertical field of view (y-coordinate) is 1080 pixels. The field of view (FOV) of the camera, from the left-perspective for Scenario-2 can be seen in Figure 1.

In our software, hand motion coordinates, regarded as the position of tool and camera in the scenarios, were represented as 3D vectors. Accordingly, the origins for eye Oeye and for hand Ohand coordinates have been represented in a 2D scene and appear in Figure 2-A and B.

Due to the low sampling rate, the analysis of saccades may become problematic when collecting data using the Eye Tribe tracker. It is reported that measuring saccade metrics require high frequency sampling (Dalmaijer, 2014). However, it has also been stated that velocity-based algorithms for saccade detection that works on high-frequency data do a relatively good job on the Eye Tribe data (Dalmaijer, 2014; Menekse Dalveren & Cagiltay, 2018), but still they fail to be accurate when it comes to saccadic positions (Dalmaijer, 2014). However, there are a number of studies in the literature using 50 Hz (van der Lans et al., 2011) and 60 Hz sampling rate trackers in order to classify eye events into fixations and saccades (Menekse Dalveren & Cagiltay, 2018).

Design

In the experimental study, there were two both-handed scenarios for the surgical training process used. The first one was prepared to practice on general skills, such as learning the use of surgical tools with the endoscope and developing depth-perception in a simulated 3D environment (Scenario-1). The other scenario closer to the operational procedures uses a simulated anatomical nose model (Scenario-2). Each scenario consists of ten repetitive tasks. The layout of the scenarios is shown in Figure 3-A and B, respectively.

‘Moving the Red Ball into the Box’ Scenario

In this scenario (Scenario-1), each participant is asked to approach the red ball with the haptic device, catch it, and then move it into the green box as shown in Figure 3-A. The tool is controlled with the dominant hand of the participant, whereas the camera is controlled by his/her non-dominant hand. The position of the ball and the box changes arbitrarily in each trial. The participant must complete this process successfully, which includes 10 tasks, within the allocated time period. In case of failure to complete each task within 10 seconds, the ball and the box disappear.

‘Clearing the Nose’ Scenario

In this scenario (Scenario-2), the participant must remove the green ball-like objects, which are spread through the nose model as shown in Figure 3-B. The camera is used as the light source and the cautery model as the tool to collect the objects. In case of a collision - that is, if the haptic device touches the tissue - it provides a force feedback that feels as if the device pushes back in the hands of the user.

This study is conducted as part of a research project and the content of both scenarios are prepared based on opinions of the neurosurgery and ENT domain experts. Mainly endoscopic pituitary surgery procedures are aimed to be practiced which is in the scope of both domains and performed starting from the nose holes through the pituitary area. Accordingly, the scenarios are designed for beginners of these operations.

Procedure

As shown in Figure 4, in this study, the research procedure mainly consisted of five stages. These are S1: Experimental study; S2: regenerated simulation version; S3: BIT algorithm; S4: Eye and hand metrics; and S5: Eye-hand coordination analyses.

At the beginning of the experiment, the participants were asked to fill out a questionnaire including their demographic information, dominant-hand and experience level (i.e., years in the department, and the number of operations observed, assisted, and performed). Prior to the experimental study, the participant is seated 70 cm. away from the screen. First, each participant was informed about the calibration process; more specifically that they should maintain the distance and that they are not allowed to move their heads or body once calibration is completed. After this briefing, the calibration process started. Nine calibration points appeared on the screen one after the other, with a viewing time of two seconds each. At the end, a five-star rating is displayed to provide the accuracy of the calibration. If the result of the calibration is four-star (<0.7°) or five-star (<0.5°), then it is regarded as acceptable for the experimental study. After that, a brief instructional video explaining how to perform the task was shown to each participant separately about the procedure. Next, each participant was asked to perform two scenarios, Scenario 1 and 2, using both their dominant and non-dominant hands at the same time. The eye-gaze data (i.e., pupil size, fixation, raw and smoothed X and Y coordinates of both left and right eye) and hand motion data (i.e., tool and camera position, tool and camera rotation as 3D vectors) were collected and stored using a special software (Figure 4: S1). The experimental setup can be seen in Figure 5.

Afterwards, the performance of each participant in both scenarios is regenerated in the simulation environment using a software developed in the Unity environment (Figure 4: S2). This regeneration software took each participant’s eye data with the help of the tracker device and the hand data from the haptic device while they were performing each task during the experimental study. In this way, each participant’s eye and hand coordinates are aligned within the same time scale by representing the eye location through an eye icon and hand location through a small blue sphere in the regenerated simulation re-play (See Figure 6).

This regenerated simulation allows researchers to determine the extent regarding tissue-contact, left-right hand coordination and the eye-hand coordination of each individual. In the meantime, the observation data was gathered through a questionnaire to understand such behavior differences between novice and intermediate groups. Five observers (other than the researchers) have evaluated the participants’ performances in Scenario-2. As this scenario is based on an environment similar to the operational procedures (Figure 4: S3), the observations are conducted on this scenario. All of the observers are the graduate students in the field of engineering and briefed about the observation procedure before they start the evaluations.

Using this approach, the raw coordinates for both the eye and hand movement data aligned in the time scale were obtained as the output for regenerated simulated version of the scenarios. Then, these coordinates were given as an input to the BIT Algorithm (Figure 4: S4). Afterwards, the classification process of the eye and hand movements - as either fixation or saccade events- were identified by running the algorithm, separately for the eye data and the hand data. The output of the BIT algorithm performed on the eye data is the fixation and saccade metrics, whereas metrics related hand movement were obtained as the outputs from the hand data (Figure 4: S5). Finally, all collected data is analyzed to better understand the efficiency of the proposed metrics and to objectively measure the eye-hand coordination skills of the participants (Figure 4: S6).

Metrics

In this study, using the BIT algorithm, three metrics are identified, namely Fixation Duration (FD) (the time from one saccade to another), Fixation Number (FN) (the number of fixations in an interval) and Saccade Number (SN) (the number of saccades in an interval).

As explained in the procedure, the eye movements and the hand movements of the participants are aligned in the same time scale. The authors believe that further insight regarding the eye-movement events can be also be applied to the hand movement events as well. Our main aim in this study is to test this assumption by comparing the results conducted in this study with the ones that are reported in earlier studies on eye-hand coordination skills of the surgeons. Accordingly, in the present work, new hand metrics are introduced to identify the hand movements of the participants as explained below:

The ‘Stand Still’ metric is proposed in this study as the period when the hand movement remains within a very small range and lower velocity for some time. In other words, it determines the ‘idle state’ of the hand movement. By running the BIT algorithm, such events can be classified into ‘Stand Still Duration’ (SSD) and ‘Stand Still Number’ (SSN) for hand movements. In this respect, the ‘Sudden Sharp Movement’ (SSM) metric is also proposed to identify very fast, sharp hand movements while performing any given task.

Results

A correlation analysis is performed to assess the relationship between eye-gaze and hand-motion metrics considering three pairs: the fixation duration of the eye-gaze (FD) and stand still duration of the hand motion (SSD), the fixation number of the eye-gaze (FN) and stand still number of the hand motion (SSN), and the saccade number of the eye-gaze (SN) and the sudden sharp movement (SSM) for hand motion, where the correlation coefficient r is calculated. The value of | r | from 0.1 to 0.3 represents a small correlation. From 0.3 to 0.5, the value represents a moderate correlation, while larger than 0.5 shows a strong correlation as reported by Cohen (Cohen, 1988; Laerd Statistics, 2017).

Eye- Hand Correlation Results for Scenario-1

In Scenario-1, the descriptive statistics for the two groups of participants (intermediate and novice) for both the eye metrics (FD, FN, and SN), and the hand metrics (SSD, SSM and SSN) are depicted in Figure 7. Additionally, the mean and standard deviations for all metrics are in Scenario-1 is shown in

Table 3. Descriptive Results for the Metrics in Scenario-1.

	Intermediate		Novice
Eye Metrics	M	SD	M	SD
FD	121.53	19.05	112.70	26.55
FN	12.14	1.91	11.26	2.66
SN	20.80	13.46	28.60	32.46
Hand Metrics	M	SD	M	SD
SSD	92.99	10.25	101.98	15.03
SSN	9.30	1.01	10.19	1.51
SSM	55.80	37.48	55.10	15.82

.

Eye-Hand Correlation for Intermediates

A Pearson’s product-moment correlation was run to assess the relationship between the eye-gaze and hand motion metrics, (FD- SSD and FN- SSN) and saccades (SN- SSM) for intermediates. For all of these three pairs, the preliminary analyses showed the relationship to be linear with both variables normally distributed, as assessed by the Shapiro-Wilk’s test (p > .05), and there were no outliers. Also, there was a strong negative correlation between the FD – SSD and FN- SSN metrics among the intermediates, r = -.836 and r = -.837, respectively. On the other hand, a strong positive correlation existed between the saccade metrics SN and SSM in the same group r = .755 (

Table 4. Eye- Hand Correlation Results for Scenario-1.

Skill Level	FD - SSD	FN - SSN	SN - SSM
Intermediate	-.836 Strong-	-.837 Strong-	.755 Strong+
Novice	.448 Moderate +	.448 Moderate+	.590 Strong+

).

Eye-Hand Correlation for Novices

Again, a Pearson’s product-moment correlation was run to assess the relationship between the eye-gaze and hand motion metrics, and fixations (FD- SSD and FN- SSN) and saccades (SN- SSM) for novices. For the first two pairs related to fixation, preliminary analyses showed the relationship to be linear with both variables normally distributed, as assessed by the Shapiro-Wilk’s test (p > .05) with no outliers. There was a moderate positive correlation for both the FD- SSD and FN- SSN metrics among the novice participants, r = .448. However, not all variables were normally distributed for the saccade metrics SN and SSM, as assessed by the Shapiro-Wilk’s test (p < .05). Accordingly, a Spearman’s rank-order correlation was run to assess the relationship between the saccade number of eye-gaze data and the sudden sharp movements of the hand data, with results showing a strong positive correlation for the SN and SSM measures, rs = .590 (Table 4).

Eye- Hand Correlation Results for Scenario-2

Figure 8 show the descriptive statistics for the eye (FD, FN, and SN) and hand metrics (SSD, SSM, SSN) for the two groups in Scenario-2. Additionally, the mean and standard deviations for all metrics are in Scenario-2 is depicted in

Table 5. Descriptive Results for the Metrics in Scenario-2.

	Intermediate		Novice
Eye Metrics	M	SD	M	SD
FD	151.25	30.48	133.42	42.49
FN	15.11	3.04	13.34	4.24
SN	41.40	23.58	91.90	82.30
Hand Metrics	M	SD	M	SD
SSD	121.37	16.60	118.28	15.13
SSN	12.15	1.65	11.84	1.52
SSM	395.00	143.63	486.00	143.86

.

Eye-Hand Correlation for Intermediates

A Spearman’s rank-order correlation was run to assess the relationship between the FD- SSD and FN- SSN measures since not all the variables were normally distributed, as assessed by the Shapiro-Wilk’s test (p < .05). There was a strong negative correlation for the pairs related to fixation that is the FD- SSD and FN- SSN metrics. In other words, an increase in the eye fixation duration and fixation number was strongly correlated with a decrease in the hand stand still duration and stand still number among the intermediates, rs = -.900, p < .05 (

Table 6. Eye- Hand Correlation Results for Scenario-2.

Skill Level	FD - SSD	FN – SSN	SN - SSM
Intermediate	-.900* Strong-	-.900* Strong-	.846 Strong+
Novice	-.443 Moderate-	-.441 Moderate-	.06 Small+

*Note that the correlation is significant at the 0.05 level.

). However, both variables of the saccade metrics, SN and SSM, were normally distributed, as assessed by the Shapiro-Wilk’s test (p > .05). Also, a Pearson’s correlation was run to assess the relationship between the saccade number of eye-gaze data and the sudden sharp movements of the hand data, with the outcome that there was also a strong positive correlation for the SN and SSM metrics, r = .846 (Table 6).

Eye-Hand Correlation for Novices

A Pearson’s product-moment correlation was run to assess the relationship between the eye-gaze and hand motion metrics, the fixation (FD- SSD and FN- SSN) and saccades (SN- SSM) for novices. For the first two pairs related to fixation, preliminary analyses showed the relationship to be linear with both variables normally distributed, as assessed by the Shapiro-Wilk’s test (p > .05), and there were no outliers. There was a moderate negative correlation for both FD- SSD and FN- SSN metrics in novices, r = -.443 and -.441, respectively. However, not all variables were normally distributed for the saccade metrics SN and SSM, as assessed by the same test (p < .05). Accordingly, a Spearman’s rank-order correlation was run to assess the relationship between the saccade number of the eye-gaze data and sudden sharp movements of the hand data. The results show a small positive correlation for the SN and SSM metrics for novices, rs = .06 (Table 6).

Analyzing the Questionnaire Data

Five observers have monitored the participants’ performances in Scenario-2 in order to assess the left-right hand coordination and eye-hand coordination skills of the participants. Their evaluation is based on the items given in

Table 7. Descriptive Results for Questionnaire Analysis of the Observers.

	Intermediate		Novice
Questionnaire Item	M	SD	M	SD
The participant shows developed depth perception skills in a 3D environment.	3.33	.42	2.12	.34
The participant shows developed skills in eye-hand coordination.	3.53	.30	1.92	.52

to be ranked in accordance to five alternatives (1: Strongly Disagree, 5: Strongly agree) in a Likert scale-type questionnaire. 11 out of 15 participants (3 intermediates, 8 novices) were evaluated using the questionnaire data. The descriptive results appear in Table 7.

A Mann-Whitney U test was run to determine if there were any differences in the scores for the given expressions (Table 7) between the intermediate and novice groups. The distributions of the scores for intermediates and novices were found to be dissimilar among the participants once inspected visually. According to the results, considering these expressions, the 3D depth perception and eye-hand coordination skills of the intermediates (mean rank = 10.00) were significantly higher than novices (mean rank = 4.50), U=0, z= -2.461, p=.012.

Discussion

The present study involved two main goals: first, to propose new objective metrics by adapting our knowledge in the field of eye-movement events; and, second, to test the appropriateness of these metrics in the endoscopic surgery field to objectively measure the eye-hand coordination skill levels of the surgeons.

Previous studies have attempted to evaluate the level of surgeons’ eye-hand coordination in the field of endoscopic surgery through some related task analysis (Andreatta, Woodrum, Gauger, & Minter, 2008; McDougall et al., 2006; Yamaguchi et al., 2007) through eye movement analysis (Wilson et al., 2010) or eye movements and arm movements (Snyder, Calton, Dickinson, & Lawrence, 2002). Hence, there are no current studies designed to understand the eye-hand coordination skills by analyzing both the hand movements and eye-movements of surgeons working in coordination. Accordingly, the proposed measures provide alternatives to understand such eye-hand coordination skills by analyzing the hand-movements and eye-movements of surgeons.

Based on the results of this study, in both scenarios, there exists a correlation between the average measured values of the three eye-gaze and hand motion metrics. The fixation metrics (FD-SSD and FN-SSN) are found strongly correlated for the intermediates, and moderately correlated for the novices. This outcome indicates that the novices require improvements in the related eye-hand skills. In other words, the intermediate participants’ eye-hand coordination skills are better improved compared to the novices. The results also show that, in Scenario-1, once the average fixation duration (FD) and the fixation number (FN) of intermediates increase, their average stand still duration (SSD) and stand still number (SSN) decreases. The increase in the fixation duration can be observed because of an increase in their concentration while performing the designated tasks. In other words, when their concentration increases, their hand movements become smoother and fewer occurrences of standstill take place, indicating serial and smoother hand movements. This result supports the earlier studies which reported that skilled surgeons’ hand performances are more stable than less-experienced ones (Uemura et al., 2014), and that the fixation number of experts are higher in task-relevant areas (Gegenfurtner, Lehtinen, & Säljö, 2011), but lower in task-redundant areas (Gegenfurtner et al., 2011).

Different from intermediates, in both Scenario-1, an increase in the average fixation duration (FD) and fixation number (FN) of the eye-gaze among the novice participants correlates with an increase in their hand motion metrics (stand still duration (SSD) and stand still number (SSN)). At this stage, it can be inferred that the novices’ hand movements experience more idle or stand still states when their eye fixation increases. This finding is supportive of earlier results reporting that, while manipulating the tool, experts look directly at the target location, whereas novices track the movement of the tool until it reaches the target location (Law, Atkins, Kirkpatrick, & Lomax, 2004). On the other hand, in Scenario-2, novices performed better in terms of eye-hand correlation, where the strength of the correlation remains the same as moderate, but the direction changes from positive to negative. This result indicates that in Scenario-2, when the eye fixation increases their hand movements also become smoother. This may be an indicator that their eye-hand movement coordination is improved for this specific task in Scenario-2 after practicing in Scenario-1.

A strong correlation occurs between SN and SSM in both scenarios for intermediates. In Scenario-2 the correlation coefficient increases slightly for this group. The result shows that, when there is an increase in the average number of saccades in the intermediates’ eye movements, their average sudden sharp movement also increases. Similarly, in Scenario-1 a strong correlation is also found for the novices on SN and SSM. However, in Scenario-2, this value is significantly smaller for the novice group, indicating that there is a need for the novices’ to improve their eye-hand coordination skills, while further supporting an earlier study which reported that novices control on the environment and monitor is lower compared to the experts (G. Tien, Atkins, Zheng, & Swindells, 2010). In this earlier study, the researchers analyzed eye movements of experts and novices and reported that novices concentrate on the surgical display and, as such, lose track of patient’s status, whereas experts also observe these conditions at the same time (G. Tien et al., 2010).

Similarly, the results obtained based on the questionnaire data considering the observers’ evaluations on the skill levels of the intermediates in terms of their 3D depth perception and eye-hand coordination are reported as higher than that of the novices.

Conclusion

This study has two major contributions. Firstly, new hand movement metrics are proposed by adapting an open-source eye-movement classification algorithm (BIT) to the data collected through a computer-based simulation software using a haptic interface and eye-tracker. Secondly, it can be stated that such metrics and eye-hand correlation analyses can be used for the objective assessment of the skill levels required for endoscopic surgery trainees. As studies on eye-hand coordination in the literature are very limited, we believe that analyzing surgeons’ hand and eye data jointly is an important contribution to objective evaluations and assessment of their skills and development of some standards for surgical education programs.

In the literature, dispersion-based algorithms are recommended for eye movement event classification while using a low-frequency eye tracking device. However, in this study, an adaptive algorithm is used because the velocity of the eye and hand movements can be different. Due to the ability of automatically defining task- and individual-specific thresholds and being machine and sampling frequency independent, we believe that the BIT algorithm proposed by (van der Lans et al., 2011) is suitable for skill-based studies.

Limitations and Future Work

As it is commonly the case, the number of surgeons in the neurosurgery and ENT departments is very limited, for which reason this study was conducted with 15 participants and only intermediate and novice surgeons. In the future attempts, it may be possible to validate the results of this study with a larger number of participants and also upon a wider range of skill levels. Additionally, in the future, experimental studies should be conducted with the help of scenarios in a larger scope aiming at different tasks with different difficulty levels and under different conditions. For instance, in the present work only both-handed task performances were evaluated; whereas later, the participants’ performances may also be compared in dominant, non-dominant and both hands settings.

As we were unable to obtain the values for the exact time stamps of the provided events, in this study the average values for each metric could be analyzed because of the values provided by the BIT algorithm. Due to this drawback, further analysis needs to be carried out to examine whether the eye and hand events occur in a synchronized way using other appropriate algorithms. Also, the differences among the results of different algorithms can also be further analyzed.

Lastly, Scenario-1 in this study was designed in a more general sense and to gain endoscopic skills, while Scenario-2 was arranged as more specific to the endo-neurosurgery tasks. Accordingly, the experimental study is conducted in the same order as the scenarios (Scenario1 is performed first and followed by Scenario-2). However, to eliminate the order effect in learning, with more scenarios this effect can be brought under control.