Effects of Haptic Feedback on Precision Peg Insertion Tasks Under Different Visual and Communication Latency Conditions

Abstract

This study investigated the role of haptic feedback in precision peg insertion tasks conducted via teleoperation under varying visual resolution and communication latency conditions. Experiment 1 examined the combined effects of haptic feedback and the visual resolution, revealing that haptic feedback significantly reduces the maximum normal force and mental workload, while enhancing subjective operability, particularly in low-visual-resolution conditions. Experiment 2 evaluated the impact of communication latency, showing that the maximum normal force, operability, and mental workload are affected by increased latency. Notably, the maximum normal force is sensitive even to minimal latency (100 ms), whereas the mental workload and operability remain acceptable under lower-latency conditions. These findings underscore the importance of multi-metric evaluations, as different aspects of performance respond differently to latency. Overall, the results demonstrate the critical role of haptic feedback in enhancing task performance and the user experience in teleoperated precision tasks, offering valuable insights for the design and development of more effective and user-friendly teleoperation systems.

1. Introduction

1.1. Background

Industrial robots, specifically multi-joint manipulators, have been widely introduced in various situations, demonstrating significant benefits [1,2]. These robots are primarily employed as tools for automation. For example, in the automotive manufacturing industry, robots are used to automate processes such as material handling, welding, painting, and assembly, contributing to reduced manufacturing costs and improved production efficiency [3]. In recent years, industrial robots have also been applied in sectors such as agriculture [4,5] and food processing [6]. Furthermore, collaborative robots, designed to work alongside humans, have been developed, expanding the scope of robot applications [7].

Despite these advancements, fully automating all tasks in diverse and complex industrial environments remains a challenge. One major limitation arises from the highly variable and rare nature of failure scenarios during automation. While humans can adapt flexibly to diverse situations, incorporating such adaptability into automated processes is difficult. For instance, in automated tasks such as bin-picking based on camera information, unexpected factors such as additional attachments, deformed objects, or overlapping items may lead to recognition errors, making it difficult to correctly identify the state of objects.

As a practical solution to such failures, human operators are often deployed on site to recover from automation errors. However, this approach increases operational costs, particularly when failures are infrequent but require immediate intervention.

To address these challenges, teleoperation/telerobotics systems [8,9] are gaining attention as a promising solution. Teleoperation systems enable human operators to remotely intervene and resolve issues when automation processes encounter failures. By leveraging teleoperation systems, it becomes possible to respond flexibly to diverse situations without the need for an on-site human presence, potentially reducing operational costs.

For the successful integration of teleoperation systems in industrial settings, it is essential to determine the type of information to be transmitted based on specific criteria. Even for the same task, variations in the information transmitted to operators can influence task performance. Moreover, the optimal information for enhancing performance may vary depending on the task type. Therefore, understanding the quantitative relationship among the transmitted information, task characteristics, and operational performance across typical tasks is crucial. Such insights can provide valuable guidance for implementing teleoperation systems.

This study focused on peg-in-hole tasks as a representative example of contact-intensive tasks in manufacturing. Unlike non-contact tasks such as welding, peg-in-hole tasks are hypothesized to require not only visual information but also tactile information generated during contact. Humans achieve complex operations by perceiving the direction and magnitude of forces during contact, and it is believed that the transmission of tactile information is equally important in teleoperated tasks.

Although numerous studies have demonstrated the effectiveness of haptic feedback in teleoperation, little research has examined the interplay between visual and haptic information in assembly tasks. This study experimentally investigated how the presence or absence of haptic feedback influences task performance under conditions with different resolutions of visual information. In addition to obtaining objective performance metrics, subjective evaluations by operators were conducted to assess various effects comprehensively.

Furthermore, in real-world teleoperation scenarios, the communication latency increases as the physical distance grows, which may impact task performance. This study also evaluated the effectiveness of haptic feedback in environments with different levels of communication latency, considering both subjective and objective perspectives.

1.2. Related Works

1.2.1. Haptic Support System

To clarify the specific type of haptic information focused on in this study, we surveyed a wide range of teleoperation support systems utilizing haptic information reported in the literature. Haptic information has emerged as a promising approach for supporting teleoperation tasks [10,11]. There are two main approaches to utilizing haptic information: the haptic feedback approach, where forces and accelerations encountered by the robot are transmitted to the operator, and the haptic guidance approach, which provides force or vibration cues to guide the operator toward target positions or along desired trajectories [12,13,14]. This study focused on the former, the haptic feedback approach.

Various types of haptic information can be transmitted as feedback. For tasks such as polishing, cutaneous haptic information, including acceleration vibrations and skin deformations [15,16,17,18,19,20], may be beneficial. However, for tasks with greater environmental constraints, such as peg-in-hole tasks, the use of multi-degree-of-freedom force information [21,22] is expected to be more effective.

Therefore, this study focused on force feedback and investigated how its effectiveness changes under various conditions.

1.2.2. Haptic Feedback for Teleoperation

In teleoperation without time latencies, it has been demonstrated that haptic feedback has various quantitative effects on task performance for different types of robots. Wagner et al. reported the effects of haptic feedback on surgical robots [23]. Carvalho et al. investigated the effects of haptic feedback on excavation robots for construction sites [24]. Morosi et al. studied the effects of haptic feedback on construction robots for excavation tasks in a simulation environment [25]. Ardakani et al. reported the effects of haptic feedback on peg-in-hole tasks [26]. While many studies have reported the quantitative effects of haptic feedback on teleoperation performance, this study investigated not only the quantitative effects but also the subjective aspects, such as the operator’s perceived workload.

On the other hand, in teleoperation, visual feedback, such as camera information, is usually provided in conjunction with haptic feedback [24,25,26,27]. It is rare to perform tasks without visual feedback. In visual feedback, there are various conditions such as the frame rate and resolution. One of the key research questions addressed in this study was, “To what extent does the effectiveness of haptic feedback vary under different visual feedback conditions”?

Next, in teleoperation with time latencies, it has also been demonstrated that haptic feedback has various quantitative effects on task performance [28]. Arata et al. investigated the effects of time latencies and the presence of force feedback on objective task performance for teleoperated surgical robots [29]. Hannaford et al. and Yip et al. studied the effects of time latencies on task performance using teleoperation systems with force feedback for peg-in-hole tasks [30]. However, these studies focused only on objective metrics such as the applied forces and did not consider subjective effects on the operator. This study examined the effects of different latency conditions on the performance of haptic feedback based on both objective and subjective metrics.

1.3. Objective

This study investigated the impact of force feedback on task performance during assembly tasks conducted via teleoperated robots. First, the relationship between visual feedback and force feedback was examined. Visual feedback was manipulated by varying the resolution of camera images presented to the operator, while force feedback was controlled by its presence or absence. Next, the effect of communication latency was explored. The teleoperation performance with force feedback was tested under conditions with varying latency values. This study focused on a peg-in-hole task, a challenging assembly task, and evaluated task performance through human subject experiments. Both quantitative evaluations and subjective assessments were conducted to comprehensively analyze the effects of force feedback.

2. Method

2.1. System Architecture

A teleoperation system capable of transmitting both visual and haptic information is shown in Figure 1. Force data measured by the force sensor installed at the base of the remote hand are transmitted to the operator through a haptic interface on the local side. Additionally, video streams captured by two cameras on the remote side are displayed on a monitor on the local side.

The two cameras consist of an overhead camera, which provides an overhead view of the robot and its environment, and a hand-mounted camera, which offers a detailed view of the area around the robotic hand.

The local interface utilizes a device (Omega.6, Force Dimension, Nyon, Switzerland) capable of inputting 6-degrees-of-freedom (DoF) position and orientation information and outputting the 3-DoF translational force. In this study, only positional information was transmitted, and orientation information was not used; the robot’s orientation was fixed.

As the remote robot, a 6-DoF industrial robot (RS007L, Kawasaki Heavy Industries, Ltd., Tokyo, Japan) was employed. At the robot’s end effector, a two-finger robotic hand (2F-140, Robotiq, Quebec City, QC, Canada) was attached, and its gripper was replaced with a custom-designed component optimized for gripping pegs. A force sensor (Axia80-M20, ATI Industrial Automation, Apex, NC, USA) was installed between the robot and the hand.

2.2. Bilateral Control Method

In teleoperation systems, there are mainly two types of control methods: unilateral control and bilateral control. Unilateral control transmits the position information input from the local side to the remote side in a one-way manner. On the other hand, bilateral control extends unilateral control by also transmitting feedback information, such as reaction forces acting on the robot, back to the local side. Bilateral control enables the operator to perceive contact forces exerted on the robot, allowing for precise and delicate operations.

Various types of bilateral control exist, including the position symmetric type, force feedback type, and force reflection type. In this study, we implemented force reflection-type bilateral control, which provides stable force feedback while maintaining operability. The control method for position and force feedback is described below.

2.2.1. Position Control

The position of the leader device, denoted as ${\mathit{p}}_{\mathrm{l}}\in {\mathbb{R}}^3$, commands the follower robot position ${\mathit{p}}_{\mathrm{f}}\in {\mathbb{R}}^3$, following the position control law

$${\mathit{p}}_{\mathrm{f}}={\mathit{K}}_{\mathrm{p}}{\mathit{p}}_{\mathrm{l}},$$

(1)

where ${\mathit{K}}_{\mathrm{p}}$ is a $3\times 3$ diagonal stiffness matrix given by

$${\mathit{K}}_{\mathrm{p}}=\left[\begin{array}{ccc}0.1& 0& 0\\ 0& 0.1& 0\\ 0& 0& 0.1\end{array}\right].$$

(2)

2.2.2. Force Feedback

Force feedback is provided by transmitting the reaction force measured on the follower side, ${\mathit{F}}_{\mathrm{f}}\in {\mathbb{R}}^3$, back to the leader device, ${\mathit{F}}_{\mathrm{l}}\in {\mathbb{R}}^3$, following the force feedback control law

$${\mathit{F}}_{\mathrm{l}}=-{\mathit{K}}_{\mathrm{f}}{\mathit{F}}_{\mathrm{f}},$$

(3)

where ${\mathit{K}}_{\mathrm{f}}$ is a $3\times 3$ diagonal stiffness matrix defined as

$${\mathit{K}}_{\mathrm{f}}=\left[\begin{array}{ccc}0.1& 0& 0\\ 0& 0.1& 0\\ 0& 0& 0.1\end{array}\right].$$

(4)

These control parameters were chosen to maintain system stability and ensure smooth interaction between the operator and the remote robot. To ensure stable bilateral control and real-time operation, both the robot control and force feedback update rates were 500 Hz.

3. Experiment 1: Relationships Between Visual and Force Feedback Conditions

3.1. Experimental Task

The experimental task was to move from an initial position at a fixed distance (approximately 150 mm) from the hole and insert a peg into the hole, continuing until the peg reached a specified depth in the hole. In the initial state, the robot hand was set to hold the peg. The diameters of the peg and the hole were $9.5\mathrm{mm}$ and $10.0\mathrm{mm}$, respectively.

3.2. Experimental Participants

A total of 10 participants, consisting of 9 males and 1 female in their 20s, took part in the experiment. All participants were right-handed.

3.3. Experimental Conditions

Four feedback conditions were established by combining two levels of visual feedback resolution (high and low) and the presence or absence of haptic feedback. The visual feedback conditions were defined by the resolution of the hand-mounted camera, with two levels, $640\times 360$ and $320\times 180$, as shown in Figure 2. The $640\times 360$ resolution corresponded to the “High visual” condition, while the $320\times 180$ resolution corresponded to the “Low visual” condition. The resolution of the overhead camera (Figure 3) was fixed at $800\times 600$. Although higher-resolution communication environments can also be set up, the present conditions were adopted in this study, assuming a classical space that is not necessarily well maintained, such as a classical factory. Figure 4 shows the experimental environment.

Four trials were conducted for each condition, and the order of the four conditions was randomized. In each condition, we preliminarily allowed time for participants to become familiar with the robot’s controls. There were short breaks between each test and also for a few minutes every 30 min. Participants commented that they did not feel physically fatigued after a period of time during the experiment. The total experiment duration was approximately 2 h. Figure 5 shows an example trial and includes the z-axis position during the trial, the contact normal force, and a hand camera time lapse.

As objective evaluation metrics, the maximum normal force and task completion time were analyzed. As subjective evaluation metrics, three questionnaire items (operability, physical demand, and mental demand) were rated on a five-point scale as shown in

Table 1. Questionnaire for experiment.

Item	1 (Bad)/5 (Good)	Description
Operability	Difficult/Easy	How easy was it to operate?
Physical demand	High/Low	How much physical demand did you feel?
Mental demand	High/Low	How much mental demand did you feel?

. Participants completed the questionnaire after all four trials under each condition.

3.4. Experimental Results

3.4.1. Task Completion Time

Figure 6 shows the task completion times under the four feedback conditions. The variance observed in the experimental data included variation across participants and within individual trials. The data presented in Figure 6 represent the results of each trial, rather than aggregated participant means. For example, some trials had minimal or no peg-to-hole contact.

The normality of the task completion time data was assessed for each condition using the Shapiro–Wilk test. The results indicated that the “Low visual”, “Low visual + Haptic”, and “High visual” conditions did not meet the assumption of normality ($p<0.050$). As the majority of conditions exhibited non-normal distributions, non-parametric statistical methods were employed for further analysis.

To evaluate the overall differences in the task completion time among the four conditions, a Friedman test was conducted. The results showed a significant difference (${\chi }^2\left(3\right)=15.81$, $p=0.0012$). To investigate specific differences between conditions, pairwise comparisons were conducted using the Mann–Whitney U test with Bonferroni correction applied $(\alpha =0.05/4)$. The results are summarized as follows:

• “Low visual” vs. “High visual”: No significant difference was observed (corrected $p=0.099$).
• “Low visual” vs. “Low visual + haptic”: No significant difference was observed (corrected $p=0.69$).
• “High visual” vs. “High visual + haptic”: No significant difference was observed (corrected $p=1.0$).
• “Low visual + haptic” vs. “High visual + haptic”: No significant difference was observed (corrected $p=0.46$).

No significant differences were observed in the pairwise comparisons. These results indicate that task completion times did not vary with the resolution of visual feedback information or the presence or absence of force feedback. However, it is possible that neither the work difficulty nor work performance had changed, which will be analyzed in subsequent sections.

Despite the differences in feedback conditions, the results indicate no significant differences in the task completion time. Similar findings have been reported in a related study on teleoperation [19], suggesting that the introduction of additional sensory feedback does not necessarily lead to faster task execution. One possible explanation is that when new information is provided, operators tend to focus more on processing that information, leading to more careful and deliberate task execution.

3.4.2. Maximum Normal Force

Figure 7 shows the maximum normal force under the four feedback conditions. The normality of the data distribution for each condition was assessed using the Shapiro–Wilk test. The results indicated that all four conditions did not meet the assumption of normality ($p<0.0010$). As a result, non-parametric methods were employed for further analyses to account for the non-normal distribution of data.

To evaluate the overall differences among the four conditions, a Friedman test was conducted. The results revealed a significant overall effect ($p<0.0010$), indicating that at least one condition differed significantly from the others.

To investigate specific differences between conditions, pairwise comparisons were performed using the Mann–Whitney U test with Bonferroni correction applied to control for multiple comparisons ($\alpha =0.05/4$). The key results are summarized below:

• “Low visual” vs. “High visual”: No significant difference was found (corrected $p=0.23$).
• “Low visual” vs. “Low visual + haptic”: A significant difference was observed (corrected $p<0.0010$).
• “High visual” vs. “High visual + haptic”: A significant difference was observed (corrected $p<0.0010$).
• “Low visual + haptic” vs. “High visual + haptic”: No significant difference was found (corrected $p=1.0$).

Specific pairwise comparisons showed significant differences in peak force values between the “Low visual” and “Low visual + haptic” conditions and the “High visual” and “High visual + haptic” conditions. These findings suggest that providing haptic feedback statistically reduces the contact force irrespective of the resolution of the visual feedback.

3.4.3. Operability

Figure 8 shows the subjective operability scores under the four feedback conditions. The normality of the data distribution for each condition was assessed using the Shapiro–Wilk test. The results indicated that the “Low-Visual+Haptic” and “High-Visual+Haptic” conditions did not satisfy the assumption of normality ($p<0.0010$). As a result, non-parametric methods were employed for further analyses to account for the non-normal distribution of data.

To evaluate the overall differences among the four conditions, a Friedman test was conducted. The results revealed a significant overall effect ($p<0.0010$), indicating that at least one condition differed significantly from the others. To investigate specific differences between conditions, pairwise comparisons were performed using the Mann–Whitney U test with Bonferroni correction applied to control for multiple comparisons. The key results are summarized below:

• “Low visual” vs. “High visual”: No significant difference was found (corrected $p=0.11$).
• “Low visual” vs. “Low visual + haptic”: A significant difference was observed (corrected $p=0.023$).
• “High visual” vs. “High visual + haptic”: No significant difference was found (corrected $p=0.10$).
• “Low visual + haptic” vs. “High visual + haptic”: A significant difference was observed (corrected $p=0.046$).

Specific pairwise comparisons showed significant differences in operability scores between the “Low visual” and “Low visual + haptic” conditions and the “Low visual + haptic” and “High visual + haptic” conditions. These findings suggest that providing haptic feedback can statistically enhance subjective operability under certain conditions, though its effectiveness may vary depending on the resolution of the visual feedback.

3.4.4. Physical Demand

Figure 9 shows the subjective physical demand scores under the four feedback conditions. The normality of the data distribution for each condition was assessed using the Shapiro–Wilk test. The results indicated that the “Low visual” condition did not meet the assumption of normality ($p=0.0085$). Due to the presence of a non-normal distribution in one condition, non-parametric methods were employed for further analyses.

To evaluate the overall differences in the physical demand among the four conditions, a Friedman test was conducted. The results did not reveal a significant overall effect ($p=0.097$), indicating that there were no statistically significant differences among the conditions. Given the absence of significant overall differences in the Friedman test, no pairwise comparisons were conducted.

The results from the statistical analyses indicated that there were no significant differences in subjective physical demand scores among the four feedback conditions. This suggests that the physical demand experienced by participants was not strongly influenced by the feedback modalities. This is not necessarily a negative outcome, as force feedback could potentially increase the perceived physical effort due to the additional forces exerted on the operator. The fact that physical demand scores did not increase suggests that the introduction of force feedback did not impose an additional physical burden on the participants.

3.4.5. Mental Demand

Figure 10 shows the subjective mental demand scores under the four feedback conditions. The normality of the data distribution for each condition was assessed using the Shapiro–Wilk test. The results indicated that none of the conditions met the assumption of normality ($p<0.050$). As a result, non-parametric methods were employed for further analyses to account for the non-normal distribution of data.

To evaluate the overall differences in the mental demand among the four conditions, a Friedman test was conducted. The results revealed a significant overall effect ($p<0.0010$), indicating that at least one condition differed significantly from the others. To investigate specific differences between conditions, pairwise comparisons were performed using the Mann–Whitney U test with Bonferroni correction applied to control for multiple comparisons ($\alpha =0.05/4$). The key results are summarized below:

• “Low visual” vs. “High visual”: A significant difference was observed (corrected $p=0.011$).
• “Low visual” vs. “Low visual + haptic”: A significant difference was observed (corrected $p=0.0021$).
• “High visual” vs. “High visual + haptic”: A significant difference was observed (corrected $p=0.0012$).
• “Low visual + haptic” vs. “High visual + haptic”: A significant difference was observed (corrected $p=0.0083$).

Pairwise comparisons further revealed significant differences between all condition pairs, demonstrating that both visual and haptic feedback modalities had a substantial impact on mental demand scores. These findings suggest that the combination of haptic feedback and high-resolution visual feedback significantly altered the perceived mental workload compared to other conditions.

3.5. Experiment 2: Effect of Latency

3.6. Experimental Task

The task involved peg-in-hole insertion. The experimental details were the same as those in Experiment 1.

3.7. Experimental Conditions

The experimental conditions consisted of the presence or absence of haptic feedback and five levels of latency. The two conditions for haptic feedback were defined by whether or not the contact force measured by the force sensor was presented via the haptic device. The latency conditions included five levels, $0,100,200,300$, and $600\mathrm{ms}$, where the latency values represent round-trip times. As the system inevitably contained a slight delay, it is correct to call the present parameter an additional delay.

Each condition was tested with three trials, and the order of the five conditions was randomized. The total duration of the experiment was approximately 2 h. Breaks were set up as in Experiment 1.

The objective and subjective evaluation metrics were the same as those used in Experiment 1.

3.8. Experimental Participants

A total of 10 participants, consisting of 9 males and 1 female in their 20s, participated in the experiment. Among the participants, nine were right-handed, and one was left-handed.

3.9. Experiment Results

3.9.1. Task Completion Time

Figure 11 shows the results for the task completion time under five latency conditions (0, 100, 200, 300, 600 [ms]). The normality of the data distribution for each latency condition was assessed using the Shapiro–Wilk test. The results indicated that the “100 ms” condition did not satisfy the assumption of normality ($p=0.049$). As a result, non-parametric methods were employed for further analyses to account for the non-normal distribution of data in one condition.

To evaluate the overall differences among the five latency conditions, a Friedman test was conducted. The results revealed a significant overall effect (${\chi }^2\left(4\right)=19$, $p<0.0010$), indicating that at least one latency condition differed significantly from the others.

To investigate specific differences between latency conditions, pairwise comparisons were conducted using the Mann–Whitney U test with Bonferroni correction applied to control for multiple comparisons ($\alpha =0.05/4$). The key results are summarized below:

• A latency of “0 ms” vs. “100 ms”: No significant difference was observed (corrected $p=1.0$).
• A latency of “0 ms” vs. “200 ms”: No significant difference was observed (corrected $p=1.0$).
• A latency of “0 ms” vs. “300 ms”: No significant difference was observed (corrected $p=1.0$).
• A latency of “0 ms” vs. “600 ms”: A significant difference was observed (corrected $p=0.021$).

Specific pairwise comparisons revealed that significant differences were only observed between the “0 ms” and “600 ms” conditions. This suggests that higher latency (600 ms) had a statistically significant impact on performance compared to zero latency, while differences between other latency pairs were not significant.

3.9.2. Maximum Normal Force

Figure 12 shows the maximum normal force under the five latency conditions. The normality of the data distribution for each latency condition was assessed using the Shapiro–Wilk test. The results indicated that none of the conditions satisfied the assumption of normality ($p<0.010$). As a result, non-parametric methods were employed for further analyses.

To evaluate the overall differences in the maximum normal force among the five latency conditions, a Friedman test was conducted. The results revealed a significant overall effect (${\chi }^2\left(4\right)=52$, $p<0.00010$), indicating that at least one latency condition differed significantly from the others.

To investigate specific differences between latency conditions, pairwise comparisons were conducted using the Mann–Whitney U test with Bonferroni correction. The key results are summarized below:

• A latency of “0 ms” vs. “100 ms”: A significant difference was observed (corrected $p=0.015$).
• A latency of “0 ms” vs. “200 ms”: A significant difference was observed (corrected $p=0.0026$).
• A latency of “0 ms” vs. “300 ms”: A significant difference was observed (corrected $p<0.0010$).
• A latency of “0 ms” vs. “600 ms”: A significant difference was observed (corrected $p<0.0010$).

Pairwise comparisons further revealed significant differences between the “0 ms” and all other conditions, with an increasing latency resulting in a significantly higher maximum normal force. These findings suggest that the latency had a substantial impact on the applied normal force during the operation with force feedback.

3.9.3. Operability

Figure 13 shows the subjective operability scores under the five latency conditions. The normality of the data distribution for each latency condition was assessed using the Shapiro–Wilk test. The results indicated that the “0 ms” ($p<0.00010$), “100 ms” ($p=0.0050$), and “600 ms” ($p=0.037$) conditions did not satisfy the assumption of normality. As a result, non-parametric methods were employed for further analyses to account for the non-normal distribution in some conditions.

To evaluate the overall differences in subjective operability among the five latency conditions, a Friedman test was conducted. The results revealed a significant overall effect (${\chi }^2\left(4\right)=33$, $p<0.00010$), indicating that at least one latency condition differed significantly from the others.

To investigate specific differences between latency conditions, pairwise comparisons were conducted using the Mann–Whitney U test with Bonferroni correction. The key results are summarized below:

• A latency of “0 ms” vs. “100 ms”: No significant difference was observed (corrected $p=0.46$).
• A latency of “0 ms” vs. “200 ms”: A significant difference was observed (corrected $p=0.0069$).
• A latency of “0 ms” vs. “300 ms”: A significant difference was observed (corrected $p=0.0013$).
• A latency of “0 ms” vs. “600 ms”: A significant difference was observed (corrected $p<0.0010$).

Pairwise comparisons revealed significant differences between the “0 ms” condition and the “200 ms”, “300 ms”, and “600 ms” conditions, while no significant difference was observed between the “0 ms” and “100 ms” conditions. These findings suggest that higher latency significantly reduced subjective operability compared to the zero latency condition.

3.9.4. Physical Demand

Figure 14 shows the subjective physical demand scores under the five latency conditions. The normality of the data distribution for each latency condition was assessed using the Shapiro–Wilk test. The results indicated that the “0 ms” ($p=0.00090$), “100 ms” ($p=0.025$), and “200 ms” ($p=0.037$) conditions did not satisfy the assumption of normality. As a result, non-parametric methods were employed for further analyses to account for the non-normal distribution in some conditions.

To evaluate the overall differences in the subjective physical demand among the five latency conditions, a Friedman test was conducted. The results revealed a significant overall effect (${\chi }^2\left(4\right)=12$, $p=0.021$), indicating that at least one latency condition differed significantly from the others.

To investigate specific differences between latency conditions, pairwise comparisons were conducted using the Mann–Whitney U test with Bonferroni correction. The key results are summarized below:

• A latency of “0 ms” vs. “100 ms”: No significant difference was observed (corrected $p=1.0$).
• A latency of “0 ms” vs. “200 ms”: No significant difference was observed (corrected $p=0.65$).
• A latency of “0 ms” vs. “300 ms”: No significant difference was observed (corrected $p=0.15$).
• A latency of “0 ms” vs. “600 ms”: No significant difference was observed (corrected $p=0.20$).

Pairwise comparisons revealed no significant differences between any specific latency pairs. These findings suggest that while the latency may have influenced the subjective physical demand to some extent, the effect was not strong enough to produce significant differences between specific conditions.

3.9.5. Mental Demand

Figure 15 shows the subjective mental demand scores under the five latency conditions. The normality of the data distribution for each latency condition was assessed using the Shapiro–Wilk test. The results indicated that the “0 ms” ($p=0.00040$), “100 ms” ($p=0.017$), “300 ms” ($p=0.025$), and “600 ms” ($p=0.00030$) conditions did not satisfy the assumption of normality. As a result, non-parametric methods were employed for further analyses to account for the non-normal distribution in most conditions.

To evaluate the overall differences in the subjective mental demand among the five latency conditions, a Friedman test was conducted. The results revealed a significant overall effect (${\chi }^2\left(4\right)=27$, $p<0.00010$), indicating that at least one latency condition differed significantly from the others.

To investigate specific differences between latency conditions, pairwise comparisons were conducted using the Mann–Whitney U test with Bonferroni correction. The key results are summarized below:

• A latency of “0 ms” vs. “100 ms”: No significant difference was observed (corrected $p=0.90$).
• A latency of “0 ms” vs. “200 ms”: No significant difference was observed (corrected $p=0.064$).
• A latency of “0 ms” vs. “300 ms”: A significant difference was observed (corrected $p=0.0081$).
• A latency of “0 ms” vs. “600 ms”: A significant difference was observed (corrected $p=0.0016$).

The results revealed significant differences between the “0 ms” condition and the “300 ms” and “600 ms” conditions. These findings suggest that higher latency, particularly at 300 ms and 600 ms, significantly increased the subjective mental demand compared to the zero latency condition.

4. Discussion

4.1. Discussion of Experiment 1

The first major result of Experiment 1 was that the force feedback resulted in a statistically significant reduction in the maximum normal force, irrespective of the visual feedback resolution. Even when high-resolution visual information was presented, it was difficult to avoid contact altogether, leading to an increase in normal forces. It was shown that force feedback have the ability to compensate for such aspects in which visual information was not beneficial. In the presence of force feedback, the maximum contact force was reduced to the same extent regardless of the resolution of the visual information. In other words, it can be inferred that the reduction in the maximum contact force with force feedback is so large that differences in the visual resolution have little effect.

Another unique result in Experiment 1 was that no difference in the task completion time was found. This may be attributed to differences in task execution strategies that offset the task time between the two conditions. The overall task can be divided into two phases: the alignment phase and the insertion phase. In the condition with haptic feedback, it is considered that the alignment phase is faster, while the insertion phase is slower. During the alignment phase, the presence of haptic feedback allows the operator to quickly grasp the positional relationship between the peg and the hole through contact. Conversely, during the insertion phase, while haptic feedback enables the operator to maintain awareness of the positional relationship even when the peg is partially inserted into the hole, it also causes the operator to unconsciously attempt to correct the peg’s position if the sides of the peg and the hole come into slight contact. This unconscious correction likely increases the time required for insertion.

Regarding the subjective evaluation in Experiment 1, the effect of improved operability and a reduced mental demand was also confirmed, although the physical demand did not vary according to the feedback conditions. In particular, with regard to the mental burden, the results showed that the burden reduction due to the increased visual resolution was superimposed on the burden reduction due to the additional force feedback. In other words, the results suggest that even under conditions of high visual resolution, the addition of force feedback can be beneficial.

4.2. Discussion of Experiment 2

In Experiment 2, the evaluation metric most affected by the latency was the maximum normal force. Even at 100 ms, the smallest latency in the present study, a statistically significant difference was shown. It was also shown that the maximum normal force increases with increasing latency.

For the other evaluation metrics, the mental demand and operability were not as sensitive as the maximum normal force to a relatively small latency, as they were not statistically different from in conditions without latency, and work completion times showed statistically significant differences only at the largest latency of 600 ms. Thus, the conditions under which the effects of latency appear differ depending on the indicator being assessed. This means that the value of acceptable latency varies depending on what is to be maintained in the task.

4.3. Limitations

One limitation of this study is that the experiments were conducted with a limited age range of participants. Age-related factors, such as sensory sensitivity and motor response time, may influence the perception and effectiveness of feedback information. Although our results provide meaningful insights within the tested age group, it is important to consider that different age groups might exhibit variations in responses.

Another limitation of this study is that only two discrete visual resolution conditions (high and low) were examined. While this approach allowed us to assess the fundamental effects of resolution differences, it did not capture the full range of possible visual feedback variations in practical applications. A more detailed gradient of resolution levels could provide further insights into how different resolutions influence teleoperation performance. Future research could address this limitation by systematically exploring a broader spectrum of visual resolutions to achieve a more comprehensive understanding.

Another limitation of this study is that it did not account for certain real-world disturbances, such as network fluctuations, changes in environmental lighting, and equipment aging. Setting these factors as experimental conditions presents a challenging task, as their effects can be unpredictable and difficult to control. However, considering these disturbances in future research will be essential for improving the robustness of teleoperation systems under realistic conditions.

Although this study did not specifically examine the impact of the training duration on task performance in teleoperation, it is an important factor to consider. Different levels of prior experience and training may influence the efficiency and accuracy of remote operation.

5. Conclusions

This study investigated the effects of haptic feedback on task performance in teleoperated peg insertion tasks under varying visual resolution and communication latency conditions. Two key experiments were conducted to evaluate the interactive effects of haptic feedback with the visual resolution and communication latencies, using both quantitative metrics such as the task completion time and maximum contact force and subjective measures including operability, the physical demand, and the mental demand.

The results of Experiment 1 revealed that haptic feedback significantly reduced the contact force and reduced the mental demand. In particular, the subjective operability of the system was improved by the force feedback in conditions with low visual feedback resolution. These findings suggest that haptic feedback compensates for the lack of high-resolution visual feedback by enabling the operator to better perceive the positional relationship between the peg and the hole.

Experiment 2 investigated the negative effects of communication latency on task performance using objective and subjective measures. The maximum normal force consistently increased with increasing latency. Increased mental fatigue and reduced operability with increasing latency were also observed, although some small latencies were shown to be potentially acceptable.

Overall, the findings of this study underscore the importance of incorporating haptic feedback into teleoperation systems, particularly for tasks requiring precision and under conditions of low visual resolution or high communication latency. These insights provide valuable guidance for the design of more effective and user-friendly teleoperation systems in industrial applications.

Future work will aim to extend the evaluation to more complex tasks and diverse operator populations to further validate the generalizability of these findings.