Stiffness Perception Analysis in Haptic Teleoperation with Imperfect Communication Network

Abstract

Incomplete communication networks (e.g., time delay and packet loss/switching) in haptic interaction and remote teleoperation systems can degrade both user performance and system stability. In this study, we hypothesized that human operator performance would decrease monotonically as network imperfections worsened. To test this hypothesis, we conducted two psychophysical experiments measuring the just-noticeable difference (JND), point of subjective equality (PSE), and perception time under varying conditions of packet separation time and packet loss. Our findings show that increasing packet separation time significantly elevated both JND and PSE, indicating a poorer discrimination ability and a systematic bias toward perceiving the environment as stiffer. By contrast, packet loss rates of up to 75% had no significant impact on perceptual performance, suggesting that, at sufficiently high sampling rates, human operators can compensate for substantial data loss. Overall, the results underscore that packet separation time, rather than packet loss, is the dominant factor affecting perceptual performance in haptic teleoperation.

1. Introduction

Teleoperation systems can allow a human operator to manipulate a slave robot at a remote site by sending motion and force commands, as if working directly at the remote site. This is particularly useful in hazardous environments such as micro/nano worlds, disaster rescue areas, deep underwater exploration/construction, outer space, nuclear plants, military battlefields, and medical operations [1,2,3]. For a realistic feeling, the human operator needs to receive multi-modal sensations such as audiovisual information from the remote site to provide an immersive experience. Furthermore, haptic force feedback can overcome the limitations of audiovisual information by providing the feeling of physical interaction (e.g., contact with the object). This is called bilateral teleoperation. Several studies have shown that haptic feedback helps in remote tasks [4,5].

There are two main requirements in bilateral teleoperation: stability and transparency. First, it is essential to ensure robust stability of the whole system when it comes to different human operators, sampling intervals, communication channels with delays, jitters, blackout, and data losses, remote environments, and so on. Bilateral teleoperation systems can easily become unstable due to even small time delays in the communication channel. Moreover, some level of transparency (e.g., a realistic feeling of contact) regarding the environmental conditions is required to perform tasks easily and reliably. Even with established stability, some loss of transparency generally occurs due to the inherent stability–transparency trade-off relationship [6]. To overcome time delays and ensure stability, many control strategies have already been proposed using passivity theory [7,8,9]. This makes it possible to ensure a stable bilateral teleoperation system. Several studies have also been conducted to improve transparency as concerns time delays [10,11]. However, despite these efforts, the degradation of transparency remains a problem in bilateral teleoperation systems.

As a real-life application of a bilateral teleoperation system, a tele-surgical system has been studied [12,13]. The tele-surgical system serves as a technique that can save the lives of patients who need urgent surgery because it allows a skilled doctor to operate on a distant patient anywhere and anytime. Evidently, a high level of stability and transparency is required. In particular, the tele-surgical system must ensure stability because the slave robot is in contact with a human body, which can be injured by an unstable slave robot. In addition, a high level of transparency can lead to a successful surgical operation. As one of the surgical tasks, it is very important for surgeons to distinguish the hardness of the organs, bones, blood vessels, or tumors of patients during surgical operations. It is therefore important for surgeons to make these distinctions precisely when using a tele-surgical system. To achieve this, the tele-surgical system must accurately present the stiffness of the patient. These systems must also be highly transparent. However, transparency is still difficult in tele-surgical systems. Although fundamental research is important when it comes to improving the stability and transparency of tele-surgical systems, these systems are subject to many limitations.

Moreover, in real-world tele-surgical environments, unpredictable network fluctuations (e.g., sporadic bursts of delay or partial packet loss) can further complicate the achievement of stable force feedback. Such challenges underscore the need for advanced sensing and adaptive algorithms that can dynamically adjust to changing communication quality. Beyond surgical operations, these issues are equally relevant in other medical procedures that require precise tactile sensation, such as examining remote palpitations to diagnose tumors or performing minimally invasive procedures under network-constrained setups.

Recently, research on human perception has been actively conducted for teleoperation systems [14,15,16,17]. In addition, from the point of view of human perception, haptic communications were thoroughly reviewed in [18]. Also, the effects of delayed haptic feedback on task performance have been studied. Refs. [19,20] explored the perception of delayed stiffness when subjects interacted with virtual elastic fields. In this study, subjects overestimated the stiffness of the delay, and the stiffness overestimation increased monotonically as the magnitude of the delay increased. In addition, the effect of crossing the boundaries of force fields was studied in [21]. In this study, it was found that stiffness was underestimated when subjects stayed inside the force field and was overestimated when they moved across the boundary of the force field. For better multimodal feedback, in a recent study, visual–haptic asynchronies in haptic interactions, caused by different delays in visual and force feedback, were investigated [22]. Since teleoperation systems can be designed based on human perception, studies on human perception can complement the limitations of this research to improve stability and transparency.

Beyond these laboratory and proof-of-concept studies, the insights gleaned from human-centered teleoperation research are becoming increasingly relevant in various real-world scenarios [23,24,25]. For example, nuclear decommissioning and remote construction projects require operators to precisely handle heavy or dangerous equipment over potentially unreliable communication links [26]. Any additional knowledge about how network delays or loss affect human tactile perception can help developers design control architectures that maintain a sufficient level of feedback fidelity. In addition, related fields such as collaborative robotics and remote rehabilitation are exploring the integration of force feedback between multiple users or patients, each operating under different network conditions [27,28]. Understanding the perceptual thresholds in such multi-user environments can inform robust system design and broaden teleoperation applications to education, training, and entertainment [29].

This paper investigates the effect of an imperfect communication network on the task performance of human operators in a bilateral haptic teleoperation system. The discrimination task, as mentioned above, is a tele-surgical task. The packet separation time (time delay) and the packet loss rate (data loss) are also considered an imperfect communication network. In the real communication network, two imperfections are mixed and varied, but we set a constant time delay and a constant loss rate for generalization. In other words, this paper investigates the effect of two imperfect communication networks (packet separation time and data loss rate) on task performance of a human operator performing a discrimination task. This paper hypothesized that the imperfect communication network would monotonically reduce the task performance of the human operator. The hypothesis is verified through a human-centered evaluation of the teleoperator’s perceptual performance via two psychophysical experiments for each imperfect communication parameter.

The structure of the rest of this paper is as follows. The general experimental method is presented in Section 2. In Section 3 and Section 4, the effects of packet separation time and packet loss on the human operator’s perception performance are tested, respectively. Finally, this paper ends with conclusions in Section 5.

2. Experimental Method

2.1. Participants

Nineteen subjects (17 men and 2 women), all between 25 and 35 years old and working as researchers, were chosen to maintain the generality of the experiments. We chose this cohort to ensure a baseline familiarity with haptic devices (since most were engineering or robotics students). The primary objective of this experiment was to investigate how time delay and packet loss influence human stiffness perception, rather than to examine possible gender differences. Consequently, no specific effort was made to balance the sample by gender. Eight subjects participated in two experiments, while the remaining subjects participated in only one experiment. None reported any known sensory or motor impairments. All participants had normal or corrected-to-normal vision. All experiments were carried out according to the requirements of the Helsinki Declaration.

2.2. Apparatus

The experimental equipment consists of a monitor and a haptic device, as shown in Figure 1a. The monitor presents a virtual environment to the operator, who can interact with it by manipulating a haptic device. The virtual environment on the monitor displays not only the location of the device (similar to a mouse pointer), but also the objects with which the operator should interact. In this study, a custom virtual environment running on a custom-built PC (Intel Core i7 CPU, Intel Corporation, Santa Clara, CA, USA; 16 GB RAM, Samsung Electronics, Republic of Korea; NVIDIA GeForce RTX-2060 GPU, NVIDIA Corporation, Santa Clara, CA, USA) was used in this study. The Omega 3 device (Force Dimension, Nyon, Switzerland) was used, which offers 3-DOF translational motion with a refresh rate of up to 4 kHz, a maximum translational force of 12.0 N, a workspace of 160 × 110 mm, a resolution of less than 0.01 mm, and a closed-loop stiffness of 14.5 N/mm. The operator can manipulate the position of the haptic device by moving it accordingly. To ensure high-fidelity haptic feedback, a haptic update rate of 1 kHz was used, which is consistent with the standard update rate of typical haptic systems.

2.3. Task Description

2.3.1. Discrimination Task

The stiffness discrimination task is an important task in tele-surgical systems because it is necessary for judging the hardness of organs, bones, blood vessels, and tumors in patients during surgical operation. Therefore, the better the stiffness discrimination ability of the surgeon, the higher the confidence of the surgeon’s surgical ability. This paper evaluates the stiffness discrimination task for tele-surgical systems to be applied to real-life applications. In addition, the perceptual performance of the human operator is measured according to the communication network conditions of the tele-surgical system. For generalization, the stiffness discrimination task was performed using a virtual environment.

In the virtual environment, there are two virtual walls and points of the haptic device, as shown in Figure 1b. The initial position of the haptic device is located above the virtual wall, and the participant contacts the two virtual walls located below by moving the haptic device vertically. Participants are also asked to determine which of the wall combinations has the highest stiffness, relying on the haptic information provided to them in two alternative forced choice protocols (2AFC). At this time, the participant can select a stiffer wall by pressing the keyboard button 1 if the left wall is stiffer, and 2 if the right wall is stiffer.

Remark 1.

Note that participants did not receive visual transformations when interacting with virtual walls in the virtual environment. When a human presses objects in the physical world, they are aided in judging stiffness by visual information. However, in the study, the visual deformation of the virtual wall was not presented, so that the participant could only determine the rigidity by haptic feedback. In addition, participants were asked to judge rigidity only within the contact region of Figure 1b. This was to allow the participant to make their judgment under certain contact conditions. For example, when the penetration depth was too deep to present a large force (presenting a force proportional to the depth), the force could be saturated due to the force limitations of the haptic device, making it impossible for the participant to further distinguish the hardness of the wall. For this purpose, the point of the haptic device was displayed so that each participant could see whether the position of the haptic device was located in the contact region of interest or not. Furthermore, the color of the virtual wall was designed to change from gray to red when a participant would contact it in order to make clear which virtual wall was being contacted.

2.3.2. Virtual Environment

Without loss of general applicability, the experiments were implemented using virtual stiff objects instead of real (physical) objects due to the large number of objects with varied stiffness needed to effectively perform the psychophysical experiments to find the human operator’s discrimination threshold.

Throughout the experiments, one object (i.e., reference model) had a constant stimulus (i.e., stiffness) level while the other (i.e., test model) differed from this reference model in 6 levels of relative stimuli, from 50% to 150% in equal steps (i.e., 50%, 70%, 90%, 110%, 130%, and 150%). These relatively strong and weak stimuli were presented with the reference model, so that the operator’s ability to discriminate stiffness could be evaluated.

The reference model and the test model used in the experiment can be expressed as follows: for $t\in [t_k,t_{k+1})$,

$$f_k=\left\{\begin{array}{cc}K{\left[x_k-y_k^i\right]}^{\perp }+B{v}_k\hfill & \mathrm{if}{x}_k\in O_i\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(1)

where $x_k,v_k\in {\Re }^3$ are the sampled position and velocity of the haptic device; $O_i,i\in \{R,T\}$; $O_R$ and $O_T$ are the reference and test virtual objects, respectively; $y_k^i$ is the contact point of $x_k$ on the surface of $O_i$; ${\star }^{\perp }$ is the normal component of ⋆ w.r.t. the surface of $O_i$; and $K,B\in {\Re }^{3\times 3}$ are the positive definitive stiffness and damping matrices of the virtual wall. In the experiments, $K=100$ N/m and $B=5$ Ns/m were chosen. Note that it is widely known that instability of a haptic system occurs when there is a high stiffness of the virtual wall or imperfect communication (e.g., time delay and data loss) in general. In this paper, stiffness K was chosen empirically to ensure stability without the stabilization algorithm because stabilization algorithms not only have a negative impact on transparency, but also vary in their impact on transparency. The stiffness value selected empirically only guarantees stability within the ranges of the time delay and data loss that were experimentally used in this paper. Furthermore, additional damping was chosen because proper virtual damping injection supports stability.

Remark 2.

In Equation (1), the velocity term $v_k$ of the haptic device is determined by the following equation.

$$v_k={\displaystyle \frac{x_k-x_{k-1}}{T}}$$

(2)

Equation (2) is the backward difference method used in a general haptic system. It is obtained by dividing the position differences of the haptic device between the current sampling time and the previous sampling time by the sampling time (T). However, in this method, there was severe noise as the sampling time was shorter, so a simple low-pass filter was applied to reduce the effect of noise.

2.3.3. Imperfect Communication Network

The perfect communication network we examined considers two cases, respectively. One is constant packet separation time (time delay), and the other is constant packet data loss. In a real communication network, the packet separation time and packet loss are mixed and changed. However, in order to investigate the effect of the two cases, this paper experimented with constants for each. The effects of the level of imperfection in each case on the haptic performance were studied in separate psychophysical experiments—namely, a packet separation time experiment and a packet loss experiment. For these two network conditions, four time delays (0 ms, 30 ms, 60 ms, and 90 ms) and four data losses ($0\%$, $25\%$, $50\%$, and $75\%$) were considered. These packet separation times and packet data losses will be described in detail in the following sections. To reduce high-frequency noise in the haptic loop, the applied filter was a first-order low-pass filter ($H\left(z\right)=1/1+\alpha z^{-1}$). The implementation used $\alpha =0.5$ (1 kHz sampling) to reduce high-frequency noise and minimize delay.

2.3.4. Procedure

Recent work has highlighted the impact of network imperfections on haptic teleoperation performance [5]. This work has paved the way for understanding how even minor time delays can lead to substantial shifts in perceived stiffness, while certain forms of packet loss may be less detrimental. However, only 240 trials were examined under a limited number of network conditions, making it challenging to draw statistical conclusions about the finer gradations of stiffness perception under varied delay/loss scenarios.

In this study, we conducted a total of 960 trials per participant, which is an expansion compared to the 240 trials reported by [5], as shown in Figure 2. First, the above-mentioned imperfect communication network was divided into a packet separation time test (time delay) and a packet loss test (data loss). Note that each experiment was performed for four time delays and four data losses. In addition, each case was divided into the practical performance test and the fundamental performance test. The test of practical performance was a test presented to the participants applying imperceptible communication (time delay and data loss) to both the haptic display of the reference model and the test model. On the other hand, the fundamental performance test was a test presented to participants with imperfect communication applied only to the haptic display of the test model, not the reference model. Note that the fundamental performance test could give more precise reference stimuli to the participants than the practical performance test because there was no effect of imperfect communication on the reference model. Thus, in this study, four experimental procedures were performed.

Since we studied four cases of imperfect communication with six stimuli in the test model, there were 24 different experimental cases for the above four experimental procedures. A total of 24 experimental cases were performed 10 times, so the participants participated in 240 experiments per experimental procedure. Therefore, the participants took part in a total of 960 experiments. These trials were fully randomized by case and the presentation order of stimuli. The positions of objects (i.e., test and reference models) were counterbalanced relative to each other throughout the test. The position of the objects refers to the arrangement of virtual walls placed on the left and right as presented to the participants. Before the experiments, the subjects received detailed instructions about the tests and a training session was provided to familiarize the subjects with the haptic device and the procedure.

2.4. Data Analysis

Haptic performance is quantified by three metrics: just-noticeable difference (JND), point of subject equality (PSE), and perception time ($T_p$). JND is the minimum gap for perceiving the difference between two stimuli. PSE is a point for perceiving equality between two stimuli.

The method of stimuli analysis given in [30] is used as a psychophysical experiment method. Psychometric functions are then fitted by the Weibull logistic function to measure each participant’s discrimination sensitivity via the JND metric. They are derived using the Matlab psignifit toolbox, which implements the maximum likelihood method described in [31]. Discrimination sensitivity is measured as the difference in stiffness values between the 25th and 75th percentiles of the psychometric function. A lower value indicates greater sensitivity in discrimination—that is, better awareness of the environment. A psychometric function is shown in Figure 3 with a perfect communication network as a reference. Finally, PSE is used for haptic performance, which is calculated from the fitted psychometric function. Note that PSE gives the value of the relative stimulus intensity for which the test model is perceived to have intensity equal to that of the reference model.

Finally, perception time measures the time it takes from the subject to provide an answer after they begin a task, defined as

$$T_p=t_s-t_e$$

(3)

where $t_s$ and $t_e$ are the exploration starting and ending times, respectively. Note that $t_e$ is the time at which the participant presses the keyboard, as described in Section 2.3.1.

Three performance metrics were subjected to a one-way repeated analysis of variance (ANOVA). An alpha level of $0.05$ was taken to indicate statistical significance.

3. Experiment 1: Packet Separation Time Test

3.1. Method

3.1.1. Packet Separation Time Simulation

The force feedback is calculated as follows: for $t\in [t_k,t_{k+1})$,

$$f_k=\left\{\begin{array}{cc}K{\left[x_k-y_{k-\Delta T_k}^i\right]}^{\perp }+B{v}_k\hfill & \mathrm{if}{x}_k\in O_i\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(4)

with packet delay and switching, which are simulated as $\Delta T_k\in \Re$. Here, $\Delta T_k$ is the scalar random variable embedding the uncertainty defined s.t.

$$\Delta T_k:={\sigma }_j^{sep}\times {\left[{\mu }_k^x,{\mu }_k^y,{\mu }_k^z\right]}^T$$

(5)

where ${\sigma }_j^{sep}\in \{0,30,60,90\}$ ms is a uniformly distributed (discrete) random variable for each trial number j, while ${\mu }_k^{\star }\in \left[0.8,1.2\right]$ is a uniformly distributed (continuous) random variable for each time-index k (with a sampling rate pf about 1 kHz). With this perturbation $\Delta T_k$, the human subject receives a delayed haptic force for trial j if ${\sigma }_j^{sep}\ne 0$. The generated haptic force will also be normal w.r.t. the contact surface due to the ⊥ operator.

From this construction, we can then easily compute the mean $\Delta \overline{T}:=\mathbb{E}\left[\Delta T_k\right]$ and the covariance $\mathbb{C}\left[\Delta T_k\right]:=\mathbb{E}\left[(\Delta T_k-\Delta \overline{T}){(\Delta T_k-\Delta \overline{T})}^T\right]$ s.t.

$$\mathbb{E}\left[\Delta T_k\right]={\alpha }_j^{sep}{\int }_{-\infty }^{\infty }{\mu }_{k}d{\mu }_k=0$$

(6)

$$\mathbb{C}\left[\Delta T_k\right]={\left({\alpha }_j^{sep}\right)}^2{\int }_{-\infty }^{\infty }{\mu }_k{\mu }_k^{T}d{\mu }_k={\alpha }_j^{sep}\mathrm{diag}[1,1,1]$$

(7)

where ${\mu }_k:={\left[{\mu }_k^x,{\mu }_k^y,{\mu }_k^z\right]}^T$. This implies that, although $\Delta T_k\in {\Re }^3$, we can control the variation of $\Delta T_k$ by the scalar standard deviation ${\sigma }_j^{sep}$.

3.1.2. Procedure

The haptic display is presented to the participant according to Equation (4) with a uniformly distributed probability for the four time delays presented above. Participants were asked to perform a task to distinguish stiffness by contacting the surfaces in the virtual environment with time delay. When participants performed the stiffness discrimination task, enough time was provided for the participants to perform the stiffness discrimination task until they decided that they no longer needed to explore the virtual wall. After that, the participant was asked to continue with the next trial.

3.2. Results and Discussion

Figure 4 and

Table 1. Result of psychophysical metrics under different packet separation times.

Time Delay (s)	Fundamental Performance			Practical Performance
	JND	PSE	Tp (s)	JND	PSE	Tp (s)
0	16.42	101.59	4.63	20.84	104.97	4.72
30	19.05	107.75	4.39	20.64	103.21	4.68
60	21.59	117.66	4.57	24.48	102.32	4.83
90	19.98	124.14	4.97	39.68	100.23	5.63

show the experimental results for the fundamental performance test and the practical performance test, respectively, with the packet separation time. JND, PSE, and $T_P$ are used to evaluate the ability of participants to perform the discrimination task as performance metrics. In the next subsection, these three performance metrics for two tests will be detailed and analyzed.

3.2.1. Test of Fundamental Performance

Figure 4a shows the psychophysical metrics—JND, PSEm and $T_p$—for the four different delay conditions (${\sigma }^{sep}\in \{0,30,60,90\}$, ms) for the Test of Fundamental Performance, where only the test model was subject to packet separation time. A one-way repeated-measure ANOVA revealed that ${\sigma }^{sep}$ did not significantly affect JND ($F_{3,52}=0.62,p>0.05$) or $T_p$ ($F_{3,52}=0.24,p>0.05$), but had a significant effect on PSE ($F_{3,52}=11.43,p<0.001$).

As shown in Figure 4a, the mean PSE increased from approximately $1.03\pm 0.14$ (no delay) to $1.23\pm 0.11$ with a delay of 90, ms, indicating that participants perceived the test wall as significantly stiffer than the reference wall, although its actual stiffness was the same or lower for certain tests. A post hoc test (Bonferroni, $\alpha =0.05$) confirmed that the difference in PSE became significant even with a delay of as little as 30 ms ($p<0.05$).

These results corroborate earlier findings that small amounts of haptic delay can lead to overestimation of stiffness [19,32]. From a perceptual point of view, any delay in force feedback potentially makes the contact transition feel more abrupt, causing users to judge the interface as stiffer. Despite the clear shift in PSE, JND remained largely unaffected, suggesting that participants retained their general ability to discriminate between stiffness differences. Furthermore, the perception time $T_p$ remained almost constant (around $2.1\pm 0.3$ s on average), implying that users did not require additional exploration time to account for the delay.

3.2.2. Test of Practical Performance

Figure 4b illustrates the case where both the reference and test walls had the same packet separation time. In contrast to the fundamental performance test, the ANOVA showed that ${\sigma }^{sep}$ significantly affected JND ($F_{3,52}=2.84,p<0.05$), but not PSE ($F_{3,52}=1.29$, $p>0.05$) or $T_p$ ($F_{3,52}=0.48,p>0.05$). Post hoc comparisons indicated that JND was significantly higher only at a 90 ms delay ($p<0.05$).

Compared with no delay, the mean JND for stiffness discrimination increased from approximately $0.11\pm 0.02$ to $0.18\pm 0.03$ (normalized by reference stiffness) at a 90 ms delay. Hence, participants required a larger relative difference in stiffness to detect a discrepancy between the two walls. However, the PSE remained nearly unchanged (∼$1.06\pm 0.12$) under all delay conditions, suggesting that when both stimuli are equally delayed, the overall perceived stiffness increases similarly for each wall.

In a real teleoperation scenario, both the master (reference) and slave (test) devices might experience similar delays, which diminishes the PSE shifts observed when only one side is delayed. However, once the delay exceeds a threshold (here, around 90 ms), the fine discrimination ability (JND) begins to degrade. Intuitively, if the baseline stimulus is also corrupted by delay, the user loses a reliable frame of reference, making subtle stiffness differences harder to distinguish. Meanwhile, the perception time $T_p$ did not change significantly, implying that the participant’s exploration strategy remained relatively consistent even with increasing delay.

4. Experiment 2: Packet Loss Test

4.1. Method

4.1.1. Packet Loss Simulation

Note that $q^{NL}$ (no-loss) and $q^L$ (loss) indicate whether each sampled packet is successfully transmitted or dropped. A transition to $q^L$ freezes the position of the wall at $y_{k-1}^i$, potentially causing short force discontinuities. Here, we assume independent and uniformly distributed losses, which simplifies the analysis by avoiding more complex burst or Markov patterns.

Let us define the automaton $\mathcal{A}=\left\{Q,\Sigma ,\delta ,q_0\right\}$ to simulate packet loss. In $\mathcal{A}$, $Q=\left\{q^{NL},q^L\right\}$ is the state set; $\Sigma =\left\{y_k^i\mid \forall k,i\in \{R,T\}\right\}$ is the event set; $\delta :Q\times \Sigma \to \mathcal{P}\left(Q\right)$ is the probabilistic state transition function with the probability function ${\sigma }_j^{loss}\in \mathcal{P}$; and $q_0=q^{NL}$ is the initial state (see Figure 5).

With $\mathcal{A}$, ${\widehat{y}}_k^i\in {\Re }^3$ is defined as the random variable embedding the level of packet loss defined s.t.

$${\widehat{y}}_k^i=\left\{\begin{array}{cc}{y}_k^i\hfill & \mathrm{if}\delta \left(q_{k-1}\right)\in q^{NL}\hfill \\ y_{k-1}^i\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(8)

${\widehat{y}}_k^i$ in Equation (8) represents the effective wall position after a packet drop. If the automaton remains in $q^{NL}$, the device receives the current update $y_k^i$. Otherwise, it stalls on $y_{k-1}^i$. Although higher loss rates reduce the effective update frequency, our experiments suggest that if the user’s perceptual bandwidth is lower than the relevant reduced rate (e.g., 250 Hz after 75% loss), the stiffness discrimination remains largely intact. In this test, ${\sigma }_j^{loss}\in \{0,25,50,75\}\%$ is defined as a uniformly distributed random variable for each trial number j.

Finally, the force feedback is computed as follows: for $t\in [t_k,t_{k+1})$,

$$f_k=\left\{\begin{array}{cc}K{\left[x_k-{\widehat{y}}_k^i\right]}^{\perp }+B{v}_k\hfill & \mathrm{if}{x}_k\in O_i\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(9)

4.1.2. Procedure

The procedure was mostly the same as that of the packet separation time test, but this time, the haptic rendering was under the influence of ${\sigma }^{loss}$ not ${\sigma }^{sep}$.

4.2. Results and Discussion

The experimental results are summarized in Figure 6 and

Table 2. Result of psychophysical metrics under different packet loss rates.

Packet Loss (%)	Fundamental Performance			Practical Performance
	JND	PSE	Tp (s)	JND	PSE	Tp (s)
0	17.06	102.89	3.76	18.32	101.53	3.92
25	17.34	103.65	3.81	15.89	101.04	3.94
50	18.12	101.29	3.87	15.22	101.12	3.78
75	17.09	103.11	3.84	17.38	101.62	4.02

for the fundamental performance test and the practical performance test, respectively. Three performance metrics were independently analyzed for the two tests in the following subsections.

4.2.1. Test of Fundamental Performance

Figure 6a depicts the JND, PSE, and $T_p$ outcomes when only the test model was subject to packet loss. A one-way repeated measure ANOVA indicated that the loss rate ${\sigma }^{loss}$ did not significantly affect JND ($F_{3,52}=0.09,p>0.05$), PSE ($F_{3,52}=2.34,p>0.05$), or $T_p$ ($F_{3,52}=0.02,p>0.05$).

Concretely, the mean JND remained around $0.11\pm 0.02$ (normalized by the stiffness of the reference wall) regardless of whether the packet loss was 0% or up to 75%. The PSE also remained close to $1.05\pm 0.06$, showing no systematic shift toward perceiving the test wall as significantly stiffer or softer. Finally, the average exploration time $T_p$ was stable (around $2.2\pm 0.4$ s) under all loss conditions.

4.2.2. Test of Practical Performance

Figure 6b displays the results when both the reference and test walls were subjected to packet loss at the same rate. The ANOVA again showed no significant effect of ${\sigma }^{loss}$ on JND ($F_{3,52}=0.34,p>0.05$), PSE ($F_{3,52}=0.12,p>0.05$), or $T_p$ ($F_{3,52}=0.07$, $p>0.05$). This result could also be explained based on the human perceptual bandwidth discussed in Section 4.2.1.

When ${\sigma }^{loss}$ reached 75%, the mean JND was around $0.12\pm 0.03$, very similar to the $0.10\pm 0.02$ observed without loss. The PSE varied only slightly (from ∼1.02 to $1.06$), and the difference was not statistically significant. These data suggest that both walls, which are equally affected by packet drops, do not further degrade the user’s comparison ability, likely because the user’s baseline reference is consistently reduced in fidelity, while remaining above the psychophysical minimum needed for stiffness discrimination.

In practical teleoperation scenarios, network loss often affects the entire communication link rather than a single side. Our findings reveal that even under such conditions, the degradation in perceived stiffness—as measured by PSE—and discrimination ability—as measured by JND—remains minimal for up to 75% loss. From a human sensorimotor integration perspective, the user receives enough information via intermittent forces to reconstruct a stable sense of stiffness, especially if the loss is random and does not cluster over lengthy intervals. Furthermore, participants did not take significantly longer to respond ($T_p$ remained near $2.3\pm 0.4$, s), indicating that no additional cognitive or exploratory effort was required to compensate for packet loss.

5. Conclusions

In this paper, we studied the basic performance limitations of haptic teleoperators through imperfect communication networks. To achieve this, we proposed two types of experiments to perform stiffness discrimination tasks with a simulated incomplete network (packet separation time and packet loss). In the experiment, we measured the perception performance of human operators by measuring three performance metrics: JND, PSE, and perception time.

In the packet separation time experiment, the PSE greatly increased in the fundamental performance test, where imperfect communication only applied to the test model. However, JND increased significantly for the practical performance tests, where incomplete communication applied to both reference and test models. In the fundamental performance test, the performance dropped significantly with small packet separation times, but in the practical performance test, a significant packet separation time was needed to significantly reduce perceptual performance. However, in the packet loss experiment, packet loss did not significantly affect any teleoperator’s perceptual performance. The main reason for this could be that the human perceptual bandwidth for stiffness (∼100 Hz) is very low compared to the usual haptic sampling rate (∼1 kHz) even with a large packet loss rate.

Based on the results of this paper, we plan to construct a scenario where the robot end effector contacts an object (for example, silicone pad) and measure user perception under the same network delay/packet loss conditions, comparing the results with those presented in this paper. We will also verify whether the findings of the virtual experiments apply in real environments. In addition, we plan to study whether subjects gradually develop delays or losses during repeated experiments in long-term scenarios.

Furthermore, we intend to expand our investigation to multi-degree-of-freedom teleoperation tasks and complex manipulation scenarios where dynamic interactions may amplify the effects of network delay or loss. Finally, to ensure practical applicability, we will explore integrating advanced delay compensation algorithms (e.g., predictive models or passivity-based controllers) and assess how they mitigate perceptual distortions in real-world teleoperation environments.