Robot Learning from Teleoperated Demonstrations: A Pilot Study Towards Automating Mastic Deposition in Construction Sites

Abstract

The construction industry faces significant challenges due to the physically demanding and hazardous nature of tasks such as manual filling of expansion joints with mastic. Automating mastic filling presents additional difficulties due to the variability of mastic density with temperature, which creates a constantly changing environment that requires adaptive control strategies to ensure consistent application quality. This pilot study focuses on testing a new human–robot collaborative approach for automating the mastic application in concrete expansion joints. The system learns the task from demonstrations performed by expert construction operators teleoperating the robot. This study evaluates the usability, efficiency, and adoption of robotic assistance in joint-filling tasks compared to traditional manual methods. The study analyzes execution time and joint quality measurements, psychophysiological signal analysis, and post-task user feedback. This multi-source approach enables a comprehensive assessment of task performance and both objective and subjective evaluations of technology acceptance. The findings underscore the effectiveness of automated systems in improving safety and productivity on construction sites, while also identifying key areas for technological improvement.

1. Introduction

The construction industry has historically struggled with physically demanding and hazardous tasks [1], one of which is the manual filling of expansion joints with mastic. This process, which requires significant manual effort, not only poses substantial health and safety risks to workers but also contributes to a range of occupational health issues. Prolonged exposure to awkward working postures during mastic application can lead to musculoskeletal disorders (MSDs), causing chronic pain and discomfort [2]. Additionally, the proximity of workers to the application area increases the likelihood of accidents, intensifying safety concerns [3].

While the manufacturing industry has widely adopted automation, the construction field, due to its constantly changing environments, faces more significant challenges in automating tasks. Although the construction industry is evolving to incorporate new technologies, most current applications are stationary. Therefore, there is a pressing need to combine robotic programming techniques with real-world construction use cases to fully understand the challenges outside the laboratory [4].

One of the primary challenges in automating the application of mastic in expansion joints is the material’s temperature-dependent density, which significantly affects its application properties [5]. Particularly, the pressure required to ensure a continuous and complete flow of mastic into the joint is highly sensitive to temperature fluctuations. Furthermore, these changes in the rheological properties of the mastic also affect the optimal forward speed of the mastic gun, which must be adjusted accordingly to ensure a complete and uniform joint filling. Therefore, traditional automation approaches, which rely on fixed robot control logic, are inadequate for this task. The same program that functions effectively in summer may fail in colder conditions due to the changes in the mastic’s density.

To address this challenge, we propose a human–robot collaborative system that employs learning from demonstration (LfD) via teleoperation to capture and transfer the expert operator’s ability to adjust key operational parameters, such as the extruder’s contact force, and forward movement speed. This approach enables the robot to replicate subtle control behaviors that are essential for achieving high-quality joint filling under variable real-world conditions. Additionally, teleoperation is a particularly effective task demonstration strategy in the construction industry because it allows expert operators to control robots remotely, ensuring precise and safe execution of tasks such as mastic application in expansion joints. By leveraging teleoperation, the expertise and skill of human operators can be transferred to robotic systems, enabling them to adapt to the dynamic and often unpredictable conditions of construction sites while reducing the health and safety risks associated with manual labor. By allowing fast and reliable robotic learning from a minimal amount of human data, the system can adapt effectively to changing environmental conditions.

While the automated mastic filling solution is designed to significantly reduce the need for manual labor, it is not yet intended to operate as a fully autonomous system in this pilot phase. Certain preparatory steps, such as surface preparation, still require human intervention and expertise. In this pilot study, the mastic filling task has been demonstrated by expert workers using a teleoperated robotic system. To achieve task automation, the learning strategy presented in [6] has been utilized, as it showed promising preliminary results in laboratory environments. In this learning from demonstration scheme, a human operator remotely controls a robotic arm equipped with a mastic gun via a haptic interface, performing a single demonstration of the task. This demonstration is then used to extract control parameters to configure an admittance controller, which allows the robot to adapt to contact forces and environmental variability. This approach enables autonomous and compliant execution of the mastic application, even when encountering minor surface irregularities or changes in joint orientation, thanks to the adaptability of the admittance controller.

The primary objective of this pilot is to perform a user evaluation of the mastic application system and assess the usability of the designed system and the acceptance of the technology by its users. Secondary objectives include evaluating the quality of the joint filling system according to industry standards, comparing execution times in three different actuation modalities (manual, teleoperation, and automatic modes). Additionally, the study aims to identify potential improvements suggested by participants to enhance the system’s usability and assess users’ confidence and operators’ comfort when utilizing the teleoperation system for joint filling.

The objective evaluation assesses the system’s usability by analyzing psychophysiological signals. The use of such signals, including electrodermal activity (EDA), heart rate variability (HRV), electroencephalography (EEG), and eye-tracking, has become increasingly common in Industry 5.0 as a reliable means to assess users’ internal states during interaction [7,8]. These indicators provide insight into key human factors such as trust [9], mental workload [10], and emotional state [11], which are evaluated as part of this pilot study to better understand user experience and support the analysis of technology acceptance.

The evaluation of the subjective experiences has involved different standardized scales, including the NASA Task Load Index (NASA-TLX [12]), the System Usability Scale (SUS [13]), a short and adapted version of the IsoMetrics for Robots [14], as well as open-ended questions regarding the expected benefits and challenges in the short and long term. In addition, sociodemographic data, such as age and gender, have been collected from participants, as well as a shortened version of the Affinity for Technology Scale (ATI [15]). This blend of standardized questionnaires and open-ended questions has allowed for a holistic view of both quantitative and qualitative data.

In this study, the proposed robotic solution has been deployed in a real-world construction environment, where it performs a straight-line movement along predefined joints to apply mastic. The focus of the learning process lies in fine-tuning task-specific parameters that are critical to achieving complete and consistent joint filling under variable field conditions. These include: the force at the extruder’s tip, which ensures consistent contact with the bottom of the joint; the activation of the material flow; and the forward speed of the mastic gun, which must be adjusted in relation to the extrusion rate to avoid gaps or overflows. On the other hand, due to the simplicity of the movements involved in this task, the process does not consider spatial aspects typically addressed in LfD approaches, such as complex navigation, trajectory planning, or obstacle avoidance.

Building on this approach, the system is designed to be intuitive enough to be trained effectively using learning from demonstration (LfD) techniques, even by workers with no prior experience in robotics. The aim is to develop a robust solution that maintains optimal performance while minimizing physical load and reducing cognitive strain on operators. The robotic solution not only assumes the responsibility of applying mastic but also maintains a safe distance between workers and the application zone, thereby significantly reducing the risk of accidents. This automation will create a safer and healthier work environment, enabling workers to focus on more complex and less hazardous tasks. Additionally, the automated solution is expected to optimize material usage, minimizing waste and ensuring a more efficient utilization of mastic. This will not only reduce material costs but also contribute to a more environmentally friendly construction process, aligning with the industry’s growing emphasis on sustainability. Overall, the proposed automated mastic filling solution intends to be a significant step forward in enhancing worker safety, improving productivity, and promoting sustainability in the construction industry.

2. Materials and Methods

This section presents the materials, procedures, and evaluation methods used in the pilot study on automating the mastic filling task in construction using a robotic system. Section 2.1 provides an overview of the manual mastic filling process, its associated challenges, and the rationale for adopting a robotic solution based on learning from demonstration (LfD) and teleoperation. Section 2.2 details the selection and characteristics of the construction workers who participated in the study. Section 2.3 summarizes the hardware elements required, such as the robotic system, haptic interface, extrusion mechanism, and physiological monitoring devices employed. Finally, Section 2.4 describes both objective and subjective measures used to assess task performance, user states, and system usability, including psychophysiological data, time-based metrics, and standardized questionnaires.

2.1. Use-Case Description

In the real world, the task of filling expansion joints with mastic is a critical component of both construction site maintenance and new construction projects. This task involves the precise application of a viscous, temperature-sensitive material into expansion joints to ensure structural integrity and prevent water infiltration. Manual mastic filling is both physically demanding and a highly repetitive task where workers must maintain awkward postures and apply significant physical effort to dispense the mastic accurately inside the expansion joint. This process not only poses substantial health and safety risks but also contributes to a range of occupational health issues. The manual execution of this task could also lead to inconsistencies in the quality of the work, which could compromise the structural integrity of the construction.

Automating the mastic filling task with a robot can address some of the challenges associated with manual application. If correctly programmed, robots can perform the task with greater precision and consistency, taking over the physically demanding and hazardous aspects of the job. To retain the expertise of construction workers and avoid the need for engineers to program the robots, the use of learning from demonstration (LfD) approaches is proposed. This approach allows expert workers to demonstrate the task to the robot, ensuring that the robot can learn and replicate the necessary skills efficiently and accurately. Teleoperation is a key component of this approach, enabling expert operators to control the robot remotely from a safe distance. The learning strategy used for this pilot can be found in [6]. This not only ensures precise and safe execution of the task but also allows for real-time adjustments and fine-tuning, adapting to the dynamic conditions of construction sites.

To illustrate the transition from manual to teleoperated mastic application, Figure 1 presents two scenarios. On the left, a worker is manually filling a joint with mastic using a traditional mastic gun. This process is physically demanding and ergonomically challenging, as it requires repetitive poor postures over extended periods of time, thereby posing significant health and safety risks.

On the right, the same task is performed using the proposed teleoperation system, where the worker remotely controls the robot. This approach not only reduces the physical risks (due to the significantly improved posture) but also allows for the transfer of the worker’s expertise to the robot through LfD techniques.

To replicate the real construction environment, ten concrete slabs were constructed, each incorporating three distinct joint widths (5 mm, 10 mm, and 15 mm), as illustrated in Figure 2. These slots were designed to represent typical expansion joints encountered in the building and construction industry. During the experimental process, the concrete blocks were placed on the floor to replicate the real working site.

Before the pilot evaluation, each participant received detailed information about the study’s objectives, risks and benefits, and signed an informed consent form. They were also briefed on the teleoperation system and its components, including the physiological signal recording systems. A demonstration and training session (up to 30 min) familiarized participants with teleoperating the robot.

The experimentation phase consisted of three tasks, each repeated three times for three different joint sizes. In Task 1, participants filled a joint manually using a traditional mastic gun. In Task 2, they filled a portion of the joint using the teleoperation system; the data collected during this stage were used to extract control parameters for the automation phase. In Task 3, a new joint was filled automatically, while participants observed the robot’s operation and their physiological data were recorded.

After completing the tasks, the concrete slab was removed and labeled with the participant’s identifier for later quality analysis. Participants then completed different questionnaires. None of the participants spent more than 2.5 h on the entire experiment.

2.2. Participants and Recruitment

The recruitment for gathering expert construction workers to evaluate this pilot was conducted among Acciona’s workers. Acciona is a global company based in Spain that specializes in the development and management of sustainable infrastructure, renewable energy, and services. A site manager informed some workers about the opportunity to participate in the experiment and encouraged them to discuss it with other workers and site supervisors, creating a snowball effect within the company. Those interested in participating contacted the site manager to be assigned a date and time for the experiments, observations, or interviews.

A total of 10 construction workers from Acciona participated in the experimental process. Despite efforts to include female participants, none were enrolled. Nevertheless, the sample accurately reflects the current demographics of the construction workforce, as women rarely occupy manual labor roles in this context. Therefore, for the pilot evaluation, a total of N = 10 users (all male) participated in the evaluation. All participants reported (each on a scale from 1 to 5) little to no experience working with robots (mean (M) = 2.0 and standard deviation (SD) = 0.7). However, nearly all believed that robotic systems can be beneficial for so-called 3D jobs (dirty, dull, and dangerous). They also overwhelmingly believed that robotic systems are necessary to take over some strenuous tasks (M = 4.9 and SD = 0.3). Furthermore, frequent use of robotic systems will reduce musculoskeletal-related pain (M = 4.7 and SD = 0.7). Overall, the sample displayed high openness towards new technologies (M = 3.9 and SD = 0.7).

The research and testing procedures outlined in this study complied with ethical principles, ensuring the protection of participants’ rights, privacy, and well-being. Informed consent was obtained from all participants, and their participation remained voluntary throughout the process. All personal and sensitive data collected were handled with strict confidentiality and in accordance with relevant data protection laws. The study was reviewed and approved by the Ethics Committee of the University of the Basque Country, ensuring that the research adhered to the highest ethical standards.

2.3. Equipment and Instrumentation

2.3.1. Robot

The robot utilized in this pilot study is the UR10e [16], a collaborative industrial robot manufactured by Universal Robots, which can handle payloads up to 10 kg and has a reach of 1300 mm. This robot has two characteristics that make it suitable for the job: firstly, it is equipped with advanced safety mechanisms, allowing it to work safely in close proximity to humans without the need for protective barriers. Additionally, it can be integrated with haptic devices for teleoperation, providing operators with force feedback to enhance control and precision, while also enabling remote robot operation. For the task of teaching joint filling with mastic via teleoperation, the robot was installed on a profile table at ground level, with its arm fitted with a mastic gun and controlled using a haptic device.

2.3.2. Haptic Device

The Haption Virtuose 6D [17] is a high-performance haptic device designed for precise teleoperation. It offers six degrees of freedom (6DoF) in force feedback and motion control, enabling remote robotic control with high accuracy. The robust force feedback and ergonomic design allow users to experience weight, texture, and resistance, making it suitable even for delicate tasks. The programmable buttons on the handle have been programmed to control the mastic extruder.

2.3.3. Extruder

The extruder consists of a cordless Milwaukee M18 caulk gun [18], a custom-made holder for mounting the caulk gun to the UR10e robot flange, and an actuation unit. The M18 caulk gun can extrude up to 4.5 kN and is designed to handle adhesives in cold temperatures. It features an anti-drip mechanism and a robust plunger to ensure a consistent flow. The linear pneumatic actuator, controlled by an Arduino UNO, activates the trigger of the caulk gun to start the mastic application. Although the extrusion speed can be adjusted either directly at the caulk gun or by controlling the compressed air pressure, it was kept constant during this pilot.

2.3.4. Physiological Signal Acquisition Devices

Two wearable devices from Bitbrain were used to collect physiological signals during the experiment: Diadem [19], a head-mounted EEG system, and Ring [20], a finger-worn biosensor. Bitbrain Diadem is a lightweight EEG monitoring system designed for comfort and minimal interference. It records up to 12 dry EEG channels positioned over the pre-frontal, frontal, parietal, temporal, and occipital regions, enabling the estimation of emotional and cognitive states in real time. Bitbrain Ring is a compact device worn on the finger, equipped with sensors for galvanic skin response (GSR), blood volume pulse (BVP), and a three-axis accelerometer (ACC) to capture cardiac activity and estimate motion artifacts. Together, these devices provide a comprehensive view of users’ psychophysiological states during interaction with the system.

2.4. Evaluation Metrics

2.4.1. Task Performance

Several metrics were used to evaluate task performance. The time required to complete each task was measured using a stopwatch, including the time taken to fill a joint: manually, via teleoperation, and autonomously by the robot. To assess the quality of joint filling, once the material had dried, a section of the concrete slab was cut to evaluate whether the system was able to fully fill the mastic at the bottom of the joint, which is a usual problem in actual real environments. During both the teleoperation and autonomous filling phases, additional data from the robotic system were collected. These include positions, orientations, and velocities of the robot; forces exerted on the robot; the status of the application tool and pedal; any potential errors in the robotic system. These data were used to automate the deposition task and to count the percentage of failed executions, if any.

2.4.2. Objective Assessment of User States

For the quantitative analysis, psychophysiological signals were measured to evaluate users’ emotional and physiological responses while interacting with the system. Specific psychophysiological signals to be recorded included electroencephalographic (EEG) signals, galvanic skin response (GSR), and heart rate (HR). These signals were recorded using non-invasive, dry, passive surface sensors. The EEG headband was placed on the participant’s head first, followed by the GSR and HR sensors on the index and middle fingers of the non-dominant hand. These recordings were used to determine key emotional–cognitive metrics (valence, activation, impact, engagement, workload, and memorization) that represent underlying psychological states during system interaction. The theoretical basis for mapping physiological signals to these metrics is discussed in detail in [8] and falls outside the scope of the present work.

To identify differences across operation modes (manual, teleoperated, and automatic), a Kruskal–Wallis H test was applied to each physiological metric. This non-parametric approach was selected due to the nature of the data, as normality could not be assumed for the signal-derived metrics. When significant effects were detected, post-hoc pairwise comparisons were conducted using Dunn’s test with appropriate correction for multiple comparisons.

Lastly, these metrics are linked to user acceptance, positioning them as interpretable, quantifiable indicators that can inform how users genuinely perceive and adapt to the technology. Sustainable variations in these established metrics pointed to significant differences in technology acceptance levels among users.

2.4.3. Subjective Assessment of Technology Interaction

The subjective evaluation included different questionnaires to cover different facets of usability and the participants’ experiences. Part of the evaluation included NASA-TLX Scales [12], a widely used multidimensional tool to evaluate perceived workload. It consists of six sub-scales: mental demand, physical demand, temporal demand, performance, effort, and frustration. Each sub-scale was rated for one task within a total point range with 5-point increments. Increments of high, medium, and low estimates for each point resulted in 21 gradations on the scales. All ratings were combined to form the task load index.

Another standardized questionnaire was the SUS [13]. The SUS consists of 10 items, each rated on a 5-point Likert scale, and provides a global usability score, which can range from 0 to 100. Higher scores indicate better perceived usability of the evaluated system. In addition, a scale consisting of seven items, each relating to one of the interaction principles from ISO 9241-210:2019 [14], was used to get an overview of the general interaction principles of the system.

Additionally, participants answered questions regarding their experience with robots in general, with two questions relating to the experience and four questions referring to the assessment of robots in general. Participants also answered questions about their socio-demographic background, including their position (worker or manager), gender, and age. To have more detailed information on the specific sample, a shortened version of the ATI [15] was used, which provided an overview of this sample’s general affinity for technology. To gain further insights into the participants’ physical perception of the performed task, a pain scale was developed. The items are based on the Nordic Musculoskeletal Questionnaire (NMQ [21]), which consists of 40 items and identifies areas of the body causing musculoskeletal issues. For this trial, six items were used. Five items dealt with the amount of pain a participant felt in different body regions (upper back, shoulders, upper limbs, lower back, and lower extremities) and were rated on a 4-point scale (1 = never and 4 = always). In addition to recording the current pain levels, the fifth and last item assessed the participants’ expectations regarding possible pain relief when using the robotic system on a 5-point scale.

3. Results

This section presents a comprehensive evaluation of the robotic system’s performance and user experience in the context of mastic filling tasks. The first part focuses on analyzing the quality of joint filling, task completion times, and the material deposition across the three working modalities to prove its potential for using the system in the real world in terms of efficiency and consistency.

Next, the objective assessment of the technology is discussed, based on physiological data collected during the experiment. Key emotional and cognitive states, such as valence, engagement, workload, and memory load, are analyzed to understand how users perceived the system during interaction.

Finally, the subjective assessment is presented, including standardized usability questionnaires such as the NASA-TLX and SUS, as well as qualitative feedback from open-ended responses from the construction workers that explore users’ perceptions of the system’s usability, learnability, and potential impact on physical and cognitive workload.

3.1. Task Execution

The task execution was measured according to different evaluation metrics: (1) the quality of the joint filling in automation, (2) the task execution times in the three modalities, and (3) the amount of centimeters filled in the joint.

The quality of expansion joint filling was evaluated by analyzing the consistency of mastic application. Manual methods using a classic gun often result in incomplete filling at the joint’s base, even for experienced users, due to the mastic’s changing density and visibility issues. If the joint is partially unfilled on top, it can be corrected after a quick visual inspection. However, if there is a lack of mastic inside the joint, this cannot be rectified as it is not visible to the naked eye. To assess the filling quality in this pilot study, a joint was considered properly filled if it was fully filled from the base of the joint, even if it had a few millimeters gap at the top or was slightly overfilled. Indeed, the operators were not instructed to avoid material waste but to ensure the joint was filled from the bottom in all three modalities during the training. Therefore, overfilled joints are not counted as badly filled joints in any of the modalities. This definition was used because the operators had received only 30 min of active training with the robot, which may not have been enough to achieve perfect results. Assuming this definition, 100% of the joints were adequately filled in automation mode. The robot controller ensured consistent contact with the surface, filling the joint from below and preventing internal gaps. Regarding material waste, it was not possible to quantify the excess material applied. However, instances of overfilling were observed, primarily due to variations in joint size. With a constant robot velocity, changes in joint width led to variations in volume, resulting in overfilling and material waste in some sections.

The next set of images shows the results obtained for the three filling modalities for different users. Figure 3 shows the results of the small joint filled in the 3 modalities by an operator. The manual filling of the expansion joint shows irregularities and is not homogeneous. The teleoperated demonstration is not regular either, but the learning from demonstration approach is capable of adequately extracting the robot control parameters to have regular, homogeneous, and smooth filling up to the top joint. To compare the manual (Ma in red) and automatic (A in red) mastic deposition, a portion of the concrete block in each operation mode was extracted after the mastic dried to analyze the performance and check the bottom of the expansion joint (upper-right image). Although from the upper view the joint seems to be correctly filled in manually (Ma), the reality is that the mastic does not fill the totality of the lower part of the joint.

Figure 4 shows a case of an overfilled joint in automation. As can be observed in the lower part of the image, the joint is overfilled both in the manual and automated filling. After mastic excess removal (upper-right image), it can be observed how both joints are filled up to the top, but again in the manual mode, the bottom of the joint is not correctly filled, while in automation, the mastic fits the entire joint.

Figure 5 shows the demonstration of a large joint filling performed by an experienced teleoperator, outside the pilot, with the same setup and concrete block. It shows how, with some experience using the teleoperation device, perfect results can be achieved. The zoomed area inside the red bubble provides a closer look at the automation vs. manual joint filling. As can be observed, the manual filling is more uneven compared to the automatic deposition. The automatic filling with the robot ensures the mastic tool contacts the surface [6] and provides a smooth, homogeneous finish, in addition to completely filling the expansion joint.

The next evaluation metric is the task execution times in the three modalities. Figure 6 shows a boxplot graph for the three demonstration modalities (manual, teleoperation, and automation) and the three sizes of joints (small, medium, and large), while

Table 1. Mean and standard deviation of task execution times per interaction mode and joint size.

Interaction Mode	Small	Medium	Large
Manual	00:50 ± 00:18	00:56 ± 00:20	01:21 ± 00:25
Teleoperation	00:17 ± 00:04	00:19 ± 00:02	00:27 ± 00:06
Automation	00:23 ± 00:08	00:40 ± 00:06	01:13 ± 00:20

gathers the mean and standard deviation values of task execution times per interaction mode and joint size.

The mean value for the 10 users is represented with the red horizontal lines, and the dots represent the outliers. The mean teleoperation time for small and medium tasks is 19 s and 20 s, respectively, and 27 s for large tasks. The variability in teleoperation times across users is relatively low, with most users completing tasks within a narrow range of times. This indicates that the teleoperation mode is consistent and reliable, with no significant differences in completion times among users. However, minor variations exist; for instance, there is a user with slightly longer teleoperation times, while another has slightly shorter times (the anomalies are dotted in the boxplot graph). Despite these minor differences, the teleoperation mode remains an efficient and reliable method for task demonstration.

When comparing manual and automated demonstrations the system shows significant difference regarding both joint size and operation mode. The robot’s automated mode consistently outperforms manual operation in terms of execution time, with the extent of the difference varying by joint size. For the small joint, the robot is faster for all 10 users, with an average speed advantage ratio of 2.32, indicating that automation is more than twice as fast. In the medium joint, the robot is faster for 7 out of 10 users; in the cases where manual is faster, the difference is minimal (1 s in two cases), and one case involves an error. The average speed advantage ratio in the medium joint is 1.43. For the large joint, automation is faster for six users, with the average advantage ratio dropping to 1.11. The remaining cases, where manual is faster, show negligible differences, with one case involving an error in the automatic execution. Overall, the trend across different joints demonstrates that automated robot filling outperforms human operators, particularly in smaller joints.

In terms of the amount of material filled, the robot was instructed to move linearly over 40 cm. The activation of the gun was delayed due to the time it took for the air-compressed actuator to engage, which affected the total distance filled, especially in smaller joints where the robot’s velocity was higher. This delay is less noticeable in medium and large joints, where the robot’s velocity was lower. For the large joint, the 40 cm filling was nearly achieved in all cases, likely due to the slower velocity and larger volume to fill. This effect can be observed in

Table 2. Mean and standard deviation of centimeters filled per interaction mode and joint size.

Interaction Mode	Small	Medium	Large
Manual	41.0 ± 2.3	40.6 ± 2.0	40.9 ± 1.8
Teleoperation	11.9 ± 4.2	10.8 ± 1.8	9.9 ± 1.7
Automation	34.9 ± 1.6	37.4 ± 2.6	38.6 ± 3.3

and its associated boxplot, Figure 7. The starting position for the joint was manually selected by the non-robotic operator, which could influence the path the tool took until it made contact with the surface. In the medium joint, the delay had a minimal impact on the amount of material filled compared to manual operations. The most significant difference was observed in the smaller joint, where the robot’s high velocity caused the tool to move a greater distance before the material flow began.

These findings highlight the efficiency and reliability of automated robot filling, particularly in tasks involving smaller joints, and suggest areas for further optimization to minimize delays and improve performance consistency. The robot’s automated mode outperformed manual operations in terms of execution time, consistency, and the ability to fill joints from the base, ensuring a more uniform and thorough filling. Despite some instances of overfilling due to variations in joint size and the robot’s constant velocity, the automated system demonstrated significant advantages over manual methods, especially in reducing the risk of internal gaps and achieving a smoother finish. While there is room for improvement in optimizing material usage and adjusting for different joint sizes, the quality and efficiency results of this pilot study indicate that robotic automation has the potential to significantly enhance the quality and speed of joint filling tasks.

3.2. Objective Assessment of the Technology

To obtain an objective view of technology acceptance, the participants’ physiological signals are analyzed and six key metrics extracted: valence, activation, impact, engagement, workload, and memory. Later, a cross-metric discussion that integrates the significant differences identified in the individual analyses is provided.

3.2.1. Technology Acceptance Metrics

The following paragraphs briefly outline each of the key metrics and their relevance to users’ technological acceptance. For each metric, paired boxplots are presented displaying its distribution by operation mode and joint size. Differences between modes were statistically tested using the Kruskal–Wallis H test, followed by Dunn’s post-hoc comparisons when appropriate. The statistical significance of these comparisons is indicated in the title of each boxplot.

• Activation: This captures the amplitude of physiological arousal. In isolation, it is a neutral indicator for acceptance, pointing to either beneficial states (excitement) or counterproductive ones (stress). Its interpretation, therefore, depends on its interplay with valence and impact, as well as task context. As shown in Figure 8, participants exhibited comparable levels of activation throughout all experimental stages, with no significant differences observed between scenarios.

  Figure 8. Boxplot distributions of activation by operation mode and joint size.

  Figure 8. Boxplot distributions of activation by operation mode and joint size.
• Engagement: This represents the degree of cognitive and affective immersion. High engagement fosters immediate involvement and motivation, strengthening users’ commitment to continued use. Figure 9 points to higher engagement during manual operation in comparison with teleoperated or automatic modes.

  Figure 9. Boxplot distributions of engagement by operation mode (significant differences) and joint size.

  Figure 9. Boxplot distributions of engagement by operation mode (significant differences) and joint size.
• Impact: This captures the intensity of the emotional response. A strong impact combined with positive valence reinforces acceptance, whereas the same intensity paired with negative valence can discourage continued use. Figure 10 shows the statistically significant differences both in operation mode (higher impact on manual mode) and joint size (higher impact on large size joints).

  Figure 10. Boxplot distributions of impact by operation mode and joint size, both with significant differences.

  Figure 10. Boxplot distributions of impact by operation mode and joint size, both with significant differences.
• Memory: This shows how much the task depends on recalling information or steps. If memory demands remain high even after the task has been repeated many times, it suggests the process never becomes automatic, which may reduce long-term willingness to use the system. Figure 11 reveals significant differences in memory load across operation modes, with manual mode showing the highest demands.

  Figure 11. Boxplot distributions of memory by operation mode (significant differences) and joint size.

  Figure 11. Boxplot distributions of memory by operation mode (significant differences) and joint size.
• Valence: This indicates how pleasant or unpleasant users feel during interaction with the system. A higher, more positive valence generally promotes comfort and strengthens the intention to adopt and continue using the technology. Figure 12 reveals higher valence levels in both the teleoperated and automatic modes, whereas the manual mode registered significantly lower scores.

  Figure 12. Boxplot distributions of valence by operation mode (significant differences) and joint size.

  Figure 12. Boxplot distributions of valence by operation mode (significant differences) and joint size.
• Workload: This represents the mental effort the task demands. Lower workload encourages technology acceptance, as users feel less strain and are more inclined to persist. However, adopting any new system usually involves a learning barrier and a consequent spike in workload. Tracking how this metric evolves over time offers a clear picture of the technology’s intuitiveness and overall user-friendliness. Figure 13 shows higher workload levels in both the teleoperated and automatic operation modes compared with the manual mode.

  Figure 13. Boxplot distributions of workload by operation mode (significant differences) and joint size.

  Figure 13. Boxplot distributions of workload by operation mode (significant differences) and joint size.

To consolidate these findings and prepare the ground for the cross-metric discussion in Section 3.2.2,

Table 3. Psychophysiological results by control mode.

Metric	Manual (Mean ± SD)	Teleoperated (Mean ± SD)	Automatic (Mean ± SD)	Kruskal–Wallis Significancy	Post-Hoc Comparison	Dunn Significancy
Activation	−8.75 ± 24.63	1.40 ± 19.93	0.25 ± 10.66	p = 0.097	–	–
Engagement	46.20 ± 20.63	29.77 ± 13.16	32.38 ± 11.32	p = $1.9\times {10}^{-4}$	Man. > Aut.	p = 0.0026
					Man. > Tel.	p = 0.0003
Impact	42.85 ± 33.86	10.76 ± 9.41	19.27 ± 28.23	p = $1.2\times {10}^{-5}$	Man. > Aut.	p = 0.0020
					Man. > Tel.	p < 0.0001
Memory	36.91 ± 14.54	17.24 ± 9.46	26.00 ± 11.02	p = $7.7\times {10}^{-8}$	Man. > Tel.	p < 0.0001
					Man. > Aut.	p = 0.0017
					Aut. > Tel.	p = 0.0146
Valence	−10.03 ± 22.03	2.95 ± 23.44	3.64 ± 15.55	p = 0.020	Aut. > Man.	p = 0.0317
					Tel. > Man.	p = 0.0439
Workload	20.24 ± 12.89	32.20 ± 14.99	35.18 ± 10.98	p = $9.1\times {10}^{-5}$	Aut. > Man.	p = 0.0001
					Tel. > Man.	p = 0.0018

and

Table 4. Psychophysiological results by joint size.

Metric	Small (Mean ± SD)	Medium (Mean ± SD)	Large (Mean ± SD)	Kruskal–Wallis Significancy	Post-Hoc Comparison	Dunn Significancy
Activation	−1.31 ± 17.83	−3.90 ± 17.39	−1.90 ± 23.61	p = 0.867	–	–
Engagement	37.58 ± 22.45	34.01 ± 12.91	36.76 ± 14.50	p = 0.653	–	–
Impact	14.41 ± 14.28	24.14 ± 27.48	34.34 ± 37.93	p = 0.012	Large > Small	p = 0.0210
Memory	28.17 ± 15.14	26.07 ± 13.46	25.92 ± 14.47	p = 0.722	–	–
Valence	0.64 ± 23.00	−3.50 ± 19.09	−0.58 ± 22.28	p = 0.732	–	–
Workload	28.56 ± 13.65	30.32 ± 15.34	28.74 ± 14.75	p = 0.856	–	–

summarize the descriptive statistics along with the outcomes of the Kruskal–Wallis H test and Dunn’s post-hoc comparison for every metric across control modes and joint sizes, respectively.

3.2.2. Cross-Metric Discussion

Building on the results presented above, this section integrates the individual metrics to provide a more comprehensive view of user experience and acceptance. While operation mode produced significant differences across multiple metrics, joint width showed a marginal effect, limited to the impact dimension. Consequently, the following analysis focused on operation mode as the primary factor shaping user perception.

While each metric offers valuable information on its own, examining them collectively provides the broader context needed to understand technology acceptance. To structure this perspective, the results were analyzed across three interconnected key dimensions to evaluate the system acceptance: emotional comfort, cognitive comfort, and personal involvement.

To assess the emotional comfort, three complementary signals were combined: the pleasantness of the experience (valence), the physiological arousal elicited by the emotion (activation), and the intensity of the emotional response (impact). Automatic and teleoperated modes displayed significantly higher and explicitly positive mean valence values, whereas the manual mode registered a negative mean valence. Activation remained comparable across modes, and impact was minimal in both automatic and teleoperated conditions, resulting in a pleasant and stable emotional profile aligned with sustained acceptance. In contrast, the manual mode not only produced a negative emotional tone but also paired it with high impact, intensifying negative affective states such as frustration and thereby limiting user satisfaction in comparison to the automated alternatives.

The analysis of cognitive comfort considered both short-term mental effort (workload) and long-term cognitive demand (memory). The automatic and teleoperated modes exhibited higher workload values, likely due to the initial learning curve associated with unfamiliar technologies. In contrast, the manual mode showed a significantly lower workload, reflecting participants’ familiarity with the process and the absence of a learning barrier. However, this apparent advantage was counterbalanced by a significantly higher memory load, suggesting that the manual task still requires sustained cognitive control and does not become fully intuitive over time. This finding underscores the potential of automated systems to reduce long-term cognitive strain once users have overcome the initial adaptation phase.

Building on the personal involvement, it refers to the degree to which users feel cognitively and emotionally engaged with the task. In this regard, the manual mode showed a clear advantage, with significantly higher engagement levels than both the automatic and teleoperated modes. This stronger engagement was further reinforced by high impact values, suggesting that participants felt more connected to the activity, with a greater sense of presence and participation throughout the task. This dynamic may represent a psychological barrier to the adoption of automated systems, which can be perceived as more distant or impersonal by users accustomed to direct, hands-on control.

Taken together, the results highlight a nuanced balance between emotional comfort, cognitive effort, and personal involvement, which are key elements in shaping user perception during the adoption of new technologies. While the manual mode encourages greater engagement and a stronger sense of participation, it also presents a negatively valenced emotional profile and elevated memory demands. In contrast, the automatic and teleoperated modes provide a more emotionally pleasant and cognitively sustainable experience, suggesting a higher potential for long-term user satisfaction. Although the initial workload was higher due to the learning curve, this barrier can be mitigated through appropriate training and familiarization strategies. Overall, these findings support the viability of automated solutions as a favorable path toward broader and more sustained technology acceptance.

3.3. Subjective Assessment of the Technology

3.3.1. Quantitative Analysis

The results of the NASA-TLX for the mastic application showed that the robotic system was perceived as moderately to highly mentally demanding (M = 10.1 and SD = 5.7), suggesting that the technology requires significant cognitive effort. In comparison, physical demand was rated lower (M = 3.5 and SD = 2.6), showing that the system offers physical support for the users. Temporal demands (M = 5.2 and SD = 4.2) were perceived as moderate, which indicates that the task was neither too fast nor too slow to be able to follow. Despite some participants experiencing more failures during the task, task performance was rated as somewhat successful (M = 5.3 and SD = 4.2). Required effort was reported as moderate (M = 6.4 and SD = 4.5) and frustration was present, although only somewhat (M = 4.7 and SD = 4.7). Overall, the experienced workload was moderate to low (M = 5.9 and SD = 2.8), even though there was some variance among the participants. Some participants stated they felt insecure, discouraged, irritated, stressed, or annoyed at higher levels.

For system usability measured with the SUS, a range of ratings from 50 to 97.5 was found, with the overall mean rating of M = 76.3 and SD = 14.9, which is considered acceptable. Most ratings were in the “good” or “excellent” category with ratings above 71.0 and 77.1. Only one participant rated the system as “poor”, with a rating below 51.7.

For the interaction principles, the results can be found in Figure 14. For each interaction principle, the mean value was calculated. Of the seven principles, the highest rated was user error robustness (M = 4.9 and SD = 0.3), emphasizing that the system allows errors to be corrected by the users. The second highest principle, learnability (M = 4.6 and SD = 0.5), shows that participants could easily learn how to use the system and felt that they could discover its functions through exploration. The lowest rated interaction principle was self-descriptiveness (M = 2.9 and SD = 1.3), which suggests that there was no clear and immediate feedback to the participants given by the system. The low rating might have been given because the system has no interface to directly communicate with the participants; therefore, their current position in the workflow and next steps were not always clearly understood. Self-descriptiveness was also the only interaction principle rated below a 3.8, with all other interaction principles rated 3.9 or higher. This indicates a general positive view of the system regarding the interaction principles.

Additionally, the haptic device of the system was assessed in greater detail. This included three questions, which were rated on a five-point Likert scale (1 = strongly disagree to 5 = strongly agree). Overall, participants rated the repositioning of the haptic device as rather intuitive (M = 3.9 and SD = 1.0) as well as straightforward (M = 4.2 and SD = 0.6), and participants agreed that they did not want the device removed from the process (M = 2.3 and SD = 1.1).

Regarding the physical perception of different body regions during task performance, participants rated all body regions around 2, which indicates a generally low level of pain across the sample. The lowest amount of pain was recorded in the shoulders (M = 2.0), the highest amount of pain was reported for the lower back and lower extremities (M = 2.4 and M = 2.4). All other regions were between 2.0 and 2.4, showing that a medium to low level of pain can be assumed for this sample. Regarding the participants’ expectations of possible pain relief if using the robotic system on a regular basis, the mean rating was M = 4.7, which indicates that participants can envision opportunities for the robotic system to alleviate physical discomfort. This shows that despite general pain ratings already being low, there is still more opportunity to improve for the participants.

3.3.2. Qualitative Analysis

In addition to the questionnaires, open-ended questions were posed to allow the participants to give feedback and their ideas. The questions dealt with expected changes in the working tasks if the robotic system were used on a more regular basis. The focus was on both potential benefits and challenges the participants could foresee. The possibility for better quality and more efficiency was mentioned as a benefit, and participants saw opportunities for faster and more precise work with the system. They also saw benefits for better physical ergonomics and reduced physical strain by minimizing difficult postures and heavy lifting when using the system. It was also mentioned that automation has the potential for a safer work environment, but no further specifications were given on that point.

The main point, however, was that first a higher technology readiness level (TRL) has to be achieved to be able to implement the technology successfully. Following this, many saw multiple key benefits and had a generally positive attitude towards the system.

4. Discussion

This study aims to provide a comprehensive understanding of the system’s effectiveness and user experience. By evaluating the quality of the joint filling system in manual, teleoperation, and automatic modes, its reliability and efficiency have been determined. The study also seeks to gather feedback from participants about the system, which will inform future design iterations.

The robotic system demonstrated significant improvements in efficiency and consistency for filling expansion joints with mastic, outperforming manual methods. The system’s performance was consistent across users, with low variability, and ensured high-quality joint filling, even in inner areas. The admittance controller played a crucial role in maintaining constant contact with the surface, preventing internal gaps. These results were made possible by the robot’s ability to reproduce key control parameters demonstrated by the operators, such as the extruder’s contact force and forward speed. User training and error analysis highlighted the system’s ease of use and potential for professional-grade performance.

From a psychophysiological perspective, the results suggest that automation contributes positively to user acceptance by fostering a more emotionally pleasant and cognitively sustainable experience. Both automatic and teleoperated modes showed higher levels of emotional valence and lower long-term cognitive demands compared to the manual condition, indicating a more comfortable and less mentally taxing interaction. While the workload associated with robotic modes was initially higher, this is likely attributable to the learning curve inherent in unfamiliar technologies, an effect that can be addressed through targeted training and progressive familiarization. Notably, the manual mode was associated with a negative emotional tone and higher cognitive strain over time, particularly in terms of memory load, which may reflect a lack of task intuitiveness despite participants’ familiarity.

These findings underscore the potential of robotic assistance not only to reduce physical strain but also to support emotional well-being and mental efficiency in demanding construction tasks. The physiological indicators thus provide objective evidence of the emotional and cognitive relief users may experience through automation.

However, engagement remains a critical challenge for automated solutions. The manual mode showed significantly higher engagement and emotional resonance, likely driven by users’ active control. In contrast, the passive nature of robotic operation may lead to a reduced sense of involvement. To address this, future developments should explore strategies to foster user engagement even in automated workflows, for example, through adaptive interfaces, real-time feedback, or optional manual interventions that preserve a sense of contribution and task ownership. Enhancing engagement in this way could further strengthen the emotional connection with the system and support long-term user acceptance.

Overall, the subjective evaluation revealed both strengths and areas for improvement. In general, the system’s usability was acceptable, despite there being some challenges. One challenge was the interaction principle self-descriptiveness, which was rated lowest by far among all interaction principles. The low rating indicates that the system does not always offer feedback that is clear and immediate. This can have a negative effect on the interaction, making it more difficult for the users to understand the system. To address this, more transparency and guidance given by the system to the participants could further improve the interaction. In contrast, the interaction principle user error robustness achieved a high score, indicating that the system effectively supported users in managing and recovering from user errors during operation.

Despite the need for improvement in some areas, participants were generally satisfied with the system and found the haptic device to be intuitive and straightforward. This shows that the system already meets basic usability expectations, although some developments are still needed. In particular, the principle of conformity with user expectations also showed room for improvement, which could explain some results regarding the experienced workload. Some users reported frustration, which could be directly related to the low levels of the mentioned interaction principle. However, the NASA-TLX performance ratings indicated that most users rated the task accomplishment as successful, which may be related to the high ratings for the interaction principle suitability for the task.

One key benefit frequently mentioned by participants was the system’s potential to support physical well-being, with a reduction in physical strain, fatigue, and risk of injury among the most often anticipated benefits. Beyond the health-related expectations, participants also identified performance-related benefits, including improved efficiency, accuracy, and productivity. Over time, the system came to be seen as a means of alleviating discomfort and pain associated with physically demanding tasks. Although initial reports of pain were already generally moderate to low, these results show a promising trend toward even less discomfort with prolonged use. In general, the mastic application was considered to be of acceptable usability, with potential benefits in terms of both health and performance. However, further refinements remain essential, especially regarding the interaction principle self-descriptiveness.

While the objective assessment distinguished between the three different operating modes, the subjective assessment focused only on a summative rating of the subjective perception of robotic use. This was done in order to extend the duration of technology use for each user, allowing them to gain a more thorough experience of the robotic system and, in turn, provide a summative rating for both robotic use modes. Therefore, the objective and subjective ratings cannot be combined into a single rating, but rather complement each other to provide a holistic view of the users’ perceptions. The subjective assessment indicated that while the mental demands were perceived as moderate to high, the overall workload was rated as moderate to low, while the level of effort required to achieve satisfactory performance was also moderate. These results suggest that although users were cognitively engaged, the task did not consistently overwhelm them.

Summary of the Results

To provide a clear overview of the experimental findings,

Table 5. Summary of key results and findings.

Category	Metric	Key Finding
Task Execution	Filling quality	100% of joints were successfully filled using the automated system. The manual process showed internal gaps.
	Execution time	The automated approach performed the task faster than the manual mode.
	Filled length	Small joints are more affected by the delay of the gun activation in automation
	Training time	A 30-min training session was sufficient to achieve acceptable performance levels.
Physiological response	Valence	Positive emotional response was significantly higher in teleoperated and automated modes; manual execution was associated with a more negative experience.
	Workload	Workload was perceived as higher in automation and teleoperation, possibly due to the learning curve.
	Engagement	Highest in manual mode, likely due to increased user control and involvement.
	Impact	Manual had higher emotional impact. It was also the only aspect where joint size made a difference, with large joints producing greater emotional impact than small ones.
	Memory	Manual required greater memory load; automation reduced long-term cognitive demand.
	Activation	Activation levels were similar across all operation modes and joint sizes, showing no significant physiological arousal differences.
Subjective assessment	NASA-TLX	Moderate workload and moderate mental demand.
	SUS score	Usability was generally perceived as acceptable, with most participants rating the system in the “good” to “excellent” range.
	Interaction principles	Users rated the system highly for error recovery and learnability, but pointed that Self-Descriptiveness should be improved.
User feedback	Benefits	Expected improvements in quality, efficiency, ergonomics, and safety.
	Challenges	Key concerns included the need to increase the system’s TRL and to improve interaction intuitiveness.

presents our key results across task execution, psychophysiological metrics, and subjective usability ratings, along with the main insights identified in each area.

5. Conclusions and Future Work

This pilot study evaluated the robotic system using a combination of qualitative and quantitative methods to gather comprehensive feedback on its performance and user experience. Qualitative data were collected through questionnaires and brief interviews, while quantitative data included task execution times, filling quality, and physiological signals.

Automating the mastic filling task with a robot is a highly beneficial approach that addresses the numerous challenges associated with manual application. Robots perform the task with greater precision and consistency, reducing the risk of errors and ensuring higher-quality results. By taking over the physically demanding and hazardous aspects of the job, robots can significantly enhance worker safety and reduce the incidence of musculoskeletal disorders (MSDs) and other occupational health issues. Teleoperation is a particularly effective solution for demonstrating and executing the mastic filling task from a safe distance. Through teleoperation, expert operators have controlled the robot remotely, maintaining control over the process while avoiding the physical and safety risks associated with direct manual labor. This approach not only ensures that the task is performed efficiently but also allows for real-time adjustments and fine-tuning, adapting to the dynamic conditions of construction sites. By leveraging teleoperation, the automated system can achieve optimal performance and reliability, ultimately leading to a safer, healthier, and more productive work environment.

In general, workers see the mastic application as having great potential to improve physical ergonomics, particularly by reducing strain on the musculoskeletal system, while at the same time improving performance, efficiency, and productivity. To build on this positive perception and encourage greater user acceptance, future developments should aim to maintain the system’s benefits for emotional well-being and physical relief, while increasing user engagement and clarity of interaction. Balancing these aspects will be essential to optimize both the overall user experience and the long-term integration of the application into everyday working practices.

To do so, future work will focus on different lines of development. Firstly, we plan to implement a visual system to enhance the awareness of the environment. Utilizing artificial vision, expansion joints could automatically be located in the scene. That way, the system would position the mastic gun autonomously over them without human intervention. Additionally, visual servicing would also be used to detect underfilling or overfilling of joints, allowing the robot’s speed to adjust accordingly. This feature will enable the robot to adjust its speed and deposition quality accordingly, reducing the need for manual intervention. We also plan to integrate the robotic system onto a mobile platform to enable easy movement through the workplace. The mastic extruder will be adapted to provide a continuous supply of mastic, eliminating the need for manual tube changes and minimizing manual intervention. Finally, we propose to have longer training sessions with operators to enhance the results of the mastic deposition combined with the development of a feedback system for providing clear feedback of the system’s status to the user.