Next Article in Journal
Teenage Pregnancy and Mental Health
Next Article in Special Issue
Robots Working with Humans or Humans Working with Robots? Searching for Social Dimensions in New Human-Robot Interaction in Industry
Previous Article in Journal
Special Education Teacher Leadership in Jordan: Current State and Constraints
Previous Article in Special Issue
Robots, Industry 4.0 and Humans, or Why Assembly Work Is More than Routine Work
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

First Application of Robot Teaching in an Existing Industry 4.0 Environment: Does It Really Work?

1
Automation and Control Institute, Vienna University of Technology, 1040 Vienna, Austria
2
PROFACTOR GmbH, 4407 Steyr, Austria
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Societies 2016, 6(3), 20; https://doi.org/10.3390/soc6030020
Submission received: 29 February 2016 / Revised: 4 July 2016 / Accepted: 11 July 2016 / Published: 20 July 2016
(This article belongs to the Special Issue Robots and the Work Environment)

Abstract

:
This article reports three case studies on the usability and acceptance of an industrial robotic prototype in the context of human-robot cooperation. The three case studies were conducted in the framework of a two-year project named AssistMe, which aims at developing different means of interaction for programming and using collaborative robots in a user-centered manner. Together with two industrial partners and a technological partner, two different application scenarios were implemented and studied with an off-the-shelf robotic system. The operators worked with the robotic prototype in laboratory conditions (two days), in a factory context (one day) and in an automotive assembly line (three weeks). In the article, the project and procedures are described in detail, including the quantitative and qualitative methodology. Our results show that close human-robot cooperation in the industrial context needs adaptive pacing mechanisms in order to avoid a change of working routines for the operators and that an off-the-shelf robotic system is still limited in terms of usability and acceptance. The touch panel, which is needed for controlling the robot, had a negative impact on the overall user experience. It creates a further intermediate layer between the user, the robot and the work piece and potentially leads to a decrease in productivity. Finally, the fear of the worker of being replaced by an improved robotic system was regularly expressed and adds an additional anthropocentric dimension to the discussion of human-robot cooperation, smart factories and the upcoming Industry 4.0.

Graphical Abstract

1. Introduction

The last few decades have demonstrated how many sectors of production can benefit from fast, vast and inexpensive industrial robots. However, direct human-robot cooperation is still considered as not safe enough for the human. Furthermore, the wide deployment of industrial robots went along with the replacement of human labor. In order to develop competitive alternatives, the human and the robot should work as a team to achieve more flexible human-robot interaction [1]. Such an approach has to take the User Experience (UX) within these robotic assembly lines into account. Usability aspects play an important role in terms of the feasibility of a robotic system for different application areas, as well as the user acceptance of the system when integrated into daily working routines. Following the definition of Alben [2], UX is a broad concept including cooperation and usability, as well as factors, such as perceived safety, stress and emotion [3]. Especially in the industrial context, UX factors have to be understood as dynamic over time, which makes them a crucial element in the production process [4].
The idea that an industrial robot can take the role of a cooperative and supportive tool for the workers is part of a relatively new paradigm called “Industry 4.0”. This paradigm describes the upcoming fourth industrial revolution and envisions “smart factories” in which humans and robots will work more and more closely together. The first industrial revolution was the mechanization with water and steam power; it was followed by the second industrial revolution: mass production using assembly lines and electrical energy. After that, the digital revolution began, and the use of electronics and IT in the production processes became common. The term “Industry 4.0” is collectively used for technologies and concepts of the value chain organization characterized by strong adjustment of the products under conditions of highly flexible (high volume) production (mass customization) [5]. Its automation technology features self-diagnosis, cognition, self-tuning and self-configuring methods based on the technological concepts of cyber-physical systems, the “Internet of Things” and the “Internet of Services”; it facilitates the vision of the “smart factory” [6,7]. These smart factories of the Industry 4.0 are modular structures and provide cyber-physical systems, which monitor physical processes, create a virtual copy of the physical world and make decentralized decisions in order to become smarter and to better help people in their increasingly complex working routines.
Up to now, robot-based assistive systems are still not widely spread in the manufacturing industry, due to missing research on their full potential and a lack of usability and user acceptance aspects. In this article, we present research performed within the project AssistMe; a two-year project aiming to develop and evaluate novel means of interaction for cooperative robotics from a user-centered perspective. The project consists of three major development cycles. In the first stage, a cooperative robotic system is used “out-of-the-box” and implemented for two different use cases: (1) assembling automotive combustion engines; and (2) polishing continuous casting molds. These implementations were evaluated within three case studies in terms of usability and user acceptance. The goal of these case studies is to identify the major usability and user acceptance problems of off-the-shelf robotic systems within the smart factory context in order to derive suggestions for improvements (i.e., reducing programming complexity, programming duration, system reliability and process quality). These improvements will be implemented and evaluated in the second and third expansion stage later in the project.
The research presented in this article focuses on the first expansion stage and its user-centered approach. In the three case studies, workers cooperated with the robotic prototype in different application contexts, including laboratory conditions (two days), factory conditions (one day) and assembly line conditions (three weeks). By using video analysis, questionnaires and established interview guidelines [8], we ensured a quantitative and phenomenologically-oriented empirical perspective on the usability and user acceptance of an off-the-shelf robotic system implemented for two different application scenarios. Before we go into detail with the three case studies, we will give an overview of relevant related work with respect to assistive robot systems in the industrial context and user-centered development processes in human-robot cooperation.

2. Related Work

Industrial robots are used in the production lines of factories for manufacturing and fast product assembly. The industry is highly dependent on its equipment and, therefore, must be able to adapt to changes to ensure fast and cost-effective production. The following state of the art gives an overview of Human-Robot Interaction (HRI) in today’s industrial environment and the user-centered research and development approaches in the factory context.

2.1. Human-Robot Interaction in the Industrial Environment

Significant core operations in factories of fast product assembly, in addition to various auxiliary functions (e.g., cleaning), are joining, handling, testing and adjusting. The value-adding operations of the primary assembly (joining) can be differentiated from non-value-adding activities of the secondary assembly (handling, testing, adjusting). The work in [9] defines a cognitive assistance system for a visual-instructive assembly guidance based on an environment-dependent statement generation. The work in [10] provides an assistance system for the automation of the secondary assembly process “testing” with a special focus on completeness and correctness. The work in [11] is a major project for the implementation of a software framework for assembling assistive robots with a similar function as in [12,13,14,15] or as addressed in numerous other projects.
However, from the perspective of executing the named primary and secondary operations, manipulation systems play a crucial role in assembling assistance. These systems can be differentiated with respect to their degree of automation [16,17]. Cranes and balancers allow gravity compensation. The work in [18] defines cobots as robots designed for direct physical interaction with the workers while their design is subject to research until today [19]. Cobots are next to the users and can overcome inertia and friction forces, but do not allow pre-programmed (autonomous) movements. Assistive robots work, as opposed to industrial robots, in the same work area as the operator. Therefore, they have safely limited speeds and forces, automatic restart and allow guiding the robot by hand. Robot systems that can be operated without spatial safety areas have gained acceptance in the market since 2010 [20,21]. The work in [22] presents a historical overview of autonomous robot-based manipulation systems and shows their intense research for the last 30 years. The sparse industrial use of autonomous manipulation systems is based on the conservative attitude of the industry (especially with regard to safety and standardization issues), the high system costs, the lack of flexibility, as well as on the low feasibility of basic research results (technology transfer gap, especially in Europe [23]).
Robotic systems used in the industrial context can be programmed via various interfaces and interaction levels [24]. The work in [25] divides programming procedures into automatic (learning, demonstration and instructional-based) and manual (text-based or graphical) methods. “Learning from Demonstration” (LfD) is especially considered to provide an easy and intuitive way to program robot behaviors, potentially reducing development time and costs tremendously. The work in [26] defined criteria to assess the applicability of state of the art LfD frameworks in industry. Furthermore, there is a distinction between online processes (runtime of the robot system, e.g., programming by demonstration [27], manual teach-in) and offline methods (CAD based, etc.). Another categorization can be made on the level of interaction and the interaction medium. The work in [28] gives an overview of physical Human-Robot Interaction (pHRI). The work in [29] outlines the relevant pHRI topics, such as security and standardization [30,31], mechanical and control engineering design, system stability and benchmarks. Current research, for example, is the learning of motor skills by pHRI [32] and the industry-oriented application [33]. Industrial systems allow only the teach-in of discrete individual positions by pHRI and cannot directly imitate smooth movements.
In modern robotics, the prominent input devices for programming are Tangible User Interfaces (TUI). These user interfaces allow the person to interact with the digital representation of the robot through the physical environment. In this setting, the robot has an equivalent in the digital world, and the TUI represents the mechanisms for its interactive control. The spectrum of TUIs ranges from haptic interfaces (often with a force feedback function) in telepresence systems [34] to robotic systems whose surface is equipped with tactile sensors in order to allow the manual control of individual robot segments [35]. The first industrial input devices, which are not available for all robot manufacturers and types, take the orientation of the input device into account in order to derive the orientation and the direction of the tool (but not the position) [36].
To summarize, the programming and usage of assistive robotic systems for collaboration is equivalent to standard industrial robots, when trained robot programmers are the target group. However, programming can be simplified using macros or the possibility of hand guidance during system teach-in for unexperienced users. These aspects are evaluated in the AssistMe project, with an off-the-shelf robotic system. Industrial installations of off-the-shelf collaborative robots can be considered to remain inflexible and to be unintelligent playback machines for movements and process technology, such as intelligent cameras, etc. It remains complicated and almost impossible with commercially available systems to integrate that which renders adaptive behavior. The AssistMe projects wants to enable naive operators to manually teach a robotic arm for their purposes with little pre-knowledge requested. Afterwards, a safe and user-friendly cooperation with the robot in the production process should be possible. The first series of case studies presented in this article therefore aims at identifying the most severe usability and user acceptance aspects that need to be resolved in order to valuably integrate an assistive robotic system into the assembly line.

2.2. User-Centered Development and Evaluation

In the last 20 years, the importance of usability and acceptance has become more important in the field of industrial robotics. Companies, such as KUKA, are investing more and more into User-Centered Development (UCD); usability standards, such as ISO/TR16982:2002 were developed, and user-friendly industrial robots, such as Baxter [37], were brought to the market. Nevertheless, there is yet little user-oriented research concerning assistive robots in the industrial context (e.g., [3,38,39]), including the teach-in of industrial robots and the user-centered development in the industrial development [40]. In addition, the evaluation is another special challenge in and of itself: How can we assess and compare the usability and acceptance of a robotic system for two different application contexts?
It can basically be distinguished between five evaluation methods in human-robot interaction: (1) self-assessment questionnaires, e.g., the Godspeed questionnaire series as a prominent example for a semantic differential especially developed for HRI [41]; (2) interviews [42]; (3) behavior measurements [43]; (4) psychophysiological measurements [44]; and (5) task performance measures [45]. All of these methods have strengths and weaknesses. In order to minimize the weaknesses, a methodological triangulation is often used, and quantitative and qualitative methods are combined. Moreover, the industrial context also includes gender-relevant aspects, and as with most research in human-robot interaction, investigating and exploring these issues relies heavily on the used evaluation methodology and scenario [46].
In order to get a better understanding of the users, their working procedures and the contextual constraints, it is important to actually leave the laboratory and go into the field where the action takes place, in order to know the real context and how the product will be used. The context should be analyzed in the very beginning to know which factors affect the product development, so UCD activities can be better tailored to development phases [13]. To get knowledge of the context, several methods can be applied, e.g., participating in training sessions [23], by means of a contextual inquiry or an ethnographic approach [14].
Harsh environments, such as the cleanroom of a semiconductor factory, have been studied using contextual inquiry [9] to learn about the working routines of maintainers. In harsh environments, the researchers have to make compromises and work-arounds, such as the investigation of the context of a rock-crushing factory where the researchers were confronted with serious problems, like noise and dirt [22]. The actual integration of user-centered methods in the product life cycle can be achieved with different strategies. The process of increasing the understanding of usability should include all stakeholders, and usability goals need to be described in detail for all of them. Furthermore, the understanding of UCD and good design practices can be spread by presenting the results of the usability team on a regular basis. However, the user-centered development methods will only impact the product development life-cycle if the usability professionals speak the same language as the people in the companies. How to increase the awareness of usability matters with industrial partners is described by [40,47]. The latter confronted industrial partners with usability problems via watching a test situation with real users. This approach helped the developers to get a feeling for when the users had troubles interacting with their software.
To summarize, up to now, still, little empirical user-centered research in the industrial context has been performed in order to improve the usability and user acceptance aspects of assistive robotic systems. Studying robots in the assembly lines with actual end users is a challenging endeavor, not only methodology-wise, but also with respect to all of the different stakeholders involved. In the AssistMe project, we follow a user-centered development and evaluation approach, involving all stakeholders (industrial partners, as well as technological partners) in order to flexibly automatize selected production steps in an economically-viable way. Within two years, we plan to iteratively evaluate the same robotic assistance in different stages of expansion for two different use cases. Stage 1 is an off-the-shelf robotic arm from Universal Robotics. Stage 2 will be further enhanced with a 3D sensor and Stage 3 with force feedback. Every stage of expansion will be evaluated together with representative target users from our industrial partners with respect to usability, user experience and acceptance. After every evaluation, the implications for improvement for the next expansion stage will be derived to keep the operators’ point of view in the development process. The AssistMe project thereby follows a very similar user-centered design approach as presented in [40], and the evaluation activities are methodologically grounded in the USUS (Usability, Social Acceptance, User Experience, and Societal Impact) evaluation framework [48].

3. Motivation and Objectives

The motivation and aim of the AssistMe project is the development of innovative haptic and optic concepts for human-robot cooperation in two different application contexts, namely the assembly of automotive combustion engines (see Figure 1), while the other one treats the machining (polishing) of casting molds (see Figure 2). These concepts can be used during set-up (teach-in) and interaction (collaboration in the assembly line) with an assistive robotic system.
The assembly of a combustion engine includes the installation of a cylinder head cover. The installation is carried out manually by stacking the cover with pre-inserted screws onto the motor block and tightening the screws with a manual power tool. The electronic screwdriver of the manual workplace is fitted with a push-start mechanism, electronic control unit and a shut-off clutch and, therefore, starts rotating when pushed onto the screw and stops motion when retracted, respectively, when a predefined torque is reached. The working instruction of the workstation includes several additional process steps. An automatic screw tightening system is expected to provide assistance and to reduce the workload at the workstation for the human worker. A state of the art collaborative robot system [49] is equipped with the power tool (Figure 1) and programed to perform screw tightening operations in the required order and accuracy to meet a defined process quality (screw-in depth, torque, etc.).
The same robotic system was equipped with a polishing tool. Casting molds are crafted from flat material by wire electro discharge machining that leaves eroded surfaces without the required surface finish quality. Manual polishing (Figure 2) by air pressure-driven oscillating polishing machines is extraordinarily labor intensive, non-ergonomic and harmful to health. Prolonged exposure to hand transmitted vibration from powered processes or tools is associated with an increased occurrence of symptoms and signs of disorders in the vascular, neurological and osteoarticular systems of the upper limbs [50]. Setup and programming time are crucial for the use case, since continuous casting molds are usually one of a kind products, manufactured in a lot size of one, with polishing being by far the most labor-intensive production step, causing umpteen hours of labor per mold. Therefore, an assistive system, that is easy to program and setup, is desirable, which can reduce the amount of labor, especially for ergonomic and health reasons.

4. Studies with the Off-The-Shelf System

The three case studies presented in this article were all performed during the first stage of the AssistMe project with the off-the-shelf robotic system from Universal Robots in order to evaluate the industrial feasibility of the state of the art and to create a benchmark in terms of usability and user acceptance measures for comparison studies of the subsequent expansion stages. Our approach is strongly influenced by [40] and prioritized the participation of all partners in all phases of the evaluation (i.e., use case definition, methodology definition, actual evaluation and data interpretation and the deduction of novel interaction paradigms). This close cross-linkage helps us learn from our industrial partners how to conduct studies in their specific environments. Thus, we understood all of the constraints and possibilities coming along with our interdisciplinary cooperation. The whole cooperation with our industrial partners lasted about eight months, consisting of a kick-off meeting (two days), a preparation phase for each use case (one week each), the three case studies (see Table 1), the data analysis (eight weeks) and the final reporting (two weeks).
First use case: The assistive system supports the worker as a third hand, which fastens the screws at taught-in positions on a cylinder head. Real parts were used for the user study. Thereby, the study participants should teach trajectories for the robotic arm.
Second use case: Participants should teach the robot trajectories and positions of the polishing tool. Here, several polishing operations are to simulate with different objectives and the subsequent control and improvement by participants until the robot arm provides satisfactory results. For each use case, accurate movement and positioning, as well as force control were crucial aspects that had to be as close to the requirements as possible.
Third use case: The assistive robot system was deployed in the automotive assembly line for three weeks. There, the worker and the robot collaborated on the task of tightening the screws on the engine block (the operator positioned the screws, and the robot tightened them).

4.1. Research Questions

The focus of our research was on two aspects: (1) identification of major usability problems with the off-the-shelf system for the two application contexts when teaching the robotic system in order two develop new interaction paradigms; (2) collecting suggestions for improvement of the human-robot collaboration in terms of user acceptance and adding a building block to the existing knowledge on the dynamics of user experience in human-robot cooperation in the factory context.
Regarding Aspect 1, we were interested in the following research questions:
(1)
Is there a difference between the two application contexts in terms of the usability of the teach-in (set-up) process?
(2)
Will participants’ attitudes towards robots change due to teaching the robotic system?
(3)
How is the workload of teaching the robot perceived by the users with respect to the application context?
Regarding Aspect 2, we were interested in the following research question:
(1)
How is the collaboration with the robot experienced in the assembly line with respect to user acceptance?
In the following, we will describe the procedure for how we integrated the UCD approach into the implementation of the robotic system for the two application contexts.

4.2. Phase 1: Project Kick-Off Meeting

The first phase consisted of a kick-off meeting with every partner of our consortium in order to introduce each other. The industrial partners explained the specific environmental and technical conditions in order to get a better understanding of the application scenarios of the robot. We introduced ourselves and our role within the project as the scientific HRI-focused partner. The consortium consisted of two industrial partners (providing the testbeds for the use cases), one technological partner (responsible for the implementation of the robot, which was bought from a retailer [49]) and ourselves (responsible for the user-centered evaluation). After clarifying each other’s goals and motivations, we started to prepare how to study the robot in two different application scenarios.

4.3. Phase 2: Preparation of Use Cases and Study Material

In the preparation phase, we defined the three different use cases based on the needs and resources of our industrial partners. After that, the study guidelines and questionnaires were specified, and finally, our industrial partners started the recruitment of participants.
(1)
Use cases: The industrial partner together with the technological partner developed the use cases in order to define how the study participants will interact with the robot within a defined working process. The three use cases were based on the usual working processes and described how the participants had to teach the robot’s trajectories and positions: manually or with the teach pendant. A few weeks before the user studies were conducted, the technological partners visited the industrial partners in order to adapt the robot prototype to the specific process requirements and conditions.
(2)
Study guidelines and questionnaires: The choice of the questionnaires and development of the study guidelines was based on the use cases and research questions. We used established and validated questionnaires (System Usability Scale (SUS) [51], Negative Attitude towards Robot Scale (NARS) [52] and NASA Task Load Index (NASA-TLX) [53]; see Section 4.6 for details) and developed one questionnaire on usability and acceptance questions ourselves. The same study guidelines and questionnaires were used in Use Cases A and B. For the field trial of three weeks (Use Case C), we used an additional interview guideline in order to explore the long-term impressions of the workers in a qualitative manner.
(3)
Recruitment: A week before the each study was conducted, the industrial partners recruited the participants. The participants were all employed at the industrial partners’ factories and were experts in the concerned working processes. The recruitment criteria in terms of age, sex, handedness and prior knowledge of robot control were set to reflect their respective distribution in the factories of our industrial partners. The detailed demographic characteristics of the recruited participants are listed in Table A1 (see the Appendix).

4.5. Phase 3: Performing User Studies

The short-term user studies in which participants should teach the robot specific movements (screw positions vs. polishing positions) took place in an automotive factory (Use Case A) and in a laboratory environment (Use Case B). The first study was conducted within one day; the latter within two days. The studies started by introducing every person (participant, experimenters, technological partner). Next, the participant filled in pre-study questionnaires, and then, the aim of the study and how the robot system can be taught was explained by the technical partner. Afterwards, participants were asked to put on a mobile eye-tracker and to teach the robot the same way as the technical partner demonstrated. In order to understand the thought processes of the participants, that is why they are doing something, they were asked to “think aloud” during the study. During the task, the participant could ask the experimenter and the technological partner for help. After the teaching was completed and the robot performed the learned procedure, the participants had to assess how successful the robot will be in executing the taught movements. In a second run, the participants had to repeat the task, but without the possibility to ask anyone for help. The technological partners and the industrial partners were observing the participants in order to get an impression of the product in real use and to help in emergency situations. The procedure of how we conducted these studies is described in detail in the following:
(1)
Pre-study questionnaires: The questionnaires (demographic questionnaire and NARS [52] questionnaire on attitudes towards robots) were filled in by the participants before any interaction with the robot happened.
(2)
Introduction of the robot: Each participant was introduced to the assistive robot, the control mechanisms and the goal of the trial. In order to relieve stress and increase compliance, the participants were assured that the focus of investigation was only the robot’s performance and there were no negative implications for them.
(3)
Conducting the user study: Each participant was audio- and video-taped with two cameras. Thus, every interaction could be recorded from different distances and angles in order to reproduce the subjective perspective, as well as the context of interaction:
(a)
Videotaping of the first-person view: A head-mounted camera (eye-tracker from Pupil Labs [54]) recorded the participants’ interaction with the robotic arm, the work piece and the touch panel during the teaching procedure. The recording of this point-of-view camera provided additional information concerning the participants’ focus shifts for where he/she was approximately looking.
(b)
Videotaping the whole scenario: One camera was used to get an overview of the scenario for the data interpretation of the eye-tracking recordings and for documenting the think-aloud information.
The use of a first-person and a context-oriented camera helped us with generating a holistic perspective during the later analysis of both use case groups (see Figure 3).
(4)
Post-study questionnaires: The questionnaires (NARS [52], SUS [51], NASA-TLX [53] and a self-developed questionnaire on usability and user acceptance) were filled in by the participants performing the teaching task with the assistive robot.
The field trial took place at the assembly line in an automotive factory (Use Case C) with the participants from Use Case A. The robot system was deployed for three weeks and operated at the same time as the worker in order to cooperatively tighten the screws on an engine block.
(1)
Introduction of the robot at the assembly line: First, the robot was introduced at the assembly line in the industrial partner’s factory. The participants were informed about the aim of the study and how to interact with the robotic system (either manually or by using the touch panel; see Figure 4) during the working process. This field test lasted for three weeks and should help the worker with developing profound opinions concerning the daily cooperation with the robot. Our technological partner was only present in case of a system malfunction.
(2)
Post-field test interview: An interview was conducted after the field test by using established interview guidelines [8] in order to ensure a phenomenological-oriented empirical perspective on the usability and user experience. This approach focuses on: (1) general aspects regarding the work with the robot; (2) experiences before the introduction of the robot; (3) first confrontation with the new system/enrollment; (4) special experiences working with the new system; and (5) special aspects of the HRI (e.g., safety). This open and narrative interview technique evokes personal experiences [55], which were recorded in order to reveal opinions, rumors, impressions, emotional issues, concerns and ideas for improvements.

4.6. Data Analysis

For each participant in Use Cases A and B, the footage of the different views was separately analyzed. For the video analysis of the eye-tracking data, a coding scheme with respect to the interaction modes with the robot (manually or touch panel) was created. The expressed thought processes (think-aloud) were transcribed and grouped content-wise for the final report. The analysis of the questionnaire data followed the established procedures for these validated scales:
  • SUS (System Usability Scale) [51]: a low-cost standardized usability scale. It is used to quickly measure the effectiveness, efficiency and satisfaction of a system and includes 10 statement-based items. Participants can rate them on a five-point Likert scale. After the system has been evaluated, the overall score for the usability of the investigated system is calculated (the scores on individual items are not meaningful). The overall score is between zero and 100 with scores being grouped as follows: “80–100: users like the system”; “60–79: users accept the system”; “0–59: users do not like the system”. Additionally, the SUS can be separated into a usability and a learnability dimension for a further differentiation of the results [56].
  • NARS (Negative Attitude towards Robot Scale) [52]: a psychological scale to measure the negative attitudes of humans against robots. It should become visible which factors are the reasons preventing individuals from interacting with robots. The NARS questionnaire consists of 14 questions, which are grouped into three sub-scales: S1 = negative attitude toward the situations of the interaction with robots; S2 = negative attitude toward the social influence of robots; S3 = negative attitude toward emotions in the interaction with robots.
  • NASA-TLX questionnaire [53]: The NASA-TLX is a subjective workload assessment tool. NASA-TLX allows users to perform subjective workload assessments on operator(s) working with various human-machine systems. NASA-TLX is a multi-dimensional rating procedure that derives an overall workload score based on a weighted average of ratings on six sub-scales. These sub-scales include mental demands, physical demands, temporal demands, own performance, effort and frustration. The sub-scales are rated for each task within a 100-point range with five-point steps.
Furthermore, we developed additional questionnaire items in order to get insights concerning the subjective usability and acceptance of the users (see the Appendix). It features a six-point Likert scale (0–5) and concerns the following aspects:
Performance expectancy is defined as the degree to which an individual believes that using the system will help him or her to attain gains in job performance. Its estimated value is the mean of Item 1 and the inverted Item 6.
Effort expectancy is defined as the degree of ease associated with the use of the system. Its estimated value is the mean of Item 2 and Item 7.
Perceived competence describes the user’s belief that the system has the ability to recognize, understand and react adequately to given tasks with a sufficient set of applicable skills. Its estimated value is the mean of Item 3 and Item 8.
System trust describes the user’s belief that the system performs with the needed functionality to do a required task. Furthermore, the system has to demonstrate being able to effectively provide help when needed (e.g., through a help menu) and to reliably or consistently operate without failing. Its estimated value is the mean of Item 4 and Item 9.
Perceived safety describes to what extent the robotic system is perceived as safe by its users. Its estimated value is the mean of Item 5 and Item 10.
The post-field test interviews (Use Case C) were conducted by the authors of this paper, who are independent of the factory administration and the robot developers. The interviews were recorded and analyzed following an established thematic analysis approach [57]. The aim of the analysis was the structuring and the interpretation of the data and the derivation of findings. In order to avoid predefined categories, we structured the relevant subjective feedbacks and experiences using an exploratory and data-focused method. In the first step, the single feedbacks and experiences were tagged according to their keywords, which made a rough identification of the relevant issues possible. In the second step, the issues were compared with regard to these keywords in order to merge overlapping topics into distinct categories or to clearly differentiate them from each other.

4.7. Final Discussion of the Study Report with the Industrial Partner

A final report was created including the results of the interviews, the questionnaires and the video analysis. This report was presented to all of the consortium partners in order to clarify the strengths and weaknesses of the current prototype and to deliver vital input for a further development iteration.

5. Results

All participants in Case Studies A and B completed both teaching rounds (i.e., with and without the help of the technological partner), except for one individual in use case B, who completed only the first trial. However, every participant was recorded and filled in the post- and pre-trial questionnaires. The demographic background of participants was very similar for Use Cases A and B (see the Appendix, Table A1). The fact that there were more men can be explained by the men-women ratio in the industrial context. Although the number of participants clearly is too small in order to derive statistical significances, we will report trends one can estimate from having a look at the descriptive statistics of Use Cases A and B. For the questionnaire results, we had the following pre-assumptions:
NARS: The negative attitude towards the robot decreases after the human-robot interaction. This expectation is based on other studies [58], which showed such a decrease after short-term human-robot interactions.
NASA-TLX: All scores are less than 50%, which indicates low workload. There are no differences expected between the application contexts.
SUS: The touch panel for teaching the robot is rated lower in terms of usability (due to its off-the-shelf complexity) than the assistive robot itself (manual teaching).
Self-developed items on perceived usability and acceptance: Use Case A should be rated slightly better than Use Case B, as the screwing task could be more robustly automized with the off-the-shelf robot than the polishing task.
The results of the NARS mainly show a decrease of the negative attitude towards the robot in all sub-scales after the human-robot cooperation (see Table 2). The subscales were calculated as accumulated mean values over several items according to [52].
The results mainly go in line with our assumption that the negative attitude should decrease after the interaction. However, this effect is more observable for Use Case A than B. This could be due to the fact that participants of Use Case B overall considered the robot as less useful for their working routines than in Use Case A (according to the think-aloud data).
The differences between the two application contexts (teaching screw driving and polishing movements) were also observable in the results of the NASA-TLX, the SUS and the additional items concerning subjective usability and acceptance (see Table 3, Table 4 and Table 5). For Use Case B, the participants rated the overall performance lower (71.00%) and the task as more frustrating (27.00%) than for Use Case A (see Table 3). In general, as we expected, most of the NASA-TLX scales were below 50%, which indicates the potential of the off-the-shelf robot to be a useful tool for collaboration after some usability improvements.
Furthermore, the manual control of the robot was rated more positively than the touch panel in both use cases as the results of the SUS questionnaire revealed (participants were asked to fill in the SUS twice, once for the robot itself and once only for the teach panel). However, participants in Use Case B rated both control methods less positive than participants in Use Case A (see Table 4).
Similar differences between the application contexts are evident in the additional items concerning subjective impressions. All items were rated slightly higher in Use Case A (see Table 5).
The biggest differences can be observed for the scales safety and perceived competence. This can be interpreted with the additional fact that participants partly blamed the system and partly themselves for the lack of performance in the polishing task.
The gathered qualitative data from the analysis of the think-aloud data from the videos showed that the touch panel in its off-the shelf version is experienced as not feasible and too complex to control the robot for both use cases. A strong tendency was observed in the video to omit the panel as an intermediate device and to directly control the robot using physical force (manually) (see Table 6).
However, the manual control was also limited with respect to feasibility due to the bulky robot arm. Furthermore, the dissatisfaction with the systems can be explained by the fact that the actual teaching was only a fraction of the whole process (see Figure 5).
Finally, the interview data from the field trial in the automotive factory revealed that the robot’s working pace was perceived as not flexible enough, which bears the risk of re-establishing a rigid production line logic (see Table 7). Overall, after three weeks of field-testing (Use Case C), all participants were anxious of being replaced by robots in the near future. More details on the interview data can be found here [59].

6. Interpretation and Recommendation

The three conducted user studies covered laboratory conditions, the industrial context and a field test of three weeks in length. We could learn that the cooperation and the user experience strongly depend on the system’s control modes and degree of adaption. This helps us with differentiating the following aspects for a further development iteration:
(1)
Touch panels: Panels are an intermediate device to control the robot. This implies a continuous switching of visual attention between the robot arm, work piece and touch panel. This implies a high cognitive and attention-related workload for the user. Furthermore, only one third of the time was used for actual teaching, and the rest was used for other activities (e.g., menu navigation). Thus, the application of usability guidelines in this real-time system is absolutely necessary (more intuitive, easier to learn, more shortcuts), and the physical handling of the panel must have high priority (smaller, lighter, more sensitive, without a cord). These insights are in line with Pan et al. [24], who emphasize the importance of teach pendants in the factory context, and Fukui et al. [60], who stated that such input devices should be easy to hold and to operate.
(2)
Manual control: Although the robot arm was described as too bulky to use in practice, there is an overall tendency of preferring manual control over control with an intermediate device. There are various possibilities to integrate manual control, including direct hand pushing, via joystick or via gesture recognition. However, this approach needs an increase of sensors and artificial intelligence in order to live up to its potential. These insights are in line with research concerning teaching motor skills by physical HRI [32] and industry-oriented application [35]. Our technological partner drew similar conclusions concerning the need to improve the manual HRI. Direct interaction should be improved by using more sensors, including cameras for the (1) recording of the whole working place and (2) the detection of potential human-robot movement collisions. The teaching process can be significantly improved by using, e.g., a 3D joystick system.
(3)
Impact on “work”: The field test revealed that the robot prototype was not well perceived with regard to its degree of adaptation towards the users’ individual rhythm, speed and working steps. The robot determined the working pace of the operator, due to its lack of sensors and intelligence. We assume that this essential lack in flexibility is at least partly responsible for the other mentioned shortcomings in usability, perceived safety, system trust, general helpfulness and personal working rhythm of the workers. Furthermore, all workers expressed their fear of being replaced by such a robot in the near future. Although the robot was introduced as a tool and cooperative agent, this fear was not reduced after three weeks of personal human-robot interaction. These empirically-derived insights are in line with Bonekamp and Sure [61], whose review of the current literature on the implications of Industry 4.0 on human labor and work organization revealed a rather consistent view, particularly on job redundancies for low-skilled jobs. However, the factory management discussed the results with the work council, who has a veto concerning further technical and social developments.
The goal of our research was to evaluate an off-the-shelf robotic system in three different use cases for two different application contexts. The results showed us what has to be done in order to develop a robot system that acts as a supportive tool for the workers. However, there is a thin line between a supportive robot and a robot that limits the temporal, physical or social needs of workers, and ultimately replaces them. We believe that the cooperative approach would help with easing the transition of working with a new robot and to increase its overall user experience. Further research and field tests are planned in order to get a deeper understanding and improvement of the mentioned shortcomings. However, firstly, the system will be technically revised to the next expansion stage, including additional sensors for posture recognition in order to measure the persons’ movements and waiting positions, which may improve the factor of perceived safety, as well. More details on this can be found in [62].

7. Conclusions

This article reported three case studies with a cooperative off-the-shelf assistive robot prototype in two application contexts: (1) the assembly of automotive combustion engines; and (2) the machining (polishing) of continuous casting molds. We followed a UCD approach, in which we studied how workers, recruited from our industrial partners’ factories, performed the teach-in (set-up) and collaboration with the robot w.r.t. usability and acceptance aspects. The operators worked with the robotic prototype in laboratory conditions (two days), in a factory context (one day) and in an automotive assembly (three weeks). We used a multi-dimensional qualitative and quantitative methodology in order to get insights into how the human-robot collaboration was experienced. The results of the case studies show the urgency of adaptation of assistive robot systems in order to avoid a change of working routine for the workers. This importance of flexibility is also reflected in the ratings of the teach pendant (a touch panel needed to control the robot), which should be completely replaced through direct physical control. Furthermore, all workers were afraid of being replaced by this or similar robotic systems in the near future. This anthropocentric dimension of human-robot cooperation was also revealed in related research [38], which also suggests that hybrid robots (combining a social- and tool-like appearance and behavioral elements) will be potentially preferred by naive users in the industrial context in the future. Our research so far could not show that a more socially-designed robot would ease collaboration, but that a tool-like robot, which still allows human flexibility and socializing with co-workers, due to high usability and intuitiveness, will be more accepted in the long run.
Our industrial partners drew similar conclusions concerning the need to improve the human-robot collaboration, preferring intuitive and manual solutions, which no longer require an additional touch panel. The precision, safety and adaptation can be improved by the employment of further sensors. The discussion and decision loop also includes the work council, who has a veto concerning further technical and social developments.
The importance of the insights gained through the presented research can be considered as high due to the empirical basis of the approach used (three weeks of field testing; real assembly line; representative workers). However, the limitation of the presented research is the small number of participants (three use cases with five participants each). However, five participants for each use case should in this specific project not be considered as a small sample size, as in the factories of our industrial partners, only these five recruited workers are actually collaborating with the robot. Moreover, the user studies for Use Cases A and B do not aim at quantifying acceptance parameters, but at identifying major usability problems and, by that, to develop new interaction concepts. A small participant number of five can be sufficient to identify the most severe usability problems, as was already discussed by [63]. However, the variety of use cases and the multi-dimensional approach used helped us to get a better understanding of the technological and anthropocentric challenges in Industry 4.0 factories where humans and robots will work more and more closely together.

Acknowledgments

This work was funded by “Produktion der Zukunft”, a program of the Federal Ministry of Science and Research of Austria (848653-AssistMe).

Author Contributions

Astrid Weiss and Andreas Huber equally contributed to the article. They performed the UCD research together and wrote the major part of the manuscript. Jürgen Minichberger and Markus Ikeda implemented the robotic solutions for the two testbeds and contributed to the Related Work section and the description of the use cases. All authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix

Table A1. Demographic characteristics of the participants in each study (SD = standard deviation).
Table A1. Demographic characteristics of the participants in each study (SD = standard deviation).
Use Cases A and C (Screw Tightening)Use Case B (Surface Polishing)
(n = 5)(n = 5)
Age (years)
Mean (SD)45.2 (11.3)45.4 (5.7)
Gender
Male (n (%))3 (60.0%)5 (100.0%)
Female (n (%))2 (40.0%)0 (0.0%)
Handedness
Left (n (%))0 (0.0%)1 (20.0%)
Right (n (%))4 (80.0%)4 (80.0%)
Both (n (%))1 (80.0%)0 (0.0%)
Color Blindness
Green-Red or Blue-Yellow (n (%))0 (0.0%)0 (0.0%)
Spatial Imagery
Below-average (n (%))0 (0.0%)0 (0.0%)
Average (n (%))4 (80.0%)5 (100.0%)
Above-average (n (%))1 (80.0%)0 (0.0%)
Excellent (n (%))0 (0.0%)0 (0.0%)
Computer Skills
No experience (n (%))1 (20.0%)1 (20.0%)
Beginner (n (%))0 (0.0%)3 (60.0%)
Experienced (n (%))3 (60.0%)1 (20.0%)
Expert (n (%))1 (20.0%)0 (0.0%)
Computer Usage per Day
None (n (%))1 (20.0%)1 (20.0%)
Less than 1 h (n (%))0 (0.0%)2 (40.0%)
1–3 h (n (%))2 (40.0%)1 (20.0%)
3–6 h (n (%))0 (0.0%)1 (20.0%)
More than 6 h (n (%))2 (40.0%)0 (0.0%)
Usually-used OS
Windows (n (%))5 (100.0%)4 (0.0%)
Unix/Linux (n (%))0 (0.0%)0 (0.0%)
Mac (n (%))1 (20.0%)0 (0.0%)
Used Applications
Word Processors (n (%))4 (80.0%)3 (60.0%)
Spreadsheet Applications (n (%))2 (40.0%)1 (20.0%)
E-Mail Applications (n (%))5 (100.0%)3 (60.0%)
Graphics Software (n (%))1 (20.0%)0 (0.0%)
Music Applications (n (%))1 (20.0%)0 (0.0%)
Desktop-Publishing Software (n (%))0 (0.0%)1 (20.0%)
Industrial Machine Skills
No experience (n (%))0 (0.0%)0 (0.0%)
Beginner (n (%))2 (40.0%)4 (80.0%)
Experienced (n (%))2 (40.0%)0 (0.0%)
Expert (n (%))1 (20.0%)1 (20.0%)
Industrial Robot Skills
No experience (n (%))1 (20.0%)3 (60.0%)
Beginner (n (%))3 (60.0%)2 (40.0%)
Experienced (n (%))1 (20.0%)0 (0.0%)
Expert (n (%))0 (0.0%)0 (0.0%)
Industrial Robot Experience
None (n (%))3 (60.0%)5 (100.0%)
Less than 3 months (n (%))0 (0.0%)0 (0.0%)
3–6 months (n (%))0 (0.0%)0 (0.0%)
6–12 months (n (%))1 (0.0%)0 (0.0%)
1–3 years (n (%))0 (0.0%)0 (0.0%)
More than 3 years (n (%))1 (20.0%)0 (0.0%)
Table A2. Subjective acceptance questionnaire. Trust-scale (Item 1 and inverted Item 6), safety (Items 2 and 7), perceived competence (Items 3 and 8), performance expectancy (Items 4 and 9) and effort expectancy (Items 5 and 10).
Table A2. Subjective acceptance questionnaire. Trust-scale (Item 1 and inverted Item 6), safety (Items 2 and 7), perceived competence (Items 3 and 8), performance expectancy (Items 4 and 9) and effort expectancy (Items 5 and 10).
1. I can trust the robot.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
2. The robot seems harmless to me.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
3. I feel capable of handling the robot to perform tasks.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
4. The use of the robot increases my professional effectiveness.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
5. It is easy for me to make the robot do what I want to accomplish.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
6. The robot gives me the creeps.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
7. The robot seems very reliable.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
8. I can operate the robot good enough to carry out work.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
9. I can fulfill my tasks more efficiently with the help of the robot.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree
10. It is easy for me to operate the robot.
OOOOOO
Strongly disagreeDisagreeRather disagreeRather agreeAgreeStrongly agree

References

  1. Young, M. The Technical Writer’s Handbook; University Science Books: Mill Valley, CA, USA, 1989. [Google Scholar]
  2. Alben, L. Quality of experience: Defining the criteria for effective interaction design. Interactions 1996, 3, 11–15. [Google Scholar] [CrossRef]
  3. Buchner, R.; Wurhofer, D.; Weiss, A.; Tscheligi, M. Robots in time: How user experience in human-robot interaction changes over time. In International Conference on Social Robotics; Herrmann, G., Pearson, M.J., Lenz, A., Bremner, P., Spiers, A., Leonards, U., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; Volume 8239, pp. 138–147. [Google Scholar]
  4. Obrist, M.; Reitberger, W.; Wurhofer, D.; Förster, F.; Tscheligi, M. User experience research in the semiconductor factory: A contradiction? In Human-Computer Interaction—INTERACT 2011; Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2011; Volume 6949, pp. 144–151. [Google Scholar]
  5. Hermann, M.; Pentek, T.; Otto, B. Design Principles for Industrie 4.0 Scenarios. 2015. Available online: http://www.snom.mb.tu-dortmund.de/cms/de/forschung/Arbeitsberichte/Design-Principles-for-Industrie-4_0-Scenarios.pdf (accessed on 25 February 2016).
  6. Jasperneite, J.; Niggemann, O. Systemkomplexität in der Automation beherrschen. Intelligente Aisstenzsysteme unterstützen den Menschen. Available online: http://www.ciit-owl.de/uploads/media/Jasperneite_Niggemann_edi_09_S.36-44.pdf (accessed on 19 July 2016).
  7. Kagermann, H.; Wahlster, W.; Helbig, J. Recommendations for Implementing the Strategic Initiative Industrie 4.0. Final report of the Industrie 4.0 Working Group. Available online: http://www.acatech.de/fileadmin/user_upload/Baumstruktur_nach_Website/Acatech/root/de/Material_fuer_Sonderseiten/Industrie_4.0/Final_report__Industrie_4.0_accessible.pdf, 2013 (accessed on 13 July 2016).
  8. Meneweger, T.; Wurhofer, D.; Fuchsberger, V.; Tscheligi, M. Erlebniszentrierte Interviews zur Erfassung von User Experience Mit Industrierobotern. In Design 2 Product. Beiträge zur empirischen Designforschung, Band 5; Bucher: Hohenems, Austria, 2016. (In German) [Google Scholar]
  9. Zäh, M.F.; Wiesbeck, M.; Engstler, F.; Friesdorf, F.; Schubö, A.; Stork, S.; Bannat, A.; Wallhoff, F. Kognitive Assistenzsysteme in der manuellen Montage. Werkstattstech. Online 2007, 97, 644–650. [Google Scholar]
  10. Rooker, M.; Hofmann, M.; Minichberger, J.; Ikeda, M.; Ebenhofer, G.; Fritz, G.; Pichler, A. Flexible and assistive quality checks with industrial robots. In Proceedings of the ISR/Robotik 2014, the 41st International Symposium on Robotics, Munich, Germany, 2–3 June 2014; pp. 1–6.
  11. LIAA Project. Available online: http://www.project-leanautomation.eu/ (accessed on 25 February 2016).
  12. Plasch, M.; Pichler, A.; Bauer, H.; Rooker, M.; Ebenhofer, G. A plug & produce approach to design robot assistants in a sustainable manufacturing environment. In Proceedings of the 22nd International Conference on Flexible Automation and Intelligent Manufacturing (FAIM 2012), Helsinki, Finland, 10–13 June 2012.
  13. Hvilshøj, M.; Bøgh, S. “Little Helper”—An autonomous industrial mobile manipulator concept. Int. J. Adv. Robot. Syst. 2011, 8, 80–90. [Google Scholar] [CrossRef]
  14. Helms, E.; Schraft, R.D.; Hagele, M. rob@work: Robot assistant in industrial environments. In Proceedings of the 11th IEEE International Workshop on Robot and Human Interactive Communication, Berlin, Germany, 25–27 September 2002; pp. 399–404.
  15. Wögerer, C.; Bauer, H.; Rooker, M.; Ebenhofer, G.; Rovetta, A.; Robertson, N.; Pichler, A. LOCOBOT-low cost toolkit for building robot co-workers in assembly lines. In Intelligent Robotics and Applications; Springer: Berlin/Heidelberg, Germany, 2012; pp. 449–459. [Google Scholar]
  16. Hägele, M.; Neugebauer, J.; Schraft, R.D. From robots to robot assistants. In Proceedings of the 32nd ISR (International Symposium on Robotics), Seoul, Korea, 19–21 April 2001.
  17. Feldmann, K.; Schöppner, V. Handbuch Fügen, Handhaben und Montieren; Carl Hanser Verlag: München, Germany, 2013; p. 231. [Google Scholar]
  18. Peshkin, M.A.; Colgate, J.E.; Wannasuphoprasit, W.; Moore, C.A.; Gillespie, R.B.; Akella, P. Cobot architecture. IEEE Trans. Robot. Autom. 2001, 17, 377–390. [Google Scholar] [CrossRef]
  19. Gambao, E.; Hernando, M.; Surdilovic, D. A new generation of collaborative robots for material handling. Gerontechnology 2012, 11, 368. [Google Scholar] [CrossRef]
  20. MRK-Systeme. Available online: http://www.mrk-systeme.de/index.html (accessed on 25 February 2016).
  21. KUKA Laboratories. Available online: http://www.kuka-healthcare.com/de/lightweight_robotics/ (accessed on 25 February 2016).
  22. Bøgh, S.; Hvilshøj, M.; Kristiansen, M.; Madsen, O. Autonomous industrial mobile manipulation (AIMM): From research to industry. In Proceedings of the 42nd International Symposium on Robotics, Chicago, IL, USA, 21–24 March 2011.
  23. Griffiths, S.; Voss, L.; Röhrbein, F. Industry-academia collaborations in robotics: Comparing Asia, Europe and North-America. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation, Hongkong, China, 31 May–7 June 2014.
  24. Pan, Z.; Polden, J.; Larkin, N.; van Duin, S.; Norrish, J. Recent progress on programming methods for industrial robots. Robot. Comput. Integr. Manuf. 2012, 28, 87–94. [Google Scholar] [CrossRef]
  25. Biggs, G.; Macdonald, B. A survey of robot programming systems. In Proceedings of the Australasian Conference on Robotics and Automation, Brisbane, Australia, 1–3 December 2003; pp. 1–3.
  26. Ko, W.K.H.; Wu, Y.; Tee, K.P.; Buchli, J. Towards industrial robot learning from demonstration. In Proceedings of the 3rd International Conference on Human-Agent Interaction (HAI’15), New York, NY, USA, 21–24 October 2015; pp. 235–238.
  27. Pires, J.N.; Veiga, G.; Araújo, R. Programming-by-demonstration in the coworker scenario for SMEs. Ind. Robot Int. J. 2009, 36, 73–83. [Google Scholar] [CrossRef]
  28. Argall, B.D.; Billard, A.G. A survey of tactile human–robot interactions. Robot. Auton. Syst. 2010, 58, 1159–1176. [Google Scholar] [CrossRef]
  29. De Santis, A.; Siciliano, B.; De Luca, A.; Bicchi, A. An atlas of physical human–robot interaction. Mech. Mach. Theory 2008, 43, 253–270. [Google Scholar] [CrossRef]
  30. ISO 10218-1:2011 (Robots). Available online: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=51330 (accessed on 25 February 2016).
  31. ISO 10218-1:2011 (Robot Systems and Integration). Available online: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=41571 (accessed on 25 February 2016).
  32. Ikemoto, S.; Amor, H.B.; Minato, T.; Jung, B.; Ishiguro, H. Physical human-robot interaction: Mutual learning and adaptation. IEEE Robot. Autom. Mag. 2012, 19, 24–35. [Google Scholar] [CrossRef]
  33. Meyer, C. Aufnahme und Nachbearbeitung von Bahnen bei der Programmierung Durch Vormachen von Industrierobotern. Ph.D. Thesis, Universität Stuttgart, Stuttgart, Germany, 2011. [Google Scholar]
  34. Geomagic Touch. Available online: http://geomagic.com/en/products/phantom-omni/overview (accessed on 25 February 2016).
  35. Fritzsche, M.; Elkmann, N.; Schulenburg, E. Tactile sensing: A key technology for safe physical human robot interaction. In Proceedings of the 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Lausanne, Switzerland, 8–11 March 2011; pp. 139–140.
  36. Intuitive an Mobile Robot Teach-In. Available online: http://www.keba.com/en/industrial-automation/ketop-operation-monitoring/mobile-operating-panels/ketop-t10-directmove/ (accessed on 25 February 2016).
  37. Baxter. Available online: http://www.rethinkrobotics.com/products/baxter (accessed on 25 February 2016).
  38. Stadler, S.; Weiss, A.; Mirnig, N.; Tscheligi, M. Antropomorphism in the factory: A paradigm change? In Proceedings of the 8th ACM/IEEE International conference on Human-Robot interaction, Tokyo, Janpan, 3–6 March 2013; pp. 231–232.
  39. Weiss, A.; Buchner, R.; Fischer, H.; Tscheligi, M. Exploring human-robot cooperation possibilities for semiconductor manufacturing. In Proceedings of the 2011 International Conference on Collaboration Technologies and Systems, Philadelphia, PA, USA, 23–27 May 2011.
  40. Buchner, R.; Mirnig, N.; Weiss, A.; Tscheligi, M. Evaluating in real life robotic environment—Bringing together research and practice. In Proceedings of the RO-MAN 2012, 21st IEEE International Symposium on Robot and Human Interactive Communication, Paris, France, 9–13 September 2012.
  41. Bartneck, C.; Kulić, D.; Croft, E.; Zoghbi, S. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int. J. Soc. Robot. 2009, 1, 71–81. [Google Scholar] [CrossRef]
  42. Weiss, A.; Igelsböck, J.; Wurhofer, D.; Tscheligi, M. Looking forward to a Robotic Society“?—Imaginations of future human-robot relationships. Special issue on the human robot personal relationship conference. Int. J. Soc. Robot. 2011, 3, 111–123. [Google Scholar] [CrossRef]
  43. Lohse, M.; Hanheide, M.; Rohlfing, K.; Sagerer, G. Systematic interaction analysis (SINA) in HRI. In Proceedings of the 4th ACM/IEEE International conference on Human Robot Interaction, La Jolla, CA, USA, 11–19 March 2009; pp. 93–100.
  44. Strasser, E.; Weiss, A.; Tscheligi, M. Affect misattribution procedure: An implicit technique to measure user experience in HRI. In Proceedings of the 7th ACM/IEEE international conference on Human-robot interaction, New York, NY, USA, 5–8 March 2012.
  45. Stadler, S.; Weiss, A.; Tscheligi, M. I Trained this Robot: The Impact of Pre-Experience and Execution Behavior on Robot Teachers. In Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK, 25–29 August 2014.
  46. Seo, S.H.; Gu, J.; Jeong, S.; Griffin, K.; Young, J.E.; Bunt, A.; Prentice, S. Women and men collaborating with robots on assembly lines: Designing a novel evaluation scenario for collocated human-robot teamwork. In Proceedings of the 3rd International Conference on Human-Agent Interaction (HAI’15), New York, NY, USA, 21–24 October 2015; pp. 3–9.
  47. Bernhaupt, R.; Weiss, A. How to convince stakeholders to take up usability evaluation results? In COST294-MAUSE Workshop Downstream Utility: The Good, the Bad and the Utterly Useless Usability Feedback; Institute of Research in Informatics of Toulouse: Toulouse, France, 2007; pp. 37–39. [Google Scholar]
  48. Weiss, A.; Bernhaupt, R.; Lankes, M.; Tscheligi, M. The USUS evaluation framework for Human-Robot Interaction. In Proceedings of the Symposium on New Frontiers in Human-Robot Interaction, Edinburgh, UK, 6–9 April 2009; pp. 158–165.
  49. Universal Robots. Available online: https://en.wikipedia.org/wiki/Universal_Robots (accessed on 13 July 2016).
  50. Bovenzi, M. Health effects of mechanical vibration. G. Ital. Med. Lav. Ergon. 2005, 27, 58–64. [Google Scholar] [PubMed]
  51. Brooke, J. SUS-A quick and dirty usability scale. Usability Eval. Ind. 1996, 189, 4–7. [Google Scholar]
  52. Nomura, T.; Suzuki, T.; Kanda, T. Measurement of negative attitudes toward robots. Interact. Stud. 2006, 7, 437–454. [Google Scholar] [CrossRef]
  53. Hart, S.G.; Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Adv. Psychol. 1988, 52, 139–183. [Google Scholar]
  54. Pupil Labs. Available online: https://pupil-labs.com/ (accessed on 13 July 2016).
  55. Flick, U. Episodic interviewing. In Qualitative Researching with Text, Image and Sound; Bauer, M.W., Gaskell, G., Eds.; Sage: London, UK, 2000; pp. 75–92. [Google Scholar]
  56. Lewis, J.R.; Sauro, J. The factor structure of the system usability scale. In Proceedings of the International Conference HCII 2009, San Diego, CA, USA, 19–24 July 2009.
  57. Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef]
  58. Weiss, A. Validation of an Evaluation Framework for Human-Robot Interaction. The Impact of Usability, Social Acceptance, User Experience, and Societal Impact on Collaboration with Humanoid Robots. Ph.D. Thesis, University Salzburg, Salzburg, Austria, 2010. [Google Scholar]
  59. Weiss, A.; Huber, A. User experience of a smart factory robot: Assembly line workers demand adaptive robots. 2016; arXiv:1606.03846. [Google Scholar]
  60. Fukui, H.; Yonejima, S.; Yamano, M.; Dohi, M.; Yamada, M.; Nishiki, T. Development of teaching pendant optimized for robot application. In Proceedings of the 2009 IEEE Workshop on Advanced Robotics and its Social Impacts, Tokyo, Japan, 23–25 November 2009; pp. 72–77.
  61. Bonekamp, L.; Sure, M. Consequences of Industry 4.0 on human labour and work organisation. J. Bus. Media Psychol. 2015, 6, 33–40. [Google Scholar]
  62. Ebenhofer, G.; Ikeda, M.; Huber, A.; Weiss, A. User-centered assistive robotics for production—The AssistMe Project. In Proceedings of the OAGM-ARW2016: Joint Workshop on “Computer Vision and Robotics, Wels, Austria, 11–13 May 2016.
  63. Why You Only Need to Test with 5 Users. Available online: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ (accessed on 13 July 2016).
Figure 1. Collaborative screwdriver robot system.
Figure 1. Collaborative screwdriver robot system.
Societies 06 00020 g001
Figure 2. Manual polishing workplace.
Figure 2. Manual polishing workplace.
Societies 06 00020 g002
Figure 3. Schematic representation of the use case set-up. (i) In Use Case A, the teacher had to program way points for screwing for the robot; (ii) in the Use Case B study, the teacher had to program way points for the polishing for the robot; (iii) in Use Case C, the participants worked together with the robot, which was deployed in the automotive assembly line for three weeks.
Figure 3. Schematic representation of the use case set-up. (i) In Use Case A, the teacher had to program way points for screwing for the robot; (ii) in the Use Case B study, the teacher had to program way points for the polishing for the robot; (iii) in Use Case C, the participants worked together with the robot, which was deployed in the automotive assembly line for three weeks.
Societies 06 00020 g003
Figure 4. The Universal Robots arm, which can be manually moved to teach-in positions or with the depicted touch panel (teach pendant).
Figure 4. The Universal Robots arm, which can be manually moved to teach-in positions or with the depicted touch panel (teach pendant).
Societies 06 00020 g004
Figure 5. During the second teaching process in both short-term studies: (a) the percentage of the duration of the manual and the touch panel mode; (b) the percentage of the duration of the actual teaching and other activities (e.g., navigating the menu architecture, administration of already taught positions).
Figure 5. During the second teaching process in both short-term studies: (a) the percentage of the duration of the manual and the touch panel mode; (b) the percentage of the duration of the actual teaching and other activities (e.g., navigating the menu architecture, administration of already taught positions).
Societies 06 00020 g005
Table 1. Overview of the three use cases.
Table 1. Overview of the three use cases.
Use Case AUse Case BUse Case C
DescriptionTeaching of screw positionsTeaching of polishing positionsCooperative screwing
EnvironmentFactoryLaboratoryFactory (assembly line)
Number of Participants555
Duration1 day2 days3 weeks
Research InstrumentsObservation
Questionnaires
Observation
Questionnaires
Interviews
Table 2. Comparison of the Negative Attitude towards Robot Scale (NARS) rating before and after the human-robot interaction before and after each study (0 = “low”, 4 = “high”).
Table 2. Comparison of the Negative Attitude towards Robot Scale (NARS) rating before and after the human-robot interaction before and after each study (0 = “low”, 4 = “high”).
Sub-ScaleNARS Sub-Scale DescriptionUse Case A (n = 5)Use Case B (n = 5)
Pre (SD)Post (SD)Pre (SD)Post (SD)
S1Negative attitude towards situations and interactions with robots.1.27 (0.52)0.75 (0.52)1.40 (0.51)1.23 (0.46)
S2Negative attitude towards social influence of robots.1.92 (0.54)1.40 (0.96)2.08 (0.46)2.08 (0.46)
S3Negative attitude towards emotions in interaction with robots.2.87 (0.42)1.50 (0.50)2.33 (0.99)2.33 (0.31)
Table 3. Comparison of the NASA-TLX rating before and after the human-robot interaction for both use cases. Each NASA-TLX scale has 21 grades, and the estimates are converted into percentages to facilitate comparison (0% = “very low”, 100% = “very high”).
Table 3. Comparison of the NASA-TLX rating before and after the human-robot interaction for both use cases. Each NASA-TLX scale has 21 grades, and the estimates are converted into percentages to facilitate comparison (0% = “very low”, 100% = “very high”).
NASA-TLX Item DescriptionUse Case A (n = 5)Use Case B (n = 5)
Mental Demand: How mentally demanding was the task?51.25%48.00%
Physical Demand: How physically demanding was the task?37.50%20.00%
Temporal Demand: How hurried or rushed was the pace of the task?20.63%59.00%
Performance: How successful were you in accomplishing what you were asked to do?86.88%71.00%
Effort: How hard did you have to work to accomplish your level of performance?53.13%38.00%
Frustration: How insecure, discouraged, irritated, stressed and annoyed were you?9.38%27.00%
Table 4. Comparison of the System Usability Scale (SUS) ratings for the control panel and manual robot teaching (0 = “low”, 4 = “high”). Subscales according to [56].
Table 4. Comparison of the System Usability Scale (SUS) ratings for the control panel and manual robot teaching (0 = “low”, 4 = “high”). Subscales according to [56].
Sub-ScaleSUS Sub-Scale DescriptionUse Case A (n = 5)Use Case B (n = 5)
PanelRobotPanelRobot
S1Usability (item mean)2.693.152.002.38
S2Learnability (item mean)3.383.631.702.20
Total score (0%–100%)70.73%81.15%48.50%58.50%
Score groupUsers accept the systemUsers like the systemUsers do not like the systemUsers do not like the system
Table 5. Comparison of additional items related to subjective acceptance after each study (0 = “very low”, 5 = “very high”).
Table 5. Comparison of additional items related to subjective acceptance after each study (0 = “very low”, 5 = “very high”).
Sub-ScaleSubjective Acceptance Sub-Scale DescriptionUse Case A (SD) (n = 5)Use Case B (SD) (n = 5)
S1System Trust (mean of Item 1 and inverted 6)3.75 (0.71)2.90 (0.42)
S2Safety (mean of Items 2 and 7)4.13 (0.18)3.00 (0.00)
S3Perceived Competence of the System Handling (mean of Items 3 and 8)4.38 (0.18)3.30 (0.71)
S4Performance Expectancy (mean of Items 4 and 9)3.00 (0.00)2.40 (0.00)
S5Effort Expectancy (mean of Items 5 and 10)4.13 (0.18)3.30 (0.71)
Total score (0%–100%)79.50%61.60%
Table 6. Frequency of user behavior and stated feedback during the short-time user studies. The feedback categories are derived by clustering the video-taped results.
Table 6. Frequency of user behavior and stated feedback during the short-time user studies. The feedback categories are derived by clustering the video-taped results.
Short-Time Feedback CategoryUse Case A (n = 5)Use Case B (n = 5)Total (n = 10)
Manual robot guidance is very bulky, especially with one hand.5510
The menu architecture of the touch panel is overwhelmingly complex.5510
Some menu buttons are too small to touch accurately.448
Tap/swipe/scroll on the touch panel sometimes does not work.437
Manual robot guidance is preferred (hand-pushed or via joystick).325
The connecting cable of the touch panel is very bulky.213
The angle of the robot arm/tool should be controllable (also for manual guidance).123
The high number of coordinate axes makes controlling cognitively demanding.202
Robot/screwdriver should be prepared so that shorter people can better handle it.202
The teaching process should be linear (straightforward) featuring more feedback.112
The robot arm should have more sensors to detect edges and obstacles.022
The whole teaching process takes too long.022
For smaller persons, the visual switch between panel and robot arm is difficult.101
The automatic processing of surfaces should be configurable.011
Table 7. Frequency of statements after three weeks of field testing in the assembly line. The feedback categories are derived by clustering the interview results.
Table 7. Frequency of statements after three weeks of field testing in the assembly line. The feedback categories are derived by clustering the interview results.
Long-Term Feedback CategoryUse Case C (n = 5)
The robot will replace my job in the near future.5
The robot determines my work and speed.4
The robot should be more adaptive.4
The robot does not stop at physical contact.4
To work with the robot was a big transition.3
The robot is a helpful tool.2
The robot should have more anthropomorphic features.2

Share and Cite

MDPI and ACS Style

Weiss, A.; Huber, A.; Minichberger, J.; Ikeda, M. First Application of Robot Teaching in an Existing Industry 4.0 Environment: Does It Really Work? Societies 2016, 6, 20. https://doi.org/10.3390/soc6030020

AMA Style

Weiss A, Huber A, Minichberger J, Ikeda M. First Application of Robot Teaching in an Existing Industry 4.0 Environment: Does It Really Work? Societies. 2016; 6(3):20. https://doi.org/10.3390/soc6030020

Chicago/Turabian Style

Weiss, Astrid, Andreas Huber, Jürgen Minichberger, and Markus Ikeda. 2016. "First Application of Robot Teaching in an Existing Industry 4.0 Environment: Does It Really Work?" Societies 6, no. 3: 20. https://doi.org/10.3390/soc6030020

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop