Concept and Evaluation of a Method for the Integration of Human Factors into Human-Oriented Work Design in Cyber-Physical Production Systems

Abstract

Due to the shift from mainly manual labor to an increased portion of cognitive tasks in manufacturing caused by the introduction of cyber-physical systems, there is a need for an updated collection of adequate design principles for user interfaces between humans and machines. Thus, we developed a method for the determination and evaluation of such design principles. It is based on human factors methods and facilitates the assessment of specific work design elements which are supposed to have a significant effect on work performance and the perception of work in cyber-physical production systems (CPPS). Within the application of the developed method, we derived an overview of key design elements in CPPS, developed an experimental platform, and conducted two empirical studies with a total of n = 68 participants. This way, three design elements were investigated, and the findings transferred into preliminary design principles. We can state that the method can be used both for a better understanding of the mechanisms between human factors and work in CPPS. Besides, it helps to provide a catalogue of design principles applicable to SMEs to promote more efficient and successful integration of workers into CPPS.

1. Introduction

Cyber-physical systems (CPS) describe a new type of technological systems in which physical elements are equipped with computers (embedded systems). These are networked with each other and can exchange information. Thus, real objects and processes are associated with virtual objects and processes [1]. This results in an increasing intelligence of products and systems [2]. These smart products are characterized by the fact that they are equipped with computers for storing product-specific information which can be used for data exchange and localization. The intelligent products can thus influence their own manufacture, use, or disposal [3].

In the field of manufacturing environments, intelligent systems include intelligent machine tools, test equipment, transport vehicles, and products. They continuously collect data about their condition and share it with other cyber-physical systems. The analysis and use of these data [1] reveal great potential for process improvements [4,5,6]. An example is the Smart Production, whose functionalities are made possible as a result of data exchange [7,8]. In detail, there are various applications for cyber-physical systems in production. For example, autonomous cooperating production processes, use of a collaborative robot, preventive maintenance or virtual reality/augmented reality systems can be used and implemented [3,5,6]. Many of these ideas are not new and therefore not entirely attributable to Industry 4.0, but can now unfold their potential as a result of the greater amount of data available [6]. The reorganization of manufacturing as a result of automation or changes in work systems will have an impact on existing jobs. It can be assumed that people in the Industry 4.0 environment both perform different activities and require different qualifications for these tasks [5].

How should the Work 4.0 associated with Industry 4.0 be designed in a both socially and economically sustainable way? This question is of main interest in the present paper. Section 1 provides background information and existing fundamentals on the change of work systems, human-oriented work design, human–machine interaction, and human factors both in general and in cyber-physical production systems (CPPS). In Section 2, we outline the underlying method of carrying out a human factors study and analyze available human-centered research in CPPS to determine their level of integrating human factors principles and methods. Section 3 is separated into four subsections and presents the results of our research. Firstly, work design elements for Human–machine interfaces in CPPS are proposed (Section 3.1). Further, a method for the integration of human factors into human-oriented work design in CPPS is shown. It facilitates the assessment of the proposed work design elements and quantifies their effect on work performance and the perception of work in CPPS and thus leads to design principles (Section 3.2). Then, an experimental platform designed to carry out these CPPS-related human factors studies easily is presented (Section 3.3). Finally, we apply and validate the developed method. This includes the hypothesis of key design elements out of the proposed work design elements, the design and implementation of representative work tasks as a human factors study, the execution, and evaluation of the human factors study, and the final development of design principles for interfaces in CPPS (Section 3.4). Conclusively, in Section 4, the followed approach is summarized and interpreted. We also point out subsequent future research ideas.

1.1. Change of Human Work in Manufacturing

Work in future manufacturing systems will be transformed by cyber-physical systems due to the technical and organizational shift described above. Despite the increased computerization and automation, studies say that human work is considered to remain very important for the future [9]. As a result, these studies expect the development of hybrid production systems, which involve collaborative work between people and machines. The specific application of the physical and mental abilities of humans can generate further added value [9,10]. Hirsch-Kreinsen [10] described the design of the human–machine interface as a central challenge. Only if the operators, which are part of the CPPS, can control and comprehensively understand them, decisions can be made that are consistent with the automated system (cf. Ironies of Automation [11]).

Further, CPPS lead to rationalization effects through digitization, networking, and automation [12]. Therefore, experts expect less simple, repetitive work and more qualified work in relation with the cyber-physical systems [9]. As a result, the type of work that will be required in the future is likely to be different. According to a study by the Prognos Institute, increasing demand for highly qualified workers and decreasing demand for low-skilled work and auxiliary activities can be assumed [13]. This effect is typical for technical and organizational change processes [12]. Additionally, Brossardt [14] highlighted changes in particular task areas: production-related occupations are declining. Here, technological development has an impact on the form of rationalization and automation. An increase can be observed for the knowledge-oriented field of occupation. This reflects the tendency towards a knowledge society with a lower need for manual but a higher need for knowledge-based activities [14].

1.2. Anthropocentric Work Design in CPPS

Work organization deals with actions that contribute to changing existing work systems or creating new ones. These actions include the design of work organization, links to other work systems, work tasks, equipment, workplaces, or working environment. They can be distinguished by different perspectives:

• From an economic or engineering perspective, the work design aims at improving the performance and efficiency of the value creation process.
• From a human factors perspective, work design deals with the creation of work systems that enable safe work that is neither physically nor mentally exhaustive [15].

Significant contributions to human-oriented work design were made by Hackman and Oldham [16], Luczak [17], Rohmert [18], and Ulich [19]. Their criteria include both physical and mental human needs and attributes. For example, the employees’ work should be designed in a way that feasibility, non-harmfulness, and appropriateness of the tasks is ensured, along with satisfaction and personal development. Various standards (e.g., DIN 6385 [20] and DIN EN 894 [21]) also include principles and approaches in the ergonomic design of (interactive) work systems. The work systems must be designed to be compatible with the performance of the person, the expectations of the operator, and the type of work task.

1.2.1. Human–Machine Interaction

Human–machine interaction is a part of every socio-technical work system in which people and technology collaborate. This refers both to systems in which the machines are merely human work equipment and to systems in which humans and machines act as collaborating, independent actors. It enables two-way dependent communication between humans and machines, so called dialogues. Each dialogue process consists of several interfaces: Input and output, dialogue, and tools. DIN EN ISO 9241 provides a set of rules in this regard, which serves to design and evaluate these dialogues [22]. They should, therefore, be appropriate, self-explanatory, faithful to expectations, adaptable, robust, and conducive to learning. Further design principles were proposed by Butz and Krüger [23]. They provided basic design rules that focus on the accessibility of human–machine interfaces and intuitive operation. Regarding human–machine interaction in CPPS, Peissner and Hipp [24] identified further challenges and preliminary solutions. They considered human-oriented design, the role of the humans, multimodal interaction, the design of automation, and individualization as essential success factors.

Table 1. Approaches to human–machine interaction in CPPS according to Peissner and Hipp [24].

Approach	Exemplary Measures
Human-oriented design	Attractive design of user interfaces to increase user acceptance and work performance
	Human-oriented development of work systems through early involvement of the work force
	Iterative design processes of the work areas
	Improving the user experience and work motivation through the use of gamification or innovative tools
Role of human actors	Clear Presentation and Context-Sensitive Display of Information for Complexity Reduction and Error Avoidance
	Mobile workstations for fast reactions to events
	Availability of manifold information through real-time data, assistance systems, and flexible tools
	Use of physical analogies and consistency in interface design
	Active error avoidance through use of constraints and plausibility checking
Multi-modal interaction	Use of different input and output methods for usability improvement and error avoidance
Design of automation	Use of comprehensible automation for improved acceptance and error avoidance
Individualization	User-dependent design of interfaces

provides an overview of these measures and gives examples for each.

1.2.2. Human Factors

In addition to the disciplines of work design and human–machine interaction, the area of human factors also covers issues raised by the design of CPPS. Human factors can be understood as all physical, psychological, and social characteristics of the humans, which influence the action in socio-technical systems [25]. The associated scientific discipline is also referred to as human factors, and it deals with the role of humans in complex systems, the design of work equipment, and the human-oriented adaptation of the working environment [26]. On the one hand, it aims to carry out basic research in the field of the interaction between people and technology and, on the other hand, to produce solutions for practical problems. The aim is to avoid the negative consequences that can arise from this interaction. As a result, human well-being can be increased, and the safety and function of the human–machine system improved [25]. Human-oriented design of the CPPS is thereby of high importance.

In the area of manufacturing systems, the human factors address the workers who operate or monitor production machines. The gain of knowledge is mainly achieved through social scientific research methods such as behavioral observations, interviews, or questionnaires. A combination of these methods enables the recording of externally observable actions and utterances as well as the acquisition of “inner” information, such as action strategies, satisfaction, or expert knowledge [25].

1.2.3. Human Factors in CPPS

For CPPS in particular, cognitive ergonomics play an essential role. In contrast to conventional ergonomics, it is less concerned with physical and anthropometric characteristics than with the cognitive, emotional, and motivational needs and abilities of people. It does not only affect those informational processes that occur during task processing. Rather, through the design of the work equipment or the task, it also has an impact on how the work is organized by the worker. It affects the entire work process, starting with the task preparation and ending with the completion of the task [27]. Part of cognitive ergonomics with regard to human–computer interaction is also the consideration of emotional and motivational aspects. For example, researchers investigate how assistance systems should give hints or recommendations to workers in order to trigger the desired consideration and response from users, but avoid defense reactions or ignorance [25].

CPPS show a high process and work complexity as well as an increased level of automation. These aspects need to be considered more closely about human factors. Complex situations have specific characteristics. First, they are extensive in terms of recording and processing the work task. They contain a large number of variables that have to be included in the consideration. These variables are not isolated from each other but interlinked. As a result, they influence each other in direction, type, and intensity. Measures on one variable may, therefore, have (undesirable) side-effects on other variables. Furthermore, complex situations are characterized by opacity, which can cause parts of the overall situation to remain hidden for the acting person. For example, variables may be unknown, their interlinking unclear, or their current state not determinable. Consequently, actions must be taken under uncertainty. Finally, complex situations are dynamic. The variables and their interaction change over time. Thus, the states of the variables can change even without the influence of the acting person or show a different state at the time of implementation of a measure than at the time of the first observation [28]. While in everyday life, many tasks can be met by routines, complex conditions lead to the necessity to actively organize tasks. There are, therefore, various ways of dealing with complexity: behavioral prevention and structural prevention. Behavioral prevention aims to provide strategies to deal with new and complex situations by training. Thus, it increases the flexibility of the workers. Structural prevention tries to reduce complexity by design in the best possible way. The reduced complexity then shows a higher similarity with standard situations and can be handled better [28].

The change of work as a result of the introduction of CPPS leads to the computerization of workplaces and further automation of processes (see also Section 1.1). Therefore, many of the future workplaces present themselves as human–machine systems. Consequently, in these cases, tasks are not carried out by only one person but by the interaction of (several) workers and machines. At this point, the term machine includes both production machines and cyber-physical assistance systems. The design of these systems in terms of a human-oriented work is in the area of interest of the human factors. Automation is the process of transferring activities from humans to machines. The automated activities can be manual or cognitive: Manzey [29] mentioned manual work, manual control, and control activities on the one hand, and judgment and decision-making activities on the other hand. An important design issue of automated systems lies in the distribution of tasks among the various actors of the human–machine system. This applies both to the allocation of tasks to different people and the assignment of tasks to workers or machines. The assignment of tasks to machines serves different purposes: utilization of economic advantages, error reduction, quality improvement, or improvement of work for humans.

The quality of the division of tasks between people and machines in practice depends crucially on the interaction between them [29]. There are various approaches to answering these allocation questions. The first approach is an automation strategy with a focus on technology and costs: according to this strategy, all tasks which can be easily and cost-effectively automated are transferred to machines. Humans are left with those tasks where automation is impossible or costly. The goals of this strategy are primarily to increase the efficiency and reliability of processes. There is little consideration given to the impact of this strategy on workers [26,29]. A second approach is an automation strategy that focuses on the capabilities of the actors. They should be relevant to the tasks assigned to them. Often, however, an isolated view of people on the one hand and machines, on the other hand, cannot meet the idea of a system-wide perspective. As a result of this criticism, a third approach regards people and automation as a complementary, hybrid work system. According to this idea, humans always bear the responsibility for the systems and thus take on the role of a so-called “leading control” [30]. Their tasks include:

• the planning of the tasks of the machines;
• the transfer of tasks to the machines;
• the monitoring of the execution of the activities;
• the intervention in case of unintended outcomes; and
• the learning from experience in handling the machines.

Examples of implementing this human role in the design include the creation of active involvement of humans and the access to complete information about the activities of the automation. Humans and machines should be considered as equal and independent actors of the system [29,30]. The usability and performance of a human–machine system which contains automated components depend to a large extent on humans having a reasonable degree of confidence in automation. An appropriate situational awareness is present when the operator is informed about the current system status and can foresee its further development [29,31]. Various reasons can lead to a lack of situational awareness. Examples are a lack of monitoring of the automation, changed feedback channels (for example, haptic or visual impressions of the production machine may no longer exist when the task is changed to leading control), or lack of transparency of the automation. Often a lack of situational awareness goes hand in hand with a loss of manual skills. This can be critical if operators need to regain control of the system or have to identify errors (see again [11]).

2. Materials and Methods

The application of methods is an elementary part of human factors research. They are utilized to conduct evaluations and predictions on the interaction among humans and machines concerning productivity, safety, and work satisfaction [32]. In this way, the performance of a human–machine system can be evaluated, and the requirements and effects of the work on the working persons can be assessed. The findings can be used to develop new and improved systems [32,33]. In many cases, the starting point for a human factors research project lies in the need to improve a human–machine system in terms of work conditions and efficiency. In addition, the objective may be to describe fundamental relationships in the interaction of humans and machines that are detached from a concrete problem [32]. Thus, research can be fundamental or applied. Fundamental studies are used to understand the factors that influence the function of the human–machine system. The results of these studies are basic principles of action and theories [32,34]. Applied research aims to transfer the results of fundamental research into the development and evaluation of human–machine systems [32].

2.1. Design and Analysis of Experiments

A standard method of human factors are experimental studies [32]. They can be used to show whether there are causal relationships between the design of a human–machine system and effects on the humans involved, the system, or the system performance. Therefore, in an experimental study, one or more variables are modified in order to induce observable effects [32,33]. The goal of a subsequent statistical evaluation of the experiments is to answer the question whether the selected independent variables (factors) have an influence on the dependent variables. Both descriptive and closing statistics methods are used. While the former summarizes and presents the collected data, the latter concludes whether the identified differences are significant or only the result of a random variation [33]. Descriptive statistics in the field of human factors often include the representation of mean values and standard deviations. Typically, the values of the dependent variables are determined for the respective levels of the independent variables and statements are made about their variance [33]. If there is a factorial design with several independent variables, the main effect of an independent variable and interaction effects between several independent variables can be determined [35]. For experimental designs that follow a factorial design with several independent variables in different levels, an analysis of variance (ANOVA) is often performed (if different conditions are met, see [35]). ANOVA provides a statistical test that takes into account the observed mean values of the different groups. It allows a decision to be made whether these differences are large enough to conclude that there are similar differences in the underlying populations [36].

2.2. Available Human Factors Research in CPPS

In the past, several research projects have dealt with the effects of human-oriented working conditions on the physical and mental condition of employees on the one hand and the economical parameters of companies on the other hand. These works revealed connections between human-oriented measures (e.g., ergonomic design or cognitive simplification) and their positive consequences (e.g., increased job satisfaction and improved product quality) [37,38,39,40,41]. In addition, a cost–benefit assessment of ergonomic measures in companies showed mostly positive effects of these measures by their ergonomic intentions as well as short amortization phases [42]. While the majority of these studies were carried out based on a classical understanding of ergonomics, in the context of Industry 4.0, the human factor is currently once again the subject of research. Various papers deal with the analysis, potential, and impact of human factors [43,44,45,46]. In addition, an interdisciplinary approach to the implementation of the human factors, the consideration of social changes in product development, and the role of technology acceptance by users has been discussed.

According to the previous explanations in Section 1.1, the work systems in semi-automated production systems are characterized by intensive use of the interface between humans and machines. Here, cognitive assistance systems are used to establish this connection between the automated production parts and the workers. In a study by Rauch et al. [47], existing research and contributions were identified using a systematic literature review, which pursued a human-oriented approach in the development of cognitive assistance systems. These research contributions are examined in more detail below in order to evaluate them in terms of considering human factors in the design process of the cognitive assistance systems.

Chen et al. [48] developed a framework for early consideration of human factors during the planning process of an assembly system. By integrating ergonomics analyses into a virtual assembly system, the expected stress on the employees can be evaluated. The result of the work of Yang et al. [49] is a concept for a human-oriented virtual factory in which work-relevant issues, such as ergonomics, collaboration, or training, are integrated into the planning process at an early stage. These factors are taken into account both in the product design process and in factory and process planning. Santochi and Failli [50] considered possible approaches for sustainable work in production, which is the result of the interplay of physical and mental working conditions and motivational influencing factors. They also suggested possible approaches for improving a typical assembly workplace using cognitive assistance systems. Romero et al. [51] proposed a human-centered reference architecture for the conception of semi-automated systems. They intended to develop production systems which are characterized by adaptable automation and thus support employees meeting their needs, for example, the requirements of elderly employees or trainees. Hold et al. [52] described a procedure for integrating physical and cognitive assistance systems into CPPS. They aimed to uncover suitable application possibilities and potentials.

Ohtsuka et al. [53] described a cognitive assistance system for collaboration with robots. The system recognizes the desired operation of the employee and provides support. This results in faster task processing. The work of Weiss et al. [54] contains the idea of a collaborative work system between humans and robots. For the developed system, studies were carried out to determine the interaction between the actors and to derive improvement potentials. The authors used human factors methods. They proposed an assistance system integrated into the worker’s clothing for robot control and monitoring. Soffker et al. [55] presented a method for mapping human–process interaction in manufacturing work. It can be used to develop intelligent assistance systems, which contribute to an in-depth process of awareness. The authors proposed an assessment of suitable channels to be used to interact with workers (e.g., auditory or visual channels). Pacaux-Lemoine et al. [56] proposed starting points for cognitive assistance systems to be used by workers with control tasks in autonomous production systems. A prototypical implementation led to both a reduction of the energy consumption and of the throughput times.

The work of Gorecky et al. [57] deals with the development of virtual training and knowledge transfer systems for worker qualification and for improving communication in production systems. The authors described the prototypical implementation of two exemplary systems. Paelke [58] considered the development of an augmented reality assistance system to support employees in part selection and assembly during an assembly process. Different visualization concepts are developed for the information to be displayed. In addition, a test of the system with a large number of study participants is described. Another augmented reality system was presented by Rauh et al. [59]. Here, an augmented reality assistance system is developed to support employees during maintenance activities in the production process. This work includes a human-oriented development and evaluation of the prototype system by involving the users in different phases of the design process.

Level of Integration of Human Factors

The presented research works deal with cognitive assistance systems, which can be used either in the planning process of production systems or during the manufacturing process of the products. They aim to improve working conditions for employees and to optimize process quality. A human-oriented approach was followed in each case. These ideas were evaluated concerning the type and extent of consideration of human-oriented design principles and methods of human factors. They were checked qualitatively for the fulfillment of the following criteria:

• Explicit consideration of designing a human-oriented assistance system (use of design principles of human-oriented work design and human–machine interaction in general and for automated systems in particular)
• Explicit consideration of the interface between worker and cognitive assistance system
• Methodical integration of human factors into the design process of assistance systems (use of an approach which integrates human factors concepts and standard methods)

Table 2. Analysis of research papers taken out of a review by Rauch et al. [47] on available human-oriented approaches for the development of cognitive assistance systems: Design, explicit consideration of designing a human-oriented assistance system; Interface, explicit consideration of the interface between worker and cognitive assistance system; and Methods, methodical integration of human factors into the design process of assistance systems.

Contribution	Design	Interface	Methods
Chen et al. [48]
Yang et al. [49]
Santochi and Failli [50]
Romero et al. [51]	x
Hold et al. [52]
Ohtsuka et al. [53]
Weiss et al. [54]			x
Soffker et al. [55]	x
Pacaux-Lemoine et al. [56]
Gorecky et al. [57]
Paelke [58]		x
Rauh et al. [59]		x	x

shows the results. A checked research paper indicates the fulfillment of a corresponding criterion. It can be noted that only a low level of criteria compliance could be found. Only two out of twelve research papers fulfill one of the criteria. Altogether, we can conclude that the focus of the contributions is mainly on the technical and organizational development of cognitive assistance systems, but only to a limited extent on the methodological integration of the human factors.

3. Integration of Human Factors into Human-Oriented Work Design in Cyber-Physical Production Systems

The results of the presented work can be separated in four parts: First, we derived crucial work design elements in CPPS (Section 3.1). Second, we developed a method for the integration of human factors into human-oriented work design in CPPS, which is based on the evaluation of the impact of work design elements on humans in a work environment (Section 3.2). Third, we developed an experimental platform for carrying out human factors studies as a part of the presented method (Section 3.3). Fourth, we applied and validated the method using the developed platform (Section 3.4).

3.1. Design Elements for Human–Machine Interfaces in CPPS

The findings presented in the previous sections on work in CPPS and on human factors in human–machine systems show the availability of a wide range of research work in this area. The consideration of work design in CPPS, however, shows only a few findings thus far. In addition, the applicability of many standards and design rules of work design for conventional production systems must be questioned. Since most of this research has been done before the introduction of cyber-physical work systems, it often involves an outdated understanding of work in production systems. Due to new tasks, a changed allocation of tasks between humans and machines as well as new assistance systems, cognitive work, in particular, becomes predominant, while the share of physical work decreases. As a result, an overview of new design elements was developed to meet the changing requirements of cyber-physical work systems. The basis is formed partly by research contributions on the current and future development of work in CPPS and partly by research on the human-oriented design of human–machine systems.

Table 3. Basis of the new design elements concerning human factor-oriented work in CPPS and their corresponding contributions.

Area	Contribution	Reference
Cyber-physical production systems	technological capabilities, new products and processes	Section 1
Change of work	New requirements, changed division of labour, new work equipment	Section 1.1
Human–Machine- Interaction	Interface Design Principles for hybrid, semi-automated cyber-physical systems	Section 1.2.1
Human factors	Human needs and abilities as part of cyber-physical, socio-technical systems	Section 1.2.2

shows the areas of origin of these references and the contribution they provide to the finding of new design principles. We also refer to the corresponding sections of this publication, which contain detailed information on the specific areas. The subject of this discussion are always working systems in CPPS, which are designed as semi-automated systems.

An overview of the new design principles is shown in Figure 1. It represents an excerpt of the aspects that work design in the sense of human-oriented work in CPPS must additionally take into account. Due to the thematic overlap of the subject areas, the design elements may originate from one or more of these areas. During their derivation, a maxim was set that they are of special relevance in the practical context of work systems in CPPS. In the context of this paper, they represent a basis for Phase 1 of the method (see Section 3.2), in which crucial design elements for the present work system are derived and selected for the subsequent phase steps, respectively. In the following, the design principles are referred to as design elements. They are explained briefly below:

Usability describes a facilitated usage of the interface for the worker. This is characterized by increased effectiveness, efficiency, and satisfaction when using the interface [60]. For example, usability can be increased by improving the readability of information or the simplicity of its presentation, as well as by placing dialog elements intuitively. The design element Attractive Design goes beyond the previously described aspect of usability and wants to create an additional positive experience for the working person when using the interface. Work tasks are perceived as easier if they can be performed with the help of attractive work equipment [61]. This can be achieved, for example, by exceptional design or the implementation of new technologies [26]. The Adaptability of the human–machine interface describes the possibility on behalf of the worker to individualize it. For example, special individual requirements can be taken into account by setting options such as font size, language, or contrast. Besides, adaptability can be used to personalize the use of existing assistance systems according to the skill level [22,62].

The design element Robustness describes the tolerance of the interface against wrong inputs and errors. Consequently, the design should aim to minimize the impact of errors on the outcome of the work. This can be done, for example, by automatic plausibility checks or the use of an “Undo” function [23,62]. Affordance describes the “invitation” of an object to use it for specific actions. For example, three-dimensionally designed buttons could be used, which seem to protrude spatially. Thus, they motivate the worker to click or press them, since they look like buttons on physical devices [23]. Constraints are (physical) restrictions of the interaction with an object [23]. For example, a drag and drop operation might only allow an object that has been picked up to be placed in the designated areas. This way, hints can be given on the intended processing of the work task, and errors are avoided. Mapping is a design element that provides a suitable assignment of real objects to the interface [23]. For example, with spatial mapping, the arrangement of production machines in production would be transferred in the same way to the arrangement of sensor data for these production machines on the interface.

The element Consistency refers to the use of constant layouts, colors, or terminology for same and similar issues. For workers, this results in greater reliability in identifying functions and predicting their effects [23,62]. Physical analogies describe the behavior of buttons based on physics [63]. For example, activated buttons could be shown as pushed in buttons. Thus, the activation can be intuitively perceived by the worker. By using Assistance systems, workers should be supported in the processing their work tasks. A typical example is an augmented reality goggle that complements the perceived reality with helpful support. Such tools can represent the human–machine interface themselves but can also be designed as a function on an existing interface (e.g., on a tablet computer) and thus support the worker in different ways [64,65]. This kind of assistance system could, for example, be designed as color highlighting of valid areas when placing objects within a user interface.

The design element Amount of information describes the provision of the appropriate amount of information which is necessary and useful for solving the work tasks [66]. Only such information shall be displayed that contributes to an improved or simplified problem-solving process. ViaFeedback workers can be offered feedback both about the activities of the cyber-physical work system and about the quality of their work results. For example, this design element can range from confirmations or process status information to details on the user’s goal achievement during or after task processing [22,23].

Gamification links work with elements from the field of computer games. For instance, it describes the use of virtual rewards (badges), a high score list, or different game levels in the remote context of a work system. This way, the intrinsic motivation of the workers can be increased, as the rewards lie in the work system itself and are not brought in externally [67,68]. The design element Multimodal interaction intends the use of several ways of communication between humans and machines. As a consequence, input and output should not only be carried out one way, for example by using a touch display but should also include other types of communication such as speech, gestures or facial expressions. This allows the human–machine interaction to be approximated to the natural communication between humans and their environment, and thereby to improve the information flow [69].

By Participation of the workers in the design process of interfaces, the interaction quality can be improved, and the efficiency of the design process can be increased. On the one hand, this participation can provide a complete picture of the work system during the requirements analysis and, on the other hand, can improve the acceptance of the interface at its introduction and operation [70,71]. Work allocation deals with the assignment of tasks within the human–machine system and allocates them to humans and automation [29]. This division of functions follows the present automation strategy and influences the quality of use and acceptance of CPPS [72]. The Complexity of a work task is a consequence of the required action and the available information and determines the work performance [73]. Complex conditions in terms of work design can be addressed by behavioral prevention (training) or structural prevention (reduction of complexity) [28].

Actions derived for the new design elements should be evaluated in terms of their effects on human work. Existing evaluation possibilities in the field of conventional ergonomics are somatographic aids. These are human-shaped models used to check the anthropometric design of workplaces. This procedure may also be implemented with the help of computers as an Augmented Reality application [74]. Furthermore, digital human models can be used in CAD models of workstations for vision or accessibility checks. These evaluation methods increase productivity and lead to cost reductions [75]. In addition to conventional ergonomics, usability engineering can be used in the evaluation of human–machine systems. It describes the process of continuously considering the usability and user-friendliness of work systems already during their design process [74]. Here, observations and user surveys are carried out frequently. The goal is to design a suitable user interface that avoids unnecessary complexity and maps the required functionalities of the work system [76]. In these types of tests, the users are confronted with representative work tasks. During and after processing, key data on effectiveness, efficiency, and user satisfaction can be collected [74].

To be able to check a large number of different design elements, a flexible way of conducting human factors studies is required. It should be usable for various derived actions. As a part of the method presented in the following section on the integration of human factors into the work design of CPPS, the development of an experimental platform is carried out.

3.2. Method for the Integration of Human Factors into Human-Oriented Work Design in CPPS

The method aims to support the understanding of the mechanisms between human factors and work in CPPS. It focuses on the area of work design. Here, the design of interfaces between humans and machines in manufacturing serves as a research subject. A method shall be designed to enable the assessment and evaluation of specific work design elements. The method should be capable of providing a first catalogue of rules regarding the interface design in CPPS.

We postulate that the integration of the new design elements presented in Section 3.1 has a significant effect on work performance and the perception of work. Our method is designed to assess and evaluate this expected effect. Its basic idea is to conduct experiments in which workers interact with cyber-physical systems via human–machine interfaces. The participants are asked to solve work tasks. Here, various key figures and inputs are recorded. At the same time, questionnaires are used during the experiments to gather personal, work psychological information about the test persons. Consequently, quantitative data on the task-specific achievement of objectives and personal status information is determined in the experiments.

If a design element is active when performing a first work task, but inactive when performing a second work task, the difference in the outcome variables can be linked to the design element. For example, a measure derived from a design element could be the illustration of a task description. In combination with suitable study design, this allows research to show the effects of design elements or draft work designs. These data enable a statistical analysis of the relationship between particular elements of work design, the perception of work, and the achieved performance. Thus, the hypotheses on essential design elements in CPPS can be tested and transformed into design principles that correspond to the particular effect.

The methodology provides a stepwise procedure in four phases:

• Phase 1: Hypothesis of key design elements;
• Phase 2: Design and implementation of representative work tasks as a human factors study;
• Phase 3: Execution and Evaluation of the human factors study; and
• Phase 4: Development of design principles for interfaces in CPPS.

Figure 2 summarizes a simple run of the method. A rectangle symbolizes an operation and an ellipse the end of the process. The method is presented in detail in the following.

3.2.1. Phase 1: Hypothesis of Key Design Elements

Phase 1 of the method aims at identifying the key design elements in the present cyber-physical work system. At this point, the term “key” refers to the design elements that may have a significant effect on work performance or the perception of work. The selection of these elements will be initiated by a collection of potentially important design elements for human–machine interfaces in CPPS. This collection originated from previous research in the fields of human–machine interaction and human factors as presented in Section 3.1. The collection of possible key design elements is presented in the overview (see Section 3.1). These design elements will be evaluated in the next steps of the method.

3.2.2. Phase 2: Design and Implementation of Representative Work Tasks as a Human Factors Study

Phase 2 of the method implements the identified representative work tasks and the key design elements as a work design draft. This is carried out in a way that enables its implementation and subsequent use for a human factors study. Consequently, the design and preparation of the study is also part of Phase 2. First, the identified design elements and the representative work tasks are implemented on the experimental platform (see Section 3.3). As a representative work task, a model CPPS work system is used that does not originate from any concrete application case. The result is a work design draft that offers the possibility of modular integration of various design elements. This way, the design of experiments by a full factorial design is possible (see Section 2.1). Second, the design of the experiment is defined. This design of experiments includes the determination of the variables of the study and the preparation of a test plan. In the following third step, the work tasks realized on the experimental platform are transferred into a human factors study according to the design of experiments.

3.2.3. Phase 3: Execution and Evaluation of the Human Factors Study

In Phase 3 of the method, the human factors study is carried out, and the study results are evaluated statistically. As a first step, a sufficient number of adequate participants must be recruited for the study corresponding to the design of experiments. Within the study, the participants perform predefined work tasks. Here, the target variables (dependent variables) are automatically recorded by the experimental platform. Finally, the study results are provided for subsequent analysis. While carrying out the experiments, it is necessary to observe the test plan as well as the experimental conditions in order to achieve consistent and unbiased results. In a second step, the statistical evaluation of the study results takes place. Statistical analysis is performed to display significant differences in dependent variables caused by the design elements (independent variables or factors). The result provides the effect intensity of the design elements on the dependent variables (key figures).

3.2.4. Phase 4: Development of Design Principles for Interfaces in CPPS

Finally, Phase 4 aims to develop corresponding design principles for work design in CPPS based on the determined effect strengths and effect directions of the design elements. Therefore, differentiation between “Yes” and “No” rules will be made, which indicate whether an implementation of the respective design element can be recommended. The entirety of all key figures examined is explicitly considered. In the event of conflicting effects concerning individual factors, a division into “Yes” and “No” rules will also be made if a predominantly positive or negative effect can be identified. In all other cases, a “Maybe” rule is defined, which requires further investigations. Based on the study results, the effect sizes for each dependent variable are used as effect indicators for all examined design elements.

Table 4. Generation of effect indicators for absolute and assessment variables.

Effect Size ${\mathit{E}}^{+}$/${\mathit{E}}^{-}$	Effect Indicator ${\mathit{E}}^{+}$	Effect Indicator ${\mathit{E}}^{-}$
$E>+50\%$	$+++$	$---$
$+25\%<E\le +50\%$	$++$	$--$
$+10\%<E\le +25\%$	+	−
$-10\%<E\le +10\%$	0	0
$-25\%<E\le -10\%$	−	+
$-50\%<E\le -25\%$	$--$	$++$
$E\le -50\%$	$---$	$+++$

shows the generation of effect indicators for absolute or assessment variables. These are key figures where an increase or decrease are the possible changes, for instance, the processing time of a work task or the evaluation of the task difficulty on a rating scale. If the desired effect direction is a reduction (e.g., processing time), the effect indicator $E^{-}$ is used; otherwise, $E^{+}$ is used for an increase. An observed reduction of the processing time by 12%, for example, would lead to an effect indicator of + (see Table 4). The sums of the effect indicators are then compared. If a homogenous pattern of the effects can be identified for both aspects (work performance and perception of work), a “Yes” rule is assigned for a positive effect and a “No” rule for a negative effect. In the case of an indifferent pattern of effects, a “Maybe” rule is assigned.

3.3. Conception and Development of an Experimental Platform

A platform needs to provide an interface to the investigator and the participant. The investigator configures the setup of the experimental platform to set up and configure the investigation. This includes the realization and customization of the work tasks, the design elements, as well as the set-up of the study. Once the experiment is finished, the experimental platform makes the recorded results available to the investigator. The test participant uses his or her interface to the system for task processing and answering of questionnaires. All entries and responses made are recorded. The described functionality of the interfaces is shown in Figure 3.

To meet the requirement that the experimental platform should be designed as flexible as possible to cope with different design elements, the setup and configuration of the experiments should be implemented as easy as possible. For example, including a new design element in a work task should be easy to integrate. In addition, there should be a set of modules which can serve as building blocks to sequentially assemble the experiment. The modules are implemented as pages (similar to a website or app page), which are displayed on a touch screen display and can be viewed by a test person in a sequence. A sequence consists of pages that can be used for task description, task processing, or interviews. There can also be additional pages for the information supply (e.g., title pages or introductory pages).

During task processing, the experimental platform can measure various key figures and record information. This information can be observable and non-observable. The observable variables are:

• the number of display touches;,
• the time used to solve a work task; and
• the outcome of the work task.

Non-observable variables are assessed using questionnaires. Depending on the chosen test design, questions can be asked about the perceived complexity of the task or the satisfaction with the own outcome. The collected data are recorded by the system and made accessible to the investigator.

The experimental platform is based on a Raspberry Pi 2 B single board computer combined with a 7-inch touch display. Further components are shown on the right side of Figure 4 as a cross-section of the system. Due to the use of common electronic components such as a Raspberry Pi 2 B or a USB power bank, a cost-effective implementation of the experimental platform is possible. The total price of a system is 233 Euro (as of 2019). The total price is in the range of commercially available tablet computers, which can be a possible alternative to this system. Due to its modular structure, however, the platform can be flexibly extended and adapted. The use of a Raspberry Pi, for example, provides extensive connection options (e.g., for operating the LED strip). The perspex box can be used to easily accommodate additional components. Thus, the use of the experimental platform is preferred over the use of a tablet. The left side of Figure 4 shows a photo of the system.

The software of the experimental platform is a self-developed Python 3 program. The operating system used is Linux for Raspberry Pi. Requirements for the software and its use cases from the perspective of the involved users were derived as follows [77]:

• The system has to implement the representative work tasks and the design elements from Figure 1 and represent the interface between human and machine within a cyber-physical work system.
• The system has to supply the necessary input and the required output regarding the representative work tasks to the workers.
• The system has to facilitate the solution of the work task.
• The system has to provide questionnaires regarding the query of work psychological indicators and personal information.
• The system has to display feedback on the completed representative task.
• The system has to independently record the desired target values of work performance and perception of work and make them available to the investigator.
• The system has to be modular in order to be able to flexibly compose and adapt the experiment out of different modules (e.g., tasks, feedback, questionnaires) following the design of the experiment.

These requirements can be assigned to the users involved (participants and investigator). Requirements 2–5 represent the use cases of the participants, while Requirements 6 and 7 are attributed to the investigator. Requirement 1 represents the main function of the experimental platform and, therefore, cannot be assigned exclusively to the participants or the investigator. Figure 5 shows these use cases.

3.4. Application and Validation of the Method

The method developed in Section 3.2 was applied and validated (see below). For this purpose, the necessary parts, the overview of design elements in CPPS (see Section 3.1) and the experimental platform (see Section 3.3), were realized. Some design elements were selected from the overview provided and implemented on the experimental platform using representative work tasks from the field of machine scheduling. Subsequently, an extensive experimental study was conducted and analyzed, which was then used to evaluate the design elements. Finally, on this basis, initial design principles for interfaces in human–machine systems in cyber-physical work systems are proposed.

3.4.1. Phase 1: Hypothesis of Key Design Elements

The following three design elements from the overview in Section 3.1 were selected: “Feedback”, “Assistance system”, and “Usability”. Since these design elements represent starting points for human-oriented design measures only, a further specification of these measures was necessary. The design element “Feedback” was implemented as a results feedback to the worker. For the integration of the design element “Assistance system”, two different measures were used: firstly, a display of detailed information on incorrect actions by the worker and, secondly, visual support for operation and decision-making. The “Usability” design element was addressed by color differentiation of dialog elements. We postulate for all three design elements that the respective design measures would have a positive effect on work performance and perception (see Section 3.1). These hypotheses are examined below.

3.4.2. Phase 2: Design and Implementation of Representative Work Tasks as a Human Factors Study

The explanations in Section 1 of CPPS, the change of work and the existing research work on human-oriented work design results the following profile of a typical, semi-automated and cyber-physical work system:

• The worker performs a predominantly cognitive activity that takes place in interaction with automated systems.
• The human–machine interface is implemented as a cyber-physical assistance system.
• The task involves the understanding of a high process complexity resulting from the use of cyber-physical systems.

This profile was met by using a representative work task from the area of machine scheduling. The selected task confronted the participants of the study with a machine scheduling problem (a flexible flow shop problem) consisting of several production machines of different types that can be used to process several multi-step orders. The work objective was to complete all orders as soon as possible. We chose this scenario as it corresponds to the characteristic of a complex cognitive task that takes place in the manufacturing context. It was implemented on the developed experimental platform, which can be seen as an interface between human and machines within a cyber-physical assistance system. Other representative work tasks could also be considered for further applications of the method. The work tasks appeared in different versions: first, as an active task in which the machine scheduling problem is solved from scratch by the worker; and, second, as a reviewing and correcting task in which a predefined solution is evaluated and can be changed. This second version represents supervisory work tasks in CPPS, such as the monitoring of an automated production process.

The given jobs and their sets of operations (referred to as partial jobs in the following) served as input for the processing of the work task. Besides, the kind and characteristics of the production machines were mentioned (type, processing time). The predefined allocation for reviewing and corrective work tasks (referred to as corrective tasks in the following) varied according to different status: optimal, needs minor improvement, and needs major improvement. The expected output for both task versions consisted first of the capturing of the job and machine situation. This was followed by the machine scheduling (active) or the assessment of the need for changes and modifying the predefined scheduling (corrective). No other information or tools were provided to solve the tasks apart from the outlined input, and there is no time limit.

Design of Work Tasks on the Experimental Platform

The work tasks consisted both in the active and in the corrective version of two pages on the experimental platform: the task description page and the task processing page. This basis was supplemented by interview pages, which were used to record the perception of work and sometimes by a feedback page (depending on the design of experiments).

The task description page listed the existing jobs $(A1,A2,\dots ,An)$ and the corresponding partial jobs $(a,b,c,d)$. Each job contained at least one and a maximum of four partial jobs. A separate table showed the available production machines $(M1,M2,\dots ,Mn)$ and indicated for which partial job they could be used and which processing time was required. Furthermore, the available machine network was represented by an illustration. Figure 6 shows a screenshot of the task description page.

For active tasks, the task processing page contained all partial jobs of the jobs (for example, $A1.a,A1.b,A2.a$, etc.) as drag-and-drop elements. These could be picked up by the study participant and placed on the machine scheduling plan. For corrective tasks, the worker started here with predefined scheduling and had the option of making changes. On the task processing page, the processing time and the number of display touches made were measured and the final solution was saved. Figure 7 shows the task processing page.

The interview page was used to collect information about the participants’ perception of work. For active tasks, the interview was conducted after task completion. The interview contained questions on the perceived difficulty of the problem, the satisfaction with the user’s own final solution and the estimation of the quality of the solution. For corrective work tasks, a two-part questionnaire was presented. The first interview took place after the predefined scheduling plan had been assessed, and the second after the changes had been made. The first interview dealt with questions about the quality of the predefined solution and the certainty of this rating. The second interview was similar to the interview for active work tasks and contained questions on satisfaction with the solution and the evaluation of the quality of the solution.

The feedback page allowed a comparison of the participant’s work results with an optimal solution. It compared the makespan of the achieved solution with that of an optimal solution. In addition, the machine scheduling plan for an optimal solution was displayed graphically. With the help of a button, the participant could switch it to the achieved solution to compare the placement of particular partial jobs.

The design elements selected in the previous phase or their respective measures were embedded in the four page types described above. The design element “Feedback” was implemented by using the feedback page for selected work tasks. If this design element was activated, the feedback page was displayed; otherwise, this page was skipped. Thus, the study participants only received feedback on the solution of a work task when this design element was activated. To implement the design element “Assistance system”, a function was developed that displays details of the type of error in the event of incorrect actions during machine scheduling processing, e.g., “Production machine already occupied” or “Sequence of the work steps of the order not observed”. Alternatively, visual support was provided to the operator for the placement of partial orders on the machine scheduling plan. The (re-)placement of drag-and-drop elements was supported by color highlighting. If an element was picked up, possible production machines on which this element can be placed (i.e., the appropriate machine type for the selected partial job) were marked by color shading. “Usability” was implemented by color coding of the jobs. All partial jobs that belong to a single job were represented by a color and were thus delimited from other jobs.

The machine scheduling problems were of comparable difficulty. For each of these tasks, CPLEX Optimizer was used to determine an optimal solution. In addition, predefined solutions were prepared in three levels: a solution without any need for improvement (no shorter completion time possible), a solution with medium improvement potential (approximately 10 percent) and a solution with high improvement potential (approximately 20 percent).

Experimental Design and Preparation of a Human Factors Study

Two experimental human factors studies were performed to illustrate the effects triggered by the design elements. Here, the three design elements selected in Phase 1 served as independent variables (factors), which were either set to “active” or “inactive”. In Study 1, the display of error details was varied for the design element “Assistance system”, while in Study 2 the visual operating support was varied. This resulted in eight experimental conditions of a factorial $2^3$ design for both studies [32,33,35]. The dependent variables represented the aspects of work performance and perception of work to be examined.

The work performance was represented by:

• the number of display touches during task processing;
• the time required for task processing; and
• the quality of the solution (ratio of optimal makespan to achieved makespan).

The perception of work was represented by:

• the quality evaluation of the solution;
• the deviation between the solution quality and the quality evaluation;
• the satisfaction with the solution; and
• the evaluation of the task difficulty.

The dependent variables chosen are out of the areas of human–machine system performance and human subjective perception [32,78]. The solution quality in terms of the deviation from the optimal completion time can be seen as a direct indicator for work performance. The number of display touches and the time required for task processing can be seen as indirect indicators of work performance. They enable direct conclusions on the efficiency of task processing and the handling of the human–machine interface. The dependent variables concerning the perception of work were indirect indicators. Here, the quality evaluation, as well as the deviation between the solution quality and the quality evaluation, reflected the situation awareness of the participant. The evaluation of the task difficulty served as an indicator of cognitive support. Finally, the measurement of satisfaction with the user’s own, final solution enabled an assessment of job satisfaction. The automatic and objective recording of the display touches and the time spent took place invisibly for the study participants in the background.

A combination of objective and subjective techniques was used for data recording. By a hidden, non-participating, and systematic observation, the objective measurement of the dependent variables on work performance took place [32,33,74,79]. Based on a questionnaire with standardized questions and answers, the subjective measurement of the dependent variables regarding the perception of work was carried out [80]. To determine the effects triggered by the design element “Feedback”, the subsequent work task was used. The assignment of the study participants to the experimental conditions was performed as a within-subjects design with repeated measures [35]. Further, various randomizations were carried out: the problems generated occurred in random order for each study participant. Within these tasks, each experimental condition occurred at least once in a randomized sequence. Besides, there was a random sequence of active and corrective work tasks for each study participant. Finally, the order of the study participants was also randomized. The various randomizations were undertaken to minimize learning and fatigue effects [32,35].

3.4.3. Phase 3: Execution and Evaluation of the Human Factors Study

The experiments were carried out under constant environmental conditions to control external influences. Furthermore, the procedure and duration of participating in the experiment followed an identical pattern for all participants:

• reception of the experimental platform;
• start of the experiment and run of a test task;
• processing of the tasks; and
• end of the experiment after 45 min.

The studies were conducted as laboratory experiments to control the experimental conditions in the best possible way with the goal of determining basic rules for the design of human–machine interfaces in CPPS. A total of 68 persons participated in the studies ($n_1$ = 33, $n_2$ = 35). In a pilot study with nine participants, first insights were gained and problems regarding the experimental platform itself and the procedure of the experiment were identified [81]. These results were incorporated into the design of the main studies. Students from the University of Bremen and external participants were acquired as study participants. Due to their affiliation to different disciplines, professional heterogeneity of the group could be achieved, which mirrored the various application possibilities of the method. All study participants were assured an anonymous use of the results. Mostly automated execution of the experiments as well as the hidden data collection should avoid unintended external effects. In addition, the suitability of the empirical method was confirmed in advance based on various application examples (design of production cells [82], design of a multimodal user interface [83], and evaluation of different supply methods for material and work descriptions [84]).

The analysis was performed separately for each of the two studies (in the following, referred to as Study 1 and Study 2) as well as for each type of work task (in the following referred, to as Type A and Type B tasks). The studies differws regarding the chosen measure for the design element “Assistance system” (Study 1: error messages; Study 2: color highlighting) as well as the complexity of the work tasks. The work tasks occurred either as active tasks (Type A) or as corrective tasks (Type B). After completion of the studies, initially, data preparation was carried out. Here, we aimed to remove incomplete tasks from the results. All tasks in which the study participants spent less than 15 s on the assessment of the predefined solution and the task processing were removed. This threshold was set at approximately 10 percent of the average processing time. Then, 147 valid and completely solved Type A tasks and 135 Type B tasks remained for Study 1 and 106 and 68 for Study 2, respectively.

The statistical significance of the independent variables was assessed utilizing a subsequent three-factor variance analysis performed with the Minitab 18 software.

Table 5. Selected Results of the analysis of variance (Study 1, Type A, significant results are marked with a gray background).

Dependent Variable	Usability	Assistance System	Feedback
Display touches	F(1,144) = 0.06, p = 0.802	F(1,144) = 0.43, p = 0.511	F(1,144) = 3.12, p = 0.079
Processing time	F(1,144) = 0.26, p = 0.61	F(1,144) = 4.37, p = 0.038	F(1,144) = 5.98, p = 0.016
Satisfaction	F(1,144) = 6.91, p = 0.009	F(1,144) = 0, p = 0.979	F(1,144) = 0.55, p = 0.459
Quality evaluation	F(1,144) = 3.25, p = 0.074	F(1,144) = 0.11, p = 0.735	F(1,144) = 0.75, p = 0.389

shows a selection of the resulting p-values. In the sense of a practical significance, all effects showing $p<0.15$ were used for further consideration. If $p<0.15$ is present, this entry is marked in gray color in Table 5. Only significant effects were used for further analysis and interpretation. A complete overview of all p-values of both studies can be found in t Appendix A.

Following the results of the analysis of variance in Table 5, the main effects of the factors are presented in Figure 8. They show the mean values of the dependent variables for each factor by level [35]. For example, the main effects graph at the upper left (processing time, Type A) shows a mean value of 190 s for the active factors “Feedback” (+1) and a mean value of 268 s for the inactive factor ($-1$). Consequently, there is an effect of the factor “Feedback” on the processing time of 78 s. Similarly, the factor “Assistance system” reduces the processing time by 66 s, while there is no significant effect for the factor “Usability” ($p=0.61$, see Table 5).

Furthermore, for the selected factors and dependent variables in Table 5 and Figure 8, further significant effects could be identified. There is an effect of the factor “Feedback” on the number of display touches of about 10 units. “Usability” led to an increase in the evaluation of the satisfaction with the user’s own, final solution of approximately 0.7 steps on the seven-step rating scale. A similar effect was observed for the evaluation of the quality of the solution (approximately 0.5 steps). The size of these effects can be expressed as regression equations in addition to the main effect graphs. In the following, the equations are shown, again corresponding to the selected cases in Table 5 and Figure 8. Here, $y_d$ refers to the number of display touches, $y_p$ to the processing time, $y_s$ to the satisfaction with the solution [1: very low; 7: very high], $y_q$ to the quality evaluation of the study [1: very low; 7: very high], $x_u$ to “Usability”, $x_a$ to “Assistance system”, and $x_f$ to “Feedback”. Each consists of a constant baseline to which the respective effects of the independent factors are added or subtracted. For example, the combination of an active “Usability” (+1), an active “Assistance system” (+1) and an inactive “Feedback” ($-1$) would result in an expected number of 46.3 display touches ($=40.25-0.70·(+1)+1.82·(+1)-4.93·(-1)$). A complete overview of all main effect graphs can be found in Appendix A of this paper.

$$y_d=40.25-0.7·x_u+1.82·x_a-4.93·x_f$$

$$y_p=229.3-8.1·x_u+33.0·x_a-39.0·x_f$$

$$y_s=3.648+0.341·x_u+0.003·x_a+0.097·x_f$$

$$y_q=3.713+0.225·x_u+0.042·x_a+0.108·x_f$$

In summary, all significant effects (with $p<0.15$) of both studies for task Types A and B are presented in Figure 9. It can be seen that the design elements “Usability” and “Feedback” in particular induce several effects. For “Usability”, nine significant observations could be made in the four areas examined (Study 1, Type A; Study 1, Type B; Study 2, Type A; and Study 2, Type B). It led to a reduction of the processing time, to a reduction of the number of display touches, to higher quality evaluation, a lower perceived difficulty and to an overestimation of the quality of the own solution compared to the actual solution quality (deviation of solution quality $<0$).

“Feedback” led to five significant observations consisting of a shorter task processing time and perceived task difficulty. The deviation in solution quality was positive at 0.872; consequently, the actual solution quality was underestimated. For the design element “Assistance system”, three significant observations were made. Initially, an increase in the time required to solve the work tasks was observed. Additionally, the evaluation of satisfaction with the work result decreased, and the quality of the own solution was rated lower.

3.5. Phase 4: Development of Design Principles for Interfaces in CPPS

In the fourth phase of the method, the effect sizes and directions determined were transferred into design rules. Based on the effects summarized in Figure 9, the design rules were derived according to the scheme shown in Section 3.2.4.

In

Table 6. Implementation: Effect sizes of the design elements.

	Work Performance			Perception of Work
	Display Touches	Processing Time	∑	Satisfaction	Evaluation of Difficulty	Deviation of Solution Quality	∑
Study 1, Type A
Assistance system		$--$	$--$
Feedback		$++$	$++$
Usability				+			+
Study 1, Type B
Assistance system
Feedback
Usability		+	+			$++$	$++$
Study 2, Type A
Assistance system
Feedback		+	+		−		−
Usability	$++$	$++$	$++++$		−	$++$	+
Study 2, Type B
Assistance system				$--$			$--$
Feedback		$++$	$++$			$++$	$++$
Usability

, the first step of this procedure is shown: the corresponding effect indicators were determined for all dependent variables for which significant effects were found. The existing scheme for deriving the effect indicators could be used for the key figures display touches, processing time, satisfaction, and difficulty rating, since these variables are absolute or rating figures. Here, the effect strengths $E^{+}$ and $E^{+}$ were calculated by dividing the main effects by the baseline values of the regression equations and then reading the corresponding effect indicators in Table 4. For the variable deviation of solution quality, however, a different scheme was used to determine the effect indicators. This variable is a relating figure where the possible change is a change in the distance between the related variables. A distance as small as possible represents the desired direction of change.

Table 7. Caption on the formation of effect indicators for solution quality deviation.

Effect Size E	Effect Indicator
$E>3.00$	$---$
$2.50<E\le 3.00$	$--$
$2.00<E\le 2.50$	−
$1.50<E\le 2.00$	0
$1.00<E\le 1.50$	+
$0.50<E\le 1.00$	$++$
$-0.50<E\le 0.50$	$+++$
$-1.00<E\le -0.50$	$++$
$-1.50<E\le -1.00$	+
$-2.00<E\le -1.50$	0
$-2.50<E\le -2.00$	−
$-3.00<E\le -2.50$	$--$
$E\le 3.00$	$---$

shows the schema we used here.

Due to the mentioned differences between Study 1 and Study 2 as well as the task Types A and B, separate analyses were again carried out. By transferring the aggregated effect indicators, we can now compare the effects on work performance and perception of work in

Table 8. Implementation: Determination scheme of design principles, Yes-Rules: green background, No-Rules: red background, and Maybe-Rules: yellow background.

	∑ Work Performance	∑ Perception of Work	Yes-Rule	No-Rule	Maybe-Rule
Study 1, Type A
Assistance system	$--$			x
Feedback	$++$		x
Usability		+	x
Study 1, Type B
Assistance system
Feedback
Usability	+	$++$	x
Study 2, Type A
Assistance system
Feedback	+	−			x
Usability	$++++$	+	x
Study 2, Type B
Assistance system		$--$		x
Feedback	$++$	$++$	x
Usability

. This results in five “Yes” rules, in which either both aspects show positive effect indicators or only one area shows a positive effect indicator. The “Yes” rules here refer to the design elements “Usability” and “Feedback”. For work tasks with Type A, there is an additional “Maybe” rule for “Feedback”, since work performance (+) and perception of work (−) lead to a conflicting situation. Finally, two “No” rules result for “Assistance system”; here, analogous to the “Yes” rule, either one or two negative effect indicators occurred.

4. Discussion and Conclusions

Numerous approaches to human-oriented design are available from many disciplines associated with human factors [25]. In the present work, a linking of human-oriented design approaches to cyber-physical production systems enables us to focus on the cognitive elements which are essential for future work systems. The presented overview of work design elements, which we expect to be crucial for work design in CPPS, in Section 3.1 sets a starting point on the way to the development of work design principles for CPPS work systems, respectively. However, although the overview covers essential areas, it cannot be considered exhaustive. The selection, which has been made from the design elements (i.e., “Feedback”, “Assistance system” and “Usability”), could be successfully implemented and converted into (first) work design principles using the method. Besides, we assume that, for further design elements of the presented overview, corresponding measures are conceivable as well.

The developed method provides a step-by-step procedure that keeps the applicability threshold for investigators low by subdividing the necessary steps into four phases. Further, necessary resources for the application of the method are available, since they were jointly developed within the scope of this research (i.e., starting points for work design elements in CPPS, the experimental platform). The method could meet the research goal and led to the desired quantification of work performance and perception of work (see also the following part regarding the application results).

The experimental platform was well received by the study participants and could thus contribute to the later acceptance of the human–machine interface to be developed. It led to a higher level of attention than would have been attained for a web application or similar. Besides, the modular design allows a variety of adaptations (e.g., use of other interfaces with multi-modal interaction, acoustic signals, classic input devices or conventional buttons). Thus, it can be seen as an enabler for future human factors studies, which might follow a different research goal and require a different setup, respectively. In addition, the platform is robust and could be used successfully and without issues across several studies.

With regard to the outcome of the two human factors studies conducted, significant effects of all three design elements investigated could be determined. We interpret the results in such a way that the participants received valuable information via “Feedback”, which resulted in a quicker and more efficient way of finding a solution to the given tasks. The studies led to three significant observations of a faster task processing (Study 1, Type A: $-78$ s; Study 2, Type A: $-94$ s; and Study 2, Type B: $-82$ s) (see Figure 9). Since this effect occurred in both studies and both types of tasks, we assume, that “Feedback” should play a key role in CPPS work design. Besides, in terms of the effect indicators, two $++$ and one + ratings support this conclusion (see Table 6). We also conclude that the differentiation by the color of the work orders (“Usability”) supported the perception of information and thus helped to place the work orders in a meaningful way. Here, several significant effects could be observed: a higher evaluation of satisfaction (Study 1, Type A: +0.682), a shorter task processing (Study 1, Type B: $-34$ s; and Study 2, Type A: $-128$ s), and fewer display touches (Study 2, Type A). These effects have predominantly occurred more intensively in Study 2. We attribute this to the more complex work tasks given in this study. Again, regarding the effect indicators, two $++$ and two + ratings support this finding (see Table 6). Finally, “Assistance system” led to two negative observations: first, it extended the processing time (Study 1, Type A: +66 s) and, second, the satisfaction evaluation was lower (Study 2, Type B: $-1.018$). This resulted in two $--$ ratings in terms of the effect indicators, respectively. Here, we assume that the intended idea of the worker assistance (Study 1: display of error details, Study 2: visual operating support) did not meet the user’s requirements. A possible explanation is that the measures led to confusion.

Other observed significant effects for all design elements, as shown in Figure 9, do not point into a clearly positive or negative direction (“Usability”, Study 1, Type A: +0.450 quality evaluation; Study 1, Type B: +0.520 quality evaluation; $-0.718$ deviation of solution quality; Study 2, Type A: $-0.662$ deviation of solution quality; $-0.852$ difficulty evaluation; “Feedback”, Study 2, Type A: $-0.446$ difficulty evaluation; “Assistance system”, Study 2, Type B: $-0.820$ quality evaluation; and “Feedback”, Study 2, Type B: +0.872 deviation of solution quality). However, they can provide valuable information in combination with other effect observations. We consider the combined analysis of key performance indicators for work performance and work perception to be meaningful and necessary, as significant results could often only be obtained for a part of the dependent variables examined. Consequently, a consideration of a limited number of parameters would possibly lead to an inconsiderate rejection or confirmation of particular design elements. If, for example, a significant result was achieved as part of a work performance measurement, an equivalent effect on the perception of work cannot be assumed necessarily as well. Following this idea, we compared the effect indicators for work performance and work perception to derive the final design principles (see Table 8). Summarizing, we recommend the implementation of the design element “Usability”. For “Feedback”, we recommend a further examination in light of the “Maybe” rule. An implementation of “Assistance System” is not recommended.

These findings need to be considered in the context of the two studies with regard to the representative work tasks and the chosen measures and the limited number and variety of study participants. Thus, we do not claim general applicability to all CPPS work design use cases. Nevertheless, the results indicate effect sizes and directions of work design elements which can serve as starting points for future investigations.

As possible next research steps, we see the widening of the scope for deriving design elements through a focus extension. As mentioned previously, we do not expect the overview to be complete. Here, a broad review of other human factors related areas could reveal further design elements. Besides, we consider a variation of the method to enable a more practical research as very promising. This could be carried out such that the method does not start with a representative work task but with an industrial use case, which then will be transferred into a use case-specific representative task. Since human factors research is often used not only for basis research but also for applied research (e.g., to make design decisions out of several alternatives [45]), this variation could serve as a decision-making tool for SME practitioners. Additionally, a practical evaluation would provide information on a further development of the method as well. Here, experiences and insights could be used in terms of increasing the relevance of the results with regard to basic research and increasing the accessibility of the method with regard to applied research.

Further, collaborative processing of work tasks in the sense of human–machine-networks could open up promising new research possibilities. Here, the establishment of a communication link between the platforms (by using an already available XBee module, see Figure 4) as a basis for the implementation of collaborative tasks could lead to an investigation of design elements in a collaborative environment.

Finally, we see the conduction of further studies as another possible next research step. By using a different set of design elements according to Section 3.1, insights on other design elements could be achieved as well, which were not part of the presented studies. In addition, the use of other measures and representative tasks for the design elements already used in the presented studies could increase the generalizability of the results.

Overall, a set of results could be presented (work design elements, method, experimental platform, and first application results) which prove the functional capability of the procedure and expand the knowledge about human-oriented work design in CPPS. As presented in Section 3.2, the method aims to support the understanding of the mechanisms between human factors and work in CPPS and the design of interfaces between humans and machines. It can assess and evaluate specific work design elements to provide first design principles regarding the interface design in CPPS. We consider this as a valuable step for manufacturing companies to improve the integration process of CPPS ideas. This way, already at an early stage of the design and implementation process, unnecessary or even counterproductive work design measures can be minimized, and beneficial ones maximized instead at an early stage of the design and implementation process. This way, both social and economic sustainability of CPPS can be promoted.