Development of a Change Management Instrument for the Implementation of Technologies†
Work and Organizational Psychology, Faculty of Psychology and Sports Science, Bielefeld University, 33615 Bielefeld, Germany
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB, Industrial Automation Branch, 32657 Lemgo, Germany
Fraunhofer Institute for Mechatronic Systems Design, 33102 Paderborn, Germany
Author to whom correspondence should be addressed.
This paper is an extended version of our paper published in Proceedings of the 11th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2018), Island of Corfu, Greece, 26–29 June 2018.
Technologies 2018, 6(4), 120; https://doi.org/10.3390/technologies6040120
Received: 31 October 2018 / Revised: 22 November 2018 / Accepted: 11 December 2018 / Published: 13 December 2018
(This article belongs to the Special Issue The PErvasive Technologies Related to Assistive Environments (PETRA))
The manufacturing industry is increasingly being dominated by information and communication technology, leading to the development of cyber-physical systems. Most existing frameworks on the assessment of such technological advancements see the technology as a solitary system. However, research has shown that other environmental factors like organizational processes or human factors are also affected. Drawing on the sociotechnical systems approach, future technologies could be evaluated using scenarios of digitized work. These scenarios can help classify new technologies and uncover their advantages and constraints in order to provide guidance for the digital development of organizations. We developed an instrument for evaluating scenarios of digitized work on the relevant dimensions ‘technology’, ‘human’ and ‘organization’ and conducted a quantitative study applying this instrument on three different scenarios (N = 24 subject matter experts). Results show that our instrument is capable of measuring technological, human and organizational aspects of technology implementations and detecting differences in the scenarios under investigation. The instrument’s practical value is significant as it enables the user to compare and quantify scenarios and helps companies to decide which technology they should implement.
The manufacturing industry is increasingly being dominated by information and communication technology [1,2]. This creates a multitude of possibilities for the design of technological systems and enables the leap from mechatronic systems to highly automated cyber-physical systems. These cyber-physical systems have a significant impact on virtually all processes of the industrial value chain, ranging from individual assistance in product engineering and manufacturing [3,4] over organizational process management  to large-scale energy systems such as the smart-grid architecture . Advantages of cyber-physical systems include their interconnectivity, real-time capability, modularity and virtualization of physical processes. This means they can communicate autonomously with each other in real time, be adapted to fast-changing needs and record all data for troubleshooting and future reference . However, a wide variety of possibilities makes it difficult for companies to choose one technological solution that fits their needs and helps them achieve their goals. A premature investment in the wrong technology might result in workers not using the technology, performance goals not being achieved, or the technological infrastructure not being compatible, among other things. It becomes apparent, that a sole focus on the technology is too narrow so that we rather suggest considering scenarios of digitized work. A scenario is defined as a generally understandable description of a possible situation which is based on a complex network of characteristics of influencing factors . Thus, a scenario-based approach gives a deeper understanding of different technologies and highlights interdependencies of factors related to the successful technology implementation. Consequently, the aim of this study is to present an instrument that incorporates all relevant factors of technology scenarios and that companies can use to weigh advantages and disadvantages of technological systems. More specifically, we investigate if the instrument is able to detect significant differences between scenarios of digitized work.
To this day, several approaches of guidelines towards the assessment of (technological) systems were proposed. Notable examples are, among others, the framework for the implementation of additive manufacturing , a framework for the integration of collaborative robots in advanced manufacturing systems  or research on evaluation criteria for digital assistance systems . However, most of the current frameworks in the field of digitization and cyber-physical systems rely heavily on a purely technological approach, neglecting the interdependence of the technology with organizational and human factors in a scenario of digitized work. There are other approaches, such as frameworks on the assessment of human-machine systems  or frameworks for human-centred design  that include human-related and organizational factors but lack the transfer into a practical instrument and have therefore a limited practical value. An application of these frameworks often requires a deep analysis and understanding of the current (manufacturing) system, a process which can be time and resource intensive. In addition, the frameworks usually address specific issues and organizations often do not have a clear overview about the overall benefits of implementing new technologies and processes.
1.2. Need for a Sociotechnical Approach
The theoretical basis for our instrument is the sociotechnical systems approach. It claims that organizations consist of a social and a technical system which are interdependent. The technical system comprises for example tools, procedures and machines that are used in the work process, the social system contains the employees, their attitudes towards the organization and their social relationships . Thus, when implementing a new technology, these changes have an impact on the employees, their work environment and organizational processes . Therefore, in order to reduce the risk of unwanted consequences, the benefit assessment of a planned technology should include technological, human-related and organizational factors. A meta-analysis on change management from the early 1990s already showed that a joint consideration of technological, personnel-related and organizational aspects had a greater effect on financial (e.g., efficiency, error rate) and behavioural (e.g., turnover, absenteeism) performance indicators than changes with a sole focus on one of the dimensions . Aside from being able to estimate a technology’s consequences in a straightforward way, our practical instrument can also support change management to encourage employee participation.
1.3. Benefits in the Change Management Process
Change management refers to approaches and actions that are necessary for the successful design and implementation of change . To give an example, research on the digitization of governmental processes has shown that a wide array of benefits such as more efficient and higher-quality processes can be achieved through technology implementation but require a thorough identification, understanding and most of all involvement of the stakeholders (e.g., the technology users) . In this context, the literature discusses a number of factors and strategies, for example top management support, participation of affected employees, training to prepare for the new processes, incorporation into the business plan and vision and testing of the planned technology [19,20]. Employee participation in the change process is among the most studied success factors. A literature review has found that it is related to higher acceptance of change, a better understanding of the change and its benefits, a greater involvement in implementing behavioural changes, an increased attachment to the organization and lower change-related stress, among other things .
There are three reasons why employee participation is beneficial for the success of a change process. First, having a say during a (change) process is strongly related to perceiving this process to be fair, which is in turn associated with commitment to change and change-supporting behaviour [22,23]. Second, employees who can have an influence during a change process, experience more autonomy. That means they feel that they can align the planned changes in a way that is compatible with their own values . A recent study has found that the experience of autonomy mediates the relationship between participation in a change process and less change-related stress as well as positive behavioural and cognitive change attitudes . Third, the active participation in the change process can be regarded as job crafting. Job crafting refers to actions taken by employees to make their own jobs more meaningful, engaging and satisfying . It is positively related to job satisfaction, organizational commitment and performance [27,28].
Hence, in order to successfully implement a technology, employees need to participate in the process. Participation can be facilitated by letting the affected employees answer the questions from our instrument. Afterwards, the results should be presented to the employees and adaptations and further actions should be discussed together.
1.4. Aim and Contribution of Our Study
The aim of this study is to bridge the gap between theoretical frameworks and practitioners’ need for clear guidance in technology implementation. We developed an instrument to assess scenarios of digitized work on three dimensions: technology, human and organization (for more details see ). The instrument enables practitioners to reflect on consequences of the technology to be implemented and review possible alternative technologies. The current study focuses on the instrument’s ability to detect differences in the scenario characteristics. Therefore, we expect the following.
The scenario characteristics of the scenarios ‘mixed mock-up’, ‘conformity management supported by a graph database’ and ‘digital documentation of the production progress’ differ significantly on the three dimensions human, technology and organization.
In the following chapters, we first describe the instrument and its facets as well as the procedure of our study. Here, we also give a description of the scenarios that were the research objects of this study. Second, we report the results of our study and provide visualizations of the developed instrument. Lastly, we discuss our study’s implications and point to further research directions.
2. Materials and Methods
We developed an instrument for the assessment of scenarios of digitized work. First, we give an outline of the instrument development process and describe the instrument in detail. Second, we apply this instrument to three scenarios, which are described briefly. Lastly, we illustrate how we investigated the three scenarios by using the instrument and describe the study design, the sampling method and the procedure.
2.1. Instrument for the Assessment of Scenarios of Digitized Work Based on Sociotechnical Criteria
In the following, our current work concerning the relevant criteria for scenarios of digitized work will be described in detail. In a first step, the different criteria for the dimensions have been chosen based on thorough literature reviews and expert interviews. In a second step, we aimed to provide a means of quantification or categorization, ideally based on existing questionnaires that addressed the same or similar issues. For this reason, each criterion is represented by one or more factors, which cover the different core aspects of the issue. The current accumulation of criteria that is presented in this paper has been iteratively worked out. An overview of the criteria and the corresponding factors for all three dimensions—technology, human and organization—can be found in Table 1.
2.1.1. Classification and Rating of the Human Dimension
Concerning the classification of criteria for the human factors, one of the main premises of our work has been the fact that with the implementation of new technologies, the characteristics of work have fundamentally changed over the last years . Given that the tasks at hand in the scenarios of digitized work have changed, there has been a shift in the competencies which are required to successfully complete these tasks . For a classification of the required skills, we focused on research by Fleishman and Quaintance , who argued that specified tasks can be characterized by specific competency profiles. Although the categories were not initially tailored to digitized work environments we see the categorization fitting for giving an overview of the nature of the task(s) in the scenario at hand. Following Fleishman and Quaintance, five categories of abilities were used for rating the scenario: cognitive, psychomotor, physical, sensory and social/interpersonal abilities.
A criterion which goes in line with a shift in the prioritization of the abilities is that new work processes often require additional qualification of employees in order to carry out the tasks in a scenario, a factor which has been identified as a critical requirement for the successful implementation of technology . Given that the use of a new technology might require a specific skill set or certain competencies, this criterion is important for organizations since it allows an estimation of the effort that a new technology might bring in terms of workforce development. As for the rating, we have chosen to rate the qualification by the amount of effort that is needed to qualify personnel for the given scenario. In addition, for each scenario, the form of the qualification such as single-day training or workshops over multiple days were specified.
Besides the requirements of the employees, a factor that is crucial for a successful implementation of new technologies is the way the work is designed. Research has found that work design in general and work design of technologies in particular can have a significant impact on individual factors such as autonomy and cognitive ability  or organizational factors of work such as group feedback . To assess the requirements, efforts, uses and constraints of work design, the work-design questionnaire (WDQ)  has been found to be a valid means of rating the scenarios. However, given the scope of our work, we altered the dimensions of the questionnaire to fit our needs for two reasons. Firstly, using the entire scope of the WDQ would surpass our aim to develop an easily accessible tool for rating scenarios of digitized work. Secondly, the rating system of the WDQ is highly subjective, given that it was designed for employees to assess their own work environment. Since our aim is the development of an objective rating scale, some of the items were altered. One of the most important work criteria has found to be the scope of action or autonomy of the work, a criterion which has been linked to the acceptance of technological innovations in organizations . In terms of rating the scope of action of a work environment, we have chosen to adapt the three factors work scheduling, that is, the freedom of individual time management, decision making and work methods, that is, freedom in the choice of how to obtain work results. The second criterion which has been adapted from the WDQ is the possibility of social interaction. Given that factors such as social support are critical factors of well-being at work  and can help in overcoming work overload , the assessment of the work environment should not neglect this. For the rating, the factors social interaction at the workplace, interaction with external parties, interdependent work tasks as well as task independence have been adapted from the social characteristics of the WDQ.
The next criterion that has been extracted from the literature for the assessment of digitized work scenarios is the holistic nature of the task. This refers to the degree to which the job can be seen as a whole with identifiable results . Understanding the holistic nature of the task has been found to increase the attractiveness of a job , classifying this criterion as an important indicator of a scenario of digital work. The rating included the factors influence of the work on other people, holistic nature of the work and automatic performance feedback. Lastly, the criterion of work diversity has been identified for the development of the questionnaire. Work diversity includes the possibilities of using a range of skills, knowledge and competencies that can be brought in during work to reach the employees’ goals . The factors which were included in the questionnaire, cover task variety, task complexity, the amount of information processing, creativity, the variety of skills as well as the variety of tools that are needed to complete the task.
2.1.2. Classification and Rating of the Technological Dimension
Concerning the technological dimension, one of the most important concepts proves to be the criterion of technological maturity . The classification of the technology into one of the four categories of the implementation and usage (introduction, growth, maturity and decline) constitutes a simple but effective means for organizations in having a first and clear classification of the technology.
Another criterion is the compatibility of the technology, indicating its ability to share any type of information across any technology component throughout the organization . In our approach, we chose to rate the different forms of compatibility measured by the following factors: interoperability with mobile devices, interoperability with stationary devices, personalized services, situational services, time-based services, location-based services as well as the possibility of using the technology with different platforms. While compatibility has been identified a key factor in the communication and integration of the technology with other technical devices and systems, the communication between the machine and its human operator is not included in this category. Therefore, the means of (human) interaction with the technology is an additional factor in technology implementation. For the rating, we chose to assess the different means of interaction which can be used in the scenario, leading to the following factors: interaction with information tags (e.g., RFID), interaction through visual codes, direct tactile interaction, direct textual interaction, acoustic interaction as well as gesture recognition.
Another factor we included is the flexibility of the system. Research shows that flexibility of IT systems is a necessity for (rapid) changes in a business environment and the effective implementation of new technologies . Concerning the quantification of flexibility, Duncan  identified three measurable key factors: The compatibility (the ability to share any type of information across any technology), the connectivity (the ability of the technology to communicate with other components in- and outside of the organization) and the modularity (the ability of the technology and its components to be reconfigured). Furthermore, research led by Schmitt emphasizes that low technological complexity may be positively related to successful technology implementation . Concerning the assessment of complexity, it has been proven difficult to apply a quantitative means of measurement to this concept, given that for the broad range of currently used technology and IT systems, a single definition of the system’s complexity has not yet been established . In an attempt to grasp the concept of complexity, we identified the three factors (inter)connectivity of the components , degree of understanding and active participation  and specialization of the technology  for the assessment in our questionnaire.
Another important criterion is the monetary effort of technology implementation. Concerning the cost of a technology, we can distinguish between the one-time initial invest in the technology, the effort of implementation in terms of personnel as well as the cost of maintenance of the technology. While the initial invest was easily quantifiable, since every technology comes with a defined price, the effort of implementation was more difficult to grasp. Given that for each organization, the amount of available personnel, the cost of personnel as well as the competencies can differ, no clear monetary value can be given for this criterion. However, we adapted a scale used in research by Gruber and colleagues  who distinguished the effort of technology implementation in four different stages: no need for additional tools, need for simple installation and additional tools, need for implementing new (complex) system components, need for a restructuration of the system. For the costs of maintenance of a technology, we tried to assess the effort of maintenance which is needed concerning the factors overhead of the organization, system maintenance, upgrades, system management and employee training.
Lastly, we identified the safety & security of the technology. In this case, safety is defined as the protection of the environment and the user from an object. In terms of technologies, this includes occupational safety, electrostatic discharge protection and the labelling of potentially hazardous elements . For the rating of the criterion, we adapted a classification used by Saleh , who identified three main factors of safety: training of employees, supervision of the work environment and implementation of security technologies. The security of the technology goes in line with the safety of the system. With a heavier focus on the physical workspace, security addresses the protection of an information technology from its environment. Several methods of quantitative measurement of IT security have been found in the literature . Nevertheless, these measurements often require a deep and narrow understanding of the technology at hand as well as rather complex methods, making them unsuitable for being used in our questionnaire. Instead, we chose to use the factors training of employees, supervision of the work environment and implementation of security technologies by Saleh  and adapted them to the criterion of security.
2.1.3. Classification and Rating of the Organizational Dimension
In addition to the evaluation based on technological and human criteria, the organizational dimension is essential for the assessment of scenarios of digitized work. Significant for this dimension is the general categorization of the scenario in regard to its position in the product development process. The corresponding stages are strategic product planning, product development, service development and production system development .
For companies, the criterion promotion of innovation can be decisive for the introduction of a scenario of digitized work. In order to assess the benefits for the promotion of innovation, the scenario is analysed in terms of the factors product orientation and customer orientation . Furthermore, we included the reduction of time-to-market as a criterion because it helps reduce costs and product development time. For the assessment of this category, we included the following factors that can contribute to a reduced time-to-market: process control, process optimization, infrastructural changes and strengthening of interdisciplinary collaborations .
Labour cost estimates play a central role in project management . The success or failure of project work, for example within the scope of a technology implementation, is decisively dependent on them. Factors to estimate the criterion labour expenditures are the number of different activities and their complexity for execution, number of participants, training period and necessity of geographically distributed persons. Besides labour expenditures to execute scenarios of digitized work there is a need for specific competencies and expert knowledge. This criterion is required for both technological implementation and execution. The results of the Competency Development Study 2016 divides the competencies that are crucial for companies to implement digitization and interconnectedness into three areas: technology and data oriented (e.g., IT security, artificial intelligence, data science), process-/customer-oriented (e.g., process management, systems engineering, eCommerce) and infrastructure-/organization-oriented (e.g., network/database administration, protection of privacy, dealing with specific IT systems) .
An indicator for assessing the complexity of a technology implementation is its influence on the process organization. We identified the following three factors to describe the process organization: adjacent/related processes, organizational units and customers or suppliers.
The degree of collaboration describes the distribution of decision-making powers within an enterprise or its subunits. This can influence the company’s policy and strategy and is therefore important from an organizational point of view. Weaver classifies this in cooperation, coordination, collaboration and coexistence . For our instrument, we used a qualitative appraisal of the two factors organizational structure and mutual management. A scenario’s degree of collaboration results from the interaction of these two factors leading to one of the four former mentioned ways of collaborating, for example, a scenario with very strict organizational structures and no mutual management results in coordination as opposed to a scenario with flat hierarchies and a high amount of mutual management, which results in collaboration. Similar to the degree of collaboration, the degree of the blurring of boundaries of work can influence the operational organization by dissolution of ‘traditional’ structures. Factors are decentralisation  working time flexibility and project flexibility . Lastly, agility is becoming more and more important for product development in order to meet the rapidly changing requirements of customers and the system context. For the assessment of scenarios of digitized work, the influence on agility is therefore significant. For this criterion, the factors customer orientation and participation, transparency, breaking up disciplinary boundaries and decentralisation of decision making have been identified as measurable indicators for agile projects .
2.2. Scenarios Under Investigation
We assessed three different scenarios with the instrument described above. The scenarios each describe the technology, its functionalities and tasks that are supported by the technology. Next, the process while using the technology is described, beginning with the initial situation, different phases and a final situation. The users’ tasks during the different phases are described. In the following, we describe the three scenarios under investigation. In our study, we presented the scenarios in German and translated them into English for the purpose of this paper.
2.2.1. Mixed Mock-up
While testing new assembly systems, cardboard installations (so-called mock-ups) are used. This enables the employees to contribute their implicit knowledge in interdisciplinary workshops. However, some machine parts are not available before the workshop begins and the system is tested.
Due to the connection of construction data in 3D and physical mock-up by means of augmented reality (AR), the construction status of products and appliances is up-to-date and information is readily available. Because of this, the conception of the production system can begin much earlier. This shortens the building and test phases of the mock-up significantly. The visual simulation of the workplaces becomes more realistic for the employees by showing different machine parts by means of the AR technology. The employees’ task is to join the parts together to a workpiece which are displayed via AR technology and to reflect if the installation of the assembly system can be optimized.
Initial situation: A group of three to 20 interdisciplinary agents are located in a room with the necessary infrastructure (physical mock-up, AR hardware). Remote agents can connect themselves with the group via other devices.
Phase 1: The virtual prototype/parts are projected by means of an AR system onto the prototypical assembly workplace. The main agent gathers the virtual part and afterwards checks the removal process concerning ergonomics, efficiency and so forth. The other agents can track the process on additional terminals and discuss it.
Phase 2: The main agent assembles the virtual part and validates the assembly process. The other agents can track the process on additional terminals and discuss necessary changes of the assembly system.
Phase 3: The assembled component is handed over to the following assembly station. The tests of the removal and assembly processes of the parts is repeated for all stations of the assembly line, using the AR technology.
Final situation: The production system is designed and can be handed over to the department of system integration.
2.2.2. Conformity Management Supported by a Graph Database
A graph database makes it possible to test products and materials regarding specific requirements even without the expert knowledge of the employees. On the one hand, products, including components and substances from an existing system (e.g., product data management) are added to a graph database by the employees themselves, on the other hand there are all relevant standards, guidelines and so forth.
For example, the system enables employees to answer customer inquiries about the approval of products with regard to the conformity relating to specific regional requirements, such as the automatic test of a maximally allowed amount of a certain substance. The automatic test allows the output of a document on qualified conformity management. The task of the responsible employees is to maintain the database and to check the designed components before starting production.
Initial situation: It starts with a customer requesting information, whether the supplied and installed product meets the requirements of a specific country where the customer wants to sell his or her product. The graph database is filled with data about the organization’s products as well as relevant norms and so forth of the main markets of the customers.
Phase 1: An employee adds the specific standards, guidelines and so forth of the target country to the graph database. The employee transfers the customer’s request to the service system.
Phase 2: The system carries out an automated comparison of the product with the target market requirements and gives the employee an overview of the fulfilled or unfulfilled requirements including a list of the reasons (e.g., positive laboratory test or exceeding the concentration of lead).
Final situation: Positive case (the product meets all target market requirements): the employee initiates an automatic output in form of a PDF file with his digital signature and sends it to the customer to confirm the conformity. Negative case (the product does not meet all target market requirements): the employee starts a process to check the product with regard to the unfulfilled requirements.
2.2.3. Digital Documentation of the Production Progress
The documentation of the production progress in a manufacture (batch size 1) is carried out individually according to the customer’s specifications. Digital documentation enables the paperless recording of the production status. In this case, the production process is individual for each product. This enables the customer to follow the progress of the production of his or her product in real time. In the case of proposed changes, the customer can contact the producer before every step and communicate the wanted changes (e.g., colour request before painting).
The employees have to confirm the completed steps on a mobile device using a digital signature. Further information and annotations can be realized for example via a ’Voice to Text’ interface. The use of the technology shortens the process of documentation of the production progress for employees.
Initial situation: The production process as well as the documentation templates are defined and implemented in the application. The workspaces in the production are equipped with appropriate input/output devices and are interconnected. There is an inquiry from a customer about the production progress and the manufacturing process so far.
Phase 1: By accessing the database, the service employee can view the digitally documented progress and quality features of the product in real time. Optionally, there is a direct access for the customer. During production, the employees document the progress digitally. For this purpose, they sign off the completed work steps using digital signatures (for example by scanning their ID). Further quality features can be recorded in comments, for example via ’Voice to Text’ (e.g., measured values).
Final situation: The service employee initiates an automated output in form of a PDF file with the current data records and sends it to the customer.
2.3. Study Design
The study using a cross-sectional design was realized by means of an online questionnaire which was created with the online survey tool Survey Monkey. An invitation containing a link was sent to subject matter experts from the field of work psychology, technology and business via email. Due to work experience in interdisciplinary projects these experts have a holistic view over the relevant fields. The study was conducted in October 2018 in Germany. Twenty-four subject matter experts took part in the study. To examine the inter-rater agreement the intraclass correlation was calculated. Over all three dimensions the raters corresponded with ICC = 0.94. For the single dimensions the agreement was ICC = 0.95 for the human dimension, ICC = 0.96 for the technological dimension and ICC = 0.87 for the organizational dimension. According to Chicchetti , these values indicate an excellent agreement of the raters on all three dimensions.
In order to compare the scenarios, each participant was required to create a code or received a code in their invitation to take part in the study. The participants first read the first scenario and then rated it using the instrument which was programmed as an online survey. Next, they read the second scenario and proceeded as described above, rating all three scenarios. First, the human dimension was rated, followed by the technical and organizational dimensions. The questionnaire contained 104 items in total. All criteria were measured by scales containing one to sixteen items. The statistical analysis of the criterion compatibility was conducted by using a sum value of all items. For all other criteria the mean of all items was calculated. For an overview of the criteria, sample items, number of items and the scaling see Table 2. Data were analysed using SPSS (Version 24). Computer code (syntax) is available upon request.
Table 3 presents the means and standard deviations of all criteria on the dimensions human, technology and organization separated for the three scenarios mixed mock-up (MMU), conformity management supported by a graph database (CMGD) and digital documentation of the production progress (DDPP). Furthermore, Table 3 shows findings of the analysis of variance that was conducted to compare the specific scenario data. On all three dimensions we found statistically significant differences between the scenarios on at least two criteria. These findings support our hypothesis that the scenario characteristics of the scenarios mixed mock-up, conformity management supported by a graph database and digital documentation of the production progress differ significantly on the three dimensions human, technology and organization. In the following we will present the detailed results for each dimension separately.
3.1. Results Human Dimension
On the human dimension we found significant differences between the three scenarios on the following criteria: psychomotor abilities (F(2) = 30.61; p = 0.00), physical abilities (F(2) = 17.20; p = 0.00), sensory abilities (F(2) = 21.93; p = 0.00), social abilities (F(2) = 4.69; p = 0.02) and work diversity (F(2) = 9.99; p = 0.00). As shown in Figure 1, all criteria were rated highest in the MMU scenario. Regarding the need for psychomotor abilities, the difference between the MMU scenario and the CMGD scenario (t(19) = 7.32, p = 0.00) and the DDPP scenario (t(18) = 6.30, p = 0.00) was significant. The same applied to the need for physical abilities, CMGD scenario (t(19) = 5.81, p = 0.00), DDPP scenario (t(18) = 4.44, p = 0.00). The rating for the need for sensory abilities also differed significantly between the MMU scenario and the CMGD scenario (t(19) = 7.26, p = 0.00) and the DDPP scenario (t(18) = 4.40, p = 0.00). Here, we also found that the need for sensory abilities was significantly higher in the DDPP scenario than in the CMGD scenario (t(17) = 2.56, p = 0.02). The need for social abilities was again rated significantly higher in the MMU scenario compared to the CMGD scenario (t(19) = 2.32, p = 0.03) and the DDPP scenario (t(19) = 2.26, p = 0.04). The same applied to the rating of work diversity, CMGD scenario (t(19) = 2.87, p = 0.01), DDPP scenario (t(19) = 5.24, p = 0.00).
3.2. Results Technological Dimension
On the technological dimension we found significant differences between the three scenarios on the following criteria: compatibility (F(2) = 5.82; p = 0.01), means of interaction (F(2) = 5.83; p = 0.01) and safety (F(2) = 4.70; p = 0.02). As shown in Figure 2a,b, compatibility and means of interaction were rated highest in the DDPP scenario. As can be seen in Figure 2c, safety was rated highest in the MMU scenario. The compatibility was rated significantly lower in the CMGD scenario than in the MMU (t(17) = −2.16, p = 0.05) and the DDPP (t(16) = −3.70, p = 0.00). The same applied to the criterion safety, MMU scenario (t(18) = −3.43, p = 0.00), DDPP scenario (t(18) = −2.55, p = 0.02). Means of interaction was rated significantly higher in the DDPP scenario than in the CMGD scenario (t(17) = 4.32, p = 0.00).
3.3. Results Organizational Dimension
On the organizational dimension we found significant differences between the three scenarios on the following criteria: reduction of time-to-market (F(2) = 9.27; p = 0.00) and degree of collaboration (F(2) = 7.30; p = 0.00). As shown in Figure 3, ratings on both criteria were highest in the MMU scenario. Reduction of time-to-market was rated significantly lower in the CMGD scenario than in the MMU scenario (t(17) = −3.98, p = 0.00) and the DDPP scenario (t(17) = −4.02, p = 0.00). Regarding the degree of collaboration, the difference between the MMU scenario and the CMGD scenario (t(19) = 4.62, p = 0.00) and the DDPP scenario (t(18) = 2.24, p = 0.04) was significant.
In this article, we presented and tested an instrument for the assessment of technologies that incorporates criteria from the dimensions human, technology and organization. In an empirical study with subject matter experts as participants, we investigated if our instrument was capable of detecting differences between three distinct scenarios of digitized work. The results support our hypothesis and show that the ratings of a number of criteria differed between the three investigated technology scenarios.
Most existing frameworks for the assessment of technologies have a sole focus on technological criteria. Yet, our study illustrates that a sociotechnical approach is necessary because the scenarios of digitized work differed not only in technological criteria like compatibility but also in human-related criteria like need for sensory abilities and organizational criteria like reduction of time-to-market. Our results are in line with previous research that has found that change initiatives have a more positive outcome when technological, personnel-related and organizational aspects are jointly considered .
4.1. Limitations and Future Research
As the instrument is at an early stage of development and this is the first quantitative study applying it to scenarios of digitized work, it was not possible to draw conclusions concerning the validity of the instrument. Further research should investigate the instrument’s validity comprehensively by correlating it to relevant external criteria, such as job satisfaction or health (for the human dimension), increase in efficiency (for the technological dimension) and innovation (for the organizational dimension). Moreover, we suggest to also include change-related constructs such as change readiness, technology acceptance and technology affinity as these constructs are predictors of successful technology implementations [62,63]. For a comprehensive validation, our sample size is not sufficient; however, we were able to show that the instrument is able to detect differences with regard to different aspects of technologies.
Moreover, the scenarios were rated by subject matter experts rather than future users of the instruments. In contrast to users in companies, the subject matter experts had no compulsory aim of implementing these technologies at their workplace which does not correspond to the future use of the instrument. However, this is also an advantage as this allowed the experts to rate the scenarios in an un-biased way and independently. Also, this allowed us to select a variety of scenarios, as we were not restricted to specific aims.
In this study we used a within-subjects design. Although the order of scenario evaluation was not specified, the research data showed that almost all experts evaluated the scenarios in the same order. To exclude sequence effects we suggest choosing a between-subjects design in future studies.
In future, studies should be conducted in which our instrument is applied in real companies. The results support decision-making on which technology should be implemented. These application studies should also investigate the impact of the technology, ideally on pure technological outcomes, as well as on human and organizational aspects.
4.2. Practical Implications
Our instrument helps organizations to assess alternative technologies. Due to the fact that all relevant characteristics can be compared at a glance, it is easier to weigh the alternatives. For example, if one technology scenario was rated higher in terms of promotion of innovation but lower in agility, the organization could choose a technology based on which outcome was more important. Thus, the instrument facilitates and structures the decision making process.
Furthermore, the prospective nature of the instrument supports the work of the human resources department. Knowing about human-related consequences of a technology before its implementation gives the human resources department the chance to prepare employees for the upcoming changes. For example, they might need a specific training or their job tasks need to be restructured. Moreover, the instrument can be used to support change initiatives. By letting the affected employees fill out the questionnaire, they feel more involved in the change process and will more likely accept it.
Due to the fact that a growing number of work processes is supported by technologies like cyber-physical systems, it has become increasingly difficult for organizations to select one technology that helps them achieve their goals. Therefore, we have developed an easily applicable instrument for the assessment and comparison of alternative technology scenarios. Our study found that the instrument serves its purpose in detecting differences between distinct scenarios and is therefore a valuable tool during digital change endeavours.
Conceptualization, L.M., A.P., D.B., S.J., M.F., M.B., L.W., C.R. and G.W.M.; Data curation, D.B.; Formal analysis, D.B.; Funding acquisition, A.P., M.B., C.R. and G.W.M.; Investigation, L.M., A.P. and D.B.; Methodology, L.M., A.P., D.B. and S.J.; Project administration, L.M.; Supervision, C.R. and G.W.M.; Validation, L.M., A.P. and D.B.; Visualization, D.B.; Writing—original draft, L.M., A.P., D.B. and S.J.; Writing—review & editing, L.M., A.P., D.B., S.J., M.F., M.B., L.W., C.R. and G.W.M.
This research was funded by Bundesministerium für Bildung und Forschung and European Social Fund, grant number 01FI16133.
We acknowledge support for the Article Processing Charge by the Deutsche Forschungsgemeinschaft and the Open Access Publication Fund of Bielefeld University.
Conflicts of Interest
The authors declare no conflict of interest. The funding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript and in the decision to publish the results.
- Dubey, R.; Gunasekaran, A.; Childe, S.J.; Wamba, S.F.; Papadopoulos, T. The impact of big data on world-class sustainable manufacturing. Int. J. Adv. Manuf. Technol. 2016, 84, 631–645. [Google Scholar] [CrossRef]
- Rosen, R.; Wichert, G.; von Lo, G.; Bettenhausen, K.D. About the importance of autonomy and digital twins for the future of manufacturing. IFAC-PapersOnLine 2015, 48, 567–572. [Google Scholar] [CrossRef]
- Peng, D.X.; Heim, G.R.; Mallick, D.N. Collaborative product development: The effect of project complexity on the use of information technology tools and new product development practices. Prod. Oper. Manag. 2014, 23, 1421–1438. [Google Scholar] [CrossRef]
- Wang, L.; Törngren, M.; Onori, M. Current status and advancement of cyber-physical systems in manufacturing. J. Manuf. Syst. 2015, 37, 517–527. [Google Scholar] [CrossRef]
- Schuh, G.; Rozenfeld, H.; Assmus, D.; Zancul, E. Process oriented framework to support PLM implementation. Comput. Ind. 2008, 59, 210–218. [Google Scholar] [CrossRef]
- SmartGrid.gov. Available online: https://www.smartgrid.gov/ (accessed on 26 October 2018).
- Hermann, M.; Pentek, T.; Otto, B. Design principles for industrie 4.0 scenarios: A literature review. In Proceedings of the 49th Hawaii International Conference on System Sciences (HICSS), Koloa, HI, USA, 5–8 January 2016; Bui, T.X., Sprague, R.H., Jr., Eds.; Conference Publishing Services: Los Alamitos, CA, USA, 2016; pp. 3928–3937. [Google Scholar]
- Fink, A.; Siebe, A. Szenario-Management. Von Strategischem Vorausdenken zu Zukunftsrobusten Entscheidungen; Campus Verlag: Frankfurt, NY, USA, 2016; ISBN 9783593506036. [Google Scholar]
- Mellor, S.; Hao, L.; Zhang, D. Additive manufacturing: A framework for implementation. Int. J. Prod. Econ. 2014, 149, 194–201. [Google Scholar] [CrossRef][Green Version]
- Djuric, A.M.; Urbanic, R.J.; Rickli, J.L. A framework for collaborative robot (CoBot) integration in advanced manufacturing systems. SAE Int. J. Mater. Manuf. 2016, 9. [Google Scholar] [CrossRef]
- Hold, P.; Erol, S.; Reisinger, G.; Sihn, W. Planning and evaluation of digital assistance systems. Procedia Manuf. 2017, 9, 143–150. [Google Scholar] [CrossRef]
- Stowers, K.; Oglesby, J.; Sonesh, S.; Leyva, K.; Iwig, C.; Salas, E. A framework to guide the assessment of human-machine systems. Hum. Factors 2017, 59, 172–188. [Google Scholar] [CrossRef]
- Boy, G.A. The Handbook of Human-Machine Interaction. A Human-Centered Design Approach; CRC Press: Boca Raton, FL, USA, 2017; ISBN 9780754675808. [Google Scholar]
- Winter, S.; Berente, N.; Howison, J.; Butler, B. Beyond the organizational ‘container’: Conceptualizing 21st century sociotechnical work. Inf. Organ. 2014, 24, 250–269. [Google Scholar] [CrossRef]
- Jede, A.; Teuteberg, F. Understanding socio-technical impacts arising from Software-as-a-Service usage in companies. Bus. Inf. Syst. Eng. 2016, 58, 161–176. [Google Scholar] [CrossRef]
- Macy, B.A.; Izumi, H. Organizational change, design, and work innovation: A meta-analysis of 131 North American field studies—1961–1991. In Research in Organizational Change and Development: An Annual Series Featuring Advances in Theory, Methodology and Research; Woodman, R.W., Pasmore, W.A., Eds.; JAI Press: Greenwich, CT, USA, 1993; pp. 235–313. ISBN 1559385391. [Google Scholar]
- Cawsey, T.F.; Deszca, G.; Ingols, C. Organizational Change. An Action-Oriented Toolkit, 3rd ed.; Sage Publications: Los Angeles, CA, USA, 2016; ISBN 9781483359304. [Google Scholar]
- Leão, H.; Canedo, E. Best practices and methodologies to promote the digitization of public services citizen-driven: A systematic literature review. Information 2018, 9, 197. [Google Scholar] [CrossRef]
- Finney, S.; Corbett, M. ERP implementation: A compilation and analysis of critical success factors. Bus. Process. Manag. J. 2007, 13, 329–347. [Google Scholar] [CrossRef]
- Todnem By, R. Organisational change management: A critical review. J. Chang. Manag. 2005, 5, 369–380. [Google Scholar] [CrossRef]
- Oreg, S.; Vakola, M.; Armenakis, A. Change recipients’ reactions to organizational change: A 60-year review of quantitative studies. J. Appl. Behav. Sci. 2011, 47, 461–524. [Google Scholar] [CrossRef]
- Cohen-Charash, Y.; Spector, P.E. The role of justice in organizations: A meta-analysis. Organ. Behav. Hum. Decis. Process. 2001, 86, 278–321. [Google Scholar] [CrossRef]
- Michel, A.; Stegmaier, R.; Sonntag, K. I scratch your back—You scratch mine. Do procedural justice and organizational identification matter for employees’ cooperation during change? J. Chang. Manag. 2010, 10, 41–59. [Google Scholar] [CrossRef]
- Deci, E.L.; Olafsen, A.H.; Ryan, R.M. Self-determination theory in work organizations: The state of a science. Annu. Rev. Organ. Psychol. Organ. Behav. 2017, 4, 19–43. [Google Scholar] [CrossRef]
- Schlicher, K.D.; Helling, R.; Maier, G.W. Need Satisfaction Is the Key: How Organizational Support Influences Positive Change Attitude during Information Technology Implementation. Unpublished Work.
- Demerouti, E. Design your own job through job crafting. Eur. Psychol. 2014, 19, 237–247. [Google Scholar] [CrossRef]
- Leana, C.; Appelbaum, E.; Shevchuk, I. Work process and quality of care in early childhood education: The role of job crafting. Acad. Manag. J. 2009, 52, 1169–1192. [Google Scholar] [CrossRef]
- Wrzesniewski, A.; Dutton, J.E. Crafting a job: Revisioning employees as active crafters of their work. Acad. Manag. Rev. 2001, 26, 179–201. [Google Scholar] [CrossRef]
- Jenderny, S.; Foullois, M.; Kato-Beiderwieden, A.-L.; Bansmann, M.; Wöste, L.; Lamß, J.; Maier, G.W.; Röcker, C. Development of an instrument for the assessment of scenarios of work 4.0 based on socio-technical criteria. In Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference on–PETRA’18, Corfu, Greece, 26–29 June 2018; ACM Press: New York, NY, USA, 2018; pp. 319–326. [Google Scholar]
- Bundesministerium für Arbeit und Soziales. Weißbuch Arbeiten 4.0; Bundesministerium für Arbeit und Soziales, Abteilung Grundsatzfragen des Sozialstaats, der Arbeitswelt und der sozialen Marktwirtschaft: Berlin, Germany, 2016.
- Cascio, W.F.; Montealegre, R. How technology is changing work and organizations. Annu. Rev. Organ. Psychol. Organ. Behav. 2016, 3, 349–375. [Google Scholar] [CrossRef]
- Fleishman, E.A.; Quaintance, M.K. Taxonomies of Human Performance: The Description of Human Tasks; Academic Press: Orlando, FL, USA, 1984; ISBN 978-0122604508. [Google Scholar]
- Rauner, F.; Rasmussen, L.; Corbett, J.M. The social shaping of technology and work: Human centred computer integrated manufacturing systems. AI Soc. 1988, 2, 47–61. [Google Scholar] [CrossRef]
- Morgeson, F.P.; Delaney-Klinger, K.; Hemingway, M.A. The importance of job autonomy, cognitive ability, and job-related skill for predicting role breadth and job performance. J. Appl. Psychol. 2005, 90, 399–406. [Google Scholar] [CrossRef] [PubMed]
- Jong, J. The role of performance feedback and job autonomy in mitigating the negative effect of role ambiguity on employee satisfaction. Public Perform. Manag. Rev. 2016, 39, 814–834. [Google Scholar] [CrossRef]
- Morgeson, F.P.; Humphrey, S.E. The Work Design Questionnaire (WDQ): Developing and validating a comprehensive measure for assessing job design and the nature of work. J. Appl. Psychol. 2006, 91, 1321–1339. [Google Scholar] [CrossRef] [PubMed]
- Sun, H.; Zhang, P. The role of moderating factors in user technology acceptance. Int. J. Hum.-Comput. Stud. 2006, 64, 53–78. [Google Scholar] [CrossRef][Green Version]
- Wrzesniewski, A.; Dutton, J.E.; Debebe, G. Interpersonal sensemaking and the meaning of work. Res. Organ. Behav. 2003, 25, 93–135. [Google Scholar] [CrossRef]
- Ulich, E. Arbeitspsychologie; Schäffer-Poeschel: Stuttgart, Germany, 2011; ISBN 9783791030494. [Google Scholar]
- Sims, H.P., Jr.; Szilagyi, A.D.; Keller, R.T. The measurement of job characteristics. Acad. Manag. J. 1976, 19, 195–212. [Google Scholar] [CrossRef]
- Hackman, R.; Oldham, G.R. Motivation through the design of work: Test of a theory. Organ. Behav. Hum. Perform. 1976, 16, 250–279. [Google Scholar] [CrossRef]
- Roussel, P.A. Technological maturity proves a valid and important concept. Res. Manag. 1984, 27, 29–34. [Google Scholar] [CrossRef]
- Duncan, N.B. Capturing flexibility of information technology infrastructure: A study of resource. J. Manag. Inf. Syst. 1995, 12, 37–57. [Google Scholar] [CrossRef]
- Byrd, T.A.; Turner, D.E. An exploratory analysis of the value of the skills of IT personnel: Their relationship to IS infrastructure and competitive advantage. Decis. Sci. 2001, 32, 21–47. [Google Scholar] [CrossRef]
- Schmitt, P.; Fleisch, E.; Boutellier, R. Adoption und Diffusion Neuer Technologien am Beispiel der Radiofrequenz-Identifikation (RFID). Ph.D. Thesis, ETH Zürich, Zürich, Switzerland, 2008. [Google Scholar]
- Broekel, T. Measuring Technological Complexity-Current Approaches and a New Measure of Structural Complexity. Available online: https://arxiv.org/pdf/1708.07357.pdf (accessed on 29 October 2018).
- Dolata, U. Soziotechnischer Wandel, Nachhaltigkeit und politische Gestaltungsfähigkeit. In Nachhaltigkeit als radikaler Wandel; Lange, H., Ed.; Vs. Verlag für Sozialwissenschaften: Wiesbaden, Germany, 2008; pp. 261–286. ISBN 9783531150932. [Google Scholar]
- Rammert, W.; Schulz-Schaeffer, I. Technik und Handeln-Wenn Soziales Handeln Sich auf Menschliche Artefakte Verteilt. 2002. Available online: https://www.uni-due.de/imperia/md/content/soziologie/technikundhandeln.pdf (accessed on 29 October 2018).
- Gruber, A.; von Roon, S.; Pellinger, C. Lastflexibilisierung in der Industrie in Konkurrenz zu Weiteren Funktionalen Speichern. Available online: https://www.ffegmbh.de/download/veroeffentlichungen/339_Vortrag_VDI_Lastflex_und_Speicher/20130319_VDI Expertenforum.pdf (accessed on 29 October 2018).
- Hansson, S.O. Risk and Safety in Technology. In Philosophy of Technology and Engineering Sciences; Gabbay, D., Thagard, P., Woods, J., Eds.; Elsevier: Amsterdam, The Netherlands, 2009; pp. 1069–1102. ISBN 9780444516671. [Google Scholar]
- Saleh, M.S.; Alfantookh, A. A new comprehensive framework for enterprise information security risk management. Appl. Comput. Inf. 2011, 9, 107–118. [Google Scholar] [CrossRef]
- Dacier, M.; Deswarte, Y.; Kaâniche, M. Models and tools for quantitative assessment of operational security. In Proceedings of the 12th International Information Security Conference (IFIP/SEC’96), Island of Samos, Greece, 21–24 May 1996; pp. 177–186. [Google Scholar]
- Gausemeier, J.; Anacker, H.; Czaja, A.; Waßmann, H.; Dumitrescu, R. Auf dem Weg zu intelligenten technischen Systemen. In 9. Paderborner Workshop Entwurf mechatronischer Systeme; Gausemeier, J., Dumitrescu, R., Rammig, F.-J., Schäfer, W., Trächtler, A., Eds.; Heinz Nixdorf Institut: Paderborn, Germany, 2013; pp. 11–47. [Google Scholar]
- Rammer, C.; Weißenfeld, B. Innovationsverhalten der Unternehmen in Deutschland 2006. Aktuelle Entwicklungen und ein Internationaler Vergleich; Studien zum Deutschen Innovationssystem 04-2008. 2008. Available online: https://www.e-fi.de/fileadmin/Studien/StuDIS_4_2008_Innovation.pdf (accessed on 30 October 2018).
- Hoff, A. “Back to the Roots”: Vor der 4. Welle der Arbeitszeitflexibilisierung. Personalwirtschaft, 2006; 17–19. [Google Scholar]
- Aufwandsschätzung und Schätzverfahren. Available online: https://www.projektmanagementhandbuch.de/handbuch/projektplanung/aufwandsschaetzung-und-schaetzverfahren/ (accessed on 30 October 2018).
- Acatech-Deutsche Akademie der Technikwissenschaften. Kompetenzentwicklungsstudie Industrie 4.0—Erste Ergebnisse und Schlussfolgerungen. 2016. Available online: https://www.acatech.de/wp-content/uploads/2018/03/acatech_DOSSIER_neu_Kompetenzentwicklung_Web.pdf (accessed on 30 October 2018).
- Weaver, B. Coordination, Cooperation, and Collaboration: Defining the C3 Framework. Ph.D. Thesis, Bryant University, Smithfield, RI, USA, 2012. [Google Scholar]
- Ittermann, P.; Niehaus, J.; Hirsch-Kreinsen, H. Arbeiten in der Industrie 4.0–Trendbestimmungen und Arbeitspolitische Handlungsfelder; No. 308. Düsseldorf, Germany, 2015. Available online: https://www.boeckler.de/pdf/p_study_hbs_308.pdf (accessed on 29 October 2018).
- Nedbal, D.; Auinger, A.; Hochmeier, A. Addressing transparency, communication and participation in enterprise 2.0 projects. Proc. Technol. 2013, 9, 676–686. [Google Scholar] [CrossRef]
- Cicchetti, D.V. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol. Assess. 1994, 6, 284–290. [Google Scholar] [CrossRef]
- Kwahk, K.-Y.; Lee, J.-N. The role of readiness for change in ERP implementation: Theoretical bases and empirical validation. Inf. Manag. 2008, 45, 474–481. [Google Scholar] [CrossRef]
- Hwang, Y. Investigating enterprise systems adoption: Uncertainty avoidance, intrinsic motivation, and the technology acceptance model. Eur. J. Inf. Syst. 2005, 14, 150–161. [Google Scholar] [CrossRef]
Figure 1. Means of the criteria on the human dimension with statistically significant differences between the three scenarios.
Figure 2. Means of the criteria (a) compatibility, (b) means of interaction and (c) safety on the technological dimension with statistically significant differences between the three scenarios.
Figure 3. Means of the criteria on the organizational dimension with statistically significant differences between the three scenarios.
Table 1. Overview of the different criteria and the corresponding factors for the dimensions human, technology and organization.
|Prioritization of abilities||Cognitive|
|Qualification of employees||Amount of effort|
Form of qualification
|Autonomy/scope of action||Work scheduling|
|(Possibility) of social interaction||Social interaction at the workplace|
Interaction with external parties
Interdependent work tasks
|Holistic nature of the task||Influence of the work on others|
Holistic nature of the work
Automatic performance feedback
|Work diversity||Task variety|
Amount of information processing
Variety of skills
Variety of tools
|Technological maturity||Category of technological maturity (introduction, growth, maturity, decline)|
|Means of (human) interaction||Information tags|
Direct tactile interaction
Direct textual interaction
|Complexity||(Inter)connectivity of the components|
Degree of understanding and active participation
Specialization of the technology
|Effort of implementation||Implementation effort|
|Cost of maintenance||Overhead|
|Safety||Training of employees|
Supervision of the work environment
Implementation of security technologies
|Security||Training of employees|
Supervision of the work environment
Implementation of security technologies
|Product development process||Stage of the product development process (strategic product planning, product development, service development, production system development)|
|Promotion of innovation||Product orientation|
|Reduction of time-to-market||Process control|
|Labour expenditures||Number of different activities|
Complexity of activities
Number of participants
Necessity of distributed persons
|Need for competencies/expert knowledge||Technology-/data-oriented|
|Influence on process organization||Adjacent/related processes|
|Degree of collaboration||Organizational structure|
|Degree of blurring of boundaries of work||Decentralisation|
Working time flexibility
|Agility||Customer orientation and participation|
Breaking up disciplinary boundaries
Table 2. Overview of used measures.
|Criterion||Sample Item||No. of Items||Scale|
|Prioritization of abilities||To what extent are the following capabilities used in the scenario?||5||4-point Likert scale|
|Qualification of employees||There is a need for qualification.||2||4/5-point Likert scale|
|Autonomy/scope of action||The work can be planned independently.||3||4-point Likert scale|
|(Possibility) of social interaction||The exchange with other people in the workplace is given.||4||4-point Likert scale|
|Holistic nature of the task||The result of the work has a great impact on other people.||3||4-point Likert scale|
|Work diversity||A lot of information is processed at work.||6||4-point Likert scale|
|Technological maturity||The technology used in the scenario is a ... (a) pacemaker, (b) key, (c) basic, (d) repressed technology.||1||4-point Likert scale|
|Compatibility||The system offers situational services.||7||Dichotomous scale|
|Means of (human) interaction||The system uses acoustic interaction (e.g., voice control).||6||Dichotomous scale|
|Flexibility||The technology used in the scenario is highly compatible.||3||4-point Likert scale|
|Complexity||The technology used in the scenario is very specific for each application.||3||4-point Likert scale|
|Invest||Please rate the (estimated) cost of the technology.||1||4-point Likert scale|
|Effort of implementation||The installation of the technology used in the scenario requires complex re-planning and conversion of a system.||1||4-point Likert scale|
|Cost of maintenance||Please evaluate the operating effort of the technology used in the scenario with regard to system maintenance.||5||4-point Likert scale|
|Safety||With regard to security, the technology used in the scenario requires constant monitoring and documentation of the working environment.||3||4-point Likert scale|
|Security||With regard to safety, the technology used in the scenario requires a high level of training and safety instruction of employees.||3||4-point Likert scale|
|Product development process||Which product development process applies?||4||Dichotomous scale|
|Promotion of innovation||In the scenario described, product orientation is made possible in the company.||4||4-point Likert scale|
|Reduction of time-to-market||The described scenario reduces the time-to-market by improving process control.||4||4-point Likert scale|
|Labour expenditures||The scenario described must be carried out by many people.||5||4-point Likert scale|
|Need for competencies/expert knowledge||Which experts are required for data evaluation and analysis?||16||Dichotomous scale|
|Influence on process organization||The scenario described has effects on other processes.||3||4-point Likert scale|
|Degree of collaboration||The scenario described favours an open and flexible organizational structure.||2||4-point Likert scale|
|Degree of blurring of boundaries of work||The scenario described favours a decentralisation of the company organization.||4||4-point Likert scale|
|Agility||The scenario described favours customer orientation and participation in dynamic projects.||4||4-point Likert scale|
Table 3. Descriptive statistics and statistical values of the analysis of variance of all factors for the dimensions human, technology and organization.
|Mixed Mock-Up||Conformity Management Supported by a Graph Database||Digital Documentation of the Production Progress|
|Qualification of employees||2.63||0.90||2.79||1.08||2.47||0.96||0.84||0.44||0.04|
|Autonomy/scope of action||3.55||0.42||3.44||0.42||3.52||0.43||0.20||0.82||0.02|
|(Possibility) of social interaction||2.55||0.77||2.76||0.65||2.60||0.67||0.56||0.47||0.03|
|Holistic nature of the task||2.80||0.56||3.02||0.65||2.72||0.73||1.80||0.18||0.09|
|Means of interaction||0.48||0.29||0.36||0.28||0.59||0.21||5.83||0.01||0.27|
|Effort of implementation||2.84||0.90||2.53||1.26||3.11||0.88||2.04||0.14||0.10|
|Cost of maintenance||2.48||0.60||2.32||0.62||2.39||0.64||0.54||0.59||0.03|
|Promotion of innovation||2.89||0.58||2.95||0.70||2.77||0.72||0.36||0.70||0.02|
|Reduction of time-to-market||3.10||0.57||2.50||0.57||3.00||0.57||9.27||0.00||0.35|
|Influence on process organization||2.82||0.83||2.98||0.79||3.25||0.66||2.10||0.14||0.11|
|Degree of collaboration||2.79||0.84||1.75||0.62||2.29||0.92||7.30||0.00||0.40|
|Degree of blurring of boundaries of work||2.48||0.70||2.19||0.78||2.43||0.74||1.50||0.24||0.10|
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).