Next Article in Journal
Factors Influencing Physical Activity Participation among Midlife Immigrant Women: A Systematic Review
Next Article in Special Issue
Usage of eHealth/mHealth Services among Young Czech Adults and the Impact of COVID-19: An Explorative Survey
Previous Article in Journal
Observational Study on the Prevalence of Urinary Incontinence in Female Athletes
Previous Article in Special Issue
Rise of Clinical Studies in the Field of Machine Learning: A Review of Data Registered in ClinicalTrials.gov
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Searching for New Model of Digital Informatics for Human–Computer Interaction: Testing the Institution-Based Technology Acceptance Model (ITAM)

1
Graduate School of Public Administration, Korea University, Sejong 30019, Korea
2
Department of Police Administration, Seowon University, Cheongju 28674, Korea
3
Department of Public Administration, Ajou University, Suwon 16499, Korea
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(11), 5593; https://doi.org/10.3390/ijerph18115593
Submission received: 20 April 2021 / Revised: 17 May 2021 / Accepted: 18 May 2021 / Published: 24 May 2021
(This article belongs to the Special Issue Information Technology's Role in Global Healthcare Systems)

Abstract

:
The fourth industrial revolution has produced new information technology (IT) that is widely used in the healthcare industry. Although the nature of the institution affects IT adoption, the Technology Acceptance Model (TAM), a dominant theory, has dismissed its role and influence. Our research investigates how institutions influence the adoption of new IT by using the Institution-based Technology Acceptance Model (ITAM). We conducted an empirical test by using survey data collected from 300 employees in the public sector. Structural equation modeling was applied to test the proposed hypotheses. The results showed the total effect of institutions on the intention to use new IT is positive and significant. Second, IT adoption is not only affected by external institutions but also by type of institution; the external institution takes a greater role in inducing perceived usefulness, perceived ease of use, and intention to use than does the internal. Third, perceived ease of use and perceived usefulness play mediating roles between institution and IT adoption. Fourth, an alternative expanded model to which more individual and organizational factors were added confirmed the results of the base model. We concluded that institutions have a strong impact on the level of intention for IT use through perceived ease of use and perceived usefulness.

1. Introduction

The rapid development of digital technologies has led to many ideas and attempts to change developmental paths in society [1]. These recent developments, along with the advancement of social media tools and IT, provide innovative and intense motivations to strengthen interactions between organizational boundaries and individuals [2,3]. Some argue that new technological developments could pave the way for long-term growth [4]. The development of digital technologies transforms conventional industry manufacturing, service delivery, policy decision-making, public participation and also influences the digital health management related to public health. The force of technologies leads to innovative approaches in many spheres of society, which motivates many companies and governments to follow these technological advancements. Because recent IT growth has been based on digital data availability and interconnections among individuals, every person has their own preference for using IT platforms, which require extensive resources to set up [2,5,6,7]. Thus, it is necessary to use new technology that guarantees organizational process optimization through interactive communications with various stakeholders. It is believed that these technological progresses could facilitate sustainable development since they afford many stakeholders real-time opportunities to voice their grievances on day-to-day policy issues in their community; moreover, these developments advance digital data integration [3]. As its development could enhance technological usability, economic wealth, and political participation, new technology has become a major social and economic vehicle to change the ways people work, learn, and socialize [8]. With advancing developments of digital data, IT could lead to a break with precedent and prompt remarkable productivity and economic growth.
Applications of the fourth industrial revolution are considered to be revolutionizing structural elements in current society and have garnered attention in many countries [9]. As the scale and scope of change are vast and diverse, the impact of these new interactive technological developments is immense. The innovative development and diffusion of the fourth industrial revolution in the business sector is occurring faster than expected and many countries are trying to keep up with the pace of change. These changes are based on the idea that digital data sharing and interconnectedness among manufacturing stakeholders open new aspects of availability of technology, and many of these digitized devices will provide new approaches to acquiring individual demands and connecting to suppliers more conveniently [1]. Digitalization is becoming a major theme of development in every aspect as it provides a fundamental basis for recent technological innovations [9,10,11,12]. The fourth industrial revolution comprises new and transformative technologies, such as the internet of things, cloud computing, big data analytics, and automotive robotics, which can accelerate the pace of change in society.
Digital innovation is changing the fundamental structure of the public sector. Therefore, innovation strategies that consider the specific contexts and characteristics that exist in the public sector are necessary. Kromidha and Córdoba-Pachón ([13], p. 16) effectively explained the meaning of digital technologies in the public sector through adopting discursive institutionalism; digital innovation projects in the public sector are important strategic points of interaction for multifaceted stakeholders whose ideas and discourses could converge. In a transition context, coordinate discourse dominates thorough transformation projects, communicative discourse is valuable for change, participation in discourse is influenced by the power position, and the leadership philosophies’ value-led discourses are tools to justify desirable change. Moreover, Ashok et al. [14] empirically showed that when adopting innovative technology, organizational culture and transformational leadership traits positively affect the adoption of knowledge management, which has a positive effect on organizational culture to counter organizational inertia.
In order to succeed in the innovation of information and communication technology in the public domain, it is important to overcome the inertia and failure factors that exist inside. Hannan and Freeman [15] define inertia from a population ecology perspective as a persistent organizational resistance to changing architectural features. If there is inertia, management requires only minimal discretion to adapt to environmental changes. To overcome this inertia, an organization must have the ability to quickly respond to new opportunities, which presumably competes with the ability to execute consistently and accurately. Moreover, Leonard-Barton [16] found that four core capabilities—employee knowledge and skills, technical systems, managerial systems, and the values and norms—have paradoxical attributions that facilitate and block innovation. Frost [17] classified the failure factors in knowledge management implementation into causal factors (lack of performance indicator, inadequate skill of knowledge managers and workers, problems with organization culture) and resultant failure culture (lack of widespread contribution, overemphasis on formation learning, and improper implementation of technology). How does one overcome those inertia and failure factors in public sectors? Hannan and Freeman [15] stressed continued learning to adapt to the new environment. Moreover, Leonard-Barton [16] argued that to succeed in new technology innovation, one must manage the paradox of conflicting capabilities. To overcome inertia in public sectors, Ashok et al. [14] stressed organizational culture and transformational leadership in knowledge management adoption. All of these are closely related to the institutional context.
Therefore, it is necessary to consider the impact of institutions and the role of governments as key factors to accelerate the usefulness of new technology and regulate basic principles. Governments are positioned to increase data availability and usage and public employees play a critical role for innovation in government [18,19,20]. Consequently, understanding the attitudes and behavioral intentions of public employees is vital for the successful adoption and implementation of new technology. The fourth industrial revolution has diverse focus areas [21] because it includes various fields related to advanced IT developments and includes digitalization in the workplace and utilization of digital interactions in community activities [21]. As a result, the fourth industrial revolution is characterized as a collection of technologies that incorporate numerous digital transformative approaches in both the physical and biological spheres, particularly in fields such as public health [21,22]. This study is especially concerned with the examination of the relationship between institutions and the adoption of the fourth industrial revolution in the public sector.
In the new information age, new theoretical concepts and models related to informatics are required. In the literature regarding new technology adoption, the Technology Acceptance Model (TAM) has been widely used as a theoretical framework for predicting individuals’ acceptance of technology. Since the TAM was introduced by Davis [23], great theoretical progress has been made by adding more contextual variables regarding individuals’ behavioral intention to use technology [23,24]. TAM has been applied in many IT studies and has received substantial theoretical and empirical support from many researchers. Thus, TAM has the practical advantage of being extended to diverse contexts and settings. Therefore, we will base our argument on the idea of TAM that posits individual adoption of technology is mostly determined by two perceptions: usefulness and ease of use. In addition, we examine how these two core variables predict individuals’ decision-making with regard to adopting new technology in institutional settings. However, the TAM model overlooks the macro-level structure and system in that it mainly focuses on the individuals’ utility from the information system at the micro-level. Human–computer interactions differ depending on not only the direct relationship between the two but also the intervening variables. Therefore, it is necessary to consider structural variables at the macro level in TAM. In this study, we examine how not only micro-level individuals’ utility but also macro-level institutional factors affect the acceptance of new technology and extend knowledge on technology adoption behaviors. We believe our findings can contribute to the extant literature of this domain and to understand critical factors when adopting new technology.
Because IT has been extensively used in the health field, it is very important to understand the fundamental structure of technology acceptance. Technology acceptance will be critical for providing better healthcare services through online IT, especially during pandemics like COVID-19. Although new technologies help healthcare services through telemedicine and online databases, some barriers to broader adoption of new technology persist. In particular, institutional readiness and the adoption of new technology by employees are necessary. Since healthcare service in nature is a system of combining micro-level individual actors and macro-level intuitional structures, the acceptance of new digital technology for health care service should consider not only humane side factors but also institutional settings which constrain people’s attitude and perception. Digital technologies, the core of new technologies, cannot operate in isolation and require integrated institutional systems. In particular, digital data sources need to be integrated and interoperable into broader institutional settings. On the other hand, the government is one of the main actors in the institutional setting for healthcare services. The use and analysis of this data heavily depends on digital infrastructure and strong support from the public health system by governments. Governments’ responsibility and strategic foresight are especially important in high-risk situations such as a pandemic. In this regard, it is necessary to understand the adoptive mechanism of new technology in the public sector, especially focusing on the impact of institutions. Therefore, our study is expect to contribute to understanding IT services for the healthcare field.
The remainder of this study is divided into four sections. First, based on previous research, the authors propose theoretical reviews and hypotheses on technology adoption behaviors of employees and focus on the relationships with institutions and TAM, in particular. The next section discusses the methodology of this study, the sample and data used, and the variables and measurement. Next, the results are presented in the following section to show the relationships among factors affecting technology adoptions. The final section discusses the implications and limitations of the study with a conclusion.

2. Theoretical Background and Hypotheses

2.1. New Information Technology in the Fourth Industrial Revolution and Its Adoption

The increased and intensive use of IT requires governments to develop IT-based tools to directly deliver public service. The recent technological developments of the fourth industrial revolution emphasize the interconnectivity and reciprocity of the digital revolution and can lead to transformational changes at a societal level [25].
While recent IT developments bring many benefits, it is necessary to recognize the challenges of these new technologies in the public sector. The advantages of implementing new technologies are immeasurable, especially in terms of assisting public organizations in improving their tasks. The new technologies have high usability and flexibility and provide high-performance service delivery at low cost [26]. However, there are challenges regarding security, privacy, trust, and lack of understanding [26]. New technology adoption can create structural inertial pressures within organizations [14]. As March [27] suggested, internal organizational change was difficult to manipulate. Since the administrative system regulates resources, any external or internal institutional forces can cause resistance on adopting the new technical systems [15]. Also, when technological innovation enters an organization, the administration should consider how to recognize and manage the acceptable behavior of employees [16]. Therefore, considering the impact of institutions at the initial stage of technology diffusion, it is crucial to examine the employees’ perception and contextual understanding of technology adoption for the success of new technology [17]. IT development provides innovative and intense contingencies that allow societies to strengthen their positions using IT platforms [2,5,6,7]. Among these trends, there have been intense and innovative attempts to reshape structures and improve performance in the public sector using interactive technology. For example, confronted with the current COVID-19 pandemic in many countries, we recognize the potential role of new technology in the fourth industrial revolution in alleviating and resolving issues [28]. However, the usage of this technology is still a long way off and each country has different institutional arrangements and resources to utilize the benefits of new technology.
With the unprecedented efforts to utilize digital technologies in the field of government and work processes, the advent of the fourth industrial revolution ushers people into digital transformation in all areas of their personal lives. Due to advanced internet technologies, this movement mainly includes ubiquitous data communication, managerial flexibility, disruptive innovations, and autonomous systems [29]. The current fourth industrial revolution movement involves the distribution and implementation of new technologies in accordance with digital interconnectivity. Many countries also plan to launch industry programs for the new technologies and have created intelligent technical stimuli to associate key actors with their implementation, including business, university, and government counterparts. For example, as initiated in 2006 by the German federal government, the fourth industrial revolution mainly presents a strategic vision and goals of economic and social transformation through digital innovation. Represented as high-tech strategies and innovation, the key distinction of the fourth industrial revolution is its participatory decision-making processes involving multiple stakeholders [30]. Along with these movements, the German government also launched an IT-related government framework that emphasizes the role of government as a facilitator to construct sustainable digital infrastructure for the IT industry.
Many other countries also made strategic preparations to support the implementation of a wide range of innovations of the fourth industrial revolution. For example, Canada launched “Digital Canada”, which includes five critical categories in new technology: connection, protection, cybersecurity, economic opportunities, and digital government [31,32,33]. Likewise, the Korean central government has tried to establish broad strategies to establish a universal digital platform [34,35,36,37]. To consider establishing a platform that can develop a network among related stakeholders involved in the fourth industrial revolution, the Presidential Committee for the fourth industrial revolution was launched in 2017 to bolster government support for related businesses [11,36]. In particular, South Korea effectively halted COVID-19 transmission during the pandemic’s early period. It was about twice as effective as the U.S. The key to South Korea’s success mainly came from blending technology and physical testing, institutional adjustment for centralized control, and coordination between actors. As a result, we concluded that South Korea is an appropriate location for studying the fourth industrial revolution’s new technology adoption.

2.2. Technology Acceptance Model (TAM) and Its Limitation

Any innovative or transformative movement in large organizations, especially governments, should consider many relevant factors in their implementation. Even though the significant factors of IT implementation in government are diverse and complicated, they could be categorized as technological, structural, and behavioral factors [38,39,40].
Among the causal factors for successful application, institutions play a significant role when policymakers try to frame and introduce an IT policy [41,42,43]. The diffusion of technological innovation is up to actual users such as employees, stakeholders, and citizens, who are directly influenced by institutional arrangements. Institutional structures usually include legislative frameworks, managerial guidance, administrative rules, procedural standards, and conventional decision-making procedures. With this background, we seek to understand the behavioral factors of employees on new technology adoption and development in the context of the application of the fourth industrial revolution in the South Korean government. Because of its integrative and systemic approach toward a digitalized society, the government’s support and drives are critical for successful implementation [10].
TAM has been continuously studied to estimate users’ acceptance and behaviors; its application has expanded [24]. As a theoretical extension of the theory of reasoned action (TRA) [44], TAM provides a unified theory of technology acceptance and usage and persuasive explanations for individual motives for utilizing IT and its adoption [45,46]. TAM states that two fundamental perceptions determine technology acceptance behavior: perceived ease of use and usefulness. Perceived usefulness is defined as “the extent to which a person believes that using IT will enhance their performance” ([24], p. 187). Perceived ease of use is defined as “the degree to which a person believes that using an IT will be free of effort” ([24], p. 187). Therefore, TAM is recognized as a stable and manageable framework to understand user acceptance of new technologies in various organizations [47].
TAM has been continuously revised and new variables have been added to increase its theoretical explanatory power. Davis [23]’s original research showed that TAM fully explained the impact of system characteristics on users’ behavior. However, this model only accounted for 36% of the variance and thus needed additional variables to increase the explanatory power. Dishaw and Strong [48] suggest an integration of TAM and task-technology fit structures in this direction. They argue that a more integrated model will contribute to explaining more choices regarding IT use. Venkatesh and Davis [24,49] showed that the extended model accounted for 40–60% of the variance in usefulness perceptions and 34–52% of the variance in use intentions. In particular, they found a significant role of both social influence processes, e.g., subjective norm and cognitive instrumental processes, e.g., job relevance. Lucas and Spitler [50] demonstrated that organizational variables such as social norms and the characteristics of job are more important in influencing use of the technology than users’ perceptions about the technology.
However, TAM has several limitations. First, TAM overlooks various preferences because it emphasizes individual utility. According to Chtourou and Souiden [51], while the utility aspect of TAM is important, the hedonic aspect should be considered. The results suggest that product designers should develop interfaces and products that not only satisfy utilitarian needs but also hedonic and enjoyment motivations. This requires considering not only utility at the individual level but also the variables of the individual ideal. Moreover, Beglaryan et al. [52] argued that TAM research ignores factors beyond the individual, including group and social processes related to IT implementation and technology’s organizational and social consequences.
Second, there are objective variables, not just subjective, among the causal factors affecting technology acceptance. On the other hand, TAM disregards these objective variables at the initial stage of development. Since previous research had focused on the variables existing inside the model, it overlooked the external variables. For example, Hu et al. [53] and Davis [23,54] did not consider external variables in TAM. However, recently, as interest in objective variables has increased, research on external variables has increased. Venkatesh et al. [55] focused on facilitating conditions as objective factors in the environment that an individual agrees to make an act easy to accomplish. Moreover, Agarwal and Prasad [56] tested their significant role with regard to technology, tenure in the workforce, level of education, and prior similar experiences. Igbaria et al. [57] investigated internal computing support, internal computing training, management support, external computing support, and external computing training participation in training.
Third, TAM neglects the conditions, environment, and context that promote the perceived usefulness and perceived ease of use it IT adoption. Marangunić and Granić [58] indicated that a field with considerable potential is the study of various information systems and environments. Nevertheless, more efforts to examine various environmental factors, including emotion, habits, personality differences, and technology change, are necessary. Recently, more research has focused on facilitating conditions or structural constraints in TAM. For example, Ngai, Poon, and Chan [59] investigated facilitating conditions and showed how they affect technology acceptance. Venkatesh and Bala [46] elaborated on facilitating conditions that are related to individuals’ control beliefs regarding the availability of organizational resources and support structures to facilitate system use. They also found that perceived enablers or barriers in the environment influence a person’s perception of ease or difficulty of performing a task [60]. Ajibade [61] argued that IT experiences promote the ease of use of technology, while technology acceptance and intention is moderated by more structural constraints like the company’s rules, policies, and IT guidelines. Similarly, Lin and Wu [62] confirmed the role of intra-and extraorganizational factors as causal factors of end-user computing perception.
Following previous research, this study utilizes the extended TAM to explain the adoptive behavior and influencing factors of the fourth industrial revolution in governments. This study seeks to validate TAM in research on emerging new technology movement by analyzing factors that influence adoptive behavior in the central government of South Korea. Generally seen as a forerunner in the adoption and implementation of IT policies, South Korea is driving IT-based policy initiatives toward the successful transformation of the fourth industrial revolution [11]. Therefore, South Korea provides a useful field to carry out this study.
There have been many debates on the definition and concept of the fourth industrial revolution because of variation in countries’ focus on applications and content [11,21]. As the application of core components in the fourth industrial revolution requires a comprehensive understanding of its application and support from various stakeholders, the role of public employees is especially critical. For successful implementation, it is necessary to develop appropriate operating systems and institutional arrangements to increase adaptability and flexibility in various polices, especially complex fields like environment and public health. In this regard, the current study applies institutional theory to the basic TAM for a better understanding on the impact of external arrangements that are embedded in individuals’ usage of new technology. Because the interplay between technology usage and social contexts is complex and recursive, institutional aspects of IT contribute significantly to technology selection and use [63]. Institutions strongly influence peoples’ procedural decisions in the workplace [64] and employee behavior because their work procedure contents and standards are guidelines for the application of decision rules.

2.3. Hypothesis Development

TAM and extended ITAM posit complex human behaviors and decision-making processes that are subject to human–computer interaction. Researchers should take into account these intertwining relations in the model. This studies examine not only perceptional factors about institutional ones. A neoinstitutional framework makes links between the individual/behavioral and social/contextual sides of technology acceptance. Neoinstitutional theory applies its theories to explain technology implementation in an institutional environment. It regards informationization as a kind of institutional change and a process of institutionalization [65,66]. Jepperson ([67], pp. 145–152) defined an institution as “a social order or pattern that has attained a certain state or property”. Institutionalism is “a theoretical strategy that features institutional theories and seeks to develop and apply them”. According to institutionalism, institutions constrain and exert on an organization and organizational factors [68,69]. DiMaggio and Powell [68] suggested the three institutional pressures in organization: coercive, normative, and mimetic pressures. Similarly, Scott [69,70] proposed three institutional pillars (regulative, normative, and cultural-cognitive) that constrain and normalize individuals’ behavior. Institutions then “represent constraints on the options that individuals and collectives are likely to exercise, albeit constraints that are open to modification over time” ([69], p. 94).
In an information system (IS) context, institutional approaches focus not only on “how institutions influence the design, use, and consequences of technologies, either within or across organizations” ([71], p. 153) but also how they have an impact on users’ behavior in IS. Institutional theory sets the importance of contextualizing IT within the wider socioeconomic and political landscape [66]. Institutionalism focuses on the impact of institutions on technology users. Institutionalists believed in the formative power of institution and context in organization [72,73], which influences the way people think and constrains the way they act. Ciborra and Lanzara ([74], p. 70) describe formative contexts as “the set of institutional arrangements and cognitive imageries that inform the actors’ practical and reasoning routines in organizations”. Jensen et al. [75] addressed the electronic patient record (EPR) implementation works at three levels: the organizational field, the organizational/group, and the individual/sociocognitive level. In particular, macro-level structures, as well as individual interpretations, influence the adoption of IS. In the course of adoption, the institution significantly influences individual perceptions. Beglaryan et al. [52] examined the barriers of information system implementation concern and found that there are not only individual level factors: (a) perception, (b) expectancy and (c) utility of IS; however, there are also more institutional factors: (a) financial, (b) structural, (c) technical barriers (lack of infrastructure and suboptimal nature of the applied technology and solutions), (d) unavailability of facilitating conditions (involvement, training, organizational support, technical and expert support, and (e) lack of legal framework in IS implementation processes.
In the TAM model, institutional factors are regarded as external or social process elements. For example, Igbaria et al. [57] demonstrated that exogenous variables such as management support and external support have a significant impact on both perceived ease of use and perceived usefulness. Social influence processes (subjective norm, voluntariness, and image) and cognitive instrumental processes (job relevance, output quality, result demonstrability, and perceived ease of use) significantly influenced user acceptance [24]. Although institutions are not directly discernible in TAM, several variables are made available for attributions for institutions. For example, Venkatesh and Davis [24] identified significant antecedents such as subjective norm, perceived behavioral control, and self-efficacy. Subjective norms are shared collective beliefs that are regarded as an institution in institutionalism [76].
TAM also focuses on culture, which is a type of institution. The cultural-cognitive approach of neoinstitutional theory emphasizes the role of shared beliefs and ideas in shaping individual and organizational behaviors ([52], p. 52). When Straub et al. [77] compared the TAM model across three different countries, they found that TAM holds for both the United States and Switzerland but not for Japan, implying that TAM may not predict technology usage across all cultures. Beglaryan et al. [52] showed that administrative monitoring with an institutional character has a positive influence on perceived usefulness. Among the various institutional frameworks of TAM, culture is one of the challenging contexts that affect technology adoptive behavior [78]. Culture is shared with people who live within the same social environments. It consists of formal and informal rules of social interactions [79]. Therefore, culture may play a critical role in reducing uncertainty or risk avoidance behavior when confronting new technology. For example, individuals with the same culture who scored high on the uncertainty avoidance dimension may tend to seek ways to reduce risk and heavily rely on institutional arrangements [80].
However, with the wide and different ways of defining culture, it is difficult to examine and measure culture in an observable and constant manner to all individuals with the same cultural backgrounds [81]. Therefore, researchers usually regard culture as a type of collective institution or embedded institutional arrangement [82]. In this regard, McCoy et al. [81] suggested that the TAM model was considered to fit well in several countries, but some individual variations appeared as well. Thus, the TAM model might depend on culture as an institution collectively. Individual differences in the same cultural backgrounds, on the other hand, would be investigated for further consideration. Based on this discussion and previous studies, it can be assumed that institutions affect individuals’ thinking and behavior, suggesting the following hypothesis.
Hypothesis 1A.
Institution is positively related to the perceived usefulness (PU) and the perceived ease of use (PEOU) of new technology in the fourth industrial revolution.
As organizations and their members generally adapt to and operate in institutional arrangements (e.g., laws, orders, procedures, and instructions), institutions influence employees’ behavior and define appropriate behavior in these practices [64,77]. As March and Olson ([83], p. 22) have argued, the institution defines the “logic of appropriateness of behavior” that clarifies appropriate behavior and the individual’s role. Therefore, institutions regulate as a means of interlinking policy goals with individual actions [55]. Because institutions define responsibilities and goals, it is necessary for employees to accept institutional logic and applicable behaviors. This logical mechanism creates organizational and social conditions for the adoption of new technology in the workplace. The behavior of public employees can be more bounded by external institutions, such as laws and formal government procedures that consist of complex rule systems [84]. Thus, we attempted to develop an extended TAM model by adding institutional arrangement variables to further understand new technology adoption behavior of employees. Based on these arguments, we propose the following hypotheses to test the aforementioned logic.
Hypothesis 1B.
External institution is more positively related to the perceived usefulness (PU) and the perceived ease of use (PEOU) of new technology than internal institution.
The perceived ease of use is the degree to which the person believes that using the particular system would be free of effort [23,54]. Davis [23] originally hypothesized that the user’s attitude toward the information system was a major influential factor in whether the user would actually accept or reject the system. The attitude of the user was considered to be influenced by perceived usefulness and perceived ease of use. Subramanian [85] showed that perceived usefulness, not ease of use, is a main determinant of predicted future use of information systems. In addition, Chau [86] found that ease of use has the largest impact on specific software acceptance; perceived usefulness, not perceived ease of use, was found to be a significant determinant of attitude and intention for technology use. Igbaria et al. [57] indicated that perceived ease of use is a main factor in explaining perceived usefulness and system use. However, although perceived ease of use significantly predicts intention to use, the explanation power is secondary, following perceived usefulness [23].
Hypothesis 2.
Perceived usefulness (PU) is positively related to the intention to use new technology in the fourth industrial revolution.
Davis [23] suggested that “ease of use operates through usefulness” ([23], p 332). According to Davis [54], perceived ease of use has a direct influence on perceived usefulness. Davis [23] demonstrated that perceived ease of use fully mediated the effects on use intentions of perceived output quality. Davis et al. [87] also explained that perceived ease of use generally affects IT adoption indirectly through its effect on perceived usefulness because perceived ease of use is instrumental in making new IT more useful. Legris et al. [88]’s meta-study reported that the correlation between perceived ease of use and use intention had a significant positive relationship in 16 of 28 studies. However, Gefen and Straub [89] comment that the role of perceived ease of use in TAM remains controversial in that some studies show that perceived ease of use does directly affect either self-reported use or intended IT use.
Hypothesis 3A.
Perceived ease of use (PEOU) is positively related to the intention to use new technology in the fourth industrial revolution.
Davis [23] defined perceived usefulness as the degree to which a person believes that using the particular system would enhance her/his job performance. Igbaria et al. [57] and Abdullah et al. [90] showed that perceived usefulness is a variable that directly affects the intention to use an information system. When comparing the explanation of perceived usefulness and perceived ease of use, Davis et al. [23] reported that the perceived usefulness had 50% more impact than ease in determining information system use. Similarly, Keil et al. [91] demonstrated that perceived usefulness is a more important variable than ease of use in influencing system use. Legris et al. [88]’s meta-study showed the relationships between perceived ease of use and perceived usefulness demonstrated statistical significance in 21 of 28 studies.
Hypothesis 3B.
Perceived ease of use (PEOU) is positively related to perceived usefulness (PU).
According to Venkatesh and Bala [46]’s TAM 3 study, perceived ease of use and perceived usefulness play mediating roles in the relationship between their determinants and outcome factors. In particular, since perceived ease of use and perceived usefulness have different antecedents, the roles of both are very different, and therefore, they have differentiation and uniqueness as a path of influence. Abdullah et al. [90] showed that perceived ease of use and perceived usefulness mediated the relationships between external variables and students’ intention to use the e-portfolio. However, Beglaryan et al. [52] showed that perceived usefulness plays a mediating role between administrative monitoring and intention to use, but perceived ease of use does not perform such a function. They explained that this is because the perceived ease of use directly affects perceived usefulness and plays a different role as a variable.
Hypothesis 4.
Perceived ease of use (PEOU) and perceived usefulness (PU) positively mediate the relationship between institution and intention to use new technology in the fourth industrial revolution.
The following equations represent the proposed hypotheses. The baseline model posits that the adoption of new technologies is a function of the institutional space that surrounds the organization as well as individual elements.
Based on these assumptions, the structural model was constructed to test the logic and our hypotheses.
PEOU = ƒ(institution),
PU = ƒ(institution, PEOU),
Intention to use = ƒ{institution, PEOU, PU, M(PEOU, PU)}
where:
  • Institution includes laws, regulations, guidelines, strategies, and work procedures
  • Intention to use is the level of individual’s behavioral intention to adopt new technologies
  • M(PEOU, PU) plays a mediating role in explaining intention to use
Figure 1 shows the conceptual model and proposed relationships between the variables. In Structural Equation Modelling (SEM), latent variables are not directly observed but are inferred by the covariation among a set of observed variables (also called reflective indicators). SEM combines factor analysis and regression, which provides far greater flexibility to the modeler than either of these two analysis methods. This is distinct from doing a factor analysis and then inputting the factor scores into a multiple regression.
The variables in ellipses represent latent constructs that are explained by the reflective indicators depicted with rectangles. In our proposed model, latent variables IU_1 through IU_5 stand for the items in the test for intention to use and e13 through e17 stand for measurement errors (unreliability) in each item. In addition, SEM considers measurement error by modeling it explicitly when estimating latent variables from indicators [92]. These error terms are depicted with circles.
At the center of the model lies the assumption that perceived ease of use and perceived usefulness have mediating roles that have an effect on attitudes and behavioral intentions. The model suggests that the indirect pathway, from institution to behavioral intention to use, is determined by a set of personal beliefs that influence usage behavior.
As the purpose of our study is to suggest a constructive artifact of digital informatics of human–computer interaction, it is worthwhile evaluating our proposal in terms of information system perspective [93,94]. Information system research should analyze the interplay between information technology policy, organizational architecture and contexts, and user behavior [94]. In this regard, our conceptual focus is on the interplay between organization-based artifacts and technology adoption behavior. We seek to build a unified model for the development of design and behavioral science [94,95]. To evaluate the appropriateness of our proposal, we tried to apply the guidelines and principles of information system research because our study is an integrative attempt between behavioral science and design science research [95]. Hevner et al. [93] suggested the guidelines and recommendations that include design of an artifact, problem relevance, design evaluation, research contributions, research rigor, design as a search process, and communication of research. The guidelines and principles of information systems by Hevner et al. [93] and its adequacy to this study are outlined in Table 1. Although it is difficult to address all the guidelines [95], we tried to apply these criteria in our proposed model to identify the important considerations for developing a proactive framework while understanding the requirements in planning for new information technology in the fourth industrial revolution in various countries. However, even though our proposed model mostly fulfills these criteria, there are a few guidelines that are partially adequate. Thus, there are partially adequate fulfillments for our model according to Henver et al. [93]’s guidelines.
New technology in the fourth industrial revolution is highly based on interactions among individuals, technologies, organizations, and people. Therefore, it is critical to investigate the causal factors to adopt and use IS. In an IS, technology and behavior are not mutually exclusive [96,97]; research in IS is often categorized into two frameworks: behavior and design science [93]. Because our study is trying to find new digital informatics, the conceptual model is to examine the interconnection between system design factors (institution) and behavioral factors (intention to use with perceived ease of use and usefulness) to explain the adoption of new technology in the fourth industrial revolution. Our conceptual model particularly focuses on the organization and human-based artifacts that address relationships between institutions and technology adoption behavior in the fourth industrial revolution. Specifically, we introduce the concept of institutions as an enabler that can facilitate the usage behavior of new technology. It is critical to analyze institutional structures in operational processes and policy implementation because of the fundamental role of the institution as a driving factor for new technology adoption behavior. Since our model suggests a more extended IT adoption model that is broadly applicable in relevant settings, it can extend the scope of research on information system and design science. It can also contribute to top managers of organizations by suggesting technology-based solutions to solve business problems. In considering the guidelines and principles of information system perspectives, it can be concluded that the presented model and its implications will provide further explanation for IT adoption phenomena in the field of artifacts that solve important organizational problems.

3. Data and Methods

3.1. Research Method

TAM is widely applied across various fields in IS research. Many researchers have tried to provide an extension of TAM by integrating contextual and external factors. To offer a deeper explanation of technology adoption behavior, both empirical testing of the hypothesized model and investigating new influencing factors are needed. Qualitative research can provide in-depth understanding of IT adoption; however, exploring subjective experiences and subtle aspects of behaviors is a more effective way to learn the underlying reasons and motivations for IT adoption. This study focuses on the growing and recent phenomenon of TAM application in the latest industry techniques. Therefore, a quantitative research approach was chosen for this study. SEM was employed to examine our proposed hypotheses because it is the preferred method of testing a series of complex and multiple relationships constituting a set of an entire theory [92,98]. In SEM, the simultaneous effects of direct and indirect relations between variables are reflected. In addition, subjective variables such as people’s attitudes are often inaccessible to direct measures. SEM ties multiple observed measures to the substantive latent variables and takes account of measurement errors in observed measures. SEM can appropriately reduce measurement errors of constructs by estimating error variance parameters for the latent variables. Therefore, it is a suitable method to test complicated causal relationships between human perceptions and attitudes measured by multiple indicators. Compared to common quantitative methods, such as correlation and regression, the strength of SEM is that researchers can specify an a priori relationship between variables [99,100]. Researchers can test whether hypothesized relationships derived from theory are reflected in the sample data [92]. In SEM, two approaches are widely used: covariance-based SEM (CB-SEM) and partial least squares SEM (PLS-SEM). CB-SEM adopts a common factor model approach, which calculates covariance between the variables, and only that variance is included in estimating the construct measures that can be explained by the common factor (one unobserved variable) and individual random error. In contrast, PLS-SEM utilizes all the variance of independent variables that explains the variance of dependent variables. CB-SEM is preferred for theory confirmation and theory testing, while PLS-SEM is primarily used for exploratory research and hypothesis development [101,102,103,104]. In addition, CB-SEM is recommended when the sample size is large (more than 100). In this study, the CB-SEM method (specifically, maximum likelihood estimation) was chosen because our conceptual model was based on the established TAM theory and the sample size is 300.
The number of samples is a critical issue in SEM. Bentler and Yuan ([105], p. 181) explained that the most natural method for analyzing non-normal data, the asymptotically distribution-free procedure, is not defined when the sample size is less than the number of nonduplicated elements in the sample covariance. Bentler and Chou [106] believe that the number of cases needs to be five times the number of free parameters. On the other hand, Jöreskog and Sörbom [107] state that if the measured variable is less than 12, 200 cases are required, and if the number of measured variables is 12 or more, 1.5q (q + 1) equation is applied, where q represents the number of items that are used in the analysis. Mitchell [108] suggested the rule of thumb that there should be 10 to 20 times as many cases as variables. Similarly, according to Stevens [99], 15 cases per measured variable or indicator are needed. We added this discussion to the method section of the paper.

3.2. Data

To address the research questions and hypotheses, our study used a dataset from an open archive of the Korea Institute of Public Administration (KIPA). The KIPA is one of the leading public research organizations funded by the central government that mainly focuses on administrative developments and practices affecting public employees. Various surveys on policy and practical issues were exclusively conducted by KIPA to support government policymaking and implementation. Thus, datasets from the surveys are provided by formal request and review for further research [109]. According to the regulations of the KIPA on consent ownership and usage, it has been authorized for use in our research. The independent variables, mediating variables, dependent variables, and control variables used in this study were measured through the survey data on the perception of new technology in the fourth industrial revolution.
The survey into the state of new digital technology in the fourth industrial revolution was designed to identify primary factors that influence the adoption of new technologies in the public sector. The survey was conducted by KIPA through an online web page from 5 to 28 September 2018. Public employees who work in IT services in the central governments of South Korea were the target population of the survey. Random and cluster sampling methods were used for data collection. A total of 333 target respondents were randomly selected from employees of IT service divisions from 18 central government ministries, which are regulated by the Government Organization Act of South Korea. The candidates were contacted by phone prior to the survey to ensure that they agreed to participate in the survey. A website address was sent to them via e-mail and respondents logged onto the website to conduct the online survey; 300 respondents participated in the survey (90.09% of response rate). The total population size of the survey was 132,422 and our sample size was 300 with 95% confidence internal and 5.6% error. Determining an appropriate sample size for SEM is debatable where there is no prevailing consensus. Many guidelines have been suggested for the optimal sample size, such as the N: q rule [110] which uses the ratio of observations to estimated parameters, where N represents the number of cases and q represents the number of parameters that should be estimated. In this heuristic approach, the recommended sample-size-to-parameters ratio ranged from 5:1 to 20:1 [106].
It depends on not only the complexity of the model but also additional factors (e.g., the number of parameters, normality of the data). CB-SEM requires a larger sample size than PLS-SEM [92], and the median sample size is about 200 cases in studies where maximum likelihood estimation is used [111]. Barrett [112] also suggested that SEM analysis with a small sample (N < 200) may be problematic. Given these requirements, even though archival data was used, we concluded that 300 participants was appropriate to test the conceptual model.
The survey sample reflects the proportion of the total number of central government public employees and each proportion by ranking. Table 2 shows the characteristics of the respondents. The hierarchical position of respondents ranged from 3rd (managers) to 9th (lowest-level employees). Respondents included managers (grades 3–4), middle-managers (5–7), and team members (8–9, and others). In terms of gender, 71.7% of the respondents were male and 28.3% were female.

3.3. Measures

The questionnaires used in this study were designed by KIPA [109]. The items were derived from validated measurements that were used and verified in previous TAM research. To ensure the construct and content validity of the measures, items selected for the variables in this study were adapted from the KIPA reports. Multiple items were used for all of the measures in order to improve reliability and validity.
Institution. Institution can be conceived as laws, norms, and systems that confine the way people select, implement, and use IT in public organizations [70,113]. Public officials are restrained by not only laws but also administrative rules, manuals, or plans that governmental organizations accept. Therefore, broader aspects of general rules and regulations should be adequately addressed in this study. In this sense, we used indicators of “institution” that were developed by KIPA to evaluate the level of administrative innovation using the fourth industrial revolution technology in government services. The concept of institution used in this study consists of six components shown in Figure 2. Respondents were asked to rate the degree to which they agreed or disagreed with the following statement: “There are laws/acts, ordinances, work guidance, manuals, plans/strategies, and work processes that can facilitate the use of the fourth industrial revolution technologies in the administrative service delivery.”
Perceived ease of use related to computer self-efficacy, perception of external control, and computer playfulness [23,46] were measured using a five-point Likert scale. Perceived usefulness (PU) refers to subjective norm, image, job relevance, output quality, and result demonstrability [23,46]. Intention to use’s measurement focused on purpose to use, motivation to use, and sustainable effort to use [47,51].
Perceived ease of use. Perceived ease of use was measured by seven items adapted from previous research by Venkatesh and Bala [46] who suggested TAM3. However, four items (computer anxiety, perceived enjoyment, objective usability, and compatibility) were excluded due to low factor loadings. As a result, the determinants of perceived ease of use in this study consisted of three items: computer self-efficacy, perception of external control, and computer playfulness. Computer self-efficacy refers to individuals’ beliefs about their ability to use and control a computer system. Perception of external control refers to individuals’ beliefs about sufficient organizational and technological support for facilitating and adopting new technology in their organization. Computer playfulness refers to intrinsic motivation and willingness to use any new system.
Perceived usefulness. Perceived usefulness (PU) was measured by five items adapted from the study by Venkatesh and Bala [46]. They suggested five types of determinants of PU: subjective norm, image, job relevance, output quality, and result demonstrability. Subjective norm refers to “the degree to which an individual perceives that people who are important to him/her think one should or should not use the technology” ([46], p. 277). Image refers to “an enhancement of one’s social system” ([114], p. 195). Job relevance refers to the applicability of technology in one’s work. Output quality refers to “the degree to which a person believes that the technology performs a job’s tasks well” ([24], p. 191). Result demonstrability refers to a tangible, observable, and communicable improvement of work outcome caused by the use of technology.
Intention to use. Behavioral intention predicts and facilitates actual usage behavior. Respondents were asked questions about their attitude toward the use of new technologies (such as artificial intelligence, internet of things, and big data). Intention to use was measured using three items that refer to purpose and motivation to use new technologies and intention to put effort into using new technologies.

3.4. Analysis Method: Structural Equation Modeling

We use SEM as main analysis method. In terms of the statistical approach, SEM has good advantages for testing a series of complex and simultaneous relationships. SEM is a multivariate technique used to test all coefficients (relationships) in the complete model simultaneously. Simple ordinary least squares (OLS) regression cannot assess the significance of particular relationships between variables when moderating effects are hypothesized. In SEM, path coefficients (also called connection strengths) are parameters of the structural model and are estimated through a series of algebraic manipulations. The maximum likelihood fit function is used to calculate the path coefficients which are iteratively modified [107]. The path coefficient represents the change of the dependent (responding) variable according to a unit change of explanatory variable when all other variables in the model remain constant. The path coefficients are similar to b (unstandardized coefficient) or beta (β, standardized coefficient) in the regression model. The standard error (SE) of the coefficient (or standard deviation of the estimate) is taken into account in order to indicate how the estimated sample means precise an unknown mean of a population. This allows researchers to calculate a confidence interval that provides the range of observed effect size or p-value that assesses whether the estimates are significantly different from some reference value.
The purpose of SEM is to specify a model derived from theory and the estimation of the parameters of the model is derived from the conventional approach of SEM. The overall objective of the conventional approach is to connect the theory and specification of the model. Within this approach, exploratory factor analysis and confirmatory factor analysis are performed to test the proposed model. At this stage, the measurement model is estimated and the structural model can be estimated later. The full model can be structured through a continuous model modification and evaluation of goodness-of-fit until the model meets the criterion of adequate fit. This is discussed in more detail in the data and methods chapter. The meaning and interpretation of parameters in the SEM are outlined in Table 3.

4. Results

SEM consists of two powerful statistical approaches: exploratory factor analysis (EFA) and structural modeling where structural path analysis is applied. EFA is used to identify the number of hypothetical factors or latent variables that can be explained by covariance among a set of observed variables. Therefore, a priori specification of the number of variables is not required in EFA [111,115]. Instead, the structural model (also referred to as confirmatory factor analysis (CFA)) is designed to confirm priori conceptualized and hypothesized models; therefore, the exact number of variables should be specified in order to conduct CFA [116]. First, as a primary test, we conducted Harman’s single factor test by loading all variables onto a single factor without a rotation using SPSS. 12.0 [117]. Harman’s single factor test is a widespread statistical technique to identify common method bias. It is designed to test whether one component will explain more than 50% of the covariance among the measures. If common method variance exists, one factor accounts for the majority of the covariance between the variables. Results showed that a common latent factor explained 30.24% of the covariance, which suggested that there was no common methods bias.
The construct validity of measures was investigated in two steps: EFA and CFA. The characteristics of these two steps are outlined in Table 4. EFA based on principal component analysis with varimax rotation was conducted. Exploratory factor analysis, also referred to as the unrestricted factor model, is a multivariate statistical method used to uncover the underlying structure of measures and identify latent variables [92,111,115]. Common factors among measured variables are identified while the structure and correlations among these observed variables are explained. In EFA, varimax rotations are used to enhance the interpretability of retained factors and clarify the relationship among factors. In this step, potential factors were affirmed on theoretical backgrounds and the basic construct was developed according to the following results. Items with lower factor loadings were extracted (the lowest factor loading was 0.588) and a total of 17 items were selected for the measurement model. In EFA, all observed measures depend on all factors, which were not specified by researchers beforehand; thus, EFA may generate several possible structures and models, from a one-factor model to multiple-factor model [11,111,115,116]. As a result, four factors were derived and the cumulative percentage of variance explained by them was 65. The Bartlett’s test of sphericity was performed to identify whether the variances were equal for all samples. The chi-squared value for the test was high and significant (approximate χ2 = 2498.853, p < 0.0001). The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was 0.849, rejecting the null hypothesis that there was no difference in variances between the groups [107].

4.1. Measurement Model

We followed the two-step procedure proposed by Anderson and Gerbing [119], which consists of a measurement model and a structural model. In this stage, after obtaining initial instruments, we developed a measurement model and performed a CFA using AMOS 18.0. In CFA, each variable is derived from the factors specified by EFA and the researcher’s theoretical expectation; that is, restricted measurement models are identified and analyzed [111]. SEM allows researchers to evaluate relationships among variables comprehensively by providing a transition from EFA to CFA. The restricted model reflects a priori hypotheses and knowledge of the theory, which in turn allows researchers to test the relationship between observed variables and underlying latent constructs. Six common model-fit indicators were used to assess the goodness-of-fit of the measurement model. As presented in Table 5, all fit indices exceeded the recommended threshold. The comparative fit index (CFI), goodness of fit index (GFI), normed fit index (NFI), and Tucker–Lewis index (TLI) were all above 0.90. The standardized root mean residual (SRMR) was less than 0.08 [107], and the root mean square error of approximation (RMSEA) was less than 0.08 [120].
Contrary to EFA, the exact number of factors should always be defined in CFA before analysis. However, CFA does not necessarily confirm the initial restricted model that fits the data. In this case, the modified hypotheses and specified models should be proposed as alternative models [111]. In this study, CFA confirmed the hypothesized restricted model and initial constructs derived from EFA. The results of the CFA are presented in Table 6. All selected items had an acceptable value of factor loadings above 0.5 [120,121]. In addition, average variance extracted (AVE) was calculated in order to assess discriminant validity. AVE shows the level of variance that is captured by a construct compared to the level of variance due to measurement error. The values of AVE of all four factors were greater than the suggested threshold of 0.5 [112]. Overall, the measurement model indicated reasonable and acceptable convergent validity and discriminant validity.

4.2. Reliability Analysis and Correlations

Table 7 summarizes the descriptive statistics, the measure of scale reliability (Cronbach’s alpha), and the result of correlation analysis between the variables. The values of Cronbach’s alpha for all the factors were above the criterion of 0.70 [121], which means that scale items had high internal consistency. In addition, we calculated Pearson’s correlation coefficient (r) in order to evaluate the association between the variables. As shown in Table 7, correlations between the factors were statistically significant and showed predicted directions in our hypotheses. The dependent variable, intention to use, was positively related to institution (r = 0.276, p < 0.01), perceived usefulness (r = 0.551, p < 0.01), and perceived ease of use (r = 0.557, p < 0.01).

4.3. Hypothesis Tests

4.3.1. Structural Model

The structural model was identified in order to test the hypotheses. Beginning with the hypothesis tests, multiple fit statistics were examined for model fit. As shown in Table 5, the results showed a reasonably good model fit, supporting the conceptual model we suggested (CFI = 0.957; GFI = 0.924; NFI = 0.917; SRMR = 0.058; and RMSEA = 0.056). Thus, we could proceed to an analysis of the path coefficients. First, we focused on the standardized direct effect of predictor variables on the response variables in the model. As Figure 3 outlines, overall results were in the predicted directions in accordance with the aforementioned hypothesis. Results showed that institutions had a positive and significant effect on perceived ease of use (β = 0.327, p < 0.001) and perceived usefulness (β = 0.142, p < 0.05), in support of Hypothesis 1A. This suggests that institutions can play an important role in the adoption and diffusion of new technology, supporting previous studies that focused on the institutional perspective [122,123]. Institution explained 10.7% of the variance in perceived ease of use (squared multiple correlations or R2 = 0.107). In terms of Hypothesis 2, the effect of perceived usefulness on intention to use was positive and significant (β = 0.329, p < 0.001). The results also showed that perceived ease of use had a strong and positive impact on intention to use (β = 0.547, p < 0.001), supporting Hypothesis 3A. The results indicate that those who perceive that the fourth industrial revolution technology is useful and simple are more likely to have a higher intention to use it. The path coefficient between perceived ease of use and perceived usefulness was positive and statistically significant (β = 0.554, p < 0.001), in support of Hypothesis 3B. In terms of perceived usefulness, 37.8% of variance in perceived usefulness was explained by predictor variables (perceived ease of use and institution).
SEM also allows one to examine the total effect of each independent variable on any dependent variable. Total effects are calculated through the decomposition of effects, where the direct path coefficient between the two variables and the indirect effect between the two through other mediating variables are summed [124]. Total effects consider the mediating effects of the intervening variables that might potentially influence the direct effect. In doing so, total effects can provide an understanding of which factors are more important in determining the level of dependent variable. The indirect effect of institutions on intention to use through perceived ease of use and perceived usefulness was tested using the bootstrapping method. The indirect effect was positive and statistically significant (β = 0.116, p < 0.01) in support of Hypothesis 4. In addition, the total effect of institution on (Institution→ perceived ease of use→ intention to use) and (Institution→ perceived ease of use→ perceived usefulness→ intention to use) was positive and significant (β = 0.267, p < 0.01) (Table 7). The result is not surprising given the body of work in technology studies that asserted intention to use is shaped both by perceived usefulness and perceived ease of use. We focused particularly on the impact of institutions and lend support to the positive relations in TAM paths. The summary of the hypothesis test results is presented in Table 8 and Figure 3.
In order to explore more specific institutional settings and to further establish the measure of institution, we additionally adopted two-factor institutional models. The focus of the alternative models is the dimensions of institutions: whether they are created and enforced by entities inside or outside the organization.
By adopting externally legitimized formal frameworks and internal administrative process, organizations may strengthen the engagement of their internal members in change and innovations [125]. It is noted that the adoption of new technology frequently implements similar practices in the environment without clear usefulness [126]. In this respect, the diffusion of change within the organization is influenced by external forces; formation and establishment are influenced by internal forces [127]. Institutions enforce diffusion from the top down, while individuals are immersed in innovation from the bottom up. Considering this perspective, it can be assumed that institutional features might influence new technology adoption differently in the fourth industrial revolution. Thus, we are trying to examine the influence of specific settings in institutions. Institutional change often arises in the integration of top-down and bottom-up frameworks; it requires synthetic efforts toward technological change and innovation [128].
An attempt to compare alternative models lets us not only review the variety of institutional systems but also determine the superiority of the conceptual modes. These two dimensions generated two-factor institutional models illustrated in Figure 4. External institutions (laws, acts, and ordinances) are enacted by legislators who can exercise political control over the operation of public organizations. Regarding path dependence and feedback frameworks, external institutions have a strong effect on early moments in new technology adoption. Because of their strong effect, these external institutions might bring inflexibility and rigidness in utilization of new technology: just follow the paths [129].
Internal institutions (work guidance, manuals, and plans/strategies) are implemented by the top of the bureaucratic hierarchy as a means of agency management. As the interrelations between organization and institution are complicated and diverse, actors in organizations tend to choose institutional suboptimal solutions or arrangements to reduce uncertainty and ambiguity in the implementation process [130]. Therefore, we tried to examine external and internal institutional influence on the intention to use new technology in the fourth industrial revolution.
Our results showed that the directions of path coefficients adhere to the general TAM hypotheses. In terms of model fit, the external institution model showed adequate fit (CFI = 0.975, NFI = 0.942, SRMR = 0.043, RMSEA = 0.047) and the internal institution model also showed adequate fit (CFI = 0.940, NFI = 0.903, SRMR = 0.052, RMSEA = 0.069). In the external institution model (Figure 4), external institutions had a positive and significant effect on perceived ease of use, and the standardized total effect on intention to use was 0.291 (p < 0.01). Regarding the internal institution model (Figure 5), perceived ease of use and perceived usefulness had positive and significant relationships with internal institutions. However, the internal institution explained less variance of dependent variables than did the external institution. The standardized total effect of internal institution on intention to use was 0.210 (p < 0.01). The findings may indicate that institutional arrangements made by external entities can be more powerful than managerial actions when adopting new technologies in the public sector, therefore confirming our Hypothesis 1B.
Regarding the characteristics of organizational structure, it should be considered that there are possible differences in institutional influences between organizations with high and low hierarchical levels. Because our study was based on a high hierarchical organization (central government), it is worthwhile to discuss possible differences when we apply our two-factor institutional models to a low hierarchical organization (for example, a public expert organization). Regarding organizational members’ tendency to reduce uncertainty in the decision-making process [131], the members in low hierarchical organization might show the same results of institutional influences when adopting new technologies.
However, in terms of an organization’s complexity, centralization, and formalization, it would be considered that the influence of internal institutions would have a strong effect on new technology adoption in low hierarchical organization [132,133]. Because a low hierarchical organization has fewer managerial layers, organizational members could easily participate in the decision-making process, especially when making internal institutions like manuals and operational standards. Therefore, when adopting new technologies, internal institutions would show strong influence unlike high hierarchical organizations. Furthermore, hierarchical levels represent the system of authority for coordinating tasks and ordering the specialization of functions. These systems use varied amounts of hierarchy and control to shape behaviors; decisions are made from above and organizational members mainly receive orders. Therefore, when adopting new technologies, external institutions have strong impacts with well-defined links in the chain of command in high hierarchical organizations. Meanwhile, low hierarchical organizations have relatively few links; they might more focus on their internal institutions than external [134]. Therefore, it is possible to assume that the result of two-factor institution models in our study might be different if we apply this model in low hierarchical organization.

4.3.2. Alternative Expanded Model

In SEM studies, the standard procedures provide a baseline model based on general theory and test a modified model based on possible relationships between other indicators [99]. During the process, theoretical support and key conceptual assumptions of the model should be maintained. Researchers can change paths or variables and observe the coefficient and model fit changes compared to other models. This can be done by adding new variables or removing problematic latent variables. All steps need to be backed up with logical reasoning and theory. After several iterations of paths and variables, we decided to use the alternative expanded model, explained in Figure 6.
In doing so, we tried to give a more comprehensive explanation of complex human behaviors and attitudes that are constrained by various conditions and circumstances. The purpose of examining an additional alternative expanded model becomes clear when considering the fundamental nature of technology diffusion and adoption in organizations. Technology adoption and diffusion occur as a continuous process based on a series of individual decisions that are mostly calculated by the incremental benefits of using the technology. In addition, technology adoption is usually requested by a top-down management and often seen as an investment decision made by managers [70]. Researchers suggested that technology adoption occurs in stages according to individual differences. It is often modified by environmental conditions and uncertainty [71].
Therefore, the alternative expanded model posits that contextual factors may have direct and indirect effects on intention to use. In order to give a more comprehensive explanation of complex new technology phenomena, we constructed an alternative expanded model and tested non-mediated models. The proposed baseline model only considered causal path relationships of institutions, perceived ease of use, and perceived usefulness to intention to use of new technology in the fourth industrial revolution. To examine the influence of individual factors, we included intrinsic factors (experience, voluntariness), organizational factors (culture, security), and task-related factors (specialty, manager’s concern) [46,134]. Among individual intrinsic factors, experience might be important for adopting new technology [135]. New ways of collecting and sharing digital information have already been established by digital-based ubiquitous technologies, users who are accustomed to these trends might easily accept new technology. Thus, experience may be important for adopting new technology [136]. The degree to which prospective users perceive their decision is voluntary is important as it can affect when they adopt new technologies [56]. When technology adoption is organizationally imperative, user motivations vary and certain users restrain such type of mandate [128]. In this respect, voluntariness may affect adoptive behavior of new technology.
When introducing new technologies, the ability for individuals to adapt and modify their patterns is challenged. In this respect, a strong cultural foundation will facilitate the acceptance and success of new technologies [14,137]. Hence, culture can have an effect on new technology adoption. Because of the importance of new technology for social and economic growth, as well as an increasing focus on data collection, privacy has become a public concern [138]. It is critical to find the right balance between technology deployment and data security with user privacy, especially in the public domain [139]. Consequently, privacy can have an effect on new technology adoption.
New technology in the fourth industrial revolution might influence the types of job tasks required of individuals, which health professionals during the current pandemic have experienced [140]. Thus, we tried to include task-related factors (specialty, manager’s concern) in the alternative expanded model. In recent technological advances, including social networking, smartphones, sensors, and embedded systems, different attitudes and skills have been required [141], which has added to the burden of some employees. Consequently, specialty might have an effect on new technology adoption. Regarding managers’ organizational role, manager behavior has been discussed as one of the most significant factors in adopting new technologies [142]. Managers can assist in the technology adoption process by advising all users in the organization [143]. In this respect, managers’ concern will impact the adoption of new technology in the fourth industrial revolution.
Therefore, we examined the possible impact of individual features (experience, voluntariness, and specialty) and organizational conditions (culture, security, and managers’ concern). Individual experience was measured by “I have experienced public services provided by new technologies in the fourth industrial revolution (such as artificial intelligence, internet of things, and big data)”. Voluntariness was measured by “I think it is important to foster public service innovation using new technologies in the fourth industrial revolution”. Specialty was measured by “I have job specialties so that I can adopt new technologies in the fourth industrial revolution”. Organizational conditions were measured by the following: “My organization is trying to foster a culture of new technology in the fourth industrial revolution utilization”, “My organization provides technical security measures to protect IT-enabled public services”, and “Managers in my organization try to foster IT-enabled public services innovation.”
Our approach to the expanded modification model with individual and organizational factors can contribute to the development of information system research. From the perspective of “good design science” [93], studies must be presented effectively to both technology-oriented and management-oriented audiences. In this regard, results must be addressed to both the technology scholars and management communities with adequate objectivity and significance. Following this standard, a modified model can provide a thorough discussion of the behavioral processes for new technology adoption and managerial actions that are necessary. Our study results can provide useful solutions for managers and employees when they face obstacles to adopting new technology.
We tested whether significant improvement in model fit was made when adding new variables [92]. In order to investigate the differential effects of each condition and to compare model fits among the proposed models, changes of path and variables were added in stages from Models 1 to 4. In the first modification, a direct relationship between institution and intention to use was presumed (Submodel 1). The second modification was made to the structural model by adding individual factors (Submodel 2). The third modification was made by adding organizational factors to the baseline model (Submodel 3). In the final modification in Model 4, both individual and organizational factors were added and tested. Overall model fit indices are presented in Table 9.
Generally, the proposed alternative models were acceptable; however, the model fits were not greatly improved over the baseline model. Thus, we concluded that alternative models may not be preferable to the baseline model. Although alternative models did not appear superior in terms of model fits, it should be noted that more conditional variables were considered and the simultaneous effects were examined.
Beyond these model fit results, significant path coefficients were found in most of the expected directions and the results supported the verified causal relationships of the baseline model. Table 9 represents the total effects of each variable on intention to use and Figure 6 only represents direct effects between the variables. The total effect is combined with direct and indirect effects. In SEM, the pathway from the exogenous latent variable to endogenous outcome variable through the mediator calculates indirect or mediating effect. As shown in Table 10, the significant and positive total effects of institution and perceived ease of use were maintained in all of the alternative models. More importantly, coefficients of both variables were raised incrementally as alternative factors were added, thus confirming the positive relation between institution and technology adoption behavior. This also suggests that the impacts of institution and perceived ease of use on intention to use remains dominant when other individual and work conditions are controlled. In addition, the direct effect of institution on intention to use did not approach significance in Figure 6; however, the total effect was significantly mediated by perceived ease of use and perceived usefulness. This suggests institutions can affect intention to use only through mediating effects of perceived ease of use and perceived usefulness. Again, perceived ease of use and perceived usefulness had full mediating effects on the relationship between institution and intention to use. Perceived ease of use also had a significant and positive effect on intention to use in Submodel 2 and Submodel 3; however, significance was not found in the full Submodel 4. Of the three predictors, the improvement of the coefficient of perceived ease of use was noticeable compared to other variables, indicating that perceived ease of use is relatively more important in determining an outcome. Thus, it may be concluded that institutions strengthen the level of intention to use mainly through perceived ease of use. The summary of the hypotheses test results are as follows.
As with any previous TAM studies, the effects of both perceived ease of use and perceived usefulness on intention to use were positive and significant in the basic parsimonious model. Furthermore, institutions significantly affect perceived ease of use and perceived usefulness. Institutions have a significant and strong indirect influence on intention to use, which would lead one to expect that institutional factors may have a crucial role in new technology acceptance of public employees in South Korean central government agencies. The test results of the alternative models lend support to the important role of institutions as an external facilitating factor, thus confirming Hypothesis 1. While the direct effects of institutions on perceived usefulness did not show significance in submodel 4, its total influence on intention to use was still reinforced by mediating variables. In terms of the total effects of contextual variables, specialty and security showed a negative and significant effect on intention to use in submodel 2 and 3, respectively. However, direct effects were not significant (Figure 6). In submodel 4, only specialty had a significant and negative effect on institution (total effect) and the direct effect was also significant as shown in Figure 6.

5. Discussion and Implication

5.1. Discussing the Main Results

Our study aimed to analyze the links between the impact of institutions, perceived ease of use, perceived usefulness, and intention to use of new technology adoption in the fourth industrial revolution [11,12,20,31,144]. By using individual-level data from a survey of public employees in the Korean central government conducted in 2018, this study empirically examined the important factors for adopting new technology. The understanding and recognition of public employees engaging in new emerging technology is important because of their role in decision-making and value creation in collective issues like the environment and public health [145,146].
The contributions of our study to the healthcare field are suggested as follows: First, the usage of digital technology and developments in the healthcare field are expected to increase significantly following COVID-19. Regarding the role of information systems in COVID-19, this study implies that institutional environments should be considered when initiating digital health related policies or projects. After tracking websites’ digital communication strategies in Latin America hospitals during the COVID-19 pandemic, Tejedor et al. [147] suggested that digital media could constitute legitimate resources for healthcare information consumption, so their accuracy and proper development seem to be significant to becoming a genuine source. In addition, after studying Electronic Health Record (EHR), Beglaryan et al. [52] indicated that institutional factors as well as personal benefits at the micro level should be considered. In particular, they suggest that the critical role of social factors, like institutions, hinders success of informatization at the organizational level. Moreover, Ahmad et al. [148] found that social influence had a positive impact on patients’ continued intention to use digital health wearables. These studies suggest not only utility-based managerial factors but also social structural ones should be considered in investigating new technology adoption. While it is important to emphasize the convenience and benefits of operationalization of new technologies, it is also important to construct basic institutional arrangements for their utilization. The usage of information technology for pandemic management is still a long way from achieving an integrated system focused on effective cooperation between humans and machines for disease prevention and treatment [28]. As legal frameworks have been critical to more promptly responding to the COVID-19 crisis, the need for more comprehensive regional and national disease registries is required. It is also emphasized that coordination between agencies and organizations is required for successful testing and quarantine for individuals with COVID symptoms using new technology such as contact tracing [139]. However, without clear guidance and recommendations, successful coordination among stakeholders is difficult to achieve. Therefore, the results of this study suggest that institutional adjustments are required to promote the acceptance of new technology, especially during crisis like COVID-19.
Second, our findings suggest that institutions had an indirect impact on the intention to use new technology of the fourth industrial revolution. It is important to consider what factors affect government employees’ intentions to use new technology as they introduce and organize various collective strategies on public issues such as pandemics. The institutions do not ensure that the actual application of emerging technologies can occur without the mediation of presumed ease and usefulness. As a result, when we introduce new technologies, we must understand how people actually perceive the ease of use and usefulness of such technologies. In particular, the COVID-19 pandemic involved huge demands that necessitated the collection and analysis of digital health data using new technology. As a result, the role of government employees is critical. Thus, it is worthwhile to investigate the factors of new technology adoption, especially when it is intended to be used by public employees confronting this unprecedented issue.
Third, when considering the design of information systems for healthcare services, it needs to consider whether external and internal institutions had a differential effect on the intention to use new technology [128]. In the results of our study, in terms of uncertainty reduction, the effect of internal institutions is stronger. As a consequence, we would conclude that when implementing new technologies, it is important to provide employees with a standardized procedure and best practices [149]. However, external institutions had a stronger total effect on intention to use than internal institutions, where perceived ease of use and perceived usefulness have mediating roles in our result. Thus, we can conclude that official government schemes and systems may have superiority over managerial actions when implementing new technology. Therefore, it is necessary to provide adequate external institutions (like laws and ordinances) for effective adopting new technology in healthcare fields.
In Table 11, we summarized the contributions of our study in the fourth industrial revolution and TAM studies. This section will provide a concise description of the experimental results, their interpretation, and the experimental conclusions that can be drawn.

5.2. Practical Implications

Our study empirically investigated the influence of institutions on new technology adoption of the fourth industrial revolution. The characteristics of these new technological developments are complicated [21], and it is expected that the fourth industrial revolution will bring new changes by growing networking, data-based intelligence, and knowledge sharing between humans by real-time computer interaction [153]. When new technology is initiated in an organization, institutional influence is normally based on centralization; however, simply adopting external requirements could result in short-term usage of new technology like perceived ease of use but not long-term goals like usefulness and intention to use in our study [13].
New technological advancements like the fourth industrial revolution should be closely interconnected with institutional arrangements such as strategies that allow a structure to adjust or preserve stabilization after experiencing periods of instability [154]. For instance, in the case of South Korea, after the Middle East Respiratory Syndrome (MERS) outbreak in 2015, structures and processes such as strengthened legislative authority for quarantine and surveillance systems encouraged people to promptly adapt to digital technologies in response to the pandemic [155]. Thus, these institutional facilitators and the groundwork helped the new technology to be quickly used for public health. These institutional arrangements for disease control that include relative digital data management have enabled South Korea to extend accelerated testing and case recording to a greater portion of the population when faced with COVID-19 [155]. Contrariwise, the United States, where lengthy lines for coronavirus testing wrapped around the block early in the pandemic, demonstrated a defective monitoring process even several months after the pandemic outbreak.
Countries like Singapore and Australia launched a digital app and QR tracing, while data encrypted technology was reserved for government-based utilization [156,157]. In Singapore’s attempts to control the COVID-19 pandemic, technologies such as artificial intelligence and data analytics have played a critical role. The Australian experience also shows that specific institutional responses are crucial for public health prevention. For example, with the launch of a monitoring and tracing application for smartphones, COVIDSafe, the Australian federal government rapidly replaced the conventional contact tracing methods. These strong institutional adjustments increase user safety [158]; the application was downloaded by six million people in a short period.
Given the pandemic’s uncertain nature and the risks it presents, it made sense for government authorities to first centralize decision making, such as school openings, and create or implement strategies for delivering instruction to fulfill regulatory requirements [159]. Nevertheless, as we can see with cases in various countries, administrative regulatory decisions during the pandemic require unambiguous evidence of the reasons and necessities for the public. Without a large amount of obvious data from reliable objective technological procedures, public uncertainty and doubt cannot be easily alleviated. We should consider public confidence in public health officials and other policy makers. While the latest pandemic is unsettling, we can envision a worse situation in the future.
As we have seen in the previous examples of response to the COVID-19 pandemic in various countries, new technology adoption does not always lead to complete adoption and use. One of the main reasons for the failure of U.S. governance was not due to technological difficulty but rather to institutional instability and perceived utility [160]. Thus, as our study suggested, detailed considerations should be prepared when established institutions adopt new technology in the fourth industrial revolution.

6. Conclusions

In this study, we assessed the causal path relationships of new technology adoption in the fourth industrial revolution by an empirical survey of public employees of the central government in South Korea. Since government employees play such a significant role in the decision to construct new technology deployment [161,162], they are in a key position to introduce and utilize the advantages of new technology. The convergence of physical and digital technologies is one of the key features of the fourth industrial revolution [153] and it could open new opportunities to transform conventional human–computer interaction, as we have witnessed during the current global COVID-19 pandemic [155].
We identified that institutions influence perceived ease of use and usefulness, which mediate the impact of institutions on intention to use new technology [163]. According to our findings, institutions have a significant influence on the intention of public employees to use new technology in terms of ease of use and usefulness, which has a mediating effect on intention to use. This means that if digital-based new technology is assumed to be the new means of information usage, the behavioral patterns of new technology adoption would not be strongly differentiated from conventional technology use [23,152], except for institutional impact. In addition, as we have witnessed, the applications of new technology in the fourth industrial revolution to confront the pandemic are constantly increasing. However, without appropriate consideration of users’ perception and behavior, this technological use would not be as effective [164].
In addition, as the results of our alternative expanded model showed, institutions cannot directly impact intention to use new technology without the mediating effect on perceived ease of use and usefulness if we do not control the causal path relationship [165]. It is important to note that the casual path direction of the influence of institutions is only through ease of use, not usefulness in the expanded alternate model, implying that perceived ease of use should receive significant consideration when new technology is introduced in a society [165,166]. In conclusion, this study specified the determinants of new technology adoption in the fourth industrial revolution and, in particular, sought a causal path between institutions and the intention to use new technology with perceived ease of use and usefulness as mediators.
The findings of this study could be meaningful in many respects. However, they also present certain limitations that will provide a basis for future research. This study performed statistical tests and analyses based on cross-sectional survey data collected at a single point in time. Thus, it is not possible to use a panel analysis methodology that examines the trend of usage over time. In addition, even though we conducted several statistical tests to avoid possible methodological bias in our model, mono source bias or correlated measurement error might not be completely excluded.
Because of the contextual features of the South Korean public sector, it is necessary to consider biases from contextual factors like culture in further research [167]. Furthermore, given that our alternative two-factor institution models are based on high hierarchical level organization, the results might show differently if they were applied to low hierarchical level organization. In addition, to offer greater insight into the phenomenon of usage and utilization, the use of case studies should be considered to investigate more interpretive research of new technology in the fourth industrial revolution phenomena [13,14]. Future studies are encouraged to explore the issues and practices of new technology and its effectiveness in the context of determinants.

Author Contributions

Conceptualization Y.K. and N.C.; data acquisition, N.C.; writing—theoretical framework and analysis, Y.K. and S.K.; writing—methodology and analysis, N.C. and Y.K.; writing—review and editing, S.K and N.C. All authors have read and agreed to the published version of the manuscript.

Funding

This article is financially supported by the Graduate School of Public Administration at Korea University in 2021. Nakbum Choi is financially supported by Seowon University.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to regulations and guideline of data open policy according to the Korea University.

Acknowledgments

This paper makes use of research material produced by the Korea Institute of Public Administration (KIPA) and has been authorized for use according to KIPA’s regulations on the ownership and use of said research material.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Brynjolfsson, E.; McAfee, A. The Second Machine age: Work, Progress, and Prosperity in a Time of Brilliant Technologies; WW Norton & Company: New York, NY, USA, 2014. [Google Scholar]
  2. Bertot, J.C.; Jaeger, P.T.; Munson, S.; Glaisyer, T. Social media technology and government transparency. Computer 2010, 43, 53–59. [Google Scholar] [CrossRef]
  3. Ziemba, E. The contribution of ICT adoption to the sustainable information society. J. Comput. Inf. Syst. 2019, 59, 116–126. [Google Scholar] [CrossRef]
  4. Sachs, J.D. The Age of Sustainable Development; Columbia University Press: New York, NY, USA, 2015. [Google Scholar]
  5. Sobaci, M.L.; Karkin, N. The use of twitter by mayors in Turkey: Tweets for better public services? Gov. Inf. Q. 2013, 30, 417–425. [Google Scholar] [CrossRef]
  6. Mergel, I. Open innovation in the Public Sector: Drivers and barriers for the adoption of challenge.gov. Pub. Manag. Rev. 2018, 20, 726–745. [Google Scholar] [CrossRef]
  7. Glyptis, L.; Christofi, M.; Vrontis, D.; Del Giudice, M.; Dimitriou, S.; Michael, P. E-Government implementation challenges in small countries: The project manager’s perspective. Technol. Forecast. Soc. Chang. 2020, 152, 119880. [Google Scholar] [CrossRef]
  8. Pahl-Wostl, C. A conceptual framework for analyzing adaptive capacity and multi-level learning processes in resource governance regimes. Glob. Environ. Chang. 2009, 19, 354–365. [Google Scholar] [CrossRef]
  9. Schwab, K. The Fourth Industrial Revolution; Currency: New York, NY, USA, 2017. [Google Scholar]
  10. Mian, S.H.; Salah, B.; Ameen, W.; Moiduddin, K.; Alkhalefah, H. Adapting universities for sustainability education in Industry 4.0: Channel of challenges and opportunities. Sustainability 2020, 12, 6100. [Google Scholar] [CrossRef]
  11. Sung, T.K. Industry 4.0: A Korea perspective. Technol. Forecast. Soc. Chang. 2018, 132, 40–45. [Google Scholar] [CrossRef]
  12. Safar, L.; Sopko, J.; Dancakova, D.; Woschank, M. Industry 4.0—Awareness in South India. Sustainability 2020, 12, 3207. [Google Scholar] [CrossRef] [Green Version]
  13. Kromidha, E.; Córdoba-Pachón, J.R. Discursive institutionalism for reconciling change and stability in digital innovation public sector projects for development. Gov. Inf. Q. 2017, 34, 16–25. [Google Scholar] [CrossRef]
  14. Ashok, M.; Al, M.S.M.A.B.; Madan, R.; Dzandu, M.D. How to counter organisational inertia to enable knowledge management practices adoption in public sector organisations. J. Knowl. Manag. 2021, 24. [Google Scholar] [CrossRef]
  15. Hannan, M.T.; Freeman, J. Structural inertia and organizational change. Am. Sociol. Rev. 1984, 49, 149–164. [Google Scholar] [CrossRef]
  16. Leonard-Barton, D. Core capabilities and core rigidities: A paradox in managing new product development. Strat. Manag. J. 1992, 13, 111–125. [Google Scholar] [CrossRef]
  17. Frost, A. A synthesis of knowledge management failure factors. Recuper. El 2014, 22, 1–22. [Google Scholar]
  18. Anton, C.; Camarero, C.; San José, R. Public employee acceptance of new technological processes: The case of an internal call centre. Public Manag. Rev. 2014, 16, 852–875. [Google Scholar] [CrossRef]
  19. Baldwin, J.N.; Gauld, R.; Goldfinch, S. What public servants really think of e-government. Public Manag. Rev. 2012, 14, 105–127. [Google Scholar] [CrossRef]
  20. Pfeiffer, S. The vision of “Industrie 4.0” in the making—A case of future told, tamed, and traded. Nanoethics 2017, 11, 107–121. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Lee, M.; Yun, J.J.; Pyka, A.; Won, D.; Kodama, F.; Schiuma, G.; Park, H.; Jeon, J.; Park, K.; Jung, K.; et al. How to respond to the fourth industrial revolution, or the second information technology revolution? Dynamic new combinations between technology, market, and society through open innovation. J. Open Innov. Technol. Mark. Complex. 2018, 4, 21. [Google Scholar] [CrossRef] [Green Version]
  22. World Economic Forum. The Future of Jobs: Employment, Skills and Workforce Strategy for the Fourth Industrial Revolution; Global Challenge Insight Report; WEF: Geneva, Switzerland, 2016. [Google Scholar]
  23. Davis, F. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef] [Green Version]
  24. Venkatesh, V.; Davis, F.D. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manag. Sci. 2000, 46, 186–204. [Google Scholar] [CrossRef] [Green Version]
  25. Ali, O.; Soar, J.; Yong, J. An investigation of the challenges and issues influencing the adoption of cloud computing in Australian regional municipal governments. J. Inf. Secur. Appl. 2016, 27, 19–34. [Google Scholar] [CrossRef]
  26. Ali, O.; Soar, J.; Yong, J. Impact of cloud computing technology on e-government. Inf. Soft. Technol. 2014, 456, 272–290. [Google Scholar]
  27. March, J.G. Footnotes to organizational change. Adm. Sci. Q. 1981, 26, 563–577. [Google Scholar] [CrossRef]
  28. Mehraeen, M.; Dadkhah, M.; Mehraeen, A. Investigating the capabilities of information technologies to support policymaking in COVID-19 crisis management; A systematic review and expert opinions. Eur. J. Clin. Investig. 2020, 50, e13391. [Google Scholar] [CrossRef] [PubMed]
  29. Ślusarczyk, B. Industry 4.0—Are we ready? Pol. J. Manag. Stud. 2018, 17, 232–248. [Google Scholar] [CrossRef]
  30. Reischauer, G. Industry 4.0 as policy-driven discourse to institutionalize innovation systems in manufacturing. Technol. Forecast. Soc. Chang. 2018, 132, 26–33. [Google Scholar] [CrossRef]
  31. Wolfe, D. A Digital Strategy for Canada: The Current Challenge. IRPP Insight, 25. 2019. Available online: https://irpp.org/wp-content/uploads/2019/01/A-Digital-Strategy-for-Canada.pdf (accessed on 17 October 2020).
  32. Mexican Ministry of Economy. Crafting the Future: A Roadmap for Industry 4.0 in Mexico. 2016. Available online: https://knoware.biz/wp-content/uploads/2018/02/industry-4.0-mexico.pdf (accessed on 11 October 2020).
  33. Liao, Y.; Loures, E.R.; Deschamps, F.; Brezinski, G.; Venâncio, A. The impact of the fourth industrial revolution: A cross-country/region comparison. Production 2018, 28. [Google Scholar] [CrossRef] [Green Version]
  34. KISDI(Kora Information Development Institute). ICT Industry Outlook of Korea. 2019. Available online: https://www.kisdi.re.kr/kisdi/jsp/fp/eng/research/KE_25000.jsp (accessed on 9 September 2020).
  35. O’Reilly, T. Government as a Platform. Innov. Technol. Gov. Glob. 2011, 6, 13–40. [Google Scholar] [CrossRef]
  36. The Government of the Republic of Korea. People-Centered plan for the Fourth Industrial Revolution to promote Innovative Growth I-Korea 4.0. 2017. Available online: https://www.4th-ir.go.kr/article/detail/220 (accessed on 14 September 2020).
  37. Presidential Committee on the Fourth Industrial Revolution (PCFIR) of Korea. Recommendations to the Government for the Fourth Industrial Revolution. 2019. Available online: https://www.4th-ir.go.kr/article/download/623 (accessed on 17 September 2020).
  38. Hilty, L.M.; Aebischer, B. ICT for sustainability: An emerging research field. In ICT Innovations for Sustainability; Hilty, L., Aebischer, B., Eds.; Springer: Cham, Switzerland, 2015; pp. 3–36. [Google Scholar]
  39. Zhang, H.; Xu, X.; Xiao, J. Diffusion of e-government: A literature review and directions for future directions. Gov. Inf. Q. 2014, 31, 631–636. [Google Scholar] [CrossRef]
  40. Carter, L.; Bélanger, F. The utilization of e-government services: Citizen trust, innovation and acceptance factors. Inf. Syst. J. 2005, 15, 5–25. [Google Scholar] [CrossRef]
  41. Peters, B.G. The Future of Governing: Four Emerging Models; University Press of Kansas: Lawrence, KS, USA, 1996. [Google Scholar]
  42. Weaver, R.; Rockman, B. Assessing the effects of institutions. In Do Institutions Matter? Weaver, K., Rockman, B., Eds.; The Brookings Institute: Washington, DC, USA, 1993; pp. 2–18. [Google Scholar]
  43. Decramer, A.; Smolders, C.; Vanderstraeten, A.; Christiaens, J. The impact of institutional pressures on employee performance management systems in higher education in the low countries. Brit. J. Manag. 2012, 23, S88–S103. [Google Scholar] [CrossRef]
  44. Fishbein, M.; Ajzen, I. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research; Reading, Addison-Wesley: Boston, MA, USA, 1975. [Google Scholar]
  45. Kim, J. Toward an understanding of web-based subscription database acceptance. J. Am. Soc. Inf. Sci. Technol. 2006, 57, 1715–1728. [Google Scholar] [CrossRef]
  46. Venkatesh, V.; Bala, H. Technology acceptance model 3 and a research agenda on interventions. Decis. Sci. 2008, 39, 273–315. [Google Scholar] [CrossRef] [Green Version]
  47. Verma, S.; Bhattacharyya, S.S.; Kumar, S. An extension of the technology acceptance model in the big data analytics system implementation environment. Inf. Process. Manag. 2018, 54, 791–806. [Google Scholar] [CrossRef]
  48. Dishaw, M.T.; Strong, D.M. Extending the technology acceptance model with task–technology fit constructs. Inf. Manag. 1999, 36, 9–21. [Google Scholar] [CrossRef]
  49. Venkatesh, V.; Davis, F.D. A model of the antecedents of perceived ease of use: Development and test. Decis. Sci. 1996, 27, 451–481. [Google Scholar] [CrossRef]
  50. Lucas, H.C., Jr.; Spitler, V.K. Technology use and performance: A field study of broker workstations. Decis. Sci. 1999, 30, 291–311. [Google Scholar] [CrossRef]
  51. Chtourou, S.; Souiden, N. Rethinking the TAM model: Time to consider fun. J. Consum. Mark. 2010, 27, 336–344. [Google Scholar] [CrossRef]
  52. Beglaryan, M.; Petrosyan, V.; Bunker, E. Development of a tripolar model of technology acceptance: Hospital-based physicians’ perspective on EHR. Int. J. Med. Inf. 2017, 102, 50–61. [Google Scholar] [CrossRef]
  53. Hu, P.J.; Chau, P.Y.; Sheng, O.R.L.; Tam, K.Y. Examining the technology acceptance model using physician acceptance of telemedicine technology. J. Manag. Inf. Syst. 1999, 16, 91–112. [Google Scholar] [CrossRef]
  54. Davis, F.D. A Technology Acceptance Model for Empirically Testing New End-user Information Systems: Theory and Results. Doctoral Dissertation, MIT Sloan School of Management, Cambridge, MA, USA, 1986. [Google Scholar]
  55. Venkatesh, V.; Morris, M.G.; Davis, G.B.; Davis, F.D. User acceptance of information technology: Toward a unified view. MIS Q. 2003, 27, 425–478. [Google Scholar] [CrossRef] [Green Version]
  56. Agarwal, R.; Prasad, J. The role of innovation characteristics and perceived voluntariness in the acceptance of information technologies. Decis. Sci. 1997, 28, 557–582. [Google Scholar] [CrossRef]
  57. Igbaria, M.; Zinatelli, N.; Cragg, P.; Cavaye, A.L. Personal computing acceptance factors in small firms: A structural equation model. MIS Q. 1997, 279–305. [Google Scholar] [CrossRef]
  58. Marangunić, N.; Granić, A. Technology acceptance model: A literature review from 1986 to 2013. Univers. Access. Inf. Soc. 2015, 14, 81–95. [Google Scholar] [CrossRef]
  59. Ngai, E.T.; Poon, J.L.; Chan, Y.C. Empirical examination of the adoption of WebCT using TAM. Comput. Educ. 2007, 48, 250–267. [Google Scholar] [CrossRef]
  60. Teo, T. Examining the influence of subjective norm and facilitating conditions on the intention to use technology among pre-service teachers: A structural equation modeling of an extended technology acceptance model. Asia Pac. Educ. Rev. 2010, 11, 253–262. [Google Scholar] [CrossRef]
  61. Ajibade, P. Technology acceptance model limitations and criticisms: Exploring the practical applications and use in technology-related Studies, mixed-method, and qualitative researches. Lib. Phil. Prac. 2018. Available online: https://core.ac.uk/download/pdf/189486068.pdf (accessed on 12 March 2021).
  62. Lin, F.H.; Wu, J.H. An empirical study of end-user computing acceptance factors in small and medium enterprises in Taiwan: Analyzed by structural equation modeling. J. Comput. Inf. Syst. 2004, 44, 98–108. [Google Scholar]
  63. Luna-Reyes, L.F.; Gil-Garcia, J.R. Using institutional theory and dynamic simulation to understand complex e-government phenomena. Gov. Inf. Q. 2011, 28, 329–345. [Google Scholar] [CrossRef]
  64. Thornton, P.H.; Ocasio, W. Institutional logics. In Handbook of Organizational Institutionalism; Greenwood, R., Oliver, C., Sahlin-Andersson, K., Suddaby, R., Eds.; Sage: Thousand Oaks, CA, USA, 2008; pp. 165–201. [Google Scholar]
  65. Currie, W.L. Institutional theory of information technology. In The Oxford Handbook of Management Information Systems: Critical Perspectives and New Directions; Gallier, R.D., Currie, W.L., Eds.; Oxford University Press: New York, NY, USA, 2011; pp. 137–173. [Google Scholar]
  66. Butler, T. Institutional change and green IS: Towards problem-driven, mechanism-based explanations. In Information Systems Theory. Integrated Series in Information Systems; Dwivedi, Y., Wade, M., Schneberger, S., Eds.; Springer: New York, NY, USA, 2012; pp. 383–407. [Google Scholar]
  67. Jepperson, R.L. Institutions, institutional effects, and institutionalism. In The New Institutionalism in Organizational Analysis; Powell, W.W., DiMaggio, P.J., Eds.; University of Chicago Press: Chicago, IL, USA, 1991; pp. 143–163. [Google Scholar]
  68. DiMaggio, P.J.; Powell, W.W. The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. Am. Sociol. Rev. 1983, 147–160. [Google Scholar] [CrossRef] [Green Version]
  69. Scott, W.R. Institutions and Organizations, 2nd ed.; Sage: Thousand Oaks, CA, USA, 2001. [Google Scholar]
  70. Scott, W.R. Lords of the dance: Professionals as institutional agents. Organ. Stud. 2008, 29, 219–238. [Google Scholar] [CrossRef] [Green Version]
  71. Barley, S.R.; Tolbert, P.S. Institutionalization and structuration: Studying the links between action and institution. Org. Stud. 1997, 18, 93–117. [Google Scholar] [CrossRef] [Green Version]
  72. Orlikowski, W.J.; Stephen, R.B. Technology and institutions: What can research on information technology and research on organizations learn from each other? MIS Q. 2001, 25, 145–165. [Google Scholar] [CrossRef]
  73. Douglas, M. How Institutions Think; Syracuse University Press: Syracuse, NY, USA, 1986. [Google Scholar]
  74. Ciborra, C.U.; Lanzara, G.F. Formative contexts and information technology: Understanding the dynamics of innovation in organizations. Accou. Manag. Inf. Technol. 1994, 4, 61–86. [Google Scholar] [CrossRef]
  75. Jensen, T.B.; Kjærgaard, A.; Svejvig, P. Using institutional theory with sensemaking theory: A case study of information system implementation in healthcare. J. Inf. Technol. 2009, 24, 343–353. [Google Scholar] [CrossRef]
  76. Scott, W.; Richard, M.; John, W. Institutional Environment and Organizations: Structural Complexity and Individualism; Sage: Thousand Oaks, CA, USA, 1994. [Google Scholar]
  77. Straub, D.; Keil, M.; Brenner, W. Testing the technology acceptance model across cultures: A three country study. Inf. Manag. 1997, 33, 1–11. [Google Scholar] [CrossRef]
  78. Lluch, M. Healthcare professionals’ organisational barriers to health information technologies—A literature review. Inter. J. Med. Inf. 2011, 80, 849–862. [Google Scholar] [CrossRef]
  79. Hofstede, G.; Hofstede, G.J.; Minkov, M. Cultures and Organizations: Software of the Mind; Mcgraw-Hill: New York, NY, USA, 2005. [Google Scholar]
  80. Hofstede, G. Culture’s Consequences: International Differences in Work-Related Values; Sage: Beverly Hills, CA, USA, 1980. [Google Scholar]
  81. McCoy, S.; Galletta, D.F.; King, W.R. Applying TAM across cultures: The need for caution. Eur. J. Inf. Syst. 2007, 16, 81–90. [Google Scholar] [CrossRef]
  82. Triandis, H.C. Cross-cultural social and personality psychology. Pers. Soc. Psychol. Bull. 1977, 3, 143–158. [Google Scholar] [CrossRef]
  83. March, J.G.; Olsen, J.P. Rediscovering Institutions; Simon and Schuster: New York, NY, USA, 1989. [Google Scholar]
  84. Fountain, J.E. Building the Virtual State: Information Technology and Institutional Change; Brookings Institution Press: Washington, DC, USA, 2004. [Google Scholar]
  85. Subramanian, G.H. A replication of perceived usefulness and perceived ease of use measurement. Decis. Sci. 1994, 25, 863–874. [Google Scholar] [CrossRef]
  86. Chau, P.Y. An empirical investigation on factors affecting the acceptance of case by systems developers. Inf. Manag. 1996, 30, 269–280. [Google Scholar] [CrossRef]
  87. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. Extrinsic and intrinsic motivation to use computers in the workplace. J. Appl. Soc. Psychol. 1992, 22, 1111–1132. [Google Scholar] [CrossRef]
  88. Legris, P.; Ingham, J.; Collerette, P. Why do people use information technology? A critical review of the technology acceptance model. Inf. Manag. 2003, 40, 191–204. [Google Scholar] [CrossRef]
  89. Gefen, D.; Straub, D.W. The relative importance of perceived ease of use in IS adoption: A study of e-commerce adoption. J. Assoc. Inf. Syst. 2000, 1, 8. [Google Scholar] [CrossRef] [Green Version]
  90. Abdullah, F.; Ward, R.; Ahmed, E. Investigating the influence of the most commonly used external variables of TAM on students’ perceived ease of use and perceived usefulness of e-portfolios. Comput. Hum. Behav. 2016, 63, 75–90. [Google Scholar] [CrossRef]
  91. Keil, M.; Beranek, P.M.; Konsynski, B.R. Usefulness and ease of use: Field study evidence regarding task considerations. Decis. Support Syst. 1995, 13, 75–91. [Google Scholar] [CrossRef]
  92. Kaplan, D. Structural Equation Modeling: Foundations and Extensions; Sage Publications: Thousand Oaks, CA, USA, 2008. [Google Scholar]
  93. Hevner, A.R.; March, S.T.; Park, J.; Ram, S. Design science in information systems research. MIS Q. 2004, 28, 75–105. [Google Scholar] [CrossRef] [Green Version]
  94. Hevner, A.; Chatterjee, S. Design science research in information systems. In Design Research in Information Systems; Springer: Boston, MA, USA, 2010; pp. 9–22. [Google Scholar]
  95. Arnott, D.; Pervan, G. Design science in decision support systems research: An assessment using the Hevner, March, Park, and Ram Guidelines. J. Assoc. Inf. Syst. 2012, 13, 1. [Google Scholar] [CrossRef]
  96. Orlikowski, W.J. Using technology and constituting structures: A practice lens for studying technology in organizations. Organ. Sci. 2000, 11, 404–428. [Google Scholar] [CrossRef]
  97. Lee, A. Systems thinking, design science, and paradigms: Heeding three lessons from the past to resolve three dilemmas in the present to direct a trajectory for future research in the information systems field, keynote address. In Proceedings of the Eleventh International Conference on Information Management, Taipei, Taiwan, 20 May 2000. [Google Scholar]
  98. Hair, J.; Anderson, R.; Tatham, R.; Black, W.l. Multivariate Data Analysis, 5th ed.; Prentice-Hall: Upper Saddle River, NJ, USA, 1998. [Google Scholar]
  99. Stevens, J. Applied Multivariate Statistics for the Behavioral Sciences; Erlbaum: Mahwah, NJ, USA, 1996. [Google Scholar]
  100. Sung, W.; Kim, C. A study on the effect of change management on organizational Innovation: Focusing on the mediating effect of members’ innovative behavior. Sustainability 2021, 13, 2079. [Google Scholar] [CrossRef]
  101. Weston, R.; Gore, P.A., Jr. A brief guide to structural equation modeling. Couns. Psychol. 2006, 34, 719–751. [Google Scholar] [CrossRef]
  102. Hair, J.F., Jr.; Matthews, L.M.; Matthews, R.L.; Sarstedt, M. PLS-SEM or CB-SEM: Updated guidelines on which method to use. Int. J. Multivar. Data Anal. 2017, 1, 107–123. [Google Scholar] [CrossRef]
  103. Hair, J.F., Jr.; Sarstedt, M. Factors versus composites: Guidelines for choosing the right structural equation modeling method. Proj. Manag. J. 2019, 50, 619–624. [Google Scholar] [CrossRef]
  104. Ashok, M.; Narula, R.; Martinez-Noya, A. How do collaboration and investments in knowledge management affect process innovation in services? J. Knowl. Manag. 2016, 20, 1004–1024. [Google Scholar] [CrossRef]
  105. Bentler, P.M.; Yuan, K.H. Structural equation modeling with small samples: Test statistics. Multivar. Behav. Res. 1999, 34, 181–197. [Google Scholar] [CrossRef]
  106. Bentler, P.M.; Chou, C. Practical issues in structural modeling. Sociol. Methods Res. 1987, 16, 78–117. [Google Scholar] [CrossRef]
  107. Jöreskog, K.G.; Sörbom, D. LISREL 7 User’s Reference Guide; SPSS Publications: Chicago, IL, USA, 1989. [Google Scholar]
  108. Mitchell, R.J. Path analysis: Pollination. In Design and Analysis of Ecological Experiments; Scheiner, S.M., Gurevitch, J., Eds.; Chapman and Hall: New York, NY, USA, 1993; pp. 211–231. [Google Scholar]
  109. KIPA (Korea Institute of Public Administration). A Survey Results of Public Service Innovation through the Use of Information Technology in the Fourth Industrial Revolution: A Research Paper. Seoul, Korea. 2018. Available online: https://www.kipa.re.kr/site/mblk/sta/selectBaseView.do?gubun=IA&seqNo=STAT_000000000001000 (accessed on 2 May 2020).
  110. Jackson, D.L. Revisiting sample size and number of parameter estimates: Some support for the n:q hypothesis. Struct. Equ. Model. A Multidiscip. J. 2003, 10, 128–141. [Google Scholar] [CrossRef]
  111. Kline, R.B. Principles and Practice of Structural Equation Modeling; Guilford Publications: New York, NY, USA, 2016. [Google Scholar]
  112. Barrett, P. Structural equation modelling: Adjudging model fit. Pers. Individ. Differ 2007, 42, 815–824. [Google Scholar] [CrossRef]
  113. Fountain, J.E. Bureaucratic Reform and E-Government in the United States: An Institutional Perspective; National Center for Digital government; Routledge: New York, NY, USA, 2008. [Google Scholar]
  114. Moore, G.C.; Benbasat, I. Development of an instrument to measure the perceptions of adopting an information technology innovation. Inf. Syst. Res. 1991, 2, 192–222. [Google Scholar] [CrossRef] [Green Version]
  115. Orcan, F. Exploratory and confirmatory factor analysis: Which one to use first. J. Meas. Eval. Educ. Psychol. 2018, 9, 414–421. [Google Scholar] [CrossRef] [Green Version]
  116. Worthington, R.L.; Whittaker, T.A. Scale development research: A content analysis and recommendations for best practices. Couns. Psychol. 2006, 34, 806–838. [Google Scholar] [CrossRef]
  117. Harman, H.H. Modern Factor Analysis; Chicago Univer: Chicago, IL, USA, 1960. [Google Scholar]
  118. Costello, A.B.; Osborne, J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Prac. Assess Res. Eval. 2005, 10, 1–9. [Google Scholar]
  119. Anderson, J.C.; Gerbing, D.D. Structural equation modeling in practice: A review and recommended two-step approach. Psychol. Bull. 1988, 103, 411–423. [Google Scholar] [CrossRef]
  120. Williams, L.J.; Vandenberg, R.J.; Edwards, J.R. 12 Structural equation modeling in management research: A guide for improved analysis. Acad. Manag. Ann. 2009, 3, 543–604. [Google Scholar] [CrossRef]
  121. Nunally, J.C.; Bernstein, I.H. Psychometric Theory; McGraw-Hill: New York, NY, USA, 1994. [Google Scholar]
  122. Klein, H.K.; Hirschheim, R.A. Legitimation in information systems development: A social change perspective. Off. Technol. People 1989, 5, 29–46. [Google Scholar] [CrossRef]
  123. Chen, A.J.W.; Boudreau, M.; Watson, R.T. Information systems and ecological sustainability. J. Syst. Inf. Technol. 2008, 10, 186–201. [Google Scholar] [CrossRef]
  124. Fox, J. Effect analysis in structural equation models: Extensions and simplified methods of computation. Sociol. Methods Res. 1980, 9, 3–28. [Google Scholar] [CrossRef]
  125. Meyer, J.W.; Rowan, B. Institutionalized organizations: Formal structure as myth and ceremony. Am. J. Sociol. 1977, 83, 340–363. [Google Scholar] [CrossRef] [Green Version]
  126. Pfeffer, J. New Directions for Organization Theory: Problems and Prospects; Oxford University Press on Demand: Oxford, UK, 1997. [Google Scholar]
  127. Scott, W.R. Institutions and Organizations: Ideas, Interests, and Identities; Sage Publications: Thousand Oaks, CA, USA, 2013. [Google Scholar]
  128. Hu, Q.; Hart, P.; Cooke, D. The role of external and internal influences on information systems security–a neo-institutional perspective. J. Strateg. Inf. Syst. 2007, 16, 153–172. [Google Scholar] [CrossRef]
  129. Pierson, P. Public policies as institutions. In Rethinking Political Institutions: The Art of the State; Shapiro, I., Skowronek, S., Galvin., D., Eds.; New York University Press: New York, NY, USA, 2006; pp. 114–131. [Google Scholar]
  130. Fountain, J. On the effects of e-government on political institutions. In Routledge Handbook of Science, Technology and Society; Kleinman, D.L., Moore, K., Eds.; Routledge: London, UK, 2014; pp. 471–488. [Google Scholar]
  131. Lips-Wiersma, M.; Hall, D. Organizational career development is not dead: A case study on managing the new career during organizational change. J. Organ. Behav. 2007, 28, 771–792. [Google Scholar] [CrossRef]
  132. Lloyd, K.; Boer, D.; Keller, J.; Voelpel, S. Is my boss really listening to me? The impact of perceived supervisor listening on emotional exhaustion, turnover intention, and organizational citizenship behavior. J. Bus. Ethic 2015, 130, 509–524. [Google Scholar] [CrossRef]
  133. Robbins, S.F.; Judge, T.A. Organizational Behavior, 12th ed.; Pearson Prentice Hall: Upper Saddle River, NJ, USA, 2007. [Google Scholar]
  134. Venkatesh, V. Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Inf. Syst. Res. 2000, 11, 342–365. [Google Scholar] [CrossRef] [Green Version]
  135. Weinberg, B.A. Experience and Technology Adoption; IZA Discussion Paper No. 1051; Institute of Labor Economics: Bonn, Germany, 2004. [Google Scholar]
  136. Zuboff, S. In the Age of the Smart Machine: The Future of Work and Power; Basic Books Inc.: New York, NY, USA, 1984. [Google Scholar]
  137. Borkovich, D.J.; Skovira, R.J.; Breese-Vitelli, J. New technology adoption: Embracing cultural influences. Issue Inf. Syst. 2015, 16, 138–147. [Google Scholar]
  138. Flyverbom, M.; Deibert, R.; Matten, D. The governance of digital technology, big data, and the internet: New roles and responsibilities for business. Buss. Sociol. 2019, 58, 3–19. [Google Scholar] [CrossRef]
  139. Strickland, L.S.; Hunt, L.E. Technology, security, and individual privacy: New tools, new threats, and new public perceptions. J. Am. Sociol. Inf. Sci. Technol. 2005, 56, 221–234. [Google Scholar] [CrossRef]
  140. Hancher-Rauch, H.L.; Bishop, C.; Campbell, A.; Cecil, K.; Yazel, L. Effects of COVID-19 pandemic on the professional roles and responsibilities of health educators. Health Promot. Pr. 2021, 22, 156–162. [Google Scholar] [CrossRef]
  141. Fitzgerald, M.; Kruschwitz, N.; Bonnet, D.; Welch, M. Embracing digital technology: A new strategic imperative. MIT Sloan Manag. Rev. 2014, 55, 1. [Google Scholar]
  142. Harrison, D.A.; Mykytyn, P.P., Jr.; Riemenschneider, C.K. Executive decisions about adoption of information technology in small business: Theory and empirical tests. Inf. Syst. Res. 1997, 8, 171–195. [Google Scholar] [CrossRef]
  143. Gagnon, Y.C. The behavior of public managers in adopting new technologies. Public Perform Manag. Rev. 2001, 24, 337–350. [Google Scholar] [CrossRef]
  144. Alekseev, A.N.; Buraeva, E.V.; Kletskova, E.V.; Rykhtikova, N.A. Stages of formation of industry 4.0 and the key indicators of its development. In Industry 4.0: Industrial Revolution of the 21st Century; Elena, G.P., Yulia, V.R., Aleksei, V.B., Eds.; Springer: Cham, Switzerland, 2019; pp. 93–100. [Google Scholar]
  145. Kulin, J.; Sevä, I.J. The role of government in protecting the environment: Quality of government and the translation of normative views about government responsibility into spending preferences. Int. J. Sociol. 2019, 49, 110–129. [Google Scholar] [CrossRef] [Green Version]
  146. Schuster, C.; Weitzman, L.; Sass Mikkelsen, K.; Meyer-Sahling, J.; Bersch, K.; Fukuyama, F.; Kay, K. Responding to COVID-19 through surveys of Public Servants. Pub. Adm. Rev. 2020, 80, 792–796. [Google Scholar] [CrossRef]
  147. Tejedor, S.; Pérez-Escoda, A.; Ventín, A.; Tusa, F.; Martínez, F. Tracking Websites’ Digital Communication Strategies in Latin American Hospitals During the COVID-19 Pandemic. Int. J. Environ. Res. Public Heal. 2020, 17, 9145. [Google Scholar] [CrossRef] [PubMed]
  148. Ahmad, A.; Rasul, T.; Yousaf, A.; Zaman, U. Understanding Factors Influencing Elderly Diabetic Patients’ Continuance Intention to Use Digital Health Wearables: Extending the Technology Acceptance Model (TAM). J. Open Innov. Technol. Mark. Complex. 2020, 6, 81. [Google Scholar] [CrossRef]
  149. Tulenko, K.; Vervoort, D. Cracks in the system: The effects of the Coronavirus pandemic on public health systems. Am. Rev. Public Adm. 2020, 50, 455–466. [Google Scholar] [CrossRef]
  150. Horst, J.; Santiago, F. What Can Policymakers Learn from Germany’s Industrie 4.0 Development Strategy; Inclusive and Sustainable Industrial Development; Working Paper Series, WP, 22; United Nations Industrial Development Organization: Vienna, Austria, 2018; pp. 1–35. [Google Scholar]
  151. Corrocher, N.; Fontana, R. Objectives; Obstacles and drivers of ICT adoption: What do IT managers perceive? Inf. Econo. Pol. 2008, 20, 229–242. [Google Scholar] [CrossRef]
  152. Holden, H.; Rada, R. Understanding the influence of perceived usability and technology self-efficacy on teachers’ technology acceptance. J. Res. Technol. Educ. 2011, 43, 343–367. [Google Scholar] [CrossRef] [Green Version]
  153. Soh, C.; Connolly, D. New frontiers of profit and risk: The Fourth Industrial Revolution’s impact on business and human rights. New Political Econ. 2021, 26, 168–185. [Google Scholar] [CrossRef]
  154. Capano, G.; Woo, J.J. Resilience and robustness in policy design: A critical appraisal. Policy Sci. 2017, 50, 399–426. [Google Scholar] [CrossRef]
  155. Kelly, J.A. The New Covid-19 home office worker: Evolving computer-human interactions and the perceived value of workplace technology. Tech. Soc. Sci. J. 2020, 13, 575. [Google Scholar] [CrossRef]
  156. Woo, J.J. Capacity-building and Pandemics: Singapore’s Response to Covid-19; Springer Nature: Berlin/Heidelberg, Germany, 2020. [Google Scholar]
  157. Staley, B.; O’Boyle, J.; Armstrong, E.; Coonan, E.; Taylor, L.; Dutton, J. The impact of COVID-19 on professional practice in the northern territory, Australia. Perspect. Asha Spec. Int. Groups 2020, 5, 1789–1792. [Google Scholar] [CrossRef]
  158. Budd, J.; Miller, B.S.; Manning, E.M.; Lampos, V.; Zhuang, M.; Edelstein, M.; McKendry, R.A. Digital technologies in the public-health response to COVID-19. Nat. Med. 2020, 26, 1183–1192. [Google Scholar] [CrossRef]
  159. Stone-Johnson, C.; Weiner, J.M. Principal professionalism in the time of COVID-19. J. Prof. Cap. Community 2020, 5, 367–374. [Google Scholar] [CrossRef]
  160. Roca, J.B.; Vaishnav, P.; Morgan, G.M.; Fuchs, E.; Mendonça, J. Technology forgiveness: Why emerging technologies differ in their resilience to institutional instability. Technol. Forecast. Soc. Chang. 2021, 166, 120599. [Google Scholar] [CrossRef]
  161. Gupta, B.; Dasgupta, S.; Gupta, A. Adoption of ICT in a government organization in a developing country: An empirical study. J. Strat. Inf. Syst. 2008, 17, 140–154. [Google Scholar] [CrossRef]
  162. Nedović-Budić, Z.; Godschalk, D.R. Human factors in adoption of geographic information systems: A local government case study. Public Adm. Rev. 1996, 554–567. [Google Scholar] [CrossRef]
  163. Andrés, A.R.; Amavilah, V.; Asongu, S. Linkages Between Formal Institutions, ICT Adoption, and Inclusive Human Development in Sub-Saharan Africa. In Catalyzing Development through ICT Adoption; Kaur, H., Lechman, E., Marszk, A., Eds.; Springer: Cham, Switzerland, 2017; pp. 175–203. [Google Scholar]
  164. Lin, C.; Braund, W.E.; Auerbach, J.; Chou, J.H.; Teng, J.H.; Tu, P.; Mullen, J. Policy decisions and use of information technology to fight coronavirus disease, Taiwan. Emerg. Infect Dis. 2020, 26, 1506. [Google Scholar] [CrossRef] [PubMed]
  165. Pagani, M. Determinants of adoption of high speed data services in the business market: Evidence for a combined technology acceptance model with task technology fit model. Inf. Manag. 2006, 43, 847–860. [Google Scholar] [CrossRef]
  166. Shen, C.C.; Chiou, J.S. The impact of perceived ease of use on Internet service adoption: The moderating effects of temporal distance and perceived risk. Comput. Hum. Behav. 2010, 26, 42–50. [Google Scholar] [CrossRef]
  167. House, R.J.; Hanges, P.J.; Javidan, M.; Dorfman, P.W.; Gupta, V. (Eds.) Culture, Leadership, and Organizations; Sage: Thousand Oaks, CA, USA, 2008. [Google Scholar]
Figure 1. Conceptual and hypothesized model for new technology adoption during the fourth industrial revolution.
Figure 1. Conceptual and hypothesized model for new technology adoption during the fourth industrial revolution.
Ijerph 18 05593 g001
Figure 2. The measurement and components of institution.
Figure 2. The measurement and components of institution.
Ijerph 18 05593 g002
Figure 3. Results of the baseline model. Note: N = 300. * p < 0.05, *** p < 0.001.
Figure 3. Results of the baseline model. Note: N = 300. * p < 0.05, *** p < 0.001.
Ijerph 18 05593 g003
Figure 4. Results of the external institution model. Note: N = 300. *** p < 0.001.
Figure 4. Results of the external institution model. Note: N = 300. *** p < 0.001.
Ijerph 18 05593 g004
Figure 5. Results of the internal institution model. Note: N = 300. * p < 0.05, *** p < 0.001.
Figure 5. Results of the internal institution model. Note: N = 300. * p < 0.05, *** p < 0.001.
Ijerph 18 05593 g005
Figure 6. Alternative expanded model for the acceptance of new information technology. Note: N = 300. ** p < 0.01, *** p < 0.001.
Figure 6. Alternative expanded model for the acceptance of new information technology. Note: N = 300. ** p < 0.01, *** p < 0.001.
Ijerph 18 05593 g006
Table 1. Guidelines and principles of information systems by Hevner et al. [93] and its application in this study.
Table 1. Guidelines and principles of information systems by Hevner et al. [93] and its application in this study.
Guidelines and PrinciplesDescriptionApplication for This Study
Design of an artifactDesign science research must produce a viable artifact in the form of a construct, a model, a method, or an instantiation.This study suggests the Institution-based Technology Acceptance Model (ITAM) as a framework to construct more effective information system arrangement and organizational settings in the fourth industrial revolution.
Problem RelevanceThe objective of design science research is to develop technology-based solutions to important and relevant business problems.This study suggests technology-based (security), organization-based (internal institution and manager’s concern), and people-based (attitude toward technology and culture) artifacts to address new technology acceptance issues.
Design EvaluationThe utility, quality, and efficacy of a design artifact must be rigorously demonstrated via a well-executed evaluation method.The proposed model in this study is evaluated for informed argument from the knowledge base (e.g., perception of users) to explain its possible utility. It also integrates prototype IT artifacts that can be mathematically evaluated.
Research ContributionsEffective design science must provide clear and verifiable contributions in the areas of design artifacts, design foundations, and/or design methodologies.This study and the artifact, ITAM provide both research and practical contributions to South Korea and other countries which adopt new information technology in the fourth industrial revolution. The result of this study provides institutional requirements and constraints for new information technology adoption.
Research RigorDesign science research relies upon the application of rigorous methods in both the construction and evaluation of design artifact.Our proposed model addresses possible alternatives for IT artifacts that can be applied to managerial and behavioral changes within appropriate environments. The applicability of the model and causal relationships between latent variables are verified by utilization of sophisticated statistical methods.
Design as a Search ProcessThe search for an effective artifact requires utilizing available means to reach desired ends while satisfying laws in the problem environment.The proposed model provides effective solutions to address new technology acceptance problem. The possible solutions suggested in this study include laws, managerial actions, and technical issues which in turn provide a pragmatic approach for design science research.
Communication of the ResearchDesign science research must be presented effectively both to the technology oriented as well as management-oriented audiences.The results provide both technological and managerial implications to enable the artifact to be implemented. It also provides an analytic framework for researchers as well as managers to evaluate new IT artifact.
Table 2. Characteristics of the respondents.
Table 2. Characteristics of the respondents.
FrequencyPercent
Position3rd62.0
4th3411.3
5th10635.3
6th7123.7
7th4916.3
8th20.7
9th20.7
Other3010.0
GenderMale21571.7
Female8528.3
Total300100.0
Table 3. The meanings and interpretations of statistics in the SEM.
Table 3. The meanings and interpretations of statistics in the SEM.
StatisticsMeaningInterpretation
S.E
(Standard error)
The standard error means an estimate of the standard deviation (S.D) of the coefficient. The standard error allows us to identify the magnitude of error which is made in estimating an outcome variable from an independent variable.
Estimate
(β, standardized coefficient)
The standardized coefficient is calculated by multiplying the unstandardized coefficient by the ratio of standard deviation of explanatory variable and outcome variable. Each of the estimated parameters represents the amount of change in the dependent variable as a function of a single unit change in the explanatory variable.
AVE
(Average Variance Extracted)
AVE is the level of variance that is captured by a construct compared to the level of variance. The suggested threshold that is normally higher than 0.50 would be acceptable.
R2
(R-Square)
As the independent variables are correlated in SEM, the R2 of each estimate indicates the partial effect of each variable on the dependent variable.The R2 of the structural model can be interpreted as a proportion of variance explained. The full structural model relationships between latent variables and direct variables have the highest partial coefficient.
CFI
(The Comparative Fit Index)
CFI is an incremental relative fit index that measures the relative improvement in the fit of the researcher’s model.CFI is a revised form of NFI. It ranged from 0 to 1. The recommended threshold is 0.9 or more.
GFI
(The Goodness of Fit)
GFI is the proportion of variance accounted for by the estimated population covariance.GFI indicates the proportion of variance explained by the estimated population covariance. The recommended threshold is 0.9 or more.
NFI (The Normed-Fit Index)/
TLI (Tucker–Lewis
Index)
NFI indicates whether the proposed model improves the fit compared to the null model. TLI (also called non-Normed-Fit) is preferable for a small sample.NFI ranges between 0 and 1. The recommended threshold for NFI is 0.9 or more.
SRMR (Standardized Root Mean Square Residual)SRMR is the standardized difference between the residuals of the observed sample covariance matrix and the predicted hypothesized model.SRMR ranges between 0 and 1.
The recommended threshold for SRMR is 0.08 or less.
RMSEA (Root Mean Square Error of Approximation)RMSEA represents a parsimony-adjusted index.RMSEA ranges from 0 to 1. Values closer to 0 represent a good fit. The recommended threshold is 0.08 or less.
Table 4. The characteristics of explanatory factor analysis (EFA) and confirmatory factor analysis (CFA).
Table 4. The characteristics of explanatory factor analysis (EFA) and confirmatory factor analysis (CFA).
EFACFA
PurposeDetermining latent variables; Developing scale [115]Investigating model assumption;
Testing validity of items [118]
NecessityExplaining the existing structure [118]Investigating previous proven structure;
Requiring strong model assumption [116]
ProcedureInitial testing between items [115]Following EFA, evaluating or confirming the extent [116]
UsageFactor decision when the number of factors between items is not known; Resulting in a preliminary rather than definite outcome [116]Prior knowledge of the expected relationships between items and factors are required [116]
Table 5. Fit indices for the measurement and structural models.
Table 5. Fit indices for the measurement and structural models.
Fit IndicesRecommended ValueMeasurement ModelStructural Model
CFI>0.900.9650.957
GFI>0.900.9310.924
NFI>0.900.9240.917
TLI>0.900.9560.947
SRMR<0.080.0550.058
RMSEA<0.080.0510.056
Table 6. Results of confirmatory factor analysis (CFA).
Table 6. Results of confirmatory factor analysis (CFA).
Survey ItemsFactor LoadingsS.EAVE
InstitutionI_10.7030.0450.672
I_20.7850.038
I_30.8370.033
I_40.8120.030
I_50.6550.047
I_60.6640.046
PUPU_10.7970.0400.632
PU_20.9060.028
PU_30.5480.065
PU_40.5320.051
PU_50.5920.057
PEOUPEOU_10.7040.0480.581
PEOU_20.5610.059
PEOU_30.7140.050
IUIU_10.5440.0750.701
IU_20.7490.047
IU_30.8320.036
Note: PU: perceived usefulness; PEOU: perceived ease of use; IU: intention to use.
Table 7. Descriptive statistics, reliabilities, and correlations.
Table 7. Descriptive statistics, reliabilities, and correlations.
VariableMeanS.DReliability1234
1. Institution3.030.6360.8961
2. PU3.620.5560.8190.308 **1
3. PEOU3.630.5950.6990.285 **0.409 **1
4. IU3.840.5590.7360.276 **0.551 **0.557 **1
Note: ** p < 0.01, PU: perceived usefulness; PEOU: perceived ease of use; IU: intention to use.
Table 8. Summary of the hypothesis test results (standardized direct effect).
Table 8. Summary of the hypothesis test results (standardized direct effect).
PathEstimateHypothesesTest ResultsR2
InstitutionPEOU0.327 (p = 0.000)H 1_1Supported0.107
Institution
PEOU

PU
PU
0.142 (p = 0.020)
0.554 (p = 0.000)
H 1_1
H3_2
Supported
Supported
0.378
PU
PEOU

IU
IU
0.329 (p = 0.000)
0.547 (p = 0.000)
H2
H3_1
Supported
Supported
0.624
Note: PU: perceived usefulness; PEOU: perceived ease of use; IU: intention to use.
Table 9. Comparison of the goodness-of-fit.
Table 9. Comparison of the goodness-of-fit.
SubmodelsCFIGFINFITLISRMRRMSEA
Submodel 10.9570.9240.9170.9470.0580.056
Submodel 20.9550.9210.9080.9430.0540.053
Submodel 30.9610.9240.9120.9490.0540.049
Submodel 40.9560.9180.9030.9400.0520.049
Note: The dependent variable is intention to use. Bootstrap, bias-corrected two-tailed tests used to calculate significance of the total effects. N = 300.
Table 10. Path coefficients for the alternative models (standardized total effects).
Table 10. Path coefficients for the alternative models (standardized total effects).
VariablesSubmodel 1Submodel 2Submodel 3Submodel 4
Institution0.267 **0.310 *0.370 **0.387 *
Perceived ease of use0.694 *0.737 **0.885 **1.032 *
Perceived usefulness0.274 **0.282 **0.294 *0.277
Experience 0.093 0.085
Voluntariness 0.171 0.173
Specialty −0.192 * −0.264 *
Culture −0.007−0.019
Security −0.284 *−0.296
Manager’s concern −0.0170.028
Note: The dependent variable is intention to use. Bootstrap, bias-corrected two-tailed tests used to calculate significance of the total effects. N = 300. * < 0.05, ** p < 0.01.
Table 11. Theoretical contributions of this study to representative previous research.
Table 11. Theoretical contributions of this study to representative previous research.
ReferenceStudy AimsOur Contributions
Sung
[11]
To analyze practices of the fourth industrial revolution and industry 4.0 plan in Korea, along with guidelines and recommendations; to suggest that institutional infrastructure of central governments to lead all initiatives are requiredBased on the survey of public employees in Korea, our study tested the basic theoretical framework for technological adoption of the fourth industrial revolution practices with the TAM approach.
Safar et al. [12]To examine opinions and attitudes of inhabitants of South India with a survey method; to emphasize insufficient knowledge of the fourth industrial revolution and industry 4.0 of the potential workforce and to suggest education and requalification is necessaryOur study showed the impact of institution on technology adoption of the fourth industrial revolution is critical. In our analysis, institution is a macro-level concept that includes work guidance, manual, plans, and strategies. Thus, institution deals with proper education and requalification to each section
Anton [18]To examine the adoption of new technological processes of public employees of internal call centers with the TAM approach; to emphasize the role of previous experience of public employees on technology, and to suggest further investigation on the effect of the environmental factors is requiredFollowing the suggestion, our study focused on the influence of institution on technology adoption, especially huge dynamic changes of the fourth industrial revolution on public employees
Baldwin [19]To examine ICT use of public employees in New Zealand and whether technological development could comprehensively change administrative process; to suggest technology usage is not just a technical issue but managerial investment is neededThe purpose of our study is to find causal factors of new technology adoption by public employees. Our study proved that institution strongly influences TAM framework. The alternative model also showed that institution does not unintentionally relate to technology usefulness in the fourth industrial revolution.
Pfeiffer [20]To discuss current status of the fourth industrial revolution with in-depth analysis; to suggest further investigation of actors in various sectors about the trendsOur study aimed to investigate the adoptative behaviors of technology practices in the fourth industrial revolution, with a focus on public employees for diverse analysis about the current issue.
Lee et al. [21]To suggest various recommendations with brainstorming techniques on the fourth industrial revolution; to emphasize the role of institution in increasing creativity in organization.Our study empirically analyzed the influence of institution with the TAM model. We showed the critical role of institution on the development of the fourth industrial revolution.
Venkatesh and Davis [24]To examine the impact of subjective and individual factors by extending TAM to address causal antecedents; to suggest adding designing patterns and system uses for structural consideration and functional designFollowing the extended TAM, our study applied significant extensive factors like culture, experience, and voluntariness on our alternative model to confirm its role in the model. For extending theoretical constructs, our model focused on the role of institution as structural prerequisite for TAM to find whether there is causal antecedent with TAM.
Reischauer [30]To discuss and clarify the contents and identity of the fourth industrial revolution and Industry 4.0; to address various policy implications including the development institutionalization of the fourth industrial revolution for innovationOur study empirically tested the impact of institution on technology adoption of the fourth industrial revolution by survey of public employees to confirm the role of institution is critical.
Horst and Santiago [150]To review and discuss the role of actors in policy process in various countries; to suggest that an institutionalized platform reframed and managed by the government is necessaryOur study investigated institution as a significant factor on technology adoption and examined its influence on the technology adoptive behavior of public employees.
Liao [33]To review and identify influential public policy and challenges by cross country comparison; to suggest various policy implications for inclusion in clear guidelines and process for policy implementationOur study regarded institution as a composition of liability, structural formation, and procedural requirements for empirical test.
Corrocher et al. [151]To examine the obstacles and drivers of ICT adoption by surveying IT managers in Italy; to suggest from their empirical findings that contexts, compatible standards, and information diffusion are significant. Furthermore, authors indicated the sensitivity of institutional environment is strong and critical.Following empirical results of ICT adoption, we empirically verified whether the role of institution is still valuable in the new technology context of the fourth industrial revolution.
Fountain [130]To explain how information technologies affects decision-making in complex organizations, especially with theoretical and qualitative approaches to the institutional perspectiveAdopting the idea of basic framework, our study tried to confirm the role of institution on technology adoption with empirical results.
Verma, Bhattacharyya and Kumar [47]To empirically examine TAM with the system characteristics of quality and belief as causal antecedents; to suggest the influence of system and integrated model are neededAdopting the idea of system characteristics, our study included institution with TAM to analyze its impact on technology adoption of the fourth industrial revolution.
Holden and Rada [152]To apply TAM with extensive variables of self-efficacy on attitudes toward using; to suggest that the role of external variable needs to be studiedTo examine the impact of external variables, we selected institution as a significant external variable in the emerging trends of the fourth industrial revolution
Alekseev et al.
[144]
To review and analyze the process of formation of Industry 4.0; to suggest possible barriers and overcoming strategies in each stage, technology usefulness would be a key factor among themTo follow their proposition, our study examined the role of technology usefulness with extensive institutional TAM by empirically tested.
Agarwal and Prasad [56]To examine extended TAM with emotional variables like efficacy, anxiety, and managerial variables like innovativeness on the adoption of mobile-based money; to confirm significant impact of perceived ease of use in the model and also suggest the use of SEM technique to control the issues of endogeneity issues in TAM.Following their findings to the context of the fourth industrial revolution, we regarded perceived ease of use of new technology as an important mediating factor in the model.
Luna-Reyes and Gil-Garcia [63]To analyze e-Government failure with regard to focus on ICT perspective on case study approach, authors demonstrated the important relationships between institutions, organization forms, and technology adoption of e-GovernmentBased on qualitative analysis by authors, our study focused on the role of institutions on new technology adoption of the fourth industrial revolution.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kang, Y.; Choi, N.; Kim, S. Searching for New Model of Digital Informatics for Human–Computer Interaction: Testing the Institution-Based Technology Acceptance Model (ITAM). Int. J. Environ. Res. Public Health 2021, 18, 5593. https://doi.org/10.3390/ijerph18115593

AMA Style

Kang Y, Choi N, Kim S. Searching for New Model of Digital Informatics for Human–Computer Interaction: Testing the Institution-Based Technology Acceptance Model (ITAM). International Journal of Environmental Research and Public Health. 2021; 18(11):5593. https://doi.org/10.3390/ijerph18115593

Chicago/Turabian Style

Kang, Youngcheoul, Nakbum Choi, and Seoyong Kim. 2021. "Searching for New Model of Digital Informatics for Human–Computer Interaction: Testing the Institution-Based Technology Acceptance Model (ITAM)" International Journal of Environmental Research and Public Health 18, no. 11: 5593. https://doi.org/10.3390/ijerph18115593

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop