Next Article in Journal
Intrapreneurial Fit and Misfit: Enterprising Behavior, Preferred Organizational and Open Innovation Culture
Next Article in Special Issue
Multidexterity—A New Metaphor for Open Innovation
Previous Article in Journal
Modeling a Logistics Hub Using the Digital Footprint Method—The Implication for Open Innovation Engineering
Previous Article in Special Issue
Linking Business Ecosystem and Natural Ecosystem Together—A Sustainable Pathway for Future Industrialization
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JOItmC
Volume 7
Issue 1
10.3390/joitmc7010060
JOItmC-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Effect of Healthcare Data Analytics Training on Knowledge Management: A Quasi-Experimental Field Study

by
Byeong Jo Kim
1,* and
Maria Tomprou
2
1
Graduate School of Public Administration, Seoul National University, Seoul 08826, Korea
2
Tepper School of Business, Carnegie Mellon University, Pittsburgh, PA 15213, USA
*
Author to whom correspondence should be addressed.
J. Open Innov. Technol. Mark. Complex. 2021, 7(1), 60; https://doi.org/10.3390/joitmc7010060
Submission received: 19 December 2020 / Revised: 27 January 2021 / Accepted: 3 February 2021 / Published: 7 February 2021
(This article belongs to the Special Issue Ambidextrous Open Innovation: Technology, Market and Complexity)
Versions Notes

Abstract

:
This study aims to provide practice-oriented evidence regarding the implementation of healthcare data analytics and its impact on the use of new data analytics tools and relevant analytical skills improvement. A quasi-experimental pre-test/post-test controlled study was conducted in a large medical system in the eastern United States. Healthcare data analytics training program participants (N = 21) and a comparison group comprising trainee-identified peers completing comparable work (N = 27) were compared at the start of training and one year later. Results showed that both trainees and peers demonstrated improved healthcare data analytics skills over time, related to concomitant increases in their healthcare data analytics-related learning and performance goals. This study suggests that healthcare organizations aiming at successfully implementing a new data analytics infrastructure should provide well-designed training that enables trainees to develop specific learning and performance goals as well as improve relevant skills and ability to use new tools.

1. Introduction

There is a high demand for organizational leaders and managers in healthcare to be able to implement effective data analytics systems [1,2,3]. Data analytics implementation facilitates the intensive use of diverse forms of organizational data, permits systematic attention to be given towards the quality of information available to practitioners, and uses statistics and modeling to inform organizational decisions [1]. For example, data analytics in healthcare systems can help derive insights ranging from systematic waste of resources to predicting trends, improving outreach, and managing the spread of diseases. Thus, it can improve areas ranging from business management to patient care quality and diagnosis, to name just a few. As part of the larger practice of attending to the quality of evidence used in management decisions [4,5], colleges have developed new programs and courses and firms have invested in new positions and infrastructure to promote the use of such healthcare data analytics. Though relatively recent and limited, academic research suggests that the use of healthcare data analytics is positively related to organizational performance, such as asset utilization and market value [6], return on investment (ROI) and sales growth [7], and supply chain performance [8]. For example, a recent study of data analytics found that healthcare data analytics accounted for 53% of the variation in organizational performance (ROI, sales growth) even when firm size and date of adoption were controlled [7]. As a consequence, data analytics competencies are increasingly valued in healthcare.
The present study contributes to emerging research on the implementation of healthcare data analytics by offering a case and quasi-field experiment of a large-sized hospital system’s data analytics implementation. As a form of practice-oriented research [9], it examines the impact of a training program commissioned and developed by executives leading healthcare data analytics implementation; in doing so, it seeks to provide evidence that is useful to practitioners in regard to the challenges of healthcare data analytics implementation and ways these might be addressed. The requisite skills necessary for healthcare data analytics implementation represent an important consideration. Huselid [3] focuses on critical thinking and excellent communication skills for healthcare data analytics in the context of human resource management. Other scholars maintain that training in support of healthcare data analytics should have a dual focus, domain knowledge related to analytical and IT skills on the one hand, and communication skills to disseminate and improve decision making in complex data-centric business environments on the other [1,2]. The present study provides empirical evidence regarding both the role of technical competencies and change-related skills in healthcare data analytics implementation. Through a quasi-field experiment, our study examines the effects of a training program undertaken to carry out an organization-wide project on healthcare data analytics implementation. Because of the increased use of healthcare data analytics and the inter-departmental coordination challenges such initiatives can bring [2], investigating the role of training in implementing healthcare data analytics has practical importance for human resource managers and change agents.
From an organizational change and open innovation perspective [10,11], it is also essential to understand the experiences of employees whose work is affected by healthcare data analytics implementation. Previous research suggests that adopting healthcare data analytics is a painstaking process in which even the system’s early adopters report continuing implementation problems over time [12,13]. In this context, we consider the contributions that motivational factors may have on such implementation. Specifically, we investigate the role that personal goals and leader vision may play in the acquisition of healthcare data analytics-related skills and the use of healthcare data analytics-related tools. In doing so, we seek to identify ways healthcare data analytics implementation can be supported in addition to training.
In the present study, we evaluate a training intervention on the first day of training and then a year later. Trainees are compared with peers who did not receive training, whom they identify as involved in healthcare data analytics implementation. The use of new skills at work is a critical indicator of training transfer and success [14,15]. Two outcome variables are used to assess the training transfer: First, the use of healthcare data analytics tools indicates on-the-job behavior and application. Second, increased healthcare data analytics skills indicates the acquisition of competencies that support the diffusion of the healthcare data analytics initiative; particularly important for the solving of on-going implementation challenges. The further part of the present study is organized as follows: Section 2 reviews the relevant literature and develops hypotheses, Section 3 describes research context, data collection, and the measurement of variables, Section 4 delineates the results of data analysis, Section 5 discusses the meanings of our findings, and finally, Section 6 draws conclusions for research and comments on the practical implications of our findings.

2. Relevant Literature and Hypotheses

2.1. Training

Training is a formal, planned effort to help employees obtain job-relevant knowledge and skills [16,17]. It is the subject of several major literature reviews [15,18,19,20,21] and at least five meta-analyses have been conducted on its effectiveness [22,23,24,25,26,27]. Together, the body of evidence suggests that organizational training is widely effective, though its success varies as a function of the training method used, the skills and tasks for which employees receive training, and the criteria used to measure the effectiveness. Further, most reviews conclude that successful training implementation depends on trainee motivation to apply new skills and the extent to which the working environment favors and supports the use of such new skills [28]. Training transfer thus depends on both individual (personal motivation) and organizational (opportunities for on-the-job practice) factors. Although an exhaustive review of the training literature is outside the scope of the current study, we rely on specific well-supported training findings to guide our study of one case of organizational training in the context of healthcare data analytics implementation.

2.2. Data Analytics Training and Employee Outcomes

Training in support of healthcare data analytics implementation can contribute to its success by helping employees acquire data analytics-specific skills. Data analytics skills are thought to range from IT competencies (e.g., use of relevant software to inform decisions), critical thinking (e.g., evaluating the quality of data), and forecast modeling (e.g., developing decision trees) to the ability to diffuse data-driven results to other units and contribute accordingly to decision making [1,29]. An additional competency of healthcare employees who use such tools is the ability to critically evaluate and appraise the quality of the evidence they use [4]. Appraising evidence quality and the use of scientific evidence on the job are evidence-based decision-making competencies that manifest when employees know how to formulate critical questions and hypotheses, test them using organizational and scientific evidence, and make evidence-informed decisions in a reflective manner.
Nonetheless, the ability to use healthcare data analytics tools and practice evidence-based decision-making are important, but unlikely to be sufficient for successful healthcare data analytics implementation. The benefits of implementing healthcare data analytics appear to derive from making data analysis a business process rather than a merely analytic or technical one [2]. Data analytics implementation requires problem-solving and change implementation competencies in order to overcome the tendency of organizations to work in silos [30]. Thus, healthcare data analytics-related training should help employees to develop implementation-related skills that promote the diffusion of healthcare data analytics across the organization. In the context of the healthcare data analytics implementation studied here, a skill assessment conducted on behalf of the organization, advised that successful implementation required employees to develop competencies they need in order to act as change agents and promoters of healthcare data analytics. These implementation competencies refer to the ability of change agents to help others use healthcare data analytics and support related practices.
Hypothesis 1.
Participation in a data analytics training will be positively related to healthcare data analytics skills (a) and the use of healthcare data analytics tools (b).

2.3. The Role of Personal Goals

Implementation of healthcare data analytics is a particular type of organizational change [31]. Similar to other types organizational change, such implementation requires employees to replace old habits with new work practices (e.g., by replacing more straightforward analytics with sophisticated modeling), redefine their performance standards (e.g., serving new internal data clients), and learn new ways of working (e.g., helping others learn new tools). Such changes are more likely to be effective when individuals are motivated to both learn new things and perform in new ways. Goals are internal standards that individuals are motivated to adopt that, in turn, direct their attention and effort in ways that help realize them [32,33]. In the context of change, we posit that both learning and performance goals are relevant. First, employees need to be motivated to learn the new skills and knowledge that would enable them to perform in new ways; second, they need to be motivated to perform in the new ways that change their goals. The importance of both learning and performance goals to organizational change is illustrated by LePine [34], who through a decision-making simulation, investigated the ability of teams to adapt to change. Results indicated that when the change occurred, the team members that adjust best were those with a high level of performance goals that were willing to learn. When teams were low on learning motivation and high on performance goals, the effects were detrimental. In the context of complex change, such as healthcare data analytics implementation, employees need to be motivated to pursue both new learning goals and performance goals in order to implement new work practices. We hypothesize that employees whose work is affected by healthcare data analytics-related changes will adapt better (i.e., increased use of healthcare data analytics tools and healthcare data analytics skills) when they have higher learning and performance goals.
Hypothesis 2.
Learning and performance goals will be positively related to data analytics-related outcomes such that learning goals will be positively related to healthcare data analytics skills (a) and the use of healthcare data analytics tools (b), and performance goals will be positively related to healthcare data analytics skills (c) and the use of healthcare data analytics tools (d).

2.4. Moderating Role of Leader Vision

An important impediment for widespread healthcare data analytics implementation is attributed to the absence of top management support [35]. On the other hand, support from top management is believed to be a key feature of the successful implementation of healthcare data analytics within an organization [13]. Based on interviews with healthcare data analytics implementers, the researchers conclude that leader vision is one of the critical factors in healthcare data analytics implementation; a weak vision that is not thoroughly understood can undermine the adoption of healthcare data analytics, whereas a strong vision provides clearer messages about organizational strategy and direction [13]. As such, leader vision, communication, and support for a distant and challenging goal are expected to spur excitement among employees by providing them with a direction and a sense of purpose [36]. Leader vision can help employees to focus their attention on the performance and learning goals important for change implementation [37,38]. Research has shown that employees open to change are more likely to adapt when they have a strong leader vision [39]; for employees who are less open to change, that strong vision also has positive effects, although to a lesser extent [40]. Such findings suggest that when employees are motivated to engage in change, they are more likely to do so where leader vision is strong. We note that experiments indicate vision affects individual attitudes but not performance [41]. Moreover, Hackman and Wageman [42] suggest that leader vision acts as a moderator, enabling a relationship between individual motivation and performance rather than affecting performance directly. Consequently, we hypothesize that an employee’s perceptions of the leader’s vision will strengthen a positive (high) relationship between their own healthcare data analytics-related goals and outcomes.
Hypothesis 3.
Leader vision will positively moderate the relationship of learning and performance goals with data analytics-related outcomes such that leader vision will positively moderate the relationships between learning goals and healthcare data analytics skills (a), learning goals and the use of healthcare data analytics tools (b), performance goals and healthcare data analytics skills, and performance goals and the use of healthcare data analytics tools (d).

3. Materials and Methods

3.1. Organizational Context

The study’s field site was in a large American medical system in the eastern United States, which we refer to by the pseudonym Eastern Medical. Eastern Medical devoted a multi-million-dollar budget to implementing a database and management system intended to integrate clinical, genomic, financial, administrative, and operational data from over 200 sites within its hospitals, health plans, and related entities. Eastern Medical’s stated ultimate goal was to drive more personalized patient care and better patient outcomes through decisions informed by information obtained by exploiting the distinct advantages derived from access to organization-wide cross-functional data.
The senior executives overseeing healthcare data analytics implementation constituted a 5-person top management team from key areas targeted for data integration. Their strategic focus was supported by an IT leadership team, mainly brought in from industry to implement the healthcare data analytics-related changes. During the initial stages of planning the healthcare data analytics implementation, Eastern Medical commissioned a consulting firm to conduct a needs analysis of the skills, competencies, and infrastructure required for the successful implementation of healthcare data analytics. By identifying several goals for the training program and indicators of healthcare data analytics success, this firm’s final report informed the development of the training program and our assessment of its effects. These included: acquiring a better understanding of requirements for, and use of, a central repository of integrated clinical, consumer, and financial data; increased access and use of these data in producing reports that leverage these data using advanced analytics; and greater capacity to cultivate support for the healthcare data analytics system within the organization.
As part of the healthcare data analytics implementation, a program was co-developed by Eastern Medical’s leadership and a local university in order for the training analysts and managers to act as skilled users and change agents. The program consisted of six courses of six weeks each that took place over a period of nine months. Six cohorts ranging from 12 to 18 participants went through the program at three- to six-month intervals. Course subjects included evidence-based decision-making, statistics and modeling, forecasting, and data mining. We note that we compared our pre-test sample of trainees and peers (N = 138) with masters level students (N = 71) at a local university to gain a sense of the motivation level of our participants. Compared to graduate students (M = 4.82, SD = 1.21), Eastern Medical respondents (M = 5.61, SD = 0.96) scored significantly higher (p < 0.001) in proactivity, as measured by three-item scale of taking charge behavior [43]. This suggests that the individuals studied here tend to be self-starters, motivated to overcome obstacles in pursuit of their goals [44].

3.2. Participants in Training Program and Comparison Group

Seventy-seven employees from five different functions of the organization—finance, provider, research, health plan, and strategic planning—were nominated by their managers to participate in the nine-month training course on the healthcare data analytics training program as trainees. Of these trainees, who constitute our intervention group, 98.7% (N = 76) completed the pre-test during their first week in the program. Their mean age was 39.5 (SD = 9.96); 46% were female. To create a comparison group, trainees were asked to nominate a list of peers who reported to the same manager as trainees and whose job was related to healthcare data analytics but who did not participate in the training program. Among a total of 203 peers identified, 42.4% (N = 86) participated in the pre-test. We treated them as our comparison group. Their mean age was 37.9 (SD = 10.9); 53% were female.

3.3. Procedure

Pre-test and post-test were conducted at the beginning and after the completion of the healthcare data analytics training program. Eight trainees and sixteen peers left the organization in the interim. At the post-test, we invited 68 trainees and 70 peers to participate; 21 trainees and 27 peers responded with an overall response rate of 34.8% (N = 48). Independent sample t-tests between those who participated in post-test (N = 48) and non-participants (N = 90) revealed no significant differences in the pre-test measures of demographics, expertise in healthcare data analytics, and learning or performance goals.

3.4. Measures

We measure the healthcare data analytics-related outcomes with two indicators. First was the extent to which participants were using data analytic tools and second was the extent to which participants were using healthcare data analytics-related skills such as understanding of healthcare data analytics, diffuse of them, and cultivating the support needed for healthcare data analytics implementation. Below is the information on how we measured these outcomes.

3.4.1. Use of Healthcare Data Analytics Tools

At the pre-test, we asked respondents to indicate what proportion of their daily job was devoted to using healthcare data analytics tools. The scale ranged from 1 “0–20%” to 5 “80–100%”. This single item assessed the extent to which trainees and peers had the opportunity to practice healthcare data analytics on the job. At the post-test, we asked respondents to indicate the extent to which their daily use of healthcare data analytics tools had changed over the past year. Their responses ranged from −5 “decreased greatly” to +5 “increased greatly” with 0 “no change”.

3.4.2. Data Analytics Skills

This eight-item scale was developed for this study representing skills identified as important to effective healthcare data analytics implementation. We cross-validated the skills the consulting firm had recommended with findings from extensive interviews with seven of the organization’s top executives and four members of its healthcare data analytics committee. Based on consensus among them, we developed items that assessed an employee’s understanding of healthcare data analytics and the ability to diffuse healthcare data analytics-related knowledge and cultivate support within the organization. Two of the ten items initially developed were dropped because their inclusion led to a two-factor solution with cross-loading of over 0.4 at the post-test. Eight items were factor analyzed using principal component analysis with varimax rotation. The analysis yielded a single factor explaining 74.6% of the variance at the pre-test and 65.5% of the variance at the post-test. All eight had primary loadings over 0.6. A sample item is “I am effective in cultivating support for use of advanced healthcare data analytics”. Responses ranged from 1 “Strongly disagree” to 6 “Strongly agree”. Cronbach’s alphas were 0.95 at pre-test and 0.92 at post-test.

3.4.3. Learning and Performance Goals

An individual’s motivation regarding healthcare data analytics implementation was assessed using two 2-item measures, assessing the extent individuals had set healthcare data analytics-related learning and performance goals, respectively. A sample learning item is “I have set specific goals for developing my advanced healthcare data analytics-related skills” and a sample performance item is “I have set specific milestones for implementing advanced healthcare data analytics”. Responses ranged from 1 “Very inaccurate” to 5 “Very accurate”. Cronbach’s alpha for learning goals was 0.62 at pre-test and 0.55 at post-test; for performance goals 0.92 at pre-test, 0.86 at post-test.

3.4.4. Leader Vision

Three items adopted from Griffin, Parker, and Mason [40] were rephrased to assess a leader’s vision regarding the healthcare data analytics initiative. A sample item is “The leaders of healthcare data analytics (i.e., the development and use of the organization-wide data warehouse) create an exciting and attractive image of where it is going”. Responses ranged from 1 “Very inaccurate” to 5 “Very accurate”. Cronbach’s alpha was 0.87 at pre-test and 0.91 at post-test. A principal component factor analysis of the three items of leader vision using varimax rotation yielded a single factor explaining 80.8% of the variance at pre-test, 85.6% at post-test. All items had primary loadings over 0.8.

3.4.5. Control Variables

We collected categorical data on gender, age, and education. Due to the absence of any correlation with study variables, these were dropped from further analyses. We also controlled pre-existing expertise in healthcare data analytics. A single item at the pre-test assessed each respondent’s self-reported level of healthcare data analytics expertise; responses ranged from 1 “novice” to 5 “expert”. This measure served as a control variable in our analyses.

4. Results

Means, standard deviations, and correlations for our study’s measures at both times are presented in Table 1. Independent sample t-tests between the training group (N = 21) and comparison group (N = 27) revealed no differences in demographics, leader vision, expertise in healthcare data analytics, and use of healthcare data analytics tools at the pre-test. However, the training group rated their healthcare data analytics skills, learning goals and performance goals more highly than did the comparison group (healthcare data analytics skills, t(46) = 1.83, p < 0.08., learning goals (t(45) = 2.80, p < 0.01) and performance goals (t(45) = 2.09, p < 0.04). Next, we tested training and comparison group differences in pre-test outcomes—healthcare data analytics skills and use of healthcare data analytics tools— while controlling for learning and performance goals at the pre-test phase. Multivariate analysis of covariance showed no between-group difference, Wilk’s λ = 0.93, F(2, 39) = 1.59, p = ns, η2 = 0.08. A separate univariate analysis of variance was conducted for each outcome at the pre-test and confirmed no between-group difference.
To test Hypothesis 1, we conducted a multivariate analysis of covariance (See Table 2). Two dependent variables at the post-test are the level of healthcare data analytics skills and the changes in the use of healthcare data analytics tools. Covariates include pre-test scores of use of healthcare data analytics tools and healthcare data analytics skills, which allow the post-test measures to indicate whether a change has occurred from the pre-test. We also controlled for expertise in healthcare data analytics measured at the pre-test. Given the limited statistical power due to our small sample size, we adopted a 0.10 significance level. In an initial analysis, no multivariate effect was obtained, Wilk’s λ = 0.91, F(2, 40) = 2.09, p = ns, η2 = 0.09. However, the assumption of independence of covariates of the ANCOVA test was violated by the initial significant difference between the trainee and comparison group on the pre-test measure of healthcare data analytics skills (t(46) = 1.83, p = 0.08). Instead, we conducted a series of univariate analyses of covariance, which showed effects for healthcare data analytics training on the changes in the use of data analytical tools (F(1, 41) = 3.51, p = 0.07, η2 = 0.08). The effect of training on healthcare data analytics skills, however, was not significant, F(1, 41) = 1.24, p = ns, η2 = 0.03. As such, Hypothesis 1a was not supported.
To verify the multivariate effects of healthcare data analytics training on the use of healthcare data analytics tools (Hypothesis 1b), we re-did the univariate analysis of covariance for the outcome variable alone. Univariate analyses of covariance retained the effect of healthcare data analytics training on the changes in the use of data analytical tools, F(1, 42) = 5.79, p < 0.05, η2 = 0.12. Hypothesis 1b, thus, was supported.
To test Hypothesis 2 on the effect of training-related learning and performance goals on employee outcomes (Hypothesis 2a–d), we conducted a multiple regression analysis, controlling for the pre-test measures of both the dependent and independent variables as well as the participation in the training program (Table 3). We tested the effect of the changes in learning and performance goals on healthcare data analytics skills. Regression analysis indicates effects of learning goals, R2 = 0.23, F(4, 41) = 3.11, p < 0.05, and performance goals, R2 = 0.39, F(4, 41) = 6.64, p < 0.01. healthcare data analytics skills at post-test are positively related to increases in both learning goals (β = 0.33; p = 0.06) and performance goals (β = 0.56; p < 0.01). Thus, Hypotheses 2a and 2c were supported.
Since healthcare data analytics training had no effect on post-test healthcare data analytics skills, our results suggest that increased healthcare data analytics skills were attributable to a broader increase in employee motivation to support the healthcare data analytics initiative rather than to the direct effects of healthcare data analytics training. We found no effect of changes in either learning or performance goals on use of healthcare data analytics tools at post-test (R2 = 0.14, F(4, 41) = 1.57, p = ns; learning goals, β = 0.15, p = ns; performance goals, β = 0.05, p = ns). Thus, hypotheses 2b and 2d were not supported.
Finally, we tested Hypothesis 3, the moderating role of leader vision at the pre-test on learning goal effects at the post-test. The regression equation testing Hypothesis 3a (R2 = 0.24, F(6, 39) = 2.10, p = 0.08) and 3c (R2 = 0.40, F(6, 39) = 4.27, p < 0.01) were significant, but not that for Hypothesis 3b (R2 = 0.21, F(6, 38) = 1.66, p = ns) and 3d (R2 = 0.18, F(6, 38) = 1.37, p = ns). However, no interaction was found between leader vision at pre-test and learning goals at post-test for either healthcare data analytics skills (β = 0.46, p = ns) or use of healthcare data analytics tools (β = 0.49, p = ns). Further, no interaction was found between leader vision at pre-test and performance goals at post-test for either healthcare data analytics skills (β = 0.32, p = ns) or use of healthcare data analytics tools (β = 1.20, p = ns). Thus, Hypothesis 3 was not supported by the data.

5. Discussion

5.1. Healthcare Data Analytics Training and Knowledge Management

This study was designed as a form of practice-oriented research [45]. That is, rather than propose and test new scientific constructs and hypotheses, its focus was on providing evidence that practitioners can use in the implementation of healthcare data analytics. In keeping with this focus, our first set of practice-related hypotheses evaluated a common tactic recommended in healthcare data analytics implementation, a healthcare data analytics-specific training intervention. Two outcome variables used to evaluate training transfer were derived from changes targeted by leaders of the healthcare data analytics intervention, that is, increases in the use of healthcare data analytics tools and the development of healthcare data analytics skills. Based on a quasi-experimental pretest–posttest controlled design, employees receiving healthcare data analytics-related training reported an increase in the use of healthcare data analytics tools in their job, than their peers in the comparison group did. From these results, we conclude that there is practical value in developing and supporting training to give employees working in healthcare data analytics-related jobs critical skills and knowledge to use new tools and to think more critically about issues of evidence quality. At the same time, we found no evidence that healthcare data analytics implementation-related skills changed due to training, but what this study did observe has implications for healthcare data analytics implementation and change management broadly.
The second set of practice-related hypotheses sought to assess whether changes might happen independently of the training program and the mechanisms through which these might occur. In this regard, we return to findings regarding healthcare data analytics skills, specifically that both trainees and non-trainees increased their implementation-related skills over time. Independent of training, the increase in employee motivation over the course of the change corresponded with an increase in their healthcare data analytics skills. This finding aligns with previous research that highlights the importance of employee motivation in organizational change implementation [46].
Further, findings suggest that two motivational factors may be particularly important for healthcare data analytics implementation. Learning goals, where individuals are committed to acquiring the skills and knowledge that a specific healthcare data analytics implementation involves, are likely to be critical since the innovative nature of healthcare data analytics makes it highly unlikely that members will have all requisite capabilities to use healthcare data analytics methodologies at the outset. In line with findings from experiments on employee motivation and training [47], the pressure to quickly introduce healthcare data analytics-related practices can impair the ability of individuals to devote time and effort to acquire important skills and knowledge. Signaling support for learning is important to healthcare data analytics implementation. Although training is one form of learning support, our findings suggest that other forms of learning can occur on-the-job, since non-trainees reported increased healthcare data analytics skills over time. At the same time, performance goals, where individuals are committed to engaging in healthcare data analytics-related practices also appear to contribute to acquiring healthcare data analytics skills. Again, changes in these goals occurred independently of participation in formal healthcare data analytics training. We note that previous findings [34] suggest that when performance goals are supported by learning goals individuals are better able to adapt to and engage in change. Although our study’s design has insufficient power to test for such an interaction, it is likely that the effects of performance goals are enhanced by learning goals, at least in the early stages of healthcare data analytics.
Interestingly despite the considerable attention paid to leader vision in research on organizational change and the implementation of healthcare data analytics, we found no effect on any healthcare data analytics-related outcome measures. Although leader vision is expected to affect individual motivation in the context of change, our results suggest that other factors may motivate individuals to engage in the change. Such factors might include the benefits individuals attribute to the change, based on their professional values, technical knowledge, or prior training. Such perceived benefits are likely to motivate individuals to formulate personal goals in support of the change [33]. This attribution is likely to be particularly relevant in the sample of respondents studied here, as their self-reported proactivity scores were greater than our comparison sample of graduate students, a group where one might otherwise expect proactivity to be reasonably high.

5.2. Healthcare Data Analytics Training, Goals, and Open Innovation

Our study also provides insightful directions for a culture of open innovation applying data analytics and employee motivation. Open innovation requires cultivating a culture of organizational learning, where employees are allowed to experiment, make mistakes and fail, and actively reflect on them [48]. As healthcare organizations seek to identify innovative solutions that are patient-centered optimizing business value, evidence-based practices such as the use of data analytics to improve decisions and change old mindsets can promote such endeavors. However, as our study shows, training does not need to function in a vacuum to promote change and innovation but align with enterprise-wide change initiatives. Such broader initiatives organizational change may create a fruitful ground for employees set personal standards for learning and performance. As enterprise-wide data analytics initiatives are implemented in healthcare, our study provides evidence of how such efforts can contribute to innovative solutions and optimize processes.

6. Conclusions

6.1. Research Implications

One of the main challenges of healthcare data analytics adoption is that it is a long process (years) requiring both cross-level developments in capability and information use and lateral changes in inter-unit communication and coordination [2,7]. We recommend that future research adopts multi-wave methodologies to better understand the challenges of healthcare data analytics implementation. Our findings pertain to the early phases of healthcare data analytics adoption and do not cover anticipated later phases where later steps in the Davenport [2] framework might be attained. Previous research also suggests that social networks are important to organizational change and technology adoption in particular; early adopters can create stronger ties and strong ties that influence diffusion [49,50,51]. Moreover, the lack of the widely expected effect of leader vision in our study opens up the possibility that change can occur even in its absence. We note that “substitutes for leadership” are known to exist in the form of professional norms and strong personal values [52], promoting individual and collective motivation in the absence of strong leaders. In considering an organization’s readiness for change and member willingness to support it, greater research attention should be devoted to non-leader-based alternatives for promoting successful change. Lastly, as practice-oriented research in management is relatively rare [45], we recommend continuing efforts in this regard where the current practices organizations use, their effects and implications for managers and employees are described and evaluated. In doing so, we suggest the use of archival data from the organization as a means of examining effects using data practitioners find relevant.

6.2. Practical Implications

Data analytics trainees increased their use of tools compared to their peers. Training is an important part of healthcare data analytics implementation. At the same time, individuals whether specially trained or not can be motivated to learn, use, and promote healthcare data analytics. Hiring talented people motivated to develop their professional skills can further healthcare data analytics implementation, particularly if the formal environment provides them with opportunities to share knowledge and learn. Signaling support for learning is important for healthcare data analytics implementation. Although training is one form of learning support, our findings suggest that other valuable forms of learning related to healthcare data analytics can occur on-the-job. These can take the form of learning from peers, self-guided readings, lunch-and-learn sessions, and related activities. Change leaders should pay special attention to ways of promoting learning and to clarifying the kinds of performance goals most helpful in supporting the implementation of healthcare data analytics.

6.3. Limitations

As with all studies, there are limitations to this study. Though we adopted a vigorous research design, that is, a quasi-field experiment measuring participants twice; before and after the training and compared them with their peers to examine whether differences were due to the training or enterprise-wide change, attrition occurred during the year in which this study took place in both the organization and our sample. Consequently, the primary limitation of this study is that it is likely to be somewhat underpowered. That is, its small sample size may decrease the likelihood of observing effects that are otherwise meaningful. Another limitation is also employee self-reports that may increase common-source biases.

6.4. Conclusions

Data analytics in healthcare change drastically how hospitals and healthcare enterprises operate. Using practice-oriented evidence, this study conducted a quasi-experimental pre/post-test study to examine the effect of data-analytics training and broader enterprise-wide organizational change in implementing healthcare data analytics. Our findings show that such interventions work in tandem which enables employees to develop their goals both in setting new learning and performance standards as well as skill acquisition and implementation of data-analytics related tools. This study provides practice-oriented evidence regarding the effect of healthcare data analytics on the use of new data analytics tools and the improvement of relevant analytical skills. In doing so, we suggest that healthcare organizations aiming to successfully implement a new data analytics infrastructure should provide well-designed training that enables trainees to develop specific learning and performance goals as well as improve relevant skills and ability to use new tools.

Author Contributions

Conceptualization, B.J.K. and M.T.; methodology, B.J.K.; software, B.J.K.; validation, B.J.K. and M.T.; formal analysis, B.J.K. and M.T.; investigation, B.J.K. and M.T.; resources, B.J.K. and M.T.; data curation, B.J.K. and M.T.; writing—original draft preparation, M.T.; writing—review and editing, M.T.; visualization, B.J.K.; supervision, B.J.K. and M.T.; project administration, B.J.K.; funding acquisition, B.J.K. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2017S1A3A2066084).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Carnegie Mellon University (protocol code IRB00000352, 21 October 2015).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Acknowledgments

The authors would like to thank Denise M. Rousseau for constructive feedback on early draft, and the second author would like to thank the Center for the Future of Work of Heinz College at Carnegie Mellon University for their support.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chen, H.; Chiang, R.H.L.; Storey, V.C. Business Intelligence and Analytics: From Big Data to Big Impact. MIS Q. 2012, 36, 1165–1188. [Google Scholar] [CrossRef]
  2. Davenport, T.H. Competing on Analytics. Harv. Bus. Rev. 2006, 84, 98–107. [Google Scholar]
  3. Huselid, M. Workforce Analytics for Strategy Execution. In The Rise of HR: Wisdom from 73 Thought Leaders; Ulrich, D., Schiemann, W.A., Sartain, L., Eds.; HR Certification Institute: Alexandria, VA, USA, 2015; pp. 301–315. [Google Scholar]
  4. Barends, E.; Rousseau, D.M.; Briner, R.B. Evidence-Based Management: The Basic Principles; The Center for Evidence-Based Management: Oxford, UK, 2014. [Google Scholar]
  5. Pfeffer, J.; Sutton, R.I. Evidence-Based Management. Harv. Bus. Rev. 2006, 84, 62–74. [Google Scholar] [PubMed]
  6. Brynjolfsson, E.; Hitt, L.M.; Kim, H.H. Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electron. J. 2011. [Google Scholar] [CrossRef]
  7. Elbashir, M.Z.; Collier, P.A.; Davern, M.J. Measuring the effects of business intelligence systems: The relationship between business process and organizational performance. Int. J. Account. Inf. Syst. 2008, 9, 135–153. [Google Scholar] [CrossRef]
  8. Trkman, P.; McCormack, K.; de Oliveira, M.P.V.; Ladeira, M.B. The Impact of Business Analytics on Supply Chain Performance. Decis. Support Syst. 2010, 49, 318–327. [Google Scholar] [CrossRef] [Green Version]
  9. Rousseau, D.M. The Oxford Handbook of Evidence-Based Management; Oxford University Press: Oxford, UK, 2012. [Google Scholar]
  10. Yun, J.J.; Zhao, X. Business Model Innovation through a Rectangular Compass: From the Perspective of Open Innovation with Mechanism Design. J. Open Innov. Technol. Mark. Complex. 2020, 6, 131. [Google Scholar] [CrossRef]
  11. Shum, K.; Kodama, F.; Shibata, T. Towards a Longitudinal Outlook on Industry Transition Management. J. Open Innov. Technol. Mark. Complex. 2020, 6, 79. [Google Scholar] [CrossRef]
  12. Davenport, T.; Harris, J.G.; Cantrell, S. Enterprise systems and ongoing process change. Bus. Process. Manag. J. 2004, 10, 16–26. [Google Scholar] [CrossRef]
  13. Yeoh, W.; Koronios, A. Critical Success Factors for Business Intelligence Systems. J. Comput. Inf. Syst. 2013, 50, 23–32. [Google Scholar]
  14. Ford, J.K.; Quiñones, M.A.; Sego, D.J.; Sorra, J.S. Factors Affecting the Opportunity to Perform Trained Tasks on the Job. Pers. Psychol. 2006, 45, 511–527. [Google Scholar] [CrossRef]
  15. Tannenbaum, S.; Yukl, G. Training and Development in Work Organizations. Annu. Rev. Psychol. 1992, 43, 399–441. [Google Scholar] [CrossRef]
  16. Goldstein, I.L.; Ford, J.K. Training in Organizations: Needs Assessment, Development, and Evaluation; Wadsworth Publishing Company: Belmont, CA, USA, 2002. [Google Scholar]
  17. Asim, Z.; Sorooshian, S. Exploring the Role of Knowledge, Innovation and Technology Management (KNIT) Capabilities that Influence Research and Development. J. Open Innov. Technol. Mark. Complex. 2019, 5, 21. [Google Scholar] [CrossRef] [Green Version]
  18. Campbell, J.P. Personnel Training and Development. Annu. Rev. Psychol. 1971, 22, 565–602. [Google Scholar] [CrossRef]
  19. Wexley, K.N. Personnel Training. Annu. Rev. Psychol. 1984, 35, 519–551. [Google Scholar] [CrossRef] [PubMed]
  20. Latham, G.P. Human Resource Training and Development. Annu. Rev. Psychol. 1988, 39, 545–582. [Google Scholar] [CrossRef]
  21. Salas, E.; Cannon-Bowers, J.A. The Science of Training: A Decade of Progress. Annu. Rev. Psychol. 2001, 52, 471–499. [Google Scholar] [CrossRef] [Green Version]
  22. Alliger, G.M.; Tannenbaum, S.I.; Bennett, W.; Traver, H.; Shotland, A. A Meta-Analysis of the Relations among Training Criteria. Pers. Psychol. 1997, 50, 341–358. [Google Scholar] [CrossRef]
  23. Arthur, W., Jr.; Bennett, W., Jr.; Edens, P.S.; Bell, S.T. Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features. J. Appl. Psychol. 2003, 88, 234–245. [Google Scholar] [CrossRef] [Green Version]
  24. Keith, N.; Frese, M. Effectiveness of error management training: A meta-analysis. J. Appl. Psychol. 2008, 93, 59–69. [Google Scholar] [CrossRef] [Green Version]
  25. Blume, B.D.; Ford, J.K.; Baldwin, T.T.; Huang, J.L. Transfer of Training: A Meta-Analytic Review. J. Manag. 2010, 36, 1065–1105. [Google Scholar] [CrossRef] [Green Version]
  26. Colquitt, J.A.; LePine, J.A.; Noe, R.A. Toward an Integrative Theory of Training Motivation: A Meta-Analytic Path Analysis of 20 Years of Research. J. Appl. Psychol. 2000, 85, 678–707. [Google Scholar] [CrossRef] [Green Version]
  27. Guzzo, R.A.; Jette, R.D.; Katzell, R.A. The Effects of Psychologically Based Intervention Programs on Worker Productivity: A Meta-Analysis. Pers. Psychol. 1985, 38, 275–291. [Google Scholar] [CrossRef]
  28. Tracey, J.B.; Tannenbaum, S.I.; Kavanagh, M.J. Applying Trained Skills on the Job: The Importance of the Work Environment. J. Appl. Psychol. 1995, 80, 239–252. [Google Scholar] [CrossRef]
  29. Huselid, M. The science and practice of workforce analytics: Introduction to the HRM special issue. Hum. Resour. Manag. 2018, 57, 679–684. [Google Scholar] [CrossRef] [Green Version]
  30. Laursen, G.H.N.; Thorlund, J. Business Analytics for Managers: Taking Business Intelligence beyond Reporting; John Wiley & Sons: Hoboken, NJ, USA, 2016. [Google Scholar]
  31. Kim, J.; Choi, S.O. The Intensity of Organizational Change and the Perception of Organizational Innovativeness; with Discussion on Open Innovation. J. Open Innov. Technol. Mark. Complex. 2020, 6, 66. [Google Scholar] [CrossRef]
  32. Locke, E.A.; Latham, G.P. A Theory of Goal Setting & Task Performance; Prentice-Hall, Inc.: Upper Saddle River, NJ, USA, 1990. [Google Scholar]
  33. Locke, E.A.; Latham, G.P. Building a Practically Useful Theory of Goal Setting and Task Motivation. A 35-Year Odyssey. Am. Psychol. 2002, 57, 705–717. [Google Scholar] [CrossRef] [Green Version]
  34. Lepine, J.A. Adaptation of Teams in Response to Unforeseen Change: Effects of Goal Difficulty and Team Composition in Terms of Cognitive Ability and Goal Orientation. J. Appl. Psychol. 2005, 90, 1153–1167. [Google Scholar] [CrossRef] [Green Version]
  35. LaValle, S.; Lesser, E.; Shockley, R.; Hopkins, M.S.; Kruschwitz, N. Big Data, Analytics and the Path from Insights to Value. MIT Sloan Manag. Rev. 2011, 52, 21–32. [Google Scholar]
  36. Armenakis, A.A.; Harris, S.G. Crafting a change message to create transformational readiness. J. Organ. Chang. Manag. 2002, 15, 169–183. [Google Scholar] [CrossRef]
  37. Colbert, A.E.; Kristof-Brown, A.L.; Bradley, B.H.; Barrick, M.R. CEO Transformational Leadership: The Role of Goal Importance Congruence in Top Management Teams. Acad. Manag. J. 2008, 51, 81–96. [Google Scholar] [CrossRef] [Green Version]
  38. Conger, J.A.; Kanungo, R.N. Toward a Behavioral Theory of Charismatic Leadership in Organizational Settings. Acad. Manag. Rev. 1987, 12, 637–647. [Google Scholar] [CrossRef]
  39. Huynh, T.; Duong, M.H.; Phan, T.T.; Van Do, T.; Do, T.T.T.; Nguyen, K.T. Team Dynamics, Leadership, and Employee Proactivity of Vietnamese Firms. J. Open Innov. Technol. Mark. Complex. 2019, 5, 16. [Google Scholar] [CrossRef] [Green Version]
  40. Griffin, M.a.; Parker, S.K.; Mason, C.M. Leader vision and the development of adaptive and proactive performance: A longitudinal study. J. Appl. Psychol. 2010, 95, 174–182. [Google Scholar] [CrossRef] [Green Version]
  41. Kirkpatrick, S.A.; Locke, E.A. Direct and Indirect Effects of Three Core Charismatic Leadership Components on Performance and Attitudes. J. Appl. Psychol. 1996, 81, 36. [Google Scholar] [CrossRef]
  42. Hackman, J.R.; Wageman, R. A Theory of Team Coaching. Acad. Manag. Rev. 2005, 30, 269–287. [Google Scholar] [CrossRef]
  43. Morrison, E.W.; Phelps, C.C. Taking Charge at Work: Extrarole Efforts to Initiate Workplace Change. Acad Manag. Rev. 1999, 42, 403–419. [Google Scholar]
  44. Grant, A.M.; Ashford, S.J. The dynamics of proactivity at work. Res. Organ. Behav. 2008, 28, 3–34. [Google Scholar] [CrossRef]
  45. Rousseau, D.M. Designing a Better Business School: Channelling Herbert Simon, Addressing the Critics, and Developing Actionable Knowledge for Professionalizing Managers. J. Manag. Stud. 2012, 49, 600–618. [Google Scholar] [CrossRef] [Green Version]
  46. Beer, M.; Nohria, N. Breaking the Code of Chang; Harvard Business School Press: Boston, MA, USA, 2000. [Google Scholar]
  47. Kanfer, R.; Ackerman, P.L. Motivation and Cognitive Abilities: An Integrative/Aptitude-Treatment Interaction Approach to Skill Acquisition. J. Appl. Psychol. 1989, 74, 657–690. [Google Scholar] [CrossRef]
  48. Amabile, T.M. A model of creativity and innovation in organizations. Res. Organ. Behav. 1988, 10, 123–167. [Google Scholar]
  49. Burkhardt, M.E.; Brass, D.J. Changing Patterns or Patterns of Change: The Effects of a Change in Technology on Social Network Structure and Power. Adm. Sci. Q. 1990, 35, 104. [Google Scholar] [CrossRef]
  50. Tenkasi, R.V.; Chesmore, M.C. Social Networks and Planned Organizational Change: The Impact of Strong Network Ties on Effective Change Implementation and Use. J. Appl. Behav. Sci. 2003, 39, 281–300. [Google Scholar] [CrossRef]
  51. Kim, B.J.; Rousseau, D.M.; Tomprou, M. Training and Diffusion of Change: The Impact of Training and Proactivity on Change-Related Advice Giving. J. Appl. Behav. Sci. 2019, 55, 453–476S. [Google Scholar] [CrossRef]
  52. Jermier, J.M.; Kerr, S. “Substitutes for Leadership: Their Meaning and Measurement”-Contextual Recollections and Current Observations. Leadersh. Q. 1997, 8, 95–101. [Google Scholar] [CrossRef]
Table
Table 1. Means, standard deviations, and correlations of variables at pre-test and post-test.
Table 1. Means, standard deviations, and correlations of variables at pre-test and post-test.
TraineesPeers
MSDMSDp12345678910111213
10.520.510.560.510.83
23.290.962.781.160.11−0.12
32.671.202.071.170.09−0.020.53 **
42.550.892.221.220.31−0.21−0.02−0.10
Pre-test
53.270.923.050.840.40−0.13−0.22−0.220.17
63.790.493.041.140.01−0.100.210.020.49 **0.39 *
72.551.071.920.980.04−0.050.08−0.060.44 **0.45 **0.52 **
83.381.412.651.330.07−0.32 *−0.06−0.240.68 **0.41 **0.51 **0.74 **
91.480.931.741.320.44−0.08−0.20−0.29 *0.34 *0.190.220.36 *0.44 **
Post-test
103.000.893.280.710.23−0.030.140.33 *−0.090.31 *0.020.07−0.100.04
113.830.663.590.750.25−0.100.030.030.33 *0.33 *0.57 **0.44 **0.34 *0.200.21
123.190.912.940.820.330.030.170.080.44 **0.110.37 *0.53 **0.31 *0.130.140.61 **
134.300.773.890.950.12−0.170.020.040.33 *0.130.290.38 **0.37 *−0.050.030.39 **0.58 **
141.471.260.700.870.020.020.06−0.020.140.110.36 *0.230.31 *0.110.070.29 *0.140.25
* p < 0.05, ** p < 0.01. N = 48. TG, training group (N = 21); CG, comparison group (N = 27); 1. Female; 2. Age; 3. Education; 4. Expertise in healthcare data analytics at pre-test; 5. Leader vision at pre-test; 6. Learning goals at pre-test; 7. Performance goals at pre-test; 8. Healthcare data analytics skills at pre-test; 9. Use of healthcare data analytics tools at pre-test; 10. Leader vision at post-test; 11. Learning goals at post-test; 12. Performance goals at post-test; 13. Healthcare data analytics skills at post-test; 14. Changes in the use of healthcare data analytics tools.
Table
Table 2. Results of multivariate and univariate analysis of covariance.
Table 2. Results of multivariate and univariate analysis of covariance.
Model (Dependent Variable)Wilk’s λdf 1df 2Fpη2
Multivariate0.912402.090.140.09
Improved data analytics skills0.971411.240.270.03
Use of data analytics tools0.921413.510.070.08
Univatiate
Improved data analytics skills0.951422.330.130.05
Use of data analytics tools0.881425.790.020.12
Covariates for multivariate ANCOVA models include pre-test scores of use of healthcare data analytics tools and healthcare data analytics skills as well as expertise in healthcare data analytics measured at the pre-test.
Table
Table 3. Results of multiple regression analysis.
Table 3. Results of multiple regression analysis.
Dependent VariableData Analytics SkillsUse of Data Analytics Tools
M1M2M3M4M5M6M7M8
Learning Goal (pre)−0.09−0.04 0.170.24
Learning Goal (post)0.33 *0.11 0.15−0.11
Performance Goal (pre) −0.16−0.18 0.04−0.03
Performance Goal (post) 0.56 **0.34 0.05−0.77
Leader Vision (pre) −0.41 −0.18 −0.42 −0.72
Leader Vision (pre) ×
Learning Goal (post)		 0.46 0.49
Leader Vision (pre) ×
Performance Goal (post)		 0.32 1.2
R-square0.230.240.390.400.200.210.140.18
F3.11 **2.10 *6.64 **4.27 **2.43 *1.661.571.37
* p < 0.10, ** p < 0.05. N = 48. Standardized coefficients are reported. Estimated models include control variables such as participation in the training program and the use of healthcare data analytics tools at the pre-test.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kim, B.J.; Tomprou, M. The Effect of Healthcare Data Analytics Training on Knowledge Management: A Quasi-Experimental Field Study. J. Open Innov. Technol. Mark. Complex. 2021, 7, 60. https://doi.org/10.3390/joitmc7010060

AMA Style

Kim BJ, Tomprou M. The Effect of Healthcare Data Analytics Training on Knowledge Management: A Quasi-Experimental Field Study. Journal of Open Innovation: Technology, Market, and Complexity. 2021; 7(1):60. https://doi.org/10.3390/joitmc7010060

Chicago/Turabian Style

Kim, Byeong Jo, and Maria Tomprou. 2021. "The Effect of Healthcare Data Analytics Training on Knowledge Management: A Quasi-Experimental Field Study" Journal of Open Innovation: Technology, Market, and Complexity 7, no. 1: 60. https://doi.org/10.3390/joitmc7010060

Article Metrics

Back to TopTop