Next Article in Journal
Ecological Risk Assessment of Heavy Metals in Adjoining Sediment of River Ecosystem
Next Article in Special Issue
The Role of Policy Perceptions and Entrepreneurs’ Preferences in Firms’ Response to Industry 4.0: The Case of Chinese Firms
Previous Article in Journal
The Determinants of Global Expansion: A Study on Food and Beverage Franchisors in Malaysia
Previous Article in Special Issue
Intrinsic Motivation Sources as Pillars of Sustainable Internal Marketing Communication in Turbulent Post-Pandemic Times
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 13
Issue 18
10.3390/su131810329
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Machine-Learning Classification Tree Model of Perceived Organizational Performance in U.S. Federal Government Health Agencies

by
In-Gu Kang
1,*,
Nayoung Kim
2,
Wei-Yin Loh
3 and
Barbara A. Bichelmeyer
4
1
Department of Organizational Performance and Workplace Learning, College of Engineering, The Boise State University, Boise, ID 83706, USA
2
Center for Tobacco Research and Intervention, School of Medicine and Population Health, The University of Wisconsin-Madison, Madison, WI 53711, USA
3
Department of Statistics, The University of Wisconsin-Madison, Madison, WI 53706, USA
4
Office of the Provost, The University of Kansas, Lawrence, KS 66045, USA
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(18), 10329; https://doi.org/10.3390/su131810329
Submission received: 21 August 2021 / Revised: 3 September 2021 / Accepted: 12 September 2021 / Published: 16 September 2021
(This article belongs to the Special Issue Sustainable Human Resource Management in Industry 4.0)
Browse Figures
Versions Notes

Abstract

:
Perceived organizational performance (POP) is an important factor that influences employees’ attitudes and behaviors such as retention and turnover, which in turn improve or impede organizational sustainability. The current study aims to identify interaction patterns of risk factors that differentiate public health and human services employees who perceived their agency performance as low. The 2018 Federal Employee Viewpoint Survey (FEVS), a nationally representative sample of U.S. federal government employees, was used for this study. The study included 43,029 federal employees (weighted n = 75,706) among 10 sub-agencies in the public health and human services sector. The machine-learning classification decision-tree modeling identified several tree-splitting variables and classified 33 subgroups of employees with 2 high-risk, 6 moderate-risk and 25 low-risk subgroups of POP. The important variables predicting POP included performance-oriented culture, organizational satisfaction, organizational procedural justice, task-oriented leadership, work security and safety, and employees’ commitment to their agency, and important variables interacted with one another in predicting risks of POP. Complex interaction patterns in high- and moderate-risk subgroups, the importance of a machine-learning approach to sustainable human resource management in industry 4.0, and the limitations and future research are discussed.

1. Introduction

An organization’s member shapes her or his self-concept and self-esteem by how they perceive the image and prestige of the organization that s/he belongs to [1]. Being a member of a high performing and highly regarded organization helps organizational members develop self-continuity, self-distinctiveness, and self-esteem [2] since people tend to “bask in reflected glory”, even though successes or achievements are not a direct consequence of their efforts or activities [3]. Thus, employees who perceive that their organization performs better than its competitors have a more favorable view of the prestige and status of their organization and strong organizational identification with the high level of perceived organizational performance (POP) [1].
POP is referred to as “the subjective measurement of employee perceptions regarding an organization’s overall performance when compared to the rivals in the same sector” [4]. POP influences employees’ attitudes and behaviors, which in turn improve or impede organizational sustainability. Sustainable human resource management (HRM) practices can engineer those attitudes and behaviors through targeted activities and practices. For instance, POP due to perceived relationships within an organization strongly influences employee attitudes or behaviors such as retention and turnover, and a favorable POP is closely and positively correlated with organizational ability to retain talent [5]. The effect of POP on employees’ retention or turnover ultimately helps improve an organization’s sustainable competitive advantage [6]. Contrarily, an unfavorable POP has the opposite effect. Therefore, organizations must improve POP to maintain a sustainable business.
Improving organizational performance has also been a critical concern of public administration and management in the public sector [7,8,9]. Particularly with New Public Management, a result- and business-oriented management approach has been emphasized in public organizations such as governments to enhance organizational performance and effectiveness for sustainability during several decades [9]. In an effort to improve organizational performance in the public sector, many scholars have investigated factors that are associated with POP to better understand how organizational environments and climates, and employees’ attitudes and behaviors affect POP [7,10].
Prior studies identified job satisfaction and employees’ perceptions of workplace environmental factors that influence POP. In a study examining multiple job-related and workplace environmental predictors of POP in a government setting, job satisfaction, defined as “an affective or emotional response toward various facets of one’s job”, was the strongest predictor of POP [11]. Among workplace environmental factors, culture can affect POP. Specifically, cultivating a performance- or results-oriented culture by implementing incentives, recognition systems, and performance management systems [12], is positively associated with POP [13]. In one study that integrated leadership and federal program performance, leadership had a significant effect on organizational performance [14]. Along with a performance-oriented culture and leadership in the public sector, physical work environments such as security and safety on the job are important to enhancing organizational performance. One prominent example is the one that investigates the implications of the Hawthorne studies in the public sector, in which physical work environments, security, and safety were significant factors in improving POP [15]. Organizational procedural justice, referred to as “the extent to which the dynamics of the decision process are judged to be fair [16]”, is also considered an antecedent of POP. In a recent study of examining the mediating effect of organizational procedural justice in the relationship between diversity and POP, organizational procedural justice had a significant impact on POP directly and indirectly [17]. Another workplace environmental factor that influences POP is organizational commitment. Organizational commitment that promotes positive employee identification and active engagement in an organization was found to have a significant association with POP because highly committed employees are willing to put in more effort to achieve organizational goals and to remain in the current organization [18]. According to a goal-setting theory, employees strive to perform better when the goals are set up clearly and specifically [19]. Teams with a higher level of goal clarity showed better team performance [20]. In addition to these variables that affect POP, employees’ perceptions of many other workplace environmental factors such as diversity management [21], trust in leadership [22], and employee empowerment [23] have been examined in the public sector.
Although a wealth of predictors has been identified in prior studies, it is not easy to integrate findings due to inconsistences and limitations in prior studies regarding study design, different measurement selection, and analytic technique. Recently, Kang and Bichelmeyer [24,25] attempted to systematically integrate findings and developed a comprehensive human and organizational performance model. Results were gathered from a representative sample of U.S. federal government employees, in which the model addressed various organizational performance factors that affect employee behavior and POP, including workplace and environmental factors. Previous performance models took a piecemeal approach that included a single or small set of environmental/organizational and human factors associated with POP [26]. This piecemeal approach is problematic when developing planned interventions on performance improvement because the mechanism through which multiple environmental/organizational and human factors influence organizational performance remains unknown. Kang and Bichelmeyer [24,25] advanced the knowledge base regarding a performance model by considering an integrated and comprehensive approach, including a variety of individual and workplace environmental factors that affect POP, and testing a mechanism through which those factors impact POP directly and indirectly. Despite this integrated effort to better understand multiple predictors of POP and the mechanism, there are still limitations on capturing the complexity of interactions between multiple predictors of POP and developing targeted interventions for at-risk subgroups who may develop low levels of POP. Although previous research has contributed to an improved understanding of general indicators or processes that contribute to high levels of POP among employees, some important questions remain.
Most studies have focused on independent rather than interactive effects [11,22] and relied on data from the general workforce. Standard statistical models (e.g., logistic regression and structural equation modeling) are often used to identify predictors of POP, but they have some limitations such as lack of ability to identify complex, unspecified interaction effects. In contrast, machine-learning classification decision-tree modeling is a novel approach that efficiently identifies multiple, complex interactive effects of predictors and classifies subgroups of populations (e.g., U.S. federal public health and human services employees) at high or low risk based on a defined outcome (e.g., POP). A non-parametric, machine-learning classification modeling [27,28] gained its popularity in the human learning/development and management fields because it provides straightforward interpretation based on decision-tree models and allows the researcher to analyze a high number of variables and their interactions simultaneously with a large volume of data [28].
The literature comparing employees with high and low POP is, with some exceptions, somewhat mixed. The inconsistency may be because some prior studies, particularly in the public health and human services sector setting, have relied on small samples. Most previous studies have also been limited by solely considering univariate relationships that may have obscured important group differences between high and low POP. Some studies have used traditional multivariable statistical models to examine which risk predictors had independent relationships with the probability of making an attempt when adjusting for others. However, none have used statistical approaches that allowed for the examination of different pathways to low performance risk, possibly because of the small sample sizes. Indeed, relationships between specific risk factors and POP are not apparent for all public health and human services employees, as revealed by examinations of specific moderators. For example, in previous studies, the associations between risk factors and POP varied as a function of sex, individual characteristics, and others [11,15,18,20]. Thus, understanding differences in POP between public health and human services employees not only calls for replication in larger samples, but also requires that studies consider different ways of combining risk factors. Approaches that consider many risk variables and their interactions at one time may reveal novel information about multiple pathways that result in low POP among public health and human services employees and clues as to the processes that account for that transition. To our knowledge, this work has not yet been done among public health and human services employees.
The objective of this study is to identify patterns of risk factors that differentiate public health and human services employees who perceived their performance as low. We advance previous work by using a large sample of workforce drawn from the Federal Employee Viewpoint Survey (FEVS) database, including a broad array of individual and workplace environmental factors in a single model, and employing machine-learning classification decision-tree modeling. The strength of this approach lies in its ability to model more complex and multidimensional risk processes that move beyond univariate relationships or regression models to identify and differentiate patterns of risk factors that are associated with a higher risk of low POP. This approach allows for the observation of moderating effects that might otherwise be missed when using traditional model-based approaches. Using an exploratory approach, this study seeks to identify constellations of individual and environmental risk factors that could become the targets of intervention efforts in public health and human services employees who are at heightened risk for low POP.
We have reviewed the previous studies that investigated a variety of predictors of POP and addressed how the machine-learning classification decision-tree model overcomes the shortcomings of traditional univariate or regression models in identifying the complex interaction patterns of a large number of predictors of POP. In the following sections, the materials and methods including data, measures, and data analysis are described, the results of the machine-learning classification decision-tree model are reported, the major findings and implications for the use of a machine-learning approach to sustainable HRM in industry 4.0 are discussed, and limitations and conclusions of this study are given.

2. Materials and Methods

2.1. Data

We used data from the 2018 FEVS, a nationally representative survey of U.S. federal government employees. The U.S. Office of Personnel Management annually collects the FEVS data to assess federal employees’ satisfaction with their job and perceptions of working environments, rewards, opportunities and their agency performance. The FEVS utilized a stratified sampling technique and sampled federal government employees (e.g., full-time, part-time, permanent and non-seasonal employees) who were invited to anonymously and voluntarily participate in a secure online survey throughout a 6-week data collection period in spring 2018 that represented 83 federal agencies in the federal government, ranging from department level to independent agency. The 2018 FEVS had a government-wide 40.6% response rate. A more detailed description of the survey methodology can be found elsewhere [29]. The 2018 study included 43,029 federal government employees (weighted n = 75,706) among 10 sub-agencies in the public health and human services sector: Administration for Children and Families, Centers for Disease Control and Prevention, Centers for Medicare and Medicaid Services, Food and Drug Administration, Health Resources and Services Administration, Indian Health Service, National Institutes of Health, Office of the Secretary, Office of Inspector General, and all other sub-agencies in the Department of Health and Human Services. FEVS data are de-identified and publicly available; thus, institutional review board approval is not required in conformance with 45 CFR 46.

2.2. Measures

2.2.1. Performance Outcome

Self-reported POP was measured by a single ordinal survey question: “My agency is successful at accomplishing its mission” rated on a 5-point Likert scale that ranged from 1 = “strongly disagree” to 5 = “strongly agree”. About 81.8% (weighted n = 60,025) of the FEVS participants strongly agreed or agreed with the high level of agency performance, 13.5% (weighted n = 9922) reported a neutral response, and 4.7% (weighted n = 3432) strongly disagreed or disagreed with the high level of agency performance (See Table 1).

2.2.2. Predictor Variable

The study contained a set of 75 predictor variables that were comprehensively selected from the FEVS variables to measure federal employees’ satisfaction with their job-related factors and perceptions of working environments, and their agency performance. The predictors included demographics (self-reported gender, racial minority status, educational attainment, supervisory status, and organizational tenure) and employees’ attitude and perception related to organizational environments such as performance-oriented culture, organizational procedural justice, organizational satisfaction, task-oriented leadership, work security and safety on the job, commitment to the agency, goal clarity, intrinsic motivation, employee development, employee empowerment, management communication, trust in supervisor, rewards for innovation, senior leadership and satisfaction with job or senior leaders, diversity management. All study measures (survey items with responses used in the current study) can be found in the Supplementary Online Material (see Supplementary Table S1).

2.3. Data Analysis

The Generalized, Unbiased, Interaction Detection and Estimation (GUIDE; http://pages.stat.wisc.edu/~loh/guide.html (accessed on 3 September 2021)) [30] classification and regression-tree modeling program, GUIDE version 38.0, was used in Windows 10 to identify predictive variables and the best combinations of predictive variables associated with POP, and distinguish subgroups of federal employees in federal public health and human services agencies based on POP. Because GUIDE has no underlying distributional assumptions about predictors or outcome variables, we modeled POP as an ordered categorical (i.e., strongly agree, agree, neither agree nor disagree, disagree, strongly disagree). GUIDE is a nonparametric, machine-learning method for constructing a prediction model via binary recursive partitioning of the data, which can be graphically represented as a hierarchical decision tree based on a defined outcome. In each partitioning iteration, GUIDE conducts chi-square tests to evaluate the associations between each predictor variable and the outcome, and selects the most important predictor variable (node splitter) and optimal cutoff of the predictor variable to identify mutually exclusive and exhaustive subgroups of a population of interest (in this case the population of the U.S. federal government employees in public health and human services agencies) based on the outcome (i.e., POP). Participants within each subgroup share similar characteristics that influence the probability of belonging to the interested response group. In addition, GUIDE computes an importance score for each predictor variable and provides a threshold score (1.0) for distinguishing important predictor variables from unimportant ones. Importance scores greater than 1.0 are typically interpreted as important to outcome prediction [31]. GUIDE uses user-specified V-fold cross-validation for pruning to develop optimal tree structures. In the current study, 10-fold cross-validation was employed to estimate the misclassification cost associated with competing trees and evaluate predictive performance of the final tree model. GUIDE was selected in the current study based on some advantages over other decision-tree models (e.g., RPART) [32] in that it is unbiased in variable selection and treats missing values in the predictor variables as informative without prior imputation [33].

3. Results

Table 1 presents sample characteristics of federal government employees in public health and human services agencies in the current study. About 62.9% of federal employees in health agencies were female, 80.7% had a bachelor’s degree or higher, and 53.0% have worked within the federal government for more than 10 years. Most employees endorsed high levels of agency performance (81.8%) and satisfaction with their jobs (74.1%), while 27.9% intended to leave their agency within the next year.
The final classification tree of POP is shown in Figure 1. The GUIDE analysis identified several tree-splitting variables and identified 33 subgroups of employees with POP levels that were high risk (i.e., those who strongly disagreed or disagreed with their agency’s performance as high, 2 subgroups), moderate risk (i.e., those who neither disagreed nor agreed with their agency’s performance as high, 6 subgroups) and low risk (i.e., those who strongly agreed or agreed with their agency’s performance as high, 25 subgroups).
Performance-oriented culture was the first splitting variable in the classification tree of POP, showing higher rates of belonging to high- or moderate-risk groups (34~71%, the left branches of the tree) among those who did not endorse that their agency has high level of performance-oriented culture compared with those who endorsed. Among employees who did not endorse high levels of performance-oriented culture at their agency, employees’ organizational satisfaction was the second splitting variable predicting high or moderate risks of POP. Particularly, high risks of POP (46~71%) were observed among those who perceived that their manager poorly demonstrates task-oriented leadership in accomplishing the organizational goals and perceived that employees’ security and safety on the job are not protected properly (71%), and those who did not endorse their agency as a good place to work even though they perceived that their security and safety are properly protected at their agency (46%).
Among employees who did not endorse high levels of performance-oriented culture and were dissatisfied with their agency, those who perceived that their agency does not secure organizational procedural justice or fairness in personnel practices (e.g., illegal discrimination or violation against any employee promotion/applicant) and that employee development is not supported by their agency to acquire the job-relevant knowledge and skills necessary to accomplish the organizational goals are at a particular high risk of belonging to a moderate-risk subgroup (60%). Employees’ low levels of perceptions on task-oriented leadership, commitment to their agency, lack of goal clarity, and work security and safety and job dissatisfaction increased risks of belonging to moderate-risk subgroups (34–56%). In contrast (see the right branches of the tree), employees’ high levels of perceptions on organizational procedure justice, commitment to their agency, work-unit performance, employee development, intrinsic motivation, goal clarity for the job, and safety and satisfaction with senior leaders at agency are associated with low risks of POP.
As shown in Figure 2, performance-oriented culture had the highest importance score (an indicator of the relation of the predictor variable with POP across all possible classification trees). Organizational procedure justice, commitment to agency, and satisfaction with senior leaders also had very high importance scores. Trust in leadership, employee empowerment, management communication, rewards for innovation, and satisfaction with senior leadership had high importance scores but were not included in the final classification tree of POP.

4. Discussion

This study aimed to identify interaction patterns of risk factors that differentiate public health and human services employees who perceived their performance as low and to develop targeted interventions for at-risk subgroups to enhance their perceptions of POP by employing the GUIDE machine-learning classification decision-tree model. In this section, (1) the first and second important split variables of POP, (2) risk factors and interaction patterns in high- and moderate-risk subgroups, (3) implications for the use of a machine-learning approach to sustainable HRM in industry 4.0, and (4) the limitations and future research are discussed.

4.1. The First and Second Split Variables from the Classification Tree Model

The first split variable of the classification tree model was performance-oriented culture (particularly employee recognition), and it was strongly predictive of POP. This is consistent with the findings of prior studies showing strong associations between performance-oriented culture and POP [13,34,35]. Organizational culture defines the boundaries of acceptable action for employees and thus leads to the desirable behaviors and expected outcomes [12,36]. The Organisation for Economic Co-operation and Development highlighted that cultivating performance-oriented culture in the public sector is an essential step towards performance improvement [13,37]. Particularly in this study, employee performance recognition was the strongest predictor of POP compared to other aspects of performance-oriented culture (e.g., monetary incentives) such as merit-based promotion, pay raises based on performance criteria set by the organization, and management on poor performance. According to Nelson’s book entitled 1001 ways to Reward Employees, employee recognition based on job performance is the top driving force of employee performance [38]. In a controlled field experiment study that examined the causal effect of employee recognition on performance, the major findings of the study demonstrated that unannounced recognition to exclusive employees was the most cost-effective tool compared to monetary or material rewards for performance, especially for non-recipients since they are aware that their performance needs to be improved [39]. Thus, management in the public health and human services agencies should cultivate performance-oriented culture where a large group of best-performing workers are recognized for their performance so that not only recipients but also non-recipients of recognition hold themselves responsible for performance improvement, which in turn improves POP.
Organizational satisfaction was the second split variable of POP and was the strong moderating variable that worsens an association between low performance-oriented culture and low POP. Organizational satisfaction is defined as “perceptions of employees on how satisfied they are with the organization considering everything [40]” and is one of the three aspects of job satisfaction (e.g., pay, job, and organizational satisfaction) [41,42,43]. Prior studies confirmed that high levels of job satisfaction, including organizational satisfaction, lead to high organizational performance because employees with high job satisfaction work harder and devote themselves to the organization [44]. One study that examines associations between merit-based pay systems and three types of job satisfaction found that merit-based pay systems had a greater negative association with organizational satisfaction than other aspects of job satisfaction (e.g., pay and job satisfaction), indicating that employees working in federal agencies with merit-based pay had significantly lower organizational satisfaction than those working in other agencies [45]. This research finding is consistent with the one in the current study regarding employee recognition: providing non-monetary recognition for a group of highest-performing employees was a more cost-effective strategy for improving POP than merit-based promotion and pay raises based on performance criteria set by the organization. This frustration and dissatisfaction toward the organization stem from a failure in the financial incentive systems that may create highly competitive and stressful workplace environments. In the HRM and organizational behavior fields, it is recommended that non-monetary incentives or rewards such as non-monetary recognition enhance favorable perceptions of employees on the organization and thus increase satisfaction with the organization [46]. Although types of non-monetary recognition vary and there is no standard approach to it, the most crucial factor that determines the scheme of non-monetary recognition is organizational culture [47]. Therefore, given performance-oriented culture that promotes employee recognition, management in the public health and human services agencies strongly encourages leaders and managers to establish the scheme of non-monetary recognition and thereby increase employee satisfaction. There are three common non-financial recognition schemes: praise, vouchers, and nomination-based schemes [47]. Praise is the most obvious and simple method for personal acknowledgement. Leaders and managers in the public health and human services agencies acknowledge high performing workers by simply telling them “thank you”. Another popular scheme for non-monetary recognition is vouchers. When designing vouchers, leaders and managers in the public health organizations may consider individual preferences to vouchers so that vouchers fit the needs of recipients. Thus, agencies get employees involved in the design of vouchers through surveys or focus groups. Finally, nomination-based schemes are also an effective way of non-monetary recognition. Leaders and managers in the public health organizations may offer a trophy or certificate for an employee’s contribution and nominees may be nominated by colleagues. This non-financial employee recognition scheme is effectively implemented in the performance-oriented culture.

4.2. Risk Factors and Interaction Patterns in High- and Moderate-Risk Subgroups

As mentioned above, both performance-oriented culture, employee recognition, (the first split variable) and organizational satisfaction (the second split variable) contribute to increasing risks from public health and human services employees who perceive the agency performance as very low or neither good nor poor by interacting with other workplace environmental factors. The machine-learning classification decision-tree model identified two high-risk subgroups who perceived organizational performance as very low and six moderate-risk subgroups who perceived organizational performance as neither good nor poor. For high-risk subgroups, along with low levels of performance-oriented culture and organizational satisfaction, public health and human services employees are likely to perceive the agency performance as very low unless task-oriented leadership role is well-implemented, employees’ commitment to the agency is increased, and work safety on the job is secured. More specifically, task-oriented leadership in this study involves leadership behaviors that seek to accomplish the organizational goals and missions [14], which in turn enhances POP. In addition, this task-oriented leadership behavior of communicating the goals and priorities of the organization with employees leads to higher organizational commitment. According to Ko, Hur, and Smith-Walter [35], communicating with managers regarding the organizational goals and priorities was the most powerful predictor of organizational commitment. To promote this task-oriented leadership behavior and organizational commitment of employees, management in the public health and human services agencies need to encourage leaders and managers to evaluate and communicate the organizational progress toward achieving the agency’s goals and missions. Physical work environments such as work security and safety also played a moderating role with other risk factors in high-risk subgroups. The research finding suggested that healthier and safer work environments enable employees to perform jobs more efficiently and effectively, and increase job satisfaction [15]. Low satisfaction with work security and safety leads to very low POP of high-risk subgroups interacting with other risk factors. To improve work security and safety, training and education for health and safety at work were found to have the only significant impact on job satisfaction among other strategies such as management practices for health and safety [48]. Thus, management in the public health and human services agencies should pay attention to implementing training and education for health and safety at work.
For moderate-risk subgroups, additional factors such as goal clarity, organizational procedural justice, and employee development increased risks that public health and human services employees perceive the agency performance as neither good nor poor, interacting with the risk factors found in high-risk subgroups. While task-oriented leadership behaviors of communicating the goals and priorities with employees in high-risk subgroups were important for POP, whether or not employees themselves clearly know how their work relates to the agency’s goals had an influence on POP. The prior study suggested that employees’ goal clarity and good common understanding of it positively influence organizational performance [49]. Although self-management or high levels of decision-making autonomy could be a good venue to strengthen the relationship between the goal clarity and agency’s performance, it is not sufficient to moderate the relationship, because employees or teams may be in chaos with high levels of autonomy [20,50]. It is recommended that high autonomy with sufficient performance feedback should be given to employees or teams since performance feedback increases the effect of autonomy by clarifying the organizational goals [50]. Thus, leaders and managers in the public health and human services agencies should provide both high-level autonomy and a wealth of performance feedback for employees or teams. Organizational procedural justice was also one of the moderating factors that worsen employees’ POP in the current study. Promoting organizational procedural justice practices in strategic decision making not only produces high-quality strategic decisions but also increases the capacity to implement those decisions in an effective way [16]. Thus, management in the public health and human services agencies should organize organizational procedural justice practices into strategic decision-making processes. Employee development also had an interaction effect in the relationship between other risk factors and POP. Employees are not likely to perform better unless work units or agencies provide the job-relevant knowledge and skills necessary to achieving the agency’s goals. Therefore, leaders and managers in the public health and human services agencies should assess training needs of employees and provide employee development opportunities such as training and education for employees to obtain job-relevant knowledge and skills.

4.3. Implications for the Use of Machine-Learning Approach to Sustainable HRM in Industry 4.0

This study began with the idea that POP has an influence on employees’ attitudes or behaviors such as retention and turnover that improve or impede organizational sustainability and competitive advantage. To better understand a mechanism on how individual and workplace environmental factors interact with each other and POP, a machine-learning classification decision-tree model was employed to identify the complex interaction patterns of risk factors that worsen the relationship between predictors and POP. As human resource practitioners are faced with improving efficiency and developing and retaining talented workers for sustainability in the industry 4.0 revolution, we can take advantages of using machine-learning techniques in improving sustainable HRM practices. First, as sustainable HRM takes a holistic approach to people management, HRM departments should consider the complexity of interactions among various individual, internal workplace environmental factors, and societal and global factors, and the dynamics of them when designing and developing HRM practices. While traditional statistical models (e.g., logistic regression) only include a limited number of theory-based factors verified in the prior studies, the machine-learning classification decision-tree model can overcome this weakness of traditional statistical models by freely exploring unspecified interaction patterns by allowing a large number of predictors and their interactions that may influence an outcome of interest simultaneously with a large volume of data and find the best prediction model. It enables HRM practitioners to make a more accurate prediction on challenging organizational issues by considering the complexity and dynamics of the realities in the workplace and to help make data-driven and evidence-driven decisions. Second, while traditional statistical models target the entire population and develop interventions for the population, the machine-learning classification decision-tree model classifies subgroups of populations, especially for at-risk subgroups, and develops targeted interventions for those subgroups in a more efficient and effective way. This helps management make an investment in the limited resources to develop the cost-effective strategies for sustainable HRM practices. Coping with new challenges in industry 4.0, the machine-learning classification decision-tree model can offer a more accurate prediction model and help develop targeted interventions for subgroups.

4.4. Limitations and Future Research

Although we used machine learning classification decision-tree modeling as a novel approach that efficiently identifies multiple, complex interactive effects of predictors and classifies subgroups of the population at high or low risks based on POP with a nationally representative sample of the U.S. federal public health and human services workforce, this study has some limitations. First, as with all observational studies, causal relationships between predictors and POP cannot be established without making more assumptions about the data. Second, although POP is a strong predictor of agency performance, the latter cannot be captured by employees’ perceptions accurately because different aspects of individual and workplace environmental factors may influence actual performance. Indicators that measure actual agency performance should be used, and different dynamics and interactions of predictors should be explored. Federal agencies are required to evaluate whether they meet targeted goals and report their agency performance through performance and accountability reporting. This performance and accountability report can be used as the actual performance of federal agencies in future research to identify important predictors of actual agency performance and the complex patterns and interactions between predictors. Finally, this study used the GUIDE classification and regression-tree method to examine the complex, dynamic interaction patterns of predictors that differentiate government employees with high or low risk of POP. As a supervised machine-learning approach, GUIDE has advantages such as accurate identification of subgroups and optimal prediction of the outcome of interest. However, although GUIDE generates the best prediction model of an outcome (i.e., the best tree model) that makes interpretation easy, this analytic strategy may not reveal other important and possible scenarios. Future research may employ other computational intelligence algorithms (e.g., monarch butterfly optimization, elephant herding optimization, and slime mould algorithms) to cluster hidden patterns of subgroups of POP without the use of labeled datasets.

5. Conclusions

This study identified factors associated with public health and human services employees’ POP, classified high-, moderate-, and low-risk subgroups, and developed targeted interventions for those at-risk subgroups. The machine-learning classification decision-tree model efficiently identified that the two most important split variables of performance-oriented culture and organizational satisfaction interacted with other variables (e.g., task-oriented leadership role, work security and workplace safety, commitment to their agency, goal clarity, organizational procedural justice, and employee development) while worsening public health and human services employees’ low POP. It also highlights the value of identifying complex interaction among risk factors and developing targeted interventions for high- and moderate-risk subgroups when modeling HRM practices to improve public health and human services employees’ POP. Based on the findings of this study, leaders and managers in the public health and human services agencies should cultivate a performance-oriented culture where a large group of best performers are recognized based on their performance, thus enhancing employees’ POP. Interventions might be designed or applied to mitigate risk factors that are strongly relevant to risk subgroups of POP. Targeted interventions may increase the cost-effectiveness of interventions so that leaders and managers allocate more resources to at-risk subgroups to enhance their POP. Despite the benefits of targeted interventions for at-risk subgroups of POP, the mechanism through which risk factors influence POP, which in turn improves or impedes organizational sustainability, is still unknown. We would encourage future research to discover that mechanism for enhancing organizational sustainability. This sort of study would help determine more effective sustainable HRM practices and improve organizational sustainability.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/su131810329/s1, Table S1: Predictor Variables Included in the Machine Learning Classification Tree Model (FEVS, 2018).

Author Contributions

All authors listed conceived the research idea, designed the study, and contributed to the manuscript. I.-G.K. conceptualized the study, prepared data, interpreted data findings and drafted the manuscript. N.K. did data analysis, interpreted data findings and drafted the manuscript. W.-Y.L. performed data analysis, interpreted findings, and critically reviewed the manuscript. B.A.B. contributed to interpreting the findings and critically reviewed and edited the manuscript. All authors critically developed the manuscript and approved the final version of the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

FEVS data are de-identified and publicly available; thus, institutional review board approval is not required in conformance with 45 CFR 46.

Informed Consent Statement

Not applicable.

Data Availability Statement

Publicly available datasets were analyzed in this study. These data can be found here: [https://www.opm.gov/fevs/public-data-file/ (accessed on 3 September 2021)].

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Carmeli, A.; Gilat, G.; Waldman, D.A. The role of perceived organizational performance in organizational identification, adjustment and job performance. J. Manag. Stud. 2007, 44, 972–992. [Google Scholar] [CrossRef]
  2. Dutton, J.E.; Dukerich, J.M.; Harquail, C.V. Organizational images and member identification. Adm. Sci. Q. 1994, 39, 239–263. [Google Scholar] [CrossRef] [Green Version]
  3. Cialdini, R.B.; Borden, R.J.; Thorne, A.; Walker, M.R.; Freeman, S.; Sloan, L.R. Basking in reflected glory: Three (football) field studies. J. Personal. Soc. Psychol. 1976, 34, 366–375. [Google Scholar] [CrossRef]
  4. Berberoglu, A. Impact of organizational climate on organizational commitment and perceived organizational performance: Empirical evidence from public hospitals. BMC Health Serv. Res. 2018, 18, 1–9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Gyurák Babeľová, Z.; Stareček, A.; Koltnerová, K.; Cagáňová, D. Perceived organizational performance in recruiting and retaining employees with respect to different generational groups of employees and sustainable human resource management. Sustainability 2020, 12, 574. [Google Scholar] [CrossRef] [Green Version]
  6. Akram, M.S.; Goraya, M.; Malik, A.; Aljarallah, A.M. Organizational performance and sustainability: Exploring the roles of I*/T capabilities and knowledge management capabilities. Sustainability 2018, 10, 3816. [Google Scholar] [CrossRef]
  7. Brewer, G.A.; Selden, S.C. Why elephants gallop: Assessing and predicting organizational performance in federal agencies. J. Public Adm. Res. Theory 2000, 10, 685–712. [Google Scholar] [CrossRef] [Green Version]
  8. El-Ghalayini, Y. Human resource management practices and organizational performance in public sector organization. J. Bus. Stud. Q. 2017, 8, 65–80. [Google Scholar]
  9. Vermeeren, B.; Kuipers, B.; Steijn, B. Does leadership style make a difference? Linking HRM, job satisfaction, and organizational performance. Rev. Public Pers. Adm. 2014, 34, 174–195. [Google Scholar] [CrossRef]
  10. Packard, T. Staff perceptions of variables affecting performance in human service organizations. Nonprofit Volunt. Sect. Q. 2010, 39, 971–990. [Google Scholar] [CrossRef]
  11. Kim, S. Individual-level factors and organizational performance in government organizations. J. Public Adm. Res. Theory 2004, 15, 245–261. [Google Scholar] [CrossRef]
  12. Kwon, M.; Jeon, S.H. Do leadership commitment and performance-oriented culture matter for federal teleworker satisfaction with telework programs? Rev. Public Pers. Adm. 2020, 40, 36–55. [Google Scholar] [CrossRef]
  13. Verbeeten, F.H.; Speklé, R.F. Management control, results-oriented culture and public sector performance: Empirical evidence on new public management. Organ. Stud. 2015, 36, 953–978. [Google Scholar] [CrossRef]
  14. Fernandez, S.; Cho, Y.J.; Perry, J.L. Exploring the link between integrated leadership and public sector performance. Leadersh. Q. 2010, 21, 308–323. [Google Scholar] [CrossRef]
  15. Jung, C.S.; Lee, S.-Y. The Hawthorne studies revisited: Evidence from the US federal workforce. Adm. Soc. 2015, 47, 507–531. [Google Scholar] [CrossRef]
  16. Kim, W.C.; Mauborgne, R. Procedural justice, strategic decision making, and the knowledge economy. Strateg. Manag. J. 1998, 19, 323–338. [Google Scholar]
  17. Blouch, R.; Azeem, M.F. Effects of perceived diversity on perceived organizational performance: Mediating role of perceived organizational justice. Empl. Relat. Int. J. 2019, 41, 1079–1097. [Google Scholar] [CrossRef]
  18. Berberoglu, A. Organizational commitment and perceived organizational performance among health care professionals: Empirical evidence from a private Hospital in Northern Cyprus. J. Econ. Behav. Stud. 2015, 7, 64–71. [Google Scholar] [CrossRef]
  19. Latham, G.P.; Locke, E.A. Self-regulation through goal setting. Organ. Behav. Hum. Decis. Process. 1991, 50, 212–247. [Google Scholar] [CrossRef]
  20. Van der Hoek, M.; Groeneveld, S.; Kuipers, B. Goal setting in teams: Goal clarity and team performance in the public sector. Rev. Public Pers. Adm. 2018, 38, 472–493. [Google Scholar] [CrossRef] [Green Version]
  21. Ng, E.S.; Burke, R.J. Person–organization fit and the war for talent: Does diversity management make a difference? Int. J. Hum. Resour. Manag. 2005, 16, 1195–1210. [Google Scholar] [CrossRef]
  22. Allen, R.S.; Dawson, G.; Wheatley, K.; White, C.S. Perceived diversity and organizational performance. Empl. Relat. 2008, 30, 20–33. [Google Scholar] [CrossRef] [Green Version]
  23. Fernandez, S.; Moldogaziev, T. Empowering public sector employees to improve performance: Does it work? Am. Rev. Public Adm. 2011, 41, 23–47. [Google Scholar] [CrossRef] [Green Version]
  24. Kang, I.G.; Bichelmeyer, B.A. Advancing the knowledge base toward a comprehensive model—Part 1: An empirically tested performance model. Perform. Improv. 2019, 58, 26–36. [Google Scholar] [CrossRef]
  25. Kang, I.G.; Bichelmeyer, B.A. Advancing the knowledge base toward a comprehensive model—Part 2: A holistic approach to performance improvement practices. Perform. Improv. 2019, 58, 13–20. [Google Scholar] [CrossRef]
  26. Kang, I.G. Empirical Testing of a Human Performance Model: Understanding Success in Federal Agencies Using Second-Order Structural Equation Modeling. Ph.D. Thesis, Indiana University, Bloomington, IN, USA, 2015. [Google Scholar]
  27. Hothorn, T.; Hornik, K.; Zeileis, A. Unbiased recursive partitioning: A conditional inference framework. J. Comput. Graph. Stat. 2006, 15, 651–674. [Google Scholar] [CrossRef] [Green Version]
  28. Strobl, C.; Malley, J.; Tutz, G. An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychol. Methods 2009, 14, 323–348. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Fernandez, S.; Resh, W.G.; Moldogaziev, T.; Oberfield, Z.W. Assessing the past and promise of the Federal Employee Viewpoint Survey for public management research: A research synthesis. Public Adm. Rev. 2015, 75, 382–394. [Google Scholar] [CrossRef]
  30. Loh, W.-Y. Improving the precision of classification trees. Ann. Appl. Stat. 2009, 3, 1710–1737. [Google Scholar] [CrossRef] [Green Version]
  31. Loh, W.-Y.; Zhou, P. Variable importance scores. arXiv 2021, arXiv:2102.07765. [Google Scholar]
  32. Therneau, T.M.; Atkinson, E.J. An Introduction to Recursive Partitioning Using the RPART Routines; Technical Report 61; Mayo Clinic: Rochester, NY, USA, 1997. [Google Scholar]
  33. Loh, W.Y.; Cao, L.; Zhou, P. Subgroup identification for precision medicine: A comparative review of 13 methods. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 2019, 9, e1326. [Google Scholar] [CrossRef]
  34. Aftab, S.; Khalid, K. 4C’S of Work environment and organizational performance: Mediating roles of individual performance. Int. Trans. J. Eng. Manag. Appl. Sci. Technol. 2019, 11, 50–65. [Google Scholar]
  35. Ko, J.; Hur, S.; Smith-Walter, A. Family-friendly work practices and job satisfaction and organizational performance: Moderating effects of managerial support and performance-oriented management. Public Pers. Manag. 2013, 42, 545–565. [Google Scholar] [CrossRef]
  36. Franklin, A.L.; Pagan, J.F. Organization culture as an explanation for employee discipline practices. Rev. Public Pers. Adm. 2006, 26, 52–73. [Google Scholar] [CrossRef]
  37. Perrin, B. Implementing the Vision: Addressing Challenges to Results-Focused Management and Budgeting, Meeting on Implementation Challenges in Results Focused Management and Budgeting. 2002, pp. 11–12. Available online: https://www.oecd.org/governance/budgeting/2497163.pdf (accessed on 3 September 2021).
  38. Nelson, B. 1001 Ways to Reward Employees; Workman Publishing: New York, NY, USA, 2005. [Google Scholar]
  39. Bradler, C.; Dur, R.; Neckermann, S.; Non, A. Employee recognition and performance: A field experiment. Manag. Sci. 2016, 62, 3085–3099. [Google Scholar] [CrossRef] [Green Version]
  40. Kang, I.G.; Croft, B.; Bichelmeyer, B.A. Predictors of Turnover Intention in US Federal Government Workforce: Machine Learning Evidence that Perceived Comprehensive HR Practices Predict Turnover Intention. Public Pers. Manag. 2020. [Google Scholar] [CrossRef]
  41. Choi, S. Diversity in the US federal government: Diversity management and employee turnover in federal agencies. J. Public Adm. Res. Theory 2009, 19, 603–630. [Google Scholar] [CrossRef]
  42. Choi, S.; Rainey, H.G. Organizational fairness and diversity management in public organizations: Does fairness matter in managing diversity? Rev. Public Pers. Adm. 2014, 34, 307–331. [Google Scholar] [CrossRef]
  43. Yang, K.; Kassekert, A. Linking management reform with employee job satisfaction: Evidence from federal agencies. J. Public Adm. Res. Theory 2010, 20, 413–436. [Google Scholar] [CrossRef]
  44. Imran, R.; Majeed, M.; Ayub, A. Impact of organizational justice, job security and job satisfaction on organizational productivity. J. Econ. Bus. Manag. 2015, 3, 840–845. [Google Scholar] [CrossRef]
  45. Choi, S.; Whitford, A.B. Employee satisfaction in agencies with merit-based pay: Differential effects for three measures. Int. Public Manag. J. 2017, 20, 442–466. [Google Scholar] [CrossRef]
  46. Abdullah, A.A.; Wan, H.L. Relationships of non-monetary incentives, job satisfaction and employee job performance. Int. Rev. Manag. Bus. Res. 2013, 2, 1085. [Google Scholar]
  47. Silverman, M. Non-Financial Recognition: The Most Effective of Rewards; Institute for Employment Studies: Brighton, UK, 2004. [Google Scholar]
  48. Chandrasena, Y.; Anjala, A. Health and safety matter a lot on job satisfaction. Workforce 2019, 4, 108–112. [Google Scholar]
  49. Anderson, D.M.; Stritch, J.M. Goal clarity, task significance, and performance: Evidence from a laboratory experiment. J. Public Adm. Res. Theory 2016, 26, 211–225. [Google Scholar] [CrossRef] [Green Version]
  50. Gonzalez-Mulé, E.; Courtright, S.H.; DeGeest, D.; Seong, J.-Y.; Hong, D.-S. Channeled autonomy: The joint effects of autonomy and feedback on team performance through organizational goal clarity. J. Manag. 2016, 42, 2018–2033. [Google Scholar] [CrossRef]
Sustainability 13 10329 g001 550
Figure 1. The GUIDE Classification Tree Model of POP. Note. GUIDE = Generalized, Unbiased, Interaction Detection and Estimation; POC = Performance-oriented Culture; OS = Organizational Satisfaction; TOL = Task-oriented Leadership; WSS = Work Security and Safety; OC = Organizational Commitment; GC = Goal Clarity; ED = Employee Development; OPJ = Organizational Procedural Justice; JS = Job Satisfaction; WUP = Work-unit Performance; SSL = Satisfaction with Senior Leadership; IM = Intrinsic Motivation; H in a red circle stands for a high-risk subgroup; M in an orange circle for a moderate-risk subgroup; L in a blue circle for a low-risk subgroup.
Figure 1. The GUIDE Classification Tree Model of POP. Note. GUIDE = Generalized, Unbiased, Interaction Detection and Estimation; POC = Performance-oriented Culture; OS = Organizational Satisfaction; TOL = Task-oriented Leadership; WSS = Work Security and Safety; OC = Organizational Commitment; GC = Goal Clarity; ED = Employee Development; OPJ = Organizational Procedural Justice; JS = Job Satisfaction; WUP = Work-unit Performance; SSL = Satisfaction with Senior Leadership; IM = Intrinsic Motivation; H in a red circle stands for a high-risk subgroup; M in an orange circle for a moderate-risk subgroup; L in a blue circle for a low-risk subgroup.
Sustainability 13 10329 g001
Sustainability 13 10329 g002 550
Figure 2. Top 10 important predictor variables with importance scores greater than 1.0 in the classification tree model of POP. Note: Please refer to the Online Supplementary document (Table S1) to look at survey questionnaires for each of the top 10 important predictor variables.
Figure 2. Top 10 important predictor variables with importance scores greater than 1.0 in the classification tree model of POP. Note: Please refer to the Online Supplementary document (Table S1) to look at survey questionnaires for each of the top 10 important predictor variables.
Sustainability 13 10329 g002
Table
Table 1. Selected sample characteristics of employees in 10 public health and human services agencies, the 2018 Federal Employee Viewpoint Survey (Weighted n = 75,706).
Table 1. Selected sample characteristics of employees in 10 public health and human services agencies, the 2018 Federal Employee Viewpoint Survey (Weighted n = 75,706).
VariableUnweighted nWeighted % (SE)
Gender
 Female24,73062.9 (0.3)
 Male13,95137.1 (0.3)
Minority status
 Non-minority20,15947.4 (0.3)
 Minority17,43552.6 (0.3)
Educational attainment
 Less than college590719.3 (0.2)
 College or more32,74180.7 (0.2)
Supervisory status
 Non-supervisor/team leader32,60782.0 (0.2)
 Supervisor/manager/senior leader765218.0 (0.2)
Organizational tenure
 10 years or fewer18,48547.0 (0.3)
 Between 10 and 20 years11,42929.6 (0.2)
 More than 20 years895623.4 (0.2)
Turnover intention
 No29,83472.1 (0.2)
 Yes11,14527.9 (0.2)
Job satisfaction
 Very dissatisfied or dissatisfied453910.9 (0.2)
 Neither satisfied nor dissatisfied601014.9 (0.2)
 Very satisfied or satisfied31,27074.1 (0.2)
Perceived organizational performance
 Strongly disagree or disagree16954.7 (0.1)
 Neither agree nor disagree505913.5 (0.2)
 Strongly agree or agree35,02781.8 (0.2)
Note. SE = standard error of percent.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kang, I.-G.; Kim, N.; Loh, W.-Y.; Bichelmeyer, B.A. A Machine-Learning Classification Tree Model of Perceived Organizational Performance in U.S. Federal Government Health Agencies. Sustainability 2021, 13, 10329. https://doi.org/10.3390/su131810329

AMA Style

Kang I-G, Kim N, Loh W-Y, Bichelmeyer BA. A Machine-Learning Classification Tree Model of Perceived Organizational Performance in U.S. Federal Government Health Agencies. Sustainability. 2021; 13(18):10329. https://doi.org/10.3390/su131810329

Chicago/Turabian Style

Kang, In-Gu, Nayoung Kim, Wei-Yin Loh, and Barbara A. Bichelmeyer. 2021. "A Machine-Learning Classification Tree Model of Perceived Organizational Performance in U.S. Federal Government Health Agencies" Sustainability 13, no. 18: 10329. https://doi.org/10.3390/su131810329

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop