Psychometric Properties of Heavy Work Investment Measures: A Systematic Review

Abstract

In recent years, the study of heavy work investment (HWI) has been diversifying greatly in the various fields of application in the organizational field, for example, occupational health, human resources, quality at work among others. However, to date, no systematic review has been carried out to examine the methodological quality of the instruments designed to measure HWI. Therefore, the present systematic review examines the psychometric properties of three main measures of HWI: Workaholism Battery (WorkBAT), Work Addiction Risk Test (WART), and Dutch Work Addiction Scale (DUWAS). Five electronic databases were systematically searched, selecting psychometric articles. Of the 2621 articles identified, 35 articles met all inclusion criteria published between 1992 and 2019. The findings indicated that most of the articles were focused on reviewing psychometric properties, analyses were conducted from classical test theory, collected validity evidence based on internal structure and relationship with other variables, and reliability of scores was obtained through the internal consistency method. Of the instruments reviewed, the DUWAS is the one with the highest methodological quality. Recommendations are made for future research to address the psychometric study of these instruments based on recent advances in the field of organizational measurement.

1. Introduction

Heavy work investment (HWI) is a concept developed by Snir and Harpaz [1] who define it as the time and effort invested in work as the two central axes of HWI. Based on Weiner’s attributional framework they propose three main categories of possible predictors of HWI: background (e.g., gender, parenthood, education), internal (e.g., work addiction, workaholism), and external variables (e.g., financial needs, employer demands) [2]. Thus, the present review focuses on the possible internal predictors, i.e., work addiction and workaholism. Current theoretical proposals of HWI use the job demands–resources (JD–R) model, considering this construct as a continuum consisting of three main parts: antecedents, dimensions, and outcomes [3].

HWI is a construct that underlies many others such as work addiction, work engagement, passion for work, workaholism, among others [4]. That is, HWI is seen as a higher-level umbrella construct and that work addiction, work engagement, passion for work, and workaholism would be at a lower-level, subsumed by HWI [3]. The time invested in work consists of the use of a large number of hours, and work intensity is the effort involved in the various tasks that a person performs at the physical and intellectual level. Regarding internal predictors, work addiction presents compulsive behaviors due to the pressure received by workers [5]. Therefore, within work addiction, heavy investment of time is marked. It is important to note that HWI is not in itself a negative or positive construct, but that this will depend on the context in which it develops. Thus, a positive implication of this variable is work commitment, where the worker puts dedication and strength into his work activities [6].

Several studies have explored the relationship between workaholism and work engagement, showing significant associations [7], although with varying results, both positively and negatively [2]. In this sense, HWI has been related to greater job security, better career development opportunities, and higher salary. In parallel, HWI may also be negatively related to individual affective and behavioral outcomes that are to be avoided [8,9]. For example, workaholics tend to be associated with poorer mental and physical health, emotional and cognitive exhaustion, poor sleep habits, cardiovascular problems, poor social relationships, and work–life conflicts [10,11,12]. Thus, it is important to have tools for a correct assessment of the internal factors of HWI (workaholism or work addiction). Workaholism or workaholism includes one of the components of HWI, the heavy time investment in work, however, it does not necessarily include the other component of HWI, the intensity of work during that period [1]. Thus, workaholism or workaholism is related to one aspect of HWI.

In this regard, this review will focus on assessing the quality of three of the most widely used HWI measures reported in the literature: Workaholism Battery (WorkBAT), Work Addiction Risk Test (WART), and Dutch Work Addiction Scale (DUWAS) [13]. In addition, the definitions of standards for educational and psychological tests will be used to ensure consistency with current psychometric guidelines for instrumental studies. To date, there has not been a published study that systematically reviews research on the psychometric properties of HWI measures. This is a significant gap given the increased interest in research beyond the organizational setting. The validity of research findings is contingent on the use of tools with appropriate validity and reliability evidence that measure the constructs of interest. Therefore, the evaluation of psychometrically robust instruments through a systematic review is warranted.

The concept of validity has undergone a long evolutionary process up to the present day. In the 1985 Standards for Educational and Psychological Testing and Manuals, it is maintained that validity is a unitary concept and that the validity of a test is construct validity [14]. In the 2014 Standards, it is insisted that the term test validity is inapplicable, and therefore that of types of validity; rather, it is necessary to provide information from various sources about the purpose for which the test or measurement instrument was developed. It is reiterated that the test is not validated, but rather the inferences made from the scores of the subjects for a given purpose [15].

From this perspective, the test creator is not only responsible for the validity of the test but also the test user. Furthermore, the validity of a test is not established once and for all but is a continuous process of gathering evidence. From a scientific point of view, the only admissible validity is construct validity [16]. Therefore, the logic that underlies it and the methods used to determine it are those that correspond in general to the scientific method. Its evidence comes from various sources, which presuppose a clear definition of the construct and its dimensions or facets if these are necessary.

It is necessary to keep in mind that no study tests or validates a complete theory only concerning some of the inferences that can be derived from it. For constructs, negative results can be interpreted in three directions: the test may not measure the construct; the theoretical framework may be flawed, making it possible for incorrect inferences to be made; or the study design would not allow for proper testing of the hypotheses. These interpretations communicate poor psychometric and research theoretical training and practice, which can lead to ambiguous interpretation of negative results. Finally, it should always be kept in mind that unexpected relationships, as well as predicted ones, are part of the nomological network of the construct and provide arguments for the meaning of the scores.

Conversely, reliability is the degree to which test scores for a group of subjects are consistent in repeated applications of a measurement procedure and hence the reliability and consistency of a subject’s score is inferred [17]. It is implicit in this definition that test scores are obtained on different occasions under the same conditions as administration and scoring; it also follows that reliability refers to the precision of the measurement, regardless of what the test measures, and that the reliability of the scores is relative since it is subject to the characteristics of the group of subjects in which it is estimated [18].

2. Objectives

The objectives of the present review are to: (1) to systematically identify three of the main measures of HWI in the literature, (2) to evaluate the psychometric properties presented in the reviewed studies, and (3) to determine whether there is a gold standard measure HWI.

3. Materials and Methods

3.1. Study Design

The present review followed the official PRISMA guidelines [19] on data identification, collection, and analysis. In this sense, the article is a systematic review of instrumental studies on three of the most commonly used measures of HWI: Workaholism Battery (WorkBAT), Work Addiction Risk Test (WART), and Dutch Work Addiction Scale (DUWAS) [13].

3.2. Literature Search

First, five selection criteria were determined: (1) instrumental or psychometric studies [20], where the main objective of the article was the creation, adaptation, or psychometric analysis of an instrument measuring HWI or any of its components (Workaholism or Work addiction), specifically, the WorkBAT, WART, and DUWAS; (2) conducted in any country; (3) published up to 2020; (4) peer-reviewed articles; and (5) published in the Spanish or English language.

Three groups of keywords and phrases were used to search for potential articles. The first group is related to psychometric analysis (psychometric, validation, validity, reliability, adaptation, and dimensionality). The second group is related to HWI (heavy work investment, workaholism, work addiction, passion to work, job demands, work craving, work engagement, addiction to work, passion towards work passion for work, and heavy-work investment). The third group is related to self-report measures (questionnaire, measure, assessment, tool, instrument, scale, inventory, and battery). The final search was performed in all databases on 17 July 2021. All articles published up to 2020 were selected.

The search strategy generated a total of 2621 studies with 770 studies from Scopus, 1034 studies from Web of Science (Web of Science databases, including Science Citation Index Expanded [SCI-EXPANDED], Social Sciences Citation Index [SSCI], Arts & Humanities Citation Index [A&HCI], and Emerging Sources Citation Index [ESCI]), 654 studies from PsycNET (PsycInfo and PsycArticles), 110 studies from MEDLINE via Ovid, and 53 studies from Psychology and Behavioral Sciences Collection via EBSCO.

Table 1. Search strategy in the five electronic databases.

Database	Search Strategy
Scopus	TITLE-ABS-KEY (psychometric* OR validation OR validity OR reliability OR adaptation OR dimensionality) AND TITLE-ABS-KEY (“heavy work investment” OR workaholism OR “work addiction” OR “passion to work” OR “job demands” OR “work craving” OR “work engagement” OR “addiction to work” OR “passion towards work” OR “passion for work” OR “heavy-work investment”) AND TITLE-ABS-KEY (questionnaire OR measure* OR assessment OR tool OR instrument OR scale OR inventory OR battery) AND PUBYEAR < 2021 AND (LIMIT-TO (DOCTYPE, “ar”)) AND (LIMIT-TO (LANGUAGE, “English”) OR LIMIT-TO (LANGUAGE, “Spanish”))
Web of Science	TS = (psychometric* OR validation OR validity OR reliability OR adaptation OR dimensionality) AND TS = (“heavy work investment” OR workaholism OR “work addiction” OR “passion to work” OR “job demands” OR “work craving” OR “work engagement” OR “addiction to work” OR “passion towards work” OR “passion for work” OR “heavy-work investment”) AND TS = (questionnaire OR measure* OR assessment OR tool OR instrument OR scale OR inventory OR battery) Refined By: NOT Publication Years: 2021; Document Types: Articles; Languages: English or Spanish
PsycNET	(Any Field: psychometric* OR Any Field: validation OR Any Field: validity OR Any Field: reliability OR Any Field: adaptation OR Any Field: dimensionality) AND (Any Field: “heavy work investment” OR Any Field: workaholism OR Any Field: “work addiction” OR Any Field: “passion to work” OR Any Field: “job demands” OR Any Field: “work craving” OR Any Field: “work engagement” OR Any Field: “addiction to work” OR Any Field: “passion towards work” OR Any Field: “passion for work” OR Any Field: “heavy-work investment”) AND (Any Field: questionnaire OR Any Field: measure* OR Any Field: assessment OR Any Field: tool OR Any Field: instrument OR Any Field: scale OR Any Field: inventory OR Any Field: battery) AND Document Type: Journal Article AND Year: 0 To 2020
Psychology and Behavioral Sciences Collection via EBSCO	(psychometric* OR validation OR validity OR reliability OR adaptation OR dimensionality) AND (“heavy work investment” OR workaholism OR “work addiction” OR “passion to work” OR “job demands” OR “work craving” OR “work engagement” OR “addiction to work” OR “passion towards work” OR “passion for work” OR “heavy-work investment”) AND (questionnaire OR measure* OR assessment OR tool OR instrument OR scale OR inventory OR battery) limit year 2020
MEDLINE via Ovid	(psychometric* OR validation OR validity OR reliability OR adaptation OR dimensionality) AND (“heavy work investment” OR workaholism OR “work addiction” OR “passion to work” OR “job demands” OR “work craving” OR “work engagement” OR “addiction to work” OR “passion towards work” OR “passion for work” OR “heavy-work investment”) AND (questionnaire OR measure* OR assessment OR tool OR instrument OR scale OR inventory OR battery) limit 1 to year = “1860–2020”

shows the search strategy followed in the five electronic databases.

3.3. Selection Process

The search results were initially screened by title and abstract to exclude research that did not meet the selection criteria. Subsequently, the remaining studies were retrieved from the different databases and the full articles were evaluated according to their relevance to meet the stipulated selection criteria.

After excluding 795 duplicate searches and 1773 studies that did not meet the selection criteria (because they were qualitative studies, empirical studies, were in a language other than Spanish or English, or were psychometric but worked with other instruments), 53 articles were selected from the review of the title and abstract, and subsequently acquired the 50 articles that were reviewed in their entirety, obtaining a final sample of 35 articles containing 37 studies (one of the articles analyzed the three instruments of interest independently). The flow diagram of the selection of the preceding studies is shown in Figure 1.

3.4. Coding Process

The coding was carried out considering the following information from the items analyzed: (a) instrument name, authors, and year; (b) study design; (c) number of items in the instrument; (d) dimensions it measures; (e) number of response options; (f) sample size; (g) participant characteristics; (h) mean and standard deviation of participant age; (i) country of origin of the sample; (j) psychometric theory used for the analyses; (k) validity evidence collected; and (l) reliability evidence collected.

3.5. Literature Analysis

After selection and coding of the articles, 35 articles met all the requirements. However, one of the articles had three studies from each of the objective instruments (WorkBAT, WART and DUWAS). Therefore, 37 studies were compiled in the database for further analysis. The characteristics of each study were organized in Table S1 (Supplementary Materials). Based on the structured information, the results were elaborated to methodologically describe the three main measures of HWI.

4. Results

4.1. Characteristics of the Studies

Of the 37 studies analyzed (

Table 2. Characteristics of the studies for each instrument.

No	Instrument Name and Authors	Study Design	Dimension Measured	Psychometric Properties
				Theory	Validity	Reliability
1	Workaholism Battery (WorkBAT) (Spence and Robbins, 1992) [21]	Development	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Relations to other variables All three factors correlated with Time commitment and Work involvement. In addition, Work involvement and Drive were correlated with Job stress, Perfectionism, and Non-delegation. Finally, Drive and Work enjoyment correlated with health complaints.	Internal consistence Factors: Work involvement (α = 0.69 and 0.67), Drive (α = 0.81 and 0.67), and Work enjoyment (α = 0.86).
2	Workaholism Battery (WorkBAT) (Kanai, Wakabayashi, and Fling, 1996) [24]	Adaptation	Workaholism (Work enjoyment and Drive).	CTT	Internal structure Orthogonal two-factor model (EFA), factor loadings greater than 0.30 and variance explained 32%. Relations to other variables Both factors correlated with Time commitment, Job involvement and Perfectionism. Drive was also correlated with Job stress, Non-delegation and Health complaints.	Internal consistence Factors: Drive (α = 0.70) and Work enjoyment (α = 0.85).
3	Workaholism Battery (WorkBAT) (Burke, 1999) [25]	Psychometric properties	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Relations to other variables The three factors correlated with Job involvement, Time on the job and Hours worked. Drive correlated with Perfectionism (r = 0.42), Overtime worked (r = 0.26) and Non-delegation (r = 0.20). Work enjoyment correlated with Overtime worked (r = −0.26).	Internal consistence Factors: Work involvement (α = 0.67), Drive (α = 0.80), and Work enjoyment (α = 0.88).
4	Workaholism Battery (WorkBAT) (Burke, 2001) [26]	Psychometric properties	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Not reported.	Internal consistence Factors: Work involvement (α = 0.66 and 0.67), Drive (α = 0.73 and 0.71), and Work enjoyment (α = 0.88). Test-retest Three months. Work involvement (r = 0.52), Drive (r = 0.59), and Work enjoyment (r = 0.76).
5	Workaholism Battery (WorkBAT) (Burke and Koksal, 2002) [27]	Adaptation	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Relations to other variables The three factors were correlated with work behaviors, job satisfaction, emotional well-being, beliefs and fears, and balance values.	Internal consistence Factors: Work involvement (α = 0.56), Drive (α = 0.46), and Work enjoyment (α = 0.79).
6	Workaholism Battery (WorkBAT) (Burke, Richardsen, and Martinussen, 2002) [28]	Psychometric properties	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Internal structure Three-factor orthogonal model (PCA) and variance explained 39%.	Internal consistence Factors: Work involvement (α = 0.45 and 0.64), Drive (α = 0.81 and 0.87), and Work enjoyment (α = 0.85 and 0.84). Test-retest Six months. Work involvement (r = 0.49), Drive (r = 0.45), and Work enjoyment (r = 0.56).
7	Workaholism Battery Revised (WorkBAT-R), revised version of the Workaholism Battery (WorkBAT) (McMillan et al., 2002) [29]	Psychometric properties	Workaholism (Work enjoyment and Drive).	CTT	Internal structure Orthogonal two-factor model (EFA), factor loadings greater than 0.40, variance explained 41.43%, correlation between factors 0.22. Relations to other variables Enjoyment and Drive were correlated with job satisfaction (except for Drive), work involvement, an alternative measure of work addiction (SNAP-Work), and intrinsic work motivation. Likewise, the number of hours worked correlated 0.16 with Enjoyment and 0.22 with Drive.	Internal consistence Factors: Enjoyment (α = 0.85) and Drive (α = 0.75).
8	Workaholism Battery (WorkBAT) (Ersoy-Kart, 2005) [30]	Adaptation	Workaholism (Work involvement and Drive).	CTT	Internal structure Two-factor oblique model (EFA), factor loadings greater than 0.40, variance explained 29.60%, correlation between factors was 0.47. Relations to other variables Work involvement (r = 0.22) and drive (r = 0.24) were correlated with type A behavior.	Internal consistence Workaholism (α = 0.83, split-half reliability coefficient = 0.69). Factors: Work involvement (α = 0.81) and Drive (α = 0.81).
9	Workaholism Battery (WorkBAT) (Andreassen, Ursin, and Eriksen, 2007) [31]	Psychometric properties	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Internal structure The two-factor (no work involvement) and three-factor (CFA) models had an acceptable fit. Relations to other variables Drive and Work enjoyment were correlated with work stress, burnout and subjective health complaints. Work enjoyment was also correlated with work engagement components.	Internal consistence Factors: Work involvement (α = 0.49), Drive (α = 0.80), and Work enjoyment (α = 0.79).
10	Workaholism Battery (WorkBAT) (Huang, Hu, and Wu, 2010) [32]	Adaptation	Workaholism (Work enjoyment, Work involvement-enjoyment, Drive-work involvement, Drive, and Work involvement).	CTT	Internal structure Five-factor model (EFA), factor loadings greater than 0.30, variance explained 54.65%, correlation between factors ranged from −0.24 to 0.31. Relations to other variables The five WorkBAT factors correlated with the Work Addiction Risk Test (WART) and most of its factors, as well as with career commitment, job involvement, emotional exhaustion, and job satisfaction.	Internal consistence Factors: Enjoyment (α = 0.88), Work involvement-enjoyment (α = 0.69), Drive-work involvement (α = 0.58), Drive (α = 0.73), and Work involvement (α = 0.60).
11	Workaholism Battery (WorkBAT) (Andreassen, Hetland, and Pallesen, 2013) [13]	Psychometric properties	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Internal structure Four-factor oblique model (PCA), factor loadings greater than 0.40. The three-factor model (CFA) had a poor fit. Relations to other variables The Drive factor had the best correlations with the WART and DUWAS. Work involvement and Work enjoyment showed variable results.	Internal consistence Factors: Involvement (α = 0.63), Drive (α = 0.82), and Work enjoyment (α = 0.84). Test-retest 24–30 months. Work involvement (ICC = 0.65), Drive (ICC = 0.64), and Work enjoyment (ICC = 0.61).
12	Workaholism Battery (WorkBAT) (Boada-Grau et al., 2013) [33]	Adaptation	Workaholism (Work enjoyment and Drive).	CTT	Internal structure Two-factor oblique model (EFA), factor loadings greater than 0.30, variance explained 42.64%, correlation between factors 0.19. Two-factor related and good fit model (CFA). Relations to other variables The WorkBAT correlated with irritation, burnout and obsessive beliefs.	Internal consistence Factors: Work enjoyment (α = 0.82, CI 0.80–0.84) and Drive (α = 0.80, CI 0.78–0.83).
13	Workaholism Battery (WorkBAT) (Santos et al., 2018) [34]	Adaptation	Workaholism (Work involvement, Work enjoyment, and Drive).	CTT	Internal structure Two- and three-factor related (CFA) model, poor fit in all goodness-of-fit indices.	Internal consistence Workaholism (α = 0.81). Factors: Work involvement (α = 0.56), Drive (α = 0.82), and Work enjoyment (α = 0.76).
14	Work Addiction Risk Test Revised (WART-R), revised version of the Work Addiction Risk Test (WART) (Urbán et al., 2019) [35]	Adaptation	Work addiction (Overcommitment, Impatience, Hard-working, and Salience).	N/A	Internal structure Four-factor oblique model (EFA), factor loadings greater than 0.40 and correlations between factors from 0.16 to 0.50. Related four-factor model (CFA), good fit, factor loadings between 0.39 and 0.73 and correlations between factors from 0.47 to 0.74. Relations to other variables The amount of Time spent working, Mental health symptoms, and Hostility were significantly associated with four factors.	Not reported.
15	Work Addiction Risk Test (WART) (Robinson, Post, and Khakee, 1992) [36]	Psychometric properties	Work addiction	CTT	Not reported.	Internal consistence Work addiction (α = 0.85). Test-retest Two weeks. Work addiction (r = 0.83).
16	Work Addiction Risk Test (WART) (Robinson and Post, 1994) [37]	Psychometric properties	Work addiction	N/A	Test content Subjects were presented with five of the major symptoms of work addiction around which the 25-item measure was constructed: Overdoing, Self- worth, Control-Perfectionism, Intimacy, and Mental Preoccupation-Future Reference. Subjects were asked to select the symptom that matched each of the 25 items. The percentage of correct categorizations ranged from 40% to 96%.	Not reported.
17	Work Addiction Risk Test (WART) (Robinson and Phillips, 1995) [38]	Psychometric properties	Work addiction	N/A	Test content Ten statements unrelated to work addiction were nested throughout the 25-item test. Subjects were asked to identify the 25 items from a list of 35 statements they believed to be symptoms of work addiction. The mean percentage score of correctly identified symptoms was 89.4%. The percentages ranged from 65% to 100%.	Not reported.
18	Work Addiction Risk Test (WART) (Robinson and Post, 1995) [39]	Psychometric properties	Work addiction	CTT	Not reported.	Internal consistence Work addiction (Spearman-Brown split-half correlation coefficient = 0.85).
19	Work Addiction Risk Test (WART) (Robinson, 1996) [40]	Psychometric properties	Work addiction	N/A	Relations to other variables Workaholism correlated with generalized anxiety (r = 0.40), type A behavior patterns (r = 0.37), and with scores on the four scales of the Jenkins Activity Survey with 0.50 on the Type A scale, 0.50 on the Speed and Impatience scale, 0.39 on the Hard-driving and Competitive scale, and 0.20 on the Job Involvement scale.	Not reported.
20	Work Addiction Risk Test (WART) (Robinson, 1999) [22]	Development	Work addiction	CTT	Relations to other variables Work addiction was correlated with anxiety (r = 0.40), Type A behaviors (r = 0.37), Type A (r = 0.50), Speed and impatience (r = 0.49), Hard-driving and competitive (r = 0.38), and Job involvement (r = 0.20).	Internal consistence Work addiction (α = 0.88).
21	Work Addiction Risk Test (WART) (Flowers and Robinson, 2002) [41]	Psychometric properties	Work addiction (Compulsive tendencies, Control, Impaired communication/Self-absorption, Inability to delegate, and Self-Worth).	CTT	Internal structure Five-factor oblique model (PCA), factor loadings greater than 0.40 and variance explained 52%. Relations to other variables Four discriminant analyses were conducted to examine the correct classification rate of scores on the WART and explore which of the factors and items accounted for the differences in the average score profiles of the WG and CW. The correct classification rate for the CG remained consistent, ranging from 93.8 to 95.3. The correct classification rate for the WG varied between 57.3 and 70.	Internal consistence Work addiction (α = 0.90).
22	Work Addiction Risk Test (WART) (Taris, Schaufeli, and Verhoeven, 2005) [42]	Adaptation	Work addiction (Compulsive tendencies, Control, Impaired Communication/Self-Absorption, Inability to delegate, and Self-worth).	CTT	Internal structure Model with five first-order and one second-order factor (CFA), good fit and factor loadings between 0.30 and 0.80 (first order) and 0.85 and 0.96 (second order). Relations to other variables The WART correlated highly with the Compulsive tendencies factor (r = 0.89 and 0.93), which is proposed as a short version of the WART. Both versions correlated with job stress (job demands and overtime), job strain (work–nonwork conflict, exhaustion, and cynism), and mental health.	Internal consistence Work addiction (α = 0.93). Factors: Compulsive tendencies (α = 0.90), Control (α = 0.82), Impaired Communication/Self-Absorption (α = 0.62), and Self-worth (α = 0.56).
23	Work Addiction Risk Test (WART) (Andreassen, Hetland, and Pallesen, 2013) [13]	Psychometric properties	Work addiction (Compulsive tendencies (CT), Control (CL), Impaired communication (IC), Self-worth (SW), and Inability to delegate (ID)).	CTT	Internal structure Four-factor oblique model (PCA), factor loadings greater than 0.40. The five-factor model (CFA) had a poor fit. Relations to other variables The five WART factors (and the total) correlated with the DUWAS factors and with the WorkBAT Drive factor.	Internal consistence Work addiction (α = 0.86). Factors: CT (α = 0.77), CL (α = 0.75), IC (α = 0.59), and SW (α = 0.36). Test-retest 24–30 months. Work addiction (ICC = 0.70), CT (ICC = 0.63), CL (ICC = 0.69), IC (ICC = 0.56), ID (ICC = 0.32), and SW (ICC = 0.56).
24	Work Addiction Risk Test Portuguese of Brazil Version (WART15-PBV), adapted version of the Work Addiction Risk Test (WART) (Romeo et al., 2014) [43]	Adaptation	Work addiction (Compulsive tendencies, Control, and Impaired Communication/Self-Absorption).	CTT	Internal structure Three-factor model (CFA), good fit and factor loadings between 0.36 and 0.95. Relations to other variables WART15-PBV correlated with the DUWAS (r = 0.90) and the correlations between the factors of both tests were greater than 0.50. WART15-PBV also correlated with general health perception (r = 0.29).	Internal consistence Work addiction (α = 0.83). Factors: Compulsive tendencies (α = 0.79), Control (α = 0.54), and Impaired communication/Self-absorption (α = 0.68).
25	Work Addiction Risk Test (WART) (Ravoux et al., 2018) [44]	Adaptation	Work addiction (Compulsive tendencies, Control, Impaired communication and self-absorption, Self-worth, and Inability to delegate).	CTT	Internal structure Four-component model (PCA). Relations to other variables Work addiction and the factors Compulsive tendencies, Control, and Impaired communication and self-absorption were correlated with the visual analog scale of stress at work (r = 0.43), stress at home (r = 0.41), and the visual analog scale of well-being (r = −0.40).	Internal consistence Work addiction (α = 0.90). Factors: α between 0.57 and 0.85. Test-retest One week. The Lin concordance coefficient indicated a value of 0.90 for the total test and values between 0.66 and 0.86 for the factors.
26	Dutch Work Addiction Scale (DUWAS) (Schaufeli, Shimazu, and Taris, 2009) [23]	Development	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Internal structure Two-component model (PCA), factor loadings between 0.57 and 0.82 (TN) and 0.52 and 0.74 (JP). Explained variance of 52.5% (TN) and 46.5% (JP). Related two-factor model (CFA), good fit and relationship between factors high in TN (r = 0.50) and JP (r = 0.59). Factor invariance: MG-CFA tested the equivalence of the CFA (configural) model in the Dutch and Japanese samples. Relations to other variables All correlations between workaholism (WE and WC) and excess working time (overtime percentage and overwork) are positive. Workaholism was not related to Engagement (r = −0.19, NT; r = −0.05, JP) and was related to Burnout (r = 0.53, NT; r = 0.64, JP).	Internal consistence Factors (TN): WE (α = 0.78) and WC (α = 0.78). Factors (JP): WE (α = 0.73) and WC (α = 0.68).
27	Dutch Work Addiction Scale (DUWAS) (del Líbano et al., 2010) [45]	Psychometric properties	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Relations to other variables (DUWAS-10 and DUWAS-17): Intercorrelations between the original and shortened factors ranged between 0.92 and 0.94 in the TN and SP samples. The intercorrelations of WE and WC with perceived health and happiness were negative in both samples. Internal structure (DUWAS-10): Two-factor related (CFA) model and good fit in the TN and SP samples. Factor invariance (DUWAS-10): MG-CFA tested the equivalence of the CFA model (factor structure, covariance, and factor loadings) between the TN and SP samples.	Internal consistence DUWAS-17 Factors. TN: WE (α = 0.82) and WC (α = 0.84). SP: WE (α = 0.85) and WC (α = 0.79). DUWAS-10 Factors. TN: WE (α = 0.75) and WC (α = 0.81). SP: WE (α = 0.78) and WC (α = 0.79).
28	Dutch Work Addiction Scale -Observer Rating (DUWAS-OR) (Falco et al. 2012) [46]	Psychometric properties	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Internal structure Related two-factor model (CFA), acceptable fit, factor loadings between 0.61 and 0.86, in addition, the correlation between factors was 0.69. Relations to other variables AVE was 0.50 for WE and 0.65 for WC. In addition, the two factors correlated with the UWES (self-report), WE (r = 0.49) and WC (r = 0.43). WE and WC correlated with Workload and WE with Work–family conflict. Discriminant evidence: Evidence was provided through the Fornell and Larcker criterion for the two factors.	Internal consistence Factors: WE (α = 0.86) and WC (α = 0.89).
29	Dutch Work Addiction Scale (DUWAS) (Molino, Ghislieri, and Colombo, 2012) [47]	Psychometric properties	Working excessively	CTT	Internal structure One-factor model (EFA), factor loadings between 0.41 and 0.76 and explained a variance of 38.62%. Unifactorial model (CFA), good fit, factor loadings between 0.38 and 0.84. Relations to other variables Relationship between Working excessively and Work–family conflict (r = 0.49).	Internal consistence Working excessively (α = 0.74).
30	Dutch Work Addiction Scale (DUWAS) (Andreassen, Hetland, and Pallesen, 2013) [13]	Psychometric properties	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Internal structure Two-factor oblique model (PCA), factor loadings greater than 0.40. The two-factor model (CFA) had a poor fit. Relations to other variables The two DUWAS factors correlated with the WART factors and the WorkBAT factors.	Internal consistence Factors: WE (α = 0.69) and WC (α = 0.63). Test-retest 24–30 months. WE (ICC = 0.61) and WC (ICC = 0.65).
31	Dutch Work Addiction Scale (DUWAS) (Sharma and Sharma, 2013) [48]	Psychometric properties	Workaholism (Working excessively (WE), Working compulsively (WC), and Overwork(O)).	CTT	Relations to other variables Convergent evidence: AVE greater than 0.50 for the three factors. Discriminant evidence: Evidence was provided through the Fornell and Larcker criterion for the three factors. Internal structure Three-factor hierarchical model (CFA). Poor fit. Factor loadings between 0.50 and 0.77 (first order), and between 0.51 and 0.89 (second order).	Internal consistence Workaholism (α = 0.83). Factors: WE (α = 0.88), WC (α = 0.83), and O (α = 0.60).
32	Dutch Work Addiction Scale (DUWAS) (Littman-Ovadia, Balducci, and Ben-Moshe, 2014) [49]	Adaptation	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Internal structure Related two-factor model (CFA), acceptable fit, factor loadings between 0.38 and 0.77, in addition, the correlation between factors was 0.76. Relations to other variables Self-reports and peer-reports of workaholism (UWES answered by spouse, boyfriend, girlfriend, friend, or colleague) correlated; Workaholism (r = 0.52), WE (r = 0.50), and WC (r = 0.43). Workaholism and its scales showed positive correlations with overcommitment, the actual number of hours worked per week, burnout (emotional exhaustion), work engagement (Absorption), and intrinsic aspects of the job.	Internal consistence Workaholism (α = 0.78). Factors: WE (α = 0.61) and WC (α = 0.70). Test-retest Two or three months. Workaholism (r = 0.79). Factors: WE (r = 0.77) and WC (r = 0.71).
33	Dutch Work Addiction Scale (DUWAS) (Rantanen et al., 2015) [50]	Psychometric properties	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Internal structure Related two-factor model (CFA), an acceptable fit in both samples. However, a second-order model showed a better fit (WE: working frantically and working long hours; WC: obsessive work drive and unease if not working) with loadings greater than 0.50. Likewise, a four-factor related model (with the first-order factors of the previous model) also indicated a good fit. Factorial invariance: Second-order factor structure showed reasonable measurement invariance and stability of factor structure across the two samples and time in the Finnish sub-sample of managers with two measurement points two years apart.	Internal consistence Workaholism (α = 0.82, TN; α = 0.86, FL). Factors (TN): WE (α = 0.72) and WC (α = 0.80). Factors (FL): WE (α = 0.80) and WC (α = 0.80). Subfactors α between 0.51 and 0.80. Test-retest Two-year. Factors: WE (r = 0.66) and WC (r = 0.76). Subfactors’ r between 0.60 and 0.71.
34	Dutch Work Addiction Scale (DUWAS) (Mir, Kamal, and Masood, 2016) [51]	Adaptation	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Internal structure Two-factor model (CFA), good fit, factor loadings between 0.46 and 0.74, 5 items for WE and 3 items for WC.	Internal consistence Workaholism (α = 0.71). Factors: WE (α = 0.66) and WC (α = 0.64).
35	Dutch Work Addiction Scale (DUWAS) (Azevedo and Mathias, 2017) [52]	Psychometric properties	Addiction to work	CTT	Internal structure One-component model (PCA), factor loadings between 0.60 and 0.70, and explained variance of 44.5%. Relations to other variables Addiction to work was positively correlated with the number of shifts worked (r = 0.20), and it showed a negative correlation with age (r = −0.20).	Internal consistence Addiction to work (α = 0.86).
36	Dutch Work Addiction Scale (DUWAS) (Balducci et al., 2017) [53]	Psychometric properties	Workaholism (Working excessively (WE) and Working compulsively (WC)).	CTT	Internal structure Related two-factor model (CFA), good fit and factor loadings between 0.44 and 0.72. Factor invariance: MG-CFA tested the equivalence of the CFA model (configural, metric, factor variance and factor covariance) between the Italian sample and another Dutch sample (n = 7523). Relations to other variables DUWAS and its factors are positively related to the number of hours worked in a week, job demands (workload and work-to-family conflict), high and low arousal job-related negative affect, and psychological strain symptoms.	Internal consistence Workaholism (α = 0.82). Factors: WE (α = 0.74, CI = 0.72–0.77; ω = 0.75) and WC (α = 0.74, CI = 0.72–0.77; ω = 0.75). Test-retest One-year. Workaholism (r = 0.57). Factors: WE (r = 0.62) and WC (r = 0.54).
37	Dutch Work Addiction Scale (DUWAS) (Nonnis et al., 2017) [54]	Psychometric properties	Workaholism (Working excessively (WE) and Working compulsively (WC)).	RMT	Internal structure The parallel analysis of the residuals shows that the work engagement has two significant components. However, the eigenvalues are below the cut-off point (2) for both factors.	Internal consistence Person separation reliability R, which scored 0.49 in the WE and 0.56 in the WC.

Note. USA = United States, PCA = Principal Component Analysis, EFA = Exploratory Factor Analysis, CFA = Confirmatory Factor Analysis, CTT = Classical Test Theory, RMT = Rasch Measurement Theory, α = Alpha coefficient, ω = Omega coefficient, r = Correlation coefficient, AVE = Average Variance Extracted, Mdn = Median, CI = Confidence interval, CCI = Intraclass Correlation Coefficient, N/A = Not Applicable.

), 13 belong to WorkBAT, 12 correspond to WART and 12 worked with DUWAS. The studies were published between 1992 and 2019. Regarding the study design of the articles reviewed, most of them (n = 22) focused on reviewing the psychometric properties (validity and reliability) of the WorkBAT, WART, and DUWAS. Likewise, a significant number of adaptations were reported (n = 12), where they sought to adjust the original tests to different cultural contexts. Finally, three studies were found where the tests that are the main focus of this review were developed or constructed [21,22,23].

Regarding the number of items, it was observed that the psychometric tests analyzed varied in number due to different factors: (1) they were short versions; (2) items were eliminated due to methodological problems; or (3) they only studied some factors of the instrument. Conversely, the dimensions measured by the tests also changed from one study to another, since the structures were not fully replicated in different samples. Nevertheless, the DUWAS was the one that most of the time showed a two-factor structure (nine studies). In terms of the number of response options, the DUWAS (from “almost never” to “almost always”) and the WART (from “never true” to “always true”) presented almost the same four-point Likert scale across studies. The WorkBAT was usually answered on a five-point Likert scale (from “strongly agree” to “strongly disagree”).

As for the sample size, in most cases it did not exceed 500 participants, which is necessary to obtain more robust and stable results, especially considering the diversity of structures presented by these tests. Furthermore, in some studies, the sample was larger than 1000 people. The participants in the studies presented diverse characteristics, mainly workers from different economic sectors (e.g., manufacturing, retail, service, education, or medical). Moreover, the age of the participants was generally between 30 and 50 years old. Finally, among the countries where these instruments have been most tested are the United States, Canada, Norway, the Netherlands, Italy, and Brazil, as well as others located in Asia and Oceania. No studies were found in Latin America or Africa.

4.2. Measurement Theory

The measurement theory that predominated in the studies was the Classical Test Theory (CTT) and only one study used the Rasch Measurement Theory (RMT) [54]. No study used the Item Response Theory (IRT). In this regard, it is important to note that some studies did not use a specific theory, because their objective was to collect evidence of validity and they did not focus on the analysis of items or the reliability of the scores [37,38,40]. CTT proposes a linear relationship between the observed score, the true score, and the measurement error, with the main limitation being the dependence of the persons and the test [55]. Conversely, the IRT seeks to explain the relationship between a person’s ability and the probability of answering an item correctly, considering various characteristics such as difficulty, discrimination, or pseudo-guessing [56]. The IRT is considered a descriptive model since it tries to explain as much of the variance as possible [57]. Finally, the RMT is a prescriptive model, in which it is of interest to know whether the data fit the measurement model and, like the IRT, it focuses on the individual analysis of the items based on the interaction between an item and a person [58].

4.3. Validity

Regarding the collection of validity evidence, the studies analyzed focused mainly on two aspects: the internal structure and the relationship with other variables. However, two studies collected evidence based on test content. This source of validity evidence refers to the degree to which the content (domain definition, domain representation, domain relevance, and appropriateness of construction processes) of a test is congruent with the measurement objectives [59].

4.3.1. Validity Evidence Based on Test Content

Two studies collected this source of evidence based on the presentation of the items and their classification by a group of students or experts in the field, a method different from the one usually used to collect this evidence of validity, which consists of a group of judges giving their assessment of the items according to certain features (representativeness, relevance or clarity). However, the method used in these studies has the characteristic of analyzing the representativeness of the items [37,38]. Both studies belonged to the WART.

4.3.2. Validity Evidence Based on Internal Structure

The three tests studied are multidimensional, so this source of evidence is one of the most widely used to corroborate how the instruments are structured according to their items. The techniques used for this purpose were mainly three: principal components analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). CFA was usually employed in studies reviewing psychometric properties as the first option, but, in cases where the fit was poor, a less restrictive method, such as PCA or EFA, was chosen. Of the three psychometric instruments, the DUWAS replicated its two-factor structure the most often. Thus, in nine studies, it was observed that the DUWAS was made up of the factors Working excessively and Working compulsively. Regarding the WorkBAT, a three-factor structure (Work involvement, Work enjoyment, and Drive) was observed in eight studies, although a two-factor structure (Work enjoyment and Drive) was also found in four studies. Finally, the WART showed a unidimensional structure in six studies, while four studies reported a five-factor structure (Compulsive tendencies, Control, Impaired communication, Self-worth, and Inability to delegate).

Within this validity, evidence is also the factorial invariance analysis that was employed by some studies to compare whether the structure of the instrument was invariant concerning the country of origin of the participants [23,53]. Factor invariance was performed through multi-group confirmatory factor analysis (MG-CFA), although in some cases the description of the procedure and results was not clear.

4.3.3. Validity Evidence Based on Relations to Other Variables

Since HWI is a relatively recent construct, it is necessary to form a nomological network to determine with which other variables it is related and with which it diverges. In this sense, this source of validity evidence made it possible to relate the total and factor scores of the WorkBAT, WART, and DUWAS to other variables such as Job stress, Job involvement, Time on the job, Overtime worked, Job satisfaction, and other variables linked to the organizational field. In most cases, simple correlation coefficients or simple linear regressions were used, with few studies analyzing the relationship between variables from a more advanced perspective such as structural equation models.

4.4. Reliability

The reliability of the scores was mostly evaluated by analyzing the internal consistency of the items and using the alpha coefficient. However, in no case were the assumptions that this reliability estimator needs to be used (unidimensional, absence of correlated errors, and tau-equivalence of the measurement model) corroborated. In addition, only one study reported confidence intervals for coefficient alpha. The DUWAS was the instrument that most often presented good levels of reliability (greater than 0.70). However, in some studies, it was found to be below this criterion. The WorkBAT and the WART had greater reliability problems in some of their dimensions, probably due to the number of factors (usually four) and the number of items that composed them.

In contrast, some studies evaluated reliability based on the temporal stability or test-retest method, which is necessary to determine whether test scores are constant over time. In most cases, the relationship between scores at time 1 and time 2 was low, less than 0.70 [28]. However, this could be influenced by the use of the simple correlation coefficient used to assess the agreement between the two measures. Only two studies used appropriate statistics for this purpose, such as the intraclass correlation coefficient [13] and Lin’s coefficient of concordance [44].

5. Discussion

The present review aimed to (1) systematically identify three of the main measures of HWI in the literature, (2) evaluate the psychometric properties presented in the reviewed studies, and (3) determine whether there is a gold standard measure of HWI. Regarding the first objective, three measurement instruments were analyzed, the Workaholism Battery (WorkBAT), Work Addiction Risk Test (WART), and Dutch Work Addiction Scale (DUWAS), which are the main measures with which various studies approach the phenomenon of HWI. Regarding the second objective, the empirical evidence of the instruments was systematized concerning the validity and reliability of their scores. Finally, regarding the third objective, although the DUWAS is the test that shows the best performance, because its two-factor factor structure was replicated and presented better levels of reliability in most studies, it is slightly superior to the WorkBAT and the WART, so it cannot be concluded that it is the most optimal measure for the measurement of HWI. Thus, further studies with these instruments, based on more robust methodologies, are needed.

About the measurement theory used, the tests have not been tested through the Item Response Theory (IRT) or different models of the Rasch Measurement Theory (RMT), so the performance of the items and the test in people with different levels of HWI is not known. That is, through these models, it is possible to identify at what level of trait of the person, the instruments are more or less reliable since from the CTT reliability is only estimated globally. Moreover, from these two perspectives, it is possible to construct computerized adaptive tests, where workers do not always have to answer the same items and the same number of items, but where the items are adapted to their responses and their trait levels.

Regarding validity, it is necessary to provide evidence of the content of the items. The two studies that were found, and that collected this evidence are more than 20 years old, so the context in which they were evaluated is different from the current one. Likewise, two additional sources of validity have not been studied to date. The evidence of validity is based on response processes, which would allow us to know to what emotions, feelings, thoughts, memories, and behaviors of those evaluated associate when they read the items of the instruments. In this way, we could approximate an explanation that the items are indeed assessing aspects of HWI. In addition, the last source of evidence that has not been considered is that based on the consequences of the application of the test and that involves aspects that go beyond the assessment, such as the actions that could be taken after learning the results in a particular group.

Regarding the reliability of the measures, the results were very variable, finding in many cases values below the minimum acceptable (0.70). However, it is important to note that in no case were the alpha coefficient assumptions corroborated, so there is a possibility that they are underestimated or distorted. Likewise, as in other statistical techniques, it is necessary to consider the nature of the items being worked on, since the WorkBAT, WART, and DUWAS have a Likert-type scale, whose level of measurement is ordinal. Therefore, reliability estimators should be calculated based on a polychoric matrix, which allows a coherent evaluation of the internal consistency of the scores. Conversely, temporal stability has been used in several studies, although methodologically, the best coefficient was not used to see if the scores changed substantially from one assessment to another. In almost all cases, a correlation coefficient was used, which is responsible for seeing whether the scores would be found to be related. However, in these cases, it is suggested to use a coefficient that assesses the concordance (coincidence) of the scores.

Currently, there are systematic reviews of measures of different variables relevant to the organizational field, for example, stakeholder engagement [60] or human resource management systems [61]. However, as mentioned at the beginning of the review, this is the first work that systematically evaluates the psychometric properties of HWI measures. Nevertheless, the study by Andreassen et al. (2013) [13] empirically studied the psychometric properties of the WorkBAT, WART, and DUWAS, through a cross-sectional study where they analyzed the temporal stability and factorial structure of the aforementioned instruments. The study was conducted with 368 Norwegian workers and found that the correlations between the scales were low, showing low convergent evidence, weak temporal stability of scores, and an internal structure for the WorkBAT of four factors, as well as for the WART, while the DUWAS showed a two-factor factorial solution. The authors concluded that the three measures could be used interchangeably. This coincides with the findings of the present review since no outstanding performance of any of the measures was observed over the others.

The present review has some limitations. First, this review only synthesized those articles written in English or Spanish, and articles written in other languages were not involved. Further, three instruments measuring one side of HWI were analyzed, which, although they are not the main ones, are not the only ones, since other tests could also be observed in the selection of the final articles, although in a smaller proportion (for example, the Workaholism Analysis Questionnaire—WAQ or Bergen Work Addiction Scale—BWAS). Moreover, only articles published up to 2020 were analyzed. However, in 2021, one psychometric study has been published that present tools specific to the HWI considering Time Commitment and Work Intensity as dimensions [62]. Thus, the aforementioned limitations should be taken into consideration for future studies, where other instruments are analyzed, that can help to understand this construct that is in full development and boom in terms of research to have a clearer approximation of its functioning.

This systematic review has theoretical and practical implications related to HWI. At a theoretical level, the review of the three instruments allows us to know the different aspects of HWI, specifically workaholism and work addiction, exploring how they are theoretically related to other variables, such as job satisfaction and engagement. In this way, it allows the construction of a nomological network for HWI, where different organizational and mental health constructs are explored. However, the practical implications lie in knowing the strengths and weaknesses of the three measures analyzed to know which versions have the most empirical support and which are still to be studied.

Likewise, the article contributes to the HWI research line, especially in the current context of a health emergency, where teleworking conditions have made the line between personal and professional life increasingly thinner, or in some cases, disappearing. Therefore, it is necessary to have good measurement tools that can capture relevant details of HWI in workers who have high levels of time and effort investment. In this sense, the findings obtained suggest that the DUWAS, the WorkBAT, and the WART can be used to measure HWI. However, the DUWAS presents a slightly better psychometric performance than the other two scales, so it should be considered as a first option although the decision should be linked to other factors such as the objective of the measurement, the components to be explored, and the context of the application.

These findings suggest that HWI has consequences in the organizational field and applied settings (e.g., occupational health or human resources). Moreover, integral to theory development is the ability to differentiate a construct from its antecedents and outcomes. Therefore, developing a thorough understanding of the nature of the HWI phenomenon and its consequences requires the use of sound and appropriate psychometric tools to measure the construct.

6. Conclusions

The present systematic review constitutes the first effort to summarize the psychometric properties of three popular HWI measures (WorkBAT, WART, and DUWAS). In this way, the review provides empirical evidence of the performance of the three instruments so that researchers, psychologists, managers, and other interested parties can choose the measure that best suits their needs and objectives of use. According to the findings obtained, the three HWI measurement instruments have similar performance, so it is advisable to conduct additional psychometric studies focused on other sources of validity evidence, analysis of bias due to certain sociodemographic variables, or with more robust psychometric models (for example, from the IRT or RMT). Finally, based on this review, it is possible to propose new lines of research focused on covering under-explored or unexplored psychometric aspects of the WorkBAT, WART, and DUWAS.