Abstract
The construct validity of self-assessment tools designed to measure adaptive expertise, which is crucial for today’s complex work environments, is evaluated in this study. Although students are still novices and do not yet possess fully developed adaptive expertise, its fostering and assessment should begin during higher education, when future professionals build the foundations of their expertise. Three instruments originally developed for working professionals, the Adaptive Expertise Inventory, the Self-Adapt, and the Adaptability Scale, were examined for their applicability to higher education students. Confirmatory factor analysis revealed strong construct and ecological validity for Self-Adapt and the Adaptability Scale, consistent with previous research. The Adaptive Expertise Inventory showed less stability, with inconsistent factor loadings, potentially related to its prompt or conceptual framework. Exploratory factor analysis indicated no shared factor structures across instruments, suggesting limited conceptual clarity. A flexible approach is recommended to match instruments with program needs, particularly for high-stakes decisions such as advancement. Combining generic, domain-specific, and qualitative tools at the program level may yield deeper insights. These tools should be evaluated using adapted quality criteria to ensure valid and reliable student assessments.
1. Introduction
Adaptive expertise is an increasingly prominent concept, driven by the fact that today’s workplace environments are marked by uncertainty, complexity, and ambiguity (Krawczyńska-Zaucha, 2020). This dynamic landscape necessitates the preparation of future professionals to navigate constant change and evolving situations. All students need to develop as adaptive experts in their careers, specifically by enhancing their ability to manage uncertain and complex work situations from the start (van Tartwijk et al., 2023). Adaptive experts encompass not only efficiency in familiar and routine situations, also known as routine expertise, but also the capacity to employ flexible problem-solving strategies and devise innovative solutions in response to changing or unforeseen circumstances (Hatano & Inagaki, 1986; Ward et al., 2018). Adaptive expertise builds upon routine expertise, and adaptive performance can be seen as the visible expression of adaptive expertise, which is triggered by (contextual) change in tasks or environments (Pelgrim et al., 2022).
There is an increasing focus on how to prepare students to manage the unknown and to overcome wicked problems (Bowman et al., 2024; Groenier et al., 2025; Veltman et al., 2021). Students, as future professionals, are currently novices and have not yet developed expertise. Moreover, they have had limited opportunities to build adaptive expertise and adaptive performance. Therefore, fostering their (self)-reflection and awareness of the components of adaptive expertise and adaptive performance is crucial for their further development. Self-evaluation tools, such as self-evaluation questionnaires, can be particularly valuable in this regard because they can effectively serve a formative function: they help students map their current level of adaptive expertise, assist in setting goals for improving this expertise, and facilitate discussions with their supervisors about these goals. In this way, self-evaluation questionnaires can function as interventions that support the professional growth of students in their work environments (Yan & Carless, 2021). Despite some limitations, these tools are widely recommended for providing an initial understanding of a particular area of interest (Allen & van der Velden, 2005; Gilmore & Feldon, 2010), and in this case, they can offer insights into students’ perceptions regarding adaptive expertise and adaptive performance.
In the literature, several instruments to measure adaptive expertise or adaptive performance have been described. A recent review study on measurement instruments for adaptive expertise and adaptive performance revealed the identification of 19 evaluation instruments, including 12 self-evaluation instruments (Hissink et al., 2025). Two of the self-evaluation tools were specifically designed for students in higher education: in biomedical engineering (Fisher & Peterson, 2001) and biomedical engineering design (Ferguson et al., 2018). However, these instruments have limited evidence of validity and reliability indicators (Hissink et al., 2025). The remaining self-evaluation instruments have been studied and validated with working professionals but not with future professionals (students) as a target group. Therefore, little is known about the quality of these instruments when used with this population. It is unclear whether the underlying constructs to self-evaluate potential aspects of adaptive expertise and adaptive performance are the same or comparable for students as they are for working professionals. Gaining insight into the construct validity of these instruments is therefore essential to determine to what extent the self-evaluation instruments for measuring adaptive expertise and adaptive performance are appropriate and useful for students. This is particularly important in developing instruments applicable across the full continuum of professional development, from students in higher education to experienced professionals. If these instruments prove suitable for all target groups, students could use them to monitor their own development in these areas. This could facilitate both students and their educators in fostering awareness and reflection about adaptive expertise and adaptive performance, and enhancing self-reflection and awareness related to this concept.
2. Research Question
The aim of this study is to evaluate the quality of self-evaluation instruments designed to measure the adaptive expertise and adaptive performance when used with higher education students instead of working professionals. This leads to the following research question:
What is the construct validity of the self-evaluation measurement instruments to measure adaptive expertise and adaptive performance of higher education students?
3. Method
3.1. Study Design
The study is evaluative in nature and examines the construct validity of measures of adaptive expertise and adaptive performance among Dutch higher education students, including students from Universities of Applied Science and Research Universities. This evaluation is denoted as T1 (time 1). A subset of students participated in the second measurement (T2), to verify the initial findings. Measuring students’ adaptive expertise and/or performance was part of Adapt at Work (2024), a design-based research project on the impact of work-based learning interventions on adaptive expertise development of higher education students (Adapt at Work, 2024; Nieuwenhuis & Fluit, 2019).
3.2. Instrumentation
The selection of measurement instruments in this study is based on an extensive overview of suitable measurement instruments to measure adaptive expertise and adaptive performance (Hissink et al., 2025). In their study, 19 instruments were identified. To specify the selection of instruments, the following inclusion criteria were developed for this study:
- The instrument is a self-evaluation tool;
- All items of the instrument are accessible, either within the article, online, or through another publication;
- The instrument is broadly applicable in different professional domains and work contexts, and not limited to one or a few domains;
- The instrument has moderate or strong evidence of validity, reliability, and fairness in testing (according to the Hissink et al. overview);
- The instrument is available in English.
The 19 instruments were independently reviewed by EH (main researcher, educational specialist, dental education and healthcare education) and LN (supervisor, emeritus professor, professional and vocational education). Of the 19 instruments, 14 did not meet the inclusion criteria. Of the 19 instruments, 7 were not self-evaluation instruments. Of the 12 self-evaluation instruments, 1 instrument contained items that are not accessible, 1 instrument was not available in English, 1 instrument was not broadly applicable, and 4 instruments did not have moderate or strong evidence of validity, reliability, and/or fairness in testing. This resulted in five instruments that matched the criteria. Three of these five can be used in a wide range of workplace settings and are easily available in both English and Dutch. Therefore, these three instruments were selected, resulting in three usable instruments:
- The Adaptive Expertise Inventory (Bohle-Carbonell et al., 2016);
- The Self-Adapt (Oprins et al., 2018);
- The Adaptability Scale (van Dam & Meulders, 2020).
All three instruments are well grounded in the international literature. A Dutch version of the Self-Adapt and the Adaptability Scale had been previously analysed and described (Oprins et al., 2018; van Dam & Meulders, 2020) and a Dutch version of the Adaptive Expertise Inventory was received from the authors of this instrument (Bohle-Carbonell et al., 2016). Each instrument is briefly outlined below, including the factors of the instrument and a sample question.
3.2.1. The Adaptive Expertise Inventory (Bohle-Carbonell et al., 2016)
The Adaptive Expertise Inventory measures self-reported adaptive expertise in a wide range of workplace settings (mostly professional, scientific, technical, and educational areas). The instrument is centred on adaptive knowledge and expertise within the work domain. It discerns (1) domain-specific and (2) innovative adaptivity, which are empirically supported as two factors of adaptive expertise (Bohle-Carbonell et al., 2016).
According to Bohle-Carbonell et al., adaptive expertise is built on routine expertise as both forms of expertise contain the ability to perform standard tasks in the domain without errors. The difference only becomes apparent when confronted with a non-standard situation: individuals with adaptive expertise possess a more extensive and integrated knowledge base than do individuals with routine expertise (Hatano & Inagaki, 1986). This helps them to determine when not to rely on their automatic processes; when this happens, they can “slow down” and make conscious efforts to deal with the problem. It follows that they abandon skill- and rule-based decision-making and spend time building a mental model of the situation, in which they draw analogies between standard and novel situations (Barnett & Koslowski, 2002; Chi, 2011). According to Bohle-Carbonell, adaptive expertise is a developmental process that is being propelled by problem-solving skills (Tynjälä et al., 1997). Consequently, characteristics of traditional expertise research (e.g., repeated high performance, standardized tasks) play a lesser role compared to the need for non-standard but realistic tasks, which elicit the problem-solving skills of individuals with adaptive expertise. Table 1 shows the factors of the Adaptive Expertise Inventory, including sample items.
Table 1.
Factors and sample questions of the Adaptive Expertise Inventory.
The internal consistency of the subscales is acceptable across populations, with α = 0.79 for professionals and 0.79 for graduates on the domain skills subscale, and α = 0.78 for professionals and 0.74 for graduates on the innovative skills subscale, respectively (Bohle-Carbonell et al., 2016).
3.2.2. The Self-Adapt (Oprins et al., 2018)
The Self-Adapt is the second part of the Dutch Adaptability Dimensions And Performance Test (D-ADAPT). The D-ADAPT is used in the military and civil domain, and is twofold: respondents are asked to describe job requirements (tWork-Adapt scale) and to evaluate their own capabilities to deal with these requirements (tSelf-Adapt scale), using the same items with a different prompt. The instrument measures six dimensions of adaptability: (1) solving difficult problems; (2) handling crisis situations; (3) culturally demanding situations; (4) physically demanding circumstances; (5) handling work stress; and (6) interpersonal interactions. These six dimensions are confirmed through confirmatory factor analysis, both for the Work-Adapt scale and the Self-Adapt scale (Oprins et al., 2018). In this study only the Self-Adapt is used.
Oprins et al. (2018) view adaptability as a competency of individuals, partly determined by innate personal qualities and partly by acquired skills. Adaptability is considered a competency that can, to a certain extent, be improved by training and experience (Pulakos et al., 2000). Insight in this competency can contribute to the selection and training of employees for a specific job. Adaptability is seen as multidimensional competency (based on the dimensions of Pulakos et al., 2002) and as a situation-specific competency (J. M. Campbell & McCord, 1999; J. P. Campbell et al., 1993; Hesketh & Neal, 1999; Pulakos et al., 2000). Table 2 shows the factors of the Self-Adapt, including sample items.
Table 2.
Factors and sample questions of the Self-Adapt.
The internal consistency of the Self-Adapt dimensions varies across subscales, with Cronbach’s α values ranging from 0.68 to 0.95, indicating generally acceptable to excellent estimated reliability across the different adaptability dimensions (Oprins et al., 2018).
3.2.3. The Adaptability Scale (van Dam & Meulders, 2020)
The Adaptability Scale is used to assess self-reported employee adaptability in workplace settings. The instrument is based on organizational development and focuses on adaptive traits separated in cognitive, behavioural, and affective adaptability. The Adaptability Scale assesses employee adaptability as an individual difference construct. The individual difference construct approach views employee adaptability as a set of underlying characteristics that allow individuals to be effective under changing task conditions (Jundt et al., 2015). This approach assumes that there is a certain stability and generalizability in how individuals relate to changing, uncertain, and novel situations (Baard et al., 2013) following Ployhart and Bliese’s (2006) conceptualization of employee adaptability as individuals’ tendency to be flexible, open-minded, resilient, and ready to actively change or fit novel, changing, or ambiguous work environments. This conceptualization captures three key elements of adaptability: (1) adaptability is an individual characteristic; (2) adaptation can have both proactive and reactive components; and (3) adaptability should be considered as a state-like capacity. These three key elements show up as stable factors in a confirmatory factor analysis (van Dam & Meulders, 2020). Table 3 shows the factors of the Adaptability Scale, including sample items.
Table 3.
Factors and sample questions of the Adaptability Scale.
The internal consistency of the Adaptability Scale is good, with McDonald’s ω values around 0.85 across samples, indicating acceptable reliability (van Dam & Meulders, 2020).
3.3. Participants
Dutch higher education students from five Dutch Research Universities and six Dutch Universities of Applied Sciences participated in the study. The research was conducted in accordance with ethical guidelines, and all students provided informed consent. Data were pseudonymized prior to analysis and securely stored in the Research Drive. Ethical approval was obtained from the Ethical Review Board of the HAN University of Applied Sciences (approval number 312.12/21, Nijmegen, The Netherlands).
The universities were selected because of their attention to adaptive expertise development. They all presented an operational case, consisting of an alleged innovative work-based learning module or intervention to enhance the development of students’ adaptive expertise, by open-ended tasks in a work-based learning environment. The majority of the students were full-time students (with some being part-time students) and were between 18 and 26 years old. About two-thirds of the students were pursuing a bachelor’s degree, while one-third were pursuing a master’s program at either one of the Research Universities or one of the Universities of Applied Sciences.
All students that participated in the modules were administered a questionnaire. At the beginning of the module (T1), nearly the entire population of the students completed the questionnaire (506 students). At the end of the module (T2), the level of effort to enhance student response varied across universities. As a result, 179 students participated at both T1 and at T2, accounting for 35% of the initial participants.
3.4. Data Collection
In the second semester of 2022 or the first semester of 2023, both at the beginning and upon completion of the learning module, students were administered a questionnaire that included all items from the three measurement instruments: the Adaptive Expertise Inventory, the Self-Adapt, and the Adaptability Scale. The Dutch versions of the instruments were used at nine universities, while the English versions were used at two universities due to the presence of international student groups. In most universities, students completed the questionnaire as a classroom activity. The timing of the post-test administration depended on the length of the modules, which ranged from 6 to 24 weeks. Due to COVID regulations, the post-test was administered only online (see Table 4 for the response count).
Table 4.
Response count.
The data that support the findings of this study are openly available in are digitally stored on the secured and protected R-station of the Department of Responsive Vocational and Professional Education of HAN University of Applied Sciences.
3.5. Data Analysis
The consistency of the instruments was tested using confirmatory factor analysis (CFA) and exploratory factor analysis (EFA). The main analysis was conducted using the data from T1. An additional examination was conducted on the data from T2, acknowledging that the response rate was only 35%. In the analysis, the data from T1 and T2 were strictly separated, with the analysis of the T2 data serving to validate the results from the T1 data and for descriptive purposes.
The first analysis consisted of applying a CFA to the scores of all students in T1 and T2 for each separate instrument. In the second analysis, data from all three instruments were combined for a joint EFA to explore their communalities.
4. Results
4.1. Confirmation of the Identified Factors of the Three Instruments
A CFA was conducted based on the number of factors reported by the authors of the three instruments (though not necessarily adhering strictly to their recommendations), as follows:
- The Adaptive Expertise Inventory: 2 factors;
- The Self-Adapt: 6 factors;
- The Adaptability Scale: 3 factors.
For each instrument, the presentation of the results of the CFAs consists of four parts. Each CFA began with an exploratory analysis to determine the number of factors (Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11, Table 12 and Table 13).
Table 5.
Eigenvalues (T1 and T2) and explained variance (T1) of the factors of the Adaptive Expertise Inventory.
Table 6.
Factor loadings of the Adaptive Expertise Inventory—T1 and T2.
Table 7.
Eigenvalues (T1 and T2) and explained variance (T1) of the factors of the Self-Adapt.
Table 8.
Factor loadings of the Self-Adapt—T1 and T2.
Table 9.
Eigenvalues (T1 and T2) and explained variance (T1) of the factors of the Adaptability Scale.
Table 10.
Factor loadings of the Adaptability Scale—T1 and T2.
Table 11.
Eigenvalues (T1 and T2) and explained variance (T1) of measures of the three instruments combined.
Table 12.
Factor loadings of the three instruments combined—T1.
Table 13.
Factor loadings of the three instruments combined—T2.
- Eigenvalues: First, Table 5, Table 7, Table 9, and Table 11 present the eigenvalues. According to the Kaiser criterion, factors should have eigenvalues of 1.0 or higher. The last factor meeting this criterion is highlighted in bold in each table, indicating the number of factors to be used in the factor analysis.
- Explained variance: In the tables with the eigenvalues, the explained variance for T1 is also reported, as it provides an indication of the contribution of each factor to the total variance, as well as the relative strength of the factors. A cumulative explained variance of 60% is considered the threshold criterion.
- Scree plots: Next, the scree plots are displayed (Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8). A scree plot visualizes the eigenvalues and helps identify the number of factors for the factor analysis.Figure 1. Scree plot of the Adaptive Expertise Inventory—T1.Figure 2. Scree plot of the Adaptive Expertise Inventory—T2.Figure 3. Scree plot of the Self-Adapt—T1.Figure 4. Scree plot of Self-Adapt—T2.Figure 5. Scree plot of the Adaptability Scale—T1.Figure 6. Scree plot of the Adaptability Scale—T2.Figure 7. Scree plot of the three instruments combined—T1.Figure 8. Scree plot of the three instruments combined—T2.
- Factor loadings: Finally, the tables with the factor loadings are presented (Table 6, Table 8, Table 10, Table 12, and Table 13). Loadings below −0.4 or above 0.4 are highlighted in bold. If the CFA demonstrates that the factors are distinct from one another, there should be a clear clustering of items, with each factor forming a separate group.
4.2. The Adaptive Expertise Inventory of Bohle-Carbonell et al. (2016) (Table 5, Table 6, Figure 1 and Figure 2)
At T1, the Kaiser criterion indicates two factors, while at T2, three factors are indicated. The explained variance (T1) is over 53% for two factors, and 61% for three factors. The scree plots clearly indicates a knee at two factors. Based on the eigenvalues at T1 and on substantive reasoning, a two-factor analysis was selected, as the analyses by Bohle-Carbonell also identified two factors (Bohle-Carbonell et al., 2016).
The results of the CFA for the Adaptive Expertise Inventory differ from those reported by Bohle-Carbonell et al. (2016). A clear separation of DS and IS in two distinct factors was anticipated. However, at T1, several items expected to load onto factor 2 instead load onto factor 1. At T2, only four items load onto factor 1, with more items loading onto factor 2.
4.3. The Self-Adapt of Oprins et al. (2018) (Table 7, Table 8, Figure 2 and Figure 3)
At both T1 and T2, the Kaiser criterion and the scree plot criterion indicate six factors for the Self-Adapt. The explained variance (T1) is almost 60%. Therefore, CFA was conducted with six factors.
The results of the CFA for the Self-Adapt indicate that the factor structure reported by Oprins et al. (2018) closely matches the findings of the current factor analysis. At T1, all items are assigned to the same factors with one exception (item STS1). At T2, the vast majority of items remain assigned to the same factors, though five items exhibit lower factor loadings. Overall, the Self-Adapt scale demonstrates a stable factor structure.
4.4. The Adaptability Scale of van Dam and Meulders (2020) (Table 9, Table 10, Figure 5 and Figure 6)
Both the Kaiser criterion and the scree plot criterion indicate the number of three factors for the Adaptability Scale. The explained variance (T1) is more than 60%. Therefore, CFA was executed with three factors.
The results of the CFA for the Adaptability Scale indicate that the factor structure reported by van Dam and Meulders (2020) generally aligns with the findings of the current factor analysis. At T1, all items load onto the reported factors; however, not all items have high enough loadings to be assigned definitively to a single factor (items A1 and C1 both show small, double loadings on two factors, although they do not meet the cut-off point of >0.4 or <-0.4. Additionally, the difference between the items is more than 0.2. At T2, these two items load onto a different factor (Behavioural). Overall, the Adaptability Scale demonstrates a stable factor structure.
4.5. Convergence in the Factor Structure Among the Three Instruments (Table 11, Table 12, Table 13, Figure 7 and Figure 8)
An EFA combining the three instruments was conducted to investigate if the instruments share communalities in their factor structures. This was done by including all items of the three instruments in a single factor analysis, once with T1 data and once with T2 data.
At T1, the Kaiser criterion indicates eleven factors, while at T2, twelve factors are suggested. The explained variance (T1) is over 60% for two factors. The scree plot criterion indicates eleven factors at both T1 and T2. Since the total number of factors for the three instruments, based on the authors’ articles, amounts to eleven, an eleven-factor analysis was chosen.
The results of the EFA indicate a total number of eleven factors. However, the factor structure slightly differs from the (combined) factor structure of the three separate instruments. A detailed analysis of the results first shows that the three instruments do not merge in the overall EFA. The items of the three instruments load on different factors and do not show communalities. Most of the items adhere to the same factors as in the individual CFA analysis.
Regarding the Adaptive Expertise Inventory, we found that at T1, the overall EFA results for the Adaptive Expertise Inventory suggested a one-factor structure for this instrument, which also turned out to be a possibility in the individual CFA. At T2, a three-factor structure was suggested. Within this structure, the DS factor remained relatively intact, but the IS factor did not, as the remaining items are distributed across two factors.
The Self-Adapt shows at T1 that four factors of the Self-Adapt (CRS, CUS, IPS, and PDC) remain stable, consistent with the individual CFA analyses. The items load onto to the same factors and exhibit similar factor loadings. However, two factors (CPS and STS) in the overall EFA differ from the CFA. The CPS factor loses two items, and the STS factor splits into two smaller factors. When a 10-factor structure is applied in a CFA, the STS items merge back into one single factor without affecting the main results of the other factors. At T2, five factors of the Self-Adapt (CPS; CRS; CUS; STS; and PDC) remain stable, similar to the individual CFA analyses. The items adhere to the same factors and show approximately the same factor loadings. However, one factor (IPS) in the overall EFA differs from the CFA; this factor loses one item to another factor, and one item does not load onto any factor. Overall, a six-factor structure appears to be the best fit for the Self-Adapt, although some items either do not load onto any factor or load onto a different factor than expected.
For the Adaptability Scale, we found at T1 that the factor structure was stable and aligned with the individual analyses, with the affective factor becoming even stronger, as item A1 reassigns to this factor in the overall EFA. At T2, a three-factor structure is also suggested. However, two items from the cognitive factor load onto the behavioural factor, and only one item remains loading onto the cognitive factor.
5. Discussion
5.1. Theoretical Considerations
Adaptive expertise is crucial in today’s complex work environment. Although students are still novices and do not yet possess ample expertise, developing adaptive expertise and the resulting adaptive performance is essential for the execution of their future profession. In higher education, considerable attention is therefore already being given to its development (Mylopoulos et al., 2022). Intrapersonal, interpersonal, and organizational interactions can contribute to the development of adaptive expertise, with the practice of reflection playing a crucial role in this process (Kua et al., 2022). Self-evaluation tools are valuable for this as self-reflection enhances students’ learning achievements (Yan & Carless, 2021). To assess the adaptive expertise of students, various instruments are available, most of which are self-evaluation instruments. Though it is important to have instruments that can be applied across the entire professional trajectory, from students in higher education to practicing professionals, most instruments of sufficient quality have never been tested with students. Therefore, the aim of this study was to evaluate the construct validity of each of the self-evaluation measurement instruments—the Adaptive Expertise Inventory, the Self-Adapt, and the Adaptability Scale—with the target group of students. Part of this involved investigating the convergence in the factor structure, which had not been previously examined in either a working population or a student population.
We first evaluated the consistency of the individual instruments to determine the stability of the factors indicated in the original instruments, using CFA. The results of the Self-Adapt and the Adaptability Scale align closely with previous findings (Oprins et al., 2018; van Dam & Meulders, 2020) and we found roughly the same factor structure as reported before. These instruments demonstrate good construct validity. Additionally, since the instruments have been used in ‘real-life’ settings outside controlled conditions, they also exhibit good ecological validity. This makes the instruments suitable for both the student population and working professionals. Bohle-Carbonell et al.’s Adaptive Expertise Inventory showed less stability than reported before (Bohle-Carbonell et al., 2016). At T1, several items expected to load onto factor 2 instead load onto factor 1. At T2, only four items load onto factor 1, with more items loading onto factor 2. We can therefore conclude that, although two factors are found, the distribution of the items at both T1 and T2 is not consistent with previous results. One possible explanation might be conceptual: the distinction between the two factors (Domain Skills and Innovativeness) may not be clear enough, or the questionnaire items might be too similar. Another explanation could be related to the prompt of the questionnaire. The prompt “In the past year…” addresses a different context than the prompts of the other questionnaires, namely, “How effective do you consider yourself at performing this behaviour in your work?” (the Self-Adapt) and “To what extent do you agree with the following statements?” (the Adaptability Scale). The Adaptive Expertise Inventory might not explicitly address non-standard situations sufficiently, although conceptually, according to Bohle-Carbonell et al., this is the key element of (developing) adaptive expertise. The prompt, however, does not explicitly indicate that respondents need to reflect on non-standard situations. It might also be possible that students struggle more with the prompt “In the past year…” because they have less (work) experience and therefore have fewer experiences to draw upon that relate to adaptive expertise. Finally, Bohle-Carbonell et al. initially proposed three factors (Metacognitive Skills in addition to Domain Skills and Innovativeness). However, their factor analysis revealed only the two latter factors Domain Skills and Innovativeness. This, combined with our findings, might suggest that the instrument is based on a construct that is less precisely defined.
Administering all three instruments also provided an opportunity to examine the stability of the factor structure of the instruments together, and to determine whether they measure the same concept. The main conclusion of the combined EFA analysis is that the three instruments do not show any communalities. On the one hand, this is a remarkable finding because all instruments pretend to measure adaptive expertise or adaptive performance (the latter being the visible expression of adaptive expertise). If they were measuring the same concept, an overall analysis would show communalities with fewer factors in the factor structures. On the other hand, as Pelgrim et al. (2022) and Cupido et al. (2022) already argued, a lack of conceptual clarity exists regarding the precise or even comparable definitions of the adaptive expertise and adaptive performance. Contextual changes in tasks or environments, as well as factors like uncertainty, complexity, and ambiguity, trigger adaptive expertise and adaptive performance. However, the specific behaviours that define these concepts vary among authors and depend on the conceptualizations and operationalizations they use to develop their instruments. This is also evident in the names of the factors of the three instruments: the specific behaviours associated with adaptive expertise or adaptive performance, according to the authors, depend on the (theoretical) construct and the way in which adaptive expertise or adaptive performance is conceptualized and operationalized. This reinforces the conclusion presented by Hissink et al. (2025) in their article: the origins of the instruments exhibit considerable diversity, stemming from various academic disciplines (particularly HRM and education), each characterized by distinct conceptualizations, elaborations, and observable behaviours. The instruments stem from different conceptual islands and we did not find any bridging factors in this archipelago.
5.2. Practical Considerations
It is crucial for those using the instruments to be thoroughly informed about the different conceptualizations and operationalizations. We therefore propose an approach that can yield different outcomes for various faculties and programs. Validity, alongside reliability and fairness in testing, is one of the principal quality criteria for assessment instruments (American Educational Research Association et al., 2014). All three instruments claim to measure adaptive expertise or adaptive performance; however, the factor analyses indicate that the factors of the individual tests largely remain distinct. Different authors conceptualize and operationalize the constructs differently and a user of any of the instruments should consider what exactly needs to be (self) measured. It is important for users of the instruments to carefully consider the principles of constructive alignment to determine which instrument best fits a faculty or educational program. This ensures a logical connection, as a good education system aligns teaching methods and assessments with the learning activities outlined in the objectives (Loughlin et al., 2020). This is especially relevant when important (summative) decisions are based on one of the instruments (for instance, passing a learning module or advancing to the next level within a program).
5.3. Recommendations for Future Research
Finally, at the program level, it is worthwhile to consider instruments other than generic self-evaluation tools, as doing so can provide valuable insights and more nuanced evaluations. At the program level, measurement does not need to span across domains; instead, the focus can be on the development of adaptive expertise and adaptive performance within a single domain or even a specific profession. In addition to using a generic instrument, program-specific tools, which are often more qualitative in nature, can be employed. Examples of such tools include design scenarios (Walker et al., 2006) or mixed-method evaluations (Yoon et al., 2019; Suh et al., 2023). However, these instruments should be evaluated using adapted quality criteria, which requires further research, as well as the appropriate combination of several instruments to make qualitatively valid and reliable statements about student progress.
6. Conclusions
Building on the discussion above, the following conclusions highlight the main theoretical, methodological, and practical implications of our findings.
This study contributes to the field of adaptive expertise by demonstrating that commonly used self-evaluation instruments (the Adaptive Expertise Inventory, the Self-Adapt, and the Adaptability Scale) show distinct factor structures and limited overlap. Theoretically, this underscores the need for greater conceptual clarity in defining adaptive expertise and adaptive performance, as the operationalizations of these constructs vary across instruments and disciplines. Methodologically, this study provides evidence on the construct validity of these instruments in a student population, extending their applicability beyond working professionals. Practically, educators and program designers are encouraged to carefully consider which instrument aligns with the learning objectives and context of their programs, ensuring constructive alignment between teaching, learning activities, and assessment. Finally, future research could focus on developing domain-specific or mixed-method assessments that capture more nuanced aspects of adaptive expertise and performance, as well as investigating how these instruments can best support longitudinal development in educational and professional trajectories.
Author Contributions
Conceptualization: E.H., L.N.; Methodology: E.H., T.D.L., M.v.d.S., L.N.; Formal analysis: T.D.L.; Investigation: M.P.; Writing—original draft preparation: E.H.; Writing—review and editing: M.v.d.S., L.N.; Supervision: M.v.d.S., L.N.; Funding acquisition: L.N. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported by the Dutch Nationaal Regieorgaan Onderwijsonderzoek, grant number 40.5.19945.601.
Institutional Review Board Statement
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethical Review Board of the HAN University of Applied Sciences (protocol code ECO 312.12/21 and date of approval 23 December 2021).
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data presented in this study are openly available in R-station of the Department of Responsive Vocational and Professional Education of HAN University of Applied Sciences (https://doi.org/10.17026/SS/PVQ66Y; accessed on 11 January 2026).
Acknowledgments
This study is part of the This study is part of the Adapt at work project: a 5-year research project (July 2019–July 2024) that was in initiated in the Netherlands, involving a consortium of 6 universities of applied sciences and 5 research universities. The project centers around the development of adaptive expertise in work-based learning contexts within higher education.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Adapt at Work. (2024). Available online: https://adaptatwork.nl/en/ (accessed on 11 January 2026).
- Allen, J. P., & van der Velden, R. K. W. (2005). The role of self-assessment in measuring skills. In REFLEX working paper series No. 2. ROA external reports. ROA. [Google Scholar]
- American Educational Research Association, American Psychological Association & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association. [Google Scholar]
- Baard, S. K., Rench, T. A., & Kozlowski, S. W. J. (2013). Performance adaptation: A theoretical integration and review. Journal of Management, 40, 48–99. [Google Scholar] [CrossRef] [Scilit]
- Barnett, S. M., & Koslowski, B. (2002). Adaptive expertise: Effects of type of experience and the level of theoretical understanding it generates. Thinking & Reasoning, 8(4), 237–267. [Google Scholar] [CrossRef] [Scilit]
- Bohle-Carbonell, K., Könings, K. D., Segers, M., & van Merriënboer, J. J. G. (2016). Measuring adaptive expertise: Development and validation of an instrument. European Journal of Work and Organizational Psychology, 25(2), 167–180. [Google Scholar] [CrossRef] [Scilit]
- Bowman, S., Salter, J., Stephenson, C., & Humble, D. (2024). Metamodern sensibilities: Toward a pedagogical framework for a wicked world. Teaching in Higher Education, 29(5), 1361–1380. [Google Scholar] [CrossRef] [Scilit]
- Campbell, J. M., & McCord, D. M. (1999). Measuring social competence with the Wechsler picture arrangement and comprehension subtests. Assessment, 6(3), 215–223. [Google Scholar] [CrossRef] [Scilit]
- Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of performance. In N. Schmitt, & W. Borman (Eds.), Personnel selection in organizations (pp. 35–70). Jossey-Bass. [Google Scholar]
- Chi, M. (2011). Theoretical perspectives, methodological approaches, and trends in the study of expertise. In Y. Li, & G. Kaiser (Eds.), Expertise in mathematics instruction: An international perspective (pp. 17–39). Springer. [Google Scholar]
- Cupido, N., Ross, S., Lawrence, K., Bethune, C., Fowler, N., Hess, B., van der Goes, T., & Schultz, K. (2022). Making sense of adaptive expertise for frontline clinical educators: A scoping review of definitions and strategies. Advances in Health Sciences Education, 27(5), 1213–1243. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ferguson, J. H., Lehmann, J., Zastavker, Y. V., Chang, S., Higginson, R. P., & Talgar, C. P. (2018, June 23–July 27). Adaptive expertise: The development of a measurement instrument. 2018 ASEE Annual Conference & Exposition, Salt Lake City, UT, USA. [Google Scholar] [CrossRef] [Scilit]
- Fisher, F. T., & Peterson, P. L. (2001, June 24–27). A tool to measure adaptive expertise in biomedical engineering students. ASEE Annual Conference Proceedings (pp. 1249–1263), Albuquerque, NM, USA. [Google Scholar]
- Gilmore, J., & Feldon, D. F. (2010, April 30–May 4). Measuring graduate students’ teaching and research skills through self-report: Descriptive findings and validity. Annual Meeting of the American Educational Research Association, Denver, CO, USA. [Google Scholar]
- Groenier, M., Khaled, A., Kamphorst, B. A., de Jong, T., & Verdonschot, S. G. M. (2025). Becoming an adaptive expert through work-based learning: A realist review. Higher Education Research & Development, 44(1), 1–16. [Google Scholar] [CrossRef] [Scilit]
- Hatano, G., & Inagaki, K. (1986). Two courses of expertise. In H. Stevenson, H. Azmuma, & K. Hakuta (Eds.), Child development and education in Japan. W.Y. Freeman and Co. [Google Scholar]
- Hesketh, B., & Neal, A. (1999). Technology and performance. In D. R. Ilgen, & D. P. Pulakos (Eds.), The changing nature of performance: Implications for staffing, motivation and development (pp. 21–55). Jossey-Bass. [Google Scholar]
- Hissink, E., Pelgrim, E. A. M., Nieuwenhuis, A. F. M., Bus, L., Kuijer-Siebelink, W., & Van Der Schaaf, M. (2025). Measuring adaptive expertise in (becoming) healthcare professionals—A review of measurement instruments and their quality. Advances in Health Sciences Education, 1–27. [Google Scholar] [CrossRef] [Scilit]
- Jundt, D. K., Shoss, M. K., & Huang, J. L. (2015). Individual adaptive performance in organizations: A review. Journal of Organizational Behavior, 36, S53–S71. [Google Scholar] [CrossRef] [Scilit]
- Krawczyńska-Zaucha, T. (2020). What is the philosophy of education needed in the XXI century? Culture Society Education, 18(2), 467–478. [Google Scholar] [CrossRef] [Scilit]
- Kua, J., Teo, W., & Lim, W. S. (2022). Learning experiences of adaptive experts: A reflexive thematic analysis. Advances in Health Sciences Education, 27(5), 1345–1359. [Google Scholar] [CrossRef] [Scilit] [PubMed] [PubMed Central]
- Loughlin, C., Lygo-Baker, S., & Lindberg-Sand, Å. (2020). Reclaiming constructive alignment. European Journal of Higher Education, 11(2), 119–136. [Google Scholar] [CrossRef] [Scilit]
- Mylopoulos, M., Dolmans, D. H. J. M., & Woods, N. N. (2022). The imperative for (and opportunities of) research on adaptive expertise in health professions education. Advances in Health Sciences Education, 27(5), 1207–1212. [Google Scholar] [CrossRef] [Scilit]
- Nieuwenhuis, A. F. M., & Fluit, C. R. M. G. (2019). Flexible higher education: Lifelong professional competence development in work-based settings [Essay]. Netherlands Organization for Research (NRO). [Google Scholar]
- Oprins, E. A. P. B., van den Bosch, K., & Venrooij, W. (2018). Measuring adaptability demands of jobs and the adaptability of military and civilians. Military Psychology, 30(6), 576–589. [Google Scholar] [CrossRef] [Scilit]
- Pelgrim, E., Hissink, E., Bus, L., van der Schaaf, M., Nieuwenhuis, L., van Tartwijk, J., & Kuijer-Siebelink, W. (2022). Professionals’ adaptive expertise and adaptive performance in educational and workplace settings: An overview of reviews. Advances in Health Sciences Education, 27(5), 1245–1263. [Google Scholar] [CrossRef] [Scilit]
- Ployhart, R. E., & Bliese, P. D. (2006). Individual adaptability (I-ADAPT) theory: Conceptualizing the antecedents, consequences, and measurement of individual differences in adaptability. In C. S. Burke, L. G. Pierce, & E. Salas (Eds.), Understanding adaptability: A prerequisite for effective performance within complex environments (pp. 3–39). Elsevier. [Google Scholar] [CrossRef] [Scilit]
- Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. (2000). Adaptability in the workplace: Development of a taxonomy of adaptive performance. Journal of Applied Psychology, 85(4), 612–624. [Google Scholar] [CrossRef] [Scilit]
- Pulakos, E. D., Schmitt, N., Dorsey, D. W., Arad, S., Borman, W. C., & Hedge, J. W. (2002). Predicting adaptive performance: Further tests of a model of adaptability. Human Performance, 15(4), 299–323. [Google Scholar] [CrossRef] [Scilit]
- Suh, J. K., Hand, B., Dursun, J. E., Lammert, C., & Fulmer, G. (2023). Characterizing adaptive teaching expertise: Teacher profiles based on epistemic orientation and knowledge of epistemic tools. Science Education, 107(4), 884–911. [Google Scholar] [CrossRef] [Scilit]
- Tynjälä, P., Nuutinen, A., Eteläpelto, A., Kirjonen, J., & Remes, P. (1997). The acquisition of professional expertise—A challenge for educational research. Scandinavian Journal of Educational Research, 41, 475–494. [Google Scholar] [CrossRef] [Scilit]
- van Dam, K., & Meulders, M. (2020). The adaptability scale: Development, internal consistency, and initial validity evidence. European Journal of Psychological Assessment, 37(2), 123–134. [Google Scholar] [CrossRef] [Scilit]
- van Tartwijk, J., van Dijk, E. E., Geertsema, J., Kluijtmans, M., & van der Schaaf, M. (2023). Teacher expertise and how it develops during teachers’ professional lives. In R. J. Tierney, F. Rizvi, & K. Erkican (Eds.), International encyclopedia of education (5th ed., pp. 170–179). Elsevier. [Google Scholar]
- Veltman, M. E., van Keulen, H., & Voogt, J. M. (2021). Using problems with wicked tendencies as vehicles for learning in higher professional education: Towards coherent curriculum design. The Curriculum Journal, 32(4), 710–728. [Google Scholar] [CrossRef] [Scilit]
- Walker, J. M. T., King, P. H., & Brophy, S. P. (2006). Design scenarios as an assessment of adaptive expertise. International Journal of Engineering Science, 22(3), 645–651. [Google Scholar]
- Ward, P., Schraagen, J. M., Gore, J., & Roth, E. M. (2018). An introduction to the handbook, communities of practice, and definitions of expertise. In P. Ward, J. M. Schraagen, J. Gore, & E. M. Roth (Eds.), The oxford handbook of expertise. Oxford University Press. [Google Scholar]
- Yan, Z., & Carless, D. (2021). Self-assessment is about more than self: The enabling role of feedback literacy. Assessment & Evaluation in Higher Education, 47(7), 1116–1128. [Google Scholar] [CrossRef] [Scilit]
- Yoon, S. A., Evans, C., Miller, K., Anderson, E., & Koehler, J. (2019). Validating a model for assessing science teacher’s adaptive expertise with computer-supported complex systems curricula and its relationship to student learning outcomes. Journal of Science Teacher Education, 30(8), 890–905. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.







