Invisible Paradigms: A Critical Realist Analysis of Ontological, Epistemological, and Axiological Positioning in Three Engineering Education Research Journals

Highlights

Please indicate how your work links to systems science via your contributions to systems practice, theory, and/or methodology.

• Introduces a critical realist framework that treats philosophical positioning as important in shaping the kind of knowledge a sociotechnical systems study produces. This extends systems thinking to the level of research design.
• Offers a transferable coding methodology for philosophical infrastructure across a research field, applicable beyond engineering education to other sociotechnical systems literature.

What are the main findings and/or the implications of the main findings?

• Across 54 papers from three Q1 journals, ontological, epistemological, and axiological positions were overwhelmingly undeclared (46/54), but were reconstructible, thus indicating that these positions function as real, consequential mechanisms whether named or not.
• Nondeclaration is a field-level norm rather than individual oversight. Making OEA positioning explicit would render the field’s philosophical plurality legible to readers and practitioners without establishing a hierarchy among paradigms.

Abstract

Engineering education research (EER) draws on a wide range of philosophical traditions, yet the ontological, epistemological, and axiological (OEA) commitments that shape how knowledge is produced are rarely made explicit in published work. This study investigated the prevalence and nature of OEA positioning across a purposive sample of 54 papers published in 2024 in three Q1 engineering education journals: the Journal of Engineering Education, the European Journal of Engineering Education, and the Australasian Journal of Engineering Education. Using critical realism as a metatheoretical framework, we developed an OEA coding instrument and applied it through an AI-assisted abductive coding process, assigning ontological, epistemological, and two-level axiological codes to each paper and assessing their internal coherence. The overwhelming majority of papers carry implicit rather than declared OEA commitments, a pattern consistent across journals and methodologies. The field is genuinely philosophically plural, with ontological positions ranging from naïve realism to social constructionism and critical realism, but the lack of clarity in this space potentially has consequences for knowledge transfer to practice, cumulative knowledge-building, and the coherence of individual studies.

1. Introduction

The call for this special issue draws together scholars who have advanced the crucial insight that engineering systems are not only technical systems, but also inherently sociotechnical—involving complex interactions between multiple human stakeholders and deliberations on potential technical solutions to societal challenges. For engineering education scholars, often trained in these educational systems, they now challenge its narrow focus on technical rationality. This shift has also involved questioning the choice of research approaches. To this end, engineering education research (EER) has a long tradition of grappling with approaches to research and looking to expand its repertoire. The present paper is focused on the ongoing need for EER to look not only outwards to the phenomena it researches (engineering, educational systems, students, teachers, and curricula) but also inwards to how it positions itself as researchers and how it conducts and reports on research.

The field of engineering education research (EER) started coalescing around an emerging and increasingly globally connected community in the 1990s. At this point, US-based EER researchers had a particular interest in signaling this as a new engineering discipline with all the accoutrements that might indicate its legitimacy, such as departments, journals, professional associations, etc. (see Froyd & Lohmann 2014, who somewhat confusingly for us consider this as an “ontological” characterization of the field [1]). At the same time, there also emerged a lively strand of scholarly work deliberating on what kind of research approaches would be most appropriate for these endeavors: whether EER would develop its own approaches or whether it would draw on other “parent” disciplines in the social sciences [2]. At that time, most aspirant EER researchers had backgrounds in engineering, and there was an awareness that researching education might be different to researching engineering science or design. While realizing that new approaches, for example, qualitative methodologies, might need to be entertained, there was also an intense focus on questions of rigor, as this community sought legitimacy for its research to be considered equal in quality to that of the established engineering disciplines. Borrego described this as “a new field [developed] by engineers who have been trained in well-established technical fields to expect more clearly defined standards of rigor” (p. 6) [3]. From this recognition, two major lines of inquiry developed: what methods of research are appropriate for EER, and what is the role of theory in EER?

The methods questions were prominent from the outset, especially from this emerging community in the US. This debate centered on the very different claims for rigor that one might have to make if using qualitative methodologies, which are not amenable to traditional, statistically supported measures of reliability, validity, and generalizability. An early analysis of articles published in the Journal of Engineering Education by Koro-Ljungberg and Douglas found that there was very limited use of qualitative approaches in EER [4]. Fast-forward to the present, and an analysis of the same journal from 1993–2022 found that the proportion of studies using qualitative methods has increased over the period of record, now roughly equaling the proportion of studies using quantitative methods [5].

Looking at it from the US perspective, it seems as if the theory questions have received less attention, and this might be due to this engineering audience being more attuned to questions of technique than to philosophical underpinnings, which tend to be less discussed in engineering sciences anyway, yet a different engineering education community had been forming from the 1990s in South Africa, and here the theoretical discussions were prominent from the outset. Case and Jawitz [6], noting the predominance of constructivist theoretical perspectives even in early EER work (as represented in the first national conference on engineering education in 1997), point to the influence of those trained in science and mathematics education research traditions (such as they were), disciplines where constructivism was prominent. At the same time, surveying the proceedings of this first conference, Case and Jawitz note the difficulty of inferring a theoretical perspective in published work, and in a subsequent piece they argued for the value of making theoretical perspective explicit when reporting on EER outputs [7]. They clarified that by the term “theoretical perspective” they “include both the philosophical justification for one’s research design (methodology) and the basis on which the knowledge claims are made (epistemology)” (p. 149), and in this paper they outlined three major paradigms—positivism, constructivism and critical inquiry—with key tenets summarized in

Table 1. Comparison of three theoretical perspectives [7].

	Positivism	Constructivism	Critical Inquiry
Aim of research	Establish facts and laws	Develop useful interpretations	Achieve social change
Research is guided by	Testable hypotheses	Research questions	Social problems
Role of researcher	Objective observer	Constructor of interpretations	Change agent

.

Notably, over these last few decades, many EER outputs increasingly signaled their adoption of a constructivist perspective. But this level of debate (constructivist vs. positivist) was not the most prominent when researchers talked about “theory”: the majority of such deliberations on “theory” were focused more on what might be termed “middle-range theory” [8], i.e., focusing on a particular phenomenon like learning or teaching [9]. At this level, there emerged a rich literature introducing EER researchers to particular theoretical approaches, initially with much focus on learning theories such as situated cognition [10] and conceptual change [11], later focusing more on theories related to identity, belonging, and community. A recent survey of the use of theory in EER [9] finds that the practice of explicitly naming a theory or theoretical framework is now quite widespread in the field [9]. This analysis found that espoused theories could be grouped into four families: 1. theories of learning, 2. theories of individual choice, 3. theories of power relations in society, and 4. theories of organizational change. Another practice that has become widespread, especially in US EER, has been the inclusion of a positionality statement [12].

This brief overview of discussions on theory–method deliberations in EER sets the stage for our analysis, which brings in a critical realism perspective to offer further tools for how we might approach these still crucial questions. In this paper, we present an analysis of a purposive sampling of 54 papers published across the Australasian Journal of Engineering Education (here denoted AJEE), the Journal of Engineering Education (here denoted JEE) and the European Journal of Engineering Education (here denoted EJEE) in 2024. The analysis was performed using critical realism as a theoretical framework and using AI-assisted abductive coding. The focus is on making visible the ontological, epistemological, and axiological commitments made implicitly or explicitly in the papers using an ontological–epistemological–axiological (OEA) framework (see Section 2). We undertook this project precisely because the vast majority of papers analyzed do not discuss OEA commitments, suggesting an operational norm of nondeclaration thereof.

The aim here is not to prescribe any particular paradigm, nor is it to suggest any hierarchy. Rather, the purpose is to reveal the philosophical plurality that is at the heart of engineering education research. We suggest that making these OEA commitments explicit will assist readers to ascertain what kind of knowledge is being developed in the paper. We hold that the plurality of OEA commitments is a strength of engineering education research and recognize that engineering faculty wishing to draw from this research to inform their practice may not be cognizant of this plurality. In addition, being more explicit about OEA commitments makes the researcher more aware of the constraints and enablements of the chosen position affording potentially fruitful reflexivity with respect to both the design and potential impact of a given study.

2. Theoretical Framework

In aiming to expand our present theoretical and methodological thinking on EER, in this paper, we sought out a theoretical framework with the potential to generate new and productive insights. Critical realism has been gaining traction as a theoretical framework across the social sciences for several decades [13]. In recent years, it has also been applied to STEM disciplines [14]. One reason that STEM based education researchers have valued critical realism is that it can serve as a metatheoretical framework across both natural and social sciences and can accommodate realist and constructivist epistemologies [13]. This means that it provides a metatheoretical foundation that can be used to analyze a plural discipline such as EER and can serve both in our outward gaze to engineering as a sociotechnical system and the inward gaze to EER theory and method, as signaled in the introduction to this paper.

2.1. Critical Realism

Most science and engineering disciplines focus on the empirical and limit the apprehension of reality to that which can be measured and observed. Critical realism takes a more holistic view of reality, making a distinction between: 1. all that is potentially possible, 2. the subset thereof, what is actually in existence at any moment, and 3. the subset thereof, which is measurable. This is to say that the phenomena that are observable, measurable, and quantifiable (termed “the empirical”) are a subset of all the phenomena in the world at any given time (“the actual”). And that which is occurring at any given time (the actual) is constrained and enabled by generative mechanisms, such as causal relationships, attractive forces, etc., which interact in a probabilistic manner, making the set of all possible actual events (“the real”).

This means that research that only focuses on the empirical may provide overly simplistic explanations for observed phenomena, because that which has been measured does not necessarily give the information required to make explanatory or causal claims. In the context of this paper, this means that commitments made explicitly or implicitly in the domain of reality will constrain or enable the kinds of explanations that can be made.

Critical realism further claims that any field of practice comprises the interaction of three different domains. For engineering education research, as represented in Figure 1, this is 1. the world under investigation (engineering education practice), 2. the current canon of engineering education research (what counts as knowledge in EER). and 3. the community of engineering education researchers (who dictate what counts as valid EER, primarily through peer review of papers, conference presentations, and grant proposals) [15]. This three-domain structure is directly relevant to the argument this paper makes. The norm of not being explicit about OEA commitments is not the product of individual researchers failing to reflect on their philosophical commitments: it is produced and reproduced at the level of the third domain, through the structural expectations of journals, reviewers, and doctoral training programs that have never required such explicit consideration. From a CR perspective, this norm is itself a generative mechanism thatshapes what gets published, what counts as a complete methodology section, and therefore what kinds of knowledge the field produces and how that knowledge is read.

2.2. What Is Meant by OEA Positioning

Critical realism insists on the importance of distinguishing between:

• Ontology—what kind of reality exists.
• Epistemology—what we can know about that reality and what kinds of knowledge claims we can make.
• Axiology—what values shape the research.

As a shorthand, we are introducing the term “OEA framework” to refer to these central questions. Critical realism holds that all three of these dimensions are important because they are operationally distinct and each positioning will constrain or enable the research that takes place, and each dimension operates at a different level [16]. Thus, the ontological, epistemological, and axiological commitments are causal mechanisms in the production of the research, whether they are made explicit or not.

Ontology refers to the assumptions the researcher makes with respect to the nature of the phenomena they are studying. Do those phenomena exist independently of human perception or are they constituted by it? Is reality singular and stable or multiple and context-dependent? Does the social world have structures and mechanisms that operate independently of whether we observe or name them? One’s understanding of the nature of reality will shape what can be investigated. The principal positions in this dataset are social constructionism, naïve realism, and critical realism. Because of the potential pejorative associations with the phrase “naïve realism,” we refer to this category as “assumed realism” for the rest of the paper.

Epistemology refers to what can be known and how we come to know it. In the critical realist worldview, it is essential not to conflate epistemology and ontology. The chemical reaction producing carbon dioxide from sodium bicarbonate, which is used to create the “rise” in baking, has existed since sodium bicarbonate formed (ontology). It has been used since ancient Egypt, but the way we now understand that chemical reaction mechanistically has only been described since the mid-twentieth century (epistemology). The reaction existed prior to human use and human use existed prior to our current understanding. The epistemological understanding will shape the way in which someone might choose to further investigate this reaction, but the actual reaction is still distinct from the understanding of the reaction. One’s epistemological commitments will shape the questions one chooses to ask. The principal positions in this dataset are interpretivist, post-positivist, and critical.

Every act of research begins with a researcher who values something. In many STEM disciplines, axiology, the statement of values, is the dimension most systematically erased. Tools by which this is accomplished include the passive voice, the suppression of the first person, and the claim to objectivity. Each performs value neutralityl while enacting a particular axiological stance [15]. Critical realism rejects this erasure. Values are not external to the research act. Something to be disclosed and then bracketed—they are constitutive of it. The axiological commitments of a researcher shape what questions are asked, what counts as a meaningful contribution, and who is imagined as the beneficiary of the knowledge produced. The OEA framework, developed for the purposes of our analysis, distinguishes two analytically separable axiological dimensions.

Axiology 1 (research values) captures what the researcher values about knowledge itself. The principal positions in the dataset are: predictive/instrumental (knowledge is valuable for its capacity to predict or optimize outcomes), interpretive/humanistic (knowledge is valuable for illuminating meaning and human experience), critical/emancipatory (knowledge is valuable for revealing and challenging unjust power structures), and transformative (knowledge is valuable for producing structural change at the institutional level). Each position entails a different relationship between the researcher and the researched and holds different grounds for claiming a contribution has been made.

Axiology 2 (social and ethical orientation) captures what human good the research is ultimately oriented toward. Principal positions in the dataset include: pedagogical effectiveness (improving teaching and learning), professional formation (producing graduates and faculty with particular competencies or identities), social justice and equity (challenging structural inequalities), institutional reform (changing organizational structures or curricula), and public good (engineering’s broader responsibility to society).

The distinction between these two levels is more than taxonomic. A paper may declare social justice as its social and ethical orientation (axiology 2) while deploying a predictive, instrumental knowledge-logic (axiology 1) that treats participants as objects of measurement rather than agents within structuring conditions. This tension would not be visible if the analysis treated axiology as a single dimension.

A distinction central to the application of the OEA framework in this paper is that between critical realism and critical theory. The two traditions share a commitment to structural analysis and to the view that power relations shape social reality in consequential ways. They are, however, ontologically distinct. Critical realism holds that social structures such as institutional arrangements and cultural norms exist independently of human observation and meaning-making [16]. They are real, whether or not participants name or recognize them, and it is precisely this independence that gives them their generative, causal character. Critical theory, and the critical social constructionism with which it is often associated in education research, treats structures as real, but constituted through and reproduced by human practice, discourse, and meaning-making. Structures are practice-dependent rather than practice-independent. In coding the dataset, this distinction was operationalized through a single diagnostic question: Does the paper treat social mechanisms as operating independently of human observation or as produced through it?

2.3. An Illustration

To make the importance of OEA positioning visible, imagine a project on the trends in the graduation rate of engineers in an engineering program. Three researchers, Susan, Bruce, and Penelope, seek to bring about institutional reform (axiology 2) in order to improve the graduation rate.

Susan assumes that the graduation rate is a real, stable, measurable phenomenon. She assumes that there are identifiable causes of student attrition and completion that operate independently of context and that can be discovered through systematic data collection. Susan takes as a given that if we know what causes attrition, we can design interventions that change the causal conditions and improve outcomes. She is likely to ask research questions such as “What is the relationship between students’ prior academic achievement and their likelihood of completing their degree?” or “Which first-year assessment results are most predictive of student attrition in years 2 and 3?” These are useful questions and can provide the data necessary to put in early-warning systems and targeted support programs.

Bruce assumes that the graduation rate is a socially constructed phenomenon constituted though students’ ongoing negotiations of identity, belonging, meaning, and social recognition within a specific institutional and cultural context. Graduation rates cannot be understood without understanding the interaction of people in the system. Thus, meaning-making is valuable because it will provide insight into the institutional forces that are in operation. He is likely to ask research questions such as “What does it mean to ‘belong’ in an engineering program, and how do different students negotiate belonging differently across their degree journey?” or “How do first-year engineering students narrate their experiences of academic difficulty, and what role do these narratives play in their decisions about continuing in the program?”

Penelope assumes that the graduation rate is neither simply a stable fact waiting to be measured nor solely a story students tell about their experience, but the visible trace of deeper structures and mechanisms, such as institutional resourcing, cultural norms, and historical patterns of exclusion, that operate whether named or not. She takes as given that the patterns Susan finds and the meanings Bruce documents are both real evidence of these underlying conditions, and that neither on its own explains why students leave. She may ask research questions such as “What institutional conditions would need to be present to produce both the relationship between prior achievement and completion and the experiences of belonging students describe?” or “Do the same underlying conditions operate differently for different groups of students, producing different statistical patterns and different narratives?” Susan’s research will show which variables predict attrition, but not why those variables have the effects they do or what an institution would need to do to address the underlying conditions. Bruce’s research will show how students understand their experience in relation to institutional structures in ways that aggregate data cannot capture. Penelope’s research asks what must be true of the institution for both the regularities Susan identifies and the meanings that Bruce documents to exist.

The point here is that all three contributions are important to understand graduation rates, and each may lead to institutional reform. But statistical predictors will have no value in Bruce’s study, and the particularities of institutional culture that may be shaping student success will not appear in Susan’s. Further, Penelope’s understanding of the mechanisms operating in the institution require the data produced by both Susan’s and Bruce’s studies. The kinds of knowledge produced are shaped by and limited by their OEA positions. Susan, Bruce, and Penelope have different ontological stances towards the world of engineering education practice that they are interrogating (return to Figure 1).

In this research, Susan is taking assumed realist, post-positivist, predictive positioning. What is being measured is a good approximation of the phenomenon, and with no change, the graduation rate will remain steady. Bruce seeks to create understanding and meaning-making through social constructionist ontology and interpretivist epistemology. The context of the study and the particularity of the students are crucial to the insight. For Penelope, critical realist ontology and abductive epistemology are employed to identify the mechanisms that generate both the lived experience and the measurable pattern.

The statement of OEA positioning does not resolve the differences in the research projects, but it does make the nature and limits of all three contributions more visible to the reader, enabling different kinds of knowledge to be brought to bear productively to improve the graduation rates. It allows research findings to be used effectively in conversation with one another. If there are apparent contradictions, these may be the different methods of research illuminating different facets of a complex phenomenon.

3. Methodology

In this study, we investigated the field of EER with respect to making explicit OEA commitments. To generate a dataset for this project, we chose three Q1 journals (JEE, EJEE, and AJEE) to allow both within-field and cross-regional comparison. Articles published in 2024 issues were chosen as sufficiently recent to represent current practice while also sufficiently established for citation count to serve as a proxy filter for engagement by the field (citation count of 5 or more as reported on the journal’s website on the day of sampling—16 April 2026). We acknowledge that citation counts should not be taken as a measure of quality and may aggregate across the three journals at different rates. The choice was made pragmatically in an attempt to ensure that the data represented the established field. Reviews and editorials were excluded from the sample. This gave a dataset of 54 papers out of a total of 104 research articles.

The goal of the analysis was to determine the types of research presently conducted in the field and to ascertain whether OEA positioning was made explicit. We employed abductive coding, which is congruent with the interpretation of qualitative data in a critical realist ontology [17]. Abductive coding requires the development of a preliminary codebook that is informed by theory. The coding is then applied to the dataset, which may involve refinement of the codebook. The result is a theory-laden analysis that provides insight into potential explanatory causes for the patterns that are revealed.

In line with the central argument of this paper, we stipulate the OEA position we have taken in this paper, positioning this in the Methodology section because these choices frame the research described in this project. In designing this study, we adopted a critical realist ontology, meaning that the patterns of positioning observed in this dataset are caused by real generative mechanisms, whether they are named or not. Our epistemological position is interpretivist and operates through abductive logic. The coding for the dataset was produced through a theory-laden interpretation based in the critical realist metatheoretical framework [16]. The OEA position of each paper was inferred—when not explicitly stated—based on the philosophical assumptions that were likely to have been present to produce the methodological choices, theoretical commitments, validation strategies, and the framing of the contribution for that paper. Our research axiology (axiology 1) is critical/emancipatory, meaning that we value transparency and reflexive self-awareness to ensure that the field’s philosophical diversity is legible, enabling readers to assess what kind of knowledge is developed in the paper. The intent is to enable appropriate cumulative progress to be made in the field without the narrowing of philosophical diversity. Our social and ethical commitment (axiology 2) is to institutional reform. We hope to show the value of making philosophical commitments transparent so that engineering education research may be accountable to these commitments and the results of research may be used appropriately.

3.1. AI-Assisted Abductive Coding

The coding framework and dataset were developed through five phases using an AI language model (Claude, Anthropic) as a “research assistant” to facilitate accelerated preliminary analysis.

Phase 1—Establishing potential codes: All papers were selectively read by author 1 with attention to OEA positioning, confirming that very few papers made any such statement explicit. Initial designations were tabularized across eight columns: journal, title, authors, research questions, methods, ontology, epistemology, and axiology.

Phase 2—Refining codes with AI assistance: AI assistance was initiated by uploading this provisional spreadsheet alongside a prompt framing the study’s purpose and the special issue call, prompting Claude to act as a “research partner” using critical realism as the metatheoretical framework. A “dialogue” then ensued, establishing potential codes for ontology, epistemology, and axiology. The most substantive early exchange concerned axiology. A definitional difficulty surfaced where it became apparent that axiology was capturing both research values and the purpose of this particular research project. From a CR perspective, values that are constitutive of the act of research rather than external disclosures need to be bracketed, and thus the conflation of these two kinds of axiology was analytically problematic. The result was a restructured two-dimensional axiology as described earlier: axiology 1, capturing research values (predictive/instrumental, interpretive/humanistic, critical/emancipatory, pragmatic, transformative), and axiology 2, capturing social and ethical orientation (individual development, pedagogical effectiveness, professional formation, institutional reform, social justice, public good). This two-level structure made it possible to identify mismatches such as a paper claiming progressive ethical goals while deploying instrumental knowledge-logic. At this point, the provisional codebook comprised definitions for each code.

Phase 3—Development of anchor cases: Author 1 chose five papers to represent the range of methodologies, including at least one paper from all three journals and occupying “paradigmatic” positions [18] in the dataset. These papers were Gratchev et al. [19], Nikolic et al. [20], Young et al. [21], Dugan et al. [22], and Smith et al. [23]. Author 1 coded each of these anchor cases, applied the label of explicit or implicit to each code, uploaded the paper, and prompted Claude to interrogate each designation with respect to the provisional codebook. This process was carried out for each paper and resulted in five anchor cases that the AI assistant could use for verification. This was documented in an OEA coding framework document, which is available as a supplementary document (S1).

Phase 4—Coding the dataset: The full dataset was then coded in batches of five papers per exchange. Each paper was analyzed and coded using the anchor cases. Observable features of the text—methodological choices, theoretical frameworks, language patterns, validation strategies, units of analysis—were used to infer the best explanation of the underlying philosophical assumptions where these were not explicitly stated. The anchor cases were used by the LLM as context to interpret boundary cases not anticipated by the OEA coding framework, in alignment with the method of abductive coding [17]. A “dialogue” between author 1 and Claude allowed author 1 to determine that theoretical framework declarations should be distinguished from OEA declarations, and papers using social justice framing with instrumental methods were flagged as potentially axiologically misaligned. Thus, several coding rules were refined mid-process, including the introduction of an “explicit (partial)” status category for papers making incomplete declarations. The findings were tabularized and are available as a supplementary document (S2).

Phase 5—The first author went through the entire summary table and returned to the papers to verify the output. This process was repeated independently by the second author.

It should be noted that the role of the AI in this process was analogous to a theoretically trained research assistant rather than an autonomous analyst. It did not generate the framework, determine the codings, or produce the analytical arguments. It applied the researcher’s framework consistently across a large volume of text, flagged tensions and ambiguities, and enabled the authors each to focus interpretive attention on the cases that most required it. The dialogue record constitutes an audit trail consistent with the trustworthiness standards of interpretive research: the analytical reasoning is preserved, traceable, and open to interrogation. A methodological account is provided as a supplementary document (S3).

We recognize the ethical concerns of using AI-assisted analysis of data. It is vital to consider the risks associated with the uploading of any data to such models. The full-text versions of the papers were uploaded to a conversational AI interface limited to interpretative analysis. In this interface, the papers were not made available for reproduction, redistribution, or model training. In this case, using publicly available articles, the authors of these papers would have no expectation of privacy with respect to the contents of these papers, and concerns regarding copyright infringement were managed by using a constrained platform that assured that the data would not be used for model training.

We also acknowledge that the use of a single AI model for the analysis is a limitation of this work. All models with the necessary limits for use for data analysis available to us were trained on data that draw more heavily from Western sources written in English. This may have biased the analysis. Furthermore, the choice of the anchor cases may also have skewed the analysis in a manner we have not noticed, despite both authors checking the coding independently.

3.2. What Counts as an Explicit OEA Declaration

A central analytical decision in this study concerned what qualifies as an explicit OEA declaration. This matters because positionality statements and theoretical framework declarations can create the appearance of philosophical transparency without fully constituting it. Clarifying this distinction is an important claim herein.

A positionality statement, as the term is most commonly used in EER, discloses aspects of the researcher’s social identity, such as race, gender, institutional role, and personal relationship to the research topic. This disclosure is valuable for certain kinds of research, where the sociocultural experience of the researcher plausibly shapes interpretation. But it is not the same as declaring an OEA stance. To know that a research team includes members with marginalized identities tells the reader something about the researchers’ relationship to the phenomenon. It does not tell the reader what the researchers assume about the nature of that phenomenon (ontology), what they take to constitute valid knowledge of it (epistemology), or what they value about the knowledge they are producing (axiology). Where a positionality statement did include a commitment on one of these dimensions, usually an axiological orientation toward the purpose of the research, this was coded as partially explicit rather than fully declared, because the commitment was stated without being connected to the theoretical or methodological choices of the paper.

The declaration of theoretical framework presented a related, but distinct challenge. A paper grounding its analysis in, say, community of practice theory or community cultural wealth carries implicit OEA commitments. The theoretical framework specifies the “middle-range theory” (see above) with its conceptual vocabulary and the unit of analysis, but it does not necessarily resolve the ontological question of what kind of reality the researcher assumes or the epistemological question of what kind of knowledge claim the analysis is entitled to make. Two papers can coherently deploy the same theoretical framework from quite different philosophical positions. For this reason, theoretical framework declarations were treated as evidence useful for inferring OEA positioning, but not coded as explicit declarations in their own right. The same logic applies to methodology textbook citations. These signal a design-level choice, not a philosophical one.

These distinctions produced a three-level coding protocol: explicit (OEA position named using philosophical or paradigmatic language in the methods or methodology section), partially explicit (a commitment stated in a positionality statement, but not connected to methodological or theoretical choices, or one dimension declared while others remain implicit), and implicit (position inferred from methodological choices, theoretical frameworks, language patterns, and the framing of the contribution).

4. Results

The full coding of each paper is available in the supplementary information. That information is included so that readers can judge for themselves the veracity of AI-assisted abductive analysis. It is important to note that the specific coding for each paper is less important than the general pattern that these individual assignments contribute to generating in the argument we are developing in this paper. In addition, the sample of papers is limited in size. The absolute values of the numbers of papers in any category should not be taken as representative of the field as whole. The dataset is relatively small and could be skewed towards more accepted forms by virtue of the citation count criterion. The purpose of the analysis is to illustrate that the field is indeed plural, and therefore OEA commitments are necessarily varied.

Each paper was assigned a code for each OEA axis (ontology, epistemology, axiology 1, and axiology 2), resulting in four codes for every paper. If an OEA position was stated in the paper, this code was labeled explicit and the authors’ stated position was taken. For example, Tafahomi and Chance state that the methods they chose are “consistent with the interpretivist worldview in terms of methodology, ontology, epistemology, and axiology” [24]. This code was labeled interpretivist ontology (explicit), interpretivist epistemology (explicit), and interpretive/humanist axiology 1 (explicit). If an OEA position was stated in the positionality statement, but not linked to methodological or theoretical framing, the label “partially explicit” was applied to that code. For example, Polmear et al. acknowledge marginalized identities in their positionality statement and note that “methodological approaches and their underpinning ontology” shaped the research aim [25]. Because the structural model used treats belonging as a real latent phenomenon shaped by real social mechanisms, this was classified as being aligned with critical realism. The ontological positioning was coded as critical realism and labelled “partially explicit.” If no statement was made with respect to OEA positioning, the label “implicit” was applied. Thus, each paper was assigned four codes (O, E, A1, and A2) and each code was labeled explicit, partially explicit, or implicit.

Finally, each paper was assigned a category of coherent or partially coherent depending on whether the methods used to generate data were in alignment with the assigned codes. It is important to note that this classification should not be understood as a judgement on the quality of the paper. It is simply referring to whether the authors actively resolved any potential disjunctures in the text of the paper. Coherency was applied when there was no disjuncture. Partial coherency was assigned for three kinds of disjuncture. Firstly, some papers showed a disjuncture between the method and nature of the phenomenon being investigated: for example, using a post-positivist survey to investigate complex phenomena such as engineering identity [26]. Secondly, we noted some papers employing mixed methods that employed interpretivist and statistical data, which use different knowledge logic, but failed to comment on this distinction [27]. Thirdly and finally, we noted papers stating commitment to social justice, but treating participants as objects, for example, structural inequality measured through a post-positivist survey that treated structural discrimination as a variable [28]. There can be excellent reasons for making these choices, but they tend to reduce the phenomenon under interrogation to factors that can be measured and quantified. From a critical realist perspective, noting the simplification and justification for doing so is valuable information for the reader. Hence, as stated earlier, a classification of partially coherent is a comment on the description of the research. It does not in any way necessarily negate any findings.

4.1. The Philosophical Landscape: Plurality Across Method, Ontology, and Epistemology

The papers in the dataset varied in their use of methods for data collection. Of the 42 papers that were based on empirical data, 22 used qualitative data, 10 used quantitative data, and 10 used data from mixed methods. In addition the dataset also included seven design and evaluation studies and five conceptual or theoretical papers (Figure 2).

A little under half of the papers employed a social constructionist ontology (24 of 54). This was followed by assumed realism (17 papers). Eleven papers employed critical realism, although not all used the full explanatory power thereof. The authors of two papers declared their ontological position to be interpretivist. In our framing, these two interpretivist papers would fall under social constructionism. However, given that we were most interested in the declaration of these commitments and have noted these papers specifically for the sake of clarity, they are categorized as “Author declared: interpretivism” in Figure 3.

Epistemologically, aligned with the dominance of the social constructivist ontology, 19 papers were categorized as interpretivist. All 17 assumed realist papers aligned with post-positivist epistemology. Nine papers fell into critical epistemology. Notably, eight papers used more than one epistemology, e.g., interpretivist and post-positivist. In congruence with the decision made for ontology, the authors of one paper declared their epistemological position to be pragmatic. This is noted in Figure 4 as “Author declared: pragmatic,” even though pragmatic was not a category we were using to describe epistemology.

We reiterate here that because the focus of the paper is on declaration of OEA commitments, where authors made specific mention of an ontological or epistemological position, we accepted the labels they had given, even if they did not fall into the categories we had developed. Our goal in creating the categories was to make visible the plurality of OEA commitments in EER.

4.2. OEA Declaration: Implicit Positioning as Field Norm

This demonstrated diversity of research method, ontology, and epistemology means that the field is genuinely philosophically diverse. From the critical realist point of view, this diversity is a strength. It gives access to a broader understanding of the nature of engineering education. The failure to declare the OEA commitments, however, has implications at different levels. Firstly, for the application of knowledge generated within the EER field to be able to facilitate change in engineering education practice, it is preferable to have OEA explicit. Many engineering educators will not have the skill to be able to intuit OEA implications from the theoretical framework. This means that the value of the research may not be understood, the implications for practice may be improperly applied, and the reasons for failure in that implementation to beobscured. Secondly, it makes it more challenging for readers to locate the source of conflict for papers that seem to come to differing conclusions. In some cases, the issue may genuinely be results that show different outcomes, but in other cases the philosophical commitments may give rise to results that are operating at different levels, as illustrated in the scenario of Susan, Bruce, and Penelope described above. Thirdly, the cumulative development of knowledge might be impeded because the research conducted from different OEA positions needs to be aggregated or interpreted in different ways. This problem of methods for the aggregation of knowledge claims is noted by Power with respect to systematic reviews [29].

Figure 5 shows the prevalence of papers where there was an explicit OEA statement in at least one dimension. Quite strikingly, the overwhelming majority of papers show no explicit declaration, indicating that this is a norm in the field of engineering education. The relatively large band of partially explicit papers in JEE (4 of 54) can be explained by the requirement in JEE for a positionality statement. Neither EJEE nor AJEE have this requirement.

In Figure 6, the OEA declaration is broken down according to the four dimensions. Here it is clear that the overwhelming majority of papers have no explicit commitments on any of the four dimensions. These commitments are inferred in the analysis described in Section 3.1.

Across the full dataset, only two papers give explicit OEA commitments. Chadha and Hellgardt offer the most robust demonstration. They note: “Ontologically, our own positionality as researchers in conducting this work is that of interpretivist—hence the use of grounded theory—and epistemologically that of pragmatist, in keeping with the pragmatic stance on grounded theory as championed by Corbin and Strauss” [30]. But even this declaration is slightly unclear. Interpretivism is an epistemological position, which suggests a social constructionist ontology. Pragmatism can operate as an ontology, an epistemology, or an axiology, and is usually used in EER in the axiological sense. Corbin and Strauss employ pragmatism in both ontological and epistemological senses.

It is important to note that the lack of OEA declaration means that inadvertent simplifications, inconsistencies, or clashes may be overlooked. However, in the dataset, most papers are coherent as judged by their inferred OEA positions (Figure 7). Furthermore, there is no notable difference in the rate of coherence between the three journals (Figure 8). Importantly, this means that for 42 out of the 54 papers sampled, the lack of declaration did not generate any problem with the individual paper.

5. Discussion

We acknowledge that the OEA codes applied to any individual paper may be subject to deliberation. The codes used were developed using critical realism as a metatheoretical framework, and so the descriptors used are congruent with critical realism. The purpose herein is not to demonstrate the accuracy of AI-assisted abductive coding (since all proposed codings were checked independently by both authors) or to ensure that authors would recognize and accept the code given to their paper if they were approaching their paper from a different operational OEA paradigm. Given that the focus of this paper has been to affirm OEA plurality within EER and to demonstrate that the dominant practice is not to state OEA positioning, minor corrections of language of description for any paper insisted upon by a particular author would not substantially alter the findings of this paper.

5.1. Reflexivity Without Transparency: The Limits of the Positionality Statement

JEE requires a positionality statement, whereas neither EJEE nor AJEE do. However, most positionality statements focus on statements of researcher identity, such as race, gender, class, institutional role, personal experience, etc. For some kinds of research, this declaration of identity is important, because the sociocultural experience of the author may influence their interpretation of the data. However, very few papers discuss positionality with respect to research orientation. Here, axiological commitments towards the purpose of the research and one’s commitment to the people and phenomena being studied are important. For example, in their paper analyzing problem-solving using systems thinking, Dugan et al. make clear how their positionality shaped what they looked for and what they were committed to changing [22].

The lack of distinction between these two kinds of positionality creates a semblance of reflexivity, but may not actually be consciously enacted in the process of the research. The reader will know aspects of the personal identity of the authors, but are not given access to the commitments that have shaped the research.

5.2. The Costs of Invisible Positioning: Epistemic Fallacy and Axiological Incoherence

As noted in the Results, 12 out of the 54 papers were noted as having partial coherency with the inferred OEA positions. These problems fell into two categories: what critical realism terms epistemic fallacy and axiological incoherence.

5.2.1. Epistemic Fallacy

One notable issue made visible through critical realism is the epistemic fallacy [16]. In these cases, no distinction is made between the empirical measurement and the ontological reality that generates the phenomenon. For example, complex phenomena such as belonging and identity are narrowly defined in terms reduced to those that are measurable by a survey. This means that measured constructs are reported, but no account can be given for why the results appear as they do. Such papers can be useful, and it is not the intent here to critique the structure of these studies. But the absence of the OEA commitment statement renders the simplification invisible to the reader. The empirical is taken to be the real. When institutional changes are made as a consequence of such research, the complexity of the underlying generative mechanism may never be taken into account, the institutional change may fail or cause unexpected difficulties, and the conclusion may be drawn that the research was not valid or somehow flawed. This may create a risk for the perceived value of engineering education research.

5.2.2. Axiological Incoherence

Several papers show an orientation to social justice or equity (axiology 1) whilst operating with an instrumental or predictive research logic (axiology 2). Whilst this in no way invalidates the research findings, and noting that these methodologies can legitimately be used to serve social justice goals, the instrumental logic requires the treatment of participants as objects of measurement rather than agents operating within a structuring system, which potentially creates a dissonance for emancipatory research. This mismatch itself is not itself a methodological problem, but the lack of acknowledgement of the dissonance reveals an absence of philosophical self-examination.

5.3. Nondeclaration as a Field-Level Mechanism

For those papers that have coherence in OEA position, there is still a consequence. The nondeclaration of OEA positioning is not a neutral absence. The evidence is present in most papers sampled herein, and is therefore the result of a mechanism operating at a level in the field of engineering education research. A likely mechanism is the inherited culture of technical rationality, disciplinary training traditions, and editorial norms. This suggests that EER itself lacks some of the self-reflexivity that we are trying to induce in engineering educators. The consequence of this mechanism is that it makes the cumulative knowledge-building in the field substantially more difficult. The solution is not to narrow the norms of paradigm usage, but rather to reveal OEA status. This will allow nondominant paradigms greater valence. Their presence in the literature will allow the knowledge that they produce to be seen in the full light of their contribution, rather than being dismissed as the awkward outlier that does not easily fit into more dominant clusters.

5.4. The Myth of the Neutral Researcher

In many STEM disciplines, the unarticulated—but dominant—research paradigm sits in the assumed realist/post-positivist space. The object of interrogation has an existence independent of human interaction and often can be usefully isolated and studied, and reliable, reductive simple models can be produced to afford control and prediction. In this paradigm, the researcher is positioned as the neutral observer. But this is not accurate [16]. The fact that the experiment is reproducible by another person is a product of the nature of the problem being investigated and the knowledge being produced. The axiological position of the original conceiver of the experiment is erased from the record, but it is not neutral [15]. The consequence is that research produced from this OEA position (assumed realism/post-positivism) is presumed to be neutral and objective, which does not bear reflexive scrutiny. Whilst the knowledge may well be reproducible, the purpose of the production of this knowledge needs to be stated. In a field such as EER, firstly there are few experiments that truly meet the requirements of a closed system because the education is in nature open, recursive, and semiotic [31]. Thus, the OEA declaration of the researcher is essential to understand the limitations of their claims. Secondly, engineering educators reading research papers may have an implicit bias towards research that creates knowledge in the paradigm that they are familiar with. Thirdly, the axiological commitments of the researcher play a fundamental role in shaping what research is carried out.

5.5. The Invitation

At the level of the individual researcher, the explicit naming of OEA commitments is a discipline of reflexivity. This practice makes visible the choices that are otherwise naturalized as simply “how research is done.” For researchers working within dominant paradigms, this is an invitation to recognize the philosophical character of their own practice rather than treating it as the neutral default. For researchers working in less dominant paradigms, it is an opportunity to make the value and specificity of their contribution legible to a wider audience. In both cases, the declaration does not constrain the research: it enables the research to be read, evaluated, and used appropriately, and thus including these commitments in any publications discussing the research may be useful.

5.6. Limitations

This exploratory study was carried out on a limited dataset. The principal purpose of this study was to reveal the practice of OEA declaration. The classification of papers with respect to research methods, epistemological commitments, or ontological commitments was done relatively quickly to reveal this underlying practice. For the purposes herein, the existence of the variation was sufficient to proceed with the central claim of the importance of OEA declaration. However, any conclusions based on the patterns of variation reported herein should only be used as a point of departure for further investigation.

The use of a single LLM model to analyze data can be potentially problematic, but again, because our purpose herein did not require precision of analysis at this point of the study validation, using a second model was not deemed necessary.

6. Conclusions

The central finding of this paper is that the philosophical foundations of engineering education research are present, but invisible in the overwhelming majority of cases. The ontological, epistemological, and axiological commitments that shape what questions are asked, what counts as evidence, and what contributions are claimed are real and consequential, whether they are named or not. This study usefully extends the earlier analyses by Case and Jawitz [6,7], who focused only on epistemological commitments in their analysis of what they termed “theoretical perspective.” The patterns of nondeclaration documented at that point and further demonstrated here across a wider range of publication outlets are may reflect structural features of the field. These features are inherited norms of technical rationality, the legacy of engineering’s post-positivist research culture.

In this paper, we have argued that the consequences of nondeclaration are threefold. Firstly, it impedes the appropriate application of research to practice, because knowledge produced from different OEA positions cannot be used interchangeably. Secondly, it renders incoherence invisible: the epistemic fallacies and axiological clashes documented in this dataset are not immediately visible to authors or reviewers because neither has been required to name the philosophical assumptions that would make them apparent. Thirdly, it grants dominant paradigms an unearned neutrality, allowing assumed realist and post-positivist assumptions to function as the invisible default while other positions are treated as in need of justification. The solution this paper proposes is not to narrow the philosophical range of the field. The diversity of EER is a genuine strength, but we argue for the value of making that diversity legible. It would be useful if OEA positioning was stated in publications. The OEA framework introduced here is an invitation to the kind of reflexive self-examination that the field asks of engineers confronting sociotechnical complexity. A field that cannot examine its own philosophical foundations is limited in its capacity to equip engineers to examine theirs.