Evaluating University Engagement as Institutional Quality: Between Standardization and Systemic Integration

Abstract

The incorporation of university engagement as a mandatory dimension of institutional accreditation has reconfigured the debate on quality in higher education, particularly in regulatory contexts such as Chile. This study develops a narrative review with a comparative analytical approach to examine the evaluative rationalities that structure the assessment of university engagement within national and international quality assurance frameworks. The analysis draws on Chilean regulatory documents and key international models, including the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG), the HE-BCI system in the United Kingdom, the E3M Project, the Carnegie Community Engagement Classification, and recent literature on the evaluation of complex university–community engagement. The findings identify three structural tensions that organize contemporary evaluative frameworks: (1) standardization versus institutional diversity, reflecting the trade-off between comparability and contextual adequacy; (2) functional reduction versus systemic transversality, associated with the treatment of engagement as a discrete function or as a cross-cutting institutional dimension; and (3) fragmented evaluation versus institutional integration, linked to the degree of articulation between engagement, teaching, research, and governance within quality assurance systems. These tensions reveal that the evaluation of university engagement is not merely a technical issue of indicator design, but a structural problem embedded in institutional architecture and governance. Based on these findings, the article proposes a systemic evaluation model structured around three interrelated dimensions: strategic purpose, relational processes, and differentiated contribution and impact across temporal scales. This model seeks to reconcile the demands for comparability with the relational and contextual complexity of university engagement, while promoting its integration within the institutional quality cycle. The study contributes to positioning the Chilean case within the international debate on the third mission and advances a conceptual framework for evaluating university engagement that moves beyond indicator-based approaches toward a systemic understanding of institutional quality.

1. Introduction

During the last decades, higher education systems have undergone a profound expansion and consolidation of quality assurance mechanisms, leading to the institutionalization of regulatory frameworks that govern not only teaching and research but also the so-called third mission of the university. This process has been accompanied by the progressive formalization of standards, indicators, and accreditation systems aimed at ensuring strategic coherence, transparency, and continuous improvement (ENQA et al., 2015). Within this evolving landscape, Vinculación con el Medio—conceptually aligned with what is internationally referred to as university engagement—has transitioned from a function traditionally associated with cultural extension or community service provision to a formally assessable dimension of institutional quality.

However, the incorporation of university engagement into quality assurance systems has opened a central debate in the international literature: to what extent is it possible to evaluate a relational, territorial, and context-dependent function through standardized instruments without reducing its inherent complexity? Comparative approaches reveal divergent evaluative rationalities. On the one hand, some frameworks prioritize comparability through the production of homogeneous and quantifiable indicators, as exemplified by systems measuring university–business interaction (HESA, 2023). On the other hand, alternative models emphasize the institutionalization of community engagement as a cross-cutting dimension of organizational identity, integrating governance, curriculum, and research practices (Carnegie Foundation for the Advancement of Teaching, 2024). European initiatives aimed at operationalizing the third mission have similarly highlighted both the necessity of developing specific evaluative frameworks and the limitations associated with their quantification (E3M Project, 2012).

In parallel, contemporary quality assurance frameworks—such as the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)—promote a systemic understanding of institutional quality based on strategic coherence, public accountability, and continuous improvement (ENQA et al., 2015). From this perspective, quality is not conceived as a set of isolated functional domains but as the result of the articulation between teaching, research, governance, and societal interaction. Consistently, recent approaches oriented toward public value and sustainability advocate for a whole-institution approach, integrating teaching, research, and university engagement in response to complex social and global challenges (IAU, 2022).

This international debate has taken on a concrete institutional form in the Chilean context with the enactment of Law No. 21,091 on Higher Education, which established the mandatory accreditation of university engagement as a core dimension of institutional quality. In compliance with this regulation, the National Accreditation Commission (CNA, 2022) incorporated explicit criteria requiring institutions to demonstrate formal policies, bidirectional engagement mechanisms, impact evaluation, and articulation with teaching and research. This regulatory shift has driven the creation of dedicated organizational structures, monitoring systems, and performance indicators, consolidating university engagement as a strategic and mandatory component of quality assurance. At the same time, this development is embedded within a broader tradition of evaluation institutionalization in the Chilean public sector, where agencies such as the Directorate of Budgets have implemented systematic ex-post evaluation mechanisms—including program evaluations and impact assessments—since the late 1990s (Arteaga Viveros, 2024; DIPRES, 2015, 2024). These frameworks illustrate that evaluation systems do not merely measure outcomes but actively shape organizational practices and accountability cultures.

Despite this regulatory consolidation, significant conceptual and operational tensions remain unresolved. First, the imposition of standardized evaluative frameworks risks promoting homogeneous models of university engagement that overlook institutional, territorial, and mission-specific diversity. Second, there is a persistent tendency to reduce engagement to extension activities or external service provision, thereby limiting its potential to transform teaching and research processes. Third, the evaluation of engagement continues to oscillate between being treated as a discrete functional domain and being understood as a systemic dimension that permeates the entire institutional architecture of quality.

In this context, the central problem is not limited to measuring university engagement, but rather concerns the design of evaluative frameworks capable of accounting for its relational, territorial, and formative dimensions without distorting the broader mission of higher education institutions. Unlike research—traditionally assessed through bibliometric indicators—or teaching—evaluated through retention, completion, or learning outcomes—university engagement involves bidirectional processes, deferred effects, and organizational transformations that are not easily reducible to standardized metrics (Wendling, 2023; Osorio Cerda & Benavides Simon, 2025). Nevertheless, quality assurance systems require verifiable and comparable evidence, generating a structural tension between the production of standardized indicators and the preservation of institutional identity and public purpose.

Addressing this challenge requires distinguishing between different evaluative levels. Monitoring focuses on inputs, processes, and outputs, while impact evaluation seeks to estimate causal effects attributable to specific interventions, often through counterfactual logic (Arteaga Viveros, 2024). This distinction is critical to avoid reducing university engagement to activity-based indicators that fail to capture institutional or territorial transformation. Furthermore, engagement cannot be managed or evaluated as a parallel function; rather, it must be understood as a dimension that interacts with and shapes teaching, research, and governance processes. Consequently, the key challenge lies in developing evaluation models that integrate comparability with complexity, functional articulation with systemic coherence. Against this background, the aim of this article is to conduct a narrative review with critical analysis of the main national and international frameworks guiding the evaluation of the third mission, with particular emphasis on university engagement as a dimension of institutional accreditation. Specifically, the study seeks to problematize the normative assumptions underlying these frameworks, identify structural tensions between standardization and institutional diversity, and analyze the challenges associated with the systemic integration of engagement with teaching and research. Ultimately, the article contributes to the conceptual and methodological debate on evaluation models capable of capturing the relational and territorial complexity of university engagement without fragmenting the core functions of higher education institutions.

1.1. Background

The contemporary discussion on university engagement is part of the broader debate on the evolution of the university mission in complex societies and knowledge economies. Traditionally structured around teaching and research, the university has progressively incorporated a third dimension associated with interaction with its social, economic and cultural environment, commonly referred to as the third mission. The international literature agrees that the third mission is not a homogeneous or unequivocally defined function. Rather, it is an evolutionary construct, whose interpretation depends on the historical context, the national higher education system, and the type of institution (Compagnucci & Spigarelli, 2020). These authors point out that the concept has oscillated between approaches focused on technology transfer and broader approaches linked to civic engagement and territorial development, generating a plurality of operational definitions.

In Latin America, the debate has historically been associated with the tradition of university extension and university social responsibility, with an emphasis on the democratization of knowledge and commitment to social problems (Giménez & Bonacelli, 2018). However, the incorporation of the logic of quality assurance and international competition has stressed these approaches, introducing requirements for measurement, accountability and comparability. Consequently, the university–community engagement cannot be understood simply as an aggregation of external activities, but as a process of reconfiguration of institutional identity and its forms of production and circulation of knowledge.

1.2. Quality Assurance

The institutionalization of linkage has accelerated to the extent that quality assurance systems have broadened their focus from the evaluation of programs and academic results to the comprehensive evaluation of the institution. At the European level, the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) establish that quality should be understood as a continuous process of institutional improvement, underpinned by strategic coherence and public responsibility (ENQA et al., 2015). Although ESG does not define specific standards for the third mission, its approach implies that every university function must be evaluated in relation to the internal quality system, strategic planning, and evidence-based decision-making.

This systemic shift has been accompanied by attempts to operationalize the third mission through specific indicators. The E3M Project (2012) is a paradigmatic example of this trend, proposing structured categories to evaluate continuing education, knowledge transfer and social commitment. However, the project’s experience showed the difficulty of capturing complex relational processes through standardized metrics, especially when seeking to reconcile institutional diversity with international comparability.

In the United Kingdom, the HE-BCI system, developed by the Higher Education Statistics Agency (HESA, 2023), represents a consolidated model of quantitative measurement of university–business–community interaction. This system has made it possible to generate robust and comparable statistical series, but its emphasis on quantifiable outputs—particularly in economic transfer—has been interpreted as an evaluative rationality that privileges productive over relational dimensions. In contrast, the Carnegie Foundation for the Advancement of Teaching (2024) prioritizes the institutionalization of community engagement as a cross-cutting feature of the organization. The evaluation is based on evidence on governance, organizational culture, curricular integration, and academic recognition of engagement. The emphasis is not exclusively on measurable results, but on coherence between mission, structures, and practices.

This background shows that the field is debating between metric models oriented to comparability and institutional models focused on strategic coherence and organizational transformation.

1.3. Impact, Contribution and Evaluative Complexity

One of the most relevant debates in the literature on third mission concerns the notion of impact. Compagnucci and Spigarelli (2020) emphasize that a large part of evaluation systems operates with an implicit conception of impact associated with observable and attributable results, which is problematic in complex social contexts where the university is an actor among multiple agents. The distinction between direct impact and partial contribution becomes especially relevant in territorial processes, where changes are co-produced and depend on structural factors that exceed university action. The difficulty of attribution has led some frameworks to emphasize the institutionalization of capacities and processes, rather than the demonstration of immediate causal results (Carnegie Foundation for the Advancement of Teaching, 2024).

In addition, the temporality of insurance does not always coincide with the temporality of social change. Systems based on annual reports (HESA, 2023) tend to privilege visible outputs in the short term, while territorial transformation processes require longer time horizons. In this context, the evaluation of linkage faces an epistemological complexity: it must produce verifiable evidence without simplifying relational phenomena and without demanding causal attributions inappropriate to open social systems.

Beyond the general normative frameworks, recent literature has begun to explicitly address the methodological problem of how to evaluate complex university–community engagement actions, particularly those that involve knowledge co-production, multi-stakeholder alliances, and territorial transformation processes. One of the most systematic contributions in this line is the work of Wanjiru and Liu (2021), who propose an evaluation framework structured in three analytical domains: purpose, process, and community impact. Based on a literature review on community engagement, the authors argue that evaluating only quantifiable results is insufficient to capture the complexity of the university–community relationship. In its proposal, the analysis should consider:

(a)

The coherence between the commitment and the institutional mission (strategic purpose).

(b)

The quality of relational processes, including reciprocity and co-production.

(c)

The effects on the community, understood not only as measurable results, but as transformations in capacities and relationships.

This approach is especially relevant for complex actions, where causality is distributed and results emerge from prolonged interactions between diverse actors. In the British context, Hart and Northmore (2011) analyzed experiences of auditing and evaluating university engagement, concluding that traditional metrics focused on outputs fail to capture key dimensions such as collaborative governance, the sustainability of partnerships, and the quality of reciprocity. These authors propose that evaluation should incorporate criteria related to the institutional capacity to sustain long-term relationships, as well as the perception and active participation of external actors in evaluation processes.

In a similar vein, Spânu et al. (2024), in a study on the evaluation of community engagement in European higher education, highlights that sustainable and strategically integrated practices require evaluation frameworks that combine quantitative indicators with qualitative evidence on collaborative processes and institutional learning. The author stresses that the complexity of community engagement requires flexible tools, capable of adapting to different institutional configurations without imposing a single model.

In addition, research on community-engaged research evaluation has shown difficulties even in the field of internal academic promotion. Wendling (2023) shows that evaluation committees face challenges in categorizing and valuing community-engaged research when traditional scientific productivity metrics are applied. This finding is significant because it reveals that evaluative complexity is not limited to the external relationship with the environment, but also crosses the academic structures themselves.

These studies provide three relevant conceptual contributions to the evaluation of the university–community engagement:

(a)

Evaluative multidimensionality: complex actions require analyzing purpose, process, and impact in a differentiated but articulated way (Wanjiru & Liu, 2021).

(b)

Centrality of reciprocity and co-production: the quality of the relationship is as relevant as the observable results (Hart & Northmore, 2011).

(c)

Need for mixed evidence and extended temporality: transformative effects are usually deferred and co-produced, so the evaluation must incorporate qualitative evidence and broad time horizons (Spânu et al., 2024; Wendling, 2023).

In this way, the evaluation of the university engagement cannot be limited to accounting indicators or annual metrics (Wendling, 2023). Rather, it requires frameworks that integrate strategic, relational, and transformative dimensions, and that recognize the systemic and territorial nature of university actions.

In the Chilean context, where the university engagement has been incorporated as a mandatory dimension of institutional accreditation (CNA, 2022), these contributions acquire special relevance. The requirement to demonstrate bidirectional impact and articulation with teaching and research demands evaluative devices capable of capturing complex actions without reducing them to immediate outputs. The international literature on engagement evaluation therefore offers key conceptual inputs to enrich the national discussion and avoid an excessively metric reduction of the function.

In Chile, the international discussion on third mission and quality has been reflected in legislation through Law No. 21.091 (2018) on Higher Education, which establishes the obligation to accredit the function of university–community engagement. In compliance with this regulation, the National Accreditation Commission explicitly incorporated this dimension in its Criteria and Standards for Institutional Accreditation (CNA, 2022). Official documentation emphasizes that quality must be understood as a permanent and systematic process, evaluated in relation to the institutional project and respecting the diversity and autonomy of the institutions (CNA, 2025a). It is also required to demonstrate formal policies, bidirectionality mechanisms, impact evaluation, and articulation with teaching and research (CNA, 2022).

Recent studies on linkage trajectories in Chilean institutions indicate that regulatory formalization has driven the development of organizational structures, monitoring systems, and strategic frameworks; however, they have also identified challenges associated with impact measurement, coherence between declaration and implementation, and effective integration with substantive functions (CNA, 2025b).

The mandatory incorporation of linkage as an evaluative dimension has strengthened its institutional visibility, but has also intensified the need to clarify its conceptual foundation and systemic integration.

We can therefore outline four conceptual axes that structure the current debate on linkage with the environment:

Conceptual plurality of the third mission: there is no single definition or universally accepted framework (Compagnucci & Spigarelli, 2020).

Tension between comparability and institutional diversity: insurance systems require homogeneous standards, but linkages are configured according to specific missions and territories (CNA, 2022; Carnegie Foundation for the Advancement of Teaching, 2024).

Impact measurement challenges in complex systems: causal attribution is limited and the temporality of social change exceeds short evaluative cycles (HESA, 2023).

Need for systemic integration: the most robust frameworks emphasize articulation with teaching and research, avoiding functional fragmentation (ENQA et al., 2015; IAU, 2022).

Thus, the university engagement cannot be reduced to the expansion of indicators, but requires a theoretical reflection on its place in the institutional architecture of quality.

2. Materials and Methods

2.1. Study Design

This study adopts a narrative review design with comparative analysis of a conceptual-interpretative nature, aimed at critically examining the regulatory and evaluative frameworks that structure university engagement as a dimension of quality in higher education.

Narrative review is relevant when the objective is not to synthesize empirical results through meta-analysis or systematic review protocols, but to analyze conceptual developments, normative devices, and evaluative rationalities that make up a field of public policy (Snyder, 2019). In this case, the interest lies in identifying the epistemological and organizational assumptions that underlie the assessment models of university engagement, as well as the structural tensions that emerge from their international comparison.

The approach adopted is analytical–comparative, rather than descriptive, and is aimed at constructing interpretative categories to understand the evaluative complexity of the function. Accordingly, the study follows a conceptual and documentary design based on the interpretative comparison of evaluative frameworks.

Construction of the documentary corpus: The corpus was structured on three levels:

(a)

National regulatory framework

Law No. 21.091 (2018) on Higher Education.

Criteria and Standards for Institutional Accreditation of the National Accreditation Commission (CNA, 2022).

Institutional Accreditation Regulations.

Technical documents and studies on UCE trajectories in Chilean institutions.

These documents were selected for their binding nature and for constituting the formal framework that regulates the evaluation of UCE in Chile.

(b)

International quality assurance frameworks

Standards and Guidelines for Quality Assurance in the European Higher Education Area (ENQA et al., 2015).

Framework of the International Association of Universities (IAU, 2022).

These frameworks were selected for their influence on global assurance policies and for promoting a systemic conception of institutional quality.

(c)

Specific models of third-mission evaluation and engagement

E3M Project (2012).

HE-BCI (HESA, 2023).

Carnegie Foundation for the Advancement of Teaching (2024).

Academic literature on complex action assessment (Wanjiru & Liu, 2021; Hart & Northmore, 2011; Wendling, 2023).

The selection responded to its international relevance, methodological explicitness and diversity of evaluative approaches (metric, institutional and systemic).

2.2. Derivation and Conceptual Grounding of Analytical Categories

The analytical categories used in this study were not predefined as a fixed typology but were constructed through an iterative and abductive process that combined theoretical grounding and comparative reading of the documentary corpus. Specifically, the derivation of categories was informed by two complementary bodies of literature.

(a)

First, the conceptualization of the third mission as a heterogeneous and context-dependent construct (Compagnucci & Spigarelli, 2020) provided a basis for identifying dimensions related to the definition of the function, its institutional role, and its articulation with teaching and research. This perspective emphasizes that evaluation frameworks are not neutral instruments but embed normative assumptions about the university mission.

(b)

Second, the literature on the evaluation of complex university–community engagement actions (Wanjiru & Liu, 2021; Hart & Northmore, 2011) contributed to structuring the analytical dimensions around purpose, process, and impact, as well as issues of temporality, evidence, and stakeholder participation. These approaches highlight the limitations of purely output-based evaluation and the need to incorporate relational and processual dimensions.

Based on these theoretical inputs, an initial set of analytical categories was defined, including: conception of the function, notion of impact, evaluative temporality, type of evidence, level of aggregation, integration with teaching and research, participation of external actors, and organizational incentives. These categories were subsequently refined through iterative comparison across the selected frameworks, allowing for the identification of convergences, divergences, and underlying evaluative rationalities. This procedure ensured that the analytical framework was both theoretically grounded and empirically sensitive to the diversity of evaluation models, enhancing the interpretative validity of the study.

2.3. Analytical Strategy

The analysis was developed in three iterative phases:

Phase 1: Identification of conceptual categories

Based on the theoretical framework and the literature review on third mission, initial categories related to the following were identified:

(a)

Conception of the function.

(b)

Notion of impact.

(c)

Evaluative temporality.

(d)

Type of evidence accepted.

(e)

Aggregation level.

(f)

Integration with teaching and research.

(g)

Participation of external actors.

(h)

Organizational incentives.

These categories emerged both from the theoretical debate (Compagnucci & Spigarelli, 2020) and from the literature on the evaluation of complex actions (Wanjiru & Liu, 2021).

Phase 2: Comparative Document Coding

(a)

Each frame was analyzed according to the defined categories, recording:

(b)

Explicit definitions.

(c)

Evaluation requirements.

(d)

Suggested indicators.

(e)

Temporality of the quality cycle.

(f)

Implicit assumptions about impact and causality.

Phase 3: Identification of structural stresses by cross-sectional comparison, three recurrent analytical stresses were identified:

(a)

Standardization versus institutional diversity.

(b)

Functional reduction versus systemic cross-sectionality.

(c)

Fragmented evaluation versus institutional integration.

These tensions were not predetermined, but emerged from the comparative reading of the frameworks.

2.4. Criteria of Rigor and Interpretative Validity

Given the conceptual nature of the study, validity is not based on statistical replicability, but on the internal coherence between theoretical categories and documentary analysis; the transparency of the corpus and explicitness of selection criteria, conceptual triangulation, by integrating academic and normative literature and comparative consistency, ensuring that each framework was analyzed under the same criteria.

The procedure described allowed the construction of a comparative analytical matrix, organized around nine criteria derived from the conceptual framework and the international literature on third mission and evaluation of complex actions. The purpose of this matrix was not to hierarchically classify the models, but to identify regularities, divergences and normative assumptions that structure the evaluation of the relationship with the environment in different contexts.

The comparative analysis showed that the revised frameworks differ not only in technical instruments, but also in underlying evaluative rationalities: some privilege comparability through quantifiable indicators, others emphasize the institutionalization of commitment and strategic coherence, while others promote systemic integration into the institutional cycle of quality.

From the cross-sectional comparison, three structural tensions emerged that organize the results of the study:

(a)

Standardization versus institutional diversity, related to the degree to which the models promote organizational convergence or allow contextual adaptation.

(b)

Functional reduction versus systemic transversality, associated with the way in which engagement is conceived as a differentiated activity or as an integrated dimension in teaching and research.

(c)

Fragmented evaluation versus institutional integration, linked to the place that the function occupies in the quality cycle and in the architecture of university governance.

These tensions were not imposed a priori, but emerged from the iterative analysis of the documentary corpus and its contrast with the literature on the evaluation of complex university engagement actions. Consequently, the results presented below do not constitute a description of models, but a comparative interpretation of the evaluative configurations that currently structure the evaluation of the engagement in higher education.

3. Results

The comparative analysis of regulatory frameworks and international evaluation models allowed us to identify structural patterns in the way in which the University engagement is conceptualized and evaluated as a dimension of institutional quality. Rather than describing specific instruments, this section examines the underlying evaluative configurations that emerge from the systematic comparison of the documentary corpus. The analytical matrix presented in

Table 1. Comparison of international frameworks and the Chilean model for evaluating the link with the environment.

Comparison Criteria	CNA (Chile)	ESG (Europe)	Carnegie CE (USA)	HE-BCI (UK)	E3M (Europe)	IAU/Whole Institution
Conception of the function	Mandatory dimension of institutional accreditation	Non-specific; integrated into systemic QA	Institutionalized Community Engagement	University–business–community interaction	Third Differentiated Mission	Comprehensive institutional commitment
Predominant evaluative approach	Mixed (indicators + qualitative evidence)	Processual and systemic	Contextualized institutionalization	Metric-quantitative	Structured indicators	Strategic-integrative
Type of impact considered	Required bidirectional impact, without explicit typology	Institutional continuous improvement	Institutional and community transformation	Quantifiable results (outputs)	Categorized results	Contribution to sustainable development
Time horizon	Accreditation cycle (4–7 years)	Continuous evaluation	Cumulative Assessment	Annual Report	Variable according to indicators	Medium and long term
Aggregation level	Institutional + operational evidence	Institutional	Institutional and cultural	Quantified institutional aggregate	Specific dimensions	Transversal institutional
Type of Evidence Accepted	Indicators + reports + formal mechanisms	Documentary and systemic	Strong qualitative component	Standardized quantitative data	Quantified indicators	Integrated strategies and policies
Participation of external actors in evaluation	Required in principle (bidirectionality)	Not explicit	Explicit and structural	Limited	Non-central	Relevant in strategic alliances
Articulation with teaching and research	Declared as a requirement	Implicit in Systemic QA	Explicit transversal	Not necessarily integrated	Variable	Central in institutional approach
Identified main risk	Operational standardization	Overgeneralization	Evaluative subjectivity	Reduction to economic metrics	Simplification by ranking	Conceptual breadth difficult to measure

Source: own elaboration.

synthesizes the main criteria for comparison—conception of the function, evaluative logic, notion of impact, temporality, functional integration, and type of evidence—and constitutes the starting point for the interpretation that follows. From this matrix, the aim is not to establish hierarchies between models, but to identify structural tensions that organize the international field of evaluation of the third mission.

The analysis reveals that the different frameworks do not differ only in technical procedures, but also in evaluative rationalities that configure the place of engagement within the institutional architecture. These rationalities can be grouped around three main tensions:

-

Standardization versus institutional diversity, associated with the balance between regulatory comparability and contextual adequacy.

-

Functional reduction versus systemic transversality, linked to the understanding of engagement as a differentiated function or as an integrated dimension to teaching and research.

-

Fragmented evaluation versus institutional integration, related to the way in which engagement is inserted in the quality cycle and in university governance systems.

The following subsections develop each of these tensions, articulating the comparative analysis with the literature on evaluation of complex actions and quality assurance. In this way, the results are not presented as a normative description of models, but as a structural interpretation of contemporary evaluative configurations.

3.1. Emergent Evaluative Configurations in the Evaluation of University Engagement

The comparative analysis of the corpus allowed us to identify not only instrumental differences between regulatory frameworks, but also coherent evaluative configurations that respond to different rationalities regarding the place of the university engagement in institutional quality. These configurations are summarized in Table 1 and are organized around three emerging structural tensions: (1) standardization versus institutional diversity, (2) functional reduction versus systemic cross-cutting, and (3) fragmented evaluation versus institutional integration.

Table 1 is not a closed typology, but an analytical device that allows us to visualize how each framework combines conception of the function, evaluative model, notion of impact, temporality, and type of evidence. The revised frameworks respond to different evaluative rationalities and produce differentiated organizational incentives (CNA, 2022; ENQA et al., 2015; Carnegie Foundation for the Advancement of Teaching, 2024; HESA, 2023; E3M Project, 2012; IAU, 2022). A synthesis developed for the Chilean context reinforces this comparison, identifying five recurring domains in instruments such as HE-BCI, E3M and the Valencia Manual: technology transfer, continuing education, social or community services, cultural extension and political-administrative engagement. This convergence suggests that standardization can operate at the level of shared analytical dimensions, without preventing each institution from incorporating additional domains according to its mission and territorial context (Fleet et al., 2017).

3.2. First Tension: Standardization Versus Institutional Diversity

The comparative analysis shows that evaluation frameworks oriented toward comparability tend to structure engagement assessment around standardized indicators, thereby facilitating measurement, benchmarking, and accountability. However, this standardization introduces pressures toward organizational convergence that may limit institutional diversity. For instance, the HE-BCI system (HESA, 2023) operationalizes university engagement primarily through quantifiable outputs such as income from knowledge transfer, number of contracts, consultancy activities, and partnerships with external actors. These indicators are reported annually and aggregated at the institutional level, reinforcing a model of evaluation that privileges measurable and comparable outputs. While this approach enables longitudinal tracking and policy-level analysis, it also tends to prioritize forms of engagement that are economically codifiable, potentially marginalizing relational, cultural, or territorially embedded practices that are less easily quantifiable.

A similar dynamic can be observed in the Chilean framework established by the National Accreditation Commission (CNA, 2022). Although the CNA explicitly recognizes institutional diversity, it requires evidence of formal policies, bidirectional engagement mechanisms, impact assessment, and articulation with teaching and research. In practice, these requirements have led many institutions to develop analogous organizational arrangements—such as centralized engagement units, standardized monitoring systems, and formal reporting structures—suggesting that regulatory enforceability generates incentives toward structural homogenization. In contrast, the Carnegie Foundation for the Advancement of Teaching (2024) adopts a different evaluative logic. Rather than imposing uniform indicators, it assesses the coherence between institutional mission, governance structures, and engagement practices. Evidence is largely qualitative and context-sensitive, allowing institutions to demonstrate engagement according to their specific identity and territorial context. This approach reduces the pressure toward convergence but limits direct comparability across institutions.

These models illustrate that standardization functions as a mechanism of comparability and accountability, but may simultaneously constrain the expression of institutional diversity. The tension lies in balancing the need for shared evaluative criteria with the preservation of contextually grounded and mission-specific forms of engagement.

3.3. Second Tension: Functional Reduction Versus Systemic Transversality

A second structural tension concerns the way in which the university engagement is conceptually positioned within the institutional architecture: either as a differentiated function or as a transversal dimension integrated with teaching and research. Frameworks oriented toward measurement and operationalization tend to treat engagement as a distinct functional domain. For example, the E3M Project (2012) organizes the third mission into specific categories—continuing education, knowledge transfer, and social engagement—each associated with particular indicators. While this categorization facilitates evaluation and reporting, it may also contribute to a functional segmentation in which engagement is managed and assessed as a separate institutional activity, rather than as an element embedded across core academic processes.

Similarly, in metric-oriented systems such as HE-BCI (HESA, 2023), engagement is captured through discrete outputs linked to external interaction, without necessarily requiring explicit articulation with teaching or research activities. This reinforces an understanding of engagement as an add-on function, often aligned with economic or service-oriented outputs. In contrast, systemic quality assurance frameworks such as the ESG (ENQA et al., 2015) and the “whole institution” approach promoted by the IAU (2022) conceptualize institutional quality as the result of the articulation between all university functions. From this perspective, engagement should not be evaluated as an isolated domain, but as a dimension that shapes curricular relevance, research agendas, and governance processes. The Carnegie Foundation for the Advancement of Teaching (2024) similarly requires evidence of how community engagement is embedded in teaching, research, and institutional decision-making, emphasizing its transversal character.

The Chilean CNA model formally aligns with this systemic perspective by requiring articulation between UCE, teaching, and research (CNA, 2022). However, the structure of accreditation processes—organized into differentiated dimensions—may inadvertently reinforce a functional logic, as institutions often report engagement under a specific category rather than demonstrating its transversal integration. This tension reveals that the way engagement is evaluated has performative effects on institutional organization. When treated as a separate function, it risks being operationally reduced to extension activities or external services. Conversely, when approached as a transversal dimension, it requires more complex evaluative frameworks capable of capturing its influence across the entire institutional system.

3.4. Third Tension: Fragmented Evaluation Versus Institutional Integration

A third tension emerges in relation to the degree to which evaluation frameworks promote integrated or fragmented approaches to institutional quality. This tension is closely linked to the architecture of quality assurance systems and the role assigned to engagement within them. Frameworks that rely on segmented evaluation structures tend to assess university functions independently, which can lead to fragmented representations of institutional performance. In the Chilean case, although the CNA (2022) explicitly promotes articulation between engagement, teaching, and research, the accreditation model is organized into distinct dimensions that are evaluated separately. As a result, institutions often produce differentiated evidence for each function, which may obscure the interdependencies between them and reinforce a compartmentalized understanding of quality.

This dynamic is consistent with findings in the literature on university–community engagement evaluation, which show that audit systems frequently prioritize discrete activities and outputs over relational processes and collaborative governance (Hart & Northmore, 2011). Similarly, research on academic evaluation indicates that community-engaged practices are often assessed using conventional metrics, leading to tensions between institutional recognition systems and the complex nature of engagement (Wendling, 2023). In contrast, systemic frameworks such as ESG (ENQA et al., 2015) emphasize continuous improvement based on strategic coherence across all institutional functions. From this perspective, evaluation is not organized around isolated dimensions but around the capacity of the institution to align its mission, processes, and outcomes. The IAU (2022) reinforces this approach by promoting the integration of engagement, sustainability, teaching, and research within a unified institutional strategy.

The Carnegie Foundation for the Advancement of Teaching (2024) also advances an integrative logic by requiring institutions to demonstrate how engagement is embedded in governance, curriculum, and academic culture, rather than assessing it as a standalone function. This tension highlights that evaluation frameworks do not merely measure institutional performance but actively shape how institutions organize their activities. Fragmented evaluation models may simplify assessment processes but risk overlooking the systemic nature of engagement. Conversely, integrative approaches demand more complex forms of evidence but offer a more accurate representation of how universities generate public value.

4. Discussion

The purpose of this study was to critically analyze the evaluative rationalities that structure the assessment of the university engagement within quality assurance frameworks, with particular attention to the Chilean case in relation to international models. The findings demonstrate that the evaluation cannot be reduced to a technical problem of indicator construction; rather, it constitutes a structural issue embedded in institutional governance and the architecture of quality systems. The comparative analysis identified three structural tensions—standardization versus institutional diversity, functional reduction versus systemic transversality, and fragmented evaluation versus institutional integration—that organize the contemporary field of third mission evaluation. These tensions are not specific to the Chilean case but reflect broader dynamics documented in the international literature (Compagnucci & Spigarelli, 2020; ENQA et al., 2015).

Importantly, these challenges are not exclusive to engagement. Similar issues—such as attribution, temporal lag, and context sensitivity—are also present in the evaluation of teaching and research. However, in the case of university engagement, these complexities become more visible due to its relational, co-produced, and territorially embedded nature. This reinforces the need to move toward integrated institutional evaluation frameworks rather than function-specific metrics.

Moreover, the international comparison shows that evaluation systems not only measure existing practices, but also contribute to shaping them organizationally. Along these lines, the Chilean experience shows that bidirectionality can be translated into verifiable organizational devices (Fleet et al., 2017). This finding dialogues with studies on the institutionalization of community engagement, which highlight how evaluation frameworks influence academic structures, incentives, and cultures (Carnegie Foundation for the Advancement of Teaching, 2024). In the Chilean context, the mandatory incorporation of university engagement as an accreditation dimension under the regulation of the National Accreditation Commission (CNA, 2022) has strengthened its institutional visibility, but has also introduced pressures towards formalization and standardization.

From a systemic assurance perspective, the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ENQA et al., 2015) argue that institutional quality is based on strategic coherence and continuous improvement. In this framework, engagement should not be evaluated as a functional appendix, but as an integrated dimension of the institutional project. However, the organization of accreditation processes into differentiated dimensions can induce evaluative segmentation, even when the normative intention is integrative. This issue is closely linked to the tension between fragmented evaluation and institutional integration, where segmented accreditation structures may obscure the systemic articulation of university functions. In turn, this dynamic also reflects the tension between functional reduction and systemic transversality, as engagement is often operationalized as a discrete function despite its inherently cross-cutting nature.

A second axis of discussion refers to the balance between comparability and institutional diversity. The literature on third mission warns that the adoption of standardized frameworks can promote dynamics of organizational convergence, particularly when national systems establish homogeneous standards (Compagnucci & Spigarelli, 2020). The HE-BCI system (HESA, 2023) represents a highly comparability model based on quantifiable outputs, mainly in economic transfer. While this logic strengthens accountability, it can also induce prioritization of easily measurable activities. Studies on third-mission evaluation in European contexts have indicated that rankings and indicator systems can simplify the complexity of social engagement (E3M Project, 2012).

In contrast, the Carnegie classification privileges contextual coherence and qualitative evidence of institutionalization (Carnegie Foundation for the Advancement of Teaching, 2024). This approach reduces the risk of over-homogenization, although it limits direct comparability. In the Chilean case, the CNA (2022) emphasizes respect for institutional diversity; however, regulatory enforceability may generate incentives towards similar structures of university engagement. This tension confirms that evaluation must balance common criteria with explicit recognition of territorial and missional specificity.

This finding directly reflects the tension between standardization and institutional diversity, as evaluation frameworks that prioritize comparability tend to promote organizational convergence while constraining context-sensitive expressions of engagement.

4.1. Evaluation of Complex Actions: Purpose, Process and Impact

Recent literature on university–community engagement evaluation offers relevant conceptual tools to address this complexity. Wanjiru and Liu (2021) propose to simultaneously assess purpose, process, and impact, stressing that the quality of engagement cannot be inferred solely from quantifiable outcomes. Likewise, Hart and Northmore (2011) argue that the audit of university commitment should consider collaborative governance, reciprocity, and institutional sustainability of relationships. These dimensions are particularly relevant in contexts where engagement is developed through long-term territorial alliances.

At the European level, Spânu et al. (2024) emphasizes that sustainable engagement practices require evaluation frameworks that combine quantitative indicators with qualitative evidence on collaborative processes and institutional learning. This perspective reinforces the need to go beyond exclusively metric models.

In addition, research on the evaluation of engaged research shows that traditional academic promotion systems face difficulties in valuing community practices when conventional scientific productivity metrics are applied (Wendling, 2023). This finding shows that evaluative complexity is not limited to the external dimension, but also crosses the internal structures of academic recognition. The assessment of university engagement must explicitly acknowledge the co-produced, relational and temporally extended nature of its results.

The discussion on impact is one of the most critical points. While some systems operate with annual horizons and verifiable outputs (HESA, 2023), others privilege cumulative institutionalization processes (Carnegie Foundation for the Advancement of Teaching, 2024).

Compagnucci and Spigarelli (2020) warn that the direct attribution of social changes to university action is methodologically problematic in complex systems. In this sense, the distinction between impact and contribution acquires conceptual relevance. Evaluating contribution implies recognizing that the university is an actor within multi-stakeholder networks, where the effects emerge from co-production processes.

In the Chilean context, the CNA (2022) requires evidence of bidirectional impact, which represents an advance in terms of conceptual demand. However, the absence of explicit typologies of impact can generate disparate interpretations in their implementation.

Therefore, a robust evaluation model should distinguish between:

(a)

Immediate results (verifiable outputs).

(b)

Intermediate changes (capacities, networks, institutionalization).

(c)

Long-term contributions (co-produced territorial transformations).

(d)

This differentiation would allow aligning evaluative temporality with social complexity.

Finally, based on comparative analysis and dialogue with the literature, it is possible to propose five guiding principles for a more coherent assessment of UCE:

(a)

Contextual adequacy, recognizing institutional diversity.

(b)

Functional integration, articulating UCE with teaching and research.

(c)

Distinction between impact and contribution, with differentiated temporality.

(d)

Mixed and multi-stakeholder evidence, incorporating community perspective.

(e)

Evaluation as an institutional learning process, not just accountability.

These principles are aligned with systemic quality approaches (ENQA et al., 2015; IAU, 2022) and with models of institutionalization of commitment (Carnegie Foundation for the Advancement of Teaching, 2024), offering a conceptual framework that can enrich the Chilean debate.

The main contribution of this article is to show that the evaluation of the relationship with the environment should be understood as a problem of quality institutional architecture, where comparability, relational complexity and university governance intersect. By situating the Chilean case in dialogue with international frameworks, the study shows that the tensions identified are not contingent or local, but structural to the field of the third mission.

This approach allows us to transcend the instrumental discussion on indicators and move towards a conceptual reflection on the place of linkage in the contemporary university mission.

4.2. Impact, Temporality, and Epistemological Complexity in the Evaluation of Engagement

A cross-cutting issue emerging from the analysis concerns the epistemological complexity associated with evaluating the impact of the university–community engagement. Unlike teaching and research—where evaluation has historically been structured around more stabilized indicators—engagement involves relational, co-produced, and context-dependent processes whose effects unfold over extended temporal horizons. Metric-oriented frameworks tend to privilege short-term and observable outputs, often reported within annual or accreditation cycles (HESA, 2023; CNA, 2022). However, as highlighted in the literature, many of the most significant effects of engagement—such as capacity building, institutional learning, and territorial transformation—are cumulative and cannot be fully captured within these temporal frames (Compagnucci & Spigarelli, 2020; Wanjiru & Liu, 2021).

This temporal mismatch introduces a fundamental epistemological tension: evaluation systems require verifiable and attributable evidence, while engagement processes are characterized by distributed causality and co-production among multiple actors. In this context, the distinction between impact and contribution becomes analytically crucial. Rather than attributing outcomes exclusively to university actions, evaluation should account for the university’s role within broader socio-territorial processes.

Recognizing this complexity reinforces the need for evaluation models that incorporate differentiated temporalities and mixed forms of evidence, aligning measurement practices with the nature of the phenomena being assessed.

4.3. Towards a Systemic Model for Evaluating the University–Community Engagement

Based on the comparative analysis and the identified tensions, this study proposes a systemic evaluation model that seeks to integrate comparability requirements with the relational and contextual complexity of university engagement. The model is structured around three interrelated analytical dimensions:

(a)

Strategic purpose: the degree of alignment between engagement activities and the institutional mission, including their articulation with teaching and research priorities. This dimension evaluates whether engagement is conceived as a peripheral function or as a core component of the institutional project.

(b)

Relational processes: the quality of interaction between the university and external actors, including reciprocity, co-production of knowledge, sustainability of partnerships, and participation in decision-making processes. This dimension emphasizes that the value of engagement lies not only in its outputs but in the configuration of collaborative relationships.

(c)

Contribution and impact: the effects of engagement understood across differentiated temporalities, including immediate outputs, intermediate changes (such as capacity building and institutional learning), and long-term contributions to territorial transformation. This dimension explicitly distinguishes between attribution and contribution, recognizing the distributed causality of social change processes.

These three dimensions are embedded within the institutional cycle of continuous improvement and must be evaluated in articulation with teaching and research, avoiding functional fragmentation. The model thus proposes a shift from isolated measurement of activities toward an integrated evaluation of institutional coherence, relational quality, and transformative contribution.

These dimensions should not be understood as independent components but as an integrated evaluative architecture. The model operates through the articulation of:

(1)

Strategic alignment (purpose);

(2)

Relational quality (process);

(3)

Differentiated effects (contribution and impact).

All embedded within the institutional quality cycle. This configuration enables the simultaneous assessment of coherence, interaction, and transformation, thereby overcoming the limitations of fragmented and purely indicator-based approaches.

4.4. Limitations

This study has several limitations inherent to its narrative and conceptual design. First, the selection of documents, although theoretically informed and transparent, involves interpretative decisions that may influence the comparative analysis. Second, the study does not aim to exhaustively cover all existing frameworks but to analyze those most influential in shaping evaluative rationalities. Third, as a documentary analysis, the findings do not capture how these frameworks are implemented in practice at the institutional level. Future research could complement this approach through empirical studies examining how universities operationalize these evaluation models in specific contexts.