Next Article in Journal
Format-Free Submissions in Psychology-Related Journals
Previous Article in Journal
ISO Standards in Healthcare Organizations: Research Evolution and Trends from a Bibliometric Analysis
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluating Policy Efficacy in Higher Education: A Synthetic Control Analysis of Ecuador’s Higher Education Law on Research Productivity

by
Patricio Álvarez-Munoz
1,
Marco Faytong-Haro
2,3,*,
Dennis Alfredo Peralta Gamboa
2,
Angelo Marcos Aviles Valenzuela
2 and
Fernando Pacheco-Olea
4
1
Facultad Social Sciences, Business Education and Law, Universidad Estatal de Milagro, Milagro 091050, Ecuador
2
Facultad de Investigación, Universidad Estatal de Milagro, Milagro 091050, Ecuador
3
Ecuadorian Development Research Lab, Daule 090656, Ecuador
4
Facultad de Educación (FACE), Universidad Estatal de Milagro, Milagro 091050, Ecuador
*
Author to whom correspondence should be addressed.
Publications 2024, 12(3), 28; https://doi.org/10.3390/publications12030028
Submission received: 23 May 2024 / Revised: 6 August 2024 / Accepted: 9 September 2024 / Published: 15 September 2024

Abstract

:
Background: This paper investigates the impact of Ecuador’s Ley Orgánica de Educación Superior (LOES), enacted in 2010, which mandated comprehensive reforms in higher education. Objective: The objective of this study is to determine whether the implementation of the LOES has led to a significant increase in academic publications in Ecuador, thereby contributing to sustainable educational practices and aligning with the Sustainable Development Goals (SDGs). In this context, sustainability refers to the ongoing capacity of higher education institutions to generate research that contributes to the long-term development of knowledge, aligns with global sustainability goals, and ensures educational practices that are responsive to societal needs. Methods: Using a synthetic control method, this study evaluates the impact of the LOES on the number of academic publications recorded in Scopus. A synthetic control consisting of data from countries with similar educational and economic profiles serves as a counterfactual to Ecuador’s situation. Results: Before the LOES, the average annual number of publications was stable at 253.4 per year. Post-2010, this number surged, reaching an average of 3645.84 publications per year, representing a 358.41% increase. By 2023, the actual number of publications in Ecuador rose to approximately 6584, significantly surpassing the synthetic control estimate of 1431 publications. Conclusions: The findings confirm that the LOES has been effective in increasing research output and advancing educational sustainability in Ecuador. This analysis not only contributes to the literature on the impact of educational policies but also demonstrates the utility of synthetic control methods in assessing policy effectiveness, underscoring the potential of targeted interventions to promote academic excellence and sustainability in education.

1. Introduction

Scientific publications play a crucial role in promoting sustainability in education by facilitating the continuous flow of up-to-date knowledge and innovative practices into the curriculum and institutional strategies [1,2,3]. This dynamic infusion of research ensures that educational content remains relevant and reflective of the latest academic and industry advancements [1,4], thereby preparing students to contribute effectively to sustainable development [5]. This continuous integration of research into educational practices ensures not only academic relevance but also supports long-term educational and institutional sustainability [6,7].
The transformative impact of educational policies on research productivity, as observed in various studies, is instrumental in sustaining this flow of knowledge [8]. This relationship can be understood through the lens of policy feedback theory, which suggests that educational policies reshape institutional norms and practices, thereby sustaining research productivity over time [9,10,11]. For instance, reforms that enhance university governance—such as increasing autonomy and improving funding mechanisms—have been shown to significantly boost the quantity and quality of research outputs [12,13]. However, the long-term effectiveness of these reforms requires continuous evaluation to ensure that they not only produce immediate results but also create lasting improvements in research capacity. These improvements create a more competitive and innovative academic environment, which is critical for nurturing a culture of sustainability within educational institutions [12,14]. Sustainability here extends beyond environmental aspects to include the institutional and social sustainability of educational practices, ensuring that the growth in research output is both enduring and adaptable [15,16].
Moreover, policies that mandate publications in accredited journals for academic progression drive researchers to contribute and engage with cutting-edge research, which in turn enriches the academic community [17]. This mechanism not only enhances immediate research outputs but also builds a foundation for ongoing academic excellence and innovation, key components of sustainable education [18]. This cycle of research and publication strengthens the foundation of knowledge upon which sustainable educational practices are built, ensuring that institutions not only keep pace with global advancements but also contribute to the sustainable development of their local and global communities [19]. Improved research productivity contributes to sustainable development by ensuring that academic institutions generate knowledge that is not only cutting-edge but also relevant to addressing global challenges [20,21]. When educational reforms boost institutions’ research output, they promote ongoing learning and innovation, which helps achieve long-term social, economic, and environmental goals [22,23,24]. This aligns with the principles of sustainability, where the growth in research capacity directly contributes to societal resilience and the ability to adapt to future challenges [25]. By embedding sustainability into the core objectives of higher education, these reforms ensure that the benefits of increased research productivity are not limited to immediate academic gains but also extend to broader societal impacts [26]. For example, studies have shown that institutions with strong research outputs are better equipped to adapt to changing educational demands and contribute to long-term societal goals [27,28]. Sustainability, in this context, is not limited to environmental concerns but extends to the institutional and social sustainability of educational practices, ensuring that the growth in research output supports long-term educational and societal goals [6].
Previous research has underscored the significant influence of education policies on research productivity. A study by Aghion, Dewatripont, Hoxby, Mas-Colell, and Sapir explored the governance and performance of research universities across Europe and the United States [29]. Their findings revealed that reforms designed to enhance university governance—by increasing autonomy and boosting funding mechanisms—substantially improved both the volume and caliber of scientific outputs. The authors noted that improved governance structures cultivate a more competitive and innovative research environment, thereby enhancing scientific productivity.
Further, universities have implemented strategies such as establishing dedicated publication teams and leveraging online journal systems to streamline and oversee publication processes, thus elevating both the quality and quantity of research [30,31,32]. These changes have also aimed to make universities more competitive and autonomous, leading to diverse effects on publication practices based on the institutions’ capabilities before the reforms [33].
Crespi and Zuniga [34] addressed this by examining the relationship between innovation and productivity in six Latin American countries, demonstrating that policy reforms enhance research development activities. Despite these insights, there remains a gap in understanding the specific impact of integral policies and reforms on scientific publication rates in developing countries.
To address this gap, this study employs a synthetic control method to rigorously evaluate the long-term impacts of the Organic Law of Higher Education (LOES) reforms in Ecuador. The LOES, enacted in 2010, aimed to transform the higher education landscape in the country by improving the quality of education, promoting equity and inclusion, and enhancing the research capabilities of higher education institutions [35]. The key goals of the LOES included establishing a robust regulatory framework for higher education, promoting academic excellence, ensuring the financial sustainability of educational programs, and facilitating greater access to higher education for all segments of the population.
Ecuador’s approach is unique because the LOES was one of the first comprehensive higher education reforms in the region to mandate specific budget allocations for research and enforce strict accreditation standards [36]. This rigorous and legally binding framework sets Ecuador apart, providing valuable insights into how structured policy interventions can lead to sustained improvements in research output. Conducting such evaluations is crucial to tailor policy interventions that effectively promote long-term scientific and educational progress, offering lessons for other developing countries.
One of the significant benefits of the LOES was the mandatory allocation of resources for research. This mandatory allocation of resources was intended to be a critical step in ensuring the sustainability of research activities over time, fostering a stable and supportive environment for academic growth. According to Article 36, higher education institutions were required to allocate at least six percent (6%) of their annual budgets to indexed publications, postgraduate scholarships for professors, and research activities. This was the first time universities were obligated to invest a specific portion of their budget directly into research [18].
Additionally, the LOES led to the establishment of public institutes such as the Council for Evaluation, Accreditation, and Quality Assurance in Higher Education (CEAACES). CEAACES began auditing universities according to rigorous standards not previously seen in Ecuador. These audits included accreditation and categorization of universities, pushing them to improve various outputs, including scientific publications in high-impact journals, the number of full-time professors, and the overall quality of education and research [37]. This process ensured that institutions adhered to high quality standards, thereby enhancing the educational and research environment. The audits also promoted transparency and accountability, encouraging institutions to continuously improve their programs and research activities [35,38].
The pressure to meet these new standards led universities to invest more in hiring more full-time professors with advanced degrees and research experience [39]. This comprehensive approach to improving higher education through rigorous evaluation and substantial investment in research and faculty development marked a significant shift in Ecuador’s academic landscape [40].
This study aims to analyze the effects of the LOES on Ecuador’s scientific publication output using a synthetic control method. By comparing Ecuador’s publication data from Scimago [41,42], which includes total documents per year, to a synthetic control group, we can isolate the effect of the 2010 legislation. Results indicate a significant increase in scientific publications in Ecuador following the enactment of the LOES, suggesting that the policy reforms were effective in promoting research activities and enhancing academic productivity. By supporting sustained research productivity, these reforms ensure that educational practices remain aligned with long-term sustainability goals, contributing to both local and global educational advancements.
To assess the long-term impact of the LOES on research productivity, this study adopts a comprehensive theoretical framework that integrates policy feedback theory and the theory of educational change. Policy feedback theory suggests that policies like the LOES not only produce immediate outcomes but also reshape the institutional and social environments, leading to sustained changes in academic practices and norms [43,44]. Additionally, the theory of educational change emphasizes the need for systemic reforms to create enduring improvements in research productivity [45,46]. By combining these perspectives, we are able to evaluate how the LOES has influenced the higher education landscape in Ecuador over time, ensuring that the observed increases in research output are not only significant but also sustainable. This theoretical foundation informs our use of the synthetic control method, allowing us to rigorously analyze the long-term effects of the LOES by comparing Ecuador’s research productivity with that of a synthetic control group.
This literature on education policies and scientific output reviewed is followed by a description of the methodology, including the synthetic control approach. The results and discussion are presented, and the paper concludes with policy implications and suggestions for future research, emphasizing the need for sustainable educational advancements.

2. Methodology

2.1. Research Questions and Hypotheses

This study seeks to answer the following research question: What is the impact of the Organic Law of Higher Education (LOES) on the scientific publication output in Ecuador?
We hypothesize that the implementation of the LOES has led to a significant increase in both the quantity and quality of scientific publications in Ecuador. The study employs a synthetic control method to rigorously evaluate the impact of the policy by comparing Ecuador’s publication trends with those of a synthetic control group.

2.2. Dataset

This study uses publication data from SCImago Journal & Country Rank (SJCR) [41,42], which includes total documents per year, to analyze the effects of the Organic Law of Higher Education (LOES) on Ecuador’s scientific publication output. SJCR is a publicly available portal that includes journals and country scientific indicators developed from the information contained in the Scopus® database, property of Elsevier B.V. (Amsterdam, The Netherlands) [47,48]. These indicators can be used to assess and analyze scientific domains. The dataset encompasses citation data drawn from over 34,100 titles from more than 5000 international publishers and country performance metrics from 239 countries worldwide. The period of study spans from 1996 to 2023, encompassing both pre- and post-LOES implementation phases. The dataset includes the number of scientific publications per year for Ecuador and a synthetic control group comprising 11 countries: Brazil, Bolivia, Argentina, Chile, Colombia, Mexico, Panama, Paraguay, Peru, Uruguay, and Venezuela.

2.3. Main Variable

The primary outcome variable is the annual number of scientific publications. This measure captures the overall research productivity and output of higher education institutions in Ecuador. The pre-LOES period (1996–2010) serves as the baseline for comparison, while the post-LOES period (2011–2023) allows for an assessment of the policy’s impact.
The annual number of scientific publications was chosen because it is a direct measure of research productivity, capturing the overall output of higher education institutions [49]. This metric is widely used and allows for clear comparison over time. While other measures like citations could be used as the main variables (and indeed this is being used for robustness checks), the number of publications provides a broad view of research activity and aligns with the goals of the LOES to increase research output.

2.4. Synthetic Control Methods

A synthetic control method (SCM) [50] is used to estimate the causal impact of the LOES on scientific publication output in Ecuador. In this case, the SCM constructs a synthetic version of the treated unit (Ecuador) by optimally weighting control units (the selected 11 countries) to approximate the characteristics and trends of the treated unit before the intervention. This method allows for a comparison between the actual post-intervention outcomes of the treated unit and the estimated outcomes that would have occurred without the intervention. The first step is to select the implementation year of the LOES (2010) as the primary predictor variable. This ensures that the synthetic control method focuses on the year of the intervention as the key factor influencing the outcome variable, which is the number of scientific publications.
To construct the synthetic control, we use the Synth package from R [51]. This utilizes pre-intervention data to ensure the synthetic version accurately reflects the treated unit’s trends and characteristics. This is achieved by an optimization algorithm that minimizes the mean squared prediction error (MSPE). The algorithm adjusts the weights of the control units based on their predictor values and outcome variables from before the intervention. This weighting process ensures that the synthetic control closely mirrors the pre-intervention state of the treated unit, allowing for reliable attribution of any post-intervention differences to the intervention itself.
Once the synthetic control is constructed, the actual post-intervention outcomes of the treated unit are compared to the predicted outcomes of the synthetic control. The difference between these outcomes represents the estimated causal effect of the LOES on scientific publication output. Any significant gap between the counterfactual and the observed series in the post-treatment period is attributable to the effect of the intervention itself. By providing a credible counterfactual scenario, we allow for a rigorous evaluation of the causal impact of the LOES on scientific publication output in Ecuador.
The synthetic control method employed in this study does not compare Ecuador directly to each South American country listed initially. Instead, it uses a weighted combination of these countries to construct a synthetic version of Ecuador that serves as a counterfactual scenario. This ‘imaginary’ Ecuador represents what the research output would have likely been if the 2010 LOES had not been implemented. Therefore, the focus is not on individual comparisons with each listed country but on the aggregate effect of using these countries to create a benchmark for evaluating Ecuador’s policy impact. This approach allows us to isolate the effect of the LOES by comparing actual outcomes in Ecuador to those of the synthetic control group, which is a statistically derived combination of these countries.
The synthetic control method (SCM) was chosen for this study due to its strengths in evaluating the causal impact of policy interventions in settings where randomized control trials are not feasible. Unlike traditional comparative methods, the SCM allows for the construction of a synthetic version of the treated unit (Ecuador) by optimally weighting control units (selected South American countries) to closely match the pre-intervention characteristics of the treated unit. This approach provides a more accurate counterfactual scenario, which is crucial for isolating the effect of the LOES from other external factors.
The SCM is particularly advantageous in this context for several reasons. First, it accommodates the complexity of real-world interventions by allowing for the combination of multiple control units, which helps mitigate biases that could arise from using a single control unit. Second, the SCM is robust to the influence of time-varying confounders, making it more reliable for policy evaluation in dynamic environments. Finally, the SCM provides a transparent and replicable framework for causal inference, as it clearly delineates the process of constructing the synthetic control and allows for easy interpretation of the results.

2.5. Construction of the Synthetic Control

The selection of the 11 control countries—Brazil, Bolivia, Argentina, Chile, Colombia, Mexico, Panama, Paraguay, Peru, Uruguay, and Venezuela—was based on their geographical, economic, and socio-political similarities to Ecuador. These countries are all located in Latin America and share comparable historical and cultural contexts, making them suitable candidates for constructing a synthetic control that accurately reflects the trends and characteristics of Ecuador [52]. The weights were determined through a data-driven optimization process as mentioned before, ensuring that the synthetic control closely approximates Ecuador’s characteristics and trends before the implementation of the LOES.
The final weights assigned to each control country are presented in Table 1. Paraguay received the highest weight (0.8679097), indicating its significant contribution to the synthetic control. This suggests that Paraguay’s pre-intervention publication trends and other characteristics closely matched those of Ecuador. Brazil, on the other hand, received the lowest weight (0.001928429), indicating a more modest contribution to the synthetic control. The varying weights assigned to each country reflect their relative importance in constructing a synthetic control that accurately represents the counterfactual scenario for Ecuador.

2.6. Statistical Analysis

We applied synthetic control to compare the actual publication trends in Ecuador with the trends of the synthetic control group. The primary focus was on the difference in publication counts between the treated unit (Ecuador) and its synthetic counterpart post-LOES implementation. This difference provides an estimate of the causal effect of the LOES on Ecuador’s scientific publication output.

2.7. Robustness Check

Citations per document were utilized as robustness checks to assess the impact of the LOES on the scholarly influence of Ecuador’s publications, focusing on the study period from 1996 to 2017. Citations per document measure the average number of times a publication is cited by other works, reflecting the impact and relevance of the research [53]. This timeframe was chosen to mitigate the influence of the ‘citation lag’ effect, where more recent publications have not yet had sufficient time to accumulate citations, potentially skewing the data. Analyzing these variables up to 2017 provides a more stable and representative measure of citations, ensuring the analysis captures the true impact of the policy on the quality and influence of the research output.

3. Results

3.1. Descriptive Analysis of Publication Trends in Ecuador

Figure 1 illustrates a significant increase in the number of scientific publications in Ecuador following the implementation of the Organic Law of Higher Education (LOES) in 2010. Before the implementation, the average annual number of publications remained relatively stable, averaging around 253.4 publications per year from 1996 to 2010. Post-2010, there is a marked rise in publications. By 2015, the number of publications had increased to approximately 2000 per year. This upward trend continued, reaching about 6000 publications only in 2023.

3.2. Comparison of Actual vs. Synthetic Ecuador Publications

Figure 2 shows a comparison of actual versus synthetic scientific publication outputs for Ecuador from 1996 to 2023. The top panel illustrates the actual number of publications (red line) and the synthetic control model (blue line), which estimates the number of publications had the LOES not been implemented. The vertical dotted line marks the year the LOES was introduced (2010).
Before 2010, the actual and synthetic publication numbers are almost identical, indicating the synthetic model accurately represents the pre-LOES scenario. After 2010, there is a significant divergence, with the actual publications increasing sharply compared to the synthetic control. By 2023, the actual publications reach approximately 6584, while the synthetic control estimates around 1431 publications. This demonstrates the substantial impact of the LOES on scientific output.
The bottom panel shows the difference between the actual number of publications and the synthetic control. The difference remains minimal before 2010, but it increases sharply after 2010, reflecting the growing impact of the LOES on scientific production. By 2023, the difference peaks, highlighting the significant increase in actual publications compared to the synthetic control.
Before the LOES (1996–2010), the average number of actual Ecuador’s publications was 253.4 per year. After the LOES, this average increased to 3645.84 per year, representing a 358.41% increase. Immediately after the LOES (2010–2013), the increase was even more pronounced, with an average annual increase of 1280.99%. This dramatic rise underscores the effectiveness of the LOES in boosting scientific output in Ecuador. In contrast, the synthetic control model, which estimates the number of publications if the LOES had not been implemented, shows a much lower increase. Prior to 2010, the synthetic control aligns closely with the actual data, with an average of around 253.4 publications per year. Post-2010, while the synthetic control model also shows some increase, it is significantly less than the actual data. By 2023, the synthetic control estimates around 1431 publications per year, compared to the actual 6584 publications.

3.3. Robustness Check

To validate our findings, we conducted robustness checks using citations per document, focusing on the period from 1996 to 2017 to avoid ‘citation lag’ effects. The metric followed the same positive trend in real Ecuador compared to the synthetic control. This indicates that the LOES not only increased the number of publications but also enhanced their academic impact. The sustained increase in research output contributes to the broader goals of sustainability by fostering a continuous flow of knowledge that supports educational and societal advancements. This ensures that Ecuador’s higher education system can maintain its contributions to global sustainability goals over time. This additional metric confirms that the improvements in Ecuador’s research productivity are robust and consistent.

4. Discussion

The present study aims to evaluate the effectiveness of Ecuador’s Higher Education Law (LOES) on enhancing research productivity, specifically addressing the research question: What is the impact of the LOES on the scientific publication output in Ecuador? The major findings suggest that the LOES has led to a significant increase in research output among higher education institutions in Ecuador. This marked improvement in the quantity of research publications supports our hypothesis that targeted legislative reforms can enhance academic productivity. In addition, this finding is consistent with previous studies that have demonstrated the positive impact of policy interventions on academic productivity. For instance, Payne et al. [54] found that policy changes in the United States led to substantial improvements in research outputs, attributing these gains to increased funding and support for academic research.
In light of previous research, our data align with the notion that legislative measures contribute to academic performance by providing structured incentives and support systems for researchers. Leydesdorff et al. [55] highlighted similar outcomes in the OECD, where funding policy changes designed to enhance research funding and collaboration opportunities resulted in increases in citation impact for some countries. Our results reinforce these findings, indicating that Ecuador’s policy intervention is effective in fostering an enhanced research environment.
Comparatively, Uddin and Singh [56] conducted a study on the impact of economic indicators in various South Asian countries, revealing that stronger economies (in this case, economies with more research funding) witnessed significant improvements in both the quantity and quality of research outputs. This parallels our findings, suggesting that Ecuador’s Higher Education Law has created a more conducive environment for research through similar mechanisms. While Uddin and Singh’s study focuses on broad economic indicators, our findings suggest that targeted policy reforms under the LOES can similarly foster a conducive research environment. The key distinction lies in the mechanisms: Uddin and Singh attribute improvements to overall economic growth and increased funding, whereas our study highlights the impact of specific legislative measures designed to enhance research capacity. Despite these differences, both studies illustrate the importance of structural factors in promoting research productivity. This comparison underscores that, whether through economic stability or policy interventions, creating a supportive environment is crucial for academic growth and innovation.
Our data suggest that the observed improvements in research productivity are associated with the overall impact of the Higher Education Law, which introduced changes such as increased funding, better infrastructure, and enhanced collaborative opportunities. While the implementation of the LOES appears to have had a significant impact on increasing research output, it is important to consider other potential confounding factors that could influence the number of scientific publications. Factors such as changes in global academic trends, economic conditions, technological advancements, and increased access to research funding from international sources could also contribute to the observed increase in publications. To mitigate these potential confounders, our study employed the synthetic control method, which helps isolate the effect of the LOES by comparing Ecuador’s publication trends to those of a synthetic control group constructed from similar countries. This approach controls for external factors that could similarly affect the research output in comparable countries, thereby providing a more accurate estimate of the policy’s impact.
While the LOES has been a cornerstone of higher education reform in Ecuador since its implementation in 2010, it has undergone several amendments to adapt to the evolving educational landscape [36]. Notably, modifications in subsequent years have adjusted certain aspects of the law, including changes in governance structures, funding mechanisms, and accreditation standards to better align with current educational needs and global trends. These adjustments have ensured that the LOES remains relevant and continues to effectively support research productivity and academic excellence in Ecuador. However, the core stipulations of the LOES, such as the mandatory allocation of resources for research and the emphasis on quality and accessibility in higher education, have remained consistent, providing a stable framework for ongoing research output.
These elements are essential for creating a sustainable academic ecosystem where resources are efficiently utilized, and research activities can thrive over the long term. The LOES pushed also international collaborations which expanded the global visibility of Ecuadorian research, further boosting productivity [57]. Figg et al. [58] found that international collaborations often lead to higher citation rates and broader dissemination of research findings, which aligns with our observation of improved research quality and impact. The law’s emphasis on international collaborations further contributes to the sustainability of higher education by ensuring that institutions remain globally competitive and resilient [59].
Additionally, targeted funding initiatives and strategic policy frameworks significantly contributed to higher research output and innovation in universities [37,38,39]. This stability is vital for the long-term sustainability of higher education institutions, allowing them to plan, innovate, and adapt to future challenges and opportunities [40]. By continuously supporting high-quality research outputs and innovations, the Higher Education Law ensures that institutions can sustain their academic performance and societal contributions over time.
Interpreting these results, it is evident that the structural changes brought about by the Higher Education Law have played a significant role in creating an environment conducive to scholarly activities.
In essence, the sustainable impact of Ecuador’s Higher Education Law lies in its ability to provide a robust and resilient framework for academic research. By ensuring that higher education institutions have the necessary resources and support, the law promotes a sustainable future where research and innovation can flourish, contributing to the broader goals of societal development and global knowledge advancement.
This study contributes to the literature on education policy and its impact on scientific output, providing valuable insights for policymakers and educational institutions aiming to enhance research productivity. Beyond the increase in the number of publications, the robustness checks using a citations-per-document index provide further validation of our main findings. These metrics, which reflect the scholarly influence and impact of research, show trends that align with the growth in publication numbers, suggesting that the increase in quantity has not come at the expense of quality. The steady rise in the citations per document indicates that the publications resulting from the LOES reforms are not only numerous but also influential, aligning with broader literature that links policy interventions with improved research quality. This alignment underscores the effectiveness of the LOES in fostering a research environment that supports both the quantity and quality of academic output.
This study is not without its caveats. A primary concern could be attributing the increased research productivity solely to the Higher Education Law, as other external factors may also contribute to this trend. However, as mentioned before, the synthetic control method we employed helps mitigate this issue by isolating the effect of the policy from other variables. Additionally, we recognize the potential for an initial surge in productivity due to the novelty of the policy, which might stabilize over time. Despite this, the consistent upward trend observed over several years indicates a sustained positive impact of the policy. It is important to consider the potential for an initial surge in research productivity following the implementation of the LOES, driven by the novelty of the policy and the immediate incentives it introduced. This surge might reflect an initial period of heightened activity as institutions adapt to the new requirements and opportunities for research funding and collaboration [60]. Over time, however, this heightened productivity could stabilize as the policy’s effects become more integrated into the regular functioning of academic institutions.
This phenomenon, often observed in policy interventions, suggests that the observed increase in publication output may not maintain the same growth trajectory indefinitely. Instead, the long-term impact of the LOES is likely to depend on the sustained commitment to funding, infrastructure, and governance structures that support ongoing research activities. It also raises the question of whether the initial surge represents a temporary boost or a permanent shift in the research culture of Ecuador’s higher education institutions.
Nevertheless, our results, although they attempt to simulate an almost perfect counterfactual, should be interpreted with caution.
Despite the positive findings, this study has certain limitations. The primary limitation is the reliance on publication data as the sole measure of research productivity, which might not fully capture the broader impact of research activities. Additionally, while we demonstrate similar trends with bibliometric and citation data, these indicators do not necessarily reflect research quality or societal usefulness. It is important to recognize the limitations of our methodological approach, particularly when considering its application in other contexts. The synthetic control method, while powerful in isolating the effects of policy changes, relies heavily on the availability of comparable data and the careful selection of control units. This can be challenging in different national contexts, where data may not be as robust or where the selection of appropriate comparison groups is more difficult. Therefore, while the findings of this study provide a valuable framework for policy implementation, careful consideration must be given to these methodological limitations when replicating this approach in other countries.
In conclusion, the findings from this study have important implications for both policymakers and academic institutions internationally. The successful implementation of LOES in Ecuador serves as a compelling case study for other developing nations seeking to enhance their research output through legislative reform. The results suggest that targeted investments in higher education, coupled with structured support for research activities, can yield significant improvements in both the quantity and quality of academic publications. The key takeaway is the importance of creating a supportive policy environment that fosters academic excellence and sustainability. This study highlights that, while context-specific adaptations may be necessary, the principles underlying Ecuador’s approach—such as increased funding, international collaboration, and institutional autonomy—are universally applicable and can be adapted to suit different educational and socio-economic landscapes.
Future research should explore the long-term impacts of the Higher Education Law on research productivity, including potential shifts in research focus areas and the quality of publications. It is also recommended to investigate the impact of the law on different types of institutions, such as public versus private universities, to understand the policy’s differential effects. Additionally, studies could examine the broader socio-economic impacts of increased research productivity on Ecuador’s development. Exploring different types of research outputs beyond publication counts, such as patents, conference presentations, and collaborative research projects, would provide a more comprehensive understanding of the LOES’s impact. Extending the analysis to other countries with similar legislative frameworks could offer comparative insights and help generalize the findings. Furthermore, incorporating qualitative methods, such as interviews with policymakers and academic leaders, could uncover the mechanisms through which these policies influence research productivity, offering valuable insights for replicating Ecuador’s success in other contexts. These future research directions will provide deeper insights for policymakers aiming to enhance research outcomes and academic productivity.

Author Contributions

P.Á.-M.: Conceptualization; writing—original draft preparation; writing—review and editing. M.F.-H.: Conceptualization; writing—original draft preparation; writing—review and editing; supervision; funding acquisition. F.P.-O.: writing—original draft preparation; supervision; resources. D.A.P.G.: formal analysis; visualization. A.M.A.V.: resources. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable. This study does not involve human or animal subjects as it exclusively analyzes secondary data from public databases, specifically publication records from the SCImago Journal & Country Rank (SJCR) and citation metrics. The data used pertain to aggregated publication and citation statistics, which do not include any personal or sensitive information about individuals. Consequently, there is no requirement for ethical review or approval from an Institutional Review Board for this type of research.

Data Availability Statement

The data used in this study are publicly available and can be accessed from the SCImago Journal & Country Rank (SJCR) website at https://www.scimagojr.com/countryrank.php (accessed on 14 September 2024).

Conflicts of Interest

The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Scoffham, S.; Rawlinson, S. Sustainability Education: A Classroom Guide; Bloomsbury Publishing: London, UK, 2022; ISBN 1-350-26207-2. [Google Scholar]
  2. Alam, G.M. Sustainable Education and Sustainability in Education: The Reality in the Era of Internationalisation and Commodification in Education—Is Higher Education Different? Sustainability 2023, 15, 1315. [Google Scholar] [CrossRef]
  3. Stanciu, A.C.; Condrea, E. Sustainability in Higher Education. In Proceedings of the 9th BASIQ International Conference on New Trends in Sustainable Business and Consumption, Constanța, Romania, 8–10 June 2023. [Google Scholar]
  4. Jastrzębska-Smolaga, H. Education for Sustainable Development. Ekon. I Sr. 2001. [Google Scholar] [CrossRef]
  5. Malik, R.S. Educational Challenges in 21st Century and Sustainable Development. J. Sustain. Dev. Educ. Res. 2018, 2, 9–20. [Google Scholar] [CrossRef]
  6. Argento, D.; Einarson, D.; Mårtensson, L.; Persson, C.; Wendin, K.; Westergren, A. Integrating Sustainability in Higher Education: A Swedish Case. Int. J. Sustain. High. Educ. 2020, 21, 1131–1150. [Google Scholar] [CrossRef]
  7. Kemper, J.A.; Ballantine, P.W.; Hall, C.M. The Role That Marketing Academics Play in Advancing Sustainability Education and Research. J. Clean. Prod. 2020, 248, 119229. [Google Scholar] [CrossRef]
  8. Kanaabi, M.; Kasule, G.W.; Owino, P. Policy Support and Research Productivity among Lecturers in Ugandan Public Universities: Case of Kyambogo University. Uganda High. Educ. Rev. 2021, 9, 52. [Google Scholar] [CrossRef]
  9. Peurach, D.J.; Glazer, J.L.; Winchell Lenhoff, S. The Developmental Evaluation of School Improvement Networks. Educ. Policy 2016, 30, 606–648. [Google Scholar] [CrossRef]
  10. Øvretveit, J.; Gustafson, D. Evaluation of Quality Improvement Programmes. BMJ Qual. Saf. 2002, 11, 270–275. [Google Scholar] [CrossRef]
  11. Cooke, J. A Framework to Evaluate Research Capacity Building in Health Care. BMC Fam. Pract. 2005, 6, 44. [Google Scholar] [CrossRef]
  12. Al Raee, M.; Ritzen, J.; de Crombrugghe, D. Innovation Policy & Labour Productivity Growth: Education, Research & Development, Government Effectiveness and Business Policy. Innovation 2017, 2017, 019. [Google Scholar]
  13. Millones-Gómez, P.A.; Yangali-Vicente, J.S.; Arispe-Alburqueque, C.M.; Rivera-Lozada, O.; Calla-Vásquez, K.M.; Calla-Poma, R.D.; Requena-Mendizábal, M.F.; Minchón-Medina, C.A. Research Policies and Scientific Production: A Study of 94 Peruvian Universities. PLoS ONE 2021, 16, e0252410. [Google Scholar] [CrossRef] [PubMed]
  14. Akcigit, U.; Pearce, J.; Prato, M. Tapping into Talent: Coupling Education and Innovation Policies for Economic Growth. Rev. Econ. Stud. 2024, rdae047. [Google Scholar] [CrossRef]
  15. Wang, X.; Hu, H. Sustainability in Chinese Higher Educational Institutions’ Social Science Research: A Performance Interface toward Efficiency. Sustainability 2017, 9, 1952. [Google Scholar] [CrossRef]
  16. Pedro, E.d.M.; Leitão, J.; Alves, H. Bridging Intellectual Capital, Sustainable Development and Quality of Life in Higher Education Institutions. Sustainability 2020, 12, 479. [Google Scholar] [CrossRef]
  17. Shibayama, S.; Baba, Y. Impact-Oriented Science Policies and Scientific Publication Practices: The Case of Life Sciences in Japan. Res. Policy 2015, 44, 936–950. [Google Scholar] [CrossRef]
  18. Gurau, C. Academic Excellence through Scholarly Research and Publications: A Multi-Stakeholder Perspective. Septentrio Conf. Ser. 2019. [Google Scholar] [CrossRef]
  19. Hallinger, P.; Nguyen, V.-T. Mapping the Landscape and Structure of Research on Education for Sustainable Development: A Bibliometric Review. Sustainability 2020, 12, 1947. [Google Scholar] [CrossRef]
  20. Saric, J.; Breu, T.; Fokou, G.; Gass, S.; Kiteme, B.; Masanja, H.; Utzinger, J.; Zeleke, G.; Käser, F. Research− Implementation Organisations and Their Role for Sustainable Development. Sustain. Dev. 2023, 31, 1401–1416. [Google Scholar] [CrossRef]
  21. Sianes, A.; Vega-Muñoz, A.; Tirado-Valencia, P.; Ariza-Montes, A. Impact of the Sustainable Development Goals on the Academic Research Agenda. A Scientometric Analysis. PLoS ONE 2022, 17, e0265409. [Google Scholar] [CrossRef]
  22. Ferrer-Estévez, M.; Chalmeta, R. Integrating Sustainable Development Goals in Educational Institutions. Int. J. Manag. Educ. 2021, 19, 100494. [Google Scholar] [CrossRef]
  23. Fioramonti, L.; Giordano, C.; Basile, F.L. Fostering Academic Interdisciplinarity: Italy’s Pioneering Experiment on Sustainability Education in Schools and Universities. Front. Sustain. 2021, 2, 631610. [Google Scholar] [CrossRef]
  24. Fahim, A.; Tan, Q.; Naz, B.; ul Ain, Q.; Bazai, S.U. Sustainable Higher Education Reform Quality Assessment Using SWOT Analysis with Integration of AHP and Entropy Models: A Case Study of Morocco. Sustainability 2021, 13, 4312. [Google Scholar] [CrossRef]
  25. Vallury, S.; Smith, A.P.; Chaffin, B.C.; Nesbitt, H.K.; Lohani, S.; Gulab, S.; Banerjee, S.; Floyd, T.M.; Metcalf, A.L.; Metcalf, E.C. Adaptive Capacity beyond the Household: A Systematic Review of Empirical Social-Ecological Research. Environ. Res. Lett. 2022, 17, 063001. [Google Scholar] [CrossRef]
  26. Fien, J. Advancing Sustainability in Higher Education: Issues and Opportunities for Research. High. Educ. Policy 2002, 15, 143–152. [Google Scholar] [CrossRef]
  27. Newig, J.; Jahn, S.; Lang, D.J.; Kahle, J.; Bergmann, M. Linking Modes of Research to Their Scientific and Societal Outcomes. Evidence from 81 Sustainability-Oriented Research Projects. Environ. Sci. Policy 2019, 101, 147–155. [Google Scholar] [CrossRef]
  28. Fini, R.; Rasmussen, E.; Siegel, D.; Wiklund, J. Rethinking the Commercialization of Public Science: From Entrepreneurial Outcomes to Societal Impacts. Acad. Manag. Perspect. 2018, 32, 4–20. [Google Scholar] [CrossRef]
  29. Aghion, P.; Dewatripont, M.; Hoxby, C.; Mas-Colell, A.; Sapir, A. The Governance and Performance of Universities: Evidence from Europe and the US. Econ. Policy 2010, 25, 7–59. [Google Scholar] [CrossRef]
  30. Noche, E.B.; Ventayen, R.J.M.; Galas, E.M. Deployment of Preprint Servers and Online Journals as Campus Research Management System Using Open-Source Applications. In Proceedings of the 2023 8th International Conference on Business and Industrial Research (ICBIR), Bangkok, Thailand, 18–19 May 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 353–358. [Google Scholar]
  31. Deliyannides, T.S.; Gabler, V. The University Library System, University of Pittsburgh: How & Why We Publish; IDS Project Press: Geneseo, NY, USA, 2013. [Google Scholar]
  32. Ratnaningsih, D.R.; Swadesi, B.; Putradianto, R.R.; Risky, A.N. Research Culture and Productivity Improvement through Online Journal System Development and Optimization; RSF Press: Paris, France, 2020; Volume 1, pp. 66–76. [Google Scholar]
  33. Whitley, R.; Glaser, J. Organisational Transformation and Scientific Change: The Impact of Institutional Restructuring on Universities and Intellectual Innovation; Emerald Group Publishing: Bingley, UK, 2014; ISBN 1-78350-683-0. [Google Scholar]
  34. Crespi, G.; Zuniga, P. Innovation and Productivity: Evidence from Six Latin American Countries. World Dev. 2012, 40, 273–290. [Google Scholar] [CrossRef]
  35. Ley Orgánica de Educación Superior; Director Del Registro Oficial. Ley Orgánica de Educación Superior, LOES; Asamblea Nacional del Ecuador: Quito, Ecuador, 2010.
  36. Guzmán, A.A.R.; Cobas, D.E.P.; Aldean, J.P.M.; Varela, R.E.P. Ley Orgánica de Educación Superior En Ecuador y Su Impacto En El Aseguramiento de La Calidad. Dilemas contemporáneos: Educación, Política y Valores. 2020. Available online: https://dilemascontemporaneoseducacionpoliticayvalores.com/index.php/dilemas/article/view/2418 (accessed on 6 September 2024).
  37. Montenegro, C.W.; Flores, D.A. An Integrated Model for ICT Governance and Management Applied to the Council for Evaluation, Accreditation and Quality Assurance of Higher Education Institutions in Ecuador (CEAACES). In Proceedings of the 2015 International Conference on Computing, Communication and Security (ICCCS), Pointe aux Piments, Mauritius, 4–5 December 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 1–9. [Google Scholar]
  38. Castillo-Mayén, R.; Montes-Berges, B. Análisis de Los Estereotipos de Género Actuales. An. De Psicol. Ann. Psychol. 2014, 30, 1044–1060. [Google Scholar] [CrossRef]
  39. Avilés, G.T. Impacto de Las Nuevas Políticas de Educación Superior En Las Universidades y Escuelas Politécnicas Del Ecuador. Rev. Empres. 2016, 10, 28–34. [Google Scholar]
  40. Abad, R.G.; Castro, J.C.O. Perfeccionamiento de Los Criterios de Evaluación Asociados a La Acreditación Institucional de La Universidad Católica de Cuenca, Extensión Azogues. Mendive. Rev. De Educ. 2015, 13, 158–168. [Google Scholar]
  41. Scimago, G. SCImago Journal & Country Rank: Un Nuevo Portal, Dos Nuevos Rankings. El Prof. De La Inf. 2007. [Google Scholar] [CrossRef]
  42. SJR—About, Us. Available online: https://www.scimagojr.com/aboutus.php (accessed on 19 May 2024).
  43. Sewerin, S.; Béland, D.; Cashore, B. Designing Policy for the Long Term: Agency, Policy Feedback and Policy Change. Policy Sci. 2020, 53, 243–252. [Google Scholar] [CrossRef]
  44. Hacker, J.S.; Pierson, P. Policy Feedback in an Age of Polarization. ANNALS Am. Acad. Political Soc. Sci. 2019, 685, 8–28. [Google Scholar] [CrossRef]
  45. Hanson, M. Institutional Theory and Educational Change. Educ. Adm. Q. 2001, 37, 637–661. [Google Scholar] [CrossRef]
  46. Little, J.W. Social Network Theory and Educational Change; Harvard Education Press: Cambridge, MA, USA, 2010; ISBN 1-61250-376-4. [Google Scholar]
  47. Guz, A.N.; Rushchitsky, J.J. Scopus: A System for the Evaluation of Scientific Journals. Int. Appl. Mech. 2009, 45, 351–362. [Google Scholar] [CrossRef]
  48. Scopus Preview—Scopus—Welcome to Scopus. Available online: https://www.scopus.com/home.uri (accessed on 19 May 2024).
  49. Mostert, S.P.; Ellenbroek, S.P.; Meijer, I.; Van Ark, G.; Klasen, E.C. Societal Output and Use of Research Performed by Health Research Groups. Health Res. Policy Syst. 2010, 8, 30. [Google Scholar] [CrossRef]
  50. Abadie, A. Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects. J. Econ. Lit. 2021, 59, 391–425. [Google Scholar] [CrossRef]
  51. Abadie, A.; Diamond, A.; Hainmueller, J. Synth: An R Package for Synthetic Control Methods in Comparative Case Studies. J. Stat. Softw. 2011, 42, 1–17. [Google Scholar] [CrossRef]
  52. Méndez-Vélez, F.; Palomeque-Cantos, R.; Borja-Pozo, C. Socio-Economic Evolution of Ecuador: Main Economic Indicators and Social Phenomena in Recent Years; AIP Publishing: Melville, NY, USA, 2022; Volume 2574. [Google Scholar]
  53. Tahamtan, I.; Bornmann, L. What Do Citation Counts Measure? An Updated Review of Studies on Citations in Scientific Documents Published between 2006 and 2018. Scientometrics 2019, 121, 1635–1684. [Google Scholar] [CrossRef]
  54. Payne, A.A.; Siow, A. Does Federal Research Funding Increase University Research Output? Citeseer, 1999. Available online: https://econpapers.repec.org/article/bpjbejeap/v_3aadvances.3_3ay_3a2003_3ai_3a1_3an_3a1.htm (accessed on 6 September 2024).
  55. Leydesdorff, L.; Bornmann, L.; Wagner, C.S. The Relative Influences of Government Funding and International Collaboration on Citation Impact. J. Assoc. Inf. Sci. Technol. 2019, 70, 198–201. [Google Scholar] [CrossRef] [PubMed]
  56. Uddin, A.; Singh, V.K. Measuring Research Output and Collaboration in South Asian Countries. Curr. Sci. 2014, 107, 31–38. [Google Scholar]
  57. Herrera-Franco, G.; Montalván-Burbano, N.; Mora-Frank, C.; Bravo-Montero, L. Scientific Research in Ecuador: A Bibliometric Analysis. Publications 2021, 9, 55. [Google Scholar] [CrossRef]
  58. Figg, W.D.; Dunn, L.; Liewehr, D.J.; Steinberg, S.M.; Thurman, P.W.; Barrett, J.C.; Birkinshaw, J. Scientific Collaboration Results in Higher Citation Rates of Published Articles. Pharmacother. J. Hum. Pharmacol. Drug Ther. 2006, 26, 759–767. [Google Scholar] [CrossRef]
  59. Nandigama, S. Global Virtual Exchange as a Sustainable Higher Education Practice: Developing Innovative Teaching and Learning Strategies Using Online Collaboration among Four International Universities. 2022. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4022489 (accessed on 6 September 2024).
  60. Quimbo, M.A.T.; Sulabo, E.C. Research Productivity and Its Policy Implications in Higher Education Institutions. Stud. High. Educ. 2014, 39, 1955–1971. [Google Scholar] [CrossRef]
Figure 1. Trend in scientific publications in Ecuador before and after LOES implementation (dashed lines represent LOES enactment).
Figure 1. Trend in scientific publications in Ecuador before and after LOES implementation (dashed lines represent LOES enactment).
Publications 12 00028 g001
Figure 2. Actual vs. synthetic scientific publication trends in Ecuador post-LOES implementation (dashed lines represent LOES enactment).
Figure 2. Actual vs. synthetic scientific publication trends in Ecuador post-LOES implementation (dashed lines represent LOES enactment).
Publications 12 00028 g002
Table 1. Weights assigned to control countries in the synthetic control analysis.
Table 1. Weights assigned to control countries in the synthetic control analysis.
CountryContribution to Synthetic Control
Paraguay0.8679097
Bolivia0.03899964
Panama0.02671188
Uruguay0.01502694
Peru0.01477641
Venezuela0.009918728
Colombia0.009142159
Chile0.006639666
Argentina0.004917408
Mexico0.004029031
Brazil0.001928429
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Álvarez-Munoz, P.; Faytong-Haro, M.; Peralta Gamboa, D.A.; Aviles Valenzuela, A.M.; Pacheco-Olea, F. Evaluating Policy Efficacy in Higher Education: A Synthetic Control Analysis of Ecuador’s Higher Education Law on Research Productivity. Publications 2024, 12, 28. https://doi.org/10.3390/publications12030028

AMA Style

Álvarez-Munoz P, Faytong-Haro M, Peralta Gamboa DA, Aviles Valenzuela AM, Pacheco-Olea F. Evaluating Policy Efficacy in Higher Education: A Synthetic Control Analysis of Ecuador’s Higher Education Law on Research Productivity. Publications. 2024; 12(3):28. https://doi.org/10.3390/publications12030028

Chicago/Turabian Style

Álvarez-Munoz, Patricio, Marco Faytong-Haro, Dennis Alfredo Peralta Gamboa, Angelo Marcos Aviles Valenzuela, and Fernando Pacheco-Olea. 2024. "Evaluating Policy Efficacy in Higher Education: A Synthetic Control Analysis of Ecuador’s Higher Education Law on Research Productivity" Publications 12, no. 3: 28. https://doi.org/10.3390/publications12030028

APA Style

Álvarez-Munoz, P., Faytong-Haro, M., Peralta Gamboa, D. A., Aviles Valenzuela, A. M., & Pacheco-Olea, F. (2024). Evaluating Policy Efficacy in Higher Education: A Synthetic Control Analysis of Ecuador’s Higher Education Law on Research Productivity. Publications, 12(3), 28. https://doi.org/10.3390/publications12030028

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop