Next Article in Journal
Design of Distributed Discrete-Event Simulation Systems Using Deep Belief Networks
Previous Article in Journal
Technological Aspects of Blockchain Application for Vehicle-to-Network
Open AccessFeature PaperReview

A Systematic Review of the Application of Maturity Models in Universities

1
Doctorado en Ingeniería, Pontificia Universidad Católica del Perú, Lima 32, Peru
2
Facultad de Ingeniería y Arquitectura, Universidad Peruana Unión, Lima 15, Peru
3
Escuela de Posgrado, Universidad Peruana Unión, Lima 15, Peru
*
Author to whom correspondence should be addressed.
Information 2020, 11(10), 466; https://doi.org/10.3390/info11100466
Received: 22 August 2020 / Revised: 24 September 2020 / Accepted: 27 September 2020 / Published: 1 October 2020
(This article belongs to the Section Information Applications)

Abstract

A maturity model is a widely used tool in software engineering and has mostly been extended to domains such as education, health, energy, finance, government, and general use. It is valuable for evaluations and continuous improvement of business processes or certain aspects of organizations, as it represents a more organized and systematic way of doing business. In this paper, we only focus on college higher education. For this reason, we present a novel approach that allows detecting some gaps in the existing maturity models for universities, as they are not models that address the dimensions in their entirety. To identify these models and their validities, as well as a classification of models that were identified in universities, we carried out a systematic literature review on 27,289 articles retrieved with respect to maturity models and published in peer-reviewed journals between 2007 and 2020. We found 23 articles that find maturity models applied in universities, through exclusion and inclusion criteria. We then grouped these items into nine categories with specific purposes. We concluded that maturity models used in Universities move towards agility, which is supported by the semantic web.
Keywords: maturity model; universities; CMMI; evaluation; continuous improvement maturity model; universities; CMMI; evaluation; continuous improvement

1. Introduction

According to [1], based on the interactions of people and the differentiation of their roles, defined the university as an organization of the type “professional bureaucracies.” A bureaucracy for Max Weber is the efficient organization par excellence and that it defines even in the smallest details how things should be done [2]. That is to say, the universities mostly regulate their work, as is the case for the selection of their teachers, who assume significant control over their own work, in the teaching-learning process. Since universities are organizations, maturity models have proven to be valuable in evaluating their processes, and determine by levels the path for academic excellence. Maturity models had their origin in the field of quality proposed by Philip B. Crosby, and were consolidated in the field of software engineering [3,4]. The model “Quality Management Maturity Grid” is divided into five (5) progressive stages (uncertainty, awakening, enlightenment, wisdom, and certainty) and six (6) measurement categories (management’s attitude and understanding, organizational quality situation, problem management, cost of quality as a percentage of sales, actions for the improvement of quality and summary of the position of the company with respect to quality) that show the experience relationships that an organization has to go through. The form of evaluation proposed by Crosby is at the organizational, division or area level, and must be verified by three individuals: the quality manager of the operation, the general manager of the operation and a staff member who is not assigned to evaluation area, in order to cross opinions and establish an organization at a maturity level. This model recognizes the importance of human factors, such as leadership, attitude, and collaborative work [4,5].
Currently, the maturity models have been extended to different domains such as education, health, energy, finance, industrial sector, government and general use [6]. Moreover, a maturity model is a widely used tool in evaluating fundamental processes or certain aspects of organizations, since it represents an increasingly organized and systematic way to do business [7].
To identify eventual gaps and the level of development of the maturity models and their validities in university higher education, the maturity models selected were grouped into nine categories, according to their orientation. In this context, we carried out a systematic literature review on 27,289 articles retrieved with respect to maturity models and published in peer-reviewed journals between 2007 and 2020. We found 23 articles (presented in Appendix A) that find maturity models applied in universities, through exclusion and inclusion criteria. However, the efforts are oriented towards evaluating the maturity of specific areas in universities, however not its entire structure.
There are many relevant maturity model proposals in literature. In one study, a list of seven methods was found for the development of maturity models for software engineering and organizational management domains, based on ISO/IEC 15504 [8]. In other study, nine maturity models based on Capability Maturity Model Integration (CMMI) and Information Technology Infrastructure Library (ITIL) were found, oriented towards e-learning, computer courses, IT services and computer science courses for higher education institutions [9].
After following a systematic approach to our literature review, we have considered a selection of previous studies that are related to this research, focusing our analysis in order to identify the level of approach achieved in the maturity models in the universities.
This article seeks to contribute and expand in the field of continuous improvement with topics that answer relevant questions regarding the form of the implementation of maturity models in universities. In the field of education, generally, these models have a broad spectrum that encompasses all higher education (university, technological and pedagogical); however, this study only focuses on university higher education. For this reason, a novel approach is presented through three research questions and two bibliometric analyses that allow detecting some gaps in the existing maturity models for universities, as they are not models that address the dimensions in their entirety. The rest of the paper is organized as follows: Section 2 of this study describes maturity model foundations, and Section 3 describes the methodology addressed in our study. The results and discussions are presented in Section 4 and Section 5. Finally, Section 6 presents conclusions and future studies.

2. Maturity Model Foundations

We have presented theoretical and practical foundations of maturity models, their form of organization by levels and their purpose and field of application in universities. A maturity model is a tool that describes and analyzes the behaviors, practices and processes that enable an organization to reach reliable and sustainable results [10]. In this sense, a maturity model is a reference that allows a better understanding of reality, with the purpose of explaining a phenomenon and giving the ability to make predictions. For example, architectural models are not used to construct buildings, but rather to understand challenges represented by its construction.
There is a report that maturity models are an evolution of methodologies used to manage university quality [11], while also guiding organizations in implementing good practices [12], a maturity model with sequences of levels or stages that form a desired future [13], or logical planned routes, from an initial state to maturity [14,15,16]. In other words, a maturity model is a reference from which to understand the reality of states through which a university must pass towards excellence; that is, it defines routes to take and quality mechanisms at each level of maturity.
For [17], a maturity model can be descriptive, prescriptive or comparative. It is descriptive if it achieves a deeper understanding of the current situation of the domain; it fulfills its purpose if it establishes a level of quality at a given time. It is prescriptive if it allows a path for improvement, or if it determines mechanisms and practices needed to reach maturity levels. If a maturity model has comparison mechanisms, and is generalized with other domains, it is said to be comparative. In addition, these authors also highlight the evolution of a maturity model from a descriptive stage for the understanding of the current situation to become prescriptive, achieving substantial improvements. In other words, assessment is necessary to ensure continuous improvement and organizational maturity.
In addition, Reference [18] cataloged a mature company as an intelligent organization that manages knowledge and maintains it as a competitive advantage, along with the quality and improvement of its products and processes. In other words, the company is able to manage tactical and explicit knowledge for continuous learning.

3. Method

The methodology approach for this study is developing the following review protocol (see Figure 1).
We performed a Systematic Literature Review to develop our study, which refers to the method proposed by Kitchenham and Charters [19]. In order to meet the objective of the study, we formulated the following research questions: Under what categories are the maturity models oriented towards Universities grouped? What are the characteristics of maturity models in universities? How are maturity models in universities going to change? In addition, bibliometric questions were formulated: How has the frequency of publications on this topic evolved over time? In which academic journals are these studies found? Inclusion and exclusion criteria considered for the selection of the primary studies were the following:

3.1. Inclusion Criteria

Establishing inclusion criteria is a required practice for the design of high-quality standards in this study. Inclusion criteria are defined as the key characteristics of documents published in scientific journals. Next, in Figure 2, the criteria are presented:

3.2. Exclusion Criteria

The exclusion criteria allow us to see additional characteristics that could interfere with the success of the study. In addition, they include characteristics of the articles that meet the conditions (see Figure 3).

3.3. Temporality

We considered studies developed in the last 14 years. We evaluated the frequency of publications of maturity models within the university education domain. Table 1 shows the chains used, linked with the AND and OR logical operators, to produce a single search string. The following digital libraries were considered for their scientific relevance: Scopus, Web of Science, ScienceDirect, IEEE, ERIC, EBSCO Discovery and Wiley.

3.4. Maturity Model Selection Procedure

The maturity model selection procedure (see Figure 4) was divided into four stages:

4. Results

Table 2 lists 27,289 articles that were considered to be “relevant”. These articles were narrowed down by temporality (2007–2020), leaving 1648 “pre-candidate” articles. Pre-candidate articles were then exported to Excel according to Database characteristics, reducing the articles to 69 “candidate” articles. We proceeded to exclude articles whose titles had no relation to the object of study; we excluded duplicate articles and articles that did not refer to maturity levels from the seven Databases, to identify the final “selected” articles: 10 from Scopus, 1 from Web of Science, 2 from ScienceDirect, 3 from IEEE, 2 from ERIC, 5 from EBSCO Discovery and 0 from Wiley, for a total of 23 articles that directly focused on evaluating maturity in University Education domains.
Finally, the 23 selected articles were registered in the Mendeley tool for bibliographic reference management.

4.1. Research Questions

The following contains conclusions regarding the research questions presented above.
  • Under what categories are the maturity models oriented towards University Education grouped?
No categories were found that properly grouped the maturity models oriented to university higher education through information extraction; however, upon analyzing their purpose, the articles were organized into nine categories, according to common criteria, as shown in Table 3.
  • What are the characteristics of maturity models in university education?
The focus of this review, regarding maturity models in university education, does not include specific guidelines due to the diversity of approaches. In [9], they highlight that indicating maturity models oriented to university education give an idea of gaps, since they are focused on a specific area of the university, such as those oriented to students, curricula, online courses, government of TI among others. In this sense, none of these models present mature practices that focus on the different entities of university education. Typically, the models demonstrate “what to do”, but none of them offer “how to” use best practices for continuous improvement established by maturity levels. As in other sectors, CMM [20], together with its successor CMMI [21], are flagship models of maturity in the university environment.
  • Which direction will maturity models in higher education take?
For [22], maturity models have a tendency to move according to the agility of their components or initiatives seen from software engineering projects, with greater resources to improve and optimize the path to maturity in a sustained and incremental fashion. Likewise, Reference [7] warns that maturity evaluation with traditional methods consumes many resources, and the lack of automation often renders reference points impossible to measure. In this sense, the tendency is to apply agile methods, to automate the methods of maturity evaluation through ontologies using the semantic web.

4.2. Bibliometric Analysis

  • How has the frequency of publications on this topic evolved over time?
When analyzing the results obtained from the search chain and the maturity model selection shown in Table 2, an increase in the number of publications describing maturity models in different domains of university education can be seen in Figure 5 from 2013 onwards. Out of a total of 23 articles, 14 (61%) were published during the last seven years and nine (39%) were published between 2007 and 2013. This shows greater growth and interest in proposing models of maturity that allow for strengthening the university system.
  • What are the publications in which studies related to the subject have been found?
Table 4 presents 23 academic journals, where articles focused on university education have been found, with very specific and isolated domains.

5. Discussion

Our study generates new knowledge proposals in the field of university education maturity, considering studies and prospects from the software engineering domain.
Twenty three of the 27,289 relevant articles were selected, with titles oriented towards university education and maturity levels. The remaining 1648 articles incorporated concepts of maturity in their purposes, but were very generic and did not define a method for evaluating maturity between levels and are oriented towards software development. The challenge will be to establish a flexible maturity model that contemplates the particularities of purposes defined by Universities, in order to diminish the impact of isolated maturity initiatives.
The selected articles corresponded to nine categories: oriented towards teaching, ICT, student monitoring, intellectual capital, e-learning, university entrepreneurship, employability of graduates, strategic planning of universities and IT governance in university institutions because no criteria were found for grouping articles in the literature review. The final 23 articles were analyzed according to their purpose, results and conclusions. We established nine categories and provided an overview of the specific domains of universities.
Maturity models in university education have moved towards agility, automation through ontologies, and the semantic web. These characteristics are being incorporated in maturity models of the application scope of greater use, software engineering, with favorable results due to the use of ontologies, a term that explains the capture of knowledge regarding a specific domain of interest. Thus, in university education, maturity models must have the capacity to add value to society in the acquisition and transfer of knowledge, supported by technology, automation and the semantic web.
In the teaching-learning category, Reference [23] proposes a maturity model for curricular design of higher education institutions in Malaysia. Based on CMMI, it presents five levels of maturity with process areas for each level. Reference [24] presents a maturity model based on the capacity of academic processes, with a maturity scale divided into five levels (see Figure 6).
Reference [25] developed a maturity model with the goal of an engineering program accreditation based on CMMI and knowledge management. The maturity model compares software engineering processes with accreditation processes. It also defines how to advance from one level to another. The proposal of the authors is based on maturity models from the software engineering field; they use CMMI processes to align them with training processes of study programs or universities. Maturity model designs must be viewed from the needs and perspectives of University interest groups, to define assessment mechanisms for teaching and learning with flexible components that respond to institutional goals.
In the category of maturity models oriented towards ICT use, Reference [26] proposes an eight level maturity model, focusing on the use of generic hardware and software as the lowest level necessary for the infrastructure of student formation. Likewise, Reference [27] proposes a maturity model for IT Outsourcing in universities based on ISO 20,000, ISO 38,500, ITIL v3 standards, and Control Objectives for Information and Related Technologies (COBIT) methodologies as best practices in IT governance, with five maturity levels. Levels 1, 2 and 4 are based on ISO 20,000 & ITIL, level 3 is based on ISO 20,000 & author contributions and level 5 is strictly the author’s contribution. Assessing the maturity in the use of ICT for training with an outsourcing environment means transferring management to third parties, allowing updated technologies and services, and avoiding complications with technological waste and inventories. For [28], institutions of higher education require that ICT be part of the daily tools in educational processes. On the one hand, they must address the needs of students who expect, require and essentially demand digital technologies in their learning process. These proposals ensure maturity in the use of hardware, software and connectivity resources to reduce operating costs and improve educational services.
In the category of maturity models oriented towards student monitoring, Reference [29] frames the generational model to a model of capacity for maturity. The generational model implies an evaluation of the institutional capacity to initiate and plan student participation practices in the university. The capacity for maturity model includes five levels, a specific practice relationship, processes and categories. In addition, the model proposes a greater understanding and contextualization of student commitment. Reference [30] presents the SESR-MM as a maturity model to strengthen commitment, success and student retention in universities. The model structure has five maturity dimensions in five categories: learning, support, sense of belonging, integration and resources, as well as 18 processes and practices, and focuses on the improvement of training processes. Maturity models have components for assessing student performance throughout their education, with curricular and extracurricular activities included, and must ensure the achievement of the competencies established in the graduation profile.
In the category of maturity models focused on Intellectual Capital (IC), Reference [31] proposes an Intellectual Capital Maturity Model (ICMM) for universities within a framework divided in phases of actions for change, based on current maturity levels of significant environments. The ICMM contemplates six maturity levels: data collection, IC measurement and management relates to unstructured knowledge management; it is originated by the action, experience and participation of the university community in a specific context. For [32], the teaching process in universities represents intellectual capital for a variety of activities characterized by a mental and intellectual nature as well as a series of other traditional activities. In other words, it defines the ability to achieve the competencies that are defined in the graduation profile.
In the category of maturity models for E-learning, Reference [33] proposes a framework for the continuous improvement of university level e-learning (OCQMM). The OCQMM is based on CMMI and describes the characteristics and requirements of five maturity levels. Its structure includes processes that contemplate practices evaluated by dimensions. Reference [34] presents a pilot study using a method to compare e-learning in university institutions. The pilot was designed to evaluate the operational feasibility of a method based on the e-Learning Maturity Model (eMM), which was developed at the University of Wellington, New Zealand, which in turn was derived from the widely accepted Carnegie Mellon capacity maturity model. Reference [35] also applied a maturity model for e-learning for universities based on CMMI. It includes four process categories: learning, development, coordination and evaluation. In addition, it includes five dimensions: delivery, planning, definition, management and optimization. For [36], most universities tend to increase their use of Web 2.0 technologies to offer online courses, which implies using videos. E-learning is a field that has taken force in university education due to its ease of use and access to ICT, in addition to multiple human activities that facilitate online training. Currently, it is necessary to define criteria for the assessment of capacity and orientation in student formation.
In the category entrepreneurship maturity model category, Reference [37] proposes a business maturity model. The model describes 13 factors and 45 sub factors grouped into a business management framework that includes processes, resources and contexts. It contemplates three maturity scales and describes evaluation mechanisms. According to their objectives, some universities need criteria in order to evaluate the degree of student and graduate entrepreneurship, contributing to society in terms of productivity, skills and innovation, as a point of evaluation.
In the category of a support maturity model for university graduates for students, Pažur and Divjak [38] present a model that assesses the maturity of practices that contribute to the employability of graduates of the technology career. The model was developed using a five-step approach, from identification of purpose, to validation, following the paradigm of design science and exploiting four case studies across Europe. The final maturity model contains 65 practices within four key process areas: 13 within strategic planning, 26 within curriculum design and delivery, 16 within student support, and 10 within provision of extracurricular activities. Additionally, a description of the capacity assessment criteria is provided at the five maturity levels for each practice.
The model presented in the category of maturity model oriented to strategic planning, there is the model presented by Kaufman Hall [39]. This maturity model measures the performance of the strategy-oriented financial planning of universities. Likewise, it is based on four levels of maturity with its practices, which measure performance from the formulation of the annual budget to a budget with periodic reviews, which ensure establishing strategies that allow it to achieve its institutional purposes.
Finally, in the category of IT governance-oriented maturity models, in [40], they present a model that assesses maturity from the edge of service delivery processes, under the COBIT 4.1 framework. This has five maturity levels or indicators of process capacity: level 1-executed, level 2-managed, level 3-established, level 4-predictable and level 5-optimized of IT governance in university institutions under the model of COBIT 4.1 and contemplates the service delivery processes. Each level of maturity has continuous improvement practices in the context of the use of Information Technologies in universities. Additionally, Reference [41] proposes an integration of the BSC with COBIT 4.1 in universities for integration of IT with their organizational strategy. On the other hand, Reference [42] proposes a model that assesses digital maturity in Croatian universities, based on rubrics and qualifications. This model proposes five levels: basic, initial, e-Enabled, e-Confident and e-Mature, it also has an instrument that determines the level of maturity.

6. Conclusions

The models developed or adapted in the university environment are based on a very specific isolated domain, neglecting other key areas of university organizations.
Generally, university maturity models do not suggest areas of work and practices that allow evaluating the level achieved, in order to strengthen continuous improvement.
With this study, it contributes to the state of the art, providing a methodology for identifying maturity models for universities.
We presented the results of a systematic review of 23 academic articles found in digital libraries and indexed databases of great relevance in the scientific and academic field. Likewise, the classification of the studies by publication year is shown in the bibliometric analysis, where an increase in the number of relevant studies published between 2014 and 2020 can be noted.
The maturity models with greater applicability are those oriented towards university education, and those of less applicability towards university entrepreneurship, due to the diversity in the declaration of their objectives that direct their university work. For some universities, the declaration of their objectives is oriented towards teaching, while others focus on teaching with components of university entrepreneurship. These characteristics diverge in the graduation profile, curriculum and educational objectives of each study program and by university.
Universities must adopt descriptive and prescriptive or evaluative maturity models with agility and automation components in order to ensure quality and continuous improvement in education. These models will be developed with tools that support automation through ontologies and the semantic web in order to ensure knowledge management.
Maturity models in their distinct orientations (teaching, ICT and student monitoring) are focused on measuring the maturity of student training, face-to-face or in a virtual modality, with the establishment of criteria that measures the capacity of the formation processes.
We recommend, as future work, to focus on the maturity models of higher education—both technological and pedagogical—evaluating each reality in a concrete way.

Author Contributions

Conceptualization, E.T.-C. and S.P.C.; methodology, E.T.-C. and J.L.L.-G.; software, J.L.L.G; validation, E.T.-C., S.P.C. and J.E.T.-C.; formal analysis, E.T.-C., J.L.L.-G. and J.E.T.-C.; investigation, E.T.-C.; Writing—Review and editing, E.T.-C. and J.L.L.-G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received external funding by Universidad Peruana Unión.

Acknowledgments

The authors would like to thank the Universidad Peruana Unión (UPeU) for the financial support.

Conflicts of Interest

The authors declare that there are no conflict of interest regarding the publication of this paper.

Abbreviations

The following abbreviations are used in this manuscript:
CMMICapability Maturity Model Integration
COBITControl Objectives for Information and Related Technologies
ICMMIntellectual Capital Maturity Model
ICIntellectual Capital
ICTInformation and Communication Technology
ITILInformation Technology Infrastructure Library
ERICEducation Resources Information Center
SLRSystematic Literature Review

Appendix A

Articles selected for analysis:
  • García, V.H.M.; Giraldo, G.A.M.; Quintero, J.F.L. Model accreditation for learning in engineering based on knowledge management and software engineering. In New Advances in Information Systems and Technologies; Springer: Charm, Switzerland, 2016; Volume 444, pp. 79–88.
  • Duarte, D.; Martins, P.V. Higher education business Process improvement: Achieving BPMM level 3. In preceedings of the 2014, 9th International Conference on the Quality of Information and Communications Technology, Guimaraes, Portugal, 23–26 September 2014; pp. 18–27.
  • Marchewka, J.T. Applying the capability maturity model to assurance of learning. Commun. IIMA 2013, 13, 1.
  • Maneerat, P.; Malaivongs, K.; Khlaisang, J. The comparison of Thai qualification framework for higher education and capability maturity model integration for service. Procedia-Social Behav. Sci. 2015, 182, 225–231.
  • Chen, C.Y.; Chen, P.C.; Chen, P.Y. Teaching quality in higher education: An introductory review on a process-oriented teaching-quality model. Total Qual. Manag. Bus. Excell. 2014, 25, 36–56.
  • Silva, D.F.A.; Cabral, R.B. Maturity model for process of academic management. In Proceedings of the 2010 International Conference on Information Society, London, UK, 28–30 June 2010; pp. 444–448.
  • Alrasheedi, M.; Capretz, L.F. Applying CMM towards an m-learning context. In the Proceedings of the International Conference on Information Society (i-Society 2013), Toronto, ON, Canada, 24–26 June 2013; pp. 146–151.
  • Thong, C.L.; Jusoh, Y.Y.; Abdullah, R.; Alwi, N.H. Application of Curriculum Design Maturity Model at Private Institution of Higher Learning in Malaysia: A Case Study. In IAENG Transactions on Engineering Technologies; Yang, G.C., Ao, S., Gelman, L., Eds.; Springer: Dordrecht, Netherlands, 2013; pp. 579–590.
  • Bass, J. An early-stage ICT maturity model derived from Ethiopian education institutions. Int. J. Educ. Dev. Using ICT 2011, 7, 5–25.
  • García, V.V.; Vicente, E.J.F.; Aragonés, L.U. Applicability of the Maturity Model for IT Service Outsourcing in Higher Education Institutions. Int. J. Adv. Comput. Sci. Appl. 2014, 5, 41–50.
  • Clarke, J.A.; Nelson, K.J.; Stoodley, I.D. The place of higher education institutions in assessing student engagement, success and retention: A maturity model to guide practice. In Research and Development in Higher Education: The Place of Learning and Teaching; Wyse, P., Billot, J., Frielick, S., Hallas, J.J., Whitehead, E., Buissink-Smith, N., Eds.; Higher Education Research and Development Society of Australasia Inc.: Hammondville, Australia, 2013; Volume 36, pp. 91–101.
  • Nelson, K.; Clarke, J.; Stoodley, I.; Creagh, T. Using a Capability Maturity Model to build on the generational approach to student engagement practices. High. Educ. Res. Dev. 2015, 34, 351–367.
  • Žilvinas, G.S.S.E.P.; Leitner, M.K.H. An intellectual capital maturity model (ICMM) to improve strategic management in European universities: A dynamic approach. J. Intellect. Cap. 2015, 16, 419–442.
  • Petch, J.; Calverley, G.; Dexter, H.; Cappelli, T. Piloting a process maturity model as an e-learning benchmarking method. In Proceedings of the 5th European Conference on elearning: ECEL, Winchester, UK, 11–12 September 2006; p. 273.
  • Al-Ammary, J.; Mohammed, Z.; Omran, F. E-Learning Capability Maturity Level in Kingdom of Bahrain. Turk. Online J. Educ. Technol. TOJET 2016, 15, 47–60.
  • Gu, D.; Chen, J.; Pu, W. Online course quality maturity model based on evening university and correspondence education (OCQMM). In Proceedings of the 2011 IEEE 3rd International Conference on Communication Software and Networks, Xi’an, China, 27–29 May 2011; pp. 5–9.
  • Markuerkiaga, L.; Errasti, N.; Ochoa, C.; Arcelus, M. UNEK, an Academic Entrepreneurship Maturity Model for Technological Faculties. Dir. Organ. 2017, 61, 12–18.
  • Kosasi, S.; Vedyanto; Yuliani, I.D.A.E. Maturity levels of academic information services of higher education using IT governance. In Proceedings of the 4th International Conference on New Media Studies (CONMEDIA), Yogyakarta, Indonesia, 8–10 November 2017; pp. 56–61.
  • Mantra, I.; Hartawan, M.S.; Saragih, H.; Abd Rahman, A. Web Vulnerability Assessment and Maturity Model Analysis on Indonesia Higher Education. Procedia Comput. Sci. 2019, 161, 1165–1172.
  • Ðurek, V.; Kadoic, N.; Redep, N.B. Assessing the digital maturity level of higher education institutions. In Proceedings of the 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 21–25 May 2018; pp. 0671–0676.
  • Pawan, E.; Utami, E.; Yunita, S.; Hasan, P.; Kaharuddin. Measurement of Maturity Level Higher Education Governance Using Balanced Scorecard (BSC) and COBIT 4.1. In Proceedings of the 2019 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia, 24–25 July 2019; pp. 948–953.
  • Pazur Anicíc, K.; Divjak, B. Maturity Model for Supporting Graduates’ Early Careers Within Higher Education Institutions. SAGE Open 2020, 10, doi:10.1177/2158244019898733.
  • Philip, A. Unveils Strategic Financial Planning Maturity Model for Higher Education at NACUBO Annual Meeting. Kaufman Hall, 2019. Available online: https://www.prnewswire.com/news-releases/kaufman-hall-unveils-strategic-financial-planning-maturity-model-for-higher-education-at-nacubo-annual-meeting-300884716.html (accessed on 29 September 2020).

References

  1. Mintzberg, H. The professional bureaucracy. Organ. Gov. High. Educ. 1979, 2, 50–70. [Google Scholar]
  2. Chiavenato, I. Introducción a la teoría general de la administración; McGraw-Hill Interamericana: Mexico City, Mexico, 2019. [Google Scholar]
  3. Tarhan, A.; Turetken, O.; Ilisulu, F. Business process maturity assessment: State of the art and key characteristics. In Proceedings of the 2015 41st Euromicro Conference on Software Engineering and Advanced Applications, Funchal, Portugal, 26–28 August 2015; pp. 430–437. [Google Scholar]
  4. Crosby, P.B. Quality Is Free: The art of Making Quality Certain; McGraw-Hill: New York, NY, USA, 1979. [Google Scholar]
  5. Storbjerg, S.H.; Brunoe, T.D.; Nielsen, K. Towards an engineering change management maturity grid. J. Eng. Des. 2016, 27, 361–389. [Google Scholar] [CrossRef]
  6. Lee, D.; Gu, J.W.; Jung, H.W. Process maturity models: Classification by application sectors and validities studies. J. Softw. Evol. Process. 2019, 31, e2161. [Google Scholar] [CrossRef]
  7. Proença, D. Methods and techniques for maturity assessment. In Proceedings of the 2016 11th Iberian Conference on Information Systems and Technologies (CISTI), Las Palmas, Spain, 15–18 June 2016; pp. 1–4. [Google Scholar]
  8. Garci, G.; Garci, F. Development of maturity models: A systematic literature review. In Proceedings of the IET Conference Proceedings, Ciudad Real, Spain, 14–15 May 2012. [Google Scholar]
  9. Duarte, D.; Martins, P.V. A maturity model for higher education institutions. J. Spat. Organ. Dyn. 2013, 1, 25–44. [Google Scholar]
  10. da Rosa, I.; da Silva, M.M. A Maturity Model for Business Transformation Management. In Proceedings of the 2015 IEEE 17th Conference on Business Informatics, 2015, Lisbon, Portugal, 13–16 July 2015; pp. 60–67. [Google Scholar]
  11. Pérez-Mergarejo, E.; Pérez-Vergara, I.; Rodríguez-Ruíz, Y. Modelos de madurez y su idoneidad para aplicar en pequeñas y medianas empresas. Ing. Ind. 2014, 35, 184–198. [Google Scholar]
  12. Rosemann, M.; vom Brocke, J. The six core elements of business process management. In Handbook on Business Process Management 1; Springer: New York, NY, USA, 2015; pp. 105–122. [Google Scholar]
  13. Röglinger, M.; Pöppelbuß, J.; Becker, J. Maturity models in business process management. Bus. Process. Manag. J. 2012, 2, 328–346. [Google Scholar] [CrossRef]
  14. Becker, J.; Knackstedt, R.; Pöppelbuß, J. Developing maturity models for IT management. Bus. Inf. Syst. Eng. 2009, 1, 213–222. [Google Scholar] [CrossRef]
  15. Gottschalk, P. Maturity levels for interoperability in digital government. Gov. Inf. Q. 2009, 26, 75–81. [Google Scholar] [CrossRef]
  16. Kazanjian, R.K.; Drazin, R. An empirical test of a stage of growth progression model. Manag. Sci. 1989, 35, 1489–1503. [Google Scholar] [CrossRef]
  17. De Bruin, T.; Rosemann, M.; Freeze, R.; Kaulkarni, U. Understanding the main phases of developing a maturity assessment model. In Proceedings of the Australasian Conference on Information Systems (ACIS): Australasian Chapter of the Association for Information Systems, Sydney, Australia, 29 November–2 December 2005; pp. 8–19. [Google Scholar]
  18. Luján, D.R.; Baryolo, Y.R.; Villalobos, C.M. La utilización de la gestión del conocimiento y la toma de decisiones en el área de proceso monitoreo y control de proyecto (PMC) de CMMI. Rev. Cuba Cienc. InformÁTicas 2010, 4, 1–10. [Google Scholar]
  19. Keele, S. Guidelines for Performing Systematic Literature Reviews in Software Engineering. Available online: https://www.sis.uta.fi/~pt/TIEA5_Thesis_Course/Session_10_2013_02_18/SysLitRev-guidelines-v2.3-kitchenham-9jul07.pdf (accessed on 28 September 2020).
  20. Paulk, M.C.; Curtis, B.; Chrissis, M.B.; Weber, C.V. Capability Maturity Model for Software, Version 1.1. Software Engineering Institute. Available online: https://resources.sei.cmu.edu/asset_files/TechnicalReport/1993_005_001_16211.pdf (accessed on 28 September 2020).
  21. Team, C.P. CMMI® for Development, Version 1.3, Improving processes for developing better products and services. In Software Engineering Institute; Springer: Cham, Switzerland, 2010. [Google Scholar]
  22. Henriques, V.; Tanner, M. A systematic literature review of agile and maturity model research. Interdiscip. J. Inf. Knowl. Manag. 2017, 12, 53–73. [Google Scholar] [CrossRef]
  23. Thong, C.L.; Jusoh, Y.Y.; Abdullah, R.; Alwi, N.H. Application of Curriculum Design Maturity Model at Private Institution of Higher Learning in Malaysia: A Case Study. In IAENG Transactions on Engineering Technologies; Springer: Dordrecht, The Netherlands, 2013; pp. 579–590. [Google Scholar]
  24. Silva, D.F.A.; Cabral, R.B. Maturity model for process of academic management. In Proceedings of the 2010 International Conference on Information Society, IEEE, London, UK, 28–30 June 2010; pp. 444–448. [Google Scholar]
  25. García, V.H.M.; Giraldo, G.A.M.; Quintero, J.F.L. Model accreditation for learning in engineering based on knowledge management and software engineering. In New Advances in Information Systems and Technologies; Springer: Cham, Switzerland, 2016; pp. 79–88. [Google Scholar]
  26. Bass, J. An early-stage ICT maturity model derived from Ethiopian education institutions. Int. J. Educ. Dev. Using Ict 2011, 7, 5–25. [Google Scholar]
  27. García, V.V.; Vicente, E.J.F.; Aragonés, L.U. Applicability of the Maturity Model for IT Service Outsourcing in Higher Education Institutions. Governance 2007, 5, 41–50. [Google Scholar]
  28. Espinoza-Guzmán, J.; Zermeño, M.G.G. Maturity model for e-learning classroom, bimodal and virtual courses in higher education: A preliminary study. Int. J. Web Based Learn. Teach. Technol. (IJWLTT) 2017, 12, 19–31. [Google Scholar] [CrossRef]
  29. Nelson, K.; Clarke, J.; Stoodley, I.; Creagh, T. Using a Capability Maturity Model to build on the generational approach to student engagement practices. High. Educ. Res. Dev. 2015, 34, 351–367. [Google Scholar] [CrossRef]
  30. Clarke, J.; Nelson, K.; Stoodley, I. The place of higher education institutions in assessing student engagement, success and retention: A maturity model to guide practice. In Research and Development in Higher Education: The Place of Learning and Teaching (Volume 36)-Refereed Papers from the 36th HERDSA Annual International Conference; Higher Education Research and Development Society of Australasia, Inc.: Auckland, New Zealand, 2013; pp. 91–101. [Google Scholar]
  31. Žilvinas, G.S.S.E.P.; Leitner, M.K.H. An intellectual capital maturity model (ICMM) to improve strategic management in European universities: A dynamic approach. J. Intellect. Cap. 2015, 16, 419–442. [Google Scholar]
  32. Naser, S.S.A.; Al Shobaki, M.J.; Amuna, Y.M.A. Knowledge Management Maturity in Universities and its Impact on Performance Excellence Comparative study. J. Sci. Eng. Res. 2016, 3, 4–14. [Google Scholar]
  33. Gu, D.; Chen, J.; Pu, W. Online course quality maturity model based on evening university and correspondence education (OCQMM). In Proceedings of the 2011 IEEE 3rd International Conference on Communication Software and Networks, Xi’an, China, 27–29 May 2011; pp. 5–9. [Google Scholar]
  34. Petch, J.; Calverley, G.; Dexter, H.; Cappelli, T. Piloting a process maturity model as an e-learning benchmarking method. Electronic Journal of e-Learning 2007, 5, 49–58. [Google Scholar]
  35. Al-Ammary, J.; Mohammed, Z.; Omran, F. E-Learning Capability Maturity Level in Kingdom of Bahrain. Turk. Online J. Educ. Technol. Tojet 2016, 15, 47–60. [Google Scholar]
  36. Wiele, P.V.; Ribiere, V.; Ermine, J.L. Diagnosing Higher Education on Purposefulness: Introducing the Employability Development and Assessment Maturity Model (EDAMM). In Empowering 21st Century Learners through Holistic and Enterprising Learning; Springer: Singapore, 2017; pp. 177–188. [Google Scholar]
  37. Markuerkiaga, L.; Errasti, N.; Ochoa, C.; Arcelus, M. UNEK, an Academic Entrepreneurship Maturity Model for Technological Faculties. Dir. Organ. 2017, 61, 12–18. [Google Scholar]
  38. Pažur Aničić, K.; Divjak, B. Maturity Model for Supporting Graduates’ Early Careers Within Higher Education Institutions. SAGE Open 2020, 10, 2158244019898733. [Google Scholar] [CrossRef]
  39. Unveils Strategic Financial Planning Maturity Model for Higher Education at NACUBO Annual Meeting. 2019. Available online: https://n9.cl/hulwc (accessed on 10 September 2020).
  40. Kosasi, S.; Yuliani, I.D.A.E. Maturity levels of academic information services of higher education using IT governance. In Proceedings of the 2017 4th International Conference on New Media Studies (CONMEDIA), Yogyakarta, Indonesia, 8–10 November 2017; pp. 56–61. [Google Scholar]
  41. Pawan, E.; Utami, E.; Yunita, S.; Hasan, P. Measurement of Maturity Level Higher Education Governance Using Balanced Scorecard (BSC) and COBIT 4.1. In Proceedings of the 2019 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia, 24–25 July 2019; pp. 948–953. [Google Scholar]
  42. Đurek, V.; Kadoic, N.; Ređep, N.B. Assessing the digital maturity level of higher education institutions. In Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 21–25 May 2018; pp. 671–676. [Google Scholar]
Figure 1. Methodological protocol.
Figure 1. Methodological protocol.
Information 11 00466 g001
Figure 2. Inclusion criteria. In this condition, articles, conference papers, reviews and conference reviews were selected, all in English.
Figure 2. Inclusion criteria. In this condition, articles, conference papers, reviews and conference reviews were selected, all in English.
Information 11 00466 g002
Figure 3. Exclusion criteria.
Figure 3. Exclusion criteria.
Information 11 00466 g003
Figure 4. Maturity model selection procedure: (a) Stage 1—Execution of search string in the selected Databases; (b) Stage 2—Those who met conditions 1-i, 2-i and 4-i were considered pre-candidates. The resulting articles were exported to Excel for further refinement; (c) Stage 3—The articles were analyzed based on two conditions: 2-e, 3-e; (d) Stage 4—Articles were analyzed based on two conditions (3-i and 1-e), implying a review of the abstract and conclusions of all articles contemplated in stage 3.
Figure 4. Maturity model selection procedure: (a) Stage 1—Execution of search string in the selected Databases; (b) Stage 2—Those who met conditions 1-i, 2-i and 4-i were considered pre-candidates. The resulting articles were exported to Excel for further refinement; (c) Stage 3—The articles were analyzed based on two conditions: 2-e, 3-e; (d) Stage 4—Articles were analyzed based on two conditions (3-i and 1-e), implying a review of the abstract and conclusions of all articles contemplated in stage 3.
Information 11 00466 g004
Figure 5. Frequency of publications.
Figure 5. Frequency of publications.
Information 11 00466 g005
Figure 6. Maturity scale. Adapted from Silva and Cabral (2010).
Figure 6. Maturity scale. Adapted from Silva and Cabral (2010).
Information 11 00466 g006
Table 1. Terms and logical connectors used in the search.
Table 1. Terms and logical connectors used in the search.
TermsChain
Domain: University education
Maturity model(“maturity model” OR “capability model” OR “maturity level”)
University education(higher education) OR (university education) OR (university organization)
Table 2. Results of the maturity model selection focused on University Education.
Table 2. Results of the maturity model selection focused on University Education.
DatabaseRelevantPre CandidatesCandidatesSelected
Scopus146411273510
Web of Science332721
ScienceDirect3087842
IEEE56430773
ERIC572542
EBSCO Discovery98271175
Wiley23,8811300
Total 27,28916486923
Table 3. Categories of selected Maturity Models.
Table 3. Categories of selected Maturity Models.
CategoriesNumber of Articles
Maturity models oriented towards teaching.8
Maturity models oriented towards Information and Communication Technology (ICT).3
Maturity models oriented towards student monitoring.2
Maturity models for intellectual capital.1
Maturity models for E-Learning.3
Maturity models aimed at evaluating university entrepreneurship.1
Maturity model oriented to the employability of graduates.1
Maturity model oriented to the strategic planning of universities.1
Maturity model for IT governance in university institutions.3
Total23
Table 4. Scientific journals of selected articles.
Table 4. Scientific journals of selected articles.
Scientific JournalsQuantity
Advances in Intelligent Systems and Computing1
IEEE Xplore7
Lecture Notes in Electrical Engineering1
Total Quality Management and Business Excellence1
Journal of Intellectual Capital2
Higher Education Research and Development1
Direccion y Organizacion1
SAGE Open1
Procedia Computer Science1
Procedia - Social and Behavioral Sciences1
Electronic Journal of e-Learning1
Turkish Online Journal of Educational Technology1
Kaufman Hall1
IJEDICT1
Higher Education Research & Development1
Governance1
Total23
Back to TopTop