A Systemic Mapping Study of Business Intelligence Maturity Models for Higher Education Institutions

Abstract

Higher education institutions (HEIs) are investing in business intelligence (BI) to meet the increasing demand for information stemming from their operations. Information technology (IT) managers in higher education may turn to BI maturity models to evaluate the current state of HEIs’ BI operation capabilities and evaluate the readiness for future improvements. However, generic BI maturity models do not have domain-specific attributes that ensure a high degree of compatibility with HEIs. This study’s objective is to survey maturity models that could be used in HEIs and identify those used for BI to perform an analysis of their qualities and identify future avenues for research into HEI-specific BI maturity models. A systemic mapping was undertaken via both a keyword and snowball search of five indexing services, 6037 articles were processed using inclusion and exclusion criteria resulting in the identification of forty-one academic works regarding maturity model uses which were mapped to ten categories. The mapping reveals an increasing number of publications featuring maturity models for HEI, particularly since 2018, focused on e-learning and ICT. A single instance of a BI maturity model for HEI emerged in 2022 within the European HEI context. The HE-BIA MM has more dimensions than most other models identified, yet only a single co-occurrence of dimensions was identified in name only. We conclude that BI maturity models for HEI are emerging as a field of research with future directions for research including exploring co-occurrence of dimensions with existing maturity models, performing case studies, and validation of HE-BIA MM outside the European HEI context.

1. Introduction

Universities, colleges, trade schools, and other higher education institutes (HEIs) generate data through their day-to-day operations. These data originate as transactions recorded and extracted from internal operational systems such as supply chain management, enterprise resource planning, customer relationship management, or external data sources [1]. Institutional data are warehoused and transformed into reporting products by business intelligence (BI) practitioners to aid in data-driven decision making with the goal of improving HEI performance across functional areas [2,3]. The desire for continuous improvement in all aspects of organizational processes has led HEIs to implement tools from the quality and process management domain [4].

BI maturity models (BIMM) consist of concise descriptions and characteristics of the various aspects of BI maturity and are a recognized tool for evaluation of BI process maturity [5,6]. Existing BIMMs are not designed for complex domains, as these BIMMs lack the domain-specific characteristics necessary to inform IT management about process capability and readiness [6,7]. Maturity models for HEIs make no relevant suggestions to areas of work and practice that allow for the evaluation of the maturity level achieved nor support continuous improvement [5]. HEIs have long been recognized as a complex domain; Kimball [3] described the complexity “running a university is akin to operating all the businesses needed to support a small village. Universities are a real estate property management company, restaurants with multiple outlets, retailer, event management and ticketing agency, police department, professional fundraiser, financial services company, investment firm, venture capitalist, job placement firm, construction company and medical services provider”. BI operations at HEIs deliver BI capabilities across many, if not all, of the functions [3].

In this research study, a systematic mapping study review was conducted to determine what is currently known about BIMMs for HEIs. The purpose of this study is to explore the gap between existing BIMMs and the domain-specific needs of HEIs. More specifically, the goal of this mapping study is to determine how BIMMs for HEIs are treated in the literature, to update and consolidate the information on available BIMMs specific to HEIs. This mapping study aims to address the following research questions (RQs):

• RQ1: To what extent do maturity models for higher education exist in the literature?
• RQ2: To what extent are maturity models for higher education oriented toward business intelligence?
• RQ3: What are the differentiating characteristics of existing BI maturity models for higher education institutions?
• RQ4: What methods exist for generating BI maturity models?
• RQ5: In which sources and in which years were BI maturity models for higher education published?
• RQ6: What were the primary methods for developing BI maturity models for higher education in the published literature?

Current and future directions for BIMM research for HEIs are discussed in the context of the information gained in answering the above research questions.

The remainder of this study is organized as per the guidelines for conducting systemic mapping studies in software engineering [8]. Having established a need for the mapping in Section 1, Section 2 provides an overview of related work in the domain of business intelligence, their use in higher education, and methods of generating maturity models. Section 3 provides details on the research methods, including the research questions, search, study selection, data extraction, analysis and classification, and validity evaluations. Section 4 covers the results, presenting the outcomes of the studies as they answer the research questions. Section 5 concludes the study by directly answering the research questions, a discussion of the findings, future directions, the threats to validity, and a conclusion.

2. Background

Business intelligence (BI) is an umbrella term that covers the technologies, processes, concepts, and methods on fact-based support systems for improving decision making [9]. The definition of BI includes all aspects of data warehousing, extract-transform-load (ETL), and data presentation methods [10]. The primary concern of BI is helping businesses receive insights from their data that aid in decision making to the benefit of the businesses [2].

The concept of BI and its use as a technical solution to deliver information on demand first appears in 1958 in a paper published by IBM and authored by H.P. Luhn [10]. It describes BI as an automatic system to disseminate information to the various section of any industrial, scientific, or government organization [10,11]. The next decade saw several companies develop what were called decision support systems (DSS) based on Luhn’s work [10]. DSSs are any “information system that draws on transaction processing systems and interacts with the other parts of the overall information system to support the decision-making activities of managers and other knowledge workers in organizations” [12]. DSS is closely related to BI and it can be thought of as either a precursor or a component of BI.

The broader application of DSS in the decades following Luhn’s work saw more specialized types of decision support develop, this would include “text-oriented DSS, database-oriented DSS, spreadsheet-oriented DSS, solver-oriented DSS, and rule-oriented DSS” [13]. By the mid 1990s, a myriad of tools and technologies had been deployed into DSS processes such as query tools, data modelling, OLAP, data mining and visualizations [13].

The use of BI as an umbrella term that included DSS was popularized in 1989 by Howard Dresner of the Gartner Group [13]. Since then, the body of knowledge in BI domain has grown, and two authors have been particularly influential. Bill Inmon and Ralph Kimball have published competing visions for overall BI architecture in a series of books and papers. The data warehouse/data mart relationship and data models used are the differentiators between the two architectures. Inmon’s architecture prescribes a data warehouse built from data extracted from operational (transaction) systems, stored in a normalized state, and fed to reporting data marts that are domain specific [3]. In the Kimball BI architecture, the data warehouse is simply the collection of all individual data marts, with no central overarching schema. Data are extracted from operational systems, denormalized through transformed (typically into dimensional models), and loaded into the data warehouse [3]. While not the first major schism in BI architecture design, it serves to demonstrate how complex these systems have become.

The need to deploy BI tools to more cost effectively handle growth in the cost and complexity of BI systems is acknowledged in corporate reports [14]. To evaluate the maturity of these complex BI systems for strengths and weaknesses, a tool called the maturity model is adopted. The BIMM is an evaluation which may be performed through a systemic review of maturity and readiness [15]. In general, a maturity model is “a tool to provide organizations with a way to characterize the maturity of their processes“ [16]. Maturity models come in several different types, but common components exist; typically, they all define “a number of dimensions or process areas at several discrete stages or levels of maturity, with a description of characteristic performance at various levels of granularity” [17].

The origins of BIMMs are found in software maturity models first developed in the 1990s, specifically the CMM [13,18]. Literature reviews of maturity models have concluded that there has been little academic research performed on maturity models [15], with models being developed largely in practitioner literature.

The early development of BI systems was pioneered in HEIs during the 1970s and 1980s [13]. However, adoption of BI systems for use in HEIs is complicated by the fact that HEIs are a complex domain. In 1994, ref. [3] described this complexity: “running a university is akin to operating all the businesses needed to support a small village. Universities are a real estate property management company, restaurants with multiple outlets, retailer, event management and ticketing agency, police department, professional fundraiser, financial services company, investment firm, venture capitalist, job placement firm, construction company and medical services provider”. BI operations at HEIs deliver BI capabilities across many, if not all, of the functional groups [3] recognized. To address the challenges posed by the domain complexity, HEIs have adopted various approaches to BI that share similar goals to extract data from transactional systems and deliver it to decision makers, reducing their time to insight and improving their decision-making process [2,10]. In these differing approaches to BI, there are common elements of HEI BI systems, these include ETL tools and processes, data warehouses, and reporting tools [10].

Practitioner literature published in books and magazines are aimed at implementers in the field and are not subject to the rigorous academic scrutiny of academic research, nor are artefacts presented in practitioner literature bound to any formal development process. The HEI is a complex domain lacking an academically rigorous domain-specific maturity model as the generic BI maturity models presented in practitioner literature contain no knowledge of the solution domain; this precludes the maturity of the solution domain from being addressed [3,7,19]. Solutions for developing domain-specific maturity models do exist.

Design science is a problem-solving paradigm that generates innovative artefacts from ideas, practices, and technical capabilities through which the analysis, design, implementation, management, and use of information systems can be effectively and efficiently accomplished [20,21,22]. Utilizing design science, researchers may create and evaluate IT artefacts to solve well defined organizational problems [22]. BI Maturity models are just such an example of an IT artefact intended to solve organizational problems, specifically the problem of measuring BI maturity in a complex domain [7,22] such as an HEI.

In [22], seven guidelines are given for the generation of IT artefacts that adhere to design science principles. Guideline 6,

Table 1. Design science research guidelines [21].

Guideline	Description
Guideline 1: Design as an Artefact	Design science research must produce a viable artefact in the form of a construct, a model, a method, or an instantiation.
Guideline 2: Problem Relevance	The objective of design science research is to develop technology-based solutions to important and relevant business problems.
Guideline 3: Design Evaluation	The utility, quality, and efficacy of a design artefact must be rigorously demonstrated via well-executed evaluation methods.
Guideline 4: Research Contributions	Effective design science research must provide clear and verifiable contributions in the areas of the design artefact, design foundations, and/or design methodologies.
Guideline 5: Research Rigour	Design science research relies upon the application of rigorous methods in both the construction and evaluation of the design artefact.
Guideline 6: Design as a Search process	The search for an effective artefact requires utilizing available means to reach desired ends while satisfying laws in the problem environment.
Guideline 7: Communication of Research	Design science research must be presented effectively to both to technology-oriented as well as management-oriented audiences.

, advises that “Effective design requires knowledge of both the application domain (e.g., requirements and constraints) and the solution domain (e.g., technical and organizational)” [22]. Based on this guideline, generic BI maturity models, when used for an HEI, are not founded on design science guidelines.

The issue of general IS and BI maturity models lacking domain-specific characteristics has been identified in prior work [6,7,23]. Duarte et al. [6] identified that maturity models intended for use in higher education institutions were “too general and ignore important, specific characteristics of higher education organizations” while [5] noted “university maturity models do not suggest areas of work and practices that allow evaluating the level achieved, to strengthen continuous improvement”.

In 2015, Brooks et al. [7] developed a framework and validated a design science based BIMM for the healthcare domain. This framework may be used for any sufficiently complex domain and employs the iterative approach for design science research proposed by [24].

Given the complexity of operations in HEIs, maturity models may be adapted by HEIs for a variety of reasons. A 2020 systemic review [5] of the application of maturity models in universities found that of 23 articles, 17 could be categorized as non-IT/ICT, while only 6 were related to ICT or IT governance. None of the 6 ICT/IT maturity models found by [5] were related to business intelligence.

A 2019 proposal [25] for a lean assessment model to evaluate BI in higher education included an overview of existing BI models. In this study, Cardoso and Su identified one pre-existing domain-specific BI maturity model for HEI and indicated that a new maturity model for BI in HEI was being developed. The OCU model, developed in 2011 and last updated in 2013, has a focus on the data warehousing aspect of BI [26]. This BI maturity model, the OCU model, was generated “based on the experience of the contributors” [26] and no methodology for its generation could be found. Due to the lack of updates [25] and transparency in development, the OCU model has failed to keep up with changing needs of BI implementation. With the 2015 development of a design science approach to domain-specific BI maturity models [7] and the continuing recognition of higher education as a complex domain for BI, it follows that research into BIMMs for HEIs would begin to be published in recent years.

This study will fill several gaps in the body of knowledge. The first gap is temporality, while Tocto-Cano et al. [5] establish a baseline of articles containing maturity models and generate the categories used in this study, their work is limited to a period ending in mid-2020 and is no longer up to date. Further evidence of the temporality gap comes from [25], as BIMMs for HEIs are noted as being in development in 2019 but are not captured by Tocto-Cano [5]. There is also an opportunity to expand the scope of [5], as their focus was on universities only, while other types of HEIs such as colleges or institutes were not included. This study also includes an analysis of the keywords used to describe relevant papers, which may be used to improve future searches.

3. Materials and Methods

This chapter provides details of the methodology employed in this mapping study. This chapter focuses on the first and second phases of the review protocol as showm in Figure 1.

3.1. Mapping Protocol Development

A systemic mapping protocol was developed using the process outlined by Petersen et al. [8]. The development protocol was divided into three major phases, each with associated stages, as shown in Figure 1. The planning phase included specifying research questions that met the objective of the study and development of the mapping protocols, including the development of the search chain using keywords, identification of search resources such as databases, the establishment of inclusion and exclusion criteria, planning the study selection process, developing data collection templates and integration plans, and documenting the data classification scheme. The developed review protocols reflect an expansion and modification of prior work that was conducted on general maturity models used by higher education institutes in [5] (the modifications are explained in a subsequent section).

The second phase of development documents the plan for conducting the mapping. This includes the generation of Figure 2, which provides a detailed view of the implementation of the selection procedures. A database search is performed on AU Discover, ERIC, IEEE Explore, ACM, and Science Direct. The inclusion and exclusion criteria are applied using article metadata. The abstracts were then read and processed according to the inclusion and exclusion criteria. The following step, full-text reading, enables the papers to be processed again according to the inclusion/exclusion criteria, to record the data to the collection template, and to be mined as sources for the snowball search. The snowball search was subject to a cap at one level due to resource constraints, primarily time. Snowball candidates were processed in the same manner as the full text reading. With the data collected, the papers were then classified to one of the ten maturity model orientations.

The third phase, reporting the mapping, was developed using a modification of the suggested reporting schema found in the guidelines to conducting systemic mappings [8]. The five chapters for reporting a systemic mapping used in this study are the “introduction” which provides necessary background, including research questions, and discusses the need to conduct the mapping; the “background” (related work), which provides an overview of studies in the subject area; the “methodology” (research method), composed of subsections which detail the search, keywords, selection protocol, data extraction, quality assessments, classification scheme and validity; the ”results” which present the aggregation, visualization and classification details realized by the mapping; lastly, the ”discussion” (discussion/conclusions), where the results are interpreted. The ”appendix”, which Petersen’s guidelines advises should consist of both included and excluded borderline papers [8], is replaced by a bibliography, as no borderline papers were identified.

3.2. Review Protocol

3.2.1. Search

The search strategy employed in this study is a modified population, intervention, comparison and outcomes (PICO) approach with a focus on the population and intervention components as recommended by Petersen et al. [8]. The population targeted in this mapping study are those scholarly papers for higher education institutes focusing on a specific process improvement (intervention) known as a maturity model. The comparison is between model orientations, and no measurable outcomes are considered. Candidate keywords were identified from known papers, as shown in

Table 2. Keywords considered and corresponding sources.

Paper	Keyword or Search Term Considered
P. Brooks, O. El-Gayar, and S. Sarnikar, “A framework for developing a domain specific business intelligence maturity model: Application to healthcare”, [7]	Business intelligence, maturity level, maturity model
D. Duarte and P. V. Martins, “A Maturity Model for Higher Education Institutions”, [6]	Maturity; education; process; improvement
E. Tocto et Al., J. Linkolk, J. Turpo, and S. Paz, “A Systematic Review of the Application of Maturity Models in Universities”, [5]	(“maturity model” OR “capability model” OR “maturity level”) AND (“higher education” OR “university organization” OR “university”)
M. Spruit and C. Sacu, “DWCMM: The data warehouse capability maturity model”, [18]	Data warehousing, business intelligence, maturity modelling, mobile analytics

.

As the focus of this study is on business intelligence and maturity models for higher education institutes, the following keywords were selected from the pool of candidates presented in Table 2: maturity level, maturity model, capability model, higher education, university organization, and university. Prior studies were not found that included college and college synonyms as search keywords; however, colleges are higher education institutes that use BI and are increasingly offering four-year bachelor’s degrees [27], similar to university. To account for this similarity, college and community college were both added as keywords. Business intelligence was excluded as a keyword to remain as broad as possible in the search and allow the identification of other classifications of models. The candidates from [6] were excluded for being too broad, and all others were excluded for lacking relevance. These generalized search terms and search chain are presented in

Table 3. Conceptual terms and their search chains.

Terms	Chain
Maturity model	(“maturity model” OR “capability model” OR “maturity level”) AND
Higher education institution	(“higher education” OR “university organization” OR “university” OR “college” OR “community college”)

, showing the mapping of the concepts to the starting search chains. Indexing services such as AU Discover suggest relevant keywords as a service during search input. Where used, these suggestions and any service specific options selected are recorded in

Table 4. Article databases, utilized search string and selected options.

Indexing Service	Search String	Options
AU Discover	(maturity model OR capability model OR maturity level) AND (higher education or college or university or post secondary or postsecondary)	Limit to: scholarly (peer reviewed) journals; full-text language: English. Expanders: furthermore, search within the full text of the articles; apply equivalent subjects
ERIC	(“maturity model” OR “capability model” OR “maturity level”) AND (“higher education” OR “university organization” OR “university”)	Limit to: peer reviewed only
IEEE Explore	Command Search: (“Abstract”:“maturity model” OR “Abstract”:“capability model” OR “Abstract”:“maturity level”) AND (“Abstract”:“higher education” OR "Abstract":“university organization” OR “Abstract”:“university”) UNION (“Title”:“maturity model” OR “Title”:“capability model” OR “Title”:“maturity level”) AND (“Title”:“higher education” OR “Title”:“university organization” OR “Title”:“university”)
ACM Digital Library	Advanced Search: [[All: “maturity model”] OR [All: “capability model”] OR [All: “maturity level”]] AND [[All: “higher education”] OR [All: “university organization”] OR [All: “university”]]	Search: The ACM Guide to Computing Literature Journal and conference proceedings
ScienceDirect	(“maturity model” OR “capability model” OR “maturity level”) AND (“higher education” OR “university organization” OR “university”))	Article type: review articles, research articles; options: title, abstract, keyword search

.

The general search string used in this study is a Boolean combination of the identified keywords meant to restrict papers to the relevant domain and focus area. In each of the databases searched, the actual search string applied is shown in Table 4.

3.2.2. Search Resources

The use of abstracting and indexing services both broadens and simplifies the collection of high-quality material for study. To conduct the search, material was collected from the following abstracting and indexing services: AU Discover, ERIC, ProQuest, IEEE Explore, and Science Direct.

This study was conducted in July 2022, all material published prior to the month of July in 2022 are included in these initial results. Due to the volume of these initial search results, a reference management software, JabRef 5.6 was used for organization, duplicate removal, and to provide the bibliography of excluded papers used in the Appendix A.

3.2.3. Inclusion and Exclusion Criteria

To ensure reproducibility and high-quality standards, this study was undertaken with defined inclusion and exclusion criteria. Material was collected from indexed digital libraries available on abstracting and indexing services.

The type of material included was restricted to material dealing with maturity models applied or designed for use in the higher education institution domain with the search terms defined as in Table 2. Inclusion was further restricted to only English language articles, conference papers, reviews, and conference reviews; this was performed to ensure the exclusion of books, magazines, and other practitioner or proprietary literature. Exclusions were removed from the pool of included material. The criteria for exclusion included all articles that did not include both dimensions (or key processes) and levels in their maturity models, as these would be incomplete and untestable models. Duplicated articles were also removed. Articles where the full text was not available were excluded. All articles whose titles were not related to the objective of this study were also removed. Articles written prior to 2007 were also excluded.

3.2.4. Selection Procedures

Articles were collected using the indexing and database services, search strings, and options as presented in Table 4 resulting in the initial quantity of candidate articles shown in

Table 5. Articles successfully gathered into reference manager software.

Source	Count of Results Imported to Jabref
AU Discover	3099
ERIC	1501
IEEE Explore	635
ACM Digital Library	791
ScienceDirect	20

.

The order of selection procedures is shown in Figure 2, in sequential order from top to bottom. The inclusion criteria were applied to article metadata, titles, and abstract readings. Following this, remaining articles were subjected to the exclusion criteria through reviewing the metadata, titles, and abstracts.

A full text reading of the remaining articles was then undertaken, and the snowball method was employed. Manually, papers or maturity models referenced in the full text of review articles that were covering maturity models were reviewed and added to the selected article. The snowball effort had a single hop threshold, meaning only one level of articles was included. Efforts were made to identify systemic reviews or mappings that could point to undiscovered works. Finally, any papers found not to meet the inclusion criteria or met the exclusion criteria were removed at this phase.

3.2.5. Data Integration

After the selection procedure is performed, each selected article is read and the following data are collected: article title, author name, year published, publisher, maturity model, maturity model orientation, model framework used, dimension number, level number, dimensions, and levels. The data are recorded into a collection template which includes columns for all data items as laid out in

Table 6. Data extraction form.

Data Item	Data Description	RQ
Article title	Name of the article
Author name	Authors names
Year published	Calendar year	RQ5a
Publisher	The publishing organization	RQ5b
Maturity model	The maturity model described. Same as StudyID if none given	RQ1
Maturity model orientation	Is the model oriented toward BI, IT, ICT, students, Teaching or other areas	RQ2, RQ3
Model framework used	The framework used to generate the maturity model	RQ4, RQ6
Dimension number	Number of dimensions	RQ3
Level number	Number of levels	RQ3
Dimensions	The descriptive names for the dimensions.	RQ3
Levels	The descriptive name for the level. Recorded in ascending order

. This template was generated including data items, their descriptions, and the research questions as per [8] with inclusion of those data items necessary to address the indicated research questions. Each of the selected articles were reviewed by a single author.

3.2.6. Classification

Prior systemic reviews of maturity models in use at universities used no a priori categorical structure and imposed a categorization schema according to the purpose of the model [5]. This study adopts the nine categories of maturity model used by [5], updating the categories to include colleges (HEI in general), and adds maturity models oriented toward business intelligence in higher education institutions.

• Maturity models oriented toward teaching.
• Maturity models oriented toward information and communication technology (ICT).
• Maturity models oriented toward student monitoring.
• Maturity models for intellectual capital.
• Maturity models for e-learning.
• Maturity models aimed at evaluating higher education institution entrepreneurship.
• Maturity models oriented toward the employability of graduates.
• Maturity models oriented toward the strategic planning of higher education institutions.
• Maturity models for IT governance in higher education institutions.
• Maturity models for BI/BIA in higher education institutions.

3.2.7. Study Quality Assessment

This research was conducted to gather quantitative categorical information on the extant literature so as to give a broad overview of the topic area, as per established guidelines on conducting systemic mappings [8]. As such, no quality assessment was completed.

3.2.8. Study Reproducibility

To improve reproducibility, this study was undertaken using existing guidelines for systemic mapping as established by Petersen et al. [8]. The selection process, sources, search terms, and detailed results are reported. An appendix has been included which includes the selected papers and those papers excluded at full text reading.

4. Results

4.1. Search Results

Performing the searches using the search string and options indicated in Table 4 resulted in an initial finding of 5994 articles. Applying the inclusion and exclusion criteria removed 5607 articles, including duplicates. A further 318 articles were removed upon examination of the abstract, resulting in 69 articles qualified for full-text reading (Figure 3). During full text reading, articles were examined against inclusion/exclusion criteria and for maturity models as described in 3.2.4 (Selection Procedures), resulting in a further reduction of 39 articles. From the remaining 30 articles, 43 candidate articles were identified using the snowball method and subjected to stages 2–4, as shown in Figure 3. Further snowballing was cut off at this stage. After removing 32 of the snowball articles (primarily for duplication) the final number of selected articles was 41.

Initial search results are predominantly found in AU Discover (3099), followed by ERIC (1501), ACM Digital Library (791), IEEE Explore (635), and Science Direct (20), as shown in Table 5. After processing, sources were evaluated for their success rate at returning papers that get selected, shown in Figure 4. The snowball method was most successful at converting candidates into selected papers at 25.58%, followed by Science Direct at 10% and IEEE Explore at 2.67%. The least successful source was ERIC at 0.067%.

A keyword analysis on all selected papers was performed. Keywords were tokenized and processed using the bag of words method to generate a word cloud (Figure 5) and an undirected graph (Figure 6), with nodes representing keywords and edges representing co-occurrences of keywords. As shown in Figure 5, the top 15 keywords in descending order were maturity model, education, capability maturity model, electronic learning, organizations, information technology, planning, decision making, foreign countries, higher education, industries, IT governance, maturity level, software, and standards organizations.

The co-occurrence network graph used a window size of 30 and a threshold of 1 to capture all keyword relations. There are no isolated vertices, but there are three isolated graphs containing keywords related to green it, teaching maturity models, and the eqtic model. These isolated graphs represent papers with keywords that have no connection to the main body of work. The full graph then is disconnected with infinite radius. Removing the three isolated sub-graphs, we may evaluate what remains. The largest connected sub-graph contains 220 nodes, 1351 edges, an average degree (number of times a keyword appears with other keywords) of 12.28, a diameter of 6 (the shortest path between the two most distant keywords), a radius of 3, and an average shortest path of 3.

The top 10 keywords by degree centrality and closeness centrality are shown in

Table 7. Properties of most central keywords ranked by degree centrality.

Keyword	Degree	Average Neighbour Degree	Degree Centrality	Rank
education	67	16.0448	0.305936	1
maturity model	66	12.0303	0.30137	2
electronic learning	52	17.0192	0.237443	3
capability maturity model	42	14.881	0.191781	4
planning	39	19.0769	0.178082	5
organizations	39	17.5385	0.178082	5
industries	35	16.0286	0.159817	7
foreign countries	34	17.7647	0.155251	8
information technology	32	18.9375	0.146119	9
technology integration	30	18.6333	0.136986	10

and

Table 8. Properties of most central keywords ranked by closeness centrality.

Keyword	Degree	Average Neighbour Degree	Closeness Centrality	Rank
education	67	16.0448	0.506944	1
planning	39	19.0769	0.495475	2
maturity model	66	12.0303	0.476087	3
higher education	28	22	0.458159	4
analytical model	26	21.2692	0.450617	5
standards organizations	22	20.6818	0.449692	6
monitoring	24	20.875	0.446939	7
electronic learning	52	17.0192	0.44332	8
information technology	32	18.9375	0.435388	9
organizations	39	17.5385	0.428571	10

, respectively. In Table 7, 7 of the 10 keywords were not original search terms. Similarly, in Table 8, 7 of 10 were not originally search terms. Business intelligence (

Table 9. Properties of keyword business intelligence.

Keyword	Degree	Average Neighbour Degree	Degree Centraility	Rank (DC)	Closeness Centrality	Rank (CC)
business intelligence	4	10	0.0182648	185	0.298365	185

) ranks 185th out of all 220 nodes in terms of either centrality measure. Figure 7 shows a one-hop sub-graph from Figure 6, centred around the business intelligence keyword. The business intelligence keyword sub-graph is four normal, connecting the decision making, university, analytics, and governance nodes (Figure 7). Only the decision-making node has any direct connection to keywords outside the sub-graph.

4.2. Publication Frequency

The first year included in the search returns a single publication, Petch2007 [28]. The last full year contains five publications: Ahmad2021 [29], Almonte2021 [30], Kashfi2021 [31], Setiadi2021 [32], and Silva2021 [33]. The distribution of publications over time from 2007 to end of 2021 are as described in

Table 10. Descriptive statistics of publications over years.

Statistic	Value
Range	0–7
Mean	2.6
Median	2
Mode	1
Standard deviation	2.09
Q1, Q2, Q3, IQR	1, 2, 5, 4
Skewness	0.7758

. The distribution has a left skew, showing an increasing trend over time. The most common publisher for works on HEI maturity models at HEIs is IEEE at 41.463%, followed by Association for Computing Machinery with 12.195% of publications, as shown in Figure 8. All other publishers are below 4.878% or two published works. The range of works spanned the years 2007–2022, with no publications found in 2009, as shown in Figure 9.

4.3. Identified Maturity Models

From the 41 academic works accepted in this mapping study 33 unique maturity models were identified based on 11 frameworks. The details of the identification are shown in Appendix A,

Table A1. Maturity model identification and classification.

Identifier [Citation]	Maturity Model	Maturity Model Orientation	Basis for Maturity Model
Ahmad2021 [29]	Digital Maturity Model (DMM)	MM aimed at evaluating HEI entrepreneurship.	DMM
AlAmmary2016 [40]	e-Learning Maturity Model (eMM)	MM for E-Learning.	CMM/SPICE
Almonte2021 [30]	IT Services Management Model (ITSM)	MM oriented toward ICT.	ITSM Framework
Alqassemi2017 [37]	Service-Oriented Architecture Maturity Model (SOA MM)	MM oriented toward ICT.	None Identified
Arezki2018 [41]	IT Governance Maturity Model (ITGMM)	MM for IT governance in HEI.	EFQM
Awasthy2018 [35]	University-Industry Collaboration Maturity Model (UICMM)	MM aimed at evaluating HEI entrepreneurship.	Design Science
Baolong2018 [42]	Data Management Maturity Model (DMMb)	MM for intellectual capital.	CMM/CMMI
Behrendt2022 [43]	Appelfeller / Feldmann MM (Industry 4.0 MM)	MM oriented toward teaching.	None Identified
Boehm2013 [44]	Further Education Maturity Model (FEMM)	MM for E-Learning.	Design Science
Cardoso2022 [36]	Higher Education Business Intelligence Analytics Maturity Model (HE-BIA MM)	MM for BI/BIA in HEI.	Design Science
Carvalho2018 [45]	Test Maturity Model integration (TMMi)	MM oriented toward student monitoring.	CMMI
Clarke2013 [46]	Student Engagement, Success and Tetention Maturity Model (SESR MM)	MM oriented toward student monitoring.	None Identified
CombitaNino2020 [47]	TDWI Analytics Maturity Model (TDWI AMM)	MM oriented toward ICT.	Proprietary
Cruz2020 [48]	None Given (Cruz2020)	MM aimed at evaluating HEI entrepreneurship.	None Identified
Durek2018 [49]	DMFHEI (DMMHEI)	MM aimed at evaluating HEI entrepreneurship.	Design Science
Durek2019 [50]	Digital Maturity Framework for HEI (DMFHEI)	MM oriented toward ICT.	Delphi
Elgrably2020 [51]	Test Maturity Model integration (Tmmi)	MM oriented toward student monitoring.	CMMI
Freitas2020 [52]	MM for Learning Analytics (Freitas2020)	MM oriented toward ICT.	Design Science
Gu2011 [53]	Online Course Quality Maturity Model (OQCMM)	MM for E-Learning.	CMM
Ishlahuddin2020 [54]	Control Objectives for Information and Related Technologies (COBIT2019)	MM for IT governance in HEI.	CMMI
Jali2018 [55]	e-Learning Maturity Model (eMM)	MM for E-Learning.	CMM/SPICE
Kashfi2021 [31]	e-Learning Maturity Model (eMM)	MM for E-Learning.	CMMI
Konsky2008 [56]	Team Software Process integration (TSPi)	MM oriented toward student monitoring.	CMMI/TSPi
Kosasi2017 [57]	Control Objectives for Information and Related Technologies Maturity Model (COBIT 4.1 MM)	MM for IT governance in HEI.	CMM
KropsuVehkaperae2013 [58]	People-Capability Maturity Model (P-CMM)	MM oriented toward the strategic planning of HEI.	CMM
Marchewka2013 [59]	Capability Maturity Model for Assurance of Learning (CMM for AOL)	MM oriented toward teaching.	CMMi
Muntean2019 [38]	None (Muntean2019)	MM oriented toward the employability of graduates.	Design Science
Nsamba2019 [60]	Maturity Assessment Framework for Open Distance E-Learning (MAFODoL)	MM for E-Learning.	CMM/OCDMM
Pasini2019 [61]	Service Maturity Model (Pasini2019)	MM oriented toward ICT.	None Identified
Pawan2019 [62]	Control Objectives for Information and Related Technologies Maturity Model (COBIT 4.1 MM)	MM for IT governance in HEI.	CMM
Petch2007 [28]	e-Learning Maturity Model (eMM)	MM for E-Learning.	CMM/SPICE
Putri2014 [63]	Green IT Governance Model for Private Higher Education Institutions (Putri2014)	MM for IT governance in HEI.	CMM
Reci2017 [64]	Teaching Maturity Model (TeaM MM)	MM oriented toward teaching.	CMMi
Rossi2015 [65]	Quality Model for Educational Products based on Information and Communication Technology (eQETIC)	MM oriented toward ICT.	None Identified
Secundo2015 [66]	Intellectual Capital Maturity Model for Universities (ICMM)	MM for intellectual capital.	Expert Panel
Setiadi2021 [32]	Data Quality Management Maturity Model (DQMMM)	MM oriented toward ICT.	CMM
Silva2010 [33]	Maturity Model for Academic Process Management (Silva2010)	MM oriented toward the strategic planning of HEI.	CMM/CMMi
Silva2021 [67]	Maturity Model for Collaborative R&D University-Industry Sustainable Partnerships (Silva2021)	MM aimed at evaluating HEI entrepreneurship.	CMMI
Thong2012 [34]	Curriculum Design Maturity Model (CDMM)	MM oriented toward teaching.	CMM
Xing2018 [68]	Maturity Model for Examination Management in University (Xing2018)	MM oriented towards teaching.	CMM
Zhou2012 [69]	e-Learning Process Capability Model (ePCMM)	MM for E-Learning.	eMM

. One maturity model for BI for HEI was identified, it was developed using a design science framework.

4.4. Mapping Results

Examples of each model orientation were discovered in the selected works, as shown in Figure 10. The least occurring mappings were maturity models oriented toward business intelligence and maturity models oriented toward the employability of graduates with one work each. Maturity models oriented toward e-learning and maturity models oriented toward ICT were the most common model types found, occurring eight times each. Maturity models for e-learning were the longest existing model orientation, ranging from 2007–2021, while Maturity models for BI/BIA in HEI were the latest to appear, as shown in

Table 11. Publication range by orientation.

Maturity Model Orientation	Published Range (Years)
Maturity models for BI/BIA in higher education institutions	2022–2022
Maturity models for IT governance in higher education institutions	2014–2020
Maturity models oriented toward the employability of graduates	2019–2019
Maturity models oriented toward the strategic planning of higher education institutions	2010–2013
Maturity models aimed at evaluating higher education institution entrepreneurship	2018–2021
Maturity models for e-learning	2007–2021
Maturity Models for intellectual capital	2015–2018
Maturity models oriented toward information and communication technology	2015–2021
Maturity models oriented toward student monitoring	2008–2020
Maturity models oriented toward teaching	2012–2022

.

4.5. Maturity Model Development Frameworks

A total of 15 frameworks or approaches to developing maturity models were identified as part of the mapping. The most common model frameworks identified, seen in Figure 11, were the capability maturity model (CMM) and the capability maturity model integration (CMMi), which is based on the CMM. These process improvement maturity models are used either directly, or as the basis in 22 of the published works, as shown in Figure 11. CMM/CMMi frameworks were combined with other approaches in 5 of those 22 works. Design science approaches were identified as the reported basis of seven works since 2013, including the single maturity model oriented toward BI/BIA in HEIs. Design science approaches were most common in maturity models aimed at evaluating higher education institution entrepreneurship and maturity models oriented toward information and communication technology (ICT).

4.6. Characteristics of Maturity Models

An analysis of the maturity model dimensions, where identified, was conducted on all selected papers. Dimensions were tokenized and processed using the bag of words method to generate a word cloud (Figure 12) and an undirected graph (Figure 13), with nodes representing dimensions and edges representing co-occurrences of dimensions.

The co-occurrence network graph used a window size of 30 and a threshold of 1 to capture all dimension relations. There are two isolated vertex representing single dimension maturity models focused on single areas or processes (Thong2012 [34], Awasthy2018 [35]). The graph has an infinite radius and is disconnected, that is, not all dimension nodes can be reached from an arbitrary dimension node. The graph consists 19 disconnected sub-graphs of two or more nodes. The largest connected sub-graph contains 62 nodes, 292 edges, an average degree of 9.419, a diameter of six (the shortest path between the two most distant dimensions), a radius of three, and an average shortest path of 3.396. For analysis, several sub-graphs are coloured in Figure 13. The largest connected sub-graph contains the dimensions of the HE-BIA MM (Cardoso2022 [36]) as shown by the red nodes, orange nodes represent the dimensions of DQMMM (Setiadi2021 [32]), and green nodes are dimensions of the SOA MM (Alqassemi2017 [37]). Yellow nodes are shared dimensions between the models, such as architecture, between HE-BIA MM and SOA MM, and governance, between SOA MM and DQMMM. The triangle/polygonal sub-graphs with a single common node, as above, represent shared nodes between individual maturity models or shared nodes between clusters of maturity models with shared dimensions and can be seen throughout Figure 13.

Of the model orientations with more than one example, there is a spread between the minimum and maximum number of dimensions and the average number of dimensions used in measuring maturity. The model with the most dimensions is Silva2010 [33] with 19, where HE-BIA MM (Cardoso2022 [36]) has 18. The maximum spread is in the Maturity models oriented toward teaching, with spread of 16.

5. Discussion

5.1. Research Questions

5.1.1. To What Extent Do Maturity Models for Higher Education Exist in the Literature? (RQ1)

This mapping study identified 41 academic works, covering 33 unique maturity models for HEI over the 15-year time span covered, as per results 4.3 (Maturity Models Discovered). These maturity models were found in 10 distinct orientations (Figure 10). The models were published by 18 different publishers, of which IEEE and ACM were the most common, accounting for 41.463% and 12.195%, respectively (Figure 8). Published works regarding maturity models are increasing over time, as shown by the left skew in the distribution of publications (Figure 9/Table 10).

5.1.2. To What Extent Are Maturity Models for Higher Education Oriented towards Business Intelligence? (RQ2)

In the literature, BI maturity models for higher education are represented the least of the 10 categories, whereas maturity models for e-learning and maturity models for ICT are the most common (Figure 10). This study identifies one maturity model for HEI oriented toward BI in the HEI-BIMM (Cardoso2022 [36]), and it was published only at the very end of the reviewed time frame, as shown in Section 4.4 (Maturity Models Discovered) (Table 11). The low values for centrality (Table 9) and the single co-occurrence of keywords (Figure 6 and Figure 7) in the business intelligence keyword sub-graph provide further evidence that maturity models oriented toward BI are not as common as other model orientations. Only the decision making keyword co-occurs in other works, providing further evidence that business intelligence for higher education institutions is not a primary domain being covered by other maturity models.

5.1.3. What Are the Differentiating Characteristics of Existing BI Maturity Models for Higher Education Institutions? (RQ3)

Maturity models in this study were differentiated by orientation (Figure 10), then by number (

Table 12. Range of MM dimensions by orientation.

Model Orientation	Count of Models Found	Min Dimension Number	Max Dimension Number	Average Dimension Number
Maturity models for BI/BIA in higher education institutions.	1	18	18	18
Maturity models for IT governance in higher education institutions.	4	3	13	8.25
Maturity models oriented toward the employability of graduates.	1	3	3	3
Maturity models oriented toward the strategic planning of higher education institutions.	1	19	19	19
Maturity models aimed at evaluating higher education institution entrepreneurship.	5	1	7	3.8
Maturity models for e-learning.	8	3	6	4.875
Maturity models for intellectual capital.	1	4	4	4
Maturity models oriented toward information and communication technology (ICT).	8	3	8	5.75
Maturity models oriented toward student monitoring.	3	3	5	4
Maturity models oriented toward teaching.	5	1	17	9.4

), and kind of dimensions (Figure 12), as well as the co-occurrences of dimensions (Figure 13), by the levels used to measure the dimensions, and by the framework used as the basis for model creation (Figure 11). This study identified only one maturity model for HEIs oriented toward BI in academic literature. In [36], the OCU model is also identified but is from an industry source and did not appear in the peer reviewed literature, so it was not included. The HEI-BIMM has 18 dimensions (BIA strategy, academic analytics support, data governance, change management, sponsorship, process coverage, system usage, user capabilities, user groups, analytical and decision-making culture, training, technical integration, IT infrastructure, architecture, traditional data products, advanced analytics, data velocity, data variety) organized into seven categories (value, program management, business process development, people, technical foundations, data products, data) measured by five well-defined levels (pre-adoption, initial, managed, systemic, optimized). As seen in Figure 14, only the architecture dimension has a co-occurrence with other models. However, when the architecture dimension was evaluated using the definition provided by the models it was found that the dimensions are co-occurring in name only, and the definitions did not match. The HEI-BIMM is based on design science.

5.1.4. What Methods Exist for Generating BI Maturity Models? (RQ4)

This mapping found one BI maturity model generated using design science techniques (Table 12), including recruiting a panel of experts for iterative design and validation, the generation of an artefact, and a separate series of case studies for evaluation (Cardoso2022 [36]). This model was built using the methods described by [38] with design science considerations, as identified by [7,22]. While only one method of generating BI maturity models for HEI was identified by this mapping, 10 other methods for generating maturity models frameworks generated maturity models for HEI were found (Figure 11).

5.1.5. In Which Sources and in Which Years Were BI Maturity Models for Higher Education Published? (RQ5)

Maturity models for e-learning and student monitoring were identified as the first to be developed in 2007 and 2008, respectively (Table 11). In comparison, the BIMM is the latest, appearing in 2022. While most works identified had been published in IEEE (17) or ACM (5), the BIMM identified was published in Applied Sciences by MDPI as a peer reviewed article. The only other maturity model orientation with a single entry identified is Muntean2019 [39], which is oriented toward the employability of graduates (Table A1) and was published in 2019 in the journal Sustainability by MDPI.

5.1.6. What Were the Primary Methods for Developing BI Maturity Models for Higher Education in the Published Literature? (RQ6)

As in Section 5.1.4, design science is the primary method for developing BI maturity models for higher education in the published literature. This participatory approach, involving practitioners from a European HEI BI special interest group were recruited to develop dimensions, validate definitions, and review level details with their input being used to modify and validate the model iteratively.

5.2. Discussion

The publishing of academic literature concerning the use of maturity models for HEI has been increasing since 2017 after an initial period of steady production from 2007 to 2016, as discussed in Section 5.1.1 and shown in Figure 8. IEEE is the primary publisher of maturity models with 22.41% of all models published appearing there, followed by publications by the Association for Computing Machinery at 12.20%, as shown in Figure 9 and according to the results in Section 4.2.

CMM and CMMi are the most common model frameworks found in use at HEIs, as shown in Section 4.5. Design science methodology first appears in 2013, with over 85% of articles using design science as a framework occurring after and in 2018, including being used in the development of the HE-BIA MM.

As conjectured in Section 2, the conditions were right for the emergence of a domain-specific BI maturity model for HEIs, and indeed this has recently occurred with the development of the HE-BIA MM, as shown in Section 5.1.2. Still, as determined in Section 5.1.5, maturity models for HEIs oriented toward BI remain uncommon, accounting for only one of the identified works and leaving an opportunity for further research (see Section 5.3, Future Directions). Having only a single identified maturity mode for HEIs oriented toward BI results in only a single example of identified dimensions, levels, development contexts, and frameworks used.

The characteristics of the BI maturity model for HEIs could not be compared to existing academic examples, as none were found Section 5.1.5. Compared to other maturity models with different orientations, the BI-HEI MM had more dimensions than only one other model, as shown in Section 4.6.

The dimension co-occurrence network shown in Figure 14 allows for the discovery of potential shared dimensions based on dimension names. The architecture dimension, shown in yellow, suggests one such shared dimension between the HE-BIA MM and SOA MM. Upon review of the definitions, the architecture dimension of the HE-BIA MM and SOA MM were not equivalent, the SOA MM architecture dimension was defined in terms of cloud infrastructure [37], and the HE-BIA MM explicitly excludes the cloud in this architecture dimension, including it instead in the IT infrastructure node [36]. This lack of equivalence in dimensions is discussed further in Section 5.3 Future Directions.

5.3. Future Directions

While the immediate gap for a BIMM for HEIs has been filled, work remains. The HE-BIA MM was developed and validated within the context of European higher education [36]. There is opportunity to validate the generalizability of the model with case studies in another region, such as North America. All other model orientations, except maturity models oriented toward the employability of graduates, as shown in Figure 10, have seen the development of multiple models. Given that the HE-BIA MM is the only model identified oriented toward BI in HEIs, opportunities exist to generate other models, as has occurred in other orientations. Replication studies resulting in the generation of additional models would provide opportunities to validate the HE-BIA MM by means of comparison of dimensions, levels, and definitions.

As demonstrated by analysis of co-occurring dimensions in Figure 13 and its sub-graph containing BI dimensions (Figure 14), dimensions with the same name do not necessarily indicate equivalent concepts. To understand the relationships of dimensions between maturity models, future analysis standardizing dimensional definitions could be performed to evaluate dimensions, identify conceptual equivalences, and analyse them using co-occurrence networks. A co-occurrence network with dimensions of standardized definitions will enable the analysis of relationships between maturity models or enable the discovery of dimensions included in some models, but not others, inside the same domain. It was discussed in [36] that BIA is connected to the larger area of digital maturity; this study did not confirm that with the co-occurrence network, but a standardization of dimensions and their definitions may produce evidence of this.

Further extension of the HE-BIA MM is suggested by [36], including the development of instruments designed for sharing best practices and providing guidelines or actionable insights for HEI BI practitioners to improve their own practice. This paper contains a keyword analysis by means of co-occurrence network that may be used in future systemic works to generate improved search terms for finding maturity models oriented toward HEIs. Keywords with high centrality appeared more often with keywords of differing groups, making them ideal search terms for breadth. Of the fifteen keywords identified in Figure 6, eleven were not used in the search. This is an area for improvement in future mapping studies that could lead to more robust results. The keyword analysis also identified three isolated sub-graphs representing academic works that shared no keywords with the main graph. These keywords, including: eqetic model, online education, educational quality model, digital education solution, green it, green it model, private higher education institution, computer science education, primary and secondary school, and teaching maturity model could be included in future systemic mappings to ensure the capture of relevant yet unconnected papers. Authors producing works in the areas covered by the isolated sub-graphs keywords may add more central, relevant keywords to their own works to improve the visibility and accessibility of their works.

5.4. Limitations and Validity

Despite efforts to improve validity, there is potential for validity threats in mapping studies on a number of basis as identified by Petersen [8], while efforts were made to ensure proper recording and identification of classifications and characteristics of studies, the descriptive validity could be threatened by the use and design of the extraction forms and the reality that this study is the undertaking of a single individual without external validation. The theoretical validity of the work could be threatened by researcher bias in selection or the quality of studies; efforts were made to decrease these biases using a well-described methodology based on an existing framework, a wide number of search sources, a restriction to peer reviewed works, and the use of model orientations from prior work. However, as discussed in 3.2.7, no quality assessment was performed on the works collected for this mapping. Frameworks were self-declared by the authors without validation for this study. The threat of researcher bias extends similarly to conclusions drawn in relation to the data collected. An additional threat lies in the generalisability of the results and there were limitations identified in terms of the search engines that the author lacked access to (SCOPUS) or declined to use for technical reasons (Google Scholar) that could have changed the results. An additional limitation is the absence of industry or practitioner maturity models. This list of identified threats and limitations is unlikely to be exhaustive.

5.5. Conclusions

Higher education institutions are instituting business intelligence to make better decisions, but these HEIs require a framework to judge their current BI maturity and target higher maturity levels. Until recently, there existed no higher education-specific maturity models for BI available to them. This study has demonstrated that while maturity models and the means to generate them for higher education exist in the academic literature, it is only recently that a concurrence of published frameworks and interest in BI has led to the generation of a BI maturity model for HEI, the HE-BIA MM [36]. This study shows by means of enumeration of relevant models, comparison across model orientations, and keyword analysis showed that business intelligence had a very low ranked centrality, that this, is a new and developing area of research with opportunities for additional work in replication and validation studies, including case studies in non-European contexts. Additional research may be performed on the relationship between this model and others in the HEI domain, which may be aided with efforts to standardize dimension definitions and names across maturity models. As shown, the future of research in BI maturity models for HEIs is primed for activity and could prove a boon to HEIs looking to improve their BI maturity and gain a competitive edge in an increasingly data-driven world.