Academic Advising Systems: A Systematic Literature Review of Empirical Evidence

Abstract

This paper aims to provide the reader with a comprehensive background for understanding current knowledge on Academic Advising Systems (AAS) and its impact on learning. It constitutes an overview of empirical evidence behind key objectives of the potential adoption of AAS in generic educational strategic planning. The researchers examined the literature on experimental case studies conducted in the domain during the past ten years (2008–2017). Search terms identified 98 mature pieces of research work, but inclusion criteria limited the key studies to 43. The authors analyzed the research questions, methodology, and findings of these published papers and categorized them accordingly. The results have highlighted three distinct major directions of the AAS empirical research. This paper discusses the emerged added value of AAS research and highlights the significance of further implications. Finally, the authors set their thoughts on possible uncharted key questions to investigate both from pedagogical and technical considerations.

1. Introduction

With the advent of flexible curriculum systems in many Higher Educational Institutions (HEI) and an ever wider variety range of courses and programs being offered, there is a present need to ensure that students make the best use of available information to make more informed decisions regarding their academic plan [1]. Besides required courses, which are compulsory for each student to be taken, HEI also offer elective courses, but generally students lack information about the objectives and the content of the course and often fail to take the appropriate ones for their academic plan. Moreover, because the number and variety (in backgrounds, in knowledge, in goals) of students is also expanding rapidly, it is increasingly important to tailor learning processes to students, since the same learning pathway is unlikely to best serve all students [2].

HEI usually employ guidance counselors, people who are tasked with helping students making their choices. Three principle models of advising include developmental advising, prescriptive advising, and intrusive advising, each of which is informed by the goals of the advisor-student interaction [3]. In all approaches, this support is based on the establishment of educational collaboration between students and their academic advisors. Advisors assist students by guiding them through the university’s educational requirements, helping them schedule the most suitable modules, introducing them to pertinent resources, promoting leadership and campus involvement, assisting in career development, helping them with the timely completion of their studies, and helping them find ways to make their educational experience personally relevant [4]. However, in practice the counselors are often overloaded with too many students and not enough time, and some students are not satisfied with the quality of academic advising that the counselors provide [5]. Good advising yields a good outcome in terms of understanding, planning, and applying strategies for academic success, while bad advising will be frustrating and may have a damaging effect on students’ progress [6].

In order to support the educational process and to relief the HEI actors, a software system is needed that will handle the advisory process in an efficient and effective way. An AAS can serve as a strategic partner responsible for the process of supporting, motivating the student’s study plan, and assisting in the achievement of their educational goal [7]. However, unlike most existing recommendation systems, AAS require dealing with a large decision space, which grows combinatorially with the number of courses, programs, and the students’ different backgrounds, knowledge, and goals but is also subjected to many constraints (e.g., course prerequisites, maximum credit hour load, course priorities, etc.). In [8], the authors describe the conceptual framework of a web-based AAS. In their study, more than 60% of users, consisting of 361 students and 155 faculty members, agreed that the tasks of academic-related matter such as course registration, course selection, academic progress, course information, course scheduling, plan of study, and academic calendar should be included in the AAS. They also discuss the importance of making AAS that are more than data repositories and including more intelligence so the systems are able to provide reliable advice when helping the advisor and the student alike.

2. Motivation and Rationale of the Study

The motivation for this review is derived from the fact that empirical evidence is required for theoretical frameworks to gain acceptance in the scientific community [9]. A search in relevant literature did not reveal any reviews of empirical evidence of the added value of research in the specific domain. Consequently, there was a need to supply the audience with an accredited overview. This paper aims to fill that gap.

The value of any single study is derived from how it fits with and expands previous work, as well as from the study’s intrinsic properties. Thus, putting together all the unbiased, credible results from previous research would be a step towards understanding the whole picture and constructing a map of our knowledge of the domain. In a sense, the rationale of our study was to manage the overwhelming amount of publications through the critical exploration, evaluation, and synthesis of the previous empirical results that merit reflection.

This paper’s goal is to carry out a systematic review of empirical evidence in order to contribute towards

• a complete documentation of the applied research approaches so far;
• a feasibility study that captures the strengths and weaknesses of research in the domain, and
• the identification of possible threats, and thus to motivate the research community to redefine or refine related questions or hypotheses for further research (opportunities).

3. The Research Questions

The following research questions need to be addressed and are distinguished into primary (generalized: set to fulfill the goals of the review) and secondary (sub-objectives/specific: refine the primary—explanatory):

• RQ1 (Primary)—Research Objectives: What are the basic research directions of AAS up to now (in terms of measurable metrics) and which approaches do researchers follow to achieve these goals?
  • RQ1.1 (Secondary)—What are the most significant results from past AAS research that constitute empirical evidence with regard to their impact on the learning process?
  • RQ1.2 (Secondary)—Interpretation of the results: What do these results indicate regarding the added value of this technology?
• RQ2 (Primary)—Future challenges: Which other emerging research approaches should be explored in the AAS research area?

4. Research Methodology

Our process for finding and including or excluding papers is summarized in Figure 1. To increase our coverage, we searched for relevant papers by conducting both a systematic search of available research paper databases and by “snowballing”, starting with a set of core papers believed to be in-scope, and expanding our set of consideration based on papers referenced by these papers.

Figure 1. Survey methodology.

Two main methods were initially employed for this search. The first was to use our University’s academic search engine, which covers several academic databases such as Xplore, ScienceDirect, SpringerLink, and Wiley. The second was to consult Google Scholar (http://scholar.google.com/), which covers more publications but is less structured and provides also more low quality material. Search terms used included “academic advising systems, academic planning” in combination with “universities, higher educational institutions”, and all words were used in different combinations. The search process spanned from March 2017 to August 2017. The parameters of the search were bound within the last ten years (2008–2017), in which emergence and adoption of AAS has grown based on the number of relevant research papers.

Due to the orientation of this work towards the practical implementation and exploitation of AAS, at the end of the data collection stage, the authors explicitly determined the article inclusion/exclusion criteria (Table 1).

Table 1. Inclusion/exclusion criteria.

Include	Exclude
Articles published in Journals with Impact Factor Full-length articles published in International Conference/Workshop Proceedings Approaches applied to formal education (e.g., universities, colleges, distance education, etc.) Date from 2008 to 2017	Articles that do not present significant empirical evidence (e.g., theoretical and conceptual articles, essays, tool demonstration, etc.) Book chapters Approaches applied to non-formal education (e.g., Virtual Learning Environments, MOOC, etc.)

This first round of analysis led to the identification of only 19 unique papers. Due to such a limited number of publications, the literature base was expanded by applying a snowballing technique [10] to the initial set of papers. The reference list of each of the papers was screened for relevant contributions and 43 papers were selected for detailed analysis at the end of the process. Saturation was used as the stop criterion: we stopped the search when new papers no longer provided new challenges. The selected papers were later analyzed by classifying their content and contribution in relation to the AAS.

Next, the authors proceeded on with an article classification according to the adopted research strategy (category), recommendation technique (method), research objectives (goals), and results. Finally, we used non-statistical methods to evaluate and interpret findings of the collected studies, and to conduct the synthesis of this review.

5. Results

In this section, the authors present their findings based on the analysis of the published case studies.

According to the followed research strategy, most of the published case studies are exploratory or experimental studies. Some of them are evaluation studies, while others are empirical studies or surveys. Furthermore, the research topics differ from study to study, but most of them focus on science, technology, engineering, and mathematics (STEM).

One important parameter is the research method adopted by authors to propose personalized academic recommendations. In the field of AAS, the most popular filtering algorithms are knowledge-based, followed by Computational intelligence-based, Collaborative filtering-based, Hybrid, and content-based. Table 2 displays the classification of the key studies according to the recommendation method they adopt.

Table 2. Classification of case studies according to the research method.

Research Method	Authors & Year (Paper Ref.)
Content-based	Mostafa et al., 2014 [7], Lin et al., 2015 [11], Poeppelmann, 2011 [12]
Collaborative filtering-based	Chang et al., 2016 [1], Unelsrød, 2011 [5], Dhabi & Advisor, 2012 [13], Li, 2015 [14]
Knowledge-based	Xu et al., 2016 [2], Werghi, Naoufel, & Kamoun, 2009 [15], Roushan et al., 2014 [16], Ai-nory, 2012 [17], Mohamed, 2015 [18], Koutrika, Bercovitz, & Garcia-Molina, 2009 [19], Kristiansen, Sørensen, & Stidsen, 2011 [20], Engin et al., 2014 [21], Henderson & Goodridge, 2015 [22], Mohamed, 2016 [23], Feghali, Zbib, & Hallah, 2011 [24], Prof & Shakeel, 2012 [25], Hashemi & Blondin, 2010 [26], Laghari & Khuwaja, 2012 [27], Ahmar, 2011 [28], Aslam & Khan, 2011 [29], Naini, Sadasivam, & Tanik, 2008 [30], Albalooshi & Shatnawi, 2010 [31], Zhou & Yu, 2008 [32], (Al-ghamdi et al., 2012 [33], Nambiar & Dutta, 2010 [34], Nguyen, Hoang, Tran, Nguyen, & Nguyen, 2008 [35]
Hybrid	Deorah, Sridharan, & Goel, 2010 [36], Daramola, Emebo, Afolabi, & Ayo, 2014 [6], Sobecki & Tomczak, 2010 [37], Ragab, Mashat, & Khedra, 2012 [38], Lee & Cho, 2011 [39], Fong, Si, & Biuk-aghai, 2009 [40]
Computational intelligence-based	Abdulwahhab, Salem, & Makhmari, 2015 [41], Meller, T., Wang, E., Lin, F., & Yang, 2009 [42], Williams, 2013 [43], Adak, Yumusak, & Campus, 2016 [44], Goodarzi & Rafe, 2012 [45], Fong & Biuk-aghai, 2009 [46]

The article classification according to the research objectives (goals) is illustrated in Table 3. As seen in this table, the majority of studies investigate issues related to Selecting Courses followed by Long-Term Academic planning and Choosing Programs/Majors.

Table 3. Classification of case studies according to the research objectives.

Research Objective	Authors & Year (Paper Ref.)
Choosing Programs/Majors	Mostafa et al., 2014 [7], Meller, T., Wang, E., Lin, F., & Yang, 2009 [42], Deorah et al., 2010 [36], Aslam & Khan, 2011 [29], Zhou & Yu, 2008 [32], Ragab et al., 2012 [38], Fong et al., 2009 [40], Fong & Biuk-aghai, 2009 [46], Engin et al., 2014 [21]
Selecting Courses	Dhabi & Advisor, 2012 [13], Deline, G., Lin et al., 2015 [11], Li, 2015 [14], Daramola et al., 2014 [6], Sobecki & Tomczak, 2010 [37], Adak et al., 2016 [44], Unelsrød, 2011 [5], Chang et al., 2016 [1], Koutrika, Bercovitz, & Garcia-Molina, 2009 [19], Xu et al., 2016 [2], Huang, Chung-Yi, Chen, & Chen, 2013 [47], Koutrika, Bercovitz, Ikeda, et al., 2009 [48], Engin et al., 2014 [21], Albalooshi & Shatnawi, 2010 [31], Feghali et al., 2011 [24], Laghari & Khuwaja, 2012 [27], Ahmar, 2011 [28], Naini et al., 2008 [30], Albalooshi & Shatnawi, 2010 [31], Nguyen et al., 2008 [35], Poeppelmann, 2011 [12], Al-ghamdi et al., 2012 [33], Nambiar & Dutta, 2010 [34], Abdulwahhab et al., 2015 [41], Hashemi & Blondin, 2010 [26], Henderson & Goodridge, 2015 [22]
Long-Term Academic planning	Werghi et al., 2009 [15], Roushan et al., 2014 [16], Williams, 2013 [43], Kristiansen et al., 2011 [20], Albalooshi & Shatnawi, 2010 [31], Mohamed, 2016 [23], Feghali et al., 2011 [24], Prof & Shakeel, 2012 [25], Ai-nory, 2012 [17], Mohamed, 2015 [18]

6. Key Studies Analysis

In this section, the authors present the findings of the review process and answer the initial set research questions RQ1 and RQ1.1. The rest of the research questions (mostly the results of the case studies and their comparative evaluation, as well as current and future trends, possible gaps, and new research directions) are discussed in the next section.

RQ1: What are the basic research directions of AAS up to now (in terms of measurable metrics), and which approaches do researchers follow to achieve these goals?

Firstly, we used technology roadmapping to explore the relationships between technological resources, objectives, and the changing environment [49]. Figure 2, Figure 3 and Figure 4 show the frequency of the categories listed on Table 2 and Table 3 over the last ten years.

Figure 2. Classification of research papers by year of publication.

Figure 3. Roadmaps for Academic Advising Systems (AAS) research method evolution.

Figure 4. Roadmaps for AAS research objective evolution.

Looking at Figure 2, Figure 3 and Figure 4, we can observe some trends. In Figure 2, it seems that publications related to AAS peaked in 2010–2012, but is now beginning to rise again. The same could be said for “Knowledge-based” research method in Figure 3 and for “Selecting courses” research objective in Figure 4. However, as the counts for the remainder of the categories captured in these charts are very low overall, other trends may not be significant.

6.1. Choosing Programs/Majors

Major selection is a very important step in the academic student life. Authors in [7,36], implemented a case based reasoning (CBR) system that recommends to the candidate the most suitable major, after comparing the historical cases by the student case. Paper [42] presents two novel nearest neighbor-like classification algorithms for program recommendation, which provide a program planning service to academic advisors and students of post-secondary institutions, while paper [32] proposes an auto-decision system, which helps distance education students choose their majors. Furthermore, the basic idea in paper [29] is to design a model for testing and measuring the student capabilities like intelligence, understanding, comprehension, and mathematical concepts, plus his/her past academic record and his/her intelligence level, and applying the results to a rule-based decision support system to determine the compatibility of those capabilities with the available faculties/majors.

A HEI is always interested in predicting the number of students into programs, because it is important for the school to track the interest of students in program studies so as to ensure that adequate classroom seats are available, appropriate instructors are available, and an appropriate schedule is produced for the students. Paper [38] presents an approach using hybrid recommendation based on data mining techniques and knowledge discovery rules, while papers [40,46] propose a hybrid model of neural network and decision tree classification for tackling college admissions prediction problems for each available program.

Paper [21] suggests an expert system implemented and tested using Oracle Policy Automation (OPA) software for scholarship recommendation and eligibility checking.

6.2. Selecting Courses

As seen from Table 3, selecting courses is a primary research objective. One crucial issue in this category that authors attempt to address is how to propose careful recommendations for different courses according to students’ goals and preferences. More specifically, the authors seek to prepare course lists that satisfy several student and university constraints, some of which depend on individual student cases [2,5,6,22,24,27,28,50]. In some case studies, authors choose to apply computational, intelligence-based algorithms to reach a degree of automatic advising by combining genetic algorithms with decision trees for developing the short-term curricular schedule, as well as by combining perception marks with the registered courses [41] or by assisting in data mining and intelligent adaptive fuzzy logic for implementing an elective course suggestion system [44]. In paper [37], the authors present recommendations of student courses using ant colony optimization and concluded that their solution is promising, since it overcomes most of the disadvantages of classical approaches based on collaborative filtering in terms of performance. AAS proposed in [24] employ recommendations based on decision support tools, while other authors [33,34], use expert systems combined with semantic infostructures [22,28,47], or database management systems [6]. The course recommendation problem gets more complex when considering a large number of courses that have different weights, values, and priorities addressed by the Course-Petri net, a specialization of Petri net that is used on paper [26] as the foundation for development of an advising system. Student diversity is another aspect that complicates the problem [13,14]. For example, authors in [13] introduce an XML user-based collaborative system called Automatic Academic Advisor, which advises a student to take courses that were taken successfully by students with the same interests and academic performance. In another case study, K-means algorithm has been used to determine the similarity of the students [14].

Another orientation is anticipating students’ future course selection as a means of short/mid-term demand forecasting [12,31]. Paper [1] proposes a two-stage user-based collaborative filtering process using an artificial immune system for the prediction of student grades, along with a filter for professor ratings in the course recommendation for college students.

6.3. Long-Term Academic Planning

Long-term academic planning is a challenging and time-consuming task involved in AAS helping a student prepare a complete study plan towards graduation. More importantly, because the number and variety (in backgrounds, in knowledge, in goals) of students is expanding rapidly, it is more and more important to tailor course sequences to students, since the same learning pathway is unlikely to best serve all students [16,17,23,25]. For example, the system described in [15] implicitly implements, via the decision tree, many academic rules and allows a systematic and exhaustive browse of the different student plan instances, and it permits a methodological assessment and measurement of the appropriateness of a given student academic plan. Work in [18] follows mathematical optimization to generate a set of optimal or near optimal alternative plans that are of a similar quality and yet structurally different. Human intelligence is then employed to analyze these alternatives and to either approve one of them or refine the problem settings for generating further solutions.

To reduce the time-to-graduation, it is therefore of paramount importance for the student to elect courses in a foresighted way by taking into account the possible subsequent course sequences (including which courses are mandatory and which ones are not, and the course prerequisites) and when the various courses are offered. The problem of planning the elective courses is modelled in [20] using integer programming and three different solution approaches are suggested, including a Branch-and-Price framework using partial Dantzig–Wolfe decomposition.

RQ1.1: What are the most significant results from past AAS research that constitute empirical evidence with regard to their impact on the learning process?

According to the research objectives explored by the authors, Table 4a displays a categorization of the algorithmic-oriented findings from the collected studies. Table 4b displays a categorization of the pedagogy-oriented findings.

Table 4. (a) Cassification of the results of AAS case studies (algorithmic). (b) Classification of the results of AAS case studies (pedagogical).

(a)
Objectives	Results
Content-based	Cosine Similarity function can be used to effectively measure the similarity between the new student case and the other old cases by checking the nearest features and values [7]
	Ontology-driven Software Development is appropriate for the development of intelligent academic advising system with an ontology-based architecture and can leverage familiar and proven software engineering tools and techniques [50]
	A case-based reasoning combined with rule-based reasoning AAS is able to acquire new knowledge as usage of the system increases, while its rules can also be modified with minimal effort. This is unlike when an ontology is used for knowledge representation, which, although quite effective, require an advance investment in quality ontology development before efficient course advising can be obtained [6]
Collaborative filtering-based	The Authors in [1] use a demographic property of the student population and their department to segregate the data and to address the sparsity problem that limits the usefulness of collaborative filtering
	AAS can predict a student’s academic performance and interest for a course based on a collection of profiles of students who have similar interests and a similar academic performance in prior courses. In paper [13], students in a Computer Science major are categorized in biclusters based on their similarity on course features: comprehension skills, memorization skills, programming skills, math skills, inferential thinking skills, problem solving skills, application of strategies skills
	K-means algorithm can be used to determine the similarity of the students. It was identified that when K = 7, the clusters generated from the K means algorithm are more informative and effective [14]
	Using an existing and relatively large dataset can be helpful in testing out the collaborative filtering signals, as well as in choosing the appropriate correlation algorithm [5]
	Using friend relationships to weight the correlation used in the collaborative filtering process did not turn out to improve the accuracy of the recommendations [5]
Knowledge-based	Contrary to other contributions, which were intensively based on rule-based approaches, paper [15] proposes a Decision Support System that formulates the student planning and advising tasks as a search problem
	In paper [20], the problem of planning the elective courses is modelled using integer programming and three different solution approaches are suggested, including a Branch-and-Price framework using partial Dantzig–Wolfe decomposition. The suggested algorithms achieved better results than the currently applied meta-heuristic
	By implementing a prescriptive advising model and a developmental advising model, system testing revealed that 93% of academic advising test cases show an agreement between the system advising in course selection and human advising [28]
	Authors in [29] collected all the required criterions, abilities, and capabilities for each faculty/major and converted the knowledge into facts and rules in CLIPS syntax, which is suitable for forward reasoning and can be used easily
Hybrid	Authors in [37] concluded that Ant Colony Optimization-based method is promising, since it enables students to overcome of the disadvantages of classical approaches and gives higher values of performance measures.
	Authors in [42] conclude that the proposed nearest neighbour-like algorithms outperform the two well-known classification algorithms Naïve Bayes and J48 algorithm in terms of student classification success rate when there is uncertainty present in the data
	Classification algorithms can be used to identify what programs a student may fit into; however, to involve multiple schools would require undesirable delays in a web application [42]
	Paper [36] acquires implicit and latent interests of students in addition to the explicitly stated ones, with the help of questionnaire responses that also determine the degree of dilemma that the candidate faces and the time taken by him/her to respond to each question
Computational intelligence-based	The major advantage of fuzzy AAS is that knowledge gradually turns into wisdom and can be used as a decision making tool in critical situations that replace the conventional FAQ [45]
	In paper [46] the experiments showed that the hybrid decision tree and neural network approach improved accuracy in classification task
(b)
Objectives	Results
Choosing Programs/Majors/HEI	Authors in [46] claim that scores are an important factor for qualifying students to universities, especially for Mathematics and English courses. In their experiments, they found that when students’ score of Mathematics and English are higher than 80%, almost all of them will be admitted to the universities.
	Many students choose a university faculty/major because it has a good social reputation or their friends have chosen it [29]. In the same paper, using abilities tests, intelligence tests, and their past academic record, authors can measure some student capabilities and abilities and determine which faculty/major is suitable for them
Selecting Courses	Authors in [1] claim that more feedback information from students is needed for effective course selection and introduces a quality-control mechanism based on filtering courses with poor instructor ratings
	AAS can be used to overcome ineffective student academic advising in Distance Education aimed at predicting a student’s academic performance and interest for a course, based on a collection of profiles of students who have similar interests and academic performance on prior courses [13].
	Authors in [5,14] suggested that students’ major is a possible factor that is related to the students’ performance. Experiment claim that male students aged between 24 and 27 in the software major, mostly of European, Maori, and Asian backgrounds, as well as middle aged (around 45) students, are more likely to be high performing students. On the other hand, young male students with network majors are more likely to be low performing students
	Complex cases for AAS have to do with students that have changed from one program to another (many of them more than once) and have failed and dropped courses that are spread among different departments. Cases in which a student has failed multiply in different departments and are more intricate to handle even for the human course adviser [6]
	Authors in [44] observed that the success of students in their previous required courses and the student’s skills are found to be determinants of student’s success in elective courses
	Authors in [2] claim that an AAS framework has important implications on how the curriculum planner should design the curriculum and allocate teaching resources
	AAS approaches for rolling prediction of future course enrolments should aim at minimizing maintenance effort, especially in terms of adaptability to curricula changes and graduation requirements [12]
Scheduling Courses/Academic planning	Authors in [43] propose a multi-criteria fuzzy system model in academic advising area to advise probation students to register an appropriate number of credit hours so as to minimize their risk of losing their enrolment due to an incorrect decision. The factors investigated are: (1) the Cumulative Grade Point Average, (2) the Number of Covered Credit Hours, (3) Number of times that the student is on Probation, and (4) the maximum grade that student has received.
	The path associated with the students’ academic plan can be used to derive a metric that measures the similarity of the students’ course history. This will be quite useful for mining student profiles and analyzing and predicting student performance [15]
	In small institutions, an issue that needs to be considered in an AAS is the number of students in each class to guarantee a minimum number of enrolled students [17]
	Solving the Elective Course Planning Problem optimal can be used to reduce students’ switch to another high school, which is highly undesirable [20]
	During academic planning, an “explanation component” is important to explain to students the rationale behind courses assignments to various semesters, or changes to study plans after modifying any input setting [23]
	More than 90% of respondents rated the online AAS increased their awareness of the curriculum [24]
	Multiple course selections from different faculties and departments enable students to maximize their opportunities in registering courses of their own interest and completing their degree requirements in the best possible way [25]
Long-Term Course Planning	Students with similar background will achieve similar expected Grade Point Average (GPA) by following the same course sequence recommendation policy [2]
	By analyzing the differences between the generated academic plans, students should be able to better understand their choices and thus make appropriate decisions [18]

7. Discussion and Future Research

We examined the frequency of AAS publications and categories derived from Table 2 and Table 3 in Figure 2, Figure 3 and Figure 4. AAS publications have peaked in 2011 and declined in 2013 but now are beginning to rise again. “Knowledged-based” approaches are on the rise, peaking in 2010–2012. “Selecting courses” are also on the upwising, rising in 2015. Approaches for “Choosing Programs/Majors” show a small decline, while the remainder of the categories does not show obvious trends. From the former analysis, it becomes apparent that, recently, the educational research community has started applying sophisticated algorithmic methods on AAS. As seen in Table 4, the landscape of the AAS research combines diverse and often conflicting aspects and results; however, the authors have highlighted three distinct major axes of the AAS empirical research including:

• Pedagogy-oriented issues (e.g., student modeling, prediction of performance, assessment and feedback, reflection and awareness): Several studies focus on pedagogically meaningful analysis of students’ data in order to shed light on the whole picture. Academic advising, as a teaching and learning process, requires a pedagogy that incorporates the preparation, facilitation, documentation, and assessment of advising interactions.
• Learning analytics (e.g., content analysis, discourse analytics, prediction and information visualization): A number of studies combine institutional data, statistical analysis, and predictive modeling to create intelligence upon which learners, instructors, or administrators can change academic behavior.
• Educational Data Mining (e.g., data mining, machine learning, and statistics): Several studies focus on techniques, tools, and research designed to automatically extract meaning from large repositories of data generated by or related to people’s learning activities in educational institutions.

These three axes are not completely autonomous, since significant overlaps may occur. However, this statement could only constitute a limitation that does not reduce the added value of the findings.

RQ1.2: What do these results indicate regarding the added value of this technology?

One of the most important goals of the systematic review was to reveal the added value of the field explored. From the above findings, it follows that analysis of factors that influence academic decisions, student academic profile, and preferences in order to “recommend” the appropriate learning resources has always been a request. In many cases, a typical software solution to AAS includes a rule-based expert system. However, the dynamic nature of program requirements might turn maintenance of the system into a crippling task. Implementing a user-based collaborative AAS is an appropriate choice, although a major problem limiting the usefulness of such a system that should be addressed is the sparsity problem, which refers to a situation in which data are insufficient to identify similarities in students’ interests. AAS research results indicate that hybrid and content-based AAS are also well suited to the academic advising domain, because these approaches allow the system to adapt to the changes.

Researchers set academic advising within limits in which previously it was almost impossible to infer recommendations automatically, due to the high levels of complexity that such a process demands. In today’s advanced learning contexts, the AAS research community determines simple and/or sophisticated approaches for recommending learning resources and explores their capabilities by tracking actual changes and the progress of the learning process. The goal is to identify the most significant factors in order to develop better systems. These systems will allow HEI and students to evaluate and adjust their learning strategies and improve their performance in terms of learning outcomes.

Moreover, the learning analytics dimension and the opportunity of applying educational data mining approaches are also explored with encouraging results. Consequently, the research community could gain insight into the learning mechanisms that previously were a “black box”.

RQ2: Which other emerging research approaches should be explored in the AAS research area?

Complementary, the literature overview has revealed a number of unexplored issues in this rapidly growing domain, including (but not limited to) the following:

• Many modern educational models (for example Accelerated Study in Associate Programs (ASAP) [51] and Guided Pathways (GP) [52]) share an emphasis on acceleration, programs that offer fewer choices and more support, greater transparency of paths to completion for students, and more mandatory and intrusive advising from day one through completion [53]. In their book “Redesigning America’s Community Colleges”, Bailey, Jaggars, and Jenkins described the idea of Guided pathways, a framework around which to structure program maps, meta-majors, e-advising, and early alert systems [54]. AAS are critical to enabling the kind of monitoring and support demanded by these models and must be understood as tools that are part of the broader reform. HEI need to carefully consider and plan how to change advising structures and daily practices so that existing advisors can leverage the potential of emerging AAS research trends to improve student outcomes.
• Technology acceptance is also a well addressed issue in educational research. Regarding AAS acceptance, authors in [55] proposed a model that considers mainly two parameters: usability and efficiency. However, more parameters should be explored in order to create a reliable AAS acceptance model, e.g., effectiveness, maintainability, and portability. Researchers from the AAS domain could also examine respective models that are suitable for the purposes of AAS.
• The review process yielded very few articles related to scholarship recommendation and eligibility checking, exploring life, and career goals. Assisting students in the clarification of their life/career goals means helping students explore and define plans for the realization of these goals and evaluating the progress of their efforts. It would be interesting to take advantage of the plethora of results from AAS research by introducing innovative educational recommender systems in these areas.
• One primary way of assisting student’s career development is by helping them understand their own intrinsic interests and abilities through self-exploration and career exploration [56]. In this context, existing literature highlights a need to examine AAS in a more holistic manner, one that considers the connected nature of student’s interests, skills, and personality type. For example, one of the most frequently used classification systems guiding personality type exploration that can be utilized by AAS researchers is Holland’s [57] theory of vocational personality types and work environments.

8. Conclusions

Previous works on AAS research provided significant insight into the conceptual basis of this rapidly growing domain. However, these studies did not conduct an analysis of actual research results. The current paper presents a systematic review of empirical evidence of AAS research. We searched the literature and gathered representative, mature, and highly-cited articles of real case studies with actual data from AAS domain. The analysis of selected case studies and their results shed light on the approaches followed by the respective research communities and revealed the potential of this emerging field of educational research. Along with the arising opportunities, we discovered a number of gaps that require the researchers’ attention. Table 5 illustrates our findings regarding the strengths, weaknesses, opportunities, and threats (SWOT) of AAS research.

Table 5. Strengths, weaknesses, opportunities, threats (SWOT) of AAS research.

Strengths	Weaknesses
Large volumes of available educational data increased accuracy of experimental results. Use of pre-existing powerful and valid algorithmic methods. Interpretable multiple recommendations to support learners/teachers. More precise user models for guiding adaptation and personalization of systems. Reveal critical moments and approaches of learning.	Lack of holistic AAS approach encompassing all multi-facet factors like student competencies, interests, and personality type. Misinterpretation of results due to human judgment factors, wrong input data, etc. Heterogeneous data sources: not yet a unified data descriptive vocabulary—data representation issues. Mostly quantitative research results. Qualitative methods have not yet provided significant results. Information overload—complex systems. Uncertainty: In many cases, only skilled advisors could trust the recommendations and interpret the results correctly.
Opportunities	Threats
Modeling programs of studies and learning pathways for data standardization, data interoperability, and compatibility among different tools and applications support multiple course selections from different faculties and departments generalized platform development. Intellectual and affective learning opportunities based on sophisticated decisions. Self-reflection/self-awareness in intelligent, autonomous and massive systems. Feed machine readable results from the AAS procedures to other data-driven systems for diving decision making. Acceptance Model: e.g., perceived usefulness, goal expectancy, perceived playfulness, trust, etc.	Over-analysis: the depth of analysis becomes profound and the results lack generality. Possibility of misclassification of outcomes. Trust: contradictory findings during implementations.

Every HEI needs to have effective academic advising mechanisms in order to increase student development, which in turn can benefit enrollment, retention, and graduation rates. Many HEI offer AAS designed to help students and their academic advisors recommend learning resources and review degree requirements and the student’s progress towards the intended degree. However, existing advising support software tools can augment the student-advisor relationship, but cannot and should not replace in-person advising [6,8,16,18,58]. This is the point at which learning science, psychology, pedagogy, and computer science intersect. The issue of understanding the deeper learning processes and recommending them remains in the middle of this cross-path.

This work can be beneficial for several types of readers. For researchers interested in AAS, this paper helps one to build upon existing work, avoiding the proverbial ‘reinvention of the wheel’, helping to understand trends, and guiding efforts towards new directions. For practitioners, this paper helps demonstrate the ways in which AAS approaches can be integrated into existing system development paradigms, offering ideas on how academic advising can be adopted in practice, including pointers to work containing further details. We believe that this active research area will continue contributing with valuable pieces of work towards the development of more efficient and effective advising services to both learners and HEI.