Student and Instructor Ratings in Geographic Information Systems: A Comparative Analysis

Abstract

Geographic information system (GIS) education empowers engineering students to make informed decisions, integrate comprehensive data, and communicate effectively through maps and visualizations. In GIS education, it is common to employ problem-based learning, which can benefit from the advantages of peer assessment methods. Among the benefits of peer assessment are the enhancement of students’ capacity for analysis and synthesis, improvement in organizational and professional communication skills, and the development of critical judgement. However, a consequence of its application is that there may be variation in students’ final grades, with no consensus in the literature on this matter. This paper explores the extent to which the application of peer ratings among students can modify student grades in the field of GIS education. This was achieved by conducting an experiment in which undergraduate engineering students assessed two problem-based learning activities carried out by their peers in two different basic GIS courses. The ratings obtained after the peer assessment were compared with the grades given by the instructors. The results allowed us to debate whether the teaching benefits of this strategy compensate for the differences between the students’ grades and those given by instructors. Although no clear pattern was found in the mean ratings awarded by the two groups of evaluators, the results show that student engagement in peer assessment was high. This experience has demonstrated that the assessments of the two groups complement each other and allow students to gain a better understanding of their ratings and how to improve their skills.

1. Introduction

Geography, as an academic discipline within the spatial sciences, is devoted to the examination of the reciprocal influences between the environment and human societies. Geography investigates dimensions such as scale, movement, regions, human–environment interaction, location, and place. Essentially, geography functions as a comprehensive domain that addresses the interplay of environmental, social, and economic aspects [1]. For this reason, it becomes a fundamental discipline for any engineer who must solve problems related to territory and people during professional activities. The assimilation of geographical concepts by students is enhanced through the integration of computer tools based on geographic information systems (GISs) [2]. A GIS can be defined as a computer system used for the input, storage, transformation, visualization, mapping, and analysis of spatial and non-spatial data, which must necessarily have coordinates that position them at a location on the Earth [3]. Due to its ability to analyze spatial data and the associated quantitative and qualitative information and layer structure, the study of geographic information is based on the superimposition of layers in order to establish relationships between the information they contain [4,5]. These characteristics make GISs a tool that integrates a multitude of disciplines, and it is essential in territorial planning processes [6], environmental impact assessments [7], transport modeling and its effects [8], urban mobility [9], allocation of uses [10], and landscape [11], among others.

Processing, classifying, and mapping data using GISs, i.e., GIS knowledge and skills, spatial thinking, and problem-solving, are key competencies for technically trained bachelor’s and master’s students [12]. For this reason, engineering-related bachelor’s and master’s curricula incorporate GISs, data processing, and mapping as important training competencies for students. The acquisition of these competencies will enable students to solve a wide range of problems related to land management [6], such as those mentioned above. The use of GISs also motivates students to develop relevant skills as they recognize the contributions of spatial analysis and geographical perspectives in their training [13]. Ref. [14] found that learning GISs helped them improve their spatial thinking, which in turn strongly correlated with their performance in the GIS course. Today, GISs are taught in departments such as geodesy, geography, photogrammetry, ecology, natural resources, forestry, civil engineering, landscaping, and urban design and planning [15].

In summary, the utilization of geographic data and spatial analysis has become indispensable in the domain of engineering pertaining to territory and the environment. Notably, GISs have established themselves in recent years as a foundational tool for engineers in this field. Consequently, GIS education holds paramount significance in engineering and natural sciences as it imparts students with essential skills in spatial analysis, data integration, visualization, resource management, and infrastructure planning. It facilitates informed decision-making, comprehensive data integration, and effective communication through maps and visualizations. In this work, we have designed and implemented a peer assessment method that reinforces the acquisition of skills in communication with maps and geodatabase creation. The students, employing critical analysis of their peers’ work, quantitatively assess whether their peers have acquired such skills. The objective of this experiment is to evaluate the extent to which the application of peer ratings among students can modify student grades in the field of teaching GISs. The results of the peer assessment are compared with the grades given by the instructors, which allows us to discuss whether the teaching benefits of this strategy compensate for the differences in the grades given by the students and those given by the instructors, who have much more experience and knowledge.

The experience was conducted in the realm of teaching Information and Communication Technologies (ICTs) in engineering, specifically in GIS instruction. This domain offers various alternatives for problem-solving, fostering the development of a critical mindset and a deeper comprehension of the activity and its content. In particular, we examined differences in two assessable problem-based activities conducted in two undergraduate programs, considering the students’ grade levels. The subsequent sub-sections of this introduction describe the fundamental aspects of teaching GISs and peer student assessment. Section 2 describes the methods employed in both the implementation of peer review, and the analysis is then outlined, followed by the presentation of the results. Lastly, the main findings are presented, and a discussion is carried out and conclusions are drawn in Section 4.

1.1. Teaching of Geographic Information Systems

The teaching of GISs has a markedly practical nature. The instructor guides the students by explaining the different tools available and their application to specific cases, which must then be resolved by the students. Teaching GISs is complex; instructors and students often struggle with technical obstacles, file management, and complex software operations [16]. This can lead to what [17] calls “buttonology”, i.e., students focusing on learning how to point and click with the mouse to complete certain functions rather than engaging in the intended reasoning. The course in which the subject is taught can be a barrier, as students must have knowledge of other subjects to enable them to use the power of GISs to solve problems. This difficulty can be solved by incorporating different GIS subjects throughout the bachelor’s or master’s degree, starting with basic levels and increasing in difficulty to show more specific applications. The instructor must also ensure that students do not simply copy what the instructor has done but are able to understand the complexity of the spatial relationships between the data and identify the appropriate tools to solve each problem. The main barrier to achieving this is the limited number of hours available for teaching these subjects [18], which require a very high practical use of software. One solution is to provide students with practical cases to solve outside the classroom that have a positive impact on their final grade.

1.2. New Rating Techniques: Student Peer Assessment

To guarantee that students take an active role in the teaching–learning process, the learning environment must be interactive and cooperative. The instructor is responsible for designing a methodology to optimize learning, in accordance with the established objectives. The teaching methodology consists of a set of methods and techniques used by the teaching staff to undertake training actions and transmit knowledge and competencies designed to achieve certain objectives, in which the instructor teaches the student to learn and to learn with a critical spirit throughout life [19,20]. The teaching–learning model requires aligning assessment methods and systems with competencies [21]. The rating or assessment system must therefore be useful both for students to learn and for instructors to improve their teaching [22,23]. Competence-oriented assessment involves four fundamental aspects [23]: (1) it must be a planned process that is coherent with the competencies to be achieved, which are in turn aligned with the professional activity; (2) it must specify the level of achievement or performance of competences that are considered adequate; and (3) it must be coherent with students’ active learning; and (4) it must be formative and continuous.

The assessment of ratings is a key aspect of any learning process [24]. It conditions what and how students learn and is the most useful tool instructors have to influence how students respond to the teaching and learning process [24,25,26]. The assessment process is based on collecting information by different means (written, oral, or observation), analyzing that information and making a judgement on it and then reaching decisions based on that judgement. It is an action that continues throughout the teaching–learning process. Its functions are formative, regulatory, pedagogical, and communicative. Its communicative function is fundamental, because it contributes to the feedback of information between students and instructors, between the students themselves, and between instructors and students, i.e., it facilitates interaction and cooperation.

Some authors consider that traditional assessment methods have become obsolete as they promote passivity and should be replaced by others that encourage dynamic learning [27]. In this context, it is increasingly common for instructors to incorporate new methods for assessing courses, despite relinquishing their dominant role in a competence that was traditionally their exclusive domain [28]. Students’ participation in the assessment method gives them the necessary skills to objectively analyze other types of documents [29].

These new assessment methods include self-assessment, in which students assess their own work; peer assessment, where students assess their peers; and co-assessment, in which both students and instructors score [30]. Peer assessment consists of “a process by which students evaluate their classmates in a reciprocal manner, applying assessment criteria” [31], or can be considered “a specific form of collaborative learning in which learners make an assessment of the learning process or product of all or some students or group of students” [32]. Peer assessment is therefore perfectly adapted to the framework of the European Higher Education Area, where assessment has become another activity within the teaching–learning process and contributes to the development of competencies [33]. Its implementation is very open: it can be anonymous or not, students can be assessors and/or assessed, and it can be carried out quantitatively or qualitatively and with or without feedback [34].

1.3. Current Practice in Peer Rating Assessment

Effective peer assessment requires the use of instruments to guide the students. One of the most common assessment tools is rubrics [35]. Rubrics are scoring guides that describe the specific characteristics of a task at various levels of performance in order to assess its execution [36]. The rubric is similar to a list of specific and essential criteria against which knowledge and/or competencies are assessed [37] in order to establish a gradation of the different criteria or elements that make up the task, competence, or content. To be helpful to students, it is essential for the instructor to offer clear instructions, provide a list of aspects to consider when determining the overall grade, and use a numerical scoring system for assessment [35]. This rating method is particularly suitable for courses with predominantly practical content and which rely on active student participation. Peer assessment has been applied in many disciplines such as computer applications [38,39], developmental psychology, music didactics and the psychology of language and thought [30], physical chemistry [40], environmental education [33], linear algebra [41], and online training [42], to name a few examples. While engaged in this evaluation process, students increase their capacity for analysis and synthesis, their organizational and professional communication skills, and the development of critical judgement [43,44,45], which they will later transfer to their own work [46]. Studies suggest that this enhancement in critical judgement correlates with improved writing skills. Additionally, the more critical students were of their peers, the higher grades they achieved [43]. It can enhance students’ performance [47], engagement with the course [48], and the quality of work they present [45]. Ref. [49] reported that the integration of peer assessment enabled over 60% of students to become more aware and reflective and learn from their mistakes. This method allows students to receive more diverse feedback compared to feedback solely from the instructor. Its application in evaluating work carried out by students in a team also yields benefits. Ref. [50] concluded that the more abrupt the decrease in peer assessments, the more pronounced the rate of change in effort. Ref. [51] showed that including peer assessment in students’ assignments improves their perception of fairness in the rating process, as it is not solely in the hands of instructors. Other studies state that students find peer review useful [52,53] and they hold a consistent standard when giving scores to peers [54].

By receiving peer ratings, students also encountered distributive and procedural justice [55]; following the implementation of peer assessment, complaints from students about fairness in grading decreased, and their overall opinion about the course improved [30]. Instructors also obtain benefits from the application of peer review, in the form of greater student motivation and learning [40] and an increase in the competencies achieved with little cost in time and effort for instructors as the students carry out the task [30]. In addition, no significant differences were found between the ratings assigned by the instructors and those assigned by peers [55]. For all of these reasons, peer rating assessment is a useful and accurate tool that should be used as another learning aid [28], provided that the students feel comfortable and engaged in the learning process [56] and that it is designed and implemented in a thoughtful way to ensure its effectiveness [57].

However, it should be noted that the weaknesses of this assessment method may raise some doubts about its reliability and validity and cause some educators to have little confidence in its use [58]. These weaknesses may include the requirement for prior training, the time commitment by the student, and the choice of explicit, clear, and simple criteria [59]. Previous experience has shown that the application of these assessment methods poses several problems, mainly related to the objectivity of the grades assigned [29]. Some authors highlight students’ possible misgivings about being assessed by their peers due to bias in the rating [41]; the influence of personal relationships on the grades assigned [53], which often causes them to be high [60]; the time at which the evaluation is carried out [35], being lower if the evaluator has already made the presentation of his/her work [61]; the fact that some students consider that their peers do not have the skills required to evaluate them [62]; or that they themselves do not consider themselves to be experts [57]. The authors also argue that in the assessment of teamwork, individuals contribute to the effort to enhance others’ perceptions and, consequently, impact the ratings [63]. This all leads some authors to consider the technique to be inadequate, as the resulting scores differ greatly from those assigned by the instructor, who is kinder to the students [29] or the reverse [64]; however, other studies show the opposite, with no major differences between the two assessment sources [41,65]. Student participation in the assessment process must therefore be supervised [29] and blind in order to reduce pressure and lack of objectivity [34].

In short, peer assessment is a learning-oriented assessment strategy that helps students gain a better understanding of the activity and its content, develop critical thinking and argumentation, be more self-critical, and detect strengths and weaknesses. It may involve awarding marks other than those given by the instructor. Despite this, there are many areas in which this evaluation methodology is not widely applied, such as, to the best of our knowledge, in the field of geography and GISs. As a result, there is a lack of knowledge regarding generalizations across disciplines on how peer assessment works. We postulate that the results, benefits, and weaknesses of this methodology may be mostly context-specific, so experiences in specific disciplines are of major importance for effective deployment and for the seeking of common features in case they exist.

2. Method

In order to address our objective and evaluate the extent to which the application of peer ratings among students can modify student grades in the field of teaching GISs, a two-phase methodology is followed in this study. The first is the implementation of the peer assessment and the second is the comparison of the grades obtained by the students without peer assessment with those obtained with peer assessment. This comparison allows us to discuss if the possible differences in the grades given by the students and those given by the instructors are compensated by the teaching benefits of this strategy reported in the literature. The two phases are described after presenting the course and the activities involved in this work.

2.1. Course Presentation and Activities

The experience was carried out in two introductory GIS courses taught in the forestry engineering (FE) bachelor’s degree (second year) and the environmental technologies engineering (ETE) bachelor’s degree (third year), both taught at the Universidad Politécnica de Madrid and with a teaching load of three credits. They include the fundamentals of GISs and basic vector and raster analysis tools and applications. The learning goals of the course are divided into six groups: (i) understanding GIS core concepts, (ii) technical proficiency, (iii) data acquisition and integration, (iv) problem-solving and analysis, (v) communication, and (vi) critical thinking and evaluation. The students enrolled in these courses do not have experience in GISs whatsoever. The preferred form of assessment is continuous, and the assessable problem-based activities include the following exercises, which account for 20% of the final mark:

Activity 1. Designing a map (A1). The objective of this problem-based activity is for students to effectively communicate results through maps and visualizations. The specific learning goal, within the fifth group, is to present findings and analyses using maps. The students must produce a map of a Spanish province from a series of vector files provided by the instructors. Students work on the symbology of the map, the legend, scales, the grid of coordinates, labels, the composition of the different elements on the page, etc. The instructors provide the students with the initial data, a statement with the objectives of the exercise, and the correction rubric to resolve the exercise.

Activity 2. Editing vector data (A2). The objective of this problem-based activity is to generate a comprehensive geodatabase that integrates environmental and geographic information. The specific learning goals, within the third group, are to produce/acquire diverse spatial data types, evaluate data quality, and perform data cleaning and preparation for analysis. The students must produce two vector layers by editing their entities from an orthophoto. The students are required to create layers representing the land uses of an area of 4 km². The instructors provide the students with the initial orthophoto, a statement with the objectives of the exercise, and the correction rubric to resolve the exercise.

By implementing peer evaluation, both activities also contribute to the sixth learning goals group, specifically to evaluate and critique GIS methodologies and approaches.

2.2. Implementation of Student Peer Evaluation

A blind peer assessment is carried out, in which each student receives the exercises completed by other students without knowing who they are. The student rates the exercises he/she has been given in a quantitative way.

The peer assessment methodology consists of several stages. In the first stage, the instructors created the specific assignment included in the Moodle Workshop 3.11 application [66]. It combines tasks for instructors and students, and the transition from one to the other can be controlled by the teaching staff. This process has five phases: configuration, submission, evaluation, grading of evaluations, and closed. In the configuration phase, instructors provide instructions for the submission, and the grading and rubric used are explained. The system randomly assigns the submissions to be assessed by each student in the submission phase. In the evaluation phase, the instructor grades the students’ exercises submitted, and each student evaluates the work assigned to him/her. Finally, the grades are automatically calculated. It is possible to award two separate grades: (i) for the student’s work, derived from the other students’ evaluations, and (ii) for the coincidence of the evaluations made by the student with the evaluations of the same work by the rest of the students.

At the same time, the case is presented in class and the students are guided in its resolution and explained the evaluation process. They have 14 days to complete the exercise and send it without any identifying data.

Once the exercise has been completed and the deadline has expired, in the next stage, the instructors randomly give each student the exercise completed by three students. All students taking the subject were recruited and participation was voluntary. The students then quantitatively grade the exercise objectively according to the rubric provided by the instructors and submit the rating given to each of the three cases to the instructors. The students have 7 days to complete the assessment.

Finally, the instructors review the ratings, compare them with the ones they themselves have given, and assign the final grade.

2.2.1. Indications for the Evaluation of the Activities: Rubrics

In order for all of the students to assess each other’s work against the same criteria, a rubric is provided describing the assessment process applied by the instructors. The content of the rubric is as follows:

Activity 1 (A1)

From the starting mark, 10, one point will be deducted for each of the following errors, which are acknowledged as a partial failure in achieving the learning goal related to presenting findings and analyses using maps:

Lack of visual balance between the different elements in the map.

• A map element is missing.
• Lack of visibility of a map layer.
• The hierarchy in the legend is not properly established.
• The coordinate grid has intervals not properly established.
• Intervals in a legend item are not properly set.
• The graphical scale is not properly customized.
• Misspelling in the legend, titles, or labels.
• Difficulty in the legibility of a map element.

An incorrect scale will result in a failing grade.

The errors detected should be indicated in a constructive way and briefly described. It is also necessary to highlight the positive points of the map. The student evaluator should focus on the work and make objective judgements. An example is shown in Figure 1, where in the right panel, the following errors are visible (in parenthesis—the corresponding numbers in the rubric): linear elements (roads) are not visually balanced to convey the information (1), the margins of the map are lacking (2), items in the legend are not properly arranged and some item titles are missing, so the hierarchy is lost (e.g., the train line is within the population of cities item) (4), the intervals in the population of cities are the default, not set to meaningful ones (6), the titles and descriptions in the legend are the default ones, so they are in two different languages, without proper labels and with underscores between words in some cases (8), and the coordinate grid labels are in different colors and above the box of the geographic data (9).

Activity 2 (A2)

The starting mark is 10, and points are deducted for each error made in completing the exercise according to the following:

• The land use data are not a single layer, i.e., there are several layers for the different categories: buildings, arboretum, paths, etc. (six points).
• There are gaps and/or overlaps between the land use layer polygons (one error, one point; two to five errors, two points; six to ten errors, three points; eleven to fifteen errors, four points; sixteen to twenty errors, five points; more than twenty errors, six points).
• Poor digitizing accuracy, according to the measured difference in meters when zooming in on the screen, between the digitized feature and the orthophoto image (three to five meters, two points; five to ten meters, four points; more than ten meters; five points).
• Lack of categories or information in the table (identification of ten to eight categories; one point; seven to four categories, two points; fewer than four categories, three points; for every five features without data in the attribute table, one point, up to three).
• The limits of the study area are not properly digitized (one point).
• Lack of connectivity in areas of the road layer (two points).
• More than ten missing features (one point).
• The existence of other complementary layers (e.g., a point layer) (one point added).

The errors detected should be indicated in a constructive way and briefly described. It is also necessary to highlight the positive points of the map.

The student evaluator should focus on the work and make objective judgements. In Figure 2 an example is shown, presenting the following errors in the right panel (in parenthesis, the corresponding numbers in the rubric): four gaps, two points (2); just four categories are identified, two points (4); the limits are not properly set; one point (5); missing features, one point (compared to the left panel) (7). By digitizing several layers, this activity is linked to the learning goal of producing diverse spatial data types. Also, from the list of the rubric, errors 1, 3, 4, 5, and 7 are linked to the learning goal devoted to evaluating data quality, and errors 2 and 6 (and 8, although not an error in itself) are linked to data cleaning and preparation for analyses.

2.2.2. Final Rating

As indicated above, the instructors compare the marks assigned by peers with the marks they themselves awarded and assign the final mark according to the criteria described below. The final mark will have values between 0 and 10. These criteria are known by the students before they assess each other’s work and are intended to involve students in a positive way in the peer assessment process.

The criteria are as follows:

• The rating assigned to each student is the average of the three grades given by the students who assess their work.
• If the instructor’s rating coincides with the student’s rating, that rating will be assigned.
• If the instructor’s rating is higher than the student’s rating, the average of the two ratings is assigned as the grade.
• If the instructor’s rating is lower than the student’s rating, the average of the two ratings is assigned as the grade.
• If there is a significant discrepancy between the two grades, the instructor’s rating is assigned.
• Any student who gives an equal rating to the instructor’s ±1.0 points when grading the other students’ work will obtain one extra point in his/her final grade.

The discrepancy between instructors’ and students’ ratings is used as an indicator of the level of achievement of the learning goal that involves the evaluation and critique of GIS methods by students.

2.3. Statistical Analysis

The analysis phase of the results was carried out using statistical tests to compare means and differences between the instructors’ and students’ ratings. These analyses took into consideration both groups of students, corresponding to two different graduate programs, and the level of performance obtained in the two exercises (A1 and A2). The analyses were based on paired sample data, as each individual exercise was evaluated twice (by instructors and by students), so Student’s t-tests for paired samples were used. In cases where the differences between the instructors’ and students’ ratings were analyzed for each individual exercise, Kruskal–Wallis tests with chi-squared distribution were used instead of the actual rating values themselves. A significance level of 95% was applied in all cases.

3. Results

A total of 116 individual exercises from A1 and 114 from A2 were evaluated (

Table 1. Summary of results.

A1
	Evaluator	n	Mean	s.d.	p-Value
All students	Students	116	7.20	1.60	0.217	No significant difference between means
	Instructors	116	7.36	1.84
All students (rating > 5)	Students	104	7.49	1.33	0.045
	Instructors	104	7.75	1.49
FE Students	Students	44	6.99	1.96	0.001
	Instructors	44	7.56	1.91
ETE Students	Students	72	7.24	1.54	0.860	No significant difference between means
	Instructors	72	7.27	1.80
A2
	Evaluator	n	Mean	s.d.	p-Value
All students	Students	114	5.15	3.07	0.024
	Instructors	114	4.65	2.96
All students (rating > 5)	Students	55	7.05	2.03	0.786	No significant difference between means
	Instructors	55	7.13	1.53
FE Students	Students	45	5.17	3.14	0.860	No significant difference between means
	Instructors	45	4.72	3.38
ETE Students	Students	69	5.14	3.04	0.017
	Instructors	69	4.60	2.68

). 35% were evaluated by women and 65% were evaluated by men. In A1, the average rating given by instructors was 7.36 out of 10, and the average rating given by students was 7.20 out of 10. In contrast, the ratings were lower for A2, with averages of 4.65 and 5.15, respectively (Figure 3 top). The variability in ratings was lower in A1, with standard deviations of 1.83 for instructor ratings and 1.60 for student ratings. In A2, the standard deviations were 2.96 and 3.06 respectively. Similarly, the range of rating values in A1 was greater for the instructor ratings, while in A2 it was greater for student ratings. The p-values from the t-tests, assuming a significance level of 95%, indicated that the mean ratings given both by instructors and students were equal in A1 and different in A2.

Focusing on the ratings of students who passed the exercises (final rating > 5) (Table 1, Figure 3 below), the first notable point is that the pass rate was significantly higher in A1 (89.6%, n = 104) than in A2 (48.2%, n = 55). In the case of A1, the mean ratings given by instructors (7.75) and students (7.49) were not significantly equal, whereas the opposite was observed in A2, with higher values than when all of the students were considered (instructors 7.13; students 7.05).

When comparing the results of the two groups for the two different graduate programs (Table 1, Figure 4), it can be seen that in A1, the mean differences are positive in the FE group and nearly zero in the ETE group, while both are slightly negative in A2 (the instructors rate below the students). In this case, the p-values from the Kruskal–Wallis tests indicate that the medians of the differences are significantly equal in both A1 (FE 0.32, ETE 0.04) and A2 (FE −0.44, ETE −0.55). It can therefore be assumed that the effect on the graduate program is not notable, and the differences between instructor and student ratings do not generally exceed 0.6 points (out of 10).

Although the group does not generally appear to be a differentiating factor, the analyses by exercise and graduate program (Table 1, Figure 5) yielded contradictory results (Student’s t-tests). For the FE group, the instructors’ and students’ means were not significantly equal in A1 (p-value 0.001) but they were in A2 (p-value 0.32). Conversely, for the ETE group, the means were equal in A1 (p-value 0.85) and different in A2 (p-value 0.017). When using data from the entire group (FE + ETE), the variability in ratings was much higher in A2 compared to A1. In cases where equal means can be assumed, the standard deviations were 1.54 for instructors and 1.80 for students in A1 and 3.14 for teachers and 3.38 for students in A2. The heterogeneity in performance at the program level does not therefore appear to be a decisive factor in the similarity or dissimilarity of instructor and student ratings.

4. Discussion and Conclusions

GISs have emerged in the last few decades as a vital tool in engineering focused on territory and environmental applications. GIS education facilitates informed decision-making, data integration, and effective communication through maps. All of the concepts related to the realm of geography are encompassed in the aspects that GISs enhance in geography training [2,67]: geographic inquiry-based learning, visual–spatial comprehension power, the definition of spatial relationships, and the development of spatial thinking skills.

This study introduces a peer assessment method that enhances communication skills in map creation and geodatabase development, with associated benefits in analytical capacity, organizational skills, and critical judgement. [43,44,45]. Given that there is no consensus in the literature on this matter [29,41,55,64,65], our main objective was to assess the extent to which the application of peer assessment can modify students’ grades in the teaching environment of ICT in engineering, specifically in the teaching of GISs. We have focused on the evaluation of the results of two evaluable problem-based activities given to second- and third-year students of Forestry Engineering and Environmental Technologies, comparing the ratings given by the students with those assigned by the instructors. We believe that the evaluation presented in this article is a valuable contribution to the knowledge of the effects of participatory evaluation activities on teaching.

New learning techniques promote dynamic learning as opposed to traditional methods [27], and many instructors have incorporated new methods for this purpose. However, their implementation implies a loss of protagonism for the instructor [28], and it is therefore essential to be aware of both its positive and negative implications. Many authors argue that student participation in the assessment method allows them to acquire the necessary skills to analyze documents objectively [29] and develop critical judgement [30]. We verified that the students demonstrated this ability, as they all correctly interpreted the rubric provided and the entire evaluation process. We also noted that the students found the experience motivating, as participation was at 85%, which reinforces the fact that the implementation of peer assessment increases the overall opinion about the course [48]. However, these methods can cause problems with the reliability of the rating assigned [29]. To avoid or at least reduce this, this study was conducted blind and under supervision, as recommended by some authors [34], and with clear and simple criteria published at the beginning of the process [35].

The results show scores of 116 and 114 submissions for two basic GIS and mapping exercises of differing difficulty. Exercise 1, on creating layouts, is less technically complex than exercise 2 on editing vector entities. The second exercise requires data management skills and the use of drawing tools that can be laborious to use. This could explain why the marks awarded in all cases are lower in exercise 2. The s.d. is higher in the distributions of marks in exercise 2 given by both instructors and students. Given that students make more errors in exercise 2, one would expect their ratings to differ from those of the instructors. However, the test results show that the means of the instructors’ marks are not significantly different from those awarded by the students in all cases, coinciding with [65]. We explored the marks given by the instructors and students in the two exercises, taking into account only passing scores and differentiating the degree courses in which the subject is framed. The instructors’ evaluation differs significantly from the students’ in half of the analyses in exercise 1 and also in half of the analyses in exercise 2 (Table 1) nor did we find any significant results indicating that the marks given by the two groups differed in terms of the course they were studying (FE or ETE).

It was expected that as the students had access to rubrics and correction criteria that they themselves had to apply, they would be able to solve the exercises and obtain a better mark than in previous years when the rubric was available but when they did not have to use it to assess someone else. However, the average marks are similar to the average for previous years, with a maximum difference of ±0.5 points with respect to the average mark from 2011 to 2022. It is true that the experience is limited to one academic year and would need to be extended over more years to obtain more robust conclusions in this regard.

Although no clear pattern was found in the results, the experience was positive overall. As far as we are aware, there are no publications in the literature that use this type of collaborative assessment methodology in GIS learning. Given that the mapping exercises allow for a great diversity of solutions, their correction by means of peer assessment provides students with the chance to observe work other than their own, and also to learn from the assessment process, which is implemented as another training activity within the course. The results show that student involvement in the peer assessment process was high. In fact, 95% of the students who submitted the exercises also carried out the peer assessment. In general, there is little difference between the assessments of instructors and peers (Figure 2), indicating a high degree of involvement by the students, who undertook the assessment activity responsibly and in a critical and constructive spirit. Further evidence of this is that 27% of the students received an extra point for their good performance as markers. The experience has shown that the evaluation of the two groups is complementary and that although we do not believe that peer marking can replace marking by instructors, it does allow students to better understand their marks and how to improve them. We lecturers have noticed that the number of requests to review marks has decreased substantially, which opens up new lines of work.

Some authors consider that one of the benefits of peer review is that it reduces the instructors’ workload [30]. This benefit was not identified, as the instructors are also involved in the review process. It actually involved more work, since it was necessary to create the environment for submitting and correcting the exercises by both peers and instructors and for awarding incentives. Although the workload is greater, there are tools to make it easier, and they are increasingly available to the educational community (virtual classrooms such as Moodle [66]). The literature reports that student learning improves [40,43,53,68] when a student peer assessment is implemented. We assessed if its application can modify students’ grades and our focus was not quantifying the benefits of these assessment techniques in terms of the increase in competencies in the analysis of documents, the development of critical judgement, and involvement in their learning. One possible line of work would be to carry out surveys before and after the activity in order to know the students’ opinions.

This study is intended to address the doubts about the application of peer assessment in the grading of students in the field of GISs. However, it has a number of limitations that could be addressed in future research. On the one hand, it is limited to a specific academic year (2022–2023); a longitudinal analysis of several years would provide more information. On the other hand, as has been said, the statistical results do not directly address the improvement in student learning engagement. It would be desirable to add student interviews or surveys in the future. However, the information we provide is sufficient to confirm that this type of evaluation is beneficial and will therefore be implemented in successive years.

Assessment is a fundamental aspect of the learning process as it conditions how students learn [24]. The new assessment methods favor the students’ incorporation into the process and increase their involvement and motivation. This situation reduces the relevance of the instructor and can lead to conflicts in the grading of students [29]. This work was unable to dispel the doubts about the possible replacement of instructor ratings by peer ratings. Further work is needed to confirm students’ improvement in the management of transversal competencies and encourage their widespread implementation, at least in the sphere of university engineering schools, where their application is limited to isolated experiences in certain subjects.