Annotating the Field: Investigating the Affordances of Mixed Reality for Learning Beyond the Classroom

Abstract

While educational excursions are widely acknowledged to enhance student learning through immersive, real-world experiences, there is limited research on how students can best capture and retain knowledge during such activities. Traditional note-taking methods, such as pen and paper or digital devices, may be inadequate for recording spatial or multimodal information encountered in these dynamic environments. With the emergence of mixed reality (MR) technologies, there is an opportunity to explore spatial, immersive note-taking that aligns with the dynamic nature of field-based learning. This study compares the effectiveness of mixed reality, pen and paper, and digital note-taking during educational excursions. A total of 50 participants in grades 7 through 12 used the Apple Vision Pro headset for mixed reality notes, mobile phones for digital notes, and clipboards paired with a pen and paper for traditional notes. The information encountered was categorised as physical, textual, or video-based. The effectiveness was evaluated through three measures: content extracted and organised in notes, post-activity quizzes on retention and critical thinking, and participant feedback. For physical information, mixed reality significantly improved the content extraction and retention. For textual information, mixed reality yielded more content, but pen and paper outperformed it in terms of organisation. Statistically, all the note-taking methods were equally effective in the remaining aspects. Although mixed reality shows potential to be integrated into educational excursions, participant feedback highlighted discomfort with the headset, suggesting that mixed reality should complement, not replace, traditional approaches.

1. Introduction

By their very nature, virtual environments and immersive worlds suggest unique affordances for learning, not least of which is the potential for dynamic, embodied, and multisensory learning experiences to complement field studies in the physical world. Such environments and their associated technologies are not new, having been marketed to consumers since at least the mid-2000s—and earlier if panoramic photographs are included.

Virtual environments are, however, enjoying a recent resurgence of interest (see, for example, ref. [1]), not only because of the recent pandemic but also because of the introduction of the mixed reality headset by Apple in February 2024.

While existing research has covered virtual reality and augmented reality within classroom settings, there remains a gap in the current research in terms of understanding the application of mixed reality in contexts outside of the classroom, such as in field-based learning. To address this, this study aimed to conduct preliminary research on how note-taking in mixed reality compared with two traditional methods to understand the potential of mixed reality and its feasibility for note-taking and learning.

While extensive research has explored the effectiveness of traditional and digital note-taking methods [2], the potential of mixed reality for educational purposes remains largely underexplored. Even less is known about how mixed reality could impact note-taking during educational excursions, where learning involves interactive and observational experiences. Understanding the specific impacts of mixed reality on educational excursions could help create better field trips for more effective learning in the future.

The present study aimed to investigate the effectiveness of mixed reality note-taking compared to traditional pen and paper and digital methods during educational excursions.

The contributions of this study are as follows:

• This study extends the limited body of research exploring mixed reality as a tool for enhancing cognitive engagement and note-taking during educational excursions.
• This study provides the first analysis of note-taking using a mixed reality headset in field-like educational scenarios, offering detailed qualitative and quantitative insights into its affordances and limitations as a note-taking tool.
• This study contributes new pedagogical insights into how mixed reality technology influences cognitive processes such as critical thinking and knowledge retention during note-taking in real-world conditions.
• This study extends existing research by uncovering critical usability limitations in terms of mixed reality note-taking.

The remainder of this article is structured as follows. Section 2 reviews the relevant literature. Section 3 describes the experimental methods, including the three note-taking scenarios examined in this study. Section 4 outlines the materials used, while Section 5 presents the results. Section 6 discusses the findings, followed by Section 7, which addresses this study’s limitations. Finally, Section 8 concludes the article.

2. Literature Review

Educational excursions, or field trips, have long been used by educators as an effective method to enhance the student learning experience; they have always been an important part of schooling, as evidenced by the long history of education research. Researchers have described field trips as trips designed by schools in interacting and engaging environments as an education device [3]. According to [4], which referenced [5,6], studies have shown that students are able to remember fieldwork and recall the activities over a much longer period of time after educational excursions than traditional learning opportunities. Educational excursions allow for more effective and immersive learning, and note-taking remains a crucial part of such excursions.

Note-taking is the process of writing down, typing, or crafting graphical representation of information for later reference [7]. During field trips, note-taking promotes active engagement with the material, encouraging students to process information more deeply while storing the information for future reference. There has been much discussion about the different modalities of note-taking; see, for instance [8], which claimed that unlike typing, handwriting generates brain connectivity patterns that promote learning, or [9], which rebutted that these claims were not supported by their results. Previous research has compared different methods of note-taking [1], or the effects of note-taking on academic performance [10] or reading comprehension [11], but there has not been much research into the effectiveness of note-taking within the context of educational excursions.

Advancements in technology afford new modalities of note-taking, shifting from traditional formats such as pen and paper to digital formats; for instance, taking notes on a phone or a laptop.

Mixed reality is a seamless blend of a physical environment and a digital one. It contains the aspects of both virtual reality and augmented reality [12]. Mixed reality is increasingly being utilised in educational settings, enhancing pedagogical practices and learning outcomes, and has been shown to make learning more immersive and interactive [13]. Studies have also shown how mixed reality technology has the potential to enhance academic performance [14], further solidifying its place within the educational landscape.

Recent advancements within mixed reality technology bring about a new format for note-taking—note-taking within mixed reality. Mixed reality as a format introduces new functionality to note-taking, such as the potential to ‘annotate space’ [15]. To elaborate, adding drawings to notes to represent concepts, terms, and relationships has a significant effect on memory and learning [16], and on a field trip, mixed reality headsets could achieve a similar effect. Extending from digital note-taking, the immersive nature of these headsets can be investigated as another feature to note-taking; using the headset, users can append their digital notes directly alongside the objects they are observing in their physical realities, allowing them to maintain their focus on the subject under study.

While mixed reality (MR) has traditionally been applied to simulation-based training and immersive educational environments, its affordances for cognitive engagement also align well with the goals of note-taking. Note-taking is fundamentally a process of encoding, organising, and externalising information for future retrieval. MR environments support these processes by enhancing spatial cognition, fostering embodied interaction with information, and enabling higher levels of engagement with content.

For instance, ref. [17] found that immersive environments promote better learning outcomes via increased presence, which correlates with deeper cognitive processing. Similarly, ref. [18] observed that spatial immersion improves memory encoding when paired with visual annotation tasks. These features imply that MR’s immersive and spatial capabilities could significantly support the processes that make note-taking effective.

MR’s potential to link annotations directly to physical environments or objects supports what [19] called contiguity—the cognitive principle that people learn better when related information is spatially and temporally aligned. When applied to note-taking, this means students may better remember concepts they note down when those notes are visually or spatially anchored in the context—such as tagging a rock formation with geological notes during a field excursion. In addition, studies by [20] and later by [21] reviewing augmented and mixed reality in education showed that these technologies can support constructivist learning approaches—promoting student-generated content, which aligns closely with the goals of autonomous note-taking.

As for the evaluation of the quality of notes, prior studies have applied the following criteria.

First, the quantum/amount of content comprising notes taken in mixed reality, as compared to traditional methods. This could be helpful in the context of field trips, where post-trip activities have been proven to improve the effectiveness of field trips [22], as more content means more materials to utilise for the activities. Second, the organisation of notes is also important—according to [23], which referenced [24], it is an important feature of effective notes. Not only that, according to [25], which referenced [26], the organisation of notes has several benefits in terms of learning. Beyond merely the notes taken, we examine how mixed reality affects factual recall and conceptual understanding compared with other methods, as previous studies on note-taking commonly focused on the latter (for example, ref. [1]). Since mixed reality has been shown to improve the motivation and engagement of participants, this could suggest it has positive effects on factual recall, or on knowledge retention abilities, making it a worthy area of investigation [27]. The present study therefore applies knowledge retention as the third criterion and critical thinking as the fourth.

Table 1. Summary of the literature.

Attributes	Previous Studies	Field-Based Learning	Note-Taking	Mixed Reality
Advantages of field-based learning over classroom-based learning	Dillon et al. (2006) [5], Falk et al. (2011) [6]	V
Methods of note-taking	Mueller and Oppenheimer (2014) [2]		V
Effects of note-taking on academic performance	Nye et al. (1984) [10], Rahmani and Sadeghi (2011) [11]		V
Effect on memory and learning of adding drawings to notes	Wammes et al. (2015) [16]		V
Mixed reality and constructivism	Bacca et al. (2014) [21], Radu (2014) [20]			V
Mixed reality and its effect on learning	Makransky and Mayer (2022) [19], Almufarreh (2023) [14], Harjana et al. (2023) [13]			V
Mixed reality and the annotation of space	Moed (2002) [15]		V	V
Effect on memory of immersion and annotation	Parong and Mayer (2018) [18]		V	V

below presents a summary of the preceding survey of the literature.

Despite the extensive literature on note-taking and the growing body of work on the use of mixed reality in educational contexts, there remains a research gap concerning the intersection of these two domains. While prior studies have explored the cognitive benefits of traditional versus digital note-taking, and others have evaluated the immersive benefits of MR environments in general pedagogical settings, very few have examined how note-taking specifically functions within MR environments, especially during educational excursions. This gap underscores the need for investigation into mixed-reality-enabled note-taking during field trips, which this study aims to address by directly comparing the note quality and learning outcomes across modalities.

3. Methods

This study explored three distinct scenarios commonly encountered during educational excursions. The first involved learners observing real-world phenomena or objects in their environment (physical). The second focused on learners engaging with textual information provided to them (textual), while the third examined situations during which learners process auditory or video content (video). For each scenario, the participants were split into three groups: the first group used Apple Vision Pro headsets, each paired with a Bluetooth keyboard, members of the second group were each given a clipboard, paper, and a pen, and the third group utilised their personal smartphones for note-taking. After this study, a quiz was administered to measure the knowledge retention and critical thinking of the participants. At the end of all three scenarios, the participants completed a feedback form on their opinion of each method of note-taking. Each scenario is described in detail below.

3.1. Participant Profile

Participants were recruited through convenience sampling through the peer network of the authors and were invited to set aside an afternoon for participation in this study. This latter duration would include sufficient time for set-up, briefing on expectations, the actual ten-minute participation, and completing the post-intervention instrument. The procedures are elaborated upon in the sub-sections that follow. Informed consent was obtained in accordance with the prevailing protocols associated with the approved IRB application CRPPIRB-2024-06-KL. In total, 31 students in grades 7 through 12 were recruited. Of these, three participants were excluded from the final data analysis; one because they were unable to complete the full duration of note-taking, and two for misinterpreting what to take notes on. In the course of the introductory briefing, the authors established that none of the participants had prior experience with mixed reality headsets; conversely, all of them owned a personal smartphone.

3.1.1. Scenario 1 (Physical)

For this scenario, participants were invited to the Archives room of the Humanities and Social Studies Education Department of the National Institute of Education, Singapore. This room was selected as the setting to represent the physical observation process in field trips because it contains material regarding Singapore’s history, as well as artifacts from other cultures. All the objects for note-taking were lined up along the sides of the rooms, with tables and chairs in the middle of the room. This room was chosen such that the information present would not be common knowledge, yet it would be simple and easy for all the participants to comprehend.

Participants were instructed to assume the persona of a history student on a field trip, and they were tasked to take down descriptions and notes concerning the objects they saw within the room. This element was intentionally introduced to simulate a more immersive and structured educational excursion environment. While a real field trip to a nearby museum or historical site would have been ideal, logistical constraints made this unfeasible within the research timeline. They were given a ten-minute time limit. After the ten minutes, participants were immediately instructed to attempt a quiz administered as a Google Form on their smartphones with one question (Appendix A). They also rated their own prior knowledge of the topic from 1 to 10, with 1 being no prior knowledge at all. All the participants were instructed not to communicate with each other during both note-taking and the quiz.

The authors jointly graded the participants’ notes and quiz. Details of the rubrics are given in the subsequent Section 4.1 below. The quiz question was graded to give two scores—the KR-score, which measures knowledge retention, and the CT-score, which measures critical thinking. In cases where there was disagreement among the authors regarding how to interpret or score a participant’s response, the discrepancies were discussed among the authorial team to seek consensus and thus to ensure consistency.

3.1.2. Scenario 2 (Textual)

For this scenario, we chose an article from The Reader’s Digest (May 2024) on the topic of the harmful effects of sugar. The topic was chosen such that it would not be common knowledge, yet it would be simple enough to ensure all the participants could comprehend it.

This scenario was conducted in a room with tables and chairs to allow participants to sit down and read the article. The seating was staggered to minimise distractions among participants. Each participant had their own copy of the article.

Participants were instructed to assume the persona of a student on a field trip who was tasked with extracting key information from the text. They were given a ten-minute time limit. After the ten minutes, participants were immediately instructed to attempt a quiz administered as a Google Form on their smartphones with three factual questions and two critical-thinking questions (Appendix A). They also rated their own prior knowledge of the topic from 1 to 10, with 1 being no prior knowledge at all. All the participants were instructed not to communicate with each other during both note-taking and the quiz.

3.1.3. Scenario 3 (Video)

For this scenario, participants were shown a ten-minute video of a TED Talk (titled “How to make your cat happier—in three minutes”). The topic was chosen such that it would not be common knowledge, yet it would be simple enough to ensure all the participants could comprehend it.

This study took place in a quiet room furnished only with sufficient seats and no tables. The environment was designed as such because it was assumed that viewing videos on field trips would not be contingent upon the presence of furniture. The video was projected at the front of the room.

Participants were instructed to take down as much key information from the video as they could. After the video ended, participants were immediately instructed to attempt a quiz administered as a Google Form on their smartphones with three factual questions and two critical-thinking questions (Appendix A). They also rated their own prior knowledge of the topic from 1 to 10, with 1 being no prior knowledge at all. All the participants were instructed not to communicate with each other during both note-taking and the quiz.

For scenarios two and three, both the authors jointly graded the participants’ notes and quiz. Details of the rubrics are given in the subsequent Section 4.1 below. For all the factual questions, each idea unit was awarded a single mark. Each question focused on a different part of the material, allowing the idea units to not all be condensed in one specific area of the material. The marks for all three factual questions were thus added together to give the FA-score, which provided a good representation of the total number of idea units over a range of information that each participant had captured. The marks for both conceptual questions were divided by their full mark, respectively, to give the C1-score and C2-score. This was to make it more convenient for comparison of the note-taking performance using the percentage instead of absolute scores. In cases where there was disagreement among the authors regarding how to interpret or score a participant’s response, the discrepancies were discussed among the authorial team to seek consensus and thus to ensure consistency.

4. Materials

The Apple Vision Pro was chosen for the purposes of this study because as of June 2024, it was one of the most advanced mixed reality headsets available on the market. The Apple Vision Pro provides users with three options to take notes. The first being voice-to-text, where it takes down information in real time as one speaks. However, during educational excursions, the user is typically surrounded by many people talking and walking, and the background noise would render voice-to-text unreliable, which is why this option was not considered for the purposes of this study.

The second option is typing on the virtual keyboard. The users either use their eyes to focus on each individual key and tap their fingers together to select it, while others tap each key separately using their fingers on top of the keyboard. The new input method introduces noticeable delays while trying to achieve speed as well as fluidity close to that of typing on a traditional laptop keyboard. These methods are frustratingly slow and quite inefficient for an educational excursion. A more practical alternative, as employed in this study, involves pairing the Apple Vision Pro with an external keyboard, allowing participants to type with the same speed and efficiency as on a laptop. Figure 1 depicts the text input interface in the headset.

Participants utilising the Apple Vision Pro were observed whilst they were taking notes. Some participants were observed to be awkward or clumsy while using the keyboard. There was an incident where a participant accidentally dropped their keyboard onto the floor while walking.

Some participants opted to place their keyboards on a nearby stable surface, like a table, and type whilst observing the nearby objects in the room. There were participants that managed to type remarkably fast and take down a significant amount of information. In the absence of a convenient resting surface, participants had to balance the keyboard on one hand while typing with the other, which often reduced their efficiency.

In this study, all the participants utilising the Apple Vision Pro to take notes did not wear glasses, had minimal experience with using mixed reality headsets, and expressed that they did not suffer from motion sickness. Participants using the Apple Vision Pro were first briefed on how to operate and utilise the headset smoothly. This study proceeded upon confirmation that the participants were comfortable with using the headset to take notes.

4.1. Measures

Four criteria were used to determine the effectiveness of the note-taking. Two were based on the notes themselves, and two were from the quiz the students answered.

4.1.1. Notes

According to [28], which referenced several other papers, there have been many methods used to assess note quality, and the most common measure of quality used appears to be the number of idea units or critical lecture points recorded in the learner’s notes [29,30]. Other methods to assess note quality include the number of details, number of words, clarity, legibility, sequencing, and accuracy of notes [31]. There is also the argument that the learner’s own subjective judgment of quality concerning his or her own notes may be the most important factor to consider regarding quality; a learner may be more suited to understand notes taken in one specific format, which may appear nonsensical to other learners.

This study utilises a combination of identifying the number of idea units and measuring the organisation of said idea units as a metric for assessing the objective quality of the notes. This study then measures the participants’ knowledge retention and critical thinking through a quiz.

Although students are often encouraged to keep their notes concise, research indicates that, in general, the quantity of notes taken is positively correlated with information retention. The greater the volume of notes recorded by students, the more effectively they tend to recall the material later [10], which is why the number of idea units is utilised as an objective metric. Some researchers have argued that the most effective notes are those that can be understood by someone unfamiliar with the content of the notes [32], which is why rating the organisation of said idea units is utilised as an objective metric.

The N-score is a percentage of the total idea units identified by the participant, whereas the organisation-score (O-score) is a rating between 0 and 5. The O-score is subjectively rated, and it is based upon the legibility and organisation of the notes. If notes have illegible handwriting or are filled with typographical errors, a lower O-score would be given, typically ranging from 0 to 2. If notes have relatively fewer errors, and the idea units captured are coherent and understandable, a higher O-score would be given, ranging from 4 to 5.

4.1.2. Quiz

Beyond analysing notes, previous studies also tested the learning of participants after note-taking through factual questions and conceptual questions (for example, ref. [1,33]). This provides an understanding of how note-taking affects the knowledge retention of students, as well as their ability to process and understand information.

This study replicated the approach by testing the knowledge retention and critical thinking of participants. Previous studies used Bloom’s revised taxonomy [34] to define critical thinking [35]. This study referenced this approach for setting and grading questions to ensure the learning objectives of testing knowledge retention and critical thinking were clearly defined. Knowledge retention was tested through factual questions, which focused on the learning objective of remembering—the lowest tier of Bloom’s taxonomy. These questions had a rigid marking scheme to test the amount of knowledge participants remembered, where one mark equated to one knowledge unit. Critical thinking was tested through questions that focused on the other tiers of the revised taxonomy. A bespoke marking scheme was designed for each question based on the applicable tier(s). For example, for an evaluation question, two marks each would be awarded for addressing the thesis and antithesis, and two marks would be awarded for the conclusion.

For Scenario 1 (physical), the quiz had one question. It was graded to give the KR-score, which measures knowledge retention, and the CT-score, which measures critical thinking.

In Scenarios 2 and 3 (textual and video, respectively), the quiz had five questions. Three were factual questions and two were conceptual questions. The marks for all three factual questions were added together to give the FA-score, while the marks for both conceptual questions were divided by their full mark, respectively, to give the C1-score and C2-score.

5. Results

5.1. Scenario 1 (Physical)

5.1.1. Notes

For data with a normal residual, one-way analysis of variance was used. For data with non-normal residuals, the Kruskal–Wallis test was used. We considered a p-value less than 0.05 to be statistically significant for both methods. The normality of residuals of the data was tested with the Shapiro–Wilk test, and the results are shown in

Table 2. The p-value from the Shapiro–Wilk test.

	p-Value (Mixed Reality)	p-Value (Pen and Paper)	p-Value (Phone)
N-score	0.618	0.265	0.263
O-score	0.093	0.006	0.018

below. A p-value above 0.05 was considered normal.

One-way analysis of variance was conducted to compare the effect of the note-taking methods on the N-score. It revealed a statistically significant difference in the N-scores between at least two groups (F(2, 23) = 5.422, p = 0.012). A post hoc Tukey–Kramer test found that the N-score was significantly higher for the mixed reality notetakers than for the pen and paper notetakers (mean difference = 14.708, p = 0.017, 95% C.I. = [2.439, 26.977]), as well as for the mixed reality notetakers compared to the phone notetakers (mean difference = 15.438, p = 0.017, 95% C.I. = [2.186, 28.689]). There was no statistically significant difference in the N-scores between the phone notetakers and the pen and paper notetakers (p = 0.985, 95% C.I. = [−11.929, 10.471]).

The Kruskal–Wallis test was used to analyse the effect of the note-taking method on the O-score. It revealed there was no statistically significant difference in the N-scores amongst the groups (Chi square = 2.339, p = 0.311, df = 2).

The mean and standard deviation of the scores are shown in

Table 3. Mean and standard deviation of the scores.

	μ (N-Score)	S.D. (N-Score)	μ (O-Score)	S.D. (O-Score)
Mixed Reality	31.25	14.491	3.333	1.366
Pen and paper	16.542	8.802	4.083	0.900
Phone	15.813	6.611	4.25	0.886

below, where the mean is abbreviated as μ and the standard deviation is abbreviated as S.D.

5.1.2. Quiz

For the data with a normal residual, one-way analysis of variance was used. For the data with non-normal residuals, the Kruskal–Wallis test was used. We considered a p-value less than 0.05 to be statistically significant for both methods. The normality of residuals of the data was tested with the Shapiro–Wilk test, and the results are shown in

Table 4. The p-value from the Shapiro–Wilk test.

	p-Value (Mixed Reality)	p-Value (Pen and Paper)	p-Value (Phone)
KR-score	0.000	0.068	0.066
CT-score	0.127	0.041	0.071

below. A p-value above 0.05 was considered normal.

In Scenario 1 (physical), the quiz question was graded to give the KR-score, which measures knowledge retention, and the CT-score, which measures critical thinking. The square of the Pearson correlation coefficient ($r^2$) was calculated to reveal no statistically significant correlation between prior knowledge and the quiz scores (refer to the table below for the values). A significant $r^2$ value was considered to be one greater than 0.500. The results are shown in

Table 5. The r² value.

	${\mathit{r}}^2$ (Mixed Reality)	${\mathit{r}}^2$ (Pen and paper)	${\mathit{r}}^2$ (Phone)
KR-score	0.156	0.104	0.068
CT-score	0.043	0.103	0.014

below.

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the KR-score. It revealed a statistically significant difference in the K-scores between at least two groups (Chi square = 6.946, p = 0.031, df = 2). A post hoc Dunn’s test with the p-values adjusted with the Bonferroni method found that the KR-score was significantly higher for the mixed reality notetakers than pen and paper notetakers (z = 2.477, p = 0.040), as well as for the mixed reality notetakers than phone notetakers (z = 2.287, p = 0.067). There was no statistically significant difference in the KR-scores between the pen and paper notetakers and the phone notetakers (z = −0.133, p = 1.000).

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the CT-score. It revealed there was no statistically significant difference in the CT-scores amongst the groups (Chi square = 5.268, p = 0.072, df = 2).

The mean and standard deviation of the scores are shown in

Table 6. Mean and standard deviation of the scores.

	μ (KR-Score)	S.D. (KR-Score)	μ (CT-Score)	S.D. (CT-Score)
Mixed Reality	4.667	0.516	4.167	0.983
Pen and paper	3.333	1.155	2.583	1.443
Phone	3.400	1.174	2.700	1.418

below, where the mean is abbreviated as μ and the standard deviation is abbreviated as S.D.

5.2. Scenario 2 (Textual)

5.2.1. Notes

For data with a normal residual, one-way analysis of variance was used. For data with non-normal residuals, the Kruskal–Wallis test was used. We considered a p-value less than 0.05 to be statistically significant for both methods. The normality of residuals of the data was tested with the Shapiro–Wilk test, and the results are shown in

Table 7. The p-value from the Shapiro–Wilk test.

	p-Value (Mixed Reality)	p-Value (Pen and Paper)	p-Value (Phone)
N-score	0.082	0.058	0.094
O-score	0.001	0.000	0.111

below. A p-value above 0.05 was considered normal.

One-way analysis of variance was conducted to compare the effect of the note-taking methods on the N-score. It revealed a statistically significant difference in the N-scores between at least two groups (F(2, 28) = 5.614, p = 0.009). A post hoc Tukey–Kramer test found that the N-score was significantly higher for the mixed reality notetakers than for the pen and paper notetakers (mean difference = 3.029, p = 0.006, 95% C.I. = [0.791, 5.267]). There was no statistically significant difference in the N-scores between the phone notetakers and the mixed reality notetakers (p = 0.169, 95% C.I. = [−4.138, 0.588]), and between the phone notetakers and the pen and paper notetakers (p = 0.315, 95% C.I. = −0.841, 3.349]).

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the O-score. It revealed a statistically significant difference in the O-scores between at least two groups (Chi square = 6.571, p = 0.037, df = 2). A post hoc Dunn’s test with the p-values adjusted with the Bonferroni method found that the O-score was significantly higher for the pen and paper notetakers than mixed reality notetakers (z = 2.413, p = 0.048). There was no statistically significant difference in the O-scores between the pen and paper notetakers and the phone notetakers (z = 1.773, p = 0.229), or between the mixed reality notetakers and the phone notetakers (z = −0.713, p = 1.000).

The mean and standard deviation of the scores are shown in

Table 8. Mean and standard deviation of the scores.

	μ (N-Score)	S.D. (N-Score)	μ (O-Score)	S.D. (O-Score)
Mixed Reality	6.875	2.100	2.500	0.535
Pen and paper	3.846	1.908	3.308	0.480
Phone	5.100	2.079	2.700	1.059

below, where the mean is abbreviated as μ and the standard deviation is abbreviated as S.D.

5.2.2. Quiz

For data with a normal residual, one-way analysis of variance was used. For data with non-normal residuals, the Kruskal–Wallis test was used. We considered a p-value less than 0.05 to be statistically significant for both methods. The normality of residuals of the data was tested with the Shapiro–Wilk test, and the results are shown in

Table 9. The p-value from the Shapiro–Wilk test.

	p-Value (Mixed Reality)	p-Value (Pen and Paper)	p-Value (Phone)
FA-score	0.401	0.070	0.087
C1-score	0.018	0.850	0.215
C2-score	0.054	0.010	0.086

below. A p-value above 0.05 was considered normal.

The marks for all three factual questions were added together to give the FA-score, while the marks for both conceptual questions were divided by their full mark, respectively, to give the C1-score and C2-score. The square of the Pearson correlation coefficient square was calculated to reveal no statistically significant correlation between prior knowledge and the quiz scores (refer to

Table 10. The r² value.

	${\mathit{r}}^2$ (Mixed Reality)	${\mathit{r}}^2$ (Pen and Paper)	${\mathit{r}}^2$ (Phone)
FA-score	0.229	0	0.084
C1-score	0.019	0.04	0.013
C2-score	0.005	0.019	0.071

below for the values). A significant $r^2$ value was considered to be one greater than 0.500.

One-way analysis of variance was conducted to compare the effect of the note-taking methods on the FA-score. It revealed that there was no statistically significant difference in the FA-scores amongst the groups (F(2, 28) = 1.539, p = 0.232).

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the C1-score. It revealed there was no statistically significant difference in the C1-scores amongst the groups (Chi square = 1.509, p = 0.470, df = 2).

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the C2-score. It revealed there was no statistically significant difference in the C2-scores amongst the groups (Chi square = 5.067, p = 0.079, df = 2).

The mean and standard deviation of the scores are shown in

Table 11. Mean and standard deviation of the scores.

	μ (FA-Score)	S.D. (FA-Score)	μ (C1-Score)	S.D. (C1-Score)	μ (C2-Score)	S.D. (C2-Score)
Mixed Reality	5.625	2.560	0.542	0.148	0.438	0.222
Pen and paper	3.900	2.234	0.500	0.283	0.175	0.230
Phone	3.692	2.810	0.397	0.293	0.292	0.252

below, where the mean is abbreviated as μ and the standard deviation is abbreviated as S.D.

5.3. Scenario 3 (Video)

5.3.1. Notes

For data with a normal residual, one-way analysis of variance was used. For data with non-normal residuals, the Kruskal–Wallis test was used. We considered a p-value less than 0.05 to be statistically significant for both methods. The normality of residuals of the data was tested with the Shapiro–Wilk test, and the results are shown in

Table 12. The p-value from the Shapiro–Wilk test.

	p-Value (Mixed Reality)	p-Value (Pen and Paper)	p-Value (Phone)
N-score	0.402	0.152	0.465
O-score	0.792	0.097	0.014

below. A p-value above 0.05 was considered normal.

One-way analysis of variance was conducted to compare the effect of the note-taking methods on the N-score. It revealed there was no statistically significant difference in the N-scores amongst the groups (F(2, 31) = 1.277, p = 0.293).

The Kruskal–Wallis test was used to analyse the effect of the note-taking method on the O-score. It revealed there was no statistically significant difference in the O-scores between the groups (Chi square = 0.025, p = 0.988, df = 2).

The mean and standard deviation of the scores are shown in

Table 13. Mean and standard deviation of the scores.

	μ (N-Score)	S.D. (N-Score)	μ (O-Score)	S.D. (O-Score)
Mixed Reality	7.750	2.435	2.875	1.246
Pen and paper	6.727	2.328	2.727	1.104
Phone	6.400	1.242	2.800	1.014

below, where the mean is abbreviated as μ and the standard deviation is abbreviated as S.D.

5.3.2. Quiz

For data with a normal residual, one-way analysis of variance was used. For data with non-normal residuals, the Kruskal–Wallis test was used. We considered a p-value less than 0.05 to be statistically significant for both methods. The normality of residuals of the data was tested with the Shapiro–Wilk test, and the results are shown in

Table 14. The p-value from the Shapiro–Wilk test.

	p-Value (Mixed Reality)	p-Value (Pen and Paper)	p-Value (Phone)
FA-score	0.425	0.012	0.018
C1-score	0.120	0.000	0.005
C2-score	0.004	0.257	0.230

below. A p-value above 0.05 was considered normal.

The marks for all three factual questions were added together to give the FA-score, while the marks for both conceptual questions were divided by their full mark, respectively, to give the C1-score and C2-score. The square of the Pearson correlation coefficient was calculated to reveal no statistically significant correlation between prior knowledge and the quiz scores (refer to table below for the values). A significant $r^2$ value was considered to be one greater than 0.500. The results are shown in

Table 15. The r² value.

	${\mathit{r}}^2$ (Mixed Reality)	${\mathit{r}}^2$ (Pen and Paper)	${\mathit{r}}^2$
FA-score	0.121	0.063	0.092
C1-score	0.501	0.046	0.058
C2-score	0.032	0.035	0.031

below.

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the FA-score. It revealed there was no statistically significant difference in the FA-scores amongst the groups (Chi square = 0.632, p = 0.729, df = 2).

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the C1-score. It revealed there was no statistically significant difference in the C1-scores amongst the groups (Chi square = 2.081, p = 0.353, df = 2).

The Kruskal–Wallis test was used to analyse the effect of the note-taking methods on the C2-score. It revealed there was no statistically significant difference in the C2-scores amongst the groups (Chi square = 0.537, p = 0.765, df = 2).

The mean and standard deviation of the scores are shown in

Table 16. Mean and standard deviation of the scores.

	μ (FA-Score)	S.D. (FA-Score)	μ (C1-Score)	S.D. (C1-Score)	μ (C2-Score)	S.D. (C2-Score)
Mixed Reality	3.875	1.727	0.333	0.356	0.268	0.178
Pen and paper	3.333	1.658	0.148	0.294	0.317	0.186
Phone	3.571	1.785	0.238	0.242	0.306	0.185

below, where the mean is abbreviated as μ and the standard deviation is abbreviated as S.D.

5.4. Feedback from Participants

Finally, participants were surveyed about their experiences with note-taking. Some 62% of the participants reported that the phone was easier to use than pen and paper; 70% of participants reported that using the phone was more efficient for note-taking than using pen and paper; and 65% of participants reported that the phone was superior to the pen and paper in terms of its ability to capture the required information. However, 58% of participants thought that using pen and paper was more helpful for their thinking process. When asked which note-taking method is ultimately better, 53% of participants chose pen and paper, whilst 47% of participants chose the phone.

6. Discussion

As described in the preceding sections, there were in total three different scenarios (physical, textual, and video) investigated in this study through three different note-taking methods (mixed reality, digital, and pen and paper). Four different criteria were used to understand the most appropriate note-taking methods. The first criterion was the number of idea units captured in the notes. The second criterion was the organisation of the notes. The third criterion was how well the note-taking method facilitated knowledge retention. The fourth criterion was how well the note-taking method facilitated critical thinking. The third and fourth criteria were measured by the quiz. Each scenario will now be discussed in turn.

For Scenario 1 (physical), the data collected suggest that all the note-taking methods were equally effective in terms of the organisation of the notes as well as facilitating critical thinking. However, mixed reality note-taking was the most effective note-taking method in terms of capturing more idea units, as well as facilitating knowledge retention. Since mixed reality note-taking was the most effective note-taking method in terms of capturing more idea units, this implies that mixed reality note-taking might be a viable method if teachers are aiming for students to add more content to field notes. Previous studies have pointed out that post-trip activities, such as discussions or specific lessons related to the content [22], are important factors to ensure positive student outcomes during field trips. Hence, the usage of mixed reality note-taking could then further enhance these post-trip activities by facilitating the addition of more content to field notes, so educators and students could have more comprehensive post-trip reviews. Similarly, since mixed reality note-taking is the most effective note-taking method in terms of facilitating knowledge retention, this implies that mixed reality note-taking might be a viable method if teachers hope for students to retain more knowledge. Previous studies have suggested that immersive technologies improve the motivation and engagement of participants [27]. Likewise, in this study, when choosing which participants to use for each study, many participants actively volunteered to use the mixed reality headset. A learner’s interests and desires are the factors that lead them to exhibit flexibility, problem solving, and more efficient knowledge acquisition [36]. The participants’ interest in immersive technologies would have led them to exhibit such traits during the note-taking process, encouraging deeper interaction with the material, which may explain why mixed reality note-taking was found to be more effective for knowledge retention.

For Scenario 2 (textual), the data collected suggest that all the note-taking methods were equally effective in facilitating knowledge retention, as well as in facilitating critical thinking. However, mixed reality note-taking was the most effective note-taking method in terms of capturing more idea units, while pen and paper note-taking was the most effective method in terms of the organisation of the notes. Since mixed reality note-taking was the most effective in capturing more idea units, this suggests similar conclusions be drawn as in Scenario 1, in which mixed reality note-taking might be a viable method if teachers are aiming for students to take note of more content, especially for use in post-trip activities. Since pen and paper note-taking was the most effective in terms of the organisation of the notes, this implies that pen and paper note-taking is a viable method if teachers hope for students to take more organised notes. Effective note-taking frameworks, such as the Cornell system, help students process information [37] and have been linked to improved test scores and learning outcomes [26]. Educators should encourage students to use pen and paper for more organised notes. Finally, for Scenario 3 (video), the data collected suggest that all the note-taking methods were equally effective in capturing idea units, organisation of notes, facilitating knowledge retention, as well as facilitating critical thinking.

Overall, this study suggests that mixed reality headsets have potential to be integrated into field trips for the purposes of capturing more information in physical and textual scenarios and encouraging knowledge retention in video scenarios. This supports our initial hypothesis that mixed reality headsets provide value for learners on field trips.

We acknowledge that taking notes using the Apple Vision Pro in a physical setting is challenging, as it requires the user to balance many activities. Feedback from participants included the weight of the headsets and eye strain due to the blurred projection of the physical environment, which might potentially cause discomfort or nausea when used for longer periods of time. The virtual keyboard and eye setups were described as “janky” and “inconsistent”, which made typing and selecting options challenging. In addition, participants reported issues concerning the responsiveness of the device and how it was “discombobulating” to walk around with the headset on for long periods of time.

Positive feedback included the participants identifying a number of affordances: namely, the efficiency of the operating system of the headset, its integration with reality, and its portability. By “efficiency”, participants appreciated being able to work in several windows at the same time, placing screens anywhere in their view, making it easier to multitask. Participants brought up helpful features on the headset, like autocorrect, the large screen size, and the ability to draw or annotate without physical interaction.

Participants also offered feedback regarding digital and pen and paper note-taking. When comparing the differences between the digital and pen and paper note-taking formats, it seems that the digital formats were found by participants to be more easy and efficient to use; participants specifically expressed a preference for the digital affordances of functionality and intuitiveness. There have been dedicated notes apps built into most phones, so participants are familiar with using them. Phones can also take photos and record videos, while pen and paper do not have such functionality. However, pen and paper note-taking also has benefits, such as bringing about new dimensions of thinking and drafting. Participants can more accurately draw or write down information in whichever way suits them best, like drawing arrows to link phrases together or grouping their words in a certain way, which could be why participants thought pen and paper was more helpful for their thinking process [16].

Future updates and developments to mixed reality headsets could enhance the note-taking experience and make such headsets easier to use, but as for now, the current setup is inferior to other traditional note-taking formats. Hence, mixed reality has the potential to be integrated into field trips but should not entirely replace traditional methods of note-taking.

7. Limitations

We acknowledge the limited generalisability of this study as constrained by the size of our sample.

We also acknowledge that the note-taking feature on the Apple Vision Pro is less mature than the traditional pen and paper or digital note-taking methods. The Apple Vision Pro is fairly new, being released in February 2024, and mixed reality headsets as a whole are not widely used in society. Being unaccustomed to the concept of mixed reality potentially affected participants’ efficiency and comfort with using the mixed reality devices to take notes. While participants have been taught how to take notes on the Apple Vision Pro, it introduces an additional learning curve that may have influenced their performance.

Third, while our research methods were built upon pre-existing research, judging the quality of a note itself is still inherently subjective. The methods we used to grade the notes and quiz results, though carefully designed, may not perfectly capture all the dimensions of note quality.

Fourth, although the selected topics were designed to be niche, there remains a possibility that some participants possessed prior knowledge of the content, which could have influenced their quiz performance. To address this, participants were screened for familiarity with the topics, and a Pearson correlation test was conducted to check if prior knowledge had an effect on their quiz performance.

A limitation specific to Scenario 1 (physical) was that participants remained in the Archives room while completing the quiz. Even though the quiz data were still analysed, we acknowledge that the students were able to glance around the room to find necessary information to complete the quiz, and hence, the reliability of our data may be reduced.

8. Conclusions

As mixed reality technology evolves, these limitations are very likely to be addressed in future generations of the devices, thus making them more intuitive and accessible for education. Continued innovation may turn mixed reality into one of the important tools for modern education, and once perfected, it has the potential to transform the student learning experience and even boost academic performance.

Mixed reality technologies like the Apple Vision Pro have the potential to be integrated into field trips, especially for physical observation and textual information segments, to increase the quality of notes and knowledge retention. However, to fully reap these benefits, the device has to be improved to be more comfortable, as many of our participants expressed increasing discomfort as this study went on. Traditional note-taking methods should also still be utilised, especially in scenarios where they are equally or more effective than the Apple Vision Pro but are more comfortable and less costly.

There is much potential for further research on the potential of mixed reality for note-taking, such as for how collaborative learning can be enhanced. Our study focused on individual note-taking without outside influence. However, mixed reality allows collaboration and discussion in the virtual world, which could prove significantly beneficial for learning and student–teacher interactions during field trips.