Strengthening Science Understanding with Learning Trails

Strengthening Science Understanding with Learning Trails Wolfgang Leister1 , Ingvar Tjøstheim1, Göran Joryd2,3, Jan Alfred Andersson4, and Håvard Heggelund4 1 Norsk Regnesentral, Postboks 114 Blindern, 0314 Oslo, Norway; wolfgang.leister@nr.no (W.L.); ingvar.tjostheim@nr.no (I.T.); 2 Expology AS, Sagveien 23F, 0459 Oslo, Norway; goran@expology.no (G.J.) 3 Museum of Cultural History, Postboks 6762 St. Olavs plass, 0130 Oslo, Norway; goran.joryd@khm.uio.no (G.J) 4 Norwegian Museum of Science and Technology, Kjelsåsveien 143, 0491 Oslo, Norway; Jan.Andersson@tekniskmuseum.no (J.A.A.); Haavard.Heggelund@tekniskmuseum.no (H.H.). * Correspondence: wolfgang.leister@nr.no (W.L.) Version May 15, 2019 submitted to Preprints

encoding activities belong to the visitor-centric view of assessment. In contrast, we want to focus on 80 the installation-centric view that is discussed by Leister et al. [2,p.51]).

82
Behavioural engagement is one of the factors that has a positive correlation to achievement-related 83 outcomes [cf. 10, p. 70ff]. In informal learning arenas, this implies that engaging exhibits and 84 installations will foster better learning outcomes than exhibits that do not engage. 85 To evaluate how engaging installations are, the Engagement Profile [2] has been used alongside 86 the visitor, and c) personalise content delivered to visitors. They also explored the impact of physical 115 proximity and visitor gaze on exhibit engagement. 116 Yoshimura et al. [17] presented a study where they use Bluetooth proximity data of visitors' smart 117 phones to measure the visitors' transition between places in a museum. Moussouri and Roussos 118 [18] discussed cultural itineraries of visitors and present a study where outdoor tracking devices 119 are used to extract the paths of visitors in the London zoo. Further, Moussouri and Roussos [19] 120 proposed a methodology for representing location-based data collected by the use of smart-phones.

121
They presented three ways: a) trail-based representation; b) functional representation; and c) statistical 122 distributions of displacement.

123
The prediction of visitor's sentiments and future behaviour can be based on current observations. 124 Parsons et al. [15] suggested to use viewing times as an indicator of preference, and they propose a 125 recommendation system based on this idea. Bohnert and Zukerman [20] used viewing times as an 126 indicator for interest. They proposed non-intrusive personalisation of the museum experience based 127 on viewing times of previous visitor behaviour and evaluated two prediction approaches.

128
Besides outlining exhibit design approaches and strategies, Bitgood [21] presented three types of   for the graphical short form). We create the indicator by evaluating which term in the Engagement

157
Profile fits best to the description of the respective media form.

158
This results in the following findings: The variables C, I, U, and E have an impact on on the 159 narrative, interactive, and adaptive media forms. We also observed that the productive media form  Adaptive media forms adapt responses to the student's actions. This is supported by high values of I,

190
NTM has organised learning objectives for subjects that have been discussed in class before the 191 museum visit. Further, the museum also expected that students understand the task better when they, 192 additionally, can listen to content from an audio file.

193
There are indications from earlier observations at NTM that the students will be more quiet when 194 given organised tasks, instead of letting them explore the exhibits on their own. As in many science 195 centres, noise from school classes in the exhibition area can be annoying. Therefore, the learning trails 196 have been designed so that the single tasks are performed at different locations in the museum.   proximity level is classified into five zones A-E using the following thresholds: A: < 1m; B: < 2.5m; C: 226 < 5m; D: < 7.5m; and E: above. Depending on characteristics of the installation, we assume that a 227 participant is close by when being in Zone C, but for some exhibits Zone B is more appropriate. This 228 can be configured per exhibit.

229
While the student groups perform the learning trails, the students' devices check the beacon 230 proximity about once every second. Notice that too high sampling rates could drain the device for 231 battery power.  Table 2 gives a short 237 overview of these experiments, and illustrative photos taken during class visits are shown in Figure 3.

238
For the learning trail forces, the pirouette is an installation that can be used to explore rotation the possibility for the student to alter this. Thus, we set narrative and user control to N=2 and U=1.

265
The remaining values are set according to which phrase of the Engagement Profile definitions (see 266 Figure 1) fits best for each dimension.  do not contribute to the respective media form (cf. Figure 2).

281
Note that not all values of the Engagement Profile are equally important, and considerations on 282 the impact on each value need to be made. For instance, the impact of E is considered to be weak in 283 the case of the interactive media form, as time constraints apply for school classes (i.e., the duration of 284 the visit is limited).

285
From Table 3 we conclude that the narrative and the communicative media forms apply for the 286 learning trails in their generic formation. However, when a learning trail is performed by a single 287 student, the communicative media form does obviously not apply. As a consequence, the concept 288 of the learning trails supports predominantly the activities of attending, apprehending, discussing, 289 and debating. Note that the concept of the learning trails does not focus on debating as an activity.

290
Elements of investigaging, exploring, and experimenting are present, but not predominantly. The 291 activities of practising, articulating, and expressing are least present, and we recognise that the learning 292 trails are not developed for these activities.

294
We wanted to explore whether we can observe differences for the operation modes I and II, as 295 well as other characteristics of the learning trails. We studied this by collecting data from school classes 296 performing the learning trail and analysed these data by aligning them with observations. 297 6.1. Test setup.

298
Each of the three learning trails consisted of three experiments, here denoted as A t , B t , and C t 299 for learning trail t. After each performed experiment, the participants answered a micro survey M 300 with four questions; after the last micro-survey there was one further question denoted as survey S.

301
See Table 4 for the survey questions. Finally, all participants answered a Kahoot quiz K where the 302 correctness of the students' answers were evaluated. Thus, each group undergoes one of the sequences The answers given in the micro-surveys and 304 the positioning data were stored in the respective tablet PCs and analysed later. 305 We implemented this entire procedure for both modes, that is Mode I for the paper-based version 306 and Mode II where interactive content is pushed to the students' devices when approaching the 307 respective experiment. In our study, the participants were divided into groups of three or four; one 308 participant was pointed out as the leader of the group. All group leaders received a tablet PC that was  M 1it F How much did you like Experiment (i, t)? -scale: 1-7 (not at all -very much) M 2it R I recommend Experiment (i, t) to others who visit the science centre. -scale: [1][2][3][4][5][6][7] (totally disagree -totally agree) M 3it A When I'll visit the science centre next time, I'll use Experiment (i, t). -scale: [1][2][3][4][5][6][7] (totally disagree -totally agree) M 4it K How much did you know from your school classes about the subject of Experiment (i, t). -scale: 1-5 (1=nothing, 2=a little, but don't remember much, 3=something, 4=quite a lot, 5=very much) L Which of the experiments A t , B t , C t did you like best?
used for answering the micro-surveys, for logging the relative position of the device, and for accessing 310 the content (mode II only). Participants that were not group leaders could answer survey questions on 311 tablet PCs that were placed near the installations they visited. The participants were not aware of the 312 test setup of other groups.

313
As each session included up to nine groups of students with three to four participants each, we 314 made some precautions that groups do not interfere with each other, e.g., use the same installation 315 concurrently.

316
We developed separate apps for each of both modes: App I for the paper-based version

323
The questions of the micro-surveys given in Table 4

328
We had planned a further question whether the participants had enough time to use the 329 installation. However, after preliminary tests, we recognised that the learning trails were absolved 330 much faster than anticipated. Consequently, this question was obsolete and removed from the survey. 331

332
Students from school classes at the lower secondary school in the Oslo area participated in the 333 study. In total, five sessions were done between Autumn 2016 and Spring 2017. In total,113(34,38,41) 334 participants appear in our log files; the numbers in parenthesis denoting participants in the learning 335 trails "Forces", "Sound", and "Light", respectively. The students were divided into groups of two 336 to four to enable discussion and interaction in between them. One of each group was selected as 337 spokesperson, here denoted as the group leader. The number of group leaders was n = 41(14, 14, 13); 338 n = 14(5, 5, 4) for Mode I and n = 26(9, 9, 8) for Mode II. The number of samples was too low to 339 determine whether Mode I or Mode II was more engaging; we did not recognise obvious trends.     As we suspected irregularities in the data set caused by the technical setup of stationary tablet 350 PCs used for the micro surveys, we also extracted the data for the group leaders. The results for group 351 leaders are shown in Figure 7. Still, it is not obvious which of the two modes was more engaging.  The results for the question which of the experiments the group leaders liked best is shown in 360 Table 5. Note that some group leaders failed to register for this question. For "Forces" and "Sound", 361 these numbers are compatible with the results in Figure 7. However, for "Light", there is a discrepancy, 362 as experiment #2 received no likes while it was rated rather high in the scores. As a further observation, 363 the experiment "Pirouette" (see Figure 3a) received the highest number of mentions. We evaluated the number of correct answers to knowledge questions. Each question has four 366 alternatives where one of these is correct. Of the questions, there is always one "odd" alternative; it 367 does not seem that the participants chose these to a large degree. Table 6 shows the percentage of 368 correct answers. We marked questions that are about content that has not been presented during the 369 experiment with an asterisk ( * ).

370
For the knowledge question, it is not significant whether Mode I or Mode II is used, nor whether 371 the participants are group leaders. As a further observation, the pre-visit knowledge is in the average 372 rather low (see the factor K in Figure 8).

374
We went into the study with the expectation that Mode II would be preferred by the participants 375 and, thus, resulting in higher scores. So far, we did not find evidence for this. We recognised that the 376 number of participants in the single parts of the study is too small to show significant preferences.

377
The low impact of the mode to the result might be caused by a rather large impact of the design, There was an expectation that the students will be more quiet with organised tasks, and the 394 learning outcome will increase. During the study, we could observe that the students were more 395 quiet compared to ordinary science centre visits, although we did not perform concrete noise level 396 measurements. We leave this for future work.

397
There are indications that the characteristics of the learning trails may contribute essentially to 398 our result. The content of the learning trail for light received low scores, which can be explained by the 399 content being closer to the curriculum, being built up more theoretically, and having less engaging 400 video content than the other two learning trails. However, note that the learning outcome is not 401 necessarily related to the scores, nor to the Engagement Profile.

402
As a further note, the low scores for the experiment centrifugal governor could be a result of this 403 experiment consisting of looking of an object and solving a simple task. This is also visible in the chart 404 of Figure 5a.c. In contrast, the experiment cup (see the chart in Figure 5a.b) seems to be more engaging, 405 and will evoke more enjoyment, inspiration, creativity, activity, behaviour, and progression.

406
For NTM, the correctness level for the knowledge questions is in the usual range, compared to 407 internal studies. Commonly, the pre-visit knowledge is rather low when the students arrive at the 408 science centre. As the subjects treated by the learning trails are rather theoretical, we expected that 409 only few students were able to answer correctly. For school classes, pre-visit knowledge can often be 410 more relevant than the learning outcome from the experiments.

411
The Kahoot-quiz was performed right after the learning trails had been performed, and the learnt 412 had not yet been internalised by the students. Thus, the Kahoot could act as an engaging repetition 413 that would have helped in the internalising process of the learnt knowledge. A repetition of the Kahoot 414 some weeks after the science centre visit could have given more evidence. The photographs in Figure 3 were taken by some of the authors during the study at NTM.