In the past decades computers in all its versions including mobile devices, smartboards, multi-touch boards and tangible system have become omnipresent in educational settings. Among them tangible interfaces (TUI) are gaining in popularity. Due to their “inherent ability to interplay between the physical and digital domains” they are said to be very intuitive and so have “gathered interest for applications in numerous domains” [1
] (p. 215). One is simulations in scientific domains which enable participants to manipulate objects or parameters leading to dynamic system-generated responses which are “based on a set of underlying rules, models, or computations” [2
] (p. 729). So, participants can perform experiments and (re-)discover the properties of the underlying model (e.g., a law of physics) [3
]. For example, we showed elsewhere [4
] how two students, while engaged in a problem-solving physics activity, discovered in situ that a specific factor influences one of the target values.
This problem-solving activity [4
], which is also investigated here, is an instance of a simulation of a biker’s applied force and work done to the bike in a changeable landscape. Participants are required to repeatedly change the gears, the inclination and coarseness of the ground, as well as the distance of the trip in order to explore and discover how these adjustments impact the force and work. In this process, they are guided by a series of tasks with an increasing difficulty, each requiring the parameters to be set on different values. This and other applications were developed by teachers who participated in the ERASMUS+ project “Re-Engage” [5
] led by the Luxembourg Institute of Science and Technology (LIST). In the context of a commencing interdisciplinary and cross-institutional collaboration (LIST and UL), we saw the opportunity to conduct an ethnomethodological/conversation analytic (EMCA) informed video-based qualitative user study with three small groups of adult students. The above-mentioned bike mechanics scenario appeared to be the most appealing to us to investigate how participants engage in TUI-mediated joint problem-solving activities and to explore the implementability of the developed prototypes in educational contexts. When we first looked at the footage, we found that two groups had efficiently resolved the problem by relying on the tangible user interface. However, when we looked closer at the video data, we noticed that, from a certain moment on, both groups speeded up the resolution of the proposed assignments by relying on a specific tangible object (a red button). They did so, however, in a way that went against the intentions of the designers, since the participants no longer engaged with the targeted physics or bicycle mechanics aspect of the activity. Indeed, the red button widget (see Section 2.3
.) was introduced into the design of the activity to provide students with an opportunity to discuss and to reflect their way of setting the input variables before checking their impact on the output variables (by pressing the button). Our detailed analysis will show, that this did work out reasonably well during a certain phase of the activity but only as long as they had not uncovered all the features of the red button.
Besides teaching us some lessons about designing TUI-mediated learning activities, the moment-by-moment EMCA-based multimodal analysis [6
] of the data also provided us with a valuable opportunity to study in detail participants’ joint discovery work [4
]. This in a twofold and intertwined way: Firstly, since we had left the participants in the dark about the features of the red button widget, they had to and did engage in a situated discovering process of jointly (re-)constructing its embedded functions. Secondly, they integrated their corresponding findings in the problem-solving activity not only to discover some aspects of the underlying bicycle mechanics model but also upon uncovering the last function, to exhaust the possibilities of the underlying mechanisms of the interface in order to finish the remaining assignments in a minimum of time.
In this paper, we rely on the video data of the first group to highlight how the participants are doing
discovering. We do this in order to contribute to a better understanding of how discoveries are produced within a context of digitally mediated joint activities and to gain design knowledge with regard to supporting the triggering of episodes in which participants engage in joint reflections and discoveries. Before we get to the account of our analysis, we shall first describe the development of the corresponding TUI-mediated activity (Section 2
), provide some reflections on discovery learning and discovery work (Section 3
), lay out our study design (Section 4
), present our detailed analysis of six excerpts (Section 5
), and, finally, draw our conclusions with regard to the organization of participants’ discovery work in TUI-mediated problem-solving activities and some related design issues (Section 6
3. Discovery Learning and Discovery Work
The here designed TUI-mediated problem-solving activity incorporates the pedagogical strategy of scientific discovery learning where the main task of the learners “is to infer the characteristics of the model underlying the simulation” basically by “changing values of input variables and observing the resulting changes in values of output variable” [22
] (p. 180). In other words, the science curricular discoverables embedded in the activity correspond to what Atkinson and Delamont [23
] have termed “cold discoveries”, that is “the findings of previous scientific inquiries into settled matters” are “reenacted for pedagogical purposes” [11
] (p. 41). In contrast, a hot discovery would be an inquiry into questions where there is no available answer yet [10
]. The idea of discovery learning as a beneficial pedagogical approach has a long history [24
] and the technological developments of the last decades come with the promise of increasing its efficiency [3
]. However, research has pointed out that, overall, students very often have problems to identify variables or to generate hypotheses [3
]. Due to this and other discrepant findings there is an “ongoing discussion about guided-versus-pure discovery learning” asking for “further studies on what and how to guide and scaffold in all forms of discovery learning” [26
] (p. 1012).
One way that we attempted to guide the discovery process (from a pedagogical point of view) was the introduction of assignments/tasks with an increasing difficulty and self-revealing widgets. Hence, an aspect of our user study (not developed here) was also to investigate to what extend these measures worked and further guidance is needed to provide participants with a challenging but not frustrating learning experience. Contrary to our expectations the red button, a central widget in our design, turned out to be problematic in terms of self-explanation and ambiguous with regard to the accomplishment of the pedagogical task. Indeed, the participants of two groups, in a sense, “hacked” the red button. From a certain moment on (see Section 5.5
.), they kept it pushed while continuously changing the input variables until the corona of the widget turns green. In a sense, they favoured a simple-quantified goal (the corona should turn green as quickly as possible) instead of a more abstract goal (grasping the fundamental laws explaining the simulation). This shows how difficult it is to design an activity where the intended pedagogical aims and ideas directly translate into the learners’ conduct and it calls for a research and design approach that focuses on “what actually happens, that is, on the task-in-process, rather than on what is intended to happen, that is, on the task-as-workplan” [27
] (p. 95).
Discovery learning is generally conceived of as a constructivist approach to education in the sense of Piaget and the early Bruner [28
]. This means that it was derived from studies in cognitive psychology and basically targets learners’ conceptual change which is traditionally being conceived of as being located in the brain [26
]. Now, if discovery learning is a theory of instruction implying that things are learned more deeply when they are learned through discovery (as opposed to being told), then this raises the question of how the discovering actually gets done by the participants as an interactional matter. Since the meaning of a discovery is tied to the particulars of the situation in question, it is best studied within the specifics of that situation. From this perspective, we need to rely on a different methodology from one that involves collecting large samples, aggregating data and applying a statistical treatment leading to a general claim designed to apply to all situations. Instead, we are making a claim about how things got done in some specific occasion by some specific cohort. This is what is achieved by applying an ethnomethodological conversation analytic approach (EMCA) (see Section 4.2
Within this framework discovery work is conceived of as an interactional matter, during which participants display to one another that “something new enters the world held in common” by them and so we can witness “how the matter so treated transitions from a dawning possibility […] to an established thing” [29
]. So, we do not investigate here discovery as a psychological phenomenon referring to a private event taking place in the realm of the mental. Rather we study another part that has to do with “how we display to others that we have to come to view the world in a different way” [29
]. To differentiate this part from the psychological event of discovery and to emphasize that our research focus is on the interactional accomplishment of producing an intersubjectively ratified understanding, we shall rely here on the term “discovery work” [10
To bring about this process (and others) as a coherent and intelligible course of action, participants organize their conduct not solely in talk but through the simultaneous use of multiple semiotic resources [31
]. Thus, we relied on video recordings to document the social interactions among participants and their interactions with the TUI. So, in the following section we shall describe how we constructed and analyzed our video data.
Before analyzing the selected excerpts, we provide a brief overview on how the three participating groups coped with the problem-solving activity, and, from that, we explain why we chose the first group and how we selected the excerpts in question.
5.1. General Observations
The first group (3 participants) took in total 23:43 min to solve all the tasks correctly. Also, the second group (2 participants) managed to correctly solve the tasks but only took 17:05 min. The last group (2 participants) was the quickest one (10:27 min). However, they did not solve any of the tasks correctly. Based on the collected video data, we could observe some similarities among the groups’ approaches but more particularly between the first and the second group. During the “start” phase, all groups had a look at the different widgets and the scenes and coronas on the table. However, despite our expectations, they did not truly make use of that phase to explore the tool and realize all the functions of the widgets: group 1 and 2 quickly understood how to control parameters by rotating the widgets but did not figure out the use of the red button; and group 3 figured out the partial use of the red button but did not understand how to control the widgets (i.e., that they need to be rotated). So, for all the groups, many features were still hidden when they began to work on the first task.
Group 1 managed to figure out how to display results with the red button during task 1 (after 5:38 min) and how to view the green feedback at the end of task 2 (after 7:19 min). With these tools at hand, they then solved the tasks in a rather systematic way, verbalizing steps, forming tentative hypotheses, trying them out, drawing conclusions. After 15:00 min, they figured out that they could hold the button down and speeded up the task resolution by adjusting the widgets and relying on a mere trial and error approach. Group 2 needed 6:25 min to figure out the use of the red button, but then right away understood the three features: seeing the results, receiving feedback on the correctness and the possibility to hold it down. They then decided to go back to the first task and were able to accomplish all the tasks very quickly by holding the button down most of the time and adjusting the parameters progressively. Group 3 had most difficulties to solve the tasks. It seemed that they did not understand what exactly they were supposed to do and how they could use the table to solve the tasks. For each task, they changed some settings without engaging any further with the system reactions and quickly moved on to the next task.
Since the participants of the first group successively uncover the different functions of the red button over a series of events, the analysis of their related joint interactional work offers the richest account of discovering work with regard to one topic (the red button). Furthermore, as the detailed analysis will show, the outcome of the latter is highly relevant for the further course of their actions with regard to the science curricular discovery work targeted by the pedagogical design of the problem-solving activity. In the following, we shall provide a detailed analysis of all the instances where the red button is topicalized by the participants (excerpts 1, 2, 3, 5, 6) and of one exemplary instance of discovery work oriented to an aspect of the bicycle mechanics (excerpt 4). Furthermore, we recount how the participants organize their problem-solving activity after a relevant discovery or identification of a new function of the red button is accomplished.
5.2. The Red Button as a Potential Starting Device
After entering the room, the participants start to visually explore the semiotic space of the TUI. At first, they do this more or less independently from one another and the TUI serves as a somewhat scattered focal point of attention. A first attempt to establish a more localized shared focus is undertaken by Trish when she comments on and successively points to the coronas of the output and gear widgets and the graphical representation of the bike. Both Matthew and Jessica follow her pointing gestures with their gazes. Then, Matthew introduces a specific focus by pointing to the red button object and describing it as looking “like a button” (Figure 4
, line 1–2).
Matthew is the first one who introduces the topic of the red button by pointing to the corresponding object and pointing out its similarity to a button. Upon receiving no verbal feedback from Jessica and only a short “mhm” from Trish (line 3), he elaborates on the uttered resemblance by assuming that the red button is a starting device to get to the first task (lines 6–9). His proposal is neither accepted by Jessica or Trish, nor is it openly and directly rejected. Jessica expresses her scepticism through a lateral head movement (line 11, fg. 1.2) and Trish mitigates her discarding response through a “yes but … construction” and through establishing some kind of resemblance between her and Matthew’s suggestion (“that is also like this,” line 13). More precisely, Trish points to the task wheel widget and refers to its inscription (“task one to six test A B,” line 15). In a sense, she uses the design of this particular tangible object as a clue that the red button may not be a starting device. Note that, in his concluding reply (“this means that,” line 16), Matthew only takes up the information regarding the inscription (“there are six tasks,” line 16). Both the topic of the red button and the idea of it being a starting device—explicitly expressed by Matthew with regard to the red button and vaguely expressed by Trish with regard to the task wheel—remain unresolved and so pending until its resolution in excerpt 2.
In the minute following excerpt 1 (not transcribed here), they address the remaining widgets one by one and attempt to figure out their purpose, however without trying them out. Then, Matthew multimodally restates his previous hypothesis with regard to the red button (excerpt 2, lines 1–2) and explicitly addresses his co-participants through his gaze and his smile.
This time, Trish picks up Matthew’s idea (Figure 5
, “shall we?” line 4). She ratifies his assumption by projecting it as an appropriate next action. Through her formulation (“we?”) she lifts her invitation to take the action (pushing the button) up to the group level. Although Trish addresses Mathew with her gaze and smile, it remains unclear who shall actually press the red button (see the long pause of 1.5 s at the end of line 4 filled with smiles and chuckles). Matthew then announces that he will “do it” after providing an account for his initiative (“if no one wants,” line 7). Through their mutual gazes and their chuckling all the participants collectively display their support but also their uncertainty and curiosity with regard to the reaction of the system. So, Matthew presses the button (line 9) and, subsequently, briefly displays that he interprets the ensuing reaction of the TUI (line 10) as relevant by making an index up gesture (line 11, fg. 2.4). He, however, rapidly withdraws his hand and verbalizes that his hypothesis can no longer be sustained (line 12), which is smilingly echoed by Trish (line 13).
With regard to discovering work, we are witnessing here an initial noticing that also includes a multimodally embodied candidate proposal for a potential purpose of the red button (i.e., a starting device): Matthew is pointing and gazing at the physically present red button (fg. 1.1) while verbally suggesting a practical use that is potentially relevant for their actual conduct (pressing to start). However, his proposal is met with embodied scepticism (fg. 1.2), respectively with a mitigated negative verbal assessment that mobilizes the task wheel object with its symbolic inscriptions.
Since the starting device issue is pending, Matthew renews his previous candidate proposal in a similar multimodally embodied way for a function of the red button at the beginning of excerpt 2. This time, his initiative leads to a testing of his conjecture and a subsequent negative assessment. Notice the embodied nature of Matthew’s assessment: First, he briefly anticipates an affirmation of the tested proposal through his index up gesture. Then, he withdraws his hand and so withdraws his appraisal. Finally, he utters the negative outcome of the assessment procedure.
In a sense, we have a negative uptake here. The idea of the red button as a starting device is refuted and definitively off the table. They have come to know that the red button widget is not
a starting device. Yet, they have introduced now the function of the red button as a recurrent topic. Since this issue is not resolved and remains pending, we shall witness in the subsequent excerpts, how they interactionally elaborate an understanding of the function of the red button through discovery work (excerpt 3, 5, 6). Furthermore, we point out how their related findings are consequential on the organization of their problem-solving activity as an actual engagement in discovery work, or as a simple working off of the assignments (excerpt 4 and narrative accounts in Section 5.4
. and Section 5.5
5.3. The Red Button as a Result-Displaying Device
After reading the instructions of task 1 (“In which situation the biker needs to use the lowest force on pedal?”), participants start sharing their ideas about what impacts the force on pedal, manipulate the widgets and contemplate the changing coronas. The white boxes of the energy and force on pedal widgets catch their attention and they start wondering how they can trigger the display of the value. After having tried several types of manipulations (rotating, moving, tipping with the fingers on objects), Matthew refers to the red button and makes a suggestion in line 1 (Figure 6
Matthew appears to experience a sudden insight and shares it with his co-participants. His turn is prefaced by a change-of-state token (ah) indicating that he “has undergone some kind of change in” his “locally current state of knowledge” [47
] (p. 299). He multimodally produces his proposition by formulating it as a reversed polarity question [48
] (p. 2) and by pointing to the relevant spots (corona of force widget and red button) on the TUI-interface. Both Trish and Jessica verbally display their agreement, which corresponds to the preferred answer to this type of questions [48
]. The former through an overlapping (line 3) and the latter through an immediate affirmative response (line 4). Matthew then recycles the “display part” of his previous turn in talk (line 5) and the corresponding gesture (line 6). Trish moves her hand over the red button (line 7) when Matthew points to the corona of the force widget (line 6). So, in a sense, both participants jointly enact the two reference points of Matthews previous turn (red button and corona of force widget). By instructing Trish to press the red button (line 8), Matthew interprets her preceding hand movement and the halting position of her hand (line 7) as readiness to perform the relevant action and to try out the ratified hypothesis. Trish then promptly pushes the red button and releases it again, which triggers the brief display of the energy and force values in the corresponding coronas (line 10). Thereupon, Matthew points out that his initial proposition is now confirmed: pressing the red button does lead to the display of values in the coronas of the output widgets. To do so, he relies on the change-of-state token “ah” and the closing particle “voilà” [49
] (line 11), where the latter co-occurs with a pointing gesture (line 12).
Thus, in principle, the sequence, initiated in line 1, is now completed. Yet, it is extended through Jessica’s noticing that the values are no longer displayed (line 13). Simultaneously, Trish moves her hand again to the red button and presses it after Jessica has finished her turn. However, this time, she does not release the button immediately but holds it for some time. Consequently, the values remain on display, prompting Matthew to read aloud the value of the force on pedal, so re-affirming the red button as an output displaying tool.
In this excerpt, the three participants jointly achieve a discovery through her interactional work. Matthew produces a new proposal for a function of the red button, that is, a trigger for the display of output values, at least, in the gesturally referred to corona of the force-on-pedal widget. Through his swiping and pointing movements he mobilizes the two objects (force-on-pedal widget and red button) and visually establishes a relationship among them that he verbalizes as conditional. Matthew’s candidate proposal is then put to the test through enactment and assessed positively (lines 8–12). Furthermore, the trial is repeated to reproduce its effect (13–18) after one of them notices the disappearance of the triggered display. They have now discovered a first function of the red button device and, in the following excerpt, we can witness how they take it up to work on the accomplishment of the given scientific problem-solving task.
Following Matthews instruction (lines 1), Trish pushes the red button (line 3). Matthew’s co-occurring pointing gesture to the corona of the force widget (line 2) creates a local focus of attention that all participants chose to share (see fg. 4.1). When the TUI displays the output values (force and energy, line 4), Matthew interprets them as not meeting the demand by suggesting turning each gear to its other extreme (line 6). They then jointly carry out this adjustment: Trish acknowledges Matthew’s suggestion by releasing the red button and instructs him to turn the object of the pedal gear widget (line 9), he is holding already (line 7). Matthew follows her instruction and sets the pedal gear to the smallest one (line 10). While Trish moves her hand again to the red button and so gets ready to check the result (line 13), Jessica seizes the wheel gear (line 12) and seeks approval to set it to a bigger gear (line 11). Matthew ratifies and specifies that she should set the wheel gear to its biggest extension. Right after his affirmation, she starts shifting the wheel gear (line 15). During Jessica’s manipulation, Trish retracts her hand (line 16) and only moves it back after Jessica has finished her adjustment (line 17). Then, she pushes the button (line 17) and the TUI displays the output values. The latter are significantly lower which is noted and commented by all the participants. Trish points out that the values are now “even less” than previously (line 19), which is acknowledged in line 20 by Matthew (“yes”), who further indexically (“this way”) verbalizes their conclusion (“then”) contrasting it (“nonetheless”) to a previous setting (at the beginning of the excerpt the pedal gear is set to its biggest and the wheel gear to its smallest extension). Simultaneously, Jessica underlines the epistemic status of their finding, stating that they “know it” (line 21) and Trish releases the red button (line 23). Matthew acknowledges again and reformulates their finding by pointing out that the current gear setting corresponds to the one requiring the lowest force (line 23). To make sure of this again, he briefly pushes the red button. After Matthew utters “yes,” Jessica also produces an acknowledgement and reaches for the task wheel to proceed to the next task. Yet, she retracts her hand again when Mathew continues reformulating their finding (“so it’s from small to big,” line 27) while pointing to the wheel gear. Then, Jessica again attempts to conclude by rhetorically asking to move on (line 29) and does so without waiting for someone else’s approval (line 31). Notice that although Matthew provides the preferred answer (“yes”), he prefaces it with “eh” and produces it with some delay. Considering this markers and Matthew’s pointing movement to the corona of the energy widget, we may assume that he might have preferred lingering on the question.
Extract 4 (Figure 7
) is interesting in two ways. First, it demonstrates how their acceptance of the proposal that the red button is (only) a result-displaying device is relevant for them for the present purpose and becomes an instrument of dealing competently with their momentary understanding of the task. Second, it shows how—through discovering work
—they deal with and come to grips with an often-misconceived concept, that is, that the force on pedal decreases when selecting a larger sized gear at the wheel but increases with a larger sized gear at the pedal. To do so, they witnessably produce multimodally embodied discovery work. Mathew introduces a candidate proposal for the solution of the task (“In which situation the biker needs to use the lowest force on pedal?”), namely to inverse the adjustments of the gears (line 6). The latter is not only talked into being but also enacted through a spinning hand movement (fg. 4.3). Again, they jointly set the parameters accordingly and get ready for the ensuing assessment. After their trial, they jointly interpret the outcome of their trial—a significant reduction of the force on pedal—as a successful accomplishment of the task and so affirm their candidate.
However, the reader will have noticed that, so far, the participants have not yet solved the task as expected. To do this, they still have to reduce the coarseness of the ground from “sand” to ‘”treet.” So far, they have not realized their "mistake" since they have not figured out yet that the red button also serves to visualize whether the independent variables are set correctly or not with respect to the task. That is what we are going to see in the following excerpt.
5.4. The Red Button as a Checking Tool
After the previous extract, the group works on the second task (“In which situation the biker consumes the most energy?”). They invert the gears and progressively change the inclination and the distance to their respective maxima and the ground to the surface they esteem to be “more exhaustive” (i.e., sand). After all these adjustments, Trish presses the red button and this time a green belt around the button and a green tick flash (Figure 8
, fg. 5.1).
This extract shows us how they discover that there is more to the red button, namely that it is the green marks (tick and green belt) that signal the successful accomplishment of the task. Matthew notices the green tick (lines 2–3) and marks it as a new information (“oh”) and Trish points to the green belt (line 4). In lines 6 and 7, Mathew ratifies (“oh cool”) Trish’s contribution (the green belt) and signals through the use of “ok” that they have completed something and, consequently, Trish suggests moving on (line 9) to the next project which is working on the next task. However, Trish displays that she has gained an insight (line 8). She interrupts Jessica’s proposal to start working on the next assignment by sharing her conclusion that their solution to the previous task was wrong (see extract 4). Matthew responds to this announcement with wonder (line 11). So, Trish reaffirms her insight by providing a multimodally embodied explanation: she draws a tick in the air with her right index (fg. 5.2). Towards the end of her turn, she addresses Jessica with her gaze. This affiliation seeking conduct "pays off" when they jointly respond (lines 15–16) to Matthew’s second wondering (line 14) and simultaneously verbalize their argument (lines 15–16) why the previous task was not solved properly.
They then (passage not transcribed here) return to the first task. After restoring the settings, they had set at the end of their presumed resolution of the first task, they quickly recognize that they also have to adjust the ground to the “hard ground” (i.e., the asphalt). After checking their now correct solution by pushing the red button, they collectively express their pleasure on sight of the flashing green tic (and belt) and move on to the next assignment.
Again, we can see, how they organize their activity as discovering
work. We have an initial noticing (their solution to the previous task was wrong) through Trish, who then embodies a candidate proposal for a new function of the red button (a green tick signals that the solution is correct). The latter is jointly affirmed by Trish and Jessica through their overlapping talk (line 15 & 16) and by Matthew through his acceptance of returning to the previous task. Since the proposal is immediately followed here by an affirmation, this episode may be regarded as a weaker example of a discovery sequence, perhaps better described as an instance of a recognition [11
] (p. 43). Nevertheless, the positive assessment is followed by a highly consequential uptake in their situated activity. They return to the previous task and resolve it by relying on their newly gained knowledge about the red button. Their corresponding competence can also be witnessed in the subsequent narrative account of their resolution of the third task (“How can the biker consume the same energy as in task 2 but minimizing the required force?”).
Indeed, they rely in a systematic way on their so-far gained knowledge concerning the result-displaying and checking features of the red button. While doing so they engage in intensive discovery work. They continuously take the red button widget up as tool to move on with and to finally solve the assigned task. They name the problem, form tentative hypotheses, try out the latter, draw conclusions and, if necessary, restart the process. More precisely, they take up proposals from previous task resolutions (e.g., 08:24:00 TRI: the force on pedal must go down (.) how did we turn that [pedal gear] [...] we had it very big); they utter initial noticings (e.g., 08:59:40 MAT: this means that we have to reduce anything) and proposals for possible solutions (e.g., 09:05:40 MAT: either [we reduce] the inclination and we switch to another [wheel] gear there; 09:05:40 TRI: or here one gear up); they assess and put to the test their hypotheses (e.g., 10:27:32 MAT: [...] we have to [...] experiment somehow if we don’t know it physically); and, finally, they figure out the required settings, leading again to a joint cheering on sight of the green corona of the red button, which marks their adherence to the co-achieved resolution and their, for their present purposes, co-constructed reality.
5.5. The Red Button as a Fast-Tracking Tool
When they work on the fourth task they continue relying on the red button to inquire about the outcome of their trials. However, they considerably increase the pace: Solving the previous third task took them 3 min and 48 s, during which they pressed the red button 21 times to check the results. To solve the fourth task, they press the red button for twenty times during a period of 2 min and 17 s. They consider manipulating a widget and pressing the red button in a chronological order: first they try with one or more widgets and then they verify with the red button. However, this pattern changes in the 15th minute, when they, more or less accidently, find out how the red button can be used to have instant access to the results while manipulating the input widgets.
When Jessica has finished adjusting the inclination of the road, they all look at the centre of the TUI where the red button is located. Through her so-prefacing Jessica then verbally initiates the next action (Figure 9
, line 1), namely pressing the red button to check if their candidate solution for the assigned task is correct. The latter is done by Matthew (line 2); notice that he will now keep holding down the button until the end of this sub-excerpt. The TUI “responds” by displaying both the output values and the grey belt around the red button (line 3), indicating that the task is not solved yet. Jessica notices that the energy is below the target. She proposes to increase the distance again (line 4) (they had decreased it in previous attempts) and, at the same time, sets it to its maximum (line 5). Matthew simultaneously makes the same suggestions, so both align with one another. After Jessica has set the distance to its maximum (30 km), Matthew does something peculiar: he pushes the red button, he is already holding down. With regard to their antecedent procedure, we can say that he is following the “set and check” pattern they have previously established upon discovering the checking function of the red button widget. The circumstance that Mathew, after briefly releasing and re-pushing the red button, continues holding it down, will now “trigger” the detection of a new function of the red button.
When Jessica follows Matthew’s instruction to try out what happens when she turns the distance object (while he is still holding the red button), he multimodally displays his sudden insight that it is indeed possible to manipulate the widgets while holding down the red button (Figure 10
, lines 9–13). He moves his right hand towards his face (without touching it) while producing a deprecating utterance (“how stupid”) and then then points to the red button while verbalizing his finding (“you can hold it”). Jessica “chimes in” to complete the second part of the utterance with him (“can hold it”), marking her shared understanding of his finding. Matthew’s deprecating pre-face is in-part addressed to himself and in-part to the whole group. He does point to himself but does not complete the gesture (i.e., touching his forehead) and he omits naming himself as the recipient (e.g., “of me”). Furthermore, neither of his co-participants is producing a disagreement, which would be the preferred response to a self-deprecation.
In a sense, Mathew’s turn in line 13 comprises a noticing (“how stupid”) and a proposal that is simultaneously an assessment (“yes ok you can hold it”), that is immediately affirmed by his co-participants. Hence, this instance of discovering work is also better described as a recognition or an identification of the last feature of the red button. Here, the participants accountably downgrade their “discovery” to a long-overdue identification of an easily detectable embedded feature of the red button and display their embarrassment about their “poor” performance (i.e., not having figured out this obvious function much earlier in the process). They then immediately produce an uptake by integrating the new function in their proceeding task accomplishment and quickly find a correct solution (lines 16–26). Their collective cheering (line 27) and Jessica’s recycling of “stupid” (while reaching for the task wheel) closes this episode, as well as the discovering work regarding the functions of the red button and, soon thereafter, also regarding the curricular problem-solving activity.
Indeed, from now on they will rely on this newly discovered feature to considerably speed up the problem-solving process during the last two tasks. When Matthew suggest holding down the red button, they start increasing the pace of manipulating the widgets. At the beginning they still formulate some insights they recall or experience while watching the changing energy and force values (e.g., “no look the kilometres just change the energy,” “nothing changes with the force on pedal”). Nevertheless, their talk becomes increasingly indexical (e.g., “that one smaller,” “this one down, that one up”) and appears only to be oriented toward a fast task completion. At the end, they are even surprised when the green belt and the tick flash and Matthew utters “how did we do that? nobody knows, it doesn’t matter, okay good”). Thus, the participants’ conduct is mainly oriented to a fast task completion, pushing discovering work about mechanisms to the background.
5.6. The Process of the Second Group
As was mentioned before, particularly the second group showed similarities in their procedure of discovering the functions of the red button and accomplishing the tasks. For this reason, we provide a brief account here of how they proceeded. Already after 30 s, one participant of the second group mobilizes the red button but he does only so by touching and pressing it without verbally topicalizing it. The latter occurs twice: two respectively three minutes later, when the same participant raises the question about the purpose of the red button and the second participant displays ignorance (“I don’t know” respectively head shaking). In the following, the first participant puts his hand again over the red button but does not touch it. At this stage, the first group had uttered a hypothesis (the red button is a starting device) but common to both is that they not discover a (viable) feature of the widget in question. This happens approximately two minutes later, when both participants make their first understandings of the functions of the red button accountable. The first participant notices that the output values are displayed and the second infers that they have solved the previous assignments incorrectly (they are now at task 3). Consequently, they return to the first task and re-solve it within a minute. So, similarly to the first group, they establish the red button as a result-playing device. Upon returning to the second assignment they notice the green corona for the first time and infer from this that their current task is solved and the previous one was not. So, they go back to the first assignment and stay with it for one minute. During this time, the first participant makes verbally accountable this new insight (a green corona means that the answer is correct and a grey corona means that the answer is incorrect) and points to it as a facilitating strategy. Both is ratified by the second participant. When they return to the third assignment the first participant utters his insight “Basically now we just can fiddle until it works.” That is what they will do for most of the remaining time, leading to a considerable increase in the pace of the achievement of the remaining tasks, as it was the case for the first group.
In this article, we focused on a single case analysis and, consequently, produced an account of the multimodal organization the discovering process of the first group alone. Yet, the above summarization indicates that our work can be the first step into the establishment of a collection with regard to the accomplishment and the organization of discovery work in learning settings mediated through horizontal shared interfaces.
6. Discussion and Conclusions
Through our micro-analysis of the multimodally embodied conduct of three students engaged in a TUI-mediated joint problem-solving activity, we reconstructed how they interactionally accomplish doing discoveries in situ. While constantly displaying a joint orientation to their task, they are doing a multimodally embodied work of noticing, of directing the other’s attention, of seeking and securing mutual understanding [11
]. Finally, new insights are introduced in their shared world. In all instances, we can find the three-part sequence of proposal (sometimes preceded by a noticing), assessment and uptake as pointed out by Koschmann and Zemel [11
]. They further specify that it is “what comes between the proposal and the uptake that makes the participants’ conduct recognizably a discovery” [11
] (p. 43). This applies to excerpts 2, 3 and 4, where the proposed hypotheses are put to the test through trials which are then assessed, whereas excerpts 5 and 6 may be better described as recognitions or identifications since the proposals are immediately affirmed. Nevertheless, there is “no discontinuity between the two” but rather “a continuum of action organizations ranging from the simplest forms of identification on up to the most sophisticated forms of scientific evaluation” [11
] (p. 43).
Furthermore, through our analysis we highlighted that the participants’ discovering work were oriented to two different foci: first, the functions of the widgets and here particularly the red button, and, second, the resolution of the different assignments mediated through the TUI-activity. The participants have transformed an initially unknown object (the red button) into an effective instrument to put their hypotheses to the test. Indeed, prior to identifying that the red button can be held down to get instant access to the results while manipulating the input variables, they rely on the tangible objects, scenes and coronas to engage in discovering work with regard to the underlying bicycle mechanics model. They name problems, form tentative hypotheses, put them to the test, draw conclusions and, if necessary, restart the process. However, after recognizing what we called the “fast-tracking” function, we witness a conduct that a participant of the second group had termed “fiddling,” i.e., a quick trial and error approach exhibiting predominantly indexical and minimalistic linguistic forms. This “fiddling” conduct prevents them from engaging with the underlying physics or bicycle mechanics aspects. From a pedagogical point of view, this does certainly not correspond to a valuable outcome. Paradoxically, however, the fast-tracking procedure also corresponds to a perfecting of the mobilization of the red button widget to maximize task performance in terms of speed. In other words, they outwitted the underlying system mechanics to efficiently do what they were asked to do, namely solving the tasks.
Nevertheless, as we were able to show in our analysis, some design aspects embedded into the red button widget were, at least, partly working since they contributed to the organization of the activity as more scientific inquiry work. One such a design principle was the decoupling of the input widgets from the calculation of the results. Indeed, Hornecker [50
] argues that complex domains are forced to violate Ishii’s principle of direct and continuous mappings [51
], in order to facilitate reflection and collaborative sense-making. Our analysis is in line with this claim, since it is only when the participants restored the coupling that their discovering work ceased. Another design aspect is that the mechanical design requires active physical pressure to activate it. Other researchers used comparable spatial or physical properties (e.g., spatially separated stations [52
] or whole-body interaction on large physical spaces [53
]) to increase the burden for participants to use certain features in a tangible system, in particular those features which allow for restoring a coupling and through this, proceed with non-reflected trial and error conduct. So, in order to inhibit the use of the red button as a fast-tracking tool, one would have to render the decoupling more difficult or “costly” for the participants, for example through limiting the periods or the number of times the button can be pushed, through increasing the physical resistance of the button mechanism, or through partially disabling the input widgets while the red button is pressed.
Finally, our study thus contributes to a better understanding of how discoveries are interactionally accomplished within the context of ICT mediated joint problem-solving/discovery-learning activities, of how designed objects (such as the red button) are appropriated (partly through discovery work) by the participants to cope with the tasks at hand and of how both aspects are intertwined. So, our findings can inform developers to provide affordances to encourage learners to engage with and discover the underlying mechanisms of a simulation and burden non-reflected trial and error strategies.