Search Results (44)

Generative artificial intelligence (GenAI) is increasingly used as a cognitive aid in education, yet no single process measure captures how learners monitor and control GenAI-assisted work. Guided by a reciprocal monitoring-control account of metacognition, this descriptive, exploratory study examined what interaction logs and concurrent think-aloud speech make observable during a 90 min instructional-design task. Seventy-nine undergraduates from 11 majors completed the task with DeepSeek support. The same two-dimensional rubric was applied separately to both records: each retained semantic unit received one primary operational category (planning, monitoring, or regulation) and one evidence-depth label (implicit, awareness, strategic, or reflective). Planning was treated as prospective control and regulation as adaptive control after evaluation or feedback. The interaction-log and think-aloud records yielded 274 and 193 retained instances, respectively. In both sources, regulation was the most frequent primary label and monitoring the least frequent, while evidence-depth ratings were concentrated at awareness and strategic. The combined strategic and reflective share was 25.4 percentage points higher in think-aloud than in interaction-log monitoring units and 9.9 points higher in think-aloud than in interaction-log planning units; for regulation, it was 14.0 points higher in interaction-log than in think-aloud units. These aggregate contrasts are descriptive because coded instances are nested within participants and the sources are paired within individuals. The central contribution is that verbalized monitoring information and enacted control leave different observable traces. Combining the sources therefore provides a fuller account of metacognitive activity than either record alone, without establishing monitoring accuracy, process effectiveness, or learning outcomes. Full article

Education about AI, including its opportunities and limitations, is essential for responsible academic development. It helps students, teachers, parents, researchers and practitioners understand both the strengths and the limitations of generative AI and ensures that it is applied in ways that are ethical and socially responsible. Without this knowledge, there is a risk that generative AI will be misused or misunderstood, reducing its potential benefits. This study evaluates an educational game designed to integrate generative AI in elementary education (in art lessons), aiming to create awareness regarding the limitations of generative AI. Data was collected from 204 game sessions in two primary education schools in Greece, using serious games for art education, while the accuracy of AI outputs and the usability and level of user satisfaction were recorded. Following these tasks, researchers used the concurrent think-aloud (CTA) protocol to identify specific usability issues and responses of elementary students in biased, incomplete or inaccurate AI results. Experimental findings from this small-scale exploratory pilot study (16 students) indicated that combining RTA with serious games in art courses successfully reveal how children understand AI outputs and evaluate them based on their initial expectations. Full article

We experience narratives in different media, such as text and picture stories. While some research suggests that inference generation operates similarly across media, few studies have directly compared inference generation across media. This study investigated whether there are differences in the generation of bridging inferences for narratives presented in either texts or pictures. Participants saw wordless pictures or text versions of the same stories, and the presence of a coherence gap was manipulated within them. When there was a coherence gap, a bridging inference was required. Experiment 1 involved participants doing think aloud protocols and Experiments 2a and 2b measured processing time. The results showed an advantage of pictures for generating bridging inferences in the think-aloud task, but no difference between media in the context of processing time data. These findings suggest that differences in comprehension may be influenced by contextual factors, such as the activity that one engages (e.g., think aloud or reading silently) in when processing narratives. Full article

Users encounter privacy and permission settings across digital platforms, yet often struggle to understand, locate, and manage them effectively. Despite regulatory efforts such as the General Data Protection Regulation (GDPR) and platform mechanisms like App Tracking Transparency, these challenges persist due to interface design limitations rather than solely user capability. This study evaluates the My Information and Data Access (MIDA) framework, a user-centred privacy interface designed to support users with different levels of expertise. A between-subjects usability study was conducted with 44 participants (novice n = 15, intermediate n = 14, advanced n = 15), combining System Usability Scale (SUS) scores, task completion rates, error rates, and think-aloud protocols. The results show high usability across all groups, with SUS scores of 80 (novice), 84 (intermediate), and 92 (advanced), all exceeding the acceptability threshold of 68. Task completion rates exceeded 80%, whilst error rates remained below 25% across most tasks. These findings indicate that MIDA can support users in understanding, configuring, and managing privacy settings across different levels of expertise. This study builds on prior HCI research linking privacy management challenges to interface design limitations and provides empirical evidence that an expertise-adaptive interface can improve users’ ability to understand and manage privacy settings. Full article

Background: Shared decision-making (SDM) is particularly relevant in rheumatoid arthritis (RA), where multiple treatment options with distinct benefit–risk profiles require alignment with patient values and preferences. This study describes the development of a web-based PtDA to support treatment decision-making in RA and represents the first structured, standards-aligned PtDA in the Greek healthcare context. Methods: Guided by the Ottawa Decision Support Framework and the International Patient Decision Aid Standards, a multistage, user-centered methodology was applied, including evidence synthesis, iterative prototyping, and alpha and beta testing. Qualitative methods, including focus group discussions, semi-structured interviews, and think-aloud protocols, were used, while usability was assessed with the System Usability Scale (SUS). Methodological quality was evaluated using IPDASi v3 and UCD-11 criteria. Results: The final PtDA provides a three-step pathway supporting values clarification, comparison of medication options, and reflection on decisional confidence. It was developed as a publicly accessible, web-based tool compatible with multiple devices, with core elements also available in printable format. The tool showed good usability (mean SUS: 75.93) and strong alignment with IPDASi (83.3/100), and user-centered design criteria (11/11). Conclusions: Developing digital PtDAs is inherently complex, underscoring the importance of established methodological frameworks. The findings demonstrate acceptable usability and alignment with established standards within this early-stage development study. Further research is required to examine the tool’s impact on decision-making processes, value–choice concordance, and longer-term clinical outcomes. Full article

The current research discusses the thematic alignment of narratives and its impact on user engagement in gamified apps, using NLP Game-with-a-Purpose (GWAP) as the experimental environment. The experimental game is made up of a three-dimensional environment wherein a scene-based narrative is contextually integrated with the game world, and there is no such integration in the case of a non-scene-based narrative. Quantitative data gathered from 80 participants shows that a scene-based narrative contributes much better to user engagement, scoring higher in Focused Attention (FA) and Reward (RW) compared to the non-scene-based narrative. These findings are supported by qualitative feedback and think-aloud protocols, which showed that the participants found the scene-based narrative to be more engaging and involved than the non-scene-based material. Even though there were no statistically significant differences in cognitive load, trends of mental demand and frustration were observed indicating that thematic alignment has the potential to streamline the user experience. These findings have design implications in developing narrative-based gamified systems that can be used to improve interaction in language-related tasks. Full article

This study aims to explore teachers’ perceptions and students’ strategies when integrating AI-mediated informal digital learning of English tools (AI-IDLE) into career ESL writing instruction. This case study involved six university instructors and over 300 students in an English writing course. Although AI-IDLE has broadened English access beyond classrooms, existing research on writing skills often neglects students’ diverse strategies that correspond to their professional aspirations, as well as teachers’ perceptions. The data included a demographic questionnaire, think-aloud protocols for real-time assessment of cognitive processes during the task, and semi-structured interviews for teachers’ validation. Findings reveal three major student strategies: (1) explicit genre understanding, (2) student-driven selection of digital multimodal tools—such as Grammarly, ChatGPT, Canva with Magic Write, and Invideo—to integrate text with images, sound, and layout for improved rhetorical accessibility, and (3) alignment with students’ post-graduation career needs. Students’ work with these AI tools demonstrated that when they created projects aligned with professional identities and future job needs, they became more aware of how to improve their writing; however, the teachers expressed hopes and doubts about the tools’ effectiveness and authenticity of the students’ work. Suggestions to use AI-IDLE to improve writing were provided. Full article

Wayfinding with minimal effort is essential for reducing cognitive load and emotional stress in unfamiliar environments. This exploratory quasi-experimental study investigated wayfinding challenges in a university building housing three spatially dispersed counseling centers and three academic departments that share the building entrances, lobby, and hallways. Using mobile eye tracking with concurrent think-aloud protocols and schematic mapping, we examined visual attention patterns during predefined navigation tasks performed by 24 first-time visitors. Findings revealed frequent fixations on non-informative structural features, while existing wayfinding cues were often overlooked. High rates of null gazes indicated unsuccessful visual searching. Thematic analysis of verbal data identified eight key issues, including spatial confusion, aesthetic monotony, and inadequate signage. Participants frequently described the environment as disorienting and emotionally taxing, comparing it to institutional settings such as hospitals. In response, we developed wayfinding design proposals informed by our research findings, stakeholder needs, and contextual priorities. We used an experiential digital twin that prioritized perceptual fidelity to analyze the current wayfinding challenges, develop experimental protocols, and discuss design options and costs. This study offers a transferable methodological framework for identifying wayfinding challenges through convergent analysis of gaze patterns and verbal protocols, demonstrating how empirical findings can inform targeted wayfinding design interventions. Full article

Traditional engineering education struggles to bridge the theory–practice gap in the Industry 4.0 era, as static 2D schematics inadequately convey complex spatial relationships. While advanced visualization tools exist, their adoption is frequently hindered by requirements for specialized hardware and software, limiting accessibility. This study details the development and evaluation of a novel, web-based Digital Twin framework designed for accessible, intuitive e-learning that requires no client-side installation. The framework, centered on a high-fidelity 3D model of a historic radial engine, was assessed through a qualitative pilot case study with seven engineering professionals. Data was collected via a “think-aloud” protocol and a mixed-methods survey with a Likert scale and open-ended questions. Findings revealed an overwhelmingly positive reception; quantitative data showed high mean scores for usability, educational impact, and professional training potential (M > 4.2). Qualitative analysis confirmed the framework’s success in enhancing spatial understanding via features like dynamic cross-sections, improving the efficiency of accessing integrated documentation, and demonstrating high value as an onboarding tool. This work provides strong preliminary evidence that an accessible, web-based Digital Twin is a powerful and scalable solution for technical education that significantly enhances spatial comprehension and knowledge transfer. Full article

Computer-Aided Design (CAD) software has long served as a foundation for planning and modeling in Architecture, Engineering, and Construction (AEC). In recent years, the introduction of Augmented Reality (AR) and Virtual Reality (VR) has significantly reshaped the CAD landscape, offering novel interaction paradigms that bridge the gap between digital prototypes and real-world spatial understanding. These technologies have enabled users to engage with 3D architectural content in more immersive and intuitive ways, facilitating improved decision making and communication throughout design workflows. As digital design services grow more complex and span multiple media platforms—from desktop-based modeling to immersive AR/VR environments—evaluating usability and User Experience (UX) becomes increasingly challenging. This paper presents a longitudinal usability and UX study of a multiplatform house design pipeline (i.e., structured workflow for creating, adapting, and delivering house designs so they can be used seamlessly across multiple platforms) comprising a web-based application for initial house creation, a mobile AR tool for contextual exterior visualization, and VR applications that allow full-scale interior exploration and configuration. Together, these components form a unified yet heterogeneous service experience across different devices and modalities. We describe the iterative design and development of this system over three distinct phases (lasting two years), each followed by user studies which evaluated UX and usability and targeted different participant profiles and design maturity levels. The paper outlines our approach to cross-platform UX evaluation, including methods such as the Think-Aloud Protocol (TAP), standardized usability metrics, and structured interviews. The results from the studies provide insight into user preferences, interaction patterns, and system coherence across platforms. From both participant and evaluator perspectives, the iterative methodology contributed to improvements in system usability and a clearer mental model of the design process. The main research question we address is how iterative design and development affects the UX of the heterogeneous service. Our findings highlight important considerations for future research and practice in the design of integrated, multiplatform XR services for AEC, with potential relevance to other domains. Full article

Museums serve as essential cultural centers, yet their mostly visual exhibits restrict access for blind and partially sighted (BPS) individuals. While recent technological advances have started to bridge this gap, many accessibility solutions focus mainly on basic inclusion rather than promoting independent exploration. This research addresses this limitation by creating features that enable visitors’ independence through customizable interaction patterns and self-paced exploration. It improved upon existing interactive tangible user interfaces (ITUIs) by enhancing their audio content and adding more flexible user control options. A mixed-methods approach evaluated the ITUI’s usability, ability to be used independently, and user satisfaction. Quantitative data were gathered using ITUI-specific satisfaction, usability, comparison, and general preference scales, while insights were obtained through notes taken during a think-aloud protocol as participants interacted with the ITUIs, direct observation, and analysis of video recordings of the experiment. The results showed a strong preference for a Pushbutton-based ITUI, which scored highest in usability (M = 87.5), perceived independence (72%), and user control (76%). Participants stressed the importance of tactile interaction, clear feedback, and customizable audio features like volume and playback speed. These findings underscore the vital role of user control and precise feedback in designing accessible museum experiences. Full article

Reviews of literacy coaching show positive outcomes for teaching and learning, yet also that coaching’s impact varies widely, especially at increased scale. Thus, some scholars argue the quality of coaching interactions may matter more than broad coaching actions (e.g., co-planning, observing). Situated within Habermas’s notion of “public sphere”, we used discourse analysis to examine video-recorded pre- and post-interviews, coaching meetings, and coach retrospective think-aloud protocols of a literacy coach and elementary school teacher who described their partnership as “successful”. We examined participants’ values expressed about coaching; how each participant positioned themselves, each other, and the coaching context; and the nature of the coach–teacher discourse therein to answer the following question: what occurs in the interactional space between a coach and teacher when engaged in coaching meetings? We found four categories of values focused on participatory choice, their sense of connectedness, knowledge development, and their approach to working with/as a coach. Further, participants’ positionings signified agency for both the coach and teachers in the interactional space. While bracketing and leveraging their own authority, the coach’s language choices promoted teachers’ agency within the interactional space, providing insight into how language functions to shape the “public sphere” of coaching interactions. Full article

Reflection can help pre-service teachers in practical phases to process anecdotal evidence (and possible misconceptions contained therein) appropriately for their professional development. In an exploratory approach, we analyze the think-aloud protocols of eleven pre-service teachers for primary education who reflect on anecdotal evidence. First, we examine whether and to what extent they identify and address misconceptions. Then, we use the AIR model of epistemic cognition to investigate which epistemic processes occur when pre-service teachers reflect on anecdotal evidence. Finally, through a qualitative comparison of three protocols, we explore possible connections between the use of epistemic processes and the processing of misconceptions. While all participants noticed a misconception from the field of educational sciences and dealt with it to varying degrees of proficiency, the subject-specific didactic misconception was hardly addressed despite causing irritation, and the content-specific misconception remained unnoticed. We were able to differentiate epistemic processes in the subcategories of noticing, reasoning, conclusion, and alternative actions as well as application of epistemic ideals. Pre-service teachers who recognized a misconception and elaborated on it with scientific knowledge seem to use epistemic processes differently, which is more concretely related to the given lesson plan. We discuss the results with regard to possible implications for university teacher training and further research desiderata. Full article

Self-regulated learning (SRL) has become increasingly important for learners in the 21st century as they engage more frequently in student-centered education. Accurate self-monitoring of one’s learning process is central to SRL. However, research has shown that learners often struggle with monitoring accuracy. One possible explanation is that the process of self-monitoring itself may add cognitive load, potentially reducing both self-monitoring accuracy and overall performance. In the current study, we investigated the effects of self-monitoring on perceived mental effort and performance during problem-solving tasks of increasing complexity. Participants in the experimental condition were instructed to think aloud while problem-solving, while those in the control condition did not receive such instructions. Our findings indicate no interaction effect between the requirement for explicit and continuous self-monitoring and task complexity on perceived mental effort and performance. However, task complexity significantly impacted the perceived mental effort and performance in both conditions, while it affected monitoring accuracy exclusively in the experimental condition. Furthermore, qualitative analyses showed that the participants in the experimental conditions engaged in self-monitoring during 21.2 % of their think-aloud protocols. Contrary to previous research, self-monitoring did not appear to increase cognitive load, although the experimental participants took significantly longer to complete all the tasks. Possible explanations for these findings are discussed. Full article

This paper presents the results of a protocol study exploring the impact of various digital technologies on team collaborative design processes. Previous studies have suggested that compared to traditional methods such as sketching, digital technologies can provide further benefits for collaborative processes. However, there persists a lack of understanding about the impacts of digital technologies on such processes, particularly in relation to emerging significant digital technologies such as immersive Virtual Reality (VR). Therefore, this study aims to fill that gap by exploring team collaboration behaviours of two groups of professionals working in two digital design environments—desktop 3D modelling with Revit and immersive VR using Hyve-3D—as well as their behaviours during traditional sketching sessions for benchmarking purposes. Utilising protocol analysis method, the think-aloud data of participants was recorded, transcribed and coded using an adapted collaborative practice model. Team collaboration activities are broadly categorised as ‘Content’ or ‘Process’: content referring to design task-based activities, while process refers to activities related to the organising of group processes. The results suggest that during the design collaboration process, designers allocated the majority of their efforts towards process-oriented design activities. Differences between design environments only had a minor impact on the amount of effort expended on process-oriented activities and content-oriented activities. Moreover, traditional sketching design environments were shown to be potentially beneficial for problem-solution and associated negotiation activities. Additionally, immersive environments were associated with a reduction in the designers’ cognitive effort that was expended on exploring the design environment. Full article