Accessible Augmented Reality in Sheltered Workshops: A Mixed-Methods Evaluation for Users with Mental Disabilities

Abstract

A prominent application of Augmented Reality (AR) is to provide step-by-step guidance for procedural tasks as it allows information to be displayed in situ by overlaying it directly onto the user’s physical environment. While the potential of AR is well known, the perspectives and requirements of individuals with mental disabilities, who face both cognitive and psychological barriers at work, have yet to be addressed, particularly on Head-Mounted Displays (HMDs). To understand practical limitations of such a system, we conducted a mixed-methods user study with 29 participants, including individuals with mental disabilities, their colleagues, and support professionals. Participants used a commercially available system on an AR HMD to perform a machine setup task. Quantitative results revealed that participants with mental disabilities perceived the system as less usable than those without. Qualitative findings point towards actionable leverage points of improvement such as privacy-aware human support, motivating but lightweight gamification, user-controlled pacing with clear feedback, confidence-building interaction patterns, and clearer task intent of multimodal instructions.

1. Introduction

According to Milgram et al. [1], the reality-virtuality continuum encompasses technologies that blend the real and virtual worlds and are referred to as Mixed Reality (MR) technologies. Augmented Reality (AR) is a form of MR that augments the user’s physical environment with spatially anchored digital content, so-called holograms, while not fully immersing them. In work-related scenarios, AR can be used to present instructions and guide the user through procedural tasks by providing spatially associated text, videos, and 3D objects. Due to the technology’s contextualized way of conveying information, research has investigated various potential benefits, which mainly include improved task completion times, task completion rates, and reduced cognitive workload [2,3]. Apart from mobile and spatial (projection-based) AR, since the launch of the Microsoft HoloLens in 2015, the first realistic use cases on Head-Mounted Displays (HMDs) became increasingly possible. In that regard, research has investigated the use of AR HMDs across several domains, including healthcare [4], education [5], and manufacturing [6]. Commercial solutions offering AR-based assistance for procedural tasks are also available and primarily target frontline work [7,8].

Still, research on inclusive instructional systems for AR HMDs, in particular considering demands by users with mental disabilities, is underrepresented. Mental disabilities result from lasting psychological disorders or mental health conditions such as depression, schizophrenia, or anxiety disorder [9]. In 2021, mental health conditions were estimated to account for more than 17% of years lived with disability worldwide [10]. The terms cognitive, intellectual, and mental disability are often used interchangeably and without clear distinction [11]. We utilize the International Classification of Functioning, Disability and Health (ICF) to better understand the different nuances associated with each term [12]. It provides a framework for describing disability by shifting focus from medical diagnoses to how body functions, including mental functions, but also activities, participation, and environmental factors interact to shape a person’s functioning in daily life. Among other things, the ICF defines a set of core mental functions [12] (p. 50). Consequently, cognitive disability refers to any impairment of mental functions such as attention, memory, thought, or higher-level cognitive functions [13]. This can be due to a neurodevelopmental disorder such as an intellectual disability or a mental disability, which also impacts cognitive functioning. Crucially, mental disability is also reflected in the impairment of psychological functions such as the experience and expression of emotions (affect), handling of stress, or generation of energy and drive [14].

Having discussed the terminology, existing accessibility efforts in research and industry rarely take on this specific mental perspective. For example, Microsoft [15], Apple [16], and Meta [17] address visual, hearing, and physical barriers. Similarly, research using AR HMDs has a strong focus on visual [18] and hearing disabilities [19]. Work considering cognitive disabilities typically examines scenarios of daily living [20] with emphasis on elderly people with dementia [21,22]. In workplace settings, research primarily focuses on related technologies such as spatial AR [23,24], mobile AR [25,26], or Virtual Reality [27,28]. Yet, the specific cognitive and psychological barriers associated with mental disability may give rise to unique design requirements not fully addressed by existing accessibility efforts.

Recognizing this gap, our broader research aims to develop an AR authoring tool that enables support staff in sheltered workshops to create more effective instructions for this user group. As a crucial first step, this study seeks to gather specific requirements and perspectives by evaluating the shortcomings of a state-of-the-art system. Therefore, this work is guided by the following two research questions:

RQ1 

How does the perceived usability of a state-of-the-art AR instruction system differ between users with and without mental disabilities in a workplace setting?

RQ2 

What specific design requirements for AR instructional systems emerge from the feedback of key stakeholders, including both users with mental disabilities and their colleagues and support staff, to address this usability gap and better support cognitive and psychological needs?

To answer these questions, we conducted a user study with 29 individuals, both with and without mental disabilities. Participants used procedural instructions presented on an AR HMD as assistance for setting up a machine. We applied a mixed methods approach, gathering data through two usability questionnaires and a subsequent semi-structured interview. Quantitative data was used to evaluate perceived usability. Qualitative data provided the grounds for a thematic analysis to organize data and gain a more profound understanding of possible user requirements considering mental disability.

2. Related Work

2.1. The Use of Digital Platforms by People with Mental Disabilities

People with mental disabilities have not seen great involvement in research on the exploration of AR as a medium and its use in different contexts. To the contrary, research on AR for this user group mainly focused on diagnosing and treating mental disorders [29,30]. This is likely due to the combination of the relative immaturity of AR technology and the inherent difficulty of designing for “mental disability”—a term that is difficult to precisely define and is composed of numerous conditions. Still, in the broader digital context, research has involved this user group. McCue et al. [31] conducted a user-centered qualitative study with individuals living with major depressive disorder to inform their design of a mobile app for depression management and care. General ease of use of the app and keeping the user engaged were highlighted as particularly important by participants. Regular reminders, positive encouragements, and the ability to set own goals were perceived as helpful means of emotional support. Ludlow et al. [32] ran co-design workshops with young people with experienced mental health issues to inform the design of a digital mental health intervention platform. Authors make seven concrete recommendations for the design of such interventions, touching on the topics of customizability, relatability, and intuitiveness. Jin et al. [33] systematically reviewed barriers to user engagement of mobile apps promoting mental health. Among those, prominent barriers were unclear perceived value, fluctuating user motivation, and one-size-fits-all designs. To address these, they propose the three design dimensions of adaptivity, continuity, and multimodality to make systems personalizable to changing needs, preserve context over time, and offer alternative ways to engage. Through participatory formats, Johansson et al. [34] investigated how people with mental disabilities access and use digital services. They report a digital divide and highlight that digital services often fall short in effectively supporting individuals with mental disabilities because particular needs of this group are frequently overlooked in large-scale surveys on the use of service technologies. The authors translate their findings into the broader context of HCI and outline key design challenges, which include “making the complicated and complex more intuitive” and designing “for trust, self-esteem, and self-confidence” [34]. This precisely reflects that for mental accessibility, both cognitive and psychological barriers need to be addressed.

2.2. The Use of AR by People with Cognitive Disabilities

While research on the direct application of AR for people with mental disabilities in non-clinical settings is scarce, the adjacent field concerning cognitive disabilities has received considerably more attention. This breadth of research is evidenced by systematic literature reviews on the topic [35,36] and findings might at least be partially applicable in the context of people with mental disabilities. Involving people with cognitive impairments, Funk et al. [37] studied AR as assistance for production-related tasks and derived eight recommendations for AR-based assistance systems with a focus on spatial AR. Authors especially highlight the importance of feedback, which should be simple, direct, and personalized. Guedes et al. [25] explored the use of mobile AR by people with intellectual disabilities in the context of museum visits through focus groups, trial sessions, and field trips. The authors outline leverage points for better addressing the needs of the target group, both during a potential co-design process and the actual use of the resulting application. Exemplary named aspects are to provide adaptable hardware and interfaces, include communication supports such as text-to-speech or easy-to-read text, and foster participation by respecting individual pace and forms of expression. Koushik and Kane [38] interviewed people with cognitive disabilities and their caregivers in a participatory design format regarding their vision on how AR could support them in their everyday lives. A form factor suitable for daily life, multimodal forms of interaction, and compatibility with other assistive technology were highlighted by participants. Yin et al. [39] organized focus groups on the topic of inclusion requirements for immersive environments such as VR and AR, among others, involving ten participants with cognitive disabilities. In the case of AR, allowing for customizability, implementing multimodal interaction, and strengthening the user’s awareness of their physical surroundings were factors important to the stakeholders. The World Wide Web Consortium (W3C) [40] has also published a working note on requirements for accessible Extended Reality (XR) across different disabilities, including cognitive. Among other things, XR platforms should be highly configurable and multimodal, placing users firmly in control of interaction, timing, and safety so experiences can be tailored to diverse user requirements.

3. Methods

To investigate the perceived usability of a state-of-the-art AR instruction system (RQ1) and to derive specific design requirements for users with mental disabilities (RQ2), we applied a mixed methods approach. We compiled both quantitative and qualitative data. In the following, we outline our methodological approach by providing information about the applied use case, engaged participant group, utilized software and hardware, followed procedure, and methods of analysis.

3.1. Use Case

In cooperation with a local organization that runs shop floors employing people with disabilities, we visited multiple sites and considered different use cases. It was agreed upon that the setup of pad printing machines was the appropriate use case for this study. This workflow constitutes a realistic application scenario, as the pad printing machines are in permanent use and the organization saw actual demand and potential for AR-based guidance to enable more employees to perform the printing task. Figure 1 shows the workplace used in our study. Pad printing describes the process of applying a two-dimensional print pattern, engraved in a print plate, onto a three-dimensional target object using a squeezable silicon pad. It consists of two phases, namely (1) the assembly of the ink cup and print plate and (2) the integration of both into the machine, with eleven steps in the first and nine in the second (see

Table 1. Two-phased workflow to set up the pad printing machine. During phase 1, the ink cup and print plate are assembled in eleven steps. During phase 2, the assembled ink cup and print plate are integrated into the machine in nine steps.

Step	Description
1	Clean print and base plate
2	Place print onto base plate
3	Place ceramic ring onto plastic plate
4	Insert sealing ring into ceramic ring
5	Place ink cup onto ceramic ring
6	Check cup and ring for even gap
7	Insert magnet into ink cup
8	Configure magnet to weakest level
9	Align print plate
10	Place ink cup onto print plate
11	Sway ink cup across print plate
12	Enable setup mode
13	Start up machine
14	Switch machine to manual mode
15	Bring forward print sled
16	Fold up cup holder
17	Insert print plate into machine
18	Fasten print plate
19	Fold down cup holder
20	Confirm operability by moving print sled

).

3.2. Participants

A total of 30 participants were recruited through convenience sampling. We introduced our research intent on-site and handed out leaflets containing an overview of the planned study. One participant provided quantitative feedback only, and another one dropped out early because wearing the AR HMD caused physical discomfort. As a result, data from 29 participants was considered in this study, of which 7 were female. They had a mean age of 48 (SD = 11.82). Among the participants, 12 experience mental disability resulting from a psychological disorder or lasting mental health condition. Diagnoses among shop floor employees included depression, post-traumatic stress disorder, schizophrenia, bipolar disorder, or personality disorder. Importantly, participants report similar functional barriers at work, including but not limited to stress management, learning, decision-making, task persistence, consistency, or adapting to new circumstances. They were either undergoing vocational training or working in assembly and printing jobs. Our partner organization differentiates between work facilities for people with intellectual or mental disabilities. This study was conducted in a unit where only people with mental disabilities work. To respect privacy, no specific connection was made between individuals and their experienced disabilities. The remaining 17 participants had no diagnosed disability, were all associated with our partner organization, and included colleagues, social workers, team leaders, and site managers. All of them were either qualified occupational therapists, work education instructors, or completed additional training in special education. Of all 29 participants, five participants with and three without mental disabilities had prior experience with setting up a pad printing machine. One participant was familiar with VR on HMDs.

3.3. Apparatus

To present AR instructions to the user, the HoloLens 2 (HL2), an optical see-through AR HMD, was used as the device of choice. Instructions were created and provided to the participants using Microsoft Dynamics 365 Guides (MS Guides) [7]. MS Guides does not require the instruction author to perform any software development. The author needs to provide instructional material for each step, namely some text, a video, or an image, and select 3D objects to be used as holographic instructions. Apart from the instructional content and the spatial placement of the selected 3D objects, the system is not configurable by the instruction author. The semantic design of instructions was reviewed and discussed with two staff members who have prior experience in designing instructions with and for workers with mental disabilities. This was done to minimize the influence of instruction semantics on participants’ assessment of the system, as we wanted to evaluate participants’ views on the system’s design and functioning. MS Guides was therefore chosen because, as the primary software delivered with the HoloLens 2, it represents the current state-of-the-art and a de facto industry standard for creating this type of AR guidance.

3.4. Procedure

In the beginning, participants were briefed about the study as well as the utilized software and hardware. Explicit space for any emerging questions was given. To familiarize participants with AR on the HL2, a custom demo application (Source code available on GitHub-FamiliAR https://github.com/anjelomerte/FamiliAR (accessed on 1 December 2025)) showcasingbasic functionalities was implemented and provided for exploration. After the participants had familiarized themselves with the AR environment and the HL2, they were introduced to MS Guides through a short tutorial within MS Guides itself. In this tutorial, a few example instructions explained the general usage of the software, including how to navigate steps, use the text-to-speech feature, play videos, or reposition windows. Subsequently, participants performed the AR-guided machine setup task in the actual work environment (see Section 3.1). Study sessions took place during regular shop floor operations, which reflected a realistic usage scenario. We communicated to the participants that it was insignificant whether they completed the process successfully or not since we were in no way evaluating task performance. Whenever participants were wearing the HMD, we streamed the user’s view onto an external display to follow participants’ actions in the augmented space and assist in case of technical problems. Exemplary views of the demo application, the tutorial, and the instructions presented to the participants are shown in Figure 2.

After the practical part of each session, participants were asked to fill out three questionnaires, followed by a semi-structured interview. The questionnaires consisted of (1) a demographic survey, (2) the System Usability Scale (SUS) [41], and (3) the Post Study System Usability Questionnaire (PSSUQ) [42]. The latter two measure perceived usability and were chosen because they are straightforward, well established, freely available, applicable to AR HMDs, and can be applied to smaller sample sizes. The SUS consists of 10 statements that the user rates on a 5-point Likert scale. It yields an overall usability score between 0 (worst) and 100 (best). The PSSUQ provides more detailed insights by dividing statements into three subdimensions: System Usefulness (SU), Information Quality (IMQ), and Interface Quality (IFQ). The user grades 16 statements on a 7-point Likert scale. The questionnaires were translated into the participants’ native language. We placed particular value on preserving the original meaning of statements and considered existing translated versions during the process of translation [43,44,45]. Furthermore, the Likert scale of the PSSUQ was reversed to match the orientation of the SUS (strongly agree on the right). Based on suggestions from social workers, we supplemented the PSSUQ with an additional smiley-based scale. This practice is supported by similar research that has successfully used visual scales to adapt standard usability questionnaires for specific user groups like children [46]. The same could not be done for the SUS, as it uses both positively and negatively phrased items.

Qualitative data was obtained based on expressed feedback and the observations made during the experiment. Applying the think-aloud methodology, it was pointed out to participants that they should express their thoughts and feelings at any point during a session. Afterward, semi-structured interviews were conducted in line with Karatsareas [47]. They started by giving participants room to voice their first impressions and feedback freely. Then, they were asked open-ended questions to explore participants assessments of the system, both positive and negative, and to understand the underlying factors of system design contributing to that experience. The first level of questions of the semi-structured interview is listed in

Table 2. Open-ended questions of the semi-structured interview serve as a starting point for exploration of participants’ thoughts.

No.	Question
1	What is your general impression of the system?
2	In terms of interacting with the system, what did you like, what didn’t you like?
3	In terms of presentation of content, what did you like, what didn’t you like?
4	Did you experience any problems during the machine setup task?
5	Did anything contribute particularly positively to your experience?
6	Did anything contribute particularly negatively to your experience?
7	Do you have suggestions for improving the system?

. Follow-up questions were asked as needed to explore emergent themes. In total, each trial took roughly 60 min.

3.5. Analysis

To see whether participants with and without disability perceived usability differently, the reported SUS and PSSUQ scores are tested using a two-sided Mann-Whitney U test at a 5% significance level. The Mann-Whitney U test is chosen because it accommodates differently sized reference groups, does not make any assumptions about the underlying distribution of sample means, and ratings represent ordinal data requiring non-parametric testing. Apart from that, effect sizes are calculated to give implications about practical significance independent of sample size. Hedges’ $\mathit{g}$ is chosen due to the differently sized and rather small sample groups.

To understand and organize the qualitative data, a thematic analysis in the style of a codebook thematic analysis was performed [48]. We chose this approach because it provides a framework for organizing data descriptively but allows for the induction of new ideas while acknowledging the interpretative nature of the process. The first and second authors, who jointly facilitated the study sessions and thus were highly familiar with the data, were involved in the analysis process. The first author generated deductive codes based on notions within our interview protocol as well as inductive codes purely based on study data. Then, the second author was involved to refine, remove, or introduce codes through joint agreement. Based on this updated codebook, the first author recoded the study data, and both authors engaged in further discussion rounds, clustering codes into common themes through affinity diagramming [49]. We performed the complete analysis in the participants’ and our native language to preserve any possible nuances and translated the results afterward.

4. Results

4.1. Quantitative Results

29 participants provided quantitative feedback. Resulting distributions of reported SUS and PSSUQ scores are visualized in Figure 3. Mean ratings among disabled and non-disabled participants are listed in

Table 3. Key descriptive statistics, including mean ratings and standard deviations of the SUS, PSSUQ, and its subscales for Disabled (D) and Non-Disabled (ND) participant groups, along with p-values and effect sizes $\mathit{g}$. Asterisks (*) indicate statistical significance at p < 0.05.

Measure	Dimension	D	ND	p	g
SUS	Overall	68.96 (20.38)	82.21 (13.28)	0.025 *	0.997
PSSUQ	Overall	2.43 (0.97)	1.82 (0.16)	0.043 *	0.779
	System Usability (SU)	2.50 (1.26)	1.62 (0.49)	0.014 *	0.993
	Information Quality (IMQ)	2.62 (0.91)	2.25 (0.98)	0.325	0.393
	Interface Quality (IFQ)	1.99 (0.97)	1.54 (0.55)	0.180	0.595

. Disabled and non-disabled participants assessed usability differently. While for non-disabled users the overall SUS score was 82.21 (SD = 13.28), disabled users reported SUS scores of 68.96 (SD = 20.38). Established benchmarks for the SUS and PSSUQ are considered for contextualization. A score of 68 constitutes an average SUS score, while anything above 80 is considered well above average [50]. Applying the Mann-Whitney U test, this discrepancy is considered statistically significant ($p=0.025$) and exhibits a large effect size of $\mathit{g}=0.997$. Values larger than $0.2$, $0.5$, and $0.8$ are considered small, moderate, or large, respectively [51]. For the PSSUQ, an overall score of 2.82 is considered average, while 2.80, 3.02, and 2.49 are average for the subdimensions System Usability (SU), Information Quality (IMQ), and Interface Quality (IFQ), respectively [42]. A discrepancy in overall PSSUQ ratings between disabled (2.43) and non-disabled users (1.82) can be observed, too. The Mann-Whitney U test returns a statistically significant difference at $p=0.043$. The same is true for SU ($p=0.014$) but not IMQ ($p=0.325$) and IFQ ($p=0.180$). Small to large effect sizes are observed for overall PSSUQ ratings ($\mathit{g}=0.779$), SU ($\mathit{g}=0.993$), IMQ ($\mathit{g}=0.393$), and IFQ ($\mathit{g}=0.595$). Note that for the PSSUQ, smaller values mean better ratings.

4.2. Qualitative Results

28 participants provided qualitative feedback through their responses to our open questions during our semi-structured interview. We clustered results into five descriptive themes, namely Interaction and Navigation, Visual Presentation, System Personalization, User Engagement, and User Assistance. For contextualization and transparency, it is made clear whether feedback was provided by participants with (P_d) or without a mental disability (P_nd).

4.2.1. Interaction and Navigation

This theme encompasses all aspects of participants’ interaction with the system, which mainly involved navigation of steps and interaction with instructional content. Feedback revealed a strong preference for head-gaze navigation, where users trigger actions by dwelling on an element for a few seconds using their current viewing direction. It was highlighted that “the percentage bar helped to understand the dwell mechanism” (P_28,nd). Some participants found the dwell time to be too long and raised whether the trigger process could be sped up (P_2,nd, P_17,nd, P_29,d). Speech-based control, on the other hand, was extensively used by one participant only (P_21,nd). P_8,nd expressed potential discomfort when using speech-based interaction in group constellations, and P_20,d was concerned with involuntary activation of voice commands by others. For some users, “it just wasn’t [their] thing” (P_1,d). Hand-based and head-gaze navigation suffered from inadvertent activation, resulting in unnoticed skipping of steps. When pointed out, participants would often state, “I didn’t even notice that” (P_16,nd).

Although interaction with the system was mostly perceived as straightforward, some users expressed that, in order to properly interact with the system, “some more practice would be needed” (P_24,d). Independent of the underlying interaction modality, the fact that users could work at their own pace was perceived very positively. It gave participants a sense of “learning by doing” (P_20,d). Specific requests were also made, e.g., by P_12,d, P_15,nd, and P_16,nd who missed an option to easily advance or go back multiple steps at a time. P_1,d, P_3,d, and P_20,d desired easier hand-based navigation and asked for a physical button alongside the virtual one. P_6,d, P_8,nd, P_15,nd, and P_16,nd demanded the limitation of overall control functionalities. This was reflected by comments such as “Do I have to remember all that?” (P_26,d) considering available system controls. Besides that, P_26,nd and P_17,nd thought that short descriptive explanations for each control element would be beneficial.

4.2.2. Visual Presentation

Within this theme, we consolidated participants’ feedback regarding the visual presentation of instructions on the AR HMD. The overall presentation of instructions was received well across the board. They were fascinated by the holograms, describing them as “beautiful” (P_21,nd), “cool” (P_28,nd), and “useful” (P_1,d). “The consistent arrangement of instructions” (P_19,nd) was also considered helpful (P_24,d, P_17,nd). Videos were deemed most valuable among the available instructions. Observations confirm that many participants gradually shifted their attention towards the videos, often skipping textual instructions completely. The importance of videos was further reinforced by comments such as “Videos should take center stage” (P_19,nd) and “Text is good as an addition to the video but not the other way around” (P_30,d). The only step without an image or video was immediately criticized and caused uncertainty among participants, prompting questions such as “Why is there no video, or am I just not seeing it?” (P_13,nd).

In terms of information clarity, a recurring problem was that participants mistook videos for images because the play button on the video was not clearly visible, resulting in reactions such as “Oh, that’s a video?!” (P_6,d). Some also had problems recognizing holograms (P_25,d, P_16,nd). P_15,nd stated that “the white hologram on the white machine was difficult to recognize” and suggested a stronger contrast. Beyond that, P_5,nd stressed that “a video should show exactly what to do”. P_3,d substantiated this by pointing out that, as a left-hander, the right-handed video demonstrations were slightly disorienting. P_4,nd added that videos recorded from an exocentric perspective also cause confusion. Another issue was the limited field of view, causing virtual elements to go unnoticed or be perceived with delay. P_27,d noted that “some kind of attention grabber would have been helpful. Otherwise I might not have noticed it at all”, echoing requests for attention cues by other participants. Additional requests included the ability to control video playback (P_9,nd, P_20,d, P_27,d), zoom in on videos (P_12,d, P_13,nd, P_29,d), and toggle visibility of individual holograms (P_1,d, P_3,d, P_21,nd). To improve understanding of textual instructions, it was also suggested by multiple participants to give an overview of object names at the start of the workflow, for example, in the form of “virtual lettering next to the objects” (P_4,nd). Although all participants could read, some utilized the built-in text-to-speech functionality (P_1,d, P_24,d, P_25,d, P_30,d).

4.2.3. System Personalization

Personalization emerged as another overarching theme, which addresses the adjustment of the system to the user’s experience level as well as the customization of basic visual and functional features. Regarding the former, while most were satisfied with the instructional detail or demanded even more information, P_7,d and P_8,nd felt it was excessive already. For participants familiar with setting up a pad printing machine, it was observed that steps were partially anticipated and performed without explicit instruction. When advancing to a step that had already been performed, this contributed to uncertainty, as reflected in statements such as “I have already done that. Should I just keep going?” (P_7,d). On that front, multiple participants asked for adaptation of the instructions to their experience level. Some, having developed individual ways to set up the machine, commented, “I would have done it differently” (P_3,d) and asked if it was possible to change the instruction sequence. Regarding system personalization, common options such as customization of color (P_1,d, P_5,nd), contrast (P_7,d, P_27,d), and layout (P_14,nd, P_22,d) were mentioned. P_12,d asked for the possibility to position the video independently from the text, which MS Guides does not allow. P_6,d brought another aspect into play, wondering, “Could the system automatically adjust the window to my height?” as repositioning of the window was necessary at the beginning of the workflow.

4.2.4. User Engagement

Ideas and features contributing to the engagement and joy of the user are described by this theme. Many participants enjoyed using the AR HMD when performing the AR-guided task. Statements such as “It’s just fun” (P_29,d) or “This is fun by itself” (P_22,d) were quite common. Moreover, participants showed appreciation for the anonymity during task performance due to the individual nature of wearing an HMD. For example, P_20,d felt more comfortable and less observed because “no one notices you navigating back and forth between steps” (P_20,d). Positive reinforcement also played a critical role. “More emotional” (P_4,nd) feedback was demanded by the participants. Concrete suggestions included phrases such as “Well done! You have earned a break!” (P_6,d). Users also suggested implementing such positive reinforcement frequently (P_13,nd, P_25,d). Similar to explicit motivations and as a possible way to engage the user, gamification is another tool brought into play by some of our participants (P_6,d, P_19,nd, P_22,d, P_30,d). Importantly, it was emphasized in large parts to keep elements of gamification in a scope appropriate to the workplace. True to the motto “work is work, pleasure is pleasure” (P_30,d), elements should not be too “silly” or “childish” (P_19,nd).

4.2.5. User Assistance

The last theme addresses participants’ expressed desire and suggestions for better assistance during the AR-assisted workflow. Although some participants described the software as “helpful” (P_1,d, P_28,nd), the lack of assistance was clearly reflected in feedback, as MS Guides does not provide any support in case of questions or errors. For instance, P_2,nd encountered an unexpected error message on the machine and was unable to resolve it. Other mistakes, e.g., placing the ink cup incorrectly on the print plate, led to consequential errors. Considering this, participants suggested having the system automatically detect errors as well as confirm task completion (P_4,nd, P_20,d). P_6,d underlined that this would take away some uncertainty, which coincides with the fact that participants often reassured themselves by asking, “Is that right?” (P_12,d) or “Is that how it should be?” (P_25,d). Another request was for object-anchored holograms instead of spatially fixed ones, which led to confusion, frustration, and non-ergonomic postures. Participants generally assumed that tasks needed to be performed within a hologram’s designated area, which became apparent when P_12,d wondered whether it was permitted to complete a task in a different location. Apart from automatic assistance, participants suggested providing the user with a list of likely errors and solutions, “something like ‘This may have happened…’” (P_5,nd).

A recurring issue for users without prior printing experience was the strong magnetic snap of the ink cup onto the plate, scaring some and potentially causing damage. An existing hint in the instruction was insufficient, prompting participants to ask for dedicated warnings. P_9,nd and P_19,nd suggested requiring explicit confirmation before continuation. Other recommendations included intermediate verification through prompts saying “Take another look at the video” (P_30,d) or “The display should look like this…” (P_29,d). Incorporation of workplace organization methods such as 5S [53], already in use at the shop floor, was also imagined (P_9,nd). Lastly, despite various suggestions for virtual assistance, many users underlined their persisting appreciation for human support. P_23,d emphasized the convenience of human assistance when initially getting to know the system. P_12,d added that “I would generally prefer a human to assist me”. Going even further, P_27,d believed that “such a system will never replace a human”.

5. Discussion

In this study, we explore perspectives of users with mental disabilities on the usability of an AR-based instruction system. As a last step, we revisit our research questions and reflect on the findings of our study. First, we carefully examine quantitative results regarding the perceived usability of the system (RQ1). We then discuss qualitative feedback considering mental disability (RQ2). For that purpose, we review aspects raised by participants with a mental disability that are not yet reflected by established accessibility principles. Additionally, we address findings that coincide with known considerations from research on the design of AR-based interfaces but contain subtle nuances revealed by this study.

5.1. Users with a Mental Disability Perceive Usability Less Favorably

In terms of system usability, participants with mental disabilities perceived the AR-based instruction system as significantly less usable compared to participants without disabilities. While both SUS and PSSUQ ratings for both disabled and non-disabled user groups can be considered average or above, absolute evaluation of scores and the comparison to benchmarks can be misleading since social desirability bias may have influenced participants to rate items more positively to appeal to the researchers [54]. Besides, due to the limited pervasiveness of AR HMDs, this technology is still considered novel, which can impact usability ratings. Effects can be both negative, users associate increased complexity and are reluctant at first [55], or positive, users exhibit initial excitement that wears off [56].

Bridging quantitative and qualitative data, the discrepancy in perceived usability highlights a remaining accessibility shortcoming of MS Guides, a state-of-the-art instructional system for AR on HMDs. For example, participants requested features that MS Guides simply does not offer, such as the controllability of video playback, object-anchored holograms, or the proposition of solutions in case of errors. This calls for further efforts exploring such aspects that can explicitly involve users with mental disabilities. Microsoft in particular has contributed significantly to matters of accessibility in the academic space of XR, addressing disabilities outside the mental realm, namely visual [57,58], hearing [59,60], and mobility impairments [61,62]. Such dedicated efforts are valuable and necessary and need to be extended to consider the needs of people with mental disabilities as well.

5.2. Specific User Needs Considering Mental Disability

Our qualitative analysis revealed several key themes that illuminate the specific needs of users with mental disabilities. In the following, we discuss these themes in detail, starting with the need for a nuanced approach to human support.

Co-located but privacy-aware human support. While virtual assistance through tips, warnings, or a virtual assistant can help resolve first problems, participants with mental disabilities expressed a remaining desire for human support. Since HMDs can only be worn by one person, they compromise our fundamental psychological need of feeling connected to other people around us [63]. MS Guides offers remote support via Teams but no solution for on-site collaboration. In our study, we streamed the user’s augmented view to an external display as a workaround. However, interpreting 3D space via a 2D display has its limits. A more authentic approach would involve enabling human assistants to join the AR environment via a second HMD. On the other hand, the fact that on AR HMDs users can explore and go through and repeat instructions privately without being socially evaluated was very much valued by participants with a mental disability. “Making the consequences of errors and mistakes less ominous” for users with mental disabilities is an important factor [34]. Providing a safe space for users to perform AR-based workflows and also make mistakes would address exactly that. Interestingly, common motives for the use of HMDs over other AR technologies include their in situ character [64], hands-free use [65], and high flexibility and scalability [66]. Considering mental disability, the aspect of privacy makes another case for HMDs as the enabling AR technology. Providing the possibility for on-demand human intervention while disallowing unwanted supervision during AR-assisted work would strike a balance between the value of privacy and the concurrent need for human support.

Lightweight, motivating gamification. Gamification at work is often advocated for by research, as it can increase user engagement [67,68]. A survey among workers and supervisors at picking workplaces revealed that 66% of respondents favored a gamified over a non-gamified AR support system [69]. “Straightforward” visualizations were shown to promote higher user acceptance [68]. Among apps promoting mental health and psychological well-being, simple progress feedback is the leading form of gamification [70]. In line with that, our study participants with a mental disability did not express a strong demand for complex gamification but instead suggested that elements should mind the context of work, succinctly captured by the statement of one of our participants saying, “Work is work, pleasure is pleasure” (P_30,d). Therefore, already a simple progress bar or small animations on successful step completion are examples of lightweight gamification without distracting the user. Crucially, participants with a mental disability emphasized the importance of explicit motivation through, e.g., textual prompts. This aspect goes hand in hand with keeping gamification lightweight.

5.3. Importance of Existing Design Principles

Handling Stress. Participants with a mental disability reported stress management to be a crucial factor at work for them. This coincides with common difficulties outlined by the ICF for mental health issues [14]. With the increased application of computer vision in AR-assisted work scenarios, research has explored automatic task transitioning through contextual understanding [71,72]. However, coinciding with research involving people with cognitive disabilities [37,73], we find that user-controlled workflow pacing is strongly appreciated by study participants with mental disabilities considering potential stress caused. Importantly, the user needs to be clearly informed about any changes, even if these result from self-determined actions [74]. During our study, participants would often not notice that they, e.g., navigated to the next step, although MS Guides indicates progression through sounds and visual transition of control elements. In a busy production environment, such cues are easily missed. Therefore, even clearer feedback is necessary, for example, by announcing performed actions, showing pop-up messages, or providing confirmations that actively involve the user acknowledging workflow transitions.

Building Confidence. One important consideration in addressing the needs of users with mental disabilities is to foster the user’s confidence in interacting with a system [34]. For AR on HMDs, interaction can be facilitated by means of hand-, (head-)gaze-, or speech-based input. Interestingly, three study participants also brought the use of a physical button into play due to their familiarity with using such. While users of our study exhibited preferences for head-gaze and aversion to speech-based interaction, other studies have found divergent preferences [38,75], underlining the importance of offering different options. However, a recurring issue in our study was the unintended triggering of actions, which led to confusion, frustration, and increased tentativeness, especially among users with mental disabilities. Separating control elements and instructional content spatially more clearly and deactivating unnecessary inputs could aid in that regard. Based on repeated feedback by participants with mental disabilities, providing reassurance is considered an important factor and would also foster the user’s confidence during AR-assisted use. Proactive examples could be to display warnings, confirmations, and a list of likely errors during and prior to critical steps. Recent research is exploring ways to automatically detect mistakes [76,77] and, importantly, also propose suitable solutions [78] that would resolve potential irritations down the line due to preceding errors.

Understanding the Task. While mental disability encompasses psychological barriers, conveying the instructions understandably without cognitively burdening the user is just as important [79]. One aspect contributing to occasional confusion was the ambiguous intent conveyed by some holographic instructions. More specifically, users usually inferred a prescriptive rather than suggestive meaning from holograms. For instance, a holographic outline indicating the object of interest was interpreted as a spatial constraint for performing the corresponding action, although the action itself could be performed wherever deemed appropriate. This led to confusion, tentativeness, and non-ergonomic poses. More clearly communicating instructional intent, e.g., through explicit labeling, different color coding, or, in this case specifically, holograms anchored to the actual object instead of static ones, could improve understanding of the task at hand. Visually guiding the user towards relevant elements supports that, too. Sometimes instructional content was not immediately perceived by participants due to the restricted field of view, which is an inherent technical limitation of optical see-through HMDs [80]. To that end, research has explored both visual [81] and non-visual [82] attention guidance for limited field of view scenarios. Lastly, we found that videos and 3D holograms were valued and utilized more, sometimes exclusively. MS Guides puts the textual part of an instruction at the heart of each step. By contrast, on HMDs with a limited field of view, it is important to utilize the available space effectively, which would mean spotlighting non-textual instructions. Importantly, though, this does not make text obsolete. If anything, the use of multimodal instructions is encouraged [40,73].

6. Conclusions

In this work, we aimed to explore perspectives of individuals with mental disabilities on the use of instructional AR on HMDs. To this end, we conducted a mixed-methods user study (N = 29) involving both disabled and non-disabled individuals in a sheltered workshop. Quantitatively, participants with mental disabilities reported significantly lower usability. Qualitatively, findings reveal first leverage points for accessibility improvements considering mental disability. Potential aspects specific to the needs of users with mental disabilities include co-located, privacy-aware human support and lightweight, mainly motivating gamification. Further considerations highlighted as particularly important by participants with mental disabilities, which are rooted in general design principles, were user-controlled workflow pacing with clearer feedback to reduce stress, interaction patterns that build confidence and avoid accidental triggers, and clearer task intent of multimodal instructions to improve task understanding. Together, these results point to actionable considerations for future AR instruction systems that aim to address facets of mental disability.

Limitations and Future Work

This work should be considered an exploratory investigation of user-relevant considerations for AR instruction systems on HMDs considering mental disability. The sample and use of a single software configuration limit the generalizability of findings, which should therefore be considered hypothesis-generating rather than definitive. We chose MS Guides as it represents the industry standard for AR-based instructions, providing a baseline for the current state-of-the-art. This evaluation serves as a preliminary study for our ongoing work to develop a novel AR authoring tool that specifically incorporates design principles to support users with mental disabilities. While using a single, established system provides a clear focus, it naturally narrows participants’ feedback to the spectrum of features provided by it. A larger pool of participants with mental disabilities would also expand qualitative insights on what is important for AR systems for that user group specifically. Beyond mere size and reflecting the gender imbalance in the selected shop floor, our study exhibits more male than female participants, which future work should aim to equate. Additionally, utilization of more distinct quantitative measures would provide a more comprehensive understanding of the users’ experience. While widely used, both the SUS and PSSUQ focus on perceived usability and lack the ability to capture cognitive or emotional aspects. With AR HMDs attracting increasing attention, developing metrics specific to this technology, as seen for mobile AR [83], would be meaningful. We also acknowledge that non-disabled participants may have had a generally more positive attitude toward assistive technologies. Although we focused on qualitative feedback by participants with mental disabilities, we considered their inclusion valuable as their perspectives contribute relevant experiential knowledge of mental disability at work. Further limitations arise from our methodological choices in the qualitative analysis. We intentionally grouped the qualitative feedback from both disabled and non-disabled participants to form a holistic view. Similarly, we did not differentiate between the specific mental disabilities of participants, a decision made to gain a broad initial overview and to reflect the practical environment of the workshop where such distinctions are not made. Future work could benefit from a more targeted analysis of these distinct groups. In that regard, we believe that an accessibility approach based on a person’s mental functions rather than their diagnosis is more effective because sufficiently powered empirical groups for every diagnosis are rarely attainable, functional impairments commonly overlap across diagnoses, and it helps reduce stigma associated with diagnostic labels. Lastly, when we prompted non-disabled participants for their perspective on how the system would benefit their colleagues, their feedback was often not a clear-cut third-person assessment but rather a blend of their perceptions and usability experiences. This suggests that future research should employ more structured methods to distinctly capture these different viewpoints.

Building on the findings of this exploratory study, our current work focuses on the development and evaluation of a novel AR authoring tool. This tool will incorporate guiding mechanisms specifically designed to help AR authors in sheltered workshops to create cognitively accessible instructions, directly addressing the usability gaps and user requirements identified herein.