A Generative AI-Augmented Accessible VR Information System for Library Orientation: Prototype Development and Formative Evaluation Involving Users with Visual Impairments

Abstract

Virtual reality (VR) can support orientation in complex environments, but most VR systems remain vision-dominant and may create barriers for users with visual impairments. This study developed and formatively evaluated a Generative AI-augmented accessible VR information system for library orientation. The prototype integrated a VR library environment with visual enhancement, audio guidance, contextual notifications, speech-based interaction, and multimodal Generative AI support for environmental description, sign interpretation, bookshelf label explanation, navigation guidance, and library information access. A mixed-methods formative evaluation was conducted with eight participants with diverse visual impairment conditions. The results showed preliminary feasibility for local information access and environmental interpretation, with all participants completing sign interpretation and library information tasks. However, multi-floor navigation remained the most challenging activity, and the overall SUS score of 63.4 indicated marginal usability. Qualitative findings showed that participants valued spoken explanations and visual adjustment features, but long AI processing time disrupted exploration. These findings suggest that integrating Generative AI within an accessible VR framework is feasible for supporting library orientation, but future systems require faster processing, proactive spatial guidance, improved text recognition, and clearer feedback during AI interaction.

1. Introduction

Virtual reality (VR) and extended reality (XR) technologies are increasingly used as information systems for training, simulation, education, and situated learning in domains such as medical training, industrial simulation, and immersive classrooms [1,2,3]. By recreating physical places and task scenarios in three-dimensional space, VR allows users to explore complex environments, rehearse procedures, and practice decision-making in safe and controllable settings [4]. However, most immersive systems remain strongly vision-dominant, relying on visual attention, visual menus, visual landmarks, and visually guided movement [5,6]. These assumptions create substantial barriers for users with visual impairments, who may experience difficulty navigating virtual spaces, identifying objects, interpreting signs, reading labels, or understanding interface states. Since visual impairment includes diverse conditions such as blindness, low vision, cataracts, tunnel vision, and color vision deficiency [7,8], accessible VR systems require adaptive support rather than a single universal accessibility setting.

These challenges are particularly important in real-world orientation contexts such as libraries, museums, schools, and public service buildings, where users must understand spatial layouts, locate service points, recognize signs and labels, and access contextual information. Although VR can provide safe and repeatable orientation experiences before users visit a physical location, its usefulness depends on whether visual and spatial information can be transformed into accessible forms. Previous accessible VR approaches have often required specialized nonvisual applications or manually embedded accessibility features during development [6]. Such approaches can be difficult to scale because they depend on extensive manual labeling, continuous developer intervention, and repeated updates when environments change [9,10]. As a result, accessibility is often treated as an additional feature rather than a core part of immersive information system design [7].

Recent advances in Generative Artificial Intelligence (GenAI), Multimodal Large Language Models (MLLMs), and vision-language models (VLMs) offer new opportunities for accessible VR [11,12]. Instead of relying only on hardcoded scripts, generative AI can interpret visual scenes, identify relevant objects, process user questions, and generate natural language explanations of complex visual and spatial information [13,14]. When integrated into VR, these systems can function as conversational accessibility assistants that support environmental understanding, contextual explanation, navigation guidance, and information access [15]. This study therefore proposes and evaluates a Generative AI-augmented accessible VR information system for library orientation among users with visual impairments. The study develops an accessibility mapping model, implements a VR library orientation prototype, and conducts a formative evaluation to examine task completion, perceived usability, and user experience. Accordingly, this study was guided by the following research questions:

• RQ1. How feasible is the Generative AI-augmented accessible VR information system in supporting library orientation tasks for users with visual impairments?
• RQ2. How do users with visual impairments perceive the usability and accessibility of the proposed VR information system?
• RQ3. What usability challenges and design limitations emerge from integrating Generative AI into accessible VR-based library orientation?

2. Related Work

2.1. Accessibility Challenges for Users with Visual Impairments

The vision-first architecture of most immersive platforms creates layered barriers for people with visual impairments because essential VR tasks—locating objects, interpreting interface states, maintaining orientation, and coordinating movement—are typically communicated through visual cues [16]. Designing general accessibility is especially difficult because this population is not homogeneous: blind users often need robust non-visual substitutes (e.g., structured audio/haptic guidance), while many low-vision users depend on transforming and optimizing what remains visible (e.g., contrast enhancement, magnification, decluttering, and stabilization) rather than replacing vision entirely [6]. For low-vision users, common barriers include fixed lighting/shadows, flashing or visually busy effects, illegible text, and low-contrast UI layouts that increase search effort and cause cognitive fatigue over time [9,17].

Navigation and locomotion in 3D further intensify these issues because losing access to visual landmarks and state feedback can quickly produce disorientation and breakdowns in mobility [9]. Standard locomotion techniques such as teleportation and smooth joystick movement often assume users can visually verify destinations, detect obstacles, and confirm facing direction; without explicit multisensory verification, these techniques can become unreliable or unusable [18]. When head-mounted displays and VR apps also lack consistent accessibility configurations, non-visual UI alternatives, and stable spatial audio anchors, users may experience sensory overload (too much information, delivered unclearly or at the wrong time), which undermines independent exploration, object interaction, and sustained engagement [19].

2.2. Accessible Virtual Reality Environments

To reduce digital exclusion in immersive media, early accessible VR efforts often relied on purpose-built, non-visual architectures that replaced the screen-first paradigm with sensory substitution. A common strategy was to represent objects and spatial structure through spatialized 3D audio and grounded haptic devices (e.g., desktop force-feedback arms) that let users probe virtual geometry, trace object boundaries, and simulate cane-like exploration mechanics [20,21]. Building on these foundations, later multisensory guidance systems expanded beyond purely tactile interaction to include real-time narration and layered, localized sound cues, approaches that can communicate environmental layout, salient events, and even story-like or cinematic sequences without depending on a visual display [22]. Importantly, accessible VR has also been studied as a training medium: structured sensory environments can provide safe, repeatable practice for orientation and mobility skills, spatial reasoning, and left–right discrimination, enabling blind and visually impaired learners, particularly children and adolescents, to rehearse navigation strategies and build confidence within controlled boundaries [23].

For users with residual vision, a second major research direction focuses on enhancing rather than replacing the visual channel. Here, the goal is to make key information pop out from visual clutter and variability that otherwise overwhelms low-vision perception. Inspired by optical low-vision aids (e.g., magnification devices and selective filtering), digital approaches apply techniques such as edge enhancement, contrast manipulation, contrast reversal, dynamic zoom windows, and saliency-driven highlighting so that interactable objects and text remain legible under different lighting and background conditions [24,25]. A pivotal shift in this line of work came with SeeingVR, which demonstrated a toolkit approach, combining multiple visual and audio augmentations into modular “proxies” that can be mixed and matched depending on a user’s needs and the task context [6]. Crucially, SeeingVR also showed that accessibility can be applied post hoc: by injecting plugins into Unity-based applications, the system could alter an existing scene at runtime, adding magnification, recoloring and contrast filters, and speech-based outputs, without requiring the original developer to redesign the entire experience [6].

2.3. Generative AI and Multimodal Interaction

Vision foundation models and cross-modal generative architectures have reshaped the intersection of human–computer interaction (HCI) and artificial intelligence by enabling systems to interpret visual content with far richer meaning than earlier computer-vision pipelines [11,26]. Instead of relying mainly on brittle object detection and bounding-box classification, modern vision–language models (VLMs) can build a more semantic understanding of scenes, linking objects to their roles, relationships, and context, and support capabilities such as grounded visual question answering that is closer to how humans ask for and use information [11,26]. This shift is particularly important for assistive and interactive applications because users rarely need a list of detected objects; they need actionable explanations (e.g., what matters right now, where it is, and how to interact with it). At the same time, the move toward edge deployment has made these systems more practical in real-world settings: compact or quantized VLM variants can run on local devices, interpret the immediate visual surroundings, and produce responsive speech output with low latency, reducing dependence on cloud connectivity and making real-time interaction feel more natural [13].

When embedded into interactive systems, multimodal generative frameworks also expand beyond “seeing and describing” into “observing and adapting.” These models can fuse multiple concurrent user signals, such as gaze coordinates, hand motion trajectories, head orientation, and spoken prompts, to infer intent and predict what the user is trying to accomplish, not just what is happening visually [27]. With this kind of synchronized sensing, system behavior can change moment-by-moment: guidance can become more direct when a user appears lost, explanations can become shorter when cognitive load rises, and feedback can shift modalities depending on what the user is currently attending to. In practical terms, continuous monitoring paired with performance-aware adaptation can reduce time spent searching, lower error rates during exploration, and help manage fatigue by delivering the “right amount” of information at the right time [27]. Beyond assistive guidance, the same generative foundations can power autonomous agents that respond in more human-like ways, supporting conversational, socially interactive, and scalable multi-user ecosystems that simulate realistic communication patterns in virtual or augmented environments [1].

2.4. Generative AI-Augmented Accessibility in Virtual Reality

By tightly integrating generative models with mixed-reality runtimes, engineering frameworks can enable automated and context-aware adaptation of VR/XR content, where the system, not the developer, handles much of the accessibility transformation at run time [2,12]. This matters because many “traditional” accessible VR approaches struggle to scale beyond carefully curated demos: authoring haptic textures for every surface, labeling every object and interaction state, and writing audio descriptions for large, open-ended worlds can consume enormous development effort and are difficult to maintain as environments evolve [12,26]. Generative models offer a path around this bottleneck by programmatically parsing arbitrary scene structures, such as 3D geometry graphs, object hierarchies, and spatial relationships, and converting them into real-time multimodal outputs (e.g., structured descriptions, prioritized cues, or task-specific guidance) without requiring exhaustive manual annotation [14]. In learning contexts, this same convergence supports more than “access”; it can actively improve instruction by pairing immersive spatial layouts with generative tutoring agents that respond to a learner’s actions, which has been reported to strengthen retention and deepen contextual reasoning through interactive, situated explanations [4].

Recent progress has also pushed beyond static or pre-scripted accessibility toward real-time spatial translation and interactive querying. For example, the EnVisionVR framework leverages a vision–language model-based architecture to provide both high-level summaries (“What is this scene?”) and lower-level tools for object-focused description (“What is in front of me, and where?”), enabling users to move between overview and detail depending on their immediate goals [14]. When paired with natural voice commands and spatialized audio cues, this kind of design supports exploratory, on-demand access, allowing blind and low-vision users to ask questions about complex 3D scenes and receive guidance that improves target localization and reduces trial-and-error navigation [14]. In parallel, large-language-model-driven guides are being introduced into social VR as companion-like assistants that can accompany users through multi-user interactions, translate non-verbal spatial and social cues into accessible explanations, and provide persistent, conversational navigation support [15]. Together, these directions illustrate why generative AI is increasingly viewed as a foundational layer for inclusive VR: it enables accessibility to be personalized, situational, and continuously available, rather than static, manually authored, and limited to a small subset of experiences [15].

3. Accessibility Challenges and Design Requirements for People with Visual Impairments

People with visual impairments encounter significant challenges when interacting with VR environments because most VR systems are primarily designed for sighted users and rely heavily on visual perception for navigation, environmental awareness, and information access. As a result, users may experience difficulties identifying objects, understanding spatial layouts, locating destinations, and interpreting contextual information within virtual spaces. These challenges are particularly evident in complex environments such as libraries, museums, educational institutions, and public facilities, where successful navigation often depends on visual cues and environmental landmarks. Furthermore, the accessibility needs of visually impaired users are not homogeneous, as different conditions—including blindness, low vision, tunnel vision, cataracts, and color vision deficiency—present distinct barriers and interaction requirements.

To address these challenges, accessibility support should be tailored to the specific requirements of different visual impairment conditions rather than relying on a single universal solution. Users with blindness may require spatial audio guidance, environmental narration, and speech-based feedback, whereas users with low vision often benefit from magnification, contrast enhancement, and enlarged text. Similarly, individuals with tunnel vision may require hazard alerts and navigation assistance, while users with cataracts or color vision deficiency may benefit from visual enhancement and alternative information presentation methods. Based on these accessibility requirements, we synthesized an accessibility mapping model that associates different visual impairment conditions with AI-based scene understanding capabilities, adaptive accessibility interventions, and appropriate output modalities, as summarized in Table 1. The model integrates established accessibility design principles with Generative AI-enabled scene interpretation and multimodal interaction, providing the design foundation for the proposed accessible VR information system described in the following section.

Table 1. Accessibility Mapping Model for Diverse Visual Impairment Conditions in AI-Augmented Virtual Reality.

Visual Impairment Condition	Primary Accessibility Challenges in VR	Design Requirement: AI-Based Scene Understanding	Adaptive Accessibility Intervention	Output Modality
Blindness	No access to visual information, difficulty in navigation and object recognition [28]	Object detection, spatial mapping, OCR, landmark recognition	Spatial navigation guidance, environmental description, object explanation	Spatial audio, speech
Low Vision	Reduced visual acuity, difficulty reading text and recognizing details [29]	Text recognition, contrast analysis, object identification	Magnification, contrast enhancement, enlarged text, visual simplification	Enhanced visual display, speech
Tunnel Vision	Limited peripheral awareness, obstacle detection difficulties [30]	Hazard detection, path analysis, peripheral object recognition	Peripheral alerts, navigation assistance, hazard warnings	Spatial audio alerts
Cataracts/Blurred Vision	Blurred imagery, glare sensitivity, reduced clarity [28,31]	Contrast analysis, lighting assessment, edge detection	Brightness adjustment, glare reduction, edge enhancement	Enhanced visual display
Color Vision Deficiency	Difficulty distinguishing color-based information [32]	Color analysis, UI element recognition	Alternative color schemes, symbolic indicators, text-based cues	Enhanced visual display, speech

Note. This table presents design requirements; some AI-based functions were partially implemented in the current prototype.

4. Proposed VR Information System

4.1. System Architecture

Figure 1 illustrates the architecture of the proposed Generative AI-Augmented Accessible VR Information System developed to support library orientation for users with diverse visual impairment conditions. The system was developed using Unity 2022.3 LTS and deployed on the Meta Quest 3 headset. The architecture integrates multimodal user interaction, visual scene understanding, Generative AI reasoning, and adaptive accessibility output within a unified system architecture. User interaction begins through natural voice commands captured by the Meta Quest 3 microphone, while visual and spatial information is simultaneously acquired from the current VR scene rendered by Unity. These multimodal inputs provide complementary information regarding user intent, surrounding objects, navigation landmarks, environmental signs, and spatial context, enabling the system to deliver context-aware accessibility assistance.

Figure 1. Architecture of the Generative AI-Augmented Accessible VR Information System.

Voice input is converted into text using the OpenAI Whisper API (whisper-1 model, based on Whisper V2), whereas the user’s current viewpoint within the virtual environment is captured as a scene image for visual context analysis. The speech transcript, scene image, and contextual spatial information are integrated within the multimodal processing layer before being transmitted to OpenAI GPT-4o, which serves as the multimodal large language model (MLLM). GPT-4o performs multimodal reasoning by jointly interpreting user queries and environmental context to identify relevant objects, interpret signs and textual information, answer contextual questions, explain library facilities, and generate navigation guidance. Compared with conventional rule-based accessibility systems that rely on predefined descriptions, the proposed architecture dynamically generates personalized responses according to the user’s current location, environmental context, and information needs.

The generated responses are subsequently processed by the response generation module and synthesized into speech using the OpenAI Text-to-Speech (TTS) API before being delivered through the Meta Quest 3 audio system. In addition to speech output, the Unity application provides accessibility functions for users with residual vision, including adjustable text size, brightness, and contrast enhancement. Depending on the interaction context, the system can provide environmental descriptions, object identification, navigation assistance, contextual explanations, and location-specific guidance. The proposed architecture establishes the technological foundation for the accessible VR library prototype and the AI-enhanced accessibility functions presented in the following sections.

4.2. Development of the Accessible VR Library Environment

Figure 2 illustrates the three-dimensional virtual reality representation of Chiang Mai University Library developed as the prototype environment for this study. The development process began with the acquisition of the library’s architectural blueprints and floor plans through collaboration with the library administration. These documents provided detailed information regarding building dimensions, room configurations, service areas, and circulation pathways throughout the facility. To ensure spatial accuracy, the floor plans were used as references for constructing the overall building structure and organizing interior spaces within the virtual environment.

Figure 2. Three-Dimensional Reconstruction of the Accessible VR Library Environment.

To enhance environmental realism, photogrammetry techniques were employed to capture the visual and structural characteristics of key library spaces. A large number of overlapping photographs were collected from multiple viewpoints and processed using Agisoft Metashape Professional (version 2.1, Agisoft LLC, St. Petersburg, Russia) to generate three-dimensional meshes and texture maps. The resulting models were optimized and refined using Blender before being imported into the Unity game engine for VR development. Additional assets, including furniture, bookshelves, service counters, and navigation elements, were manually modeled and integrated with the photogrammetric models to improve visual fidelity while maintaining real-time rendering performance within the VR environment.

The resulting virtual environment consists of four interconnected floors representing the major facilities and services available within the library. The first floor primarily accommodates public service functions, including circulation services, information desks, exhibition areas, and collaborative learning spaces. The second floor contains general book collections and reading areas, while the third floor provides multimedia facilities, digital learning resources, computer workstations, and group study spaces. The fourth floor houses archives, special collections, and Northern Thai information resources. Figure 3 and Figure 4 present the floor-by-floor layouts implemented within the VR environment. The completed environment was deployed on the Meta Quest platform and served as the foundation for implementing accessibility features and Generative AI-assisted interactions described in the following sections.

Figure 3. First- and Second-Floor Implementations within the VR Library Environment.

Figure 4. Third- and Fourth-Floor Implementations within the VR Library Environment.

4.3. Accessibility Features and AI Integration

To support users with diverse visual impairment conditions, a range of accessibility features was implemented within the VR environment based on the accessibility requirements identified in Section 3. These features included visual enhancement functions, audio guidance, contextual notifications, and hazard alerts to support navigation, environmental awareness, and information access. Building upon these accessibility mechanisms, Generative AI was integrated to provide context-aware assistance during library orientation. User voice commands were captured through the Meta Quest 3 microphone and processed using the OpenAI Whisper API for speech-to-text conversion. At the same time, the system captured the user’s current viewpoint from the Unity-rendered VR scene as an image input. Before being submitted to the API, the captured image was resized and compressed to 512 × 512 pixels to reduce data size and support faster processing. The transcribed voice query and the compressed scene image were then combined as multimodal input and submitted to GPT-4o for contextual interpretation.

The Generative AI component was implemented as a context-aware interaction layer within the accessibility framework rather than as an autonomous navigation system. The interaction context was structured through a task-oriented system prompt that defined the model’s role as an accessibility assistant for library orientation and directed response generation toward environmental description, interpretation of signs and bookshelf labels, spatial guidance, and library-related information. At each interaction, the model operated on the transcribed user query together with the 512 × 512-pixel image representing the user’s current viewpoint. Response generation was therefore bounded by the visual information available within the captured scene and the user’s immediate information request, rather than by continuous visual input or access to the complete virtual environment. GPT-4o was accessed through the OpenAI API without application-specific model fine-tuning, allowing its multimodal reasoning capabilities to be integrated directly into the Unity interaction pipeline. This implementation provided on-demand contextual assistance while maintaining a functional distinction between AI-generated interpretation and the non-generative accessibility features implemented directly within the VR environment. Default API generation parameters were used, with no application-specific modification of the model decoding parameters.

Figure 5 presents examples of the accessibility features implemented within the VR library environment. The system provides audio-based guidance and contextual information through speech output, allowing users to obtain navigation assistance and environmental descriptions without relying solely on visual cues. In addition, users can customize visual settings through adjustable lighting and contrast controls to accommodate different levels of residual vision and individual accessibility preferences. Figure 6 demonstrates the application of Generative AI for contextual information interpretation and information access. The AI assistant can identify and explain bookshelf labels, interpret environmental signs, and provide contextual descriptions through natural language interaction. By transforming visual information into accessible spoken explanations, the system was intended to support independent exploration and improve accessibility within the immersive library orientation environment.

Figure 5. Audio Guidance and Visual Enhancement Features in the VR Environment.

Figure 6. Generative AI-Assisted Information Interpretation in the VR Environment.

4.4. Library Orientation Tasks

To demonstrate the practical applicability of the proposed system, a library orientation scenario was developed within the accessible VR environment. The scenario was designed to simulate the experiences of visually impaired users who are unfamiliar with the physical library and require assistance in locating facilities, accessing information, and navigating complex indoor spaces. The orientation activities were designed based on the accessibility features and Generative AI functions summarized in Table 2, ensuring that each task reflected realistic information-seeking and navigation challenges commonly encountered within academic library environments.

Table 2. Mapping of Accessibility and Generative AI Components to Visual Impairment Challenges.

Category	Component	Addressed Challenge	Target User Group
Accessibility Feature	Audio Guidance	Navigation and spatial orientation	Blindness
Accessibility Feature	Visual Enhancement	Difficulty perceiving visual information	Low Vision, Cataracts
Accessibility Feature	Contextual Notifications	Environmental awareness and hazard detection	Blindness, Tunnel Vision
Generative AI Function	Environmental Description	Understanding surrounding contexts	Blindness
Generative AI Function	Bookshelf Label Interpretation	Accessing collection information	Blindness, Low Vision
Generative AI Function	Sign Interpretation	Understanding environmental signs	Blindness, Low Vision, Cataracts
Generative AI Function	Conversational Assistance	Information access and service inquiries	All User Groups

Participants were encouraged to explore the virtual library independently while utilizing both the accessibility features and the Generative AI assistant when needed. The orientation tasks included identifying key service areas, locating designated bookshelf collections, navigating between floors, and obtaining information regarding library facilities and services. Throughout the activities, participants could interact with the AI assistant using natural voice commands to request environmental descriptions, interpret signs and labels, and receive navigation guidance based on their current location. These tasks were designed to demonstrate how the proposed system can support spatial understanding, information access, and independent navigation within an immersive virtual environment.

5. Materials and Methods

5.1. Research Design and Participants

This study adopted a research and development (R&D) approach to design, develop, and evaluate a Generative AI-augmented accessible VR information system for people with visual impairments. The study consisted of two phases: (1) prototype development and implementation and (2) user evaluation. A mixed-methods approach [33] was employed to assess the preliminary feasibility and usability of the proposed system by combining quantitative task-performance data, perceived usability scores, and qualitative interview feedback. As the study focused on formative evaluation of an early-stage prototype, the evaluation was intended to identify accessibility benefits, usability issues, and areas for future refinement rather than to establish generalizable effectiveness.

Eight participants with visual impairments were recruited through purposive sampling. Participants were eligible if they were at least 18 years old, had a self-reported or registered visual impairment, were able to communicate verbally, and were able to interact with the VR system after a short familiarization session. Participants also needed to safely wear the Meta Quest 3 headset (Meta Platforms, Inc., Menlo Park, CA, USA) and follow verbal instructions during the experiment. Individuals were excluded if they could not safely use a head-mounted display, experienced severe discomfort or cybersickness during familiarization, or were unable to complete the basic VR interaction training. The participants represented diverse visual impairment categories, including cortical visual impairment, low vision, color vision deficiency, tunnel vision, and partial blindness, as summarized in Table 3. Prior to participation, all participants received an explanation of the study objectives, procedures, potential discomforts, and their right to withdraw at any time. Written informed consent was obtained before data collection. Ethical approval was obtained from the Chiang Mai University Research Ethics Committee (COA No. 127/66).

Table 3. Participant Demographics and Visual Impairment Categories.

Participant	Gender	Age	Visual Impairment Category
P1	Female	20	Cortical visual impairment
P2	Male	21	Low vision
P3	Female	20	Color vision deficiency
P4	Male	19	Tunnel vision
P5	Female	22	Partial blindness
P6	Male	21	Low vision
P7	Female	21	Low vision and color vision deficiency
P8	Female	22	Color vision deficiency

5.2. Instruments

5.2.1. Task Completion Assessment

A task completion assessment was developed to evaluate participants’ performance during the VR library orientation activities. Performance was assessed using four indicators: task completion, completion time, the number of AI queries, and AI response latency. AI response latency was defined as the elapsed time from submission of the multimodal request to receipt of the generated response by the VR application. All participants completed the same tasks under standardized procedures, and operational definitions for each performance indicator are summarized in Table 4.

Table 4. Operational Definitions for Task Completion Assessment.

Indicator	Operational Definition	Recording Rule
Task completion	The participant successfully reached the target location, identified the required sign or bookshelf label, or obtained the requested library information.	Coded as completed or incomplete for each task.
Incomplete task	The participant stopped the task, requested direct task-specific assistance, or could not complete the task without researcher guidance.	Included in the completion rate but not interpreted as successful performance.
Completion time	Time from the end of the task instruction to successful completion or task discontinuation.	Reported in minutes; mean completion time was calculated from completed attempts.
AI queries	Spoken requests submitted to the Generative AI assistant during each task.	Repeated, rephrased, and clarification questions were counted as separate AI queries.
Researcher assistance	Support limited to safety, equipment adjustment, and repetition of standardized task instructions.	Direct answers, navigation directions, or interpretation of target information were not provided.
Task sequence	All participants completed the same five tasks in the same order.	Used to maintain consistency across participants.
Starting point	Each task began from a predefined location in the VR library environment.	Used to standardize task conditions across participants.
AI response latency	Elapsed time from submission of the multimodal request to receipt of the generated response by the VR application.	Recorded in seconds for each AI interaction.

5.2.2. System Usability Scale

The usability of the proposed system was evaluated using the System Usability Scale (SUS), a widely adopted instrument for assessing perceived system usability [34]. The SUS consists of 10 items rated on a five-point Likert scale ranging from strongly disagree to strongly agree. The instrument provides an overall usability score ranging from 0 to 100, where higher scores indicate greater perceived usability with the system.

5.2.3. Semi-Structured Interview

Following completion of the VR orientation activities, participants participated in semi-structured interviews to explore their experiences [35] with the accessible VR environment and Generative AI assistant. The interview protocol focused on participants’ perceptions of accessibility, usability, navigation support, information access, and the usefulness of AI-assisted interactions. Additional questions explored challenges encountered during system use and suggestions for future improvements. The interviews were audio-recorded and transcribed for qualitative thematic analysis.

5.3. Procedure

The study was conducted individually with each participant in a controlled laboratory setting at Chiang Mai University. Prior to the experiment, participants were informed about the objectives of the study, the experimental procedures, and the functionality of the accessible VR system. The researcher introduced the Meta Quest headset and demonstrated the basic interaction methods available within the virtual environment, including navigation controls, audio guidance functions, visual adjustment options, and voice-based interaction with the Generative AI assistant. Participants were then provided with a familiarization session to ensure that they could comfortably use the VR equipment and understand the available accessibility features before beginning the formal evaluation.

Following the familiarization session, participants entered the VR library environment and completed the orientation activities described in Section 4.4. These activities included identifying key service areas, locating bookshelf collections, navigating between floors, interpreting environmental signs, and obtaining information about library facilities and services. Throughout the session, participants were encouraged to use both the accessibility features and the Generative AI assistant whenever needed to support navigation, environmental awareness, and information access. During the experiment, task completion performance, completion time, AI interaction frequency, and AI response latency were recorded. After completing the VR activities, participants completed the SUS questionnaire. Finally, semi-structured interviews were conducted to collect qualitative feedback regarding participants’ experiences with the accessible VR environment, the usefulness of the Generative AI assistant, perceived accessibility benefits, navigation support, system usability, and suggestions for future improvements. The entire experimental session required approximately 60–75 min per participant.

5.4. Data Analysis

Quantitative data obtained from the task completion assessment and the SUS questionnaire were analyzed using descriptive statistics, including frequency, percentage, mean, and standard deviation. Task completion performance was summarized using completion rate, mean completion time, the average number of AI queries, and AI response latency. AI response latency was summarized using the mean and observed range. SUS responses were analyzed according to the standard scoring procedure to obtain an overall usability score and item-level descriptive statistics.

Qualitative data obtained from the semi-structured interviews were analyzed using thematic analysis. The interviews were conducted in Thai, audio-recorded, and transcribed verbatim. Two researchers independently reviewed the transcripts to become familiar with the data and then coded the interview content based on recurring patterns in participants’ experiences. The initial codes were compared and discussed until agreement was reached, and related codes were subsequently grouped into broader themes describing accessibility and information access, navigation and spatial orientation, user experience and system usability, and system limitations and improvement suggestions. Representative quotations were translated from Thai into English for reporting and reviewed by the research team to preserve the original meaning.

6. Results

6.1. Results of Task Completion Performance

Table 5 presents participants’ performance across the five library orientation activities conducted within the accessible VR environment. Overall, participants demonstrated relatively high task completion across most activities, suggesting the preliminary feasibility of the proposed accessibility features and Generative AI support in assisting navigation and information access. The highest completion rates were observed for interpreting environmental signs and obtaining library information, with all participants successfully completing these tasks (8/8, 100.00%). These activities also required relatively short completion times, particularly interpreting environmental signs (M = 3.28 min) and obtaining library information (M = 5.21 min). During these activities, participants made active use of the Generative AI assistant for environmental interpretation and contextual information access; however, the present evaluation design does not permit the independent contribution of the AI component to task performance to be isolated from the other accessibility features. The number of AI queries recorded during these activities also indicates that participants actively used the AI assistant when interpreting visual information that might otherwise be difficult to access through conventional VR interfaces. However, a higher number of AI queries may also reflect task difficulty or the need for repeated clarification, particularly when environmental text was small, distant, or visually complex.

Table 5. Task Completion Performance and AI Interaction Metrics.

Orientation Activity	Completion, n/N (%)	Mean Completion Time (min)	Mean AI Queries (Number)
Identify key service areas	7/8 (87.50)	6.82	4.25
Navigate to a designated floor or area	6/8 (75.00)	9.11	5.63
Identify bookshelf labels	7/8 (87.50)	6.70	6.50
Interpret environmental signs	8/8 (100.00)	3.28	5.75
Obtain library information	8/8 (100.00)	5.21	4.75

In contrast, navigation-related activities remained more challenging for some participants. Navigating to a designated floor or area achieved the lowest completion rate (6/8, 75.00%) and required the longest completion time (M = 9.11 min), suggesting that spatial orientation and wayfinding remained difficult within the virtual library environment. Individual participants with tunnel vision and partial blindness appeared to experience greater difficulty during this activity, possibly due to limitations in peripheral awareness and challenges in understanding spatial relationships between floors and destination areas. Similarly, the bookshelf identification task achieved a completion rate of 87.50% (7/8) and generated the highest number of AI queries (M = 6.50), suggesting that participants frequently relied on the AI assistant to interpret bookshelf labels and locate target collections. Some participants with low vision and color vision deficiency required repeated AI assistance during this activity, particularly when bookshelf labels were small, distant, or visually complex. These findings indicate that while the system showed promise for supporting information access and environmental interpretation, more complex navigation and collection-identification tasks still required additional guidance and system refinement. Across all AI-assisted interactions, the mean response latency was 5.08 s per interaction, with observed response times ranging from 3.20 to 9.10 s. Latency was measured from submission of the multimodal request to receipt of the generated response by the VR application.

6.2. System Usability Evaluation

Table 6 presents the results of the SUS assessment. The overall SUS score was 63.4 (SD = 7.42), indicating marginal usability of the proposed Generative AI-augmented accessible VR system. Participants reported relatively positive perceptions of ease of use and learnability, as reflected in SUS3 (M = 3.87, SD = 0.83) and SUS7 (M = 3.87, SD = 0.83). SUS1 also showed that participants had a moderately positive intention to use the system frequently (M = 3.75, SD = 0.89), while SUS5 indicated that the system functions were perceived as reasonably well integrated (M = 3.75, SD = 0.89). However, the reverse-scored items showed moderate concerns regarding system complexity, the need for technical support, inconsistency, cumbersomeness, and the learning effort required to use the system effectively.

Table 6. SUS Evaluation Results of the Generative AI-Augmented Accessible VR System.

Item	SUS Statement	Mean	SD
SUS1	I think that I would like to use this system frequently.	3.75	0.89
SUS2 (R)	I found the system unnecessarily complex.	2.50	0.93
SUS3	I thought the system was easy to use.	3.87	0.83
SUS4 (R)	I think that I would need the support of a technical person to be able to use this system.	2.87	0.99
SUS5	I found the various functions in this system were well integrated.	3.75	0.89
SUS6 (R)	I thought there was too much inconsistency in this system.	2.75	0.89
SUS7	I would imagine that most people would learn to use this system very quickly.	3.87	0.83
SUS8 (R)	I found the system very cumbersome to use.	2.62	0.92
SUS9	I felt very confident using the system.	3.62	0.92
SUS10 (R)	I needed to learn a lot of things before I could get going with this system.	2.75	1.04
Overall		63.4	7.42

Note: (R) indicates negatively worded SUS items. Item means are reported as raw responses, whereas the overall SUS score was calculated using the standard reverse-scoring procedure and converted to a 0–100 scale.

As shown in Figure 7, the overall SUS score was positioned within the marginal usability range and slightly below the commonly accepted usability benchmark. This result is consistent with observations during the experiment, where participants found the AI-assisted functions useful for navigation, sign interpretation, and information access, but some experienced frustration due to latency in voice-based interaction and image-based AI interpretation.

Figure 7. Interpretation of the Overall SUS Score for the Generative AI-Augmented Accessible VR System.

6.3. Qualitative Findings from Semi-Structured Interviews

The semi-structured interviews were analyzed using thematic analysis to examine participants’ experiences with the Generative AI-augmented accessible VR system. Four main themes emerged from the analysis: (1) accessibility and information access, (2) navigation and spatial orientation, (3) user experience and system usability, and (4) system limitations and improvement suggestions. Representative quotations are reported using participant codes and visual impairment categories.

6.3.1. Theme 1: Accessibility and Information Access

Participants described the Generative AI assistant as a useful tool for accessing visual and contextual information within the VR library environment. They frequently used the AI to identify bookshelf labels, interpret environmental signs, and obtain information about library facilities that were difficult to perceive visually. For participants with low vision, the spoken explanations were helpful for confirming information that could only be partially seen. As P2, a participant with low vision, explained, “I could see some objects, but I could not clearly read the bookshelf labels. When I asked the AI, it helped me understand what section I was looking at.” Similarly, P3, a participant with color vision deficiency, stated, “Some signs and colors were not easy for me to distinguish, but the voice explanation made the information clearer.”

Participants also described situations in which accessing information required repeated interaction with the AI assistant. Small text, densely arranged signs, or labels viewed from a distance often required participants to ask the AI assistant more than once before receiving useful information. P7, a participant with low vision, stated, “When the text was small, I had to ask again because the first answer was not clear enough.” P5, a participant with partial blindness, similarly explained, “For some signs, I needed to repeat the question two or three times before I understood the information.” P8, a participant with color vision deficiency, added, “Sometimes I had to move closer or change the viewing angle before asking again because the AI could not recognize the label the first time.”

6.3.2. Theme 2: Navigation and Spatial Orientation

Participants reported that the combination of audio guidance, environmental notifications, and Generative AI assistance supported navigation within the VR library environment. Audio guidance was frequently used to identify nearby facilities, confirm directions, and locate destinations. Warning messages for stairs and changes in floor elevation were also reported as helpful, particularly when participants approached different floor levels or transition areas. P4, a participant with tunnel vision, explained, “The warning before the stairs was very helpful because I could not always see the change in floor level. It made me feel more confident when walking.” Similarly, P5, a participant with partial blindness, stated, “The voice guidance helped me know where to go, but when I had to move to another floor, I needed to ask again to make sure I was going in the right direction.”

Participants with low vision also reported that the brightness adjustment feature improved their ability to perceive the surrounding environment. Several participants adjusted the display brightness before exploring different areas of the library and reported feeling more comfortable moving through the virtual space. P2, a participant with low vision, commented, “After increasing the brightness, I could recognize objects more easily and felt more comfortable walking around.” P6 similarly noted, “The brighter environment made it easier to notice walls, furniture, and pathways, so I felt safer when moving.” However, participants also described situations in which navigation remained difficult, particularly when moving between floors or locating destinations in unfamiliar areas. P4 stated, “Sometimes I knew the direction, but I still had to ask the AI again because I wasn’t sure if I had reached the correct place.”

6.3.3. Theme 3: User Experience and System Usability

Participants generally described the system as easy to learn and use after a short familiarization period. Most participants reported that the accessibility features, including audio guidance, brightness adjustment, environmental notifications, and AI-assisted explanations, made the VR environment easier to explore. P1, a participant with cortical visual impairment, explained, “At first, I didn’t really know what to do, but after a few minutes I got used to it. It was actually quite easy to use.” P7, a participant with low vision, also commented, “I liked that I could still use the vision I have, and whenever I couldn’t read something, the system helped me.”

Although participants appreciated the overall usability of the system, all participants described the response time of the Generative AI assistant as a major usability concern. The delay was most noticeable when asking questions about surrounding objects, bookshelf labels, or environmental signs. P6, a participant with low vision, stated, “The AI was very slow. Every time I asked something, I had to wait. Sometimes I wasn’t sure if it was still working.” P8, a participant with color vision deficiency, similarly explained, “The answers were good, but it was very slow. I just stood there waiting, and sometimes I ended up asking again.” P5, a participant with partial blindness, added, “When I asked about a sign, it took quite a while. The information was useful, but the waiting time was too long.”

6.3.4. Theme 4: System Limitations and Improvement Suggestions

When asked about limitations and possible improvements, participants most frequently referred to the waiting time during AI interaction. Participants recommended that the system should provide clearer feedback while processing voice-based queries or image interpretation requests. Suggested feedback mechanisms included a visible progress bar, a percentage indicator, an audible processing notification, or a short voice message indicating that the AI was still processing the request. P8, a participant with color vision deficiency, commented, “When I asked about something, I just had to wait. It would be better if the system could tell me that it was still processing.” P2, a participant with low vision, similarly stated, “A progress bar or a voice message saying ‘I’m still processing’ would let me know that the system is still working.”

Participants also suggested improvements to the speech output of the AI assistant. Several participants reported that some AI-generated explanations were longer than necessary during navigation or object identification tasks. They suggested that the system should allow users to adjust the speech playback speed or choose shorter responses when they needed quick information. P5, a participant with partial blindness, stated, “Sometimes the explanation was longer than I needed. It would be nice if I could make the voice speak faster.” P6, a participant with low vision, also commented, “If the answer could be shorter or faster when I was walking, it would be easier to use.”

7. Discussion

7.1. Preliminary Feasibility and Task Performance

In relation to RQ1, the task completion results suggest that the proposed system was more successful in supporting information interpretation than in supporting complex spatial navigation. The strongest performance was observed in interpreting environmental signs and obtaining library information, where all participants completed the tasks successfully. These tasks were relatively bounded and information-centered, requiring users to obtain explanations of visible signs, nearby facilities, or library-related information. This indicates that the integration of accessible VR features with Generative AI showed preliminary feasibility for transforming visual or textual information into spoken explanations. In other words, the system performed well when the user’s need was to understand “what something means” or “what information is available here.” However, this success should be interpreted as formative feasibility rather than evidence of full effectiveness, because the tasks were conducted with a small sample and within a controlled prototype environment.

By contrast, the system was less successful in supporting continuous wayfinding and fine-grained collection identification. Navigation to a designated floor or area had the lowest completion rate and longest completion time, suggesting that multi-floor spatial orientation remained difficult for users with visual impairments. This may be because VR locomotion requires not only movement control, but also reliable spatial cues, orientation feedback, and confidence in direction [36]. In addition, effective adaptive VR support should respond dynamically to user behavior and task demands, whereas the present prototype remained mainly reactive and query-based [37]. Users had to request help when uncertain rather than receiving proactive step-by-step guidance. Bookshelf identification also generated the highest number of AI queries, likely due to small or distant text, dense visual layouts, and the dependence of AI interpretation on the quality of the captured scene image. This aligns with recent AI-driven VR accessibility research showing that scene description depends on accurate scene recognition and effective conversion of visual content into non-visual feedback [38]. Overall, the prototype was more feasible for local information access and environmental explanation than for independent wayfinding, indicating the need for stronger spatial guidance, improved text recognition, and more proactive AI assistance.

7.2. Generative AI-Supported Accessibility Interaction

With respect to RQ2, the findings indicate that Generative AI played an important role in transforming visually dependent information into accessible spoken explanations. Participants used the AI assistant to interpret environmental signs, identify bookshelf labels, obtain information about library facilities, and confirm partially perceived visual content. This suggests that the value of Generative AI in accessible VR lies not only in object recognition, but also in contextual interpretation: users could ask natural language questions about the surrounding virtual environment and receive explanations that were relevant to their immediate information needs. Compared with fully pre-scripted accessibility support, this interaction model provides greater flexibility because users are not limited to predefined descriptions or fixed help menus. This aligns with recent work on AI-supported VR accessibility, which emphasizes the potential of scene description and conversational assistance to make virtual environments more understandable for blind and low-vision users [14,15].

However, the results also show that Generative AI support remained dependent on the quality of the interaction context. When labels or signs were small, distant, visually dense, or poorly framed in the captured scene image, participants often needed to repeat their questions, move closer, or adjust their viewing angle. Thus, the number of AI queries should not be interpreted only as evidence of active engagement; it also reflects moments where the system required additional clarification before producing useful information. This finding is consistent with recent discussions on GenAI for disability access and inclusive education, which emphasize that AI-generated support must be evaluated not only for response generation, but also for contextual accuracy, accessibility fit, and adaptation to diverse learner needs [39,40,41]. In this respect, the prototype demonstrates both the promise and limitations of AI-supported accessibility interaction. Generative AI can make visual information more accessible when the relevant content is sufficiently captured and interpreted, but it does not automatically solve all accessibility barriers in VR. Future systems should therefore combine AI-based scene interpretation with stronger viewpoint guidance, clearer prompts for users, confidence feedback, and mechanisms for indicating when the visual input is insufficient for reliable interpretation.

Importantly, these findings reflect the use of Generative AI as one component of a broader accessibility framework rather than its independent effect. During the evaluation, AI-based environmental interpretation and conversational assistance were available concurrently with non-generative features such as audio guidance, visual enhancement, contextual notifications, and hazard alerts. The observed task performance and usability outcomes therefore represent the combined operation of these accessibility mechanisms. Although the interaction data illustrate how participants used Generative AI for context-dependent information access, the present study design does not allow its incremental contribution to be isolated from that of the non-AI accessibility features. A controlled comparison with an otherwise equivalent non-AI condition would be required to determine the specific contribution of Generative AI to task performance and usability.

7.3. Usability Challenges and Design Implications

Regarding RQ3, the usability findings show both strengths and limitations of the prototype. On the positive side, participants were generally able to learn the system after a short familiarization period, and several accessibility features helped reduce immediate barriers during exploration. Audio guidance, spoken environmental explanations, and brightness adjustment supported users with residual vision and helped them perceive the VR library environment more comfortably. The use of natural speech input also reduced dependence on visual menus and allowed participants to request help in a flexible way. These findings suggest that the prototype was understandable and usable at a basic level, particularly for local explanation, confirmation of nearby information, and access to library-related content.

However, the most critical usability problem was the long AI processing time. The measured AI response latency averaged 5.08 s per interaction, with observed response times ranging from 3.20 to 9.10 s. Users had to stop and wait while the system processed the voice query, captured the scene image, generated a response, and converted it into speech. The measured latency, together with participants’ qualitative feedback, indicates that response delay affected the continuity of interaction during AI-assisted tasks. This delay disrupted exploration, reduced the sense of control, and caused some participants to shift their attention away from the original task. For users with visual impairments, delayed feedback is especially problematic because navigation support must be timely and situated. This concern is consistent with recent work on multimodal GenAI and image-recognition tools for blind and visually impaired users, which emphasizes the need for accurate, timely, interpretable, and interaction-aware feedback [42,43]. It also relates to uncertainty in blind users’ everyday use of GenAI, where users may be unsure whether the system has correctly understood the visual context [44]. Future versions should therefore address latency at both the interaction and system levels. Interaction-level improvements should include audio cues, progress indicators, or short processing messages, while system-level improvements could include a hybrid AI infrastructure with local or edge-based processing for frequent tasks such as speech recognition, scene preprocessing, text recognition, and retrieval of library-specific information. Caching frequently requested information, preloading spatial data, and separating fast navigation prompts from slower detailed scene interpretation would help make AI assistance more continuous and less disruptive.

7.4. Limitations and Future Work

This study has several limitations that should be considered when interpreting the findings. First, the evaluation involved only eight participants and was conducted as a formative prototype study in a controlled laboratory setting. Therefore, the results should be interpreted as preliminary evidence of feasibility and usability rather than as generalizable evidence of effectiveness. Second, although the participant group represented several types of visual impairment, including low vision, color vision deficiency, tunnel vision, cortical visual impairment, and partial blindness, the small and heterogeneous sample was not designed or sufficiently powered to compare outcomes across specific visual impairment categories. Therefore, the findings should not be interpreted as condition-specific evidence of usability or accessibility effectiveness. Furthermore, the study did not include participants who were completely blind. As a result, findings related to blindness-oriented accessibility features should be understood as design implications rather than empirically validated outcomes for completely blind users. Third, the evaluation did not include a baseline condition in which the same VR environment and accessibility features were provided without Generative AI assistance. Because the AI and non-AI accessibility components were available concurrently, their individual contributions to task performance and usability could not be isolated. Consequently, the findings should be interpreted as reflecting the integrated system rather than the independent effectiveness of the Generative AI component.

Future work should expand the evaluation with a larger and more diverse participant group, especially users who are completely blind, and compare the proposed system with conventional VR orientation, non-AI accessibility support, and real-world orientation training. In particular, a controlled comparison between AI-augmented and otherwise equivalent non-AI accessibility conditions would help determine the incremental contribution of Generative AI. Future studies should also examine whether VR-based orientation improves users’ confidence, spatial understanding, and navigation performance when they later visit the physical library. From a system perspective, future development should focus on improving AI response time, text recognition, proactive navigation guidance, and feedback during AI processing. A more optimized AI infrastructure, such as hybrid cloud and local or edge-based processing, should be explored to reduce latency and make AI assistance more continuous during movement. In addition, future versions should include clearer confidence feedback, shorter response modes, adjustable speech output, and systematic evaluation of AI interpretation accuracy to ensure that the system provides timely, reliable, and accessible support for users with different visual needs.

8. Conclusions

This study developed and formatively evaluated a Generative AI-augmented accessible VR information system for library orientation among users with visual impairments. The prototype integrated an accessible VR library environment with visual enhancement, audio guidance, contextual notifications, speech-based interaction, and multimodal Generative AI support for environmental description, sign interpretation, bookshelf label explanation, and library information access. The findings suggest that the system showed preliminary feasibility, particularly for local information interpretation and access to library-related content. However, more complex tasks, such as multi-floor wayfinding and fine-grained collection identification, remained challenging and required repeated AI assistance. The usability results further indicated that while participants could learn and use the system after familiarization, long AI processing time disrupted the continuity of exploration and reduced the sense of control. These findings highlight that AI-supported accessibility in VR should be designed not only for accurate scene interpretation, but also for timely feedback, proactive spatial guidance, and smooth interaction flow. Future work should evaluate the system with a larger and more diverse participant group, including users who are completely blind, and further optimize the AI infrastructure, text recognition, and navigation support to improve real-world orientation transfer and accessibility in immersive information environments.