Which Are the Needs of People with Learning Disorders for Inclusive Museums? Design of OLOS^®—An Innovative Audio-Visual Technology

Featured Application

The results of this research have the potential to be an important milestone in the direction of an increasingly inclusive learning environment for dyslexic people while also demonstrating the full potential of artificial intelligence and immersive reality in addressing educational problems.

Abstract

This paper proposes an innovative methodology for enhancing museum accessibility and inclusivity for visitors with specific learning disorders (SLDs) using audio-visual interfaces based on patented technology. The approach involves analyzing user preferences and dyslexic students’ self-assessments through two questionnaires. This study gathered 126 responses from both SLD-certified individuals and those without intellectual disabilities for the first questionnaire and over 1300 responses exclusively from SLD-certified individuals for the second. Results suggest practical solutions such as new visual effects, gamification methods, and user-friendly informational materials linked to an AI system. These findings serve as guidelines for developing technology to improve museum accessibility, particularly for individuals with SLDs.

1. Introduction

Technology has become an important part of modern society and has brought changes in various aspects of human life, including how museums operate [1]. Over the years, museums have begun incorporating technological advancements to enhance visitors’ experiences, including those with physical and intellectual disabilities [2].

Among those technology interventions are audio descriptions, which represent a viable solution for enhancing museum accessibility. Audio description is an audio narration that describes visual elements for people with visual impairments. In a user study at the Andy Warhol Museum in Pittsburgh, blind participants followed routes of interest while learning about the artworks. This helped researchers produce a solution to support an independent and seamless interplay between navigation and the continuous tracking of the user’s position and orientation, making art appreciation possible during the interactive museum experience [3].

In addition, museums can use QR codes, mobile applications, and simplified language to make educational information more accessible. This enables people with intellectual disabilities to access information that is typically presented in complex ways. Indeed, it must be considered, as QR codes are potential tools for teaching and learning activities [4]. For example, the “Museo Facile” project seeks to create and implement communication and multimedia material based on the principles of clarity, readability, graphic consistency, and accessibility by merging classic tools and contemporary technology (3D objects, QR Codes, etc.). In order to support the integration and active engagement of various audiences in museums, tactile panels and Italian Sign Language videos were integrated into the apparatus with the aim of encouraging an accessible and direct approach to collections for visitors with visual and auditory difficulties [5]. Visitors can scan the codes using their smartphones to access videos, photos, and other multimedia resources.

One of the most recent technologies used in museums is virtual reality. Visitors can wear virtual reality (VR) headsets to access a hyper-realistic 360-degree experience of the museum exhibits. This technology is particularly beneficial for people with physical disabilities, who may not be able to access some parts of the museum due to mobility issues. For example, The Haihunhou Relics Museum of the Han Dynasty in Nanchang exhibits historical sites and priceless cultural artifacts in an undamaged manner. By using virtual reality technology to present cultural artifacts, the museum is able to fully play up its educational value and historical responsibility [6]. In the same context, another relevant implementation is represented by the virtual and interactive exhibition called “Ancient Afrasiyab”, which features artifacts from the Afrasiab Museum in Samarkand (Uzbekistan), a city situated along the Silk Road. Visitors may interact with digital 3D items put in the application’s artificial virtual 3D environment and move around in it. It displays a variety of artifacts as 3D models that are arranged in digital spaces that represent an exhibition hall or a chamber from the era in which the artifacts were created [7]. For people with intellectual disabilities, it can help them understand the complexity of historical events through visualization. In fact, VR offers significant advantages for individuals with intellectual disabilities (IDs); it provides a safe environment for learning and practicing skills essential for independent living, overcoming real-world limitations. VR facilitates skill acquisition without fear of consequences, offers personalized scaffolding, and aids in generalizing skills across different contexts. Additionally, it enables the conveyance of abstract concepts without language, enhancing cognition and social skill development [8].

Among immersive technologies, augmented reality (AR) is also reaching popularity. This technology permits to superimpose digital information onto real-world objects. Visitors can use their head-mounted displays, smartphones, or tablets to access additional information about exhibits. For example, the usability and learning efficacy of three AR-based assistance systems for tours in medical specimen museums were assessed at the museum of the University of Yamanashi, Faculty of Medicine. The first one involved the use of AR markers. By photographing AR markers with a tablet camera, this device might show virtual label information for medical specimens. The second technique was designed to use the picture of the medical specimen as a marker, allowing visitors to access the label information without any markers obstructing the display or anatomical specimens. A head-mounted display and a natural click interaction made up the third system. The technology may provide users with a natural environment for manipulating virtual items that are scalable in the future [9]. Research indicates that AR applications in special education support self-determination, self-management, and task resolution for individuals with disabilities. Studies have shown AR’s potential to enhance learning experiences, navigation skills, and daily living skills among adults with intellectual disabilities, contributing significantly to their cognitive development and vocational skills [10].

In the end, the technology commonly referred to as “holography” (or “pepper’s ghost”) creates the appearance that virtual characters and items move, change color, or fit within other objects without the employment of personal or wearable devices. The basis of this technique lies in the use of an image with high contrast, resolution, and definition generated using a video projector or a screen. The image thus generated is reflected on a very thin transparent surface with high reflective power. Placing the image generated and the reflecting surface in such a way as to form an angle of about 45 degrees allows the users to see a “hologram” in front of them. As in the case of the European CEMEC project (2015–2019), a collaboration between universities, museums, and technical partners led to the development and evaluation of holographic showcases to promote Early Medieval arts and cultures across Europe. A traveling exhibition showed the holographic showcase’s impact on the public, emphasizing a coherent communication ecosystem integrating real and digital contents, narrative approaches, and scalability [11].

Thus, it is evident how technology has revolutionized museums’ operations, and the above-described solutions are just a few examples of how technology is being used in museums to make learning and exploring exhibits more accessible to all visitors. These technological advancements will continue to shape museum experiences, ensuring inclusivity for all visitors [12,13,14,15,16,17]. Nevertheless, incorporating immersive technologies, such as VR and AR, into education encounters several hurdles. Challenges include user acceptance problems due to technical constraints and discomfort, distractions from learning objectives, high implementation costs, safety issues like motion sickness, cybersecurity risks, communication barriers, and miscellaneous cultural and ethical concerns. Addressing these drawbacks is crucial for ensuring the effective and responsible integration of immersive technologies into educational environments, maximizing their potential benefits while minimizing adverse impacts on learners and educators alike [18].

It is worth noticing that the usefulness and typology of technology strongly vary according to the needs and issues of the people to whom they are addressed. This article focuses on specific learning disabilities (SLDs), which refer to a group of neurological disorders that impact an individual’s ability to process and use information efficiently [19,20], including reading, writing, speaking, doing math, and organizing information. These disorders can affect various aspects of learning, including difficulties with skills such as note-taking, essay organization, and expressing ideas in writing. These challenges require substantial effort to overcome, and students often experience delays and lack of communication in accessing support services. The stigma surrounding dyslexia also affects students’ willingness to seek support and interact with educators, indicating the need for improved support systems and awareness among school staff [21]. More generally, the stigma surrounding specific learning disorders has a considerable and measurable impact on individuals’ self-esteem and overall psychological adjustment, with observed effects of stereotype threat, although the consistency of these associations varies among studies and is more pronounced in psychological outcomes [22]. Common types of SLDs include dyslexia, dysgraphia, and dyscalculia.

People affected by SLDs may encounter difficulties in museums that rely heavily on reading and written material to convey information to visitors. For example, individuals with dyslexia may struggle to read labels, signs, and exhibit descriptions, making it challenging to fully understand what they are viewing [23]. Those with dysgraphia may strive to take notes, write in response to prompts, or complete any written activities or assignments that museums might offer. Individuals with dyscalculia may find it challenging to understand the mathematical concepts, such as timelines or measurements, presented in a museum exhibition.

Overall, museums need to ensure that their permanent collections, exhibits, and materials are accessible and inclusive for all visitors, including those with SLDs. This may involve offering alternative formats, such as audio or visual aids, hands-on experiences [24], or multisensory exhibits [25], to support all visitors’ full engagement and understanding.

Immersive technologies in museums have already been used but with different target groups, such as people with autism [26] and intellective [17] or physical disabilities [27]. When talking about learning disorders, virtual museums have been realized as educational tools in scholastic environments, as depicted in [28], rather than to promote the use of virtual and/or augmented reality applications inside the museum itself. Kunjir et al. affirmed that the current augmented reality applications in museums and art galleries focus on people without any disorders, recommending to consider the challenges faced by visitors suffering from learning disorders in the future design of immersive technologies for museums [29]. In addition, a recent systematic review affirms that there is still a lack of scientific publications focused on the definition of guidelines for guaranteeing the accessibility of inclusive museums for people with learning disorders [30]. From such a perspective, this paper aims to understand the needs of people with learning disorders in order to define which are the requirements for the realization of immersive applications. As a direct consequence, we seek to apply the findings in the designing of an innovative methodology to help people with SLDs enhance their museum experience. The focus of such methodology is audio-visual interfaces obtained through the European patent linked to the trademark “OLOS^®”. The “OLOS^®” methodology allows the reproduction of virtual life-size interactive human figures by using, at present, the “holographic”–“pepper’s ghost” effect. They can dialogue with visitors and re-incarnate characters from the past, thus allowing the users to revive historical, artistic, and archaeological contexts. More specifically, we seek to define the specifications for the design and realization of the “OLOS^® methodology”-based technologies in order to make them accessible and inclusive for people with SLDs. For this purpose, we implemented and administered ad hoc questionnaires, asking museum visitors about their preferences and the innovative technologies that should be included in museums to enhance accessibility, following a user-centered methodology, especially when considering people with SLDs. The present study represents the first step within the project OLOS^®DSA (DSA is the Italian acronym for SLD), aiming at improving the accessibility of museums for people with specific learning disorders.

2. OLOS^®

The Internet of Things (IoT) opens up a scenario in which things not only perceive but also communicate, facilitating an ecosystem where communication surrounds us globally and can happen anytime, anywhere. However, speaking to inanimate objects through a human voice is different from speaking to a life-size human-like virtual character who can interact with real objects in their turn [31].

Virtual human beings are capable of interacting with users and objects in different contexts whenever a visual demonstration is needed. They are suitable for one-to-many user interactions in situations as diverse as museum visits and live entertainment. Therefore, virtual human beings bring one step further the capabilities of the interaction of speaking things in that they can provide support for fully fledged dialogues using natural language. This is the main feature of the European-patented methodology linked to the trademark OLOS^® [32] (European Patent No. 2965172, Italian Patent MISE UIBM n. 000141612, European trademark UAMI n. 011115367, European design UAMI n. 002572685-001), the process for making an audio-visual interface that reproduces a life-size interactive human being. OLOS^® interfaces typically implement a hardware-inclusive component-based architecture for visualization, event management, IoT communication, natural language processing services, and speech intelligibility. Real human beings perform roles and actions captured and reproduced by the human-like interfaces. OLOS^®, therefore, exemplifies a fusion of 21st-century engineering and the creative process related to the performing arts tradition. It is intended for a full-size visualization rather than for smart devices; however, it can be effectively integrated into multiple visualization systems.

OLOS^® interactive virtual characters act as permanent storytellers and guides within museums. They are perceived visually and sensorially as real human beings present in our own space, in an extended reality where there is no need for sensory filters, such as various devices, including head-mounted displays. This allows for a broader fruition by multiple audiences with different characteristics. The choice of storyteller characters for OLOS^® interactive installations is made in agreement with the cultural heritage referents and in line with the museum’s themes. For this reason, it can fall on historical figures, as well as on multiple key characters, including contemporary ones. A single OLOS^® installation can also host more than one interactive storyteller character at a time in order to create more varied and complex experiences, allowing for the interaction with different virtual human beings, each one a bearer of different stories and facilitator of experiences. The digital storytelling conveyed by OLOS^® characters is emotionally engaging; it includes the potential of human communication but, in addition, is exponentially enhanced by the technological component. In fact, the characters’ oral stories are combined with the potential of visual effects, 2D and 3D animations, music and “holophonic” sounds (a particular technique of faithful sound reproduction based on real auditory perception), and interaction with real objects placed both inside and outside the OLOS^® installations (a strength and uniqueness of the European patent highlighted by the judging panel). In this way, in addition to telling specific stories related to certain aspects of the past, characters, places, and collection elements present in a specific museum, by way of example and not exhaustively, the characters can show images and videos related to them, can make another part of the original collection that is not currently present in the museum appear before the users, and can illuminate certain objects located in the room in sync with the narration, giving rise to interaction also with the surrounding environment. Users are able to interact with the storyteller characters through user interfaces (GUI—Graphical User Interface), establishing a dialogue and questioning the storyteller on specific museum-related topics through voice interaction or touch system (touchscreen). For questions and topics related to the narration, users have visual feedback through the GUIs, on which, always in line with the development of the interactive dialogues, the topics and themes that can be explored appear. In this way, it is possible to develop a branching conversation and personalized paths according to the choices of the users, who can choose to explore certain sub-themes rather than others and always have the possibility to go back and explore further themes, navigating within the storytelling. For unprogrammed questions, to which they do not know the answer because they are not relevant or pertinent to the experience, the characters answer by underlining the irrelevance to the museum themes. At the same time, interaction is possible through tactile interfaces (TUI—Tangible User Interface): by linking the OLOS^® system to physical objects that, when touched by users, perceive the input through advanced sensor systems, the audio-visual sequences are activated. This is also conceived with a view to greater accessibility, as the objects can represent reproductions of works actually present in the museum, to which the storytelling conveyed by the character/s is linked, which can also be sensorially perceived by users who are blind or visually impaired. It is also possible to conceive both GUI and TUI interfaces customized for children and young people, with a view to greater accessibility and involvement. Always with a view to broadening fruition and total accessibility, OLOS^® is able to go beyond human communication thanks to the multiple functions and implementations that can be foreseen. It is possible to include the multilingual functionality, which can be accessed directly through the users’ smartphones without the need to download applications or have a mobile network (cellular signal is often weak or absent in museums). OLOS^®Multicultural Development allows you to listen to the narration in a potentially infinite number of languages in sync with the character’s speech. The “OLOS^®DSA” project is the natural continuation of a path aimed at “access for all”, from the design of the physical components and user interfaces to allowing the installations to be accessible to everyone. In fact, over the years, functionalities have been developed to allow access to content for people with sensory and cognitive difficulties or disabilities through ad hoc methods. In addition to the aforementioned characteristics, OLOS^® is chosen by museums (contexts in which, especially in the Italian territory, there is a strong shortage of staff, in particular of guides and museum operators [33]) also thanks to European, national, and state incentives that promote the digital transition, as it exemplifies an investment in terms of scalability and sustainability in the long term. In fact, OLOS^® installations are open systems that are implementable and renewable over time in which it is always possible to include new content and add further functionalities, even those that have yet to be developed. Museums are also always guaranteed support in the event of malfunctions or maintenance needs, as they often do not have expert staff or personnel assigned to carry out this role.

Three basic aspects of the OLOS^® audio-visual interfaces can be identified according to the following characteristics [34]:

• Human figures derived from human originals serve as interfaces;
• Ultra-high definition (UHD) resolution levels are used to display the interface;
• An event management system (OLOS^® Event Manager—O^®EM) facilitates the execution of changes in the state of the interface in response to the input from users.

Hence, the interface assumes a human form if it is derived from a real human being. People who provide their image for the interface play their roles incorporating all the expressive qualities typical of acting in theater and movies. The effect of the holographic illusion on the user’s perception of the interface is made highly attractive and effective by the UHD resolution of the characters and elements. In a typical deployment, as shown in Figure 1, the pictures (red) are projected on a clear, suitably treated transparent polymer sheet (green) that is 1 mm thick (this thickness ensures that the refraction of the two surfaces of the sheet is minimized, preventing their overlapping). The sheet is positioned 45 degrees away from the floor. The final graphics that the viewer sees are on a vertical plane behind the reflective transparent film (yellow), totally separated from the background (blue), creating a parallax effect.

The creation of an audio-visual interface based on the patented OLOS^® method for the reproduction of life-size human characters is characterized by three phases:

• Analysis of interactions between user and virtual character;
• Special methodology for image acquisition;
• Reproduction via an optical system with vocal and gestural interaction capability.

The methodology involves the use of audio-visual sequences that can be logically managed by the system by means of the OLOS^® Event Manager (O^®EM), which will be discussed in more detail later. These sequences contain storytelling segments with the character’s reactions and interactions with the user. The first phase is closely related to the storytelling that one wishes to implement by means of the virtual character. It, therefore, starts with defining what you want to tell, including the expression, attitude, and possible modulation of these parameters. The first phase has as its output a flowchart that schematizes the analysis of the interactivity realized in harmony with the desired storytelling and the execution of appropriate actions on the peripherals connected to the system. The flowchart involves four types of sequences: (i) idle, (ii) pre-idle, (iii) action, and (iv) bridge. The system is capable of transitioning from the idle state to action or a pre-idle state when it detects the presence of a user through a volumetric sensor positioned between the wall and ceiling of the room where OLOS^® is placed, indicating to the system the beginning of the interaction to be established with the user. After that, the system waits for the request, which can be carried out verbally or manually through a touch screen. If the request is suitable, the system moves to the bridge phase by communicating the output of the request. In Figure 2, there is an example of an OLOS^® flowchart.

The second step describes the shooting method used to produce the audio-visual sequences described above. This method takes into account certain necessary parameters that result in images with three basic characteristics required for a proper visualization in the optical system:

• High and extended contrast (HDR): This is first achieved by preparing an appropriate light design for the subject. Then, a Dynamic Range value between 10 and 14 stops should be used;
• High definition: this parameter is influenced by several factors, including the resolution, the resolving power of the optics used, and the level of compression applied to the image itself. The resolution of the image must be at least 4 K (3840 × 2160 pixels), as the more details are captured, the better the visual rendering of the virtual character. The resolving power of the optics, which indicates the ability of the lens to distinguish two objects placed at extremely close distances, must at least ensure a value of modulation transfer function (MTF) of 2000 LW/PH. Finally, the compression used in this method was APPLE ProRes 422 HQ, which provides the right compromise among high color fidelity, low compression, and an affordable number of computational resources [35];
• Life size: in the reproduction of a virtual human subject, this parameter is very important, as it contributes to the increase of the user’s perception of the real presence of a virtual character [36]. This effect is achieved by a precise relationship between focal length, lens used, the distance between the subject and the shooting point, the height of the shooting viewpoint, and the format of the sensor generating the image, as can be seen in Figure 3.

The last phase focuses on the reproduction of the video segments described in the first phase within an optical apparatus with voice and gesture interaction capabilities. The optical apparatus generates a new image through the reflection of the source image on a suitably treated transparent polymer inclined at an angle between 35° and 70° to the source image. The reflection thus generated creates the optical illusion of total suspension in the vacuum of the virtual character. In Figure 4, you can see how the final image is perceived by the viewer on a vertical plane behind the plane of the transparent polymer, completely detached from the backdrop, with which it forms a parallax effect, creating the illusion of the three-dimensionality of the virtual character. This parallax effect is unique to each viewing point, which allows multiple users to simultaneously experience the same immersive experience without the use of wearables.

At the core of the interactivity of OLOS^®, there is the event manager, whose functioning is depicted in Figure 2. The OLOS^® Event Manager (O^®EM) generates a loop where different images and the corresponding actions of the interface are triggered by the actions and requests of the user. Logs of interactions can be maintained, and the knowledge upon which OLOS^® relies to provide satisfactory answers to its users can be incrementally upgraded by taking into account former failures to respond to its users.

The O^®EM can be integrated within a modular architecture so as to take advantage of a variety of technologies from IT and AI, which provide both the form and substance to the interactions, including face and speech recognition, management systems for dialogue, semantic content, and multimedia database. The OLOS^® system architecture is made up of two major subsystems, each of which has modules serving the aforementioned capabilities. An O^®EM and an IoT application infrastructure serve as examples of such subsystems.

In reality, by recognizing the presence of humans around an OLOS^® installation, the IOT application infrastructure is then able to tell the O^®EM to initiate discussion sessions with users in that area and handle the entry of human users into the area of interaction with OLOS^®. This subsystem is made up of primary parts, which are symbolized by a Wireless Sensor Network (WSN) to track the movements and locations of human users and an app designed as a common interface for touchscreens or smart personal devices; with the help of this component, users are tracked and given the option to interact with the virtual figure by asking questions using the microphone on their smart device or the one provided by the touch display. A message delivery service (MDS) is in charge of receiving verbal inputs from the app or touchscreen and sending them to the original equipment manufacturer.

The main mechanism controlling the whole procedure underpinning the administration of human-like discourse is represented by the implementation of the OLOS^® Event Manager:

• The Natural Language Processing Engine (NLPE), which converts speech signals sent by the MDS into text messages after analyzing them. It gets textual phrases ready for the O^®EM processing. A questions and answers (Q&A) engine is assigned to evaluate the textual request against a Knowledge Base holding a collection of potential replies. It is based on Voice Recognition and Speech-To-Text services implemented in the present prototypal version leveraging the Speech Recognition Platform API;
• The questions and answers (Q&A) engine, which was assigned to evaluate the textual request against a Knowledge Base holding a set of potential replies. The engine is built as a collection of logical and semantic rules in the prototypical version, which extracts the most important phrases from the request and uses them as a key to identify the best response. Each response is evaluated in relation to the word list, and the one with the best match is selected as the answer;
• The Video Dispatcher—triggered when a video answer is chosen—uses it as a query key to find the appropriate video segment in a multimedia database. Such a component—implemented as a software service—allows the virtual character to respond to the user’s spoken request and explains any potential follow-up actions that may be prompted by the response itself.

It is important to underline that the patented OLOS^® methodology is currently in use and widely applied in the field of cultural heritage, and the aim of this study is to expand its capabilities to support people who have specific learning disorders.

3. Materials and Methods

In accordance with the aim of the paper, we decided to implement an ad hoc questionnaire to understand how the user would like to introduce innovative technologies in a museum environment. In addition, in order to understand the needs of people with SLDs, the answers to an already implemented and widespread questionnaire were taken into account and analyzed. Both questionnaires are detailed in the following paragraphs.

3.1. Museum Questionnaire

A survey was created with the help of a speech-language pathologist to ensure the best understanding and ease of compilation for all users, using a universal design for learning approach. It was made available online to accelerate the data gathering concerning visitors’ interactions with museums and their awareness of the cutting-edge immersive technology that is becoming more prevalent in daily life. Participants aged 35 residing in Italy, excluding museum workers, were recruited through various channels, including educational institutions, community organizations, and online platforms. Sampling aimed to ensure representation across different cultural backgrounds, with a focus on inclusivity and diversity. Both individuals with SLDs and those without were encouraged to participate unless they had additional cognitive problems, regardless of nationality or gender. The questionnaire, named “Holography: The Museum that I Would”, was organized as follows.

The first group of questions concerned some socio-demographic data (such as gender and nationality). They asked if the interviewee was a student or a worker and if he/she had some kind of learning disability.

The second part concerned information about the interest in museums. In particular, the person being interviewed had to report (i) if he/she is used to going to museums, (ii) how many times a year he/she goes, and (iii) what kind of museum he/she prefers. Considering the first question, a score from 1 to 10 had to be selected, whereas the second one was associated with four possible choices: never, once per year, from two to five times per year, and more than five times. The last question allowed for the selection of several types of museums with a multiple-choice format, including archaeological museums, art museums, open-air museums, science museums, natural history museums, natural science museums, and history museums.

It continued by trying to figure out some personal preferences, opinions, and issues, such as the following:

• If he/she prefers to visit a museum in company or alone;
• If he/she prefers that information and contents be exposed through (multiple choice answers) guidebooks, totems next to the artworks, audio guides, giant screens next to the artworks, personal smartphones or tablets, and museum guides;
• What kind of experience he/she considers a museum tour. In this case, the interviewee could choose between three multiple-choice options: recreation, emotional, and learning;
• If it was easy to find the right information during the visit;
• If sometimes he/she is bored or tired throughout the experience, and if yes, he/she was asked to report the reasons;
• If he/she would like to be involved in activities during the tour.

The last questions of this part focused on the user‘s knowledge and viewpoint on the new technologies that are catching on inside museums: (i) if he/she had ever heard about immersive reality, (ii) if the interviewee knew the difference between augmented and virtual reality (it is important to underline that this part of the questionnaire also contained the possibility for the users to receive more information about the concepts of virtual and augmented reality through a hyperlink), (iii) if he/she had ever had an experience with a hologram, and (iv) he/she was asked for a personal opinion about this technology used during a museum tour.

In the end, the survey concerned “holographic” technology. Participants were asked for their point of view on how a holographic experience should be realized to increase the accessibility of museum contents as follows:

• The usefulness of the explanation provided by a holographic character, with the possible answers being “yes”, “no”, or “not known”;
• The kind of character’s aspect that was preferred, with the following possible choices: (i) human, (ii) robot, (iii) animal, and (iv) typology depends on narration;
• The kind of position that should be assumed by the character during the presentation, with the following possible choices: (i) standing and far, (ii) standing and close, (iii) sitting and far, and (iv) sitting and close;
• Time willing to spend on multimedia storytelling, with the following possible choices: (i) less than one minute, (ii) less than five minutes, (iii) from five to ten minutes, and (iv) more than ten minutes;
• Interest in interacting with characters or objects, with the possible answers being “yes”, “no”, or “indifferent”;
• The preferred manners of interaction, with the following possible choices: (i) facial expression recognition, (ii) vocals, (iii) limb movements, and (iv) external devices, such as mouse and touch screen;
• How should an engaging background be, with the following possible choices: (i) realistic, (ii) virtual, (iii) monocolor, and (iv) the same environment of the museum attraction;
• Favorite device to enjoy the hologram, with the possible choices: (i) personal smartphone/tablet, (ii) shared tablet, (iii) shared totem, (iv) maxi-screen, and (iv) hologram in motion without any physical support;

Even in this case, the interviewee had the possibility to receive more explanations about “holography” by using a hyperlink.

3.2. VRAIlexia Questionnaire

The second questionnaire had already been implemented, administered, and validated within the VRAIlexia project, which was carried out by the same research group involved in the present paper. Briefly, VRAIlexia [37] is a project funded by the European Commission (Erasmus+) and coordinated by the University of Tuscia. Through the development of an original and inclusive teaching strategy, it aims to develop educational resources and services for university-bound dyslexic students. Exploiting the potentialities of artificial intelligence and virtual reality, VRAIlexia allows for the realization of a web-based platform able to comprehend the primary problems faced by the students and provide them with tailored digital support approaches to remove the challenges they encounter in their academic studies.

During the project, an informal interview with dyslexic students was conducted to examine aspects relating to metacognition and learning methods. This information was used to construct a survey for self-evaluation that allowed students to talk about their study issues and the supporting strategies and tools that each of them found helpful.

As a result, a digital questionnaire was developed and subsequently placed online to facilitate the acquisition of data, starting in March. The questionnaire was structured as follows: after a few demographic inquiries, it asked about high school and university experiences, as well as the person’s status and history of dyslexia:

• Which had been the main issues experienced during the last years of the learning path;
• Which had been the most useful supporting tools;
• Which had been the most useful supporting strategies.

Each group was organized in multiple choice questions, rated from 1 to 5, concerning one of the issues, tools, or strategies that emerged from the interviews with dyslexic students. Furthermore, four other options were made available to allow the AI to identify why a certain methodology was not regarded as helpful. These options were the following: “not experienced” if the interviewee had never encountered the issues; “not useful” if he/she thought that a specific tool or strategy was worthwhile; “never tried” and “don’t know” if the student had never considered a supporting methodology.

For more details about the questionnaire, please refer to Zingoni et al. [38], Yeguas-Bolívar et al. [39], and Benedetti et al. [40].

4. Results

Questionnaire results are depicted in the following for both the Museum and VRAIlexia answers.

4.1. Museum Questionnaire

From this survey, we collected 126 replies. As observable in Appendix A.1, since this work focused on high school and academic students, it had a heterogeneous distribution of respondents’ ages, with about only 20% of the interviewees being over 30 years old. Regarding gender, a perfect balance between males and females (50% each) was achieved. The only homogeneous result concerned nationality, with about 95% being Italian users. However, it is worth noticing that the questionnaire was only spread in Italy. Regarding the status of the interviewees, most of them were students; specifically, almost half were academic students, and about 20% were high school students. Instead, talking about employment status, the answers were again balanced, highlighting that a significant share of the academic student’s group were also workers. Finally, the last field focuses on specific learning disorders. From the survey, it emerged that about 40% experienced SLDs; furthermore, almost the entirety suffered from dyslexia, and the vast majority of them had experienced at least another disorder. In summary, if we exclude nationality, we can consider the cross-section of age, gender, students, workers, and experienced disorders very heterogeneous.

In Appendix A.2, it can be seen firstly how the interviewees answered questions about museum preferences, showing that there is a large interest in museums; more than 75% of the surveyed people voted more than eight, and more than 60% of them were used to visit museums more than once a year. Considering the museum type, art museums were the most chosen, followed by science museums, while the others were on the same level. The second part focuses on some personal preferences, opinions, and issues of the interviewees who are museum visitors. Talking about preferences, a significant part of the participants (40%) highlighted the necessity of support during the visit, whether by their smartphone or a guide. Moving forward, the majority of them visit museums in the company of others, whereas the preferred modality to access content and information is the autonomous ones, with the technology system in first position, followed by the traditional ways of fruition. In this context, considering that this is a multiple-choice question, it is interesting to notice that museum guides collected only one preference. When we asked them what kind of experience a museum visit is, most of them answered it was a learning one, but due to the multiple-choice options, a relevant portion of them emphasized the importance of the emotional and recreational aspects of a museum tour. In fact, when they were later asked if it would be interesting to take part in some activity, more than 80% of them answered yes, suggesting the possible engagement through laboratories and games. While most of the surveyed have expressed the issue of feeling tired or bored during a museum tour, only one of the interviewees used the “other” option, because, given the fact that he moves in a wheelchair, he feels tired when a place is too crowded. The last part of this Table investigates the knowledge and the viewpoint of the users on new technologies that are catching on inside museums. When the respondents were asked if they knew the difference between virtual and augmented reality, one out of three declared no, and a similar percentage was collected concerning immersive reality. Instead, when the interviewees were asked if they had ever experienced holograms, a bit more than 50% answered yes, pointing out that there is still confusion regarding the differences between them in this field. The last question of this part asked for a personal opinion; in particular, the interviewees were asked if they thought that this new technology could be useful inside museums; only one person said “no”, ten of the interviewees replied “don’t know”, and all the others thought it could be useful to enjoy innovative technologies inside museums.

The results of the last part of the questionnaire, the one dedicated to the use of holography inside the museums, can be seen in Appendix A.3. The first question asked to the people surveyed concerned the use of holographic characters for explanation purposes; again, only one said “no”, twelve of the interviewed people replied “don’t know”, and all the others thought holography could be useful. The next one asked the time he/she is willing to dedicate to multimedia attraction on artworks; less than 10% said “less than one minute”, about 46% answered “less than five minutes”, 35% of them chose “from five to ten minutes”, and only 8% said “more than ten minutes”. Moving forward, when asked if it could be interesting to interact with holographic objects and characters, almost all of the interviewees said yes, with the exception of two. The fourth question asked about the aspect of the holographic character during the storytelling: the answers “robot” and “animal” reached one preference each, whereas “human” was around 45%; all the others (about 50%) thought that this choice should be taken according to the kind of storytelling. Talking about the types of interactions, the speaking one was the most chosen, with about 43% of the votes, followed by the most common input device (such as a touchscreen, mouse, or keyboard) with 27% of the votes; instead, limb movement and facial emotion recognition were the least selected. In reference to the character’s position (standing or sitting/far or close), more than 87% preferred the hologram in standing position, with three out of four of them who preferred the character close; about the others surveyed, they chose “standing and far”, except for one, who said, “sitting and close”. Concerning the background, more than 60% expressed a preference for “the same of which the artwork is placed”, and about 30% of the interviewees answered “realistic”; “virtual” and “monocolor” reached the worst scores, with 6.3% and 1.6%, respectively. The last question of the survey asked about the device where the holographic character should be visualized. Approximately half of the answers were “in a movement without physical supports”, while the others were all around the same level: “personal Smartphone/Tablet” reached a level slightly more than the other ones (17.5%) and is followed by “Totem next to the artwork” (14.3%), “Tablet next to the artwork” (12.7%), and “giant screen next to the artworks” (9.5%).

Table 1. Questionnaire excerpt focus on technologies and tools that can be implemented within the museum, and then, opinions and preferences regarding immersive realities going on to subdivided by people with and without SLDs.

	SLDs   Not SLDs
2.5 In which ways would you like the contents and information to be offered to you?
Paper guide *	21.4%	25.7%
Totem near the museum work *	32.1%	42.9%
Audio guide *	46.4%	45.7%
Maxi-screen near the museum work *	39.3%	20.0%
Personal smartphone/tablet *	32.1%	45.7%
Museum guide *	0%	2.90%
2.9 Would you like to be involved in some activities while visiting a museum?
No.	32.1%	14.3%
Yes, with an educational workshop *	57.1%	57.1%
Yes, by playing games *	17.9%	54.3%
Other *	0%	5.71%
2.13 Do you believe that these new technologies (i.e., XR) can be useful within a museum?
No	3.57%	0%
Yes	75.0%	88.6%
I do not know	21.4%	11.4%
3.3 Do you think it would be interesting to interact with holographic objects/characters?
No, I prefer a static reproduction with only language	3.57%	2.86%
Yes, that would be much more interesting	96.4%	97.1%
3.5 How would you like to interact?
Touch screen/keyboard/mouse	25.0%	28.6%
Requests made by voice	28.6%	54.3%
Limb movement (hand/arm/legs)	21.4%	14.3%
By facial expression recognition	25.0%	2.86%
3.7 What kind of background would you like behind the holographic character?
Monocolor	0%	2.86%
Virtual	7.14%	5.71%
Realistic	32.1%	28.6%
The same as the environment in which the museum works	60.7%	62.9%
3.8 Where would you like the holographic character to be displayed?
Personal smartphone/tablet	17.9%	17.1%
Big screen near the museum artwork	17.9%	2.86%
Totem near the museum artwork	7.1%	20.0%
Moving around the room without physical supports	42.9%	48.6%
Tablet near the museum artwork	14.3%	11.4%

* multiple-choice option.

shows an excerpt of the results regarding the above questionnaire with, first, a focus on technologies and tools that can be implemented within the museum, and then, opinions and preferences regarding immersive realities going on to subdivide the responses between people with and without SLDs.

4.2. VRAIlexia Questionnaire

This questionnaire, which was uploaded to the internet in March 2020, has witnessed huge participation, with more than 1300 people interviewed (last updated May 2023). Since this project started to simplify the issues experienced by academic students, the interviewees were all under 30; in fact, about 98% of the respondents declared to be academic students or have obtained a degree in the last 5 years. Concerning gender, there was a majority of females, with a two-to-three ratio. The survey continued by asking something about their schooling, such as what kind of high school, university, and course he/she attended/is attending now, showing a very heterogeneous pool of answers; a relevant percentage of students experienced class failure in the past (about 18%). More than 60% of them were attending the first or second year; a relevant part, more than 17%, was attending the third year; the rest were in the fourth or fifth year, outside prescribed times, or already graduated students. More than 80% of them were full-time students, and even here, it can be found that about a one-to-six ratio were working students, as in the case of the Museum Questionnaire. The first part of the survey ended with some personal information, such as “When have you received diagnosis of SLD?”, where the majority answered during primary school, followed by the last three years of high school, middle school, and lastly, the first two years of high school; the observed trend shows an increase in the frequency of early diagnosis. Interviewees also declared that dyslexia was not the only one experienced, but four out of five people had at least another SLD issue. Almost half of them said that they had a minimum of one family member with the same problem, usually a parent or a sister/brother.

Going on, the survey asked the students which kinds of issues he/she encountered during their schooling so that, for this paper’s purpose, only the issues that could be relevant also during a museum visit were treated, while the rest of the answers could provide useful information for the problem analysis.

The most common issues were the following:

• Reading difficulties;
• Text comprehension difficulties;
• Concentration difficulty;
• Verbal short-term memory impairment;
• Verbal long-term memory impairment/

Finally, the supporting tools proposed to interviewees for compensating their impairments (and selected according to their viability within museums) are the following:

• Audiobook with a human voice;
• Words in different colors;
• Highlighted keywords;
• Clear layout of the study material;
• Use of images for the memorization and understanding of words;
• Use of images for the memorization of concepts;
• The integration of study materials using the internet.

Please refer to Zingoni et al. [38] for more information.

5. Discussion

In this paper, the conceptual design of a new access methodology to help dyslexic people during their museum visit was presented. This work took as input two questionnaires: the OLOS^® Questionnaire, which investigates the usage preferences of a museum’s visitor and how a museum should innovate its experience by means of immersive reality according to the interviewees, and the VRAIlexia one, which consists of the self-assessment of dyslexic students’ difficulties and needs experienced during their educational period. The functioning of OLOS^® was addressed in detail to provide a new proof of concept for the methodology that will lead to its final implementation. Due to the results obtained in the previous paragraph, the following solution will be proposed.

As seen in Paragraph 2, the European patent linked to OLOS^® methodology concerns access to a sensory involvement, which offers an experience that leads to relating with humans perceived as real without the use of visual or auditory filters. In this context, OLOS^® can create an extended reality where real and virtual blend together and emotions are at the center of the experience. In this regard, the choice of using the OLOS^® methodology seems to be a good starting point, in line with the interviewee’s preferences, as detected by the “OLOS^® Questionnaire” in Appendix A.3, where people underline the preference to interact with a human holographic character. On the one hand, as it is conceived, OLOS^® can show a real actor/actress who talks to the user with his/her real voice, which is an important feature, as highlighted by the VRAIlexia Questionnaire; in addition, students revealed the uselessness of a robotic voice. The holographic layer permits, firstly, to see the character in movement without any physical support and, secondly, to see the real background of the environment in which the artwork is placed. This methodology also includes a way to interact with the character that can be performed through the voice, namely a speech recognition system and another way of interaction through a touch screen device near the installation. The narration is divided into a series of parts that are no longer than a couple of minutes and can be replayed as often as the user deems necessary. Two important features that permit, firstly, to maintain a high concentration in relation to the necessity of OLOS^® to receive constant input to continue the storytelling and, secondly, to rehear the explanation if the visitors have lost some parts of the narration.

Therefore, following the principles of Universal Design for Learning, where it is stated that each individual learns in his or her own way [41], each user, whether they had SLD or not, will enjoy new, enriched storytelling with two kinds of content: some available in the main area next to the interactive life-size character so that new information can be visualized without having to look away from the narrator and others on a personal device (e.g., smartphone), which can be viewed between narratives, accessing a web-based application for quick and easy access to compensatory tools, all without needing an internet connection, as all content will be made available via a local connection to the server where the O^®EM (and thus OLOS^®) is located. These tools are the following:

• Images and 3D Objects, realized by scanning the real artworks present inside the museum that will appear and disappear during the narration and with which the actor/actress will interact in order to focus the attention of the user on every single characteristic of the artwork and, therefore, to facilitate the comprehension of its history;
• Keywords, appearing in real-time next to the interactive character to highlight the most important aspects of the speech;
• Concept Maps, a simple but at the same time powerful tool as underlined by the VRAIlexia Questionnaire on the user’s personal device in order to ensure the right learning experience. Of course, knowing how reading is a sensitive issue for people with specific learning disorders, the following measures will be taken:

  -

  Use of a dyslexic-friendly font,

  -

  Bold and/or colored keywords;

  -

  Images or synonyms to substitute the most complex words;

  -

  Doubled letters and diphthongs highlighted;

  -

  Opportunity to listen to the entire text.

• Hyperlinks, referring to interviews and background material on the topics covered during the storytelling of the interactive character to allow users to obtain further details on the topic;
• Glossaries, to clarify the meaning of the most difficult words used during storytelling and, if needed, substitute them with synonyms;
• AI-based algorithms, algorithms to understand the presence of users with specific learning disorders. Among others, a test based on the questionnaire realized within VRAIlexia could be used to understand the main useful additional functions for each user. In addition, a second approach can cover the use of image-based algorithms to understand the engagement and attention level of the users by analyzing posture and gaze during storytelling. In this regard, it is possible to verify if visitors have reached an appropriate level of comprehension, eventually suggesting assisted fruition;
• End-of-storytelling questionnaire, a test realized with a gamification design to ensure that the visitors have reached full knowledge of the proposed content.

In addition, should these guidelines be applied in a context other than OLOS^®, it is good to keep in mind some preferences emphasized in the questionnaire found in Section 4.1, such as (i) a virtual character with human likeness or in keeping with the narrative, (ii) interacting with the user in a standing position and at close range, (iii) allowing the user to interject in multiple ways (voice requests, input devices, body movements), (iv) in a museum-like or otherwise realistic environment, and (v) able to move within the environment without the need for supports.

An example of how the O^®EM system will logically handle some of the proposed tools can be seen in Figure 5, where they are shown grouped into tools that will be visible next to the virtual character (one-to-many) and tools that will be usable through a personal device (one-to-one).

Limits of the Research

In conclusion, while our questionnaire offers valuable insights into the experiences and perceptions of young adults with and without learning disabilities in Italy, some limitations must be acknowledged. These include potential sampling bias, self-reporting biases, and the challenge of fully capturing cross-cultural variability. Additionally, the scope of this study was limited to educational settings, leaving out other influential factors like family dynamics and socioeconomic status. Despite these limitations, the findings can still inform inclusive educational policies and support systems. However, caution should be exercised in generalizing the results, and policymakers should consider contextual factors when applying them. Moving forward, addressing these limitations will be crucial for enhancing the relevance and applicability of research on learning disabilities among young adults.

6. Conclusions

This work aimed to extend the capabilities of the patented OLOS^® methodology in such a way that it could be more supportive for people with SLDs. Firstly, a questionnaire in which preferences, difficulties, and desires in the museum environment were investigated was produced and was subsequently submitted to a significant number of people, whether they had an SLD certification or not. The results of this questionnaire were combined with those produced by the VRAIlexia project, in which many university students with SLD raised the major critical issues and the most useful tools and strategies they encountered during their studies.

The fusion of these outputs produced ideas on the actions to be performed to facilitate learning in the museum environment, such as simplified and digitized information material, new visual effects and the use of gamification during the entire experience. In more detail, these ideas encompass the following components: (i) images and 3D objects, (ii) keywords, (iii) concept maps, (iv) hyperlinks, (v) glossaries, (vi) AI-based algorithms, and (vii) End-of-storytelling questionnaires.

The next step certainly involves the implementation of these new tools in the contexts in which OLOS^® is already present, followed by a final survey to be submitted to museum visitors, which can confirm the validity of the choices made and suggest new ones.

7. Patents

This work is based on the European patent “Process for making an audio-visual interface”, with Daniele Baldacci listed as the inventor.