1. Introduction
Driving is a complex activity that requires various skills, such as fast reactions against environmental events or unpredicted risks. Driver training and driving safety awareness in such risky cases, or with special groups of drivers (e.g., racers), can be safely done with the use of driving simulators, usually supported by virtual reality environments and motion platforms. Driving simulators are convenient for different reasons, namely: safety and risk reduction; costs reduction; greater trial availability; they cause no damage for the drivers; availability of trainers and students; possibility of recreating a variety of situations and weather conditions; possibility of repeating the same tests under the same conditions; ability to evaluate tests objectively; and the transfer of skills and training [
1].
The research community has long reported on the use of driving simulators and driving-oriented virtual reality applications. Although, many of them aim at assessing or promoting driving safety, the environmental conditions, targeted users, simulators’ setups, and case studies are varied. For instance, in [
2] a study was conducted regarding the fatigue among aging drivers by making use of a fixed-base driving simulator, comprising a complete automobile, fully equipped with functional pedals and a dashboard. The developed virtual reality environment was projected on a large screen reproducing a drive on a monotonous country road. Another example is shown in [
3], where the targeted users are novice drivers with autism spectrum disorder. The tests were performed with a commercial simulator that displays a 210
field of view on a curved screen inside an eight-foot cylinder. The simulator includes a seatbelt, dashboard, steering wheel, turn signal, gas and brake controls, right, left, side, and rear-view mirrors, as well as an adjustable seat. In [
4] a study was carried out to predict motor vehicle collisions in young adults, by making use of a PC-based driving simulator. In [
5] the patterns followed by drivers in adjusting the speed in curves is studied by carrying various tests in a motion-based driving simulator. In [
6], how the simulator test conditions affect the severity of the simulator sickness symptoms was studied. For the tests, different simulator setups were considered, one of them considering a motion platform.
Most of the aforementioned works make use of advanced 3D graphics and sound; some also including motion platforms. However, many of them focus mainly on driving skills, not on safety awareness. In any case, the integration of other kind of multimodal stimuli, which might increase the user’s immersion, is usually not considered, and only a few works have been reported by the research community. In this regard, one of the earliest multimodal immersive systems was the Sensorama, a system patented in 1962 by Morton L. Heilig [
7]. The technology integrated in the Sensorama allowed a single person to see a stereoscopic film enhanced with seat motion, vibration, stereo sound, wind, and aromas, which were triggered during the film. Recent research works dealing with immersive multimodal systems are reported in [
8,
9], also involving individual experiences. On the other hand, the rapid technological advancements of the last years have allowed the development of commercial solutions that integrate a variety of multimodal displays in movie theatres, such as in [
10,
11,
12], where these systems are usually referred to as 4D or 5D cinemas or theatres. Some claim that this technology shifts the cinema experience from “watching the movie to almost living it” [
13], also enhancing the cinematic experience while creating a new and contemporary version of storytelling, which can be conceptualized as a “reboot cinema” [
14]. Although these systems are not centred in driving simulation, they are of interest because they include multimodal stimuli and account for a set of users rather for a single user.
In this paper, we present the outcomes of a usability test performed with ROMOT [
15], a RObotized 3D-MOvie Theatre involving multimodal and multiuser experiences, for a case study in driving safety awareness. ROMOT follows the concept of a 3D movie theatre with a robotized motion platform and integrated multimodal devices, also supporting some level of audience-film interaction. ROMOT was initially conceived to be the central attraction of a driving safety awareness exhibition. However, ROMOT has been designed as a multi-purpose 3D interactive theatre, since it is highly versatile as it is prepared to support different types of setups and contexts, including films/animations or even simulations that could be related to a variety of contents, such as learning, entertainment, tourism, or even driving safety awareness, as shown in this paper. Currently, ROMOT supports and integrates various setups to fulfil different needs: first-person movies, mixed reality environments, virtual reality interactive environments, and augmented reality mirror-based scenes. The contents of all of the different setups are based on a storytelling and are seen stereoscopically, so they can be broadly referred to as 3D movies, mainly showing virtual generated graphics.
Due to the fact that ROMOT is a laboratory system that has been built from scratch (both hardware and software), differently from other commercial systems, it is highly versatile, being easily adapted to different kinds of public, purposes, contents, setups, etc. This opens new avenues in research related to e.g., HCI, robotics, learning, perception, etc. Additionally, the costs of such a laboratory system are significantly lower of that of similar commercial systems.
Despite ROMOT could be perfectly used as a classical driving simulator adding the vehicle controls (thanks to its versatility), the setups shown here do not represent a skill-oriented driving simulator to study how people perform while driving, but a 3D interactive theatre with simulated content, designed specifically to enhance and improve driving safety awareness. So far, similar systems have not been used for the purpose of driving safety awareness.
This paper is focused on the development and initial user evaluation of ROMOT, and is organized in the following way. First, we show the main technical aspects behind the construction of ROMOT and the integrated multimodal devices and interaction capabilities. It is worth mentioning that, differently from other existing commercial solutions, ROMOT accounts for a 180 curved screen to enhance user immersion. Then, we show the different kind of setups and contents that were created for the case study on driving safety awareness. Finally, we show the first outcomes regarding the usability of the system and the individual’s satisfaction. To the best of our knowledge, this is the first work reporting audience experiences in such a complete system.
3. Results
In this section, we present the results of some preliminary user tests made at the laboratory level (
Figure 9). A total of 14 people tested the system and participated in its evaluation, which consisted on filling out a pair of questionnaires that were related to the usability of the system and the individual’s satisfaction, where the system usability scale (SUS) [
27] was chosen to measure usability. These kind of tests are commonly used to derive quantitative evaluation of technological systems regarding to usability, and SUS has become a standard. Some recent works using these tests can be found in [
28,
29,
30].
The participants were some of the research staff of the IRTIC Institute, where seven of them were women and seven were men. Only three of the participants contributed to the present research work at some of their stages. Though this might not be considered as a full objective audience, it can give us a good first notion for a user-related evaluation of the system.
The results of the SUS questionnaire are listed in
Table 2. In questions 1 to 10 the range 0–4 means: 0: strongly disagree, 4: strongly agree. The values of the SUS score, however, range from 0 to 100, meaning 100 is the best imaginable result. In the case of the ROMOT evaluation, this score reaches 84.25 points, which can be considered excellent on the scale of scores provided by the questionnaire and taking into account the fact that a minimum score of 68 would be deemed acceptable for a tool [
31,
32].
The results of the individuals’ satisfaction questionnaires are given in
Table 3. The scores also range from 0 to 4, meaning: 0: strongly disagree, 4: strongly agree. As it can be seen, results are quite satisfactory as eight out of the 12 mean scores are over 3 points and none are under 2.5 points. The lowest mean score belongs to question 10: “I didn’t feel sick after using the ROMOT”, which is an issue difficult to tackle, as simulator sickness highly depends on the individual. On the other hand, the highest mean score belongs to question 12: “I would like to recommend others to use ROMOT”, meaning that, overall, the individuals are satisfied with the system.
In addition to the tests, we have asked the participants to give us any other kind of feedback in a section dedicated to personal comments. We collected feedback from three of the participants. Two of them coincided in that the system was “very awesome” and they enjoyed the multimodal experience. The third one also addressed that the system provided him/her with an impressive experience and the sense of immersion was very high, as he/she could really feel like driving a car through the different scenarios and situations. However, he/she felt a little bit sick after a while, due to the combination of the stereoscopic glasses and the motion platform movements. Nevertheless, he/she still recommended the experience to others, as it was “very thrilling”.
4. Discussion and Further Work
As explained in the previous section, the preliminary quantitative evaluation of ROMOT related to usability was very satisfactory. After the tests were performed in the lab environment, our system was sent to a driving safety awareness exhibition in the Middle East. The exhibition consisted of a set of interactive applications, which included ROMOT as the central attraction (
Figure 10). ROMOT and the rest of applications were designed and implemented entirely by the IRTIC institute for the same purpose, i.e., to teach and train on basic driving safety rules and transfer awareness on the importance of driving safely. In this sense, the set of interactive applications can be considered as a single learning/training tool.
As a further work, we intend to evaluate the learning and training capabilities of the whole exhibition. In order to collect some quantitative indicators, the visitors of the exhibition are asked to fulfil an online questionnaire before (ex-ante) and after (ex-post) the visit. Additionally, we are measuring the time that each individual expends at each application, the individual performance and scores obtained at each application (if any), and the trail each individual follows in the exhibition (e.g., in what order they visit the applications, if they repeat the experience, etc.). We are able to collect these data as each individual is given a unique code at the ticket entrance, which is required to activate each of the applications.
From a technological standpoint, the authors also propose and suggest testing other possibilities, such as different types of stereoscopic displays, a 6-DOF motion platform, and explore other user interaction systems different from tactile tablets. However, these kinds of improvements could raise the cost of the solution.
5. Conclusions
In this paper, we have presented the construction and first user evaluation of ROMOT, a robotized 3D-movie theatre. The work shown in this paper relates to the enhancement of audience experiences when integrating multimodal stimuli and making it interactive. As well, we show the versatility of the system by means of the different kind of generated content.
Both the setups and the film content of ROMOT can be changed for different types of user experiences. Here we have shown different setups for content related to driving safety awareness, though other filmic contents could be used, including some related to learning, training, entertainment, etc. As for the different setups, we have shown a first-person movie and others related to the technologies of virtual, augmented and mixed realities.
The outcomes regarding the usability of the system and the individual’s satisfaction are very promising, though we are aware that the system has only been evaluated at the laboratory level. It is also worth mentioning that, although there exist different commercial solutions (e.g., 4D/5D cinemas), we have not found complete research works dealing with the construction and audience evaluation of such systems.
As a further work, we intend to evaluate the developed system in a real environment (the exhibition space) with the participation of hundreds of visitors. We would also like to create more film content in order to make use of ROMOT for other purposes.