Next Article in Journal
Generative Artificial Intelligence, Human Agency and the Future of Cultural Heritage
Previous Article in Journal
Correction: Walker et al. Winds of Change: Urgent Challenges and Emerging Opportunities in Submerged Prehistory, a Perspective from the North Sea. Heritage 2024, 7, 1947–1968
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Developing a Multitasking Augmented Reality Application for Theatrical and Cultural Content

1
Department of International and European Studies, University of Piraeus, 18534 Piraeus, Greece
2
Department of Telecommunications and Informatics, National and Kapodistrian University of Athens, 10672 Athens, Greece
3
Content Management in Culture P.C., 10680 Athens, Greece
4
Department of Business Administration, University of West Attica, 12243 Aigaleo, Greece
*
Author to whom correspondence should be addressed.
Heritage 2024, 7(7), 3578-3596; https://doi.org/10.3390/heritage7070169
Submission received: 8 May 2024 / Revised: 14 June 2024 / Accepted: 26 June 2024 / Published: 5 July 2024
(This article belongs to the Section Digital Heritage)

Abstract

:
Implementing digital technologies, such as augmented reality and 3D modeling, to various cultural sectors is an ongoing procedure, especially favorable after the COVID-19 pandemic. Even though such technologies are common to museums, galleries and archaeological places all over the world, little research or practical cases have been developed when it comes to theaters and theatrical costumes. The proposed research article describes the methodology of both reference material collection and application development, while an analysis of functions and aesthetic results are described, especially when it comes to 3D modeling. This paper’s aim is to reveal that theatrical operations (as part of the cultural sector) as well as structured parts (such as acts, performances and costumes) can be preserved and served to larger audiences via technological means, such as an augmented reality application.

1. Introduction

Theater is an ark, in which the historical information of human activity and culture from every era and period is preserved. In the eyes of the contemporary observer, researcher or creator, the everyday costume of a previous era becomes a historical costume. A theatrical act is a “slice” of life of the heroes who live and move on stage. In this sense, a theatrical act provides future generations with information relating to the whole spectrum of a person’s life. This includes one’s habits, education, life status, financial situation, job, interests, age, even one’s beliefs and ethics and a whole lot more.
The Costumart project (Figure 1) is part of the National Scope Action “RESEARCH-CREATE-INNOVATE” of the Operational Programme Competitiveness, Entrepreneurship and Innovation, co-funded by the European Regional Development Fund (ERDF) and national resources, under the NSRF 2014–2020 (code number Τ1ΕΔΚ-05070). It was funded to address some of these issues. The project was structured by a set of activities aiming to develop a set of applications based on augmented reality (AR) technologies and virtual reality (VR) tools. AR is an enhanced version of the real world, a kind of interactive environment based on reality, and this new version is achieved through the use of digital, visual, audio or other sensory means provided by real-time technology. VR aims to create simulated environments by using computer technology tools and embodying users in virtual worlds.
In order to achieve its goals, the project’s leaders took the initiative and completed the following set of actions and activities:
  • Activities in order to portray historical and urban costumes (digitization, recordings, etc.);
  • Creation of an infrastructure (storage systems, creation of ontologies, documentation dictionaries in three languages, etc.);
  • Creation of applications;
  • Publicity and communication activities (clothing display, offer of data documentation, virtual tour, storytelling);
  • Educational activities (educational applications and programs for specialists, for adults, for children, for schools, etc.);
  • Dictionary and index of costume terminology.
The whole project was created with the support of a Greek, well-known theater scholar, Mr. Nikos Kamtsis. Since the whole effort was part of a research program, which provided financial resources, the costumes produced fulfilled the needs of this research work and were not really used in the theater (they remained in the possession of the project). Even under this condition, costumes and theatrical representations were historically accurate following the theater scholar’s detailed instructions, who adopted principles coming from the pioneers in the creation of innovative costumes and stage design, such as Inigo Jones, Adolphe Appia, Edward Gordon Craig and Isadora Duncan [1].
The project created open ontologies for the costume/garment, its use, its materials and its way of production as a learning machine for everyone interested in theatrical costume from an educational, theatrical or research point of view. The target audience for all educational activities was a theatrical audience, comprising adults, young people and children as well as specialists from the theatrical sector. Even though this target audience seems generic, it should be taken into account that younger people are more eager to test and experiment with such applications, while an older audience is searching for information and knowledge through using them. By following such a framework, our research team tried to provide information as accurate as possible regarding theatrical scenes and costumes, while information regarding plays and historical information was kept at a minimum level to retain the application’s entertaining character as well. Learning objectives where mainly concentrated on providing historical information associated with the architecture of the theaters and the costumes to transfer the application’s end users to the culture and the atmosphere of the proposed historical period. This project addresses and presents, via a series of applications, the clothing and dress patterns of men and women coming from four major historical periods: (a) Ancient Greek, (b) Roman, (c) Elizabethan and (d) European years. The whole procedure was carried out in two ways, by two distinct applications:
  • The incorporation of augmented reality within the theatrical act—The activity involved pilot AR use in a short-term theatrical performance where a small number of users wearing special glasses watched a theatrical performance with real and virtual characters.
  • AR application via mobile devices/augmented reality glasses—This included the development of a special augmented reality application that was applied in a showroom of theatrical costumes. The visitors had the opportunity to “scan with a mobile device or via the AR headsets” exhibits using the application. What followed was 3D actor modeling or civilians (avatars) wearing the costumes illustrating the costumes’ historical use and cultural value.
The proposed AR application works as a natural exhibition space. The visitor is able to “scan with his/her own smartphone or via an AR headset” any exhibit, travelling them back in time through the application, 3D models of actors or/and citizens (avatars) who wear these clothes, while storytelling can accompany and explain (through a synthetic voice) the historical use and the cultural value of the content provided. Moreover, the abovementioned applications contain not only the theaters but also three-dimensional models (avatars) of the representation of clothes of each era, while a series of texts and audio for educational purposes was developed (providing information about the theatrical performances and the actors of each era). Moreover, alongside with the costume/garment and its theatrical use, the application’s users can enrich their knowledge on the various architectures related to the abovementioned historical periods. A series of 3D theaters were developed permitting end users to choose the appropriate clothes/building and to see well-known historical monuments be created, while there exists an operation permitting users to explore both buildings and costumes from various aspects by using their mobile phone.
The development of 3D modeling, clothes and buildings/monuments as well as the user’s interaction with the application, were all achieved by using well-known technologies for augmented reality (AR) such as the Unity tool and Vuforia. The application has been implemented for both mobile phones with android version 10 software and IOS version 14 so that any user has the opportunity to both experience this application but also learn through it.
The current paper aims to present the project’s main result and to broaden readers’ understanding about how augmented reality, 3D modeling and virtual reality can be used to enrich a pure cultural experience such as a theatrical act. In the following sections, a brief theoretical background—mainly focused on the technological aspects and their usage in cultural activities—will be provided, technological issues related to the proposed application will be explained and the project’s results (3D modeling) from an aesthetic and operational point of view will be presented.

2. Literature Review

Augmented reality (AR) technologies alongside virtual reality (VR) technologies have been developed over the last twenty years, with an expanding amount of their usage in several kinds of cultural organizations. More and more, different cultural sectors face a need or express a willingness to implement such technologies. Cultural organizations are using both VR and AR technologies in order to reach their goals in their physical places as well as in the digital world [2], including actions such as the following:
  • Expanding their presence to the digital world and making their content available to a mass audience in the global cultural market;
  • Reusing or distributing already existing digital content (moreover expanding their collections’ digitalization);
  • Connecting physical structures/artefacts with digital content (e.g., AR technologies) in order to provide their physical audience with a more interactive experience.
The use of AR technologies in various cultural organizations (e.g., museums, galleries, archaeological places) has gathered both theoretical and practical interest. Starting from the early 2000s, researchers have been interested in interactive technologies for museums [3,4], up to 2009, when Sylaiou et al. [5] compared various forms of emerging technologies, such as VR, AR and Web3D. In the fields of museums and archaeological places, a series of AR applications were developed, with more indicative examples being (a) the Augmented Reality-based Cultural Heritage On-site GUIDE (ARCHEOGUIDE) system [6], (b) an AR museum guide for internal navigation and data information provision on cultural artefacts [7], (c) Arbela Layers Uncovered (ALU), a mobile AR guide [8] using 3D modeling as well and (d) an AR application acting guide in the archaeological site of Knossos in Greece (called KnossosAR) [9].
Technological advances and price affordability for reaching and using a smartphone [10] led to further development of a series of applications, not only as a means for digitally replacing physical artefacts as was the case in the past [11], but to enrich users’ experience as a whole [12,13] via the artefacts’ information, storytelling experiences, etc. [14,15,16]. This led to the ability to access artefacts from different time periods in a realistic way and in a dynamic environment [17], leading to what can be called “Industry 4.0” for the cultural sector/organizations [2,18].
During the development stage, the project’s team used prior experience from other projects including the following: (a) issues related with 360° videos [19,20,21,22,23], (b) various pure AR issues [24,25,26], (c) issues related with 3D content and virtual reality [27,28,29,30] and (d) theater and the implementation of advanced technologies [31,32,33,34].
As far as the theater is concerned, there is a variety of works exploring the relationship between technology and live performances [35,36,37,38,39]. Some researchers put emphasis on audience participation via technological means, a framework that increases engagement, interaction and reshapes the cultural content in real time [37]. Cerratto-Pargman et al. [36] developed a mobile application that permits audiences to choose the outcome of a production, Lee et al. [40] introduced such an application to music concerts and Rostani [41] expanded the same concept to interactive performances in general. These researchers provided indications about the innovativeness and the capabilities that AR applications can have when it comes to live performances, including the theater.
However, existing research results indicate that technologies such as the ones described previously still remain a challenge when it comes to theatrical organizations, rather than full operational systems [1]. Even though, in the forthcoming future, such technologies are expected to find practical use, for the time being, there are only few research efforts taking place. However, most of them are not using these technologies as a means to promote and preserve in a digital way, the existing living (intangible) theatrical culture. Therefore, the main research scope of the current research contribution is to empirically, if feasible, preserve and promote contemporary culture as expressed by theatrical costume through synchronous technologies of sound, color, video and augmented reality.

3. Materials and Methods

As mentioned before, two distinct technological tools (the Unity tool and Vuforia) were used, while the most interesting part was achieving the appropriate collaboration between them in order to reach the best result, from an operational and aesthetic point of view. Moreover, knowledge and skills were needed to understand the environment of Unity and how this environment translated/appeared not only into a computer but also into a mobile phone. Using Unity, a widely known 3D modeling suite, provided the advantage of the existence of a global community of developers with whom interaction may exist. Meanwhile, using Unity also ensures the viability of the project’s results, since it is a technological tool wide used and difficult to be abandoned.. Such conditions ensured both the preservation and dissemination of the project, while challenges occurred from limitations that the technological platform had itself.
Initial tests of the application were conducted in a computerized environment, to reach a no-error status, and then the corresponding procedure was conducted οn a mobile phone environment to perform the test on a smaller screen but also in different versions of both android and IOS. There were some problems with the correct movement of the characters on different screens which could not have been solved without prior knowledge. The developers also had to know which costume relates to which theater so as not to transmit incorrect information to users. Also, historical knowledge was needed in order to create the right texts for each character but also the right sound files.
The main purpose of this application is not only for each user to be able to see through a game the architecture of the buildings of each era (Ancient Greek, Roman, Elizabethan, European) but also to learn what clothes the actors wore in each ancient theater and to get updated about some theatrical performances that take place nowadays so that they can go and watch them. As mentioned above, many modern technologies have been used to create this application which will be analyzed below (Figure 2). First of all, the Unity tool is the environment in which the application was implemented and the tool through which the application was converted to android and IOS, so that it could be compatible with mobile phones. It is also the tool in which all the three-dimensional models of the application are located/processed.
The other tool is Vuforia, through which the application can scan a specially formatted image and based on this information, start the game for the user (Figure 3). The basic function of this tool is that when an image is lost from the camera of the mobile, then the game stops, as all the 3D modeling is created on this image. Also, when the image is found, the user can interact with the game, see and learn everything she/he wants.
3D technologies have also been used to create the three-dimensional models of both the theaters and the clothes of each actor (Figure 4) as well as the dressing room where the actor enters to wear the right clothes. At the current stage of the project, this phase just aims to familiarize end users with the dress code and costumes of each historical period. At a later stage, when the database is enriched with theatrical costumes, more value, added experiences will be offered.
Even for the movement of each character to go get dressed, movement technologies have been used to make them look like they are walking like real people (Figure 5). To achieve the result presented in the above figure, many techniques—related with specific operations—were used. The first operation was the actors’ walking, where a technique called timeline was used so that the walking of the actor could be more realistic. Emphasis was put on realism and on the experience gained from end users even though simple actions such as moving their character across the theatrical stage.
Furthermore, in the beginning, as can be seen in Figure 4, the characters are not wearing their costumes. The user has to find the correct actor to begin the process and make the actor move. When the actor enters the dressing room, there is an invisible collider, which starts another process and the actor puts on the correct costume. Each time the collider is activated, a new piece of cloth (part of the costume as a whole) is added, until everything is on. Moreover, the user has to choose the right theatrical environment, among four distinct 3D theaters representing different periods of time (Figure 6).
Associating time (a historical period) with the right costume and appropriate stage (environment) is this ambitious project’s goal, since it educates end users in cultural and historical trends often used in theater. Moreover, at a later stage, when more content will be added, these associations will be based on theatrical plays and not on historical periods. The project will provide the tools to educate end users on theatrical culture and the unique artistic value of relating theatrical plays to times, costumes and stages.
To facilitate end users, techniques were adopted to enlarge the theater (zoom button) but also activate and deactivate the panel with the written information for them to see costumes and frocks more easily. Since the aim was to educate end users in recognizing digital content and to associate it with specific theatrical plays, an environment of accessing information was developed. The more the content will be enriched, the more research will be needed from the end users’ part in order to avoid guessing and pass to the next phase.

4. Results

The proposed application incorporated four distinct theaters and four actors, each one coming from a different time period of the theater’s history. These 3D models were reused in each application and educational game developed. Moreover, a fifth character/avatar was developed, taking the figure of a clown and acting as a “conferacie”. Apart from 3D theaters and 3D actors—avatars (including 3D costumes), a third category of 3D modeling was developed, included a series of 3D objects (props) of either wearables from actors or objects used as part of the play.
In order to achieve the quality needed for the development of 3D modeling, extensive research was conducted. Research included finding and collecting the appropriate reference materials, coming from pictures, architectural drawings, videos, etc., while the most valuable source, related with 3D theaters/buildings, were architectural drawings (floor plans, facades and sections). Moreover, photos and videos were extensively useful when information was needed regarding construction materials (wood, stone, plaster, etc.) that were used. An example related to the Globe Theatre is given in Figure 7.
For the whole research team, it was essential to incorporate valid historical information to enrich end users’ experiences and to provide as much detail as possible. Since costumes may vary from theatrical play to theatrical play, the current stage of the project’s emphasis (in terms of validity) was put on theatrical stages as the main element that remained stable. Especially when it comes to the Globe Theatre, there were a lot of reference materials as a result of the recent restoration of the theater, which provided a treasure trove of information for researchers, such as the roof of the theater (Figure 8).
At a second stage, all reference material was evaluated and the most accurate information was used to develop 3D modeling. Architectural drawings, drawings and photos were imported to Autodesk Maya (Figure 9), a 3D graphic and motion design software. By using this material and existing information as a base of the design, a series of software techniques were applied until an appropriate representation of the building was constructed. Such techniques included (indicatively) the following: extrusion, revolution, lofting, Boolean functions, etc. Transitioning from physical content to 3D modeling is a complex activity, especially when it comes to cultural content. Details and historical accuracy are more appreciated by theatrical stakeholders rather than programming details that are related with operations such as the movement of avatars and avoiding bugs.
The third stage of development included the creation of the 2D parts that would be used to create the 3D models (Figure 10), a procedure known as UV mapping. At this stage, the two-dimensional development of the model was created which would be used to “clothe” the 3D model with textures, providing color information (albedo map), dielectric gloss (metal roughness map) and details of various anomalies (normal map). Success during this stage meant achieving the model with the smallest degree of distortions and with the least possible notches in the joints between the triangles. Even though this procedure may be known to most programmers, audiences from other fields such as theatrical stakeholders are now discovering a new world when it comes to understanding the 3D modeling procedure.
At the next stage, emphasis was put on the building’s texture, so that there existed a realistic resemblance to the “materials” that were used to build the physical building. The most accurate techniques involved the development of the abovementioned 3D pictures (albedo map, metal roughness map, normal Map). The developed model was implemented through the specialized software of Adobe Substance Painter version 7.2 (Figure 11), using information gathered during the first stage of the development, while the software’s libraries permitted the effective imitation of the properties of natural materials (in terms of how light interacts) in the 3D modeling.
Finally, the model and the special images mentioned above were inserted in the game machine, which took its final display on mobile devices. The game machine determined a series of parameters, for example, the detection properties of the various materials as an obstacle to the avatars’ movement, etc. During this stage, programmers evaluated how the various 3D models operated when implemented on mobile devices (e.g., smartphones). Optimizations were made in the models and the special images in order to ensure the satisfactory performance rate (frame rate) of the application.
As far as three out of four theaters (Figure 12, Figure 13, Figure 14 and Figure 15) are concerned (Globe Theatre, Ancient Greek theater and Roman theater), there were no problems regarding the frame rate when the proposed 3D models were implemented on mobile devices. The European theater, having hundreds of seats, seemed to pose a problematic issue. Based on the fact that its seats are similar to each other, this made it possible to use an algorithm called “GPU instancing” to keep the frame rate to an accepted level, without any detriments to the realism of the representation.
For the development of the 3D actors, the same abovementioned steps were followed, including the collection of reference material, construction of the three-dimensional models, creation of their extensions, creation of the materials and implementation in the game machine. Two more stages were added, including the production of a model with a high number of triangles (HighPoly) and the addition of animation. Based on the fact that mobile applications must operate at a satisfactory rate of performance, modern 3D graphic design techniques require the construction of a second identical 3D model with a large number of triangles in order to represent details and irregularities of the physical object. This second model (HighPoly) was used so that these details were captured in the two-dimensional special image (NormalMap) and to create the illusion of their existence (Figure 16).
For the development of the HighPoly costumes, the Marvelous Designer software version 10 was used. The proposed software is widely used in the video game industry, where the representation of clothing is particularly important. Its main characteristic is that it uses the design of the pattern of clothes and through simulation applies it to the character imitating the folds that are created. For example, in the following Figure 17, the pattern on the right and the simulated model on the left can be seen. The correspondence of the pattern pieces in the model is marked in yellow.
For the needs of the game, the corresponding characters were also created wearing simple everyday clothes. The manufacturing process remained the same, but they were standalone models due to the fact that most parts of the body were covered by fabric. Additionally, the characters acquired special properties (rigging) that allowed them to move in space (animation). These properties were given in the 3D graphics and motion design software but were controlled by the game machine. For each character, an internal skeleton was created from the joints and bones (rig), which followed a logical hierarchy (parent–child hierarchy), for example, the shoulder–arm–hand–fingers (Figure 18 and Figure 19). The latter procedure is called weighting or skinning and aimed to move the model’s points in space by turning only the individual bones.
Complementary items are an important part of the app as the user has to choose the right combination of theater–costume–items to complete the game. For each character, two types of objects were created. The Ancient Greek actor is accompanied by a mask and quills, the Elizabethan and the European actors by a hat and shoes, and the Roman by a laurel wreath and sandals. Their differences in relation to the previous types of models lie in the fact that in their final stage, they must be inserted into the skeleton hierarchy and additionally, regarding the shoes, they must acquire rigging and weighting properties to follow the deformations of the foot and ankle.
In order to have a more detailed analysis, the research team conducted an evaluation of the proposed application during the period from 24 to 26 of May 2024. During this period, the research team reached out to the audience of a specific theatrical play, asking them to use the AR application and evaluate it. An online evaluation questionnaire was provided, which users of the application filled via tablets. At the end of these 3 days, a total of 63 fully answered questionnaires were collected, while there were another three partly completed and therefore excluded from the analysis. The mean age of participants was 35.8 years old, middle-aged το 35.8, while 61.9% of the answers come from females (39 females) and 38.1% from males (24 males). Moreover, all respondents had a smartphone, 68.25% (43 persons) answered that they were familiar with VR and AR technologies, 23.81% (15 persons) were somewhat familiar and 7.94% (5 persons) were not familiar at all.
Since the application resembles a digital game, a Game Experience Questionnaire (GEQ) was used to measure the following dimensions:
  • Competence describing how users evaluated their performance regarding the application’s goals;
  • Sensory and imaginative immersion providing insight into the degree to which users felt connected with the application;
  • Flow describing how fast the time passed when using the application;
  • Tension/annoyance reflecting the degree to which users felt any of these two emotions;
  • Challenge describing how challenging the application was for users;
  • Negative affect reflecting a negative emotional experience from using the application;
  • Positive affect reflecting a positive emotional experience from using the application.
In order to quantify these seven dimensions, a set of 33 statements related with users’ experiences were set, while a five-point Likert scale was used, with the scale ranging from 1 (“Do not agree”) to 5 (“Completely agree”). The questionnaire as a whole and its dimensions were developed from Ijsselsteijn et al. [42] and were averaged to present the mean GEQ scores for each dimension, as presented in Table 1.
In order to better understand the results, the research team conducted non-structured interviews of respondents who agreed to take place in such a procedure. These interviews aimed to illuminate users’ personal opinions regarding the application’s strengths and weaknesses and potential suggestions for new features as well as their reasoning for their evaluations. In the end, 12 interviews were conducted (almost 19% of the application’s users).
The results of both the application’s evaluation and interviews revealed a strong relationship between users’ satisfaction and the digital app with cultural content. Their evaluations and answers indicated that the application created a strong and positive emotional experience (mean 4.302—Table 1), mainly coming from the fact that AR apps are not something that is common in a theatrical content. Most users were positively impressed by the perspectives of implementing AR solutions in theatrical scenes and the ease of use of the application (e.g., just downloading an app). Moreover, most respondents sensed a connection with the proposed application (mean 4.143—Table 1), but their interviews’ answers mainly indicated that such a connection was mainly coming from their interest in the app’s content (theatrical costumes and scenes) and not from the app’s technological features. Such a result is also supported by the app’s flow (mean 3.492—Table 1), which indicates that time passed easily when using it. Most users pointed out that they spent time observing the 3D costumes and theaters, having the opportunity to focus on details rather than proceeding to the app’s next phases.
Moreover, users evaluated how high their performance was as far as the application’s goals were concerned (mean 3.984—Table 1), a result supported from the fact that the app had an average challenging orientation (mean 3.016—Table 1). Since the application is more educational rather than game-oriented, the abovementioned results were expected, while respondents were not disappointed by the challenge set by the application or their effort to achieve its goals. There were respondents who expressed their willingness to see more game-oriented elements, such as “treasure hunting” or “puzzle-solving challenges and knowledge quizzes”, all taking into account the theatrical content and the information provided.
Finally, as far as negative emotions are concerned, the results indicate a rather low degree (mean 1.714—Table 1), supporting results from previous research [43]. Respondents declared that since they had no previous experience using AR applications with theatrical content, the whole effort (as expressed from the proposed app) mainly created positive emotions. Only a single older respondent expressed difficulty in effectively accessing the cultural content from his smartphone, since “everything on the screen was too small”. The abovementioned result is also supported by the fact that tension/annoyance was also rather small (mean 1.492—Table 1).

5. Discussion and Conclusions

Based on the in-depth analysis of the project’s results, an initial conclusion is that by using this application, users can enrich their knowledge regarding the clothing and architecture of four different eras of antiquity (Ancient Greek, Roman, Elizabethan, European) through play and interaction with an environment. All of these can be achieved via their mobile phone, without having to open a computer to see the costumes, masks or theaters, since all the information is in the palm of their hand.
There were four scenarios for each theater of each historical period but in each scenario, there were small variations, useful to understand the different options that the user could choose and the different things they could learn about the theaters of each season. By scanning a play’s poster via its mobile phone, the app’s user can reach the above—mentioned content and information, enriching his knowledge and understanding about various historical periods. Modern augmented reality technologies and programming skills were used, as well as a lot of research to create three-dimensional models of clothes, theaters, shoes and accessories, without which the knowledge derived could not have led to the implementation of either of these items or the application itself.
The proposed application can be used in many different fields. Initially, it can be used as a game where students from schools will be able to visit a theater and not only see a play but moreover through augmented reality will be able to have the theater in the palm of their hand and enlarge it to see and to learn all the details required. They can also see the costumes of each character and learn what they wore in each historical period, but in a much more modern and interesting way than just seeing a garment, a shoe or a mask in a shop window.
In general, the proposed application seems to create a dynamic environment, capable of delivering information that is easier for end users to understand, achieving a higher degree of implicit learning, as has happened with various such applications [44]. However, an issue remains of how such applications can be directly associated with social media in terms of further disseminating any project’s social impact [45]. Finally, it should be mentioned that an important limitation and future research goal is that no user evaluation of the experience was conducted to discover in a qualitative and quantitative way the project’s impact on end users.
It should also be noted that such technologies have barely been used in other kind of industries, such as healthcare and fitness, where wearable gadgets are in common use [46]. It seems that the abovementioned kind of technologies are not yet integrated in mainstream theater [1]. Such a limitation derives from the fact that theatrical institutions may be reluctant to adopt technologies that may change the experience of a theatrical act. The anxiety of losing the collective experience of a live performance plays a crucial role, alongside changing the role of a theatrical stage as a public space to a humanitarian, social and political forum [47].

Author Contributions

Conceptualization, I.M., G.K., G.G., G.L., A.K. and D.V.; methodology, I.M., G.K., G.G. and G.L.; software, G.K. and G.G.; validation, G.K., G.G. and G.L.; writing—original draft preparation, I.M., G.K., G.G. and A.K.; writing—review and editing, I.M. and A.K.; visualization, G.K. and G.G.; supervision, D.V.; project administration, G.L.; funding acquisition, G.L., A.K. and D.V. All authors have read and agreed to the published version of the manuscript.

Funding

The project is part of the National Scope Action “RESEARCH-CREATE-INNOVATE” of the Operational Programme Competitiveness, Entrepreneurship and Innovation, co-funded by the European Regional Development Fund (ERDF) and national resources, under the NSRF 2014–2020 (code number Τ1ΕΔΚ-05070).

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Panagou, M.; Kamtsis, N.; Gourgoulianni, S. The impact of new technologies on theater and costume. Int. J. Sci. Acad. Res. 2021, 2, 3122–3130. [Google Scholar]
  2. Loumos, G.; Kargas, A.; Varoutas, D. Augmented and Virtual Reality Technologies in cultural Sector: Exploring their Usefulness and the Perceived Ease of Use. J. Media Crit. 2018, 4, 307–322. [Google Scholar]
  3. Brown, B.; Maccoll, I.; Chalmers, M.; Galani, A. Lessons from the Lighthouse: Collaboration in a shared mixed reality system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Fort Lauderdale, FL, USA, 5–10 April 2003; pp. 577–584. Available online: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.2438 (accessed on 15 June 2023).
  4. Grinter, R.E.; Aoki, P.M.; Szymanski, M.H.; Thornton, J.D.; Woodruff, A.; Hurst, A. Revisiting the visit: Understanding how technology can shape the museum visit. In Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work—CSCW ’02, New Orleans, LA, USA, 16–20 November 2002; p. 146. [Google Scholar] [CrossRef]
  5. Sylaiou, S.; Liarokapis, F.; Kotsakis, K.; Patias, P. Virtual museums, a survey and some issues for consideration. J. Cult. Herit. 2009, 10, 520–528. [Google Scholar] [CrossRef]
  6. Vlahakis, V.; Karigiannis, J.; Tsotros, M.; Gounaris, M.; Almeida, L.; Stricker, D.; Gleue, T.; Christou, I.T.; Carlucci, R.; Ioannidis, N. ARCHEOGUIDE: First results of an augmented reality, mobile computing system in cultural heritage sites. In Proceedings of the International Symposium on Virtual Reality, Archaeology and Cultural Heritage, Glyfada, Greece, 28–30 November 2001; pp. 131–139. Available online: https://doi.org/10.1145/584993.585015 (accessed on 10 October 2023).
  7. Miyashita, T.; Meier, P.; Tachikawa, T.; Orlic, S.; Eble, T.; Scholz, V.; Gapel, A.; Gerl, O.; Arnaudov, S.; Lieberknecht, S. An Augmented Reality museum guide. In Proceedings of the 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK, 15–18 September 2008; pp. 103–106. [Google Scholar] [CrossRef]
  8. Mohammed-Amin, R.K.; Levy, R.M.; Boyd, J.E. Mobile augmented reality for interpretation of archaeological sites. In Proceedings of the Second International ACM Workshop on Personalized Access to Cultural Heritage—PATCH ’12, Nara, Japan, 2 November 2012; p. 11. [Google Scholar] [CrossRef]
  9. Galatis, P.; Gavalas, D.; Kasapakis, V.; Pantziou, G.; Zaroliagis, C. Mobile Augmented Reality Guides in Cultural Heritage. In Proceedings of the 8th EAI International Conference on Mobile Computing, Applications and Services, Cambridge, UK, 30 November–1 December 2016; pp. 11–19. Available online: https://github.com/phishman3579/android-augment-reality-framework (accessed on 10 October 2023).
  10. Kim, S.L.; Suk, H.J.; Kang, J.H.; Jung, J.M.; Laine, T.H.; Westlin, J. Using Unity 3D to facilitate mobile augmented reality game development. In Proceedings of the 2014 IEEE World Forum on Internet of Things (WF-IoT), Seoul, Republic of Korea, 6–8 March 2014; pp. 21–26. [Google Scholar] [CrossRef]
  11. Novitski, B.J. Rendering Real and Imagined Buildings: The Art of Computer Modeling from the Palace of Kublai Khan to Le Corbusier’s Villas; Rockport Publishers: Beverly, MA, USA, 1998. [Google Scholar]
  12. Karafotias, G.; Gkourdoglou, G.; Maroglou, C.; Koliniatis, C.; Loumos, G.; Kargas, A.; Varoutas, D. Developing VR applications for cultural heritage to enrich users’ experience: The case of Digital Routes in Greek History’s Paths (RoGH project). Int. J. Cult. Herit. 2022, 7, 32–53. [Google Scholar]
  13. Kargas, A.; Loumos, G.; Varoutas, D. Using Different Ways of 3D Reconstruction of Historical Cities for Gaming Purposes: The Case Study of Nafplio. Heritage 2019, 2, 1799–1811. [Google Scholar] [CrossRef]
  14. Vayanou, M.; Loumos, G.; Kargas, A.; Kakaletris, G. The IIIσETO project: Storytelling games for groups of visitors in fine art exhibitions. In CEUR Workshop Proceedings; Technical University of Aachen: Aachen, Germany, 2019; p. 2412. [Google Scholar]
  15. Vayanou, M.; Ioannidis, Y.; Loumos, G.; Kargas, A. How to play storytelling games with masterpieces: From art galleries to hybrid board games. J. Comput. Educ. 2019, 6, 79–116. [Google Scholar] [CrossRef]
  16. Vayanou, M.; Ioannidis, Y.; Loumos, G.; Sidiropoulou, O.; Kargas, A. Designing Performative, Gamified Cultural Experiences for Groups. In Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems—CHI EA ’19, Glasgow, UK, 4–9 May 2019; pp. 1–6. [Google Scholar] [CrossRef]
  17. Ch’ng, E. The Mirror Between Two Worlds: 3D Surface Computing Interaction for Digital Objects and Environments. In Digital Media and Technologies for Virtual Artistic Spaces; IGI Global: Hershey, PA, USA, 2013. [Google Scholar] [CrossRef]
  18. Kargas, A.; Varoutas, D. Industry 4.0 in Cultural Industry: A Review on Digital Visualization for VR and AR Applications. In Impact of Industry 4.0 on Architecture and Cultural Heritage; Bolognesi, C.M., Cettina, S., Eds.; IGI Global: Hershey, PA, USA, 2020; pp. 1–19. [Google Scholar] [CrossRef]
  19. De Cicco, L.; Mascolo, S.; Palmisano, V.; Ribezzo, G. Reducing the network bandwidth requirements for 360° immersive video streaming. Internet Technol. Lett. 2019, 2, e118. [Google Scholar] [CrossRef]
  20. Liu, S.J.; Agrawala, M.; DiVerdi, S.; Hertzmann, A. View-dependent video textures for 360° video. In Proceedings of the UIST 2019—Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, New Orleans, LA, USA, 20–23 October 2019; pp. 249–262. [Google Scholar] [CrossRef]
  21. Qian, F.; Han, B.; Ji, L.; Gopalakrishnan, V. Optimizing 360 video delivery over cellular networks. In Proceedings of the Annual International Conference on Mobile Computing and Networking, MOBICOM, New York, NY, USA, 3–7 October 2016; pp. 1–6. [Google Scholar] [CrossRef]
  22. Yen, S.C.; Fan, C.L.; Hsu, C.H. Streaming 360° videos to head-mounted virtual reality using DASH over QUIC transport protocol. In Proceedings of the 24th ACM Workshop on Packet Video, PV, Amherst, MA, USA, 21 June 2019; pp. 7–12. [Google Scholar] [CrossRef]
  23. Yi, J.; Luo, S.; Yan, Z. A measurement study of YouTube 360° live video streaming. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV, Amherst, MA, USA, 21 June 2019; pp. 49–54. [Google Scholar] [CrossRef]
  24. Barrow, J.; Forker, C.; Sands, A.; O’Hare, D.; Hurst, W. Augmented Reality for Enhancing Life Science Education—The University of Aberdeen Research Portal. Medical Education. 2019. Available online: https://abdn.pure.elsevier.com/en/publications/augmented-reality-for-enhancing-life-science-education (accessed on 10 October 2023).
  25. Ibáñez, M.B.; Delgado-Kloos, C. Augmented reality for STEM learning: A systematic review. Comput. Educ. 2018, 123, 109–123. [Google Scholar] [CrossRef]
  26. Pratt, P.; Ives, M.; Lawton, G.; Simmons, J.; Radev, N.; Spyropoulou, L.; Amiras, D. Through the HoloLensTM looking glass: Augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels. Eur. Radiol. Exp. 2018, 2, 2. [Google Scholar] [CrossRef] [PubMed]
  27. El-Ganainy, T.; Hefeeda, M. Streaming Virtual Reality Content|Semantic Scholar. Computer Science. 2016. Available online: https://www.semanticscholar.org/paper/Streaming-Virtual-Reality-Content-El-Ganainy-Hefeeda/0fb740da32307f4a960172eebba2efe4625dabd7 (accessed on 10 October 2023).
  28. LaValle, S. Virtual Reality; Cambridge University Press: Cambridge, UK, 2019; Available online: http://lavalle.pl/vr/ (accessed on 10 October 2023).
  29. Men, L.; Bryan-Kinns, N.; Hassard, A.S.; Ma, Z. The impact of transitions on user experience in virtual reality. In Proceedings of the IEEE Virtual Reality, Los Angeles, CA, USA, 18–22 March 2017; pp. 285–286. [Google Scholar] [CrossRef]
  30. Petry, B.; Huber, J. Towards effective interaction with omnidirectional videos using immersive virtual reality headsets. ACM Int. Conf. Proc. Ser. 2015, 11, 217–218. [Google Scholar] [CrossRef]
  31. AEA Consulting. AEA Consulting—Insights—From Live-to-Digital: Understanding the Impact of Digital Developments in Theatre on Audiences, Production and Distribution. 2022. Available online: https://aeaconsulting.com/insights/from_livetodigital_understanding_the_impact_of_digital_developments_in_theatre_on_audiences_production_and_distribution (accessed on 10 October 2023).
  32. Bertola, P.; Teunissen, J. Fashion 4.0. Innovating fashion industry through digital transformation. Res. J. Text. Appar. 2018, 22, 352–369. [Google Scholar] [CrossRef]
  33. Pope, V.C.; Dawes, R.; Schweiger, F.; Sheikh, A. The geometry of storytelling: Theatrical use of space for 360-degree videos and virtual reality. In Proceedings of the Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; pp. 4468–4478. [Google Scholar] [CrossRef]
  34. Rajan, R. Beyond Broadway: Connecting Musical Theater and Academic Learning. Child. Educ. 2016, 92, 118–125. [Google Scholar] [CrossRef]
  35. Barkhuus, L.; Rossitto, C. Acting with technology: Rehearsing for mixed-media live performances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; pp. 864–875. [Google Scholar]
  36. Cerratto-Pargman, T.; Rossitto, C.; Barkhuus, L. Understanding audience participation in an interactive theater performance. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, Helsinki, Finland, 26–30 October 2014; pp. 608–617. [Google Scholar]
  37. Cordeiro, J.; Martins de Abreu, F.; Estadieu, G. Audience Participation in Interactive Art Systems: Is Instructional Signage a Necessary Evil? In Proceedings of the 8th International Conference on Digital Arts, Macau, China, 6–8 September 2017; pp. 31–37. [Google Scholar]
  38. Friederichs-Büttner, G.; Walther-Franks, B.; Malaka, R. An unfinished drama: Designing participation for the theatrical dance performance Parcival XX-XI. In Proceedings of the Designing Interactive Systems Conference, Newcastle upon Tyne, UK, 11–15 June 2012; pp. 770–780. [Google Scholar]
  39. Owen, C.; Dobbins, A.; Rebenitsch, L. Theatre Engine: Integrating mobile devices with live theater. In Proceedings of the International Conference on Advances in Mobile Computing & Multimedia, Vienna, Austria, 2–4 December 2013; pp. 378–386. [Google Scholar]
  40. Lee, S.W.; Willette, A.; Koutra, D.; Lasecki, W.S. The effect of social interaction on facilitating audience participation in a live music performance. In Proceedings of the 2019 on Creativity and Cognition, San Diego, CA, USA, 23–26 June 2019; pp. 108–120. [Google Scholar]
  41. Rostami, A.; McMillan, D.; Márquez Segura, E.; Rossito, C.; Barkhuus, L. Bio-sensed and embodied participation in interactive performance. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction, Yokohama, Japan, 20–23 March 2017; pp. 197–208. [Google Scholar]
  42. IJsselsteijn, W.A.; de Kort, Y.A.W.; Poels, K. The Game Experience Questionnaire; Technische Universiteit: Eindhoven, The Netherlands, 2013. [Google Scholar]
  43. Johnson, D.M.; Gardner, M.J.; Perry, R. Validation of two game experience scales: The Player Experience of Need Satisfaction (PENS) and Game Experience Questionnaire (GEQ). Int. J. Hum. Comput. Stud. 2018, 118, 38–46. [Google Scholar] [CrossRef]
  44. Kargas, A.; Loumos, G.; Mamakou, I.; Varoutas, D. Digital Routes in Greek History’s paths. Heritage 2022, 5, 742–755. [Google Scholar] [CrossRef]
  45. Kargas, A.; Karitsioti, N.; Loumos, G. Reinventing Museums in 21st Century: Implementing Augmented Reality and Virtual Reality Technologies Alongside Social Media’s Logics. In Virtual and Augmented Reality in Education, Art, and Museums; Guazzaroni, G., Pillai, A.S., Eds.; IGI Global: Hershey, PA, USA, 2019; pp. 117–138. [Google Scholar] [CrossRef]
  46. Honauer, M.; Hornecker, E. Challenges for Creating and Staging Interactive Costumes for the Theatre Stage. In Proceedings of the C&C ’15: Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition, Glasgow, UK, 22–25 June 2015; Association for Computing Machinery: New York, NY, USA, 2015; pp. 13–22. [Google Scholar]
  47. Burgheim, J. Live Performances in Digital Times: An Overview; IETM—International Network for Contemporary Performing Arts: Brussel, Belgium, 2016. [Google Scholar]
Figure 1. The project’s logo.
Figure 1. The project’s logo.
Heritage 07 00169 g001
Figure 2. The project’s/applications’ architecture.
Figure 2. The project’s/applications’ architecture.
Heritage 07 00169 g002
Figure 3. How the game starts.
Figure 3. How the game starts.
Heritage 07 00169 g003
Figure 4. Characters before dressing up.
Figure 4. Characters before dressing up.
Heritage 07 00169 g004
Figure 5. Movement of characters and dressing room curtain.
Figure 5. Movement of characters and dressing room curtain.
Heritage 07 00169 g005
Figure 6. Choosing the 3D theater.
Figure 6. Choosing the 3D theater.
Heritage 07 00169 g006
Figure 7. Indicative architecture and hypothetical representation of the Globe Theatre.
Figure 7. Indicative architecture and hypothetical representation of the Globe Theatre.
Heritage 07 00169 g007
Figure 8. The roof of the Globe Theatre.
Figure 8. The roof of the Globe Theatre.
Heritage 07 00169 g008
Figure 9. Developing the Globe Theatre using Autodesk Maya software version 2020.4.
Figure 9. Developing the Globe Theatre using Autodesk Maya software version 2020.4.
Heritage 07 00169 g009
Figure 10. Models used and building’s development by the proposed models.
Figure 10. Models used and building’s development by the proposed models.
Heritage 07 00169 g010
Figure 11. The group of models in the previous image after applying the materials.
Figure 11. The group of models in the previous image after applying the materials.
Heritage 07 00169 g011
Figure 12. Globe Theatre in Unity 3D environment.
Figure 12. Globe Theatre in Unity 3D environment.
Heritage 07 00169 g012
Figure 13. Ancient Greek theater in Unity 3D environment.
Figure 13. Ancient Greek theater in Unity 3D environment.
Heritage 07 00169 g013
Figure 14. European theater in Unity 3D environment.
Figure 14. European theater in Unity 3D environment.
Heritage 07 00169 g014
Figure 15. Roman theater in Unity 3D environment.
Figure 15. Roman theater in Unity 3D environment.
Heritage 07 00169 g015
Figure 16. 3D modeling and its HighPoly model.
Figure 16. 3D modeling and its HighPoly model.
Heritage 07 00169 g016
Figure 17. European costume in HighPoly model.
Figure 17. European costume in HighPoly model.
Heritage 07 00169 g017
Figure 18. European actor’s 3D model standing.
Figure 18. European actor’s 3D model standing.
Heritage 07 00169 g018
Figure 19. Roman actor’s 3D model walking.
Figure 19. Roman actor’s 3D model walking.
Heritage 07 00169 g019
Table 1. Game evaluation scores.
Table 1. Game evaluation scores.
Evaluation QuestionMinMaxMeanSD
Competence2.005.003.9840.833
Sensory and Imaginative Immersion2.005.004.1430.820
Flow1.005.003.4920.801
Tension/Annoyance1.004.001.4920.669
Challenge1.004.003.0160.729
Negative Affect1.003.001.7140.851
Positive Affect2.005.004.3020.754
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Mamakou, I.; Karafotias, G.; Gkourdoglou, G.; Loumos, G.; Kargas, A.; Varoutas, D. Developing a Multitasking Augmented Reality Application for Theatrical and Cultural Content. Heritage 2024, 7, 3578-3596. https://doi.org/10.3390/heritage7070169

AMA Style

Mamakou I, Karafotias G, Gkourdoglou G, Loumos G, Kargas A, Varoutas D. Developing a Multitasking Augmented Reality Application for Theatrical and Cultural Content. Heritage. 2024; 7(7):3578-3596. https://doi.org/10.3390/heritage7070169

Chicago/Turabian Style

Mamakou, Irene, Georgios Karafotias, Gabriel Gkourdoglou, Georgios Loumos, Antonios Kargas, and Dimitrios Varoutas. 2024. "Developing a Multitasking Augmented Reality Application for Theatrical and Cultural Content" Heritage 7, no. 7: 3578-3596. https://doi.org/10.3390/heritage7070169

APA Style

Mamakou, I., Karafotias, G., Gkourdoglou, G., Loumos, G., Kargas, A., & Varoutas, D. (2024). Developing a Multitasking Augmented Reality Application for Theatrical and Cultural Content. Heritage, 7(7), 3578-3596. https://doi.org/10.3390/heritage7070169

Article Metrics

Back to TopTop