1. Introduction
Theater is an ark, in which the historical information of human activity and culture from every era and period is preserved. In the eyes of the contemporary observer, researcher or creator, the everyday costume of a previous era becomes a historical costume. A theatrical act is a “slice” of life of the heroes who live and move on stage. In this sense, a theatrical act provides future generations with information relating to the whole spectrum of a person’s life. This includes one’s habits, education, life status, financial situation, job, interests, age, even one’s beliefs and ethics and a whole lot more.
The Costumart project (
Figure 1) is part of the National Scope Action “RESEARCH-CREATE-INNOVATE” of the Operational Programme Competitiveness, Entrepreneurship and Innovation, co-funded by the European Regional Development Fund (ERDF) and national resources, under the NSRF 2014–2020 (code number Τ1ΕΔΚ-05070). It was funded to address some of these issues. The project was structured by a set of activities aiming to develop a set of applications based on augmented reality (AR) technologies and virtual reality (VR) tools. AR is an enhanced version of the real world, a kind of interactive environment based on reality, and this new version is achieved through the use of digital, visual, audio or other sensory means provided by real-time technology. VR aims to create simulated environments by using computer technology tools and embodying users in virtual worlds.
In order to achieve its goals, the project’s leaders took the initiative and completed the following set of actions and activities:
Activities in order to portray historical and urban costumes (digitization, recordings, etc.);
Creation of an infrastructure (storage systems, creation of ontologies, documentation dictionaries in three languages, etc.);
Creation of applications;
Publicity and communication activities (clothing display, offer of data documentation, virtual tour, storytelling);
Educational activities (educational applications and programs for specialists, for adults, for children, for schools, etc.);
Dictionary and index of costume terminology.
The whole project was created with the support of a Greek, well-known theater scholar, Mr. Nikos Kamtsis. Since the whole effort was part of a research program, which provided financial resources, the costumes produced fulfilled the needs of this research work and were not really used in the theater (they remained in the possession of the project). Even under this condition, costumes and theatrical representations were historically accurate following the theater scholar’s detailed instructions, who adopted principles coming from the pioneers in the creation of innovative costumes and stage design, such as Inigo Jones, Adolphe Appia, Edward Gordon Craig and Isadora Duncan [
1].
The project created open ontologies for the costume/garment, its use, its materials and its way of production as a learning machine for everyone interested in theatrical costume from an educational, theatrical or research point of view. The target audience for all educational activities was a theatrical audience, comprising adults, young people and children as well as specialists from the theatrical sector. Even though this target audience seems generic, it should be taken into account that younger people are more eager to test and experiment with such applications, while an older audience is searching for information and knowledge through using them. By following such a framework, our research team tried to provide information as accurate as possible regarding theatrical scenes and costumes, while information regarding plays and historical information was kept at a minimum level to retain the application’s entertaining character as well. Learning objectives where mainly concentrated on providing historical information associated with the architecture of the theaters and the costumes to transfer the application’s end users to the culture and the atmosphere of the proposed historical period. This project addresses and presents, via a series of applications, the clothing and dress patterns of men and women coming from four major historical periods: (a) Ancient Greek, (b) Roman, (c) Elizabethan and (d) European years. The whole procedure was carried out in two ways, by two distinct applications:
The incorporation of augmented reality within the theatrical act—The activity involved pilot AR use in a short-term theatrical performance where a small number of users wearing special glasses watched a theatrical performance with real and virtual characters.
AR application via mobile devices/augmented reality glasses—This included the development of a special augmented reality application that was applied in a showroom of theatrical costumes. The visitors had the opportunity to “scan with a mobile device or via the AR headsets” exhibits using the application. What followed was 3D actor modeling or civilians (avatars) wearing the costumes illustrating the costumes’ historical use and cultural value.
The proposed AR application works as a natural exhibition space. The visitor is able to “scan with his/her own smartphone or via an AR headset” any exhibit, travelling them back in time through the application, 3D models of actors or/and citizens (avatars) who wear these clothes, while storytelling can accompany and explain (through a synthetic voice) the historical use and the cultural value of the content provided. Moreover, the abovementioned applications contain not only the theaters but also three-dimensional models (avatars) of the representation of clothes of each era, while a series of texts and audio for educational purposes was developed (providing information about the theatrical performances and the actors of each era). Moreover, alongside with the costume/garment and its theatrical use, the application’s users can enrich their knowledge on the various architectures related to the abovementioned historical periods. A series of 3D theaters were developed permitting end users to choose the appropriate clothes/building and to see well-known historical monuments be created, while there exists an operation permitting users to explore both buildings and costumes from various aspects by using their mobile phone.
The development of 3D modeling, clothes and buildings/monuments as well as the user’s interaction with the application, were all achieved by using well-known technologies for augmented reality (AR) such as the Unity tool and Vuforia. The application has been implemented for both mobile phones with android version 10 software and IOS version 14 so that any user has the opportunity to both experience this application but also learn through it.
The current paper aims to present the project’s main result and to broaden readers’ understanding about how augmented reality, 3D modeling and virtual reality can be used to enrich a pure cultural experience such as a theatrical act. In the following sections, a brief theoretical background—mainly focused on the technological aspects and their usage in cultural activities—will be provided, technological issues related to the proposed application will be explained and the project’s results (3D modeling) from an aesthetic and operational point of view will be presented.
2. Literature Review
Augmented reality (AR) technologies alongside virtual reality (VR) technologies have been developed over the last twenty years, with an expanding amount of their usage in several kinds of cultural organizations. More and more, different cultural sectors face a need or express a willingness to implement such technologies. Cultural organizations are using both VR and AR technologies in order to reach their goals in their physical places as well as in the digital world [
2], including actions such as the following:
Expanding their presence to the digital world and making their content available to a mass audience in the global cultural market;
Reusing or distributing already existing digital content (moreover expanding their collections’ digitalization);
Connecting physical structures/artefacts with digital content (e.g., AR technologies) in order to provide their physical audience with a more interactive experience.
The use of AR technologies in various cultural organizations (e.g., museums, galleries, archaeological places) has gathered both theoretical and practical interest. Starting from the early 2000s, researchers have been interested in interactive technologies for museums [
3,
4], up to 2009, when Sylaiou et al. [
5] compared various forms of emerging technologies, such as VR, AR and Web3D. In the fields of museums and archaeological places, a series of AR applications were developed, with more indicative examples being (a) the Augmented Reality-based Cultural Heritage On-site GUIDE (ARCHEOGUIDE) system [
6], (b) an AR museum guide for internal navigation and data information provision on cultural artefacts [
7], (c) Arbela Layers Uncovered (ALU), a mobile AR guide [
8] using 3D modeling as well and (d) an AR application acting guide in the archaeological site of Knossos in Greece (called KnossosAR) [
9].
Technological advances and price affordability for reaching and using a smartphone [
10] led to further development of a series of applications, not only as a means for digitally replacing physical artefacts as was the case in the past [
11], but to enrich users’ experience as a whole [
12,
13] via the artefacts’ information, storytelling experiences, etc. [
14,
15,
16]. This led to the ability to access artefacts from different time periods in a realistic way and in a dynamic environment [
17], leading to what can be called “Industry 4.0” for the cultural sector/organizations [
2,
18].
During the development stage, the project’s team used prior experience from other projects including the following: (a) issues related with 360° videos [
19,
20,
21,
22,
23], (b) various pure AR issues [
24,
25,
26], (c) issues related with 3D content and virtual reality [
27,
28,
29,
30] and (d) theater and the implementation of advanced technologies [
31,
32,
33,
34].
As far as the theater is concerned, there is a variety of works exploring the relationship between technology and live performances [
35,
36,
37,
38,
39]. Some researchers put emphasis on audience participation via technological means, a framework that increases engagement, interaction and reshapes the cultural content in real time [
37]. Cerratto-Pargman et al. [
36] developed a mobile application that permits audiences to choose the outcome of a production, Lee et al. [
40] introduced such an application to music concerts and Rostani [
41] expanded the same concept to interactive performances in general. These researchers provided indications about the innovativeness and the capabilities that AR applications can have when it comes to live performances, including the theater.
However, existing research results indicate that technologies such as the ones described previously still remain a challenge when it comes to theatrical organizations, rather than full operational systems [
1]. Even though, in the forthcoming future, such technologies are expected to find practical use, for the time being, there are only few research efforts taking place. However, most of them are not using these technologies as a means to promote and preserve in a digital way, the existing living (intangible) theatrical culture. Therefore, the main research scope of the current research contribution is to empirically, if feasible, preserve and promote contemporary culture as expressed by theatrical costume through synchronous technologies of sound, color, video and augmented reality.
3. Materials and Methods
As mentioned before, two distinct technological tools (the Unity tool and Vuforia) were used, while the most interesting part was achieving the appropriate collaboration between them in order to reach the best result, from an operational and aesthetic point of view. Moreover, knowledge and skills were needed to understand the environment of Unity and how this environment translated/appeared not only into a computer but also into a mobile phone. Using Unity, a widely known 3D modeling suite, provided the advantage of the existence of a global community of developers with whom interaction may exist. Meanwhile, using Unity also ensures the viability of the project’s results, since it is a technological tool wide used and difficult to be abandoned.. Such conditions ensured both the preservation and dissemination of the project, while challenges occurred from limitations that the technological platform had itself.
Initial tests of the application were conducted in a computerized environment, to reach a no-error status, and then the corresponding procedure was conducted οn a mobile phone environment to perform the test on a smaller screen but also in different versions of both android and IOS. There were some problems with the correct movement of the characters on different screens which could not have been solved without prior knowledge. The developers also had to know which costume relates to which theater so as not to transmit incorrect information to users. Also, historical knowledge was needed in order to create the right texts for each character but also the right sound files.
The main purpose of this application is not only for each user to be able to see through a game the architecture of the buildings of each era (Ancient Greek, Roman, Elizabethan, European) but also to learn what clothes the actors wore in each ancient theater and to get updated about some theatrical performances that take place nowadays so that they can go and watch them. As mentioned above, many modern technologies have been used to create this application which will be analyzed below (
Figure 2). First of all, the Unity tool is the environment in which the application was implemented and the tool through which the application was converted to android and IOS, so that it could be compatible with mobile phones. It is also the tool in which all the three-dimensional models of the application are located/processed.
The other tool is Vuforia, through which the application can scan a specially formatted image and based on this information, start the game for the user (
Figure 3). The basic function of this tool is that when an image is lost from the camera of the mobile, then the game stops, as all the 3D modeling is created on this image. Also, when the image is found, the user can interact with the game, see and learn everything she/he wants.
3D technologies have also been used to create the three-dimensional models of both the theaters and the clothes of each actor (
Figure 4) as well as the dressing room where the actor enters to wear the right clothes. At the current stage of the project, this phase just aims to familiarize end users with the dress code and costumes of each historical period. At a later stage, when the database is enriched with theatrical costumes, more value, added experiences will be offered.
Even for the movement of each character to go get dressed, movement technologies have been used to make them look like they are walking like real people (
Figure 5). To achieve the result presented in the above figure, many techniques—related with specific operations—were used. The first operation was the actors’ walking, where a technique called timeline was used so that the walking of the actor could be more realistic. Emphasis was put on realism and on the experience gained from end users even though simple actions such as moving their character across the theatrical stage.
Furthermore, in the beginning, as can be seen in
Figure 4, the characters are not wearing their costumes. The user has to find the correct actor to begin the process and make the actor move. When the actor enters the dressing room, there is an invisible collider, which starts another process and the actor puts on the correct costume. Each time the collider is activated, a new piece of cloth (part of the costume as a whole) is added, until everything is on. Moreover, the user has to choose the right theatrical environment, among four distinct 3D theaters representing different periods of time (
Figure 6).
Associating time (a historical period) with the right costume and appropriate stage (environment) is this ambitious project’s goal, since it educates end users in cultural and historical trends often used in theater. Moreover, at a later stage, when more content will be added, these associations will be based on theatrical plays and not on historical periods. The project will provide the tools to educate end users on theatrical culture and the unique artistic value of relating theatrical plays to times, costumes and stages.
To facilitate end users, techniques were adopted to enlarge the theater (zoom button) but also activate and deactivate the panel with the written information for them to see costumes and frocks more easily. Since the aim was to educate end users in recognizing digital content and to associate it with specific theatrical plays, an environment of accessing information was developed. The more the content will be enriched, the more research will be needed from the end users’ part in order to avoid guessing and pass to the next phase.
4. Results
The proposed application incorporated four distinct theaters and four actors, each one coming from a different time period of the theater’s history. These 3D models were reused in each application and educational game developed. Moreover, a fifth character/avatar was developed, taking the figure of a clown and acting as a “conferacie”. Apart from 3D theaters and 3D actors—avatars (including 3D costumes), a third category of 3D modeling was developed, included a series of 3D objects (props) of either wearables from actors or objects used as part of the play.
In order to achieve the quality needed for the development of 3D modeling, extensive research was conducted. Research included finding and collecting the appropriate reference materials, coming from pictures, architectural drawings, videos, etc., while the most valuable source, related with 3D theaters/buildings, were architectural drawings (floor plans, facades and sections). Moreover, photos and videos were extensively useful when information was needed regarding construction materials (wood, stone, plaster, etc.) that were used. An example related to the Globe Theatre is given in
Figure 7.
For the whole research team, it was essential to incorporate valid historical information to enrich end users’ experiences and to provide as much detail as possible. Since costumes may vary from theatrical play to theatrical play, the current stage of the project’s emphasis (in terms of validity) was put on theatrical stages as the main element that remained stable. Especially when it comes to the Globe Theatre, there were a lot of reference materials as a result of the recent restoration of the theater, which provided a treasure trove of information for researchers, such as the roof of the theater (
Figure 8).
At a second stage, all reference material was evaluated and the most accurate information was used to develop 3D modeling. Architectural drawings, drawings and photos were imported to Autodesk Maya (
Figure 9), a 3D graphic and motion design software. By using this material and existing information as a base of the design, a series of software techniques were applied until an appropriate representation of the building was constructed. Such techniques included (indicatively) the following: extrusion, revolution, lofting, Boolean functions, etc. Transitioning from physical content to 3D modeling is a complex activity, especially when it comes to cultural content. Details and historical accuracy are more appreciated by theatrical stakeholders rather than programming details that are related with operations such as the movement of avatars and avoiding bugs.
The third stage of development included the creation of the 2D parts that would be used to create the 3D models (
Figure 10), a procedure known as UV mapping. At this stage, the two-dimensional development of the model was created which would be used to “clothe” the 3D model with textures, providing color information (albedo map), dielectric gloss (metal roughness map) and details of various anomalies (normal map). Success during this stage meant achieving the model with the smallest degree of distortions and with the least possible notches in the joints between the triangles. Even though this procedure may be known to most programmers, audiences from other fields such as theatrical stakeholders are now discovering a new world when it comes to understanding the 3D modeling procedure.
At the next stage, emphasis was put on the building’s texture, so that there existed a realistic resemblance to the “materials” that were used to build the physical building. The most accurate techniques involved the development of the abovementioned 3D pictures (albedo map, metal roughness map, normal Map). The developed model was implemented through the specialized software of Adobe Substance Painter version 7.2 (
Figure 11), using information gathered during the first stage of the development, while the software’s libraries permitted the effective imitation of the properties of natural materials (in terms of how light interacts) in the 3D modeling.
Finally, the model and the special images mentioned above were inserted in the game machine, which took its final display on mobile devices. The game machine determined a series of parameters, for example, the detection properties of the various materials as an obstacle to the avatars’ movement, etc. During this stage, programmers evaluated how the various 3D models operated when implemented on mobile devices (e.g., smartphones). Optimizations were made in the models and the special images in order to ensure the satisfactory performance rate (frame rate) of the application.
As far as three out of four theaters (
Figure 12,
Figure 13,
Figure 14 and
Figure 15) are concerned (Globe Theatre, Ancient Greek theater and Roman theater), there were no problems regarding the frame rate when the proposed 3D models were implemented on mobile devices. The European theater, having hundreds of seats, seemed to pose a problematic issue. Based on the fact that its seats are similar to each other, this made it possible to use an algorithm called “GPU instancing” to keep the frame rate to an accepted level, without any detriments to the realism of the representation.
For the development of the 3D actors, the same abovementioned steps were followed, including the collection of reference material, construction of the three-dimensional models, creation of their extensions, creation of the materials and implementation in the game machine. Two more stages were added, including the production of a model with a high number of triangles (HighPoly) and the addition of animation. Based on the fact that mobile applications must operate at a satisfactory rate of performance, modern 3D graphic design techniques require the construction of a second identical 3D model with a large number of triangles in order to represent details and irregularities of the physical object. This second model (HighPoly) was used so that these details were captured in the two-dimensional special image (NormalMap) and to create the illusion of their existence (
Figure 16).
For the development of the HighPoly costumes, the Marvelous Designer software version 10 was used. The proposed software is widely used in the video game industry, where the representation of clothing is particularly important. Its main characteristic is that it uses the design of the pattern of clothes and through simulation applies it to the character imitating the folds that are created. For example, in the following
Figure 17, the pattern on the right and the simulated model on the left can be seen. The correspondence of the pattern pieces in the model is marked in yellow.
For the needs of the game, the corresponding characters were also created wearing simple everyday clothes. The manufacturing process remained the same, but they were standalone models due to the fact that most parts of the body were covered by fabric. Additionally, the characters acquired special properties (rigging) that allowed them to move in space (animation). These properties were given in the 3D graphics and motion design software but were controlled by the game machine. For each character, an internal skeleton was created from the joints and bones (rig), which followed a logical hierarchy (parent–child hierarchy), for example, the shoulder–arm–hand–fingers (
Figure 18 and
Figure 19). The latter procedure is called weighting or skinning and aimed to move the model’s points in space by turning only the individual bones.
Complementary items are an important part of the app as the user has to choose the right combination of theater–costume–items to complete the game. For each character, two types of objects were created. The Ancient Greek actor is accompanied by a mask and quills, the Elizabethan and the European actors by a hat and shoes, and the Roman by a laurel wreath and sandals. Their differences in relation to the previous types of models lie in the fact that in their final stage, they must be inserted into the skeleton hierarchy and additionally, regarding the shoes, they must acquire rigging and weighting properties to follow the deformations of the foot and ankle.
In order to have a more detailed analysis, the research team conducted an evaluation of the proposed application during the period from 24 to 26 of May 2024. During this period, the research team reached out to the audience of a specific theatrical play, asking them to use the AR application and evaluate it. An online evaluation questionnaire was provided, which users of the application filled via tablets. At the end of these 3 days, a total of 63 fully answered questionnaires were collected, while there were another three partly completed and therefore excluded from the analysis. The mean age of participants was 35.8 years old, middle-aged το 35.8, while 61.9% of the answers come from females (39 females) and 38.1% from males (24 males). Moreover, all respondents had a smartphone, 68.25% (43 persons) answered that they were familiar with VR and AR technologies, 23.81% (15 persons) were somewhat familiar and 7.94% (5 persons) were not familiar at all.
Since the application resembles a digital game, a Game Experience Questionnaire (GEQ) was used to measure the following dimensions:
Competence describing how users evaluated their performance regarding the application’s goals;
Sensory and imaginative immersion providing insight into the degree to which users felt connected with the application;
Flow describing how fast the time passed when using the application;
Tension/annoyance reflecting the degree to which users felt any of these two emotions;
Challenge describing how challenging the application was for users;
Negative affect reflecting a negative emotional experience from using the application;
Positive affect reflecting a positive emotional experience from using the application.
In order to quantify these seven dimensions, a set of 33 statements related with users’ experiences were set, while a five-point Likert scale was used, with the scale ranging from 1 (“Do not agree”) to 5 (“Completely agree”). The questionnaire as a whole and its dimensions were developed from Ijsselsteijn et al. [
42] and were averaged to present the mean GEQ scores for each dimension, as presented in
Table 1.
In order to better understand the results, the research team conducted non-structured interviews of respondents who agreed to take place in such a procedure. These interviews aimed to illuminate users’ personal opinions regarding the application’s strengths and weaknesses and potential suggestions for new features as well as their reasoning for their evaluations. In the end, 12 interviews were conducted (almost 19% of the application’s users).
The results of both the application’s evaluation and interviews revealed a strong relationship between users’ satisfaction and the digital app with cultural content. Their evaluations and answers indicated that the application created a strong and positive emotional experience (mean 4.302—
Table 1), mainly coming from the fact that AR apps are not something that is common in a theatrical content. Most users were positively impressed by the perspectives of implementing AR solutions in theatrical scenes and the ease of use of the application (e.g., just downloading an app). Moreover, most respondents sensed a connection with the proposed application (mean 4.143—
Table 1), but their interviews’ answers mainly indicated that such a connection was mainly coming from their interest in the app’s content (theatrical costumes and scenes) and not from the app’s technological features. Such a result is also supported by the app’s flow (mean 3.492—
Table 1), which indicates that time passed easily when using it. Most users pointed out that they spent time observing the 3D costumes and theaters, having the opportunity to focus on details rather than proceeding to the app’s next phases.
Moreover, users evaluated how high their performance was as far as the application’s goals were concerned (mean 3.984—
Table 1), a result supported from the fact that the app had an average challenging orientation (mean 3.016—
Table 1). Since the application is more educational rather than game-oriented, the abovementioned results were expected, while respondents were not disappointed by the challenge set by the application or their effort to achieve its goals. There were respondents who expressed their willingness to see more game-oriented elements, such as “treasure hunting” or “puzzle-solving challenges and knowledge quizzes”, all taking into account the theatrical content and the information provided.
Finally, as far as negative emotions are concerned, the results indicate a rather low degree (mean 1.714—
Table 1), supporting results from previous research [
43]. Respondents declared that since they had no previous experience using AR applications with theatrical content, the whole effort (as expressed from the proposed app) mainly created positive emotions. Only a single older respondent expressed difficulty in effectively accessing the cultural content from his smartphone, since “everything on the screen was too small”. The abovementioned result is also supported by the fact that tension/annoyance was also rather small (mean 1.492—
Table 1).
5. Discussion and Conclusions
Based on the in-depth analysis of the project’s results, an initial conclusion is that by using this application, users can enrich their knowledge regarding the clothing and architecture of four different eras of antiquity (Ancient Greek, Roman, Elizabethan, European) through play and interaction with an environment. All of these can be achieved via their mobile phone, without having to open a computer to see the costumes, masks or theaters, since all the information is in the palm of their hand.
There were four scenarios for each theater of each historical period but in each scenario, there were small variations, useful to understand the different options that the user could choose and the different things they could learn about the theaters of each season. By scanning a play’s poster via its mobile phone, the app’s user can reach the above—mentioned content and information, enriching his knowledge and understanding about various historical periods. Modern augmented reality technologies and programming skills were used, as well as a lot of research to create three-dimensional models of clothes, theaters, shoes and accessories, without which the knowledge derived could not have led to the implementation of either of these items or the application itself.
The proposed application can be used in many different fields. Initially, it can be used as a game where students from schools will be able to visit a theater and not only see a play but moreover through augmented reality will be able to have the theater in the palm of their hand and enlarge it to see and to learn all the details required. They can also see the costumes of each character and learn what they wore in each historical period, but in a much more modern and interesting way than just seeing a garment, a shoe or a mask in a shop window.
In general, the proposed application seems to create a dynamic environment, capable of delivering information that is easier for end users to understand, achieving a higher degree of implicit learning, as has happened with various such applications [
44]. However, an issue remains of how such applications can be directly associated with social media in terms of further disseminating any project’s social impact [
45]. Finally, it should be mentioned that an important limitation and future research goal is that no user evaluation of the experience was conducted to discover in a qualitative and quantitative way the project’s impact on end users.
It should also be noted that such technologies have barely been used in other kind of industries, such as healthcare and fitness, where wearable gadgets are in common use [
46]. It seems that the abovementioned kind of technologies are not yet integrated in mainstream theater [
1]. Such a limitation derives from the fact that theatrical institutions may be reluctant to adopt technologies that may change the experience of a theatrical act. The anxiety of losing the collective experience of a live performance plays a crucial role, alongside changing the role of a theatrical stage as a public space to a humanitarian, social and political forum [
47].