A Review of Sustainable Urban Regeneration Approaches Based on Augmented Reality Technology: A Case of the Bund in Shanghai

Abstract

Both physical and cultural aspects are important for urban areas. In the context of urban space involving cultural resources, the application of augmented reality (AR) could physically integrate cultural resources into the physical facilities of the city. In this paper, the narrative approach is introduced to the construction of an urban cultural scene based on AR technology. Following the collection of historical materials about the Bund in Shanghai in the 1920s and 1930s, this study creates the characters and events in a non-fictional way, organizing a series of narrative elements around themes based on real historical contexts. This is in line with AR, a technology that superimposes virtual information in the real world. As part of this non-fictional narrative design, the author has chosen suitable characters as ‘avatars’ to observe and experience the development of the narrative from a first-person perspective. A participatory viewing can be brought about by adopting a non-fictional approach, rather than a simple information transfer and passive reception. The result is that instead of the traditional guide centric model, a multi-path tour model is created. In conclusion, this paper believes that the use of AR technology can link the narrative to the region more directly and closely. Further, a new cultural scene is created that allows visitors to have an immersive experience. AR technology provides a narrative perspective based on regional knowledge that reinforces and represents the image of the region in a cohesive way.

1. Introduction

As time goes on, there are more and more media that carry narrative stories, from the oral to the written word, then film, and now virtual reality (VR) and augmented reality (AR) [1]. In recent years, there have been numerous studies on the application of AR, VR, and mixed reality in the physical spaces, such as museums, which consider how to unveil the multifaceted nature of objects by providing additional multimedia contents [2,3]. Based on smartphones and tablets as supporting hardware, some augmented skins and skeletons can be viewed by visitors at the Smithsonian National Museum of Natural History in Washington D. C. [4]. Technology makes up for information that the exhibits themselves cannot directly show to the visitors. While museum exhibitions are designed to tell a story, the visitor’s journey through them is a metaphorical, a from start-to-finish way of reading the story. In museums, one can often see exhibits that bear the marks and signs of damage left by age. Although they have been skillfully restored and reinstated, or have computer-generated ‘original photographs’, the visitor may still not have a good idea of the historical or contextual information about these exhibits, and it is still up to the visitor’s imagination to expand upon them. As a result, the process of visiting an exhibition becomes a process in which the audience’s imagination is involved in the ‘conceptualization’ of the narrative story. AR technology can improve this conceptualization process by enhancing the users’ sense of presence [5], narrative engagement and reflection. However, there is little research on AR-based narratives in other types of physical places, such as open urban spaces.

According to Diao and Lu’s view, a sustainable urban regeneration should take into account both the physical space and the inheritance of cultural resources [6]. AR is defined as the synchronization of physical space with an overlay of dynamic digital data [7]. This combination is usually site- or object-specific, which commonly occurs in real time through participants’ engagement with devices, and mediates the environment in a way that previous media can’t achieve [8]. This characteristic of AR technology makes it easier to link narratives to physical places at the same time as creating a location-based experience [9]. The application of AR in the context of urban space can physically integrate cultural resources into the physical facilities of the city and can reflect the uniqueness of the architecture in its spatial and temporal dimensions [10]. This will help to express the diachronic factors of urban space in spatial places [11].

AR technology allows people to see the world in a different way. From a tourism perspective, traditional tourism mainly follows a fixed itinerary, and although it is divided into individual and guided tours, tourists lack a multi-level interactive experience of the scenic spots and local culture. Further, some attractions are not open to the public in order to better preserve, for example, historical buildings. This not only affects the visitor’s overall understanding of the area, but also limits the promotion of the location. However, if AR technology is used, then it is possible to show those attractions that are not open to the public in a certain space. This would allow visitors to have a fuller picture of the information and backstory of the tourist attraction. Today, there are still old houses that have been restored and opened to the public, but for visitors, such attractions mostly mean some ruins or restorations or replicas that do not allow for a complete and deep, as well as objective, understanding of these monuments. With AR technology, however, it is possible to input real scenes from history, based on time and space into a computer and then restore them to the real scene through digital technology. In other words, the classic look of the historical sites at various points is brought to the visitor’s attention. In addition, AR technology is well placed to overcome the many limitations and allow visitors to experience the stunning beauty of the original architecture.

However, it is challenging to create an AR-based space corresponding to the features and layout of the urban physical places. Karapanos et al. found that viewers have better mental imagery when watching a video story in a physical place where the surroundings match the narrative atmosphere [12]. Following this argument, this research introduces a non-fictional approach to create an open narrative in the physical space of the city, namely the Bund, in order to explore a new approach to sustainable urban regeneration, based on AR technology. In this approach, we want to use avatars to simulate the psychological journey of the protagonist from a first point of view, based on a real historical scene or setting, thus delivering a superior immersion.

2. The Non-Fiction Approach

2.1. Open-Ended Narrative

In contrast to a close-ended narrative, an open-ended narrative means that its ending is to some extent uncertain. That is, the author does not directly tell the audience the answer, but allows them to arrive at the end of the story on their own. Klaasen describes an open-ended narrative as an approach, which provides space for informing, forming and transforming identities in the post-apartheid South Africa. Furthermore, community, tradition, exchange and experience are used as four dimensions to interpret their narratives during the process of identity formation [13]. Scholars generally believe that the open-ended approach gives more space for people to engage with the construction of narratives.

Traditional narratives are author-driven, and the process of conveying narrative content to the reader is one-way. At the same time, the feedback mechanism for users is relatively simple, as the core of a textual narrative is the “textual content”, that is, the words, sentences and paragraphs [14]. Texts convey information primarily through narrative texts, which can be divided into two categories in their current form: one form is works traditionally published on paper or on the Internet, and the other is literary genres with an open-ended nature, such as online interactive fiction, which emphasizes interactive experiences [15]. There is a clear difference between these two forms, in terms of creativity and interaction. While the former has been empowered by the internet, which provides a platform with a greater capacity and a greater emphasis on mass communication than traditional vehicles, such as books and journals. This gives it a certain capacity for dissemination, but it still lacks more in terms of interaction. The latter is more open in terms of narrative, with logically interconnected text modules forming a three-dimensional net-like matrix that creates a complete textual structure. This textual structure allows users to participate in the production of content according to their own understanding, and their choices have a different impact on the ending of the story. The end result allows users to be deeply involved in the creation of the content. This new type of interactive online literature allows the reader–user to co-create with the author, and in this way, the reader is also transformed from a mere recipient of the information into a co-producer of the information [16].

Interactive fiction is a very representative genre that allows the authors and audiences to co-create the stories [17]. In the case of Lost Lies, videos, documents and other relevant clues are given as the breakthrough of the case, allowing the reader to take on the role of a ‘detective’ who finds the missing key characters in the narrative through different means. The users can choose from several given nodes, with each node corresponding to a different narrative direction. In 2020, the game company Madnetic Games adapted this script into an interactive video game. Kim introduced two similar games, Choices: Stories you Play and Episode—Choose Your Story, both of which were interactive fiction or ‘choose your own adventure’ games [18]. She paid particular attention to the neoliberal choices of users, arguing that storytellers should consider the differences between users, embodied in culture, economics and time, to promote a more enjoyable engagement with readers. Willis established video games as a special form of interactive fiction, and analysed their fictional content and the ways in which the research was created in order to present different types of fictional truths [19]. The development of linear logical thinking is facilitated by both textual and pictorial narratives and brought with it the significance of image-shaping, plot development and post-reading pleasure with the reader’s creativity at its core. The appeal of hypertextual narratives in digital media lies in their interactivity, with the reader’s creativity at the core, bringing the joy of role-playing, plot exploration and engagement. Players of video games such as Heroes of Might and Magic and The Sims can enter the virtual world created by the games through role-playing [20,21]. High-intelligence game programs such as Command & Conquer can continuously adapt according to the way the players want to play [22]. In the game Civilisation and SimCity 3000, players have the right and freedom to build their own countries and cities and to develop different strategies. The personal characteristics of the players will affect the progress and outcome of the game. For example, in the game Civilisation, if the player’s role is a leader who indulges in aggressive warfare or a wise governor who manages his country well, his identity will directly influence the course of history in the game to take two very different directions. It is possible for readers to read a narrative text several times to better understand it. Similarly, in interactive narrative texts, such as games, it is also desirable for players to improve their skills by repeating the game several times, or to win the game by changing the interaction to achieve greater satisfaction and pleasure.

Narratives designed for interaction, based on powerful computer processing, are no longer a single text, but a gathering of all texts. The audience involved is different, and therefore their chosen paths into the text are different, with the presentation of the narrative being different as well [23,24]. This situation is ubiquitous in interactive narratives. The audience can select the event point of their interest and enter from any node in the pre-defined text, thus entering a different event narrative situation and completing their own unique narrative. Theoretically, for an audience that chooses a textual narrative, it is completely sustainable as all texts are interconnected and exist in an intertextual form. It is a set of hypertextual narratives on the Internet, so that when people click on different buttons they can not only access different narratives, but also continue clicking on new texts in new narrative texts. The same is true for the design of interactive narratives. Based on the active participation of the audience, they pass through multiple texts one by one and complete the narrative as a result.

2.2. A Non-Fiction Approach

Non-fiction can be a way of conceptualizing creative writing in the process of creating a play with an open narrative [25]. Non-fiction writing represents a new form of narrative writing and communication that has also attracted increasing attention with the popularity of the Internet. With depth and breadth in the narrative process and open-ended plotting, non-fiction conveys the complexity of characters and stories better than other creative approaches. There is an emphasis in non-fiction on the ‘live’ nature of the narrative, which is extremely compatible with the ‘immersive’ character sought by open-ended narratives. As a result, many works of non-fiction are narrated from a first-person perspective. In the narrative, the author, through his own observations and feelings, creates a scene in which his own influences are “everywhere” and parallel to the real space.

It has been argued that creative non-fiction is a reconceptualization and reinterpretation of real history and source materials [26], enabling an easier reflection on some specific historical events or people. The immersive reading of non-fiction places the audience in the spatio-temporal context of the narrative and the audience is guided to confront some acute and insightful issues. Through the creation of non-fiction, the open-ended narrative script is dynamic and available to users who can also jump and navigate through hyperlinks. This is where the creator connects the plot, characters and scenes through their own logic, whereby different connections will take the narrative in different directions. The audience may choose different connections regarding their own judgement, so each reading will be a unique event.

The reality is the basis and starting point for the construction of a work of non-fiction, requiring facts and personal experience as the context for its creation [27]. The interpretation of non-fiction is also characterized by the authenticity of the extended elements of reality. For example, through the dual narrative ability of linguistics and imagology, non-fiction films can stretch narratives across multiple media, thus offering the possibility of extending non-fiction stories. This feature is in line with AR technics, which is based on real-world content.

A non-fiction narrative mode can be divided into “complete non-fiction” and “incomplete non-fiction”. The criteria for distinguishing the two include the subjective consciousness of the creator, the degree of historical restoration of the text reproduction and the degree of immersion experience of the audience. Generally, “complete non-fiction” works, such as biographies, dictations and reportages, must be created in strict accordance with the development process of historical events. The “incomplete non-fiction”, that is, on the basis of well documented content, can be artistically recreated narrative. At the same time, the author is allowed to have reasonable imagination and plot displacement. However, the author chooses the time segment or character prototype as the prototype of the overall content design, and the narrative logic between them must have a certain coupling with the real events. The creative genre in this study belongs to the latter.

AR, as a form of information dissemination that blends virtual information with the real environment in the age of smart media, significantly enhances the sensory experience of the audience in accessing information. The content of AR, unlike that presented by VR, is a technology characterized by complete virtualization, and does not completely recreate the virtual scene. Instead, by placing the virtual model in a real space, with a real external scene as the backdrop for the display, this greatly accentuates the practical implications. An AR, unlike holographic projects, does not require large fixed devices to display its content in real time. Instead, viewers can access content anytime and anywhere through their AR glasses, mobile phones, tablets and other smart portable devices, making viewing in a wider range of spatial and temporal scenarios possible. With this perspective, a scenario of content built by AR technology will reach into people’s lives in the near future. This study, according to narratology, argues that tangible remains, such as buildings, are the tangible carriers of stories, while at the same time, the history of the experiences of these buildings is the discourse, both of which are an important part of narratology [6]. Taking the Bund as an example, these buildings are the narrative carrier of the city’s history. This study explores how narrative can be used as a medium to present cultural connotations in the physical space. Further, the fragmented buildings are integrated in a non-fictional way, thus highlighting the cultural connotations of the city.

John Pavlik established a link between AR technology and storytelling and argued that producing contextual documentaries is the most appropriate part of media-based content creation and AR integration [28]. These documentaries further add sound and 3D images to what viewers see or hear, telling the story of historical events that take place in the user’s surroundings. By stimulating the imagination with objects based on specific locations or the real world, AR technology provides an open content production environment for narrative, thus helping to enhance the audience’s experience and understanding of narrative storytelling. Constructed narratives based on AR hybrid spaces, which tend to merge with the content of communication, have become a pathway for people to return to and engage with the centre of communication. The new media space has become a domain containing human, economic, social, cultural and other information elements. One can return to the centre of the various elements of communication through a series of means, such as weakening symbols and dissolving field boundaries. AR hybrid spaces can record and reproduce the trajectories of people’s actions in an urban space, whilst empowering people to mark, comment on and even construct spatial symbols. The space becomes a link between the virtual and the real urban space, as well as an interface for people to engage in common practice.

Unlike linear descriptions, the non-fictional approach provides a way of collecting narrative elements around a theme. Based on this, users can act to combine these narrative elements to form different narrative understandings. Briefly, it enhances the historical and interactive nature of the spatial narrative. That is, users can be guided to consciously make connections in time and space, thereby contributing as a whole to the construction of a spatial narrative.

For the scripting of the AR scenes based on the historical background of the Bund, the authors followed the basic requirements of authenticity, realism and an emphasis on the sense of presence of an immersive experience in non-fiction creation. The historical setting of the narrative is in Shanghai in the 1920s and 1930s, the financial centre of China at the time, which had been open for several decades and was full of merchants and celebrities and a booming economy and trade. The individual characters to whom the narrative leads, and the plot between them, must therefore also be in keeping with the period. During the creation process, the author did not use artistic manipulation to recreate a fictional setting. Rather, they collected historical material about the Bund in Shanghai, and in turn created characters and events based on the real historical context. In particular, the complex connections between the characters had to be designed with some connection to the real world in mind, as non-fiction plays need to be based on the real world.

3. Interpretation of the Cultural Information

The most important starting point for the application of augmented reality is to enhance the user’s perception of real-world space. The spatial element is therefore very important in the narrative of augmented reality. This study uses the Bund in Shanghai as an example to show how non-fiction can be used to construct space for narrative. The Bund, located on the banks of the Huangpu River in the Huangpu district of central Shanghai, is known as the Outer Huangpu Bund. This area, leased to the British from 1844, is considered to be the starting point of the old Shanghai lease and the modern history of Shanghai. Fifty-two classical and revival buildings of various styles stand in the Bund, known as the ‘World Architecture Expo’. They are significant cultural relics and typical buildings of modern China and one of the landmarks of Shanghai. In November 1996, it was listed by the State Council of the People’s Republic of China as part of the fourth batch of national key cultural relics protective units.

The Bund has a wide variety of multi-storey and high-rise buildings. For example, the Asia Building (formerly the Shanghai Institute of Design and Metallurgy), the Shanghai Club (now the East Wind Hotel), the Shanghai Pudong Development Bank (formerly the HSBC Building), and the Jardine Matheson Building (now the Foreign Trade Mansion) are of British Classical, Neoclassical and Renaissance styles, respectively. There are also many other styles, such as French classicism, French mansion, Gothic, Baroque, modern Western, Eastern Indian, Eclectic, Chinese-Western, etc., which co-exist with the architecture of the world. Therefore, the collection of buildings stretching from Jinling Road in the south to the Waibaidu Bridge on the Suzhou River in the north are hailed as the “World Architecture Expo”. These buildings, which blend classicism and modernism, have become the symbol of Shanghai. There are 33 buildings in the Bund, some of which are still in use by the authorities. For instance, the No. 13 Shanghai Customs House, was finished in 1927, and still houses the Shanghai Customs. The No. 14 Bank of Communications building, completed in 1948, is the most recent of these buildings to be completed. Following the foundation of the state, it has been managed by the Shanghai Trade Commission. The rest of the buildings are mostly the headquarters of foreign banks and insurance companies, as well as high-class hotels. For example, No. 1 was the Asia Building, built in 1913; the Nissin Building (also known as the Shipping Building), completed in 1925, was used to house the Japanese Shipping Company. The HSBC Building (also known as the Shanghai Municipal People’s Government Building) was built in 1925. The current East Wind Hotel, which was at first the main British social club in Shanghai, had the longest bar cabinet at 110.7 feet. The then Central Hotel No. 19 is now the Peace Hotel. No. 22, the Sassoon House, was completed in 1929, and is the tallest one in the Bund and part of the Peace Hotel. Nos. 3, 6 and 18 have all been renovated and are now high-end shopping and entertainment venues and are the centres of luxury consumption in Shanghai.

Table 1. Deconstruction of Contextualised Information at The Bund.

Buildings Nowadays	Bandao Hotel	Tianantang Theatre	Heping Hotel
Buildings in History	British Consulate	Union Church	Shaxun Building
Contextualised Information	The Bund was originally a dam on the Huangpu River, surrounded by a village called Lijiazhuang. Balfour, the first British consul general in Shanghai, bought the property of the village. He also negotiated with Daoyuan (a local official in the Qing Dynasty) and signed the Shanghai Land Regulations. At that time, the British settlement became the first official concession in Shanghai. Balfour’s choice of this mansion has become the source of the Bund’s development for more than 100 years.	Designed by the famous British architect W.M. Dowdall, the Union Church (formerly known as Lijiazhuang Church) became home to a growing number of foreigners, which made it the earliest settlement of foreign entrepreneurs. It is now the only authentic relic that has not been renovated or rebuilt.	In 1832, David Sassoon, the founder of Sassoon Bank, moved from his ancestral home in Baghdad to Bombay, India, and established Sassoon Bank. in 1864, Elias David Sassoon, the second son, was sent to Shanghai to take charge of the business there. He then bought the property at 22 Nanjing Road from Augustine Hurd & Co., in October 1877 for 82,000 taels of silver. In 1917, Elias David Sassoon’s grandson, Elias Victor Sassoon, took over the business and Sassoon Bank became one of the four major conglomerates alongside Swire, Jardine Matheson and British American Tobacco. Victor developed the land as a whole and built the Sassoon Building, which is today the Peace Hotel (North Building).

shows examples of the deconstruction of information about the representative buildings in the Bund, from a historical perspective.

4. Construction of a Non-Fiction Narrative of the Bund

In order to introduce the history of the Bund, this paper uses a non-fictional approach to create a script that deals with representative buildings. Furthermore, each building in the Bund is linked to this script. In the script, multiple characters are created, taking into account their experiences and situations, and their ‘plight’ is presented from multiple perspectives, which can hold the audience’s attention. The audience can follow the narrative line of this story as it roams through physical space, gaining access to the AR information superimposed on the building, thus enabling a ‘real-time’ and ‘experiential’ audience reception mode in real space. This script, in turn, contributes to the construction of a three-dimensional, multi-temporal mode of interactive communication. Overall, to summaries this paper, a non-fictional approach has been used to create a script, based on the historical context of the Bund (Appendix A).

As shown in Figure 1, the script was created using a non-fiction writing technique. At the same time, the existing historical buildings in the Bund are outlined in the context of the development of Shanghai after the opening of the Bund. Moreover, an aerial narrative line is used to connect the relationships between specific buildings in the scene. Not only does it follow the timeline and advance the plot in a non-fictional way, but it also incorporates narrative methods, such as voice-overs and flashbacks. In addition, the script has been written with the temporal dimension in mind. Specifically, in order to break the continuity of time and space, interactive plotting and montage effects are used after the narrative content of the sub-chapters.

For example,

Upon discovering the weapons, a series of questions popped into protagonist A’s head.” Who are these weapons for? Why am I here? I’m just an ordinary college student who got a job because I followed D to Shanghai.” A feels a deep sense of loss because A’s education requires A to stay away from these weapons, but A also knows that they are the very means by which D makes his money.

In this passage, the weapon in front of A is referred to as A’s clue, which the author uses to enable A to switch in time and space between the past and the present context of his life. The temporal montage effect created between the two segments, as well as the trajectory of virtual and real life, provides a hint for A to make ‘some kind of choice’. In contrast to traditional pictorial narratives, content superimposed and enhanced open-ended narratives allow for the creation of panoramic simulations of ‘virtual environments’, depending on the need for scene scripting. The creation of such, more integrated hybrid environments with reality, and an immersive sense of presence, greatly enhances the depth of the content.

On the technical side of creation, given that the object of study, the international architecture of the Bund, spans several kilometres in a realistic setting, the coupling between the differences in location, and the overall narrative structure based on augmented reality, had to be taken into account when creating the narrative script for the scenes in question. According to Gwilt, from a creative perspective, AR creates scenes that bring digital content into physical space, meaning that new meanings can be created for ever-changing places and scenes [29]. Conceptually, the virtual images superimposed by AR are not intended to mimic the phenomenon of one space being replaced by another, rather they are intended to understand the complex relationships between them.

Table 2. How Buildings and Their History were Integrated in the Script.

	No. 16 Terminal	Waitanyuan	Cenotaphe, the Allied War Memorial	Nanjing Road
Script	The place where the main characters A and B first land at sea is the starting point for the whole story.	The most powerful tenant merchant, D, was located in the area where the largest number of businessmen from all over the world were gathered in Shanghai, at that time.	The place where A and C have their emotional entanglement, is a famous landmark in Shanghai during the Republican era.	The path of investigation of D by A. A large number of industries owned by D are concentrated along Nanjing Road.
History	At the end of the Qing Dynasty, the county of Shanghai was divided into several shops inside and outside of the city, and the banks of the Huangpu River outside of the small east gate of the southern city belonged to the “Sixteen Shops”. Due to the concentration of wharves around the “Sixteen Shops”, it was an important water gateway where goods were distributed and transferred from north to south. It became synonymous with the port of Shanghai for a time. The subsequent development of the street market turned the area into a commercial centre, which is how Shanghai took shape as the financial centre of the Far East.	The British first established a consulate, business and missionary office in Shanghai’s Migratory Bird Garden in 1849. Chronologically, it is the earliest group of historical buildings in the Bund area. From a spatial point of view, it is the starting point of the Bund.	Built on 16 February, 1924 to commemorate the foreigners who returned to Shanghai after participating in the First World War.	The first commercial street established after the opening of Shanghai, it was a symbol of Shanghai’s commercial prosperity during the Republican era. The wealthy Jewish merchant Hardoon spent huge sums of money to pave Nanjing Road with expensive mahogany, making it one of the most famous commercial streets in the Far East, as it was praised in foreign publicity as “a road paved with gold”.
Current Situation	In December 2004, the old wharf, once the largest land and water passenger transport hub in the country, was closed.	It has become a premium commercial gathering area in the Bund.	It has been dismantled.	It has been a symbol of Shanghai’s commercial civilisation and is known as the “First Street of China”. In the 1990s, it was completely renovated and became a “pedestrian street” where no motorised traffic was allowed.

illustrates how the buildings of the Bund and their cultural information are integrated into the script.

As can be seen, the script is based on the history of the Bund, and most of these scenes are not directly visible in the present day. A scholar of cognitive narrative, Ryan introduced the concept of cognitive maps into narratology. This has also facilitated the integration of narratology and geography, expanding the path of narratology to the study of spatial history [30]. Whereas the characteristics of AR can link the narrative and places more directly and closely. For example, based on this concept, visitors, especially as their knowledge and experience is usually limited to historical contexts, can also view some of the missing content without the need for an interface, through a specialised device (e.g., a mobile phone or a tablet). Instead, the use of AR can “provide concrete visual clues about past events and representations of content, and leave room for the visitor’s imagination” [31]. Figure 2 shows the historic look of the Shaxun Building, the predecessor of the Heping Hotel, that Cyberverse has attached to it and displayed on the mobile phone interface.

In non-fiction script design, the author chooses the right characters as ‘avatars’ and observes and experiences the development of the narrative from a first-person perspective. The author needs the characters to be present, to feel present and to think in the present. For example, in this study, the author acts as a waiter in Character D’s mansion and observes the various characters in the narrative from their perspective. The audience can also perceive the presence of the author’s thoughts in the development of the narrative. Taken together, this feature of the non-fiction approach gives the narrative scripts in this study a greater sense of feeling. The narrative script in this study has a greater sense of resonance and presence precisely because of the presence of non-fictional creations. It gives the viewer an experience that is usually one of participatory viewing rather than simple information delivery and passive reception. Furthermore it is aligned with AR technology as virtual information is overlaid on the real world. It is conceived as a narrative that can transcend history as well as time and spatial reality.

5. Sustainable Cultural Resources: An Immersive Urban Experience Scene

The integration of new technologies with urban cultural venues has been developed over the years. Guangzhou Daily established a new media technology support center, with the Original Technology Department as the core, to produce content products with timeliness, visibility and immersion, working with the content production team. In 2020, the technical team involved in the production of “VR Takes You Behind the Scenes of the Cantonese Opera”, assisted journalists in filming scenes such as backstage dressing rooms, rehearsal rooms and stages, respectively, using panoramic filming equipment. Taking the dressing room as an example, the dressing room became a virtual reality space under the 360-degree VR lens, as shown in Figure 3.

Unlike VR, AR allows virtual information to be better associated with landmarks by projecting digital objects into the real environment. These landmarks, in turn, are not only carriers of historical memories, but also of cultural heritage. With the help of AR, the history of the buildings can be presented virtually. Users can also be able to break the limits of physical space and time and learn a wealth of information related to their past lives through multiple devices. For example, the Guangzhou Thirteen Hongs Museum l collaborated with an AR technology company to launch an AR guided tour function in 2018. Visitors can scan the cultural exhibits in the museum through a customised app and listen to an audio guide while viewing digital images of the artefacts on the screen. On the electronic screen, visitors can select their favourite exhibits, adjust them to the right size and position, then take a photo with them. In 2020, the Liangzhu Museum in Hangzhou, Zhejiang Province, launched an AR glasses tour, giving visitors the additional option to immerse themselves in the past lives of exhibits, rather than being limited to voice or manual tours. At the same time, this AR technology supports offline voice recognition and can also reconstruct historical scenes that cannot be restored during traditional manual and audio tours. As well as museums, AR technology has also been used in the restoration of the Da Shui Fa site. Baidu’s AR technology department connected the old Summer Palace site to the actual site, allowing a ‘complete’ view of the Da Shui Fa through a handheld mobile AR device. On the screen, the original scattered stones and pillars are presented in the form of layers of line sketches. The framework of the building erected amidst the shattered remains is colored in layer by layer. As visitors move around the site, they can see how the Da Shui Fa would have looked from different angles.

However, both exhibits in museums and Dashuifa in the Old Summer Palace are separate cultural sites, and aims to integrate the scattered cultural sites into some open places in the city, such as the Bund in Shanghai. Generally, people’s first impression of Shanghai is of the Bund, as it is the city’s calling card. As the city grows and changes, the Bund itself is also changing. It is well known that the Bund is located on the banks of the Huangpu River in the Huangpu District of Shanghai. Through AR technology, it is possible to show people the past of these buildings and the stories that cannot be told, thus making the intangible cultural meaning of ‘history’ visible. Through the medium of architecture, people can really see what makes the Bund the ‘root of Shanghai’s history and culture’. At present, the way people visit the site is still mainly a sightseeing tour. Most of these people follow a guided tour, and individual tourists are fond of taking photographs. However, all these approaches only allow the visitor to passively accept the city. AR can change this situation. It allows visitors to go from passive to active, exploring each and every sight in the city through a script-like experience. This greatly increases the motivation of visitors, thus increasing their goodwill towards the city and better reflecting its character. In addition, AR has a strong endogenous nature. For a long time, tourists have mostly been sightseeing tourists, and only for the sake of sightseeing. This type of sightseeing has resulted in visitors not having a deep impression of the destination and scenic spots. Compared to a real tour guide, a virtual tour guide created through AR technology is more approachable and interacts more freely with the visitor. In this way, the distance to the visitor is reduced, thus making sightseeing more personal and independent. It is a novel experience that also deepens the visitor’s impression of the destination. In addition, AR is highly realistic. Through the combination of panoramic projection and AR, scenes that cannot be viewed up close, can be highly recreated. All of these interactions can enhance the stickiness of users and bring a constant stream of visitors to the scenic spot. Further, a user evaluation function can be added to the AR system, allowing visitors to give their opinions and comments on the scenic spot. This can promote a better development of the scenic spot and provide better services to visitors, thus enhancing the competitiveness of the tourist destination.

Rooted in the city, its urban cultural resources can be overlooked in the process of urban modernization, which makes it difficult to maintain the city’s historical and cultural contexts [6]. Through a non-fiction approach, we explore a narrative perspective that combines physical space and cultural context, as well as finding ways to reproduce and present the cultural resources of the city.

As a media, this paper finds that the application of AR technology can enhance the presence of the Bund’s cultural resources and create a new scene, unlike anything the city has seen before. A distinctive feature of media technology since the early days of its development has been that it breaks the obligatory link between information delivery and the shared space [32]. Different from old media tools, however, technologies such as VR, AR and MR have broken the boundaries between content and audience through interactivity, resulting in thoroughly different sensory experiences. Boundaries are just similar to “the fourth wall” between the theatre stage and the audience. On a proscenium stage, there are generally three walls in the space, but actors and actresses on the stage are prone to imagine a virtual wall to separate the stage from the auditorium that enables them to better immerse themselves in the performance while ignoring the reactions of the audience. By contrast, the audience, if indulged in the performance, tends to ignore the wall. Similarly, in cases such as being fully immersed in the storyline of a movie, or paying too much attention to the movements, routes and operations of a game while ignoring other people and things, people’s minds are already immersed in the world inside the screen although they are physically sitting on the other side of the screen. A screen is similar to a wall connecting another world, through which those in a state of immersiveness seem to enter the other world. People become aware of this wall since they can perceive the existence of boundaries, such as the entrance of the stage and the screens of mobile phones and computers.

Computers have now evolved from tools for content production (mimicking the functions of typewriters, paint brushes, etc.) to equipment that can be used to create, store, distribute information and access all media. Lev Manovich characterized the way computers present media content as “human-computer cultural interface interaction” [33]. At this time, the user no longer interacts with the computer, but with the culture that it carries. The computer is thus a tool or medium for human interaction with culture, similar to newspapers, magazines and books. A combination of screen and computer has become the main method of rapid access to media information. Reality, framed in a rectangular screen, has clear boundaries, as well as being irreversible and untamperable. Furthermore, only the content presented on the screen is given an essence, being illuminated and thus viewed by people. An AR-based space, by contrast, is one in which there is barely the slightest visual blind spot for the viewer. As such, they can be seen as being ‘pulled in’ from outside the story or event. In other words, there is an immersion of the viewer’s mind in the space constructed by the media, and the perceptual boundaries between the physical world and the media space become blurred. In previous technological environments, such media spaces were more often presented on a screen, in people’s conscious imagination. In contrast, technologies such as AR and VR allow for the direct construction of a space that emphasises the viewer’s first perspective of perception and experience. Within it, people can interact directly with the environment constructed by VR technology, rather than through a third perspective. The development of networked information technologies has made both digital presence and remote presence possible, in addition to the physical corporeal presence. Human perception through the direct senses is weakened and the ‘innate instrumental perceptual experience’ is replaced by technological perception [34]. Technologies such as VR and MR are similar to building an immersive environment, that is to say building a reality that is closer to the human body’s most primal sensory experiences. When people wear devices with extended sensory capabilities, the sense of ‘being there’ acts directly on the body’s perceptual system, allowing people to consciously include their own bodily feelings and life experiences in their associations. Humans can feel the presence of their bodies through the perception and practice of the spaces constructed by the media technology system, realising the ‘re-perception’ of the body in the process of communication. In this case, a virtual body can exist in the constructed space as an ‘avatar’ of the real body. This is a new way of presenting the subject [35].

In the age of mass media, due to the existence of communication, many things are actually inaccessible to people on the spot, so they act more as observers of the scene. At the same time, their bodies have difficulty in accessing the communication network directly and achieving live communication, and they can only rely on symbols to connect with other subjects. Thus, it is necessary to create a sense of reality and immersion through imagination. The “sensory hegemony of the visual centre” has changed, as the incarnation’s presence in the hybrid space relies more on various interactive activities [36]. This also characterises the multisensory, immersive interaction of scenes in AR-based spaces.

The words and phrases of written language take the form of abstract symbols that have existed independently of humans from the very beginning. Human perceptions and ideas are systematically presented through media. In the era of images, the three-dimensional world is condensed onto a two-dimensional screen. This is then converted by the eye into pulses with specific frequencies that activate the relevant image information stored on biomolecules in the brain, allowing the brain to perceive the information more directly. In the age of immersive communication, the body is “immersed” in the communication field as the interface for information interaction. The original text- and image-oriented communication model is weakened and replaced by an immersive communication model centred on real-time human participation and aided by the activation of sensory resonance. Thereby, in the era of interpersonal interactive spoken communication, new analogue scenarios characterised by “voice narration” are created. The immersion model of communication echoes the ‘dialogic’ model of communication advocated by Idealism by changing the narrative style of the media and diminishing the function of symbols as media. The presentation of symbols in a unified form maintains the communicative paradigm of the text, while eliminating the differences and homogeneity of the narrator’s personal image. The logic of this idea is that once information is presented in symbolic form, the value of dialogue is reduced. Thus, once the simulated scenario dominated by ‘voice narration’ is restored, information with the individual as the subject of communication will be represented. Instead, the human figure will be restored in a virtual space without borders, with diachronic and synchronic coexistence.

Linear or non-linear narratives based on traditional media are characterized by a flattening of the structure. In contrast, this AR technology-based narrative, through non-fiction offers a new spatial dimension. The guide centred ‘closed-loop’ narrative text is broken, and the original one-way story structure becomes a multi-path/multi-exit tour with each ‘approach’ and ‘interaction’ of the visitor. This immersive interaction can transform the visitor’s cultural experience of the city, and the underlying cultural messages can be uncovered. One of the things that is often missing from urban cultural resources is how to bring the historical stories of what happened on the sites more directly to the visitor. This would facilitate the integration of the physical spaces in the city with the inner stories, thus strengthening the local cultural identity. The concept of narrative in a non-fictional way can help to integrate scattered cultural resources. Connecting these cultural resources located in different parts of the city in an orderly manner brings the cultural value of the city into sharper focus.

6. Conclusions

In conclusion, this paper is an attempt to apply a narrative approach to the construction of physical spaces involving a cultural significance, presenting an example of the process of applying a non-fictional approach to connect the Bund’s architecture. This cultural and historical narrative, based on essential local information, can form a new bond based on physical place. Thus, the image of place can be strengthened and represented in a powerful and cohesive way. In this paper, the use of non-fiction offers a new perspective and, considers how the story can be made coherent when applying new techniques. The application of non-fictional narratives provides a way to connect dispersed information and allow the audience to take an active and dynamic role in making spaces and activities directly relevant to them. “Without culture, there will be no future of the cities [37]”. This paper therefore attempts a new approach to construct an urban cultural scene with audience participation. The aim is to provide a better understanding of the city’s past and to have a positive impact on the cultural sustainability of the city. It enables visitors to see not only the physical preservation of the culture, but also to experience the local history and the environment in which they live.

According to the World Tourism Economy Trend Report 2022 (https://cn.wtcf.org.cn/20220307/b11199c0-4fbf-a965-15bd-4ba53da421ed-i.html, (accessed on 5 August 2022)) jointly released by the World Tourism Cities Federation and the Chinese Academy of Social Sciences, the post-epidemic global tourism economy is showing signs of structural recovery, in which the importance of sustainable tourism is increasingly prominent. Therefore, the task of developing diversification strategies and seeking new economic growth through technological change has become an important one in reshaping the tourism landscape and reviving the industry. In this regard, the use of AR technology has the potential to transform and revive it. However, it is important to note that unlike traditional narrative approaches, the role of the tour guide is weakened in the approach proposed in this study, and in its place, the visitor is given more autonomy and choice. The visitor can experience and guide the story along different narrative threads. At the same time, the visitor’s focus may, at the same time, be distracted by the diversity of story directions. Therefore, narratives based on AR technology need to identify several main lines and edit and express them along the narrative logic. At the same time, further research is needed to focus on how to remind the audience of the pre-defined focus in the immersive scenes through appropriate guidance without compromising their experience.