Virtual Restoration and Virtual Reconstruction in Cultural Heritage: Terminology, Methodologies, Visual Representation Techniques and Cognitive Models

Abstract

Today, the practice of making digital replicas of artworks and restoring and recontextualizing them within artificial simulations is widespread in the virtual heritage domain. Virtual reconstructions have achieved results of great realistic and aesthetic impact. Alongside the practice, a growing methodological awareness has developed of the extent to which, and how, it is permissible to virtually operate in the field of restoration, avoid a false sense of reality, and preserve the reliability of the original content. However, there is not yet a full sharing of meanings in virtual restoration and reconstruction domains. Therefore, this article aims to clarify and define concepts, functions, fields of application, and methodologies. The goal of virtual heritage is not only producing digital replicas. In the absence of materiality, what emerges as a fundamental value are the interaction processes, the semantic values that can be attributed to the model itself. The cognitive process originates from this interaction. The theoretical discussion is supported by exemplar case studies carried out by the authors over almost twenty years. Finally, the concepts of uniqueness and authenticity need to be again pondered in light of the digital era. Indeed, real and virtual should be considered as a continuum, as they exchange information favoring new processes of interaction and critical thinking.

1. Introduction

1.1. Cybernetic Approach to Virtual Heritage

A digital “object” can be a text message, a 2D image, a 3D model, a movie, or a sound. It can be a reproduction of a physical artefact, or it can be natively digital. The digital object, if limited to the reproduction of a real object, becomes a replica, if it is obtained through criteria and methodologies that guarantee fidelity and accuracy both at the topological/metric level and at the surface properties level. In this sense, the digital replica becomes a transmitter of information and allows the preservation of the knowledge of that object, even if the real original is lost. Moreover, through rapid prototyping techniques, it is possible to print digital replicas and obtain physical copies that can be touched or relocated in their original context to restore a fragmented or lost cultural object. Physical copies can also be used for haptic experiences addressed to the public, which is in particular useful to enhance accessibility for visually impaired persons (if following specific criteria and characterized by a recognisable tactile pattern).

However, the digital “content” should not be limited to the reproduction of an object. It should be a communicative and cognitive unity, endowed with form and meaning.

According to the cybernetic approach introduced by Gregory Bateson [1], the main objective of virtual heritage is not the massive digital recording, the model, or its objectivity and description, but is the creation of a dynamic space of relations and interactions. Therefore, what really matters is the process, the experience, what we do with the digital model, and the simulation of a multiple past open to many possible experiences. The feedback obtained from the virtual context modifies our behavior and critical thinking and stimulates understanding and attribution of meaning; thus, the process of knowledge is built and consolidated [2].

Hence, the goal of a virtual process is to increase perceptual and cognitive levels, reactivating spatial–temporal relationships and meanings of the cultural object. The virtual dimension facilitates the mental process of imagination [3], giving shape to an abstract concept (a vanished ancient context cannot be perceived and experienced), making legible and recognizable what the visitor is often unable to “decode”, identify, and contextualize.

Virtual heritage therefore requires connection. All the information about an artefact and its context must be connected according to relational maps and accessed through experiential spaces. Typically, but not necessarily, this is a three-dimensional space. The three-dimensional space amplifies our possibilities of interaction: it allows us to explore, move, and change the point of view; to construct and to deconstruct the model according to new relational hypotheses; and to create simulations, strengthening the possibilities of analysis and interpretation. In virtual space, the cognitive activity is “embodied” [4]. This means that it should affect the user on two levels: (1) perceptual motor (sensory perception of form and affordances stimulated by the cognitive activity itself) [5] and (2) symbolic reconstruction of the meaning, thus associating deeper levels of understanding with the visible appearance. The latter can be integrations of the artefact (virtual restoration), contextualization in the original place (virtual reconstruction), relationship with the environment, or values and meanings attributed to the artefact by the various societies with which it entered into relation over time [6].

On this theoretical background, we are going to discuss concepts and methodologies dealing with virtual restoration and virtual reconstruction.

1.2. Definition and Domain

1.2.1. Physical Restoration

In the book “Theory of Restoration”, a landmark theoretical essay of Cesare Brandi [7], which had a great influence on the Italian and international practice of restoration (at least in western countries), the author stated that: “restoration is the methodological moment of recognition of the work of art in its aesthetic and historic dual polarity, in view of its transmission to the future” [8]. In 1964, the “Venice Charter’’ was drafted at the Second International Congress of Architects and Technicians of Historic Monuments. The charter consists of 16 articles and defines the principles of the methodology of architectural restoration, which can be considered immutable. Here, the restoration is defined as a “a highly specialized operation. Its aim is to preserve and reveal the aesthetic and historic value of the monument and is based on respect for original material and authentic documents. It must stop at the point where conjecture begins, and in this case any extra work which is indispensable must be distinct from the architectural composition and must bear a contemporary stamp. The restoration in any case must be preceded and followed by an archaeological and historical study of the monument” [9]. Currently, modern restoration consists of eliminating the causes of deterioration and recovering the legibility of the historical evolution of an artwork [10] (p. 27). It is based on a historical–critical approach and implemented through a complete fusion of historical and technical–scientific expertise [11]. The principles developed over years, concerning the material consistency of an artwork, are currently established as international fundamentals for any restoration activity, and can be listed in the following five issues [12]:

• Respect for aesthetic and historical value: The meaning, history, and authenticity of the cultural asset should be preserved.
• Compatibility: It is necessary to know the material of which the cultural assets are made for a correct evaluation of the intervention and compatible materials to be used.
• Recognition of intervention: It concerns the legibility of the original parts. Integrations should be recognizable.
• Reversibility: Any material used in restoration should be removable in order to return the artwork to its original conditions, allowing for future restoration.
• Minimal intervention: It is necessary to repair or conserve original parts rather than replace materials to maintain the historical value.

1.2.2. Virtual Restoration

In 1994, the professor Gianfranco Fiaccadori used the term virtual restoration as a methodology that combines technique and purpose to intervene in damaged heritage in a virtual way [13] (p. 16). More recently, the Principles of Seville, which are mainly oriented around computer-based visualization in archaeological heritage, define virtual restoration as the process of “using a virtual model to reorder available material remains to visually recreate something that existed in the past. Thus, virtual restoration includes virtual anastylosis”, which “involves restructuring existing but dismembered parts in a virtual model” [14] (p. 2).

Indeed, virtual restoration consists in applying digital techniques in the field of restoration. It is bounded in the digital domain and does not imply any interference with the materiality of the artwork. For this reason, virtual restoration does not have any conflict with some principle of physical restoration (like compatibility, reversibility, or minimal intervention) nor restrictions, since all actions have no consequences for the original cultural object [14] (p. 42). This makes it possible to digitally perform operations and simulations that may be unfeasible when intervening directly on real artefacts [15].

However, virtual reconstruction shares the same notions of philologic study, authenticity, and scientific transparency with physical restoration to guarantee the reliability of the work and avoid arbitrary interpretations and reconstructions [12] (p. 25) [16] (pp. 106–124) [17].

Given this rapid evolution, virtual restoration can play different roles according to the contexts and field of applications. It is used to:

• Plan and assist physical restoration. It allows the prefiguration of the result of a real restoration, according to the principle of guided restoration, by simulating all the separate phases of intervention. An exemplary case study is the restoration of Madonna of Pietranico. 3D digital technologies were used to assist the real restoration of a fragmented terracotta statue damaged in the 2009 Italian earthquake. Before operating on the fragmented artefacts, the researcher simulated their recombination in a virtual environment. The simulation was performed using digitized 3D models of the statue fragments, reducing their manipulation, preventing damages, and increasing the capabilities to evaluate different reassembly options. The digital 3D models were also used to design and produce a physical supporting structure created with a rapid prototyping device [18]. Another interesting example is the case of the Buddha of Bamiyan destroyed by the Taliban in 2001. The virtual reconstruction was aimed at creating a digital model of the statue to support the physical preservation and restoration works on site, exploring the possibilities of a future anastylosis starting from remaining fragments of the statue [19]. In the field of painting restoration, specific digital image processing techniques have been developed over the years to analyze damaged areas, like cracks or gaps [20,21], and to solve difficulties related to the pictorial reintegration. It is the case of the Madonna with the Child, Saint Joseph, and Saint John, an oil on canvas on which researchers performed an image analysis method, based on the semi-automatic extraction approach, to automatically classify lacunae and outline different virtual operating proposals [22]. Another notable example is the restoration of Giotto’s and Cimabue’s frescoes in the Upper Basilica of Assisi, which were reduced to several hundred thousand fragments by the earthquake in 1997. The physical restoration for the recomposition of the fragments, almost impossible to calculate only by hand, was preceded by their digitization and by a computer simulation. This process was able to analyze and create correspondences among the fragment profiles, leading to a partial physical relocation of most of the fragments [23].
• Rebuild lost heritage. The destruction of cultural heritage in territories of war or because of terrorist attacks has led to an unprecedented need for digital preservation and rehabilitation of lost heritage through projects of virtual restoration. In the case of Palmyra, UNESCO has promoted plans to facilitate the restoration [24] (p. 34). Physical restoration was not always possible, and only thanks to 3D modeling technologies it was possible to digitally rebuild the damaged monuments in their former beauty. The digital rebuilding of the damaged artefacts can be used both for testing different options of possible anastylosis and, above all, to preserve and bring back the image of the monuments in their former integrity and make them accessible through virtual reality applications [25].
• Restore visual assets. In some particular fields, like photography and cinematography, virtual restoration represents the only possible technique of effective restoration able to preserve these cultural visual assets and their historical value, regardless of the material support [13] (p. 23).
• Reconstruct an artefact in its integrity. Following the principles of “stylistic restoration”, this digital intervention aims at reconstructing the unity of style of an artefact, corresponding to the hypothetical original aspect. This “total retouch” [10] (p. 31) is possible only in a virtual environment, since it only involves a digital edition of the artwork without damaging the original, especially when restoring the original in its integrity is impossible. The added value is that it integrates the physical methodologies, allowing an undisturbed reading of restored artefacts and improving their legibility both for interpretation and museum communication projects [26]. This approach is also called “virtual iconographic restoration” [27], especially when applied to paintings. The digital intervention removes alteration from the painted image and fills the gaps in a mimetic way. All the information necessary to fill the gaps is taken directly from analogous elements present on the surface of the artwork [28]. The main issue regarding the stylistic virtual restoration is authenticity. However, this approach over the years has developed a rigorous philological method. The interpretation and restoration of the missing parts is not invented, but based on the concepts of style and analogy to guarantee the reliability of the work.

This “stylistic restoration” is usually carried out on small gaps or in any case where the lacunae are replaceable on the basis of tangible evidence, or through punctual and incontrovertible deductions (conjunction or continuation of lines or colors to integrate a clearly identified figure). This approach was followed in the virtual reconstruction of the painted slabs of a Lucanian tomb [28]. Missing details like anatomical parts of the characters were philologically reconstructed, taking as an example analogous elements present in the representation itself. When the gaps were too large or there was not enough information, operational approaches were borrowed from physical restoration, avoiding recreating missing figures or contents. In [10] (pp. 31, 88), the authors describe digital techniques used for the virtual restoration of the frescoes preserved in the rock-cut church of Lama d’Antico (Italy). When it was not possible to perform the “total retouch”, other techniques, such as schematic reconstruction, neutral retouching, and chromatic dampening, were used to visually harmonize the overall iconographic program of the church.

This kind of virtual restoration is applied in all fields of research, from frescoes to mosaics, drawings to written documents, and architecture to sculpture. The virtual architectural restoration in the archaeological domain is certainly one of the most complex fields of application, both because it embraces many branches of virtual restoration (frescoes, mosaics, wooden infrastructure, etc.), and because most buildings are often preserved at the level of ruins [14] (p. 23). The digital integration of the parts that no longer exist requires specific precautions and methodologies developed over years in the field of virtual archaeology [16] (p. 107) and easily trespasses into the realm of virtual reconstruction. This topic requires a separate discussion, which will be examined in the next sections.

1.2.3. Virtual Reconstruction

According to the Principles of Seville, Virtual Reconstruction is a digital process which uses “a virtual model to visually recover a building or object made by humans at a given moment in the past from available physical evidence of these buildings or objects, scientifically -reasonable comparative inferences, and in general all studies carried out by archaeologists and other experts in relation to archaeological and historical science” [14] (p. 2). Like the stylistic virtual restoration, the virtual reconstruction implies the restitution of an artefact at the time of its creation or its successive phases of use, but, in the latter, the concept of “hypothesis” seems to play a predominant role.

However, “virtual restoration”, intended as stylistic intervention, and “virtual reconstruction” are often used as synonymous especially in the field of built heritage. While the former is preferably used in the academic field of architectural restoration, the latter is more common in archaeology. The cause of this overlap of terminology may depend on different factors.

The first factor is the different disciplinary background between archaeology and restoration. Although these disciplines often work together in the same contexts, they have their own theories and methodologies. The operational methodology of virtual restoration is a natural evolution of the physical restoration. It also shares the aims related to image restitution and legibility. In the field of archaeology, it seems more correct to use the expression “reconstruction”, as this word emphasizes more the deficient state in which the monuments are generally found and, consequently, the need for a more extensive interpretation.

As mentioned above, in archaeology, the percentage of lost volumes often exceeds what is preserved, and the information collected in the field is not sufficient to define a complete hypothesis nor to ensure legibility. To overcome the lack of information, it is necessary to push the critical hypothesis beyond and rely on other documents, testimonies, and comparisons that come from other similar contexts.

Secondly the term “reconstruction” could seem too peremptory, and its use has also long been debated in archaeology. Many academics [29,30,31] have criticized the use of the term “reconstruction”, as it could be misleading and detrimental: it may convey a false sense of knowledge with the risk of mistaking for “truth” what is actually nothing more than a simulation, a hypothetical model of the past, or the result of subjective interpretations.

In several projects of virtual archaeology, the two concepts of virtual restoration and virtual reconstruction coexist. When a building is virtually proposed in its past appearance and function, many existing parts are digitally restored (e.g., paving mosaics, fragmented columns, etc.) and integrated in a wider virtual reconstruction, aiming at giving a general idea of the cultural context. This means that in a virtual reconstruction, many “local” interventions of digital restoration are usually included, with various levels of reliability that must be differently documented. The reconstruction is often needed, especially for communication or educational purposes, because without context there is no communication [1].

1.3. Legibility, Contextualization, and Symbolic Meaning: Where and How Virtual Restorations and Reconstructions Intervene in the Communication Pipeline

Every cultural artefact consists of a combination of materials, colors, and shapes (aesthetic consistency) and a convergence of expressive values and meanings (historical values) [7,32]. An artwork is an object created by man, using any material, endowed with aesthetic characteristics, and imitating the natural or spiritual reality.

Each artwork reflects the artist’s opinions on the social, moral, cultural, ethical, or religious context of his time. In other words, a cultural object conveys a message coming from its creator, and it is addressed to a specific audience.

The main function of a museum, besides preservation, is education, which means making understandable the cultural message beyond the materiality of the artefacts, triggering in the public a process of critical interpretation and elaboration of the meaning. If the cultural information enhances connections, the visitors will be able to evolve their thinking capacity.

This is not always obvious, because many cultural objects cannot be recognized and understood any longer, as they come from a completely different cultural context or they have been damaged by the passage of time [6].

Indeed, there are three conditions for cultural transmission to take place:

• Legibility: The object must be identifiable in its shape, content, and functionality; this condition falls within the domains of both physical and virtual restoration.
• Contextualization: The object must be connected to its original context (e.g., a statue originally belonging to the decoration of a temple pediment; a painting located on an altar in a church). It falls within the domain of virtual reconstruction.
• Narration: The function and the symbolic, intangible value of an object should be narrated (like the historical events of which it paid testimony during its life, along with the meaning that different cultures and societies have attributed to it, from past to present). Here, the attention is focused on identity. This condition falls within the storytelling domain, and virtual reconstructions are often the visual background of such stories.

Therefore, we can state that:

• Physical restoration aims at the “preservation” of the materiality of the object and its cultural content.
• Virtual restoration aims at digital preservation of the information about this content, enhancing its legibility.
• Virtual reconstruction aims at the “valorization” and dissemination of the object, enhancing its meaning and function.

2. Theoretical Background and Methodological Approach

In the last 20 years, the rapid technological and cultural development has allowed the diffusion of different types of digital applications (involving virtual reality, augmented reality, mixed reality, serious games, etc.) oriented toward the communication of the past using three-dimensional content. Today, 3D technology and virtual reality allow digital interventions that in the mid-nineties were inconceivable. For these reasons, virtual restoration is now applied not only to bidimensional features like wall paintings, paintings on canvas or wood, mosaics, documents, and library materials, but also to artefacts that involve 3D restitution like sculpture, movable artefacts, and, above all, architecture.

We are witnessing an increasingly careful and widespread use of virtual reconstructions in scientific applications and in dissemination projects within exhibitions, museums, and archaeological sites. This large diffusion depends also on the recent hybridization of media in digital applications, which borrows and brings together different paradigms and languages from other fields (virtual reality, theatre, cinema, applied games, etc.), to involve users and fix knowledge through a vivid and incisive narrative cultural experience [33,34].

The main problem with this kind of application regards reliability and scientific transparency. In fact, it is very difficult, even for experts, to distinguish what is original and what is a hypothesis. Moreover, it is impossible to know what kind of sources and logical processes are used to formulate a reconstructive hypothesis. The result is a “black box” [35], and the reconstructive models are considered just fiction rather than a consistent and thorough visualization model of the past, based on a scientific approach.

2.1. The Reconstruction Dilemma

The idea of reconstructing the past has been part of archaeology almost since its origins [27] (p. 63). The first to combine virtual reality and cultural heritage was Paul Reilly, who in 1990 introduced the term Virtual Archaeology to describe the use of computer-based simulations of archaeological excavations [36] (p. 133). Since the 1990s, the method, theory, and application of computer-based visualization in cultural heritage have been discussed and criticized in several periodic conferences such as Computer Applications and Quantitaive Methods in Archaeology (CAA), Eurographics, Digital Heritage, Virtual Reality, Archaeology and Cultural Heritage (VAST), Arqueologica 2.0, Cultural Heritage and New Technologies (CHNT), and Virtual Systems and Multimedia (VSMM). From the perspective of post processual archaeology, the possibility of reconstructing the past in an accurate way was considered impossible, since all information collected in the field is always incomplete and too complex to be reconstructed in its entirety. Furthermore, the interpretation of the gaps is always flawed from the contemporary point of view. In the last two decades, thanks to the development of computer graphic technology, computer-based visualization has enormously evolved in terms of graphics rendering and realism. This development has had a great impact on virtual archaeology. The main consequence is that Reilly’s concept of “simulation” has been declined, and the term “reconstruction” has been widely used to refer to these visualizations. According to Clark, it would be preferable to use the terms “models, simulative models, or scientific models”, as they are tools for better understanding the past and not statements of reality. Models are just simplifications, subject to a selection of information and therefore useful for interpretations [29] (pp. 66–69). Baker also prefers the term “visualization” instead of reconstruction, since the former “does not pretend to show the real thing because what constitutes the real thing is open to far too much speculation” [37] (p. 164). Yet, Barratt [38] uses on purpose the term “3D approximation” to emphasize some intrinsic aspects such as subjectivity and speculation.

Another criticism concerns the high graphic quality of virtual reconstructions. Given the idiom “seeing is believing”, the realism achieved by modern visualization systems could lead the users to perceive the virtual model as “truth” instead of as the result of interpretation. For this reason, every virtual reconstruction should follow adequate procedures to declare the level of authenticity, allowing the distinguishing of what is real from what is interpreted.

Clark complains that most virtual reconstructions are not provided with such procedures. This is often because making transparent all the sources that led to the development of a hypothetical reconstruction is not easy, especially when dealing with complex contexts or conflicting information. These criticisms were certainly also generated by the fact that the first applications of virtual reality for 3D visualization of archaeological data had several weaknesses. As Forte underlines, at the beginning there were a lack of consistent virtual archaeological projects aimed at finding precise answers using digital technologies [39]. In fact, many projects were more focused on technological exhibition and dissemination aspects rather than on scientific research. The first models were not “transparent” in respect to the sources, and the reconstructions were presented as peremptory without offering alternative hypotheses [40]. Finally, there were a lack of interdisciplinary professionals who would have linked the humanities with the computer science world. In 2007, Hermon complained that the potential of 3D visualization for scientific research was greatly underestimated, and the reason could have been that, at that time, there were few archaeological reports presenting new results obtained while using 3D [41].

Debates were very intense in the 2000s about the advantages and disadvantages of virtual archaeology and the use of computer-based visualization in cultural heritage [40]. Indeed, despite prudence and criticisms, many benefits have been considered by the scientific community. Sanders, in “Why do virtual heritage”, explains the specific benefits of using interactive 3D modeling in the field of cultural heritage, saying that it is not merely visualization, but “takes advantage of the digital medium to (…) produce new insight into the past, which after all is what archaeology is supposed to be all about” [42]. According to him, among others, virtual heritage is the better way to test complex hypotheses, visualize intrasite change and development, and visually absorb complex datasets about the past. Forte, in “About virtual archaeology”, supports the great cognitive potential of virtual archaeology. Furthermore, he is convinced that this domain was not born as a consequence of a technological development and is not only methodological, but integrates theoretical aspects [39]. Hermon and Nikodem [42] also emphasize the use of 3D modeling as a research tool and underline the cognitive value of 3D visualization, as it facilitates the understanding of complex information.

The debate about benefits and drawbacks, and how these virtual reconstructions of the past should be properly produced and managed, both in research and communication, is still ongoing. The same applies to the use of terminology. For example, the term “virtual reconstruction”, although criticized, is widely used in the field of digital heritage, with an increasing awareness of its meaning.

2.2. Virtual Reconstruction as a Multidimensional and Dynamic Space of Interaction

The cybernetic approach, from which our discussion has started, can be a perfect answer to such a dilemma. In fact, the final goal of virtual heritage and cyber archaeology is not only digitization, production of a replica, or a fixed “model”. Virtual heritage is meant as the information associated with the cultural heritage that enhances its value, beyond the physical consistency of the objects. In the absence of materiality, what emerges as fundamental is the interaction processes, the feedback, and the semantic values that can be attributed to the virtual model itself [39]. The cognitive process is developed from this interaction. Regarding fragmented archaeological contexts, reconstructive hypotheses are based on available archaeological and historical information that represents the knowledge background. Thus, the result of a virtual reconstruction should educate visitors to better understand the complexity of the cultural heritage domain. It should communicate that it is just a “a possible reconstruction”, the result of an interpretation, and not the truth, a perfectible assumption that can always be updated in the light of new discoveries. Otherwise, the risk is to trivialize the heritage data instead of simplifying it, diminishing virtual reconstruction to mere sensationalism or media event.

But how can such a process of interaction within a virtual environment be triggered?

There are basically two conditions making this possible:

• Making the virtual reconstruction part of a virtual ecosystem: It should not be limited to a pure visualization, nor be disjuncted from its informative context. The simple visualization and exploration of a virtual model can satisfy only sensory motor skills, making learning partial. An informative network should be implemented, connecting the 3D elements with the knowledge available for each of them. In a virtual reconstruction, the user can learn through the experience, combining perception, movements, behaviors, and symbolic–reconstructive mental faculties. Behaviors and possibilities of interaction arise from contextualization, selection, manipulation, in depth exploration, and comparisons. Contextualization should be developed both at a holistic (general meaning of the context) and analytic level. Such a kind of approach has been theorized and adopted by our team at CNR Institute of Technologies Applied to the Cultural heritage (ITABC since the beginning of 2000s, in a pioneer project of virtual reality dedicated to the Scrovegni Chapel in Padua. In the virtual environment, the architecture and all Giotto’s painted scenes were associated with thematic layers. A cognitive space, the “cybermap”, was created, corresponding to the iconographic one. (Figure 1);
• Following the principle of “data transparency”: Interpretative sources and processes should be declared and made explicit in virtual reconstructions, to let the public distinguish what is original and certain from what is probable or evocative. In these cases, the principles established and accepted at international levels in the virtual archaeological domain are helpful to shape such a level of information, both in terms of visual grammar and of data structuring and integration.

2.3. International References Shaping the Discipline

As described above, a project of virtual restoration/reconstruction requires a scientific approach to avoid the equivocity of the black box effect. As a result of the scientific debate on virtual reconstruction, numerous projects and documents aimed at creating efficient guidelines and good practices in the field of scientific visualization of the past have been implemented over the years, such as the London Charter and Seville Principles. Scientific projects and consortiums also have contributed to the research providing tools and support, as in the case of 3DVisa, V-Must, and Ariadne infrastructure and 3D-coform (even if the latter are more oriented towards digitization and visualization services). These initiatives arise from the awareness of the potentialities of computer-based visualization of heritage, but also from the need for a theoretical debate and principles with practical implications to regulate its use and minimize its weaknesses and inconsistencies.

The London Charter is one of the most internationally recognised documents that defines a set of principles of computer-based visualization in virtual heritage, to ensure intellectual and technical integrity, reliability, documentation, sustainability, and accessibility [43]. It points out the importance of structuring and documenting not only the sources used and their metadata, but also the interpretive process made to achieve the visual representation. This aspect is particularly important when dealing with virtual reconstruction. “Sufficient information should be documented and disseminated to allow computer-based visualization methods and outcomes to be understood and evaluated in relation to the contexts and purposes for which they are deployed” [44]. The Section 4.4 underlines the importance of the knowledge claim. It states that any computer-based visualization of heritage should clearly declare its identity (e.g., 3D models of existing state, evidence-based restorations, or hypothetical reconstructions) and the extent and nature of any factual uncertainty.

The Principles of Seville is another important document, drawn up to increase the conditions of applicability of the London Charter and to improve its implementation specifically in the field of archaeological heritage. These principles seem to be inspired by the philological and restoration domain. Principle 3 on “authenticity”—(…) it should always be possible to distinguish what is real, genuine, or authentic from what is not—borrows the same concepts of physical restoration contained in principle 3, “recognition of intervention”. Principle 7.1 on “scientific transparency” focuses on the importance of the documentation to make all reconstructions testable by other researchers or professionals: “(…) to achieve scientific and academic rigour in virtual archaeology projects, it is essential to prepare documentary bases in which to gather and present transparently the entire work process” [45].

However, the mentioned principles are intended as guidelines and good practices and not as norms. There is not yet a standard solution to manage and represent the typology of technologies and the complexity of records and interpretations involved in virtual reconstruction, or to represent the reliability/uncertainty of the reconstruction. In the past few years, researchers developed and adopted different solutions: segmentation methods according to “degree of certainty” based on “level and classes” [46] or “typologies” [47]; a method borrowed from architectural restoration, based on the “data correspondence model” [17,48]; quantitative methods where the validation of the models is calculated according to a numerical “index of reliability” [49]; a method that uses structural equilibrium and constructive rules to validate architectural reconstruction [12]; methods based on systemic simulations [50]; formal languages based on stratigraphic approaches [51]; and others. Even if solving this issue with a universal method is complicated, some of these various methods have been successfully applied.

2.4. Workflow

The process of developing a virtual heritage project involving digital restoration/reconstruction of an ancient artefact, architecture, or landscape for research and cultural dissemination implies a challenging and complex workflow. Without presuming to exhaust here such a wide-ranging topic, an operating protocol used in these kinds of projects will be discussed in summary below. The process is based on bottom-up and top-down strategies. Bottom up is related to still existing material and measurable evidence (on the site or in the museum); top down refers to the information coming from interpretative studies based on literary, iconographic sources, cultural patterns, proportion rules and comparisons, and so on.

The entire work is carried out by different professionals. According to the situation, the working group can be mainly composed of computer scientists, graphic designers, archaeologists, architects, art historians, etc. The comparison, integration, and verification of their respective activities allows the improvement of a strong and reliable scientific pipeline on which to elaborate interpretation and virtual reconstruction. The workflow is organized as follows:

• Survey: The geometric and graphic acquisition of a cultural heritage in its current state of preservation is aimed at creating a digital replica of the site. This is fundamental for the formal knowledge and study of the object itself. Today, there are different image-based and range-based technologies with different characteristics in terms of accuracy, precision, portability, cost, automation, etc. The task of the surveyor is to evaluate, case by case, which are the most suitable techniques for the object’s purposes, considering the problems inherent to the artefact/site and the final product.
• Documentation: The second type of data to support the philological reconstruction concerns the whole bibliography, the written and iconographic sources related to the object investigated, and, when possible, also the data already organized and structured (GIS, database, etc.). The virtual reconstruction, therefore, requires the same philological approach used for the corrupted texts to avoid the visual outcome being just a result of imagination. The philologist, in his work, supplies the edition with a preface introducing the witnesses and their relations, uses symbols to indicate problem in the text, and finally introduces a critical apparatus, which is the critical and primary source material that accompanies the edition of a text. A similar approach to handling metadata and the explicit decision taken needs to be used in the virtual reconstruction.
• Data processing and interpretation: All data collected and processed are then analyzed and discussed among professionals in different domains to formulate a reconstructive proposal that is as likely as possible according to the available resources. Therefore, the following contents are taken into consideration and combined in the interpretative process: (1) elements still visible on site, (2) elements that were documented but are no longer visible today, (3) style and theory of proportions, (4) figurative deductions, and (5) typological comparisons and cultural patterns. After analyzing the data and solving the critical issues, the first hypothetical reconstruction is drafted, returning the formal and decorative completeness of an object. Every new version of the virtual model must be verified until the scientific committee does not approve a definitive version (or versions), discharging those fallacious or less probable.
• Creation of the 2D or 3D reconstructive hypothesis: This phase occurs simultaneously with the previous one. It can be very complex depending on the type of artefact to be reconstructed and its lacunose state of preservation. Ancient architecture is certainly one of the most complex areas, both because their structures are often preserved in a state of ruin and because a building generally includes other categories of cultural heritage, from paintings to mosaics [52] (p. 47), each of them with a different degree of conservation. It follows that each reconstructive model has different degrees of reliability that must be stated through graphic expedients that allow the recognizability of the interpretive intervention (see next point). Today, there are several approaches to managing the transparency of the data (see Section 2.2). It should be kept in mind that during the virtual reconstruction phase, the modeling process should not be considered as a mere visualization, a functional tool for the restitution of the ideal image of a cultural heritage as it should have appeared in the past, but as part of the interpretative process for the verification and synthesis of analytical data. Forte complained that in the last two decades, instruments, tools, and software were more focused on data recording rather than analyses and interpretation, and often enormous quantities of data are not completely used and the interpretation often remains hidden in the models. Nevertheless, the new phase of completely digital research could allow a better management of the entire workflow from data capturing to reconstruction, improving the capabilities of interpretation [53] (p. 126).
• Source mapping and transparency: Additional layers of decision-making and documentation. When dealing with 3D reconstruction, it is necessary to combine a large number of sources in order to formulate the reconstructive hypothesis. For each element that makes up an artefact, it is necessary to reconstruct different missing parts, along with their physical properties like shapes, materials, decorative apparatus, or styles. Given that, all data previously processed and used together with the deductive processes of analysis and synthesis that led us to design the virtual reconstruction should be traced and mapped to make the process transparent. As already mentioned, today there are several approaches to managing the transparency of the data (see Section 2.2), but not a standard protocol. Especially in archaeology and architecture, specific software solutions have been developed to create semantic models allowing several layers of documentation and decision-making to be connected with virtual reconstructions [51].

It is also necessary to keep in mind the communicative aspect concerning the final output of our work (computer graphics animations, serious games, virtual reality, augmented reality or web applications, etc.) that inevitably influences the approach to modeling. In fact, it is in view of the final product that the aesthetic language and the detail of the model is decided [54] (pp. 260–266). Here, the scientific contents are synthesized in a visual representation. The visualization of the past through virtual reconstructions becomes essential in the workflow, as it assumes both a cognitive and a communicative value. The virtual reconstruction in fact, through special presentation and simulation tools, improves cognitive processes by making it easy for anyone to understand the historical and archaeological data represented and transforming the raw data into information.

3. Purposes and Target

When dealing with the simulation of the past, two communicative approaches are possible, both founded on a scientific approach and starting from the same dataset but pursuing different purposes. The first approach is oriented toward an expert audience, and the second one is oriented toward a wide audience.

The target, in fact, greatly influences the experience design to be applied to an application dealing with virtual restoration and reconstruction of cultural heritage: more analytical in the case of a specialized audience and more mediated, narrative, or playful, in the case of a non-expert public that must be introduced to the cultural context and to its historical background.

The medium adopted for communication greatly influences the typology of the experience: a movie consists in a predefined sequence connecting perspectives and contents according to a given logical process. The spectators have no possibility of performing an active role in the knowledge process. They can of course acknowledge and reflect on what they have seen and heard, but without the possibility of interacting, performing actions, or obtaining different feedback. For this reason, a movie is useful to record and document the different phases of a process once their sequence and methodology have been confirmed. Similarly, a movie is useful to clearly communicate a meaningful story in a well-defined time to a passive audience.

In contrast, an interactive environment solicits the users into an active role, aiming at progressively shaping a personal knowledge process through an alternation of action and reactions.

Decisions regarding rendering techniques, contents and metadata, visualization technologies, and investigation tools change according to the different audiences, even if they always follow methodological awareness and scientific consolidated criteria. In the next paragraphs, this topic will be further discussed.

3.1. Analytic Investigation for an Expert Audience

If the virtual environment aims to analyze and study an artefact, its detailed topology, and its state of preservation, or to simulate an accurate virtual restoration, a 3D model with high resolution and high accuracy is needed along with all the specialized information levels perfectly mapped and connected with the geometry. Interactive tools of measurement, a set of variable lights (diffuse, spot, direct, oblique, etc.), can help the analytic approach with the structures and its elements. A connection between the 3D space with a semantic database can be very helpful to let the scholars and expert users make queries and visualize related information directly on the 3D model. Similarly, records about construction materials or executive techniques could be associated with different elements of the 3D model; additional textures could be loaded to map the items of the state of preservation (Figure 2) [55] or to make the inner stratigraphies visible and comparable (for instance, images recorded through spectrography techniques). All the contents will be closely connected to the ongoing scientific investigation. Most probably, the virtual reconstruction will serve to simulate and test interpretative hypotheses. Such a kind of VR application can be addressed to expert users with in-depth knowledge of both the cultural context and complex digital interfaces.

3.2. Holistic Representation for the Public

When the virtual environment is addressed to the public, the experience design is completely different. The holistic approach and narration prevail over analysis, and the digitization techniques are calibrated according to the level of detail and accuracy required by the effective experience. Indeed, this experience is not only conceptual, but also involves sensory and emotional aspects. Preserving the detail and realism of the 3D representation and the quality of the information deriving from the data acquired in the field is important to maintain continuity and consistency between knowledge and communication. The realism of the three-dimensional graphic representation enhances the sense of presence within the virtual world, because it relates to the degree of sensory and cognitive interaction [56].

However, in a virtual reality environment conceived for communication needs, what is fundamental is the holistic vision. This global approach better involves the user conveying the identity of a cultural context, its historical and social background, the way it was used, life dimension, and transformation over the centuries.

For this reason, virtual reconstructions addressed to the public are encouraged to show not only what is certain, on the basis of remains and evidence acquired on the field. To give a better overview, they usually propose visualization of what is hypothetical on the bases of historical and iconographic sources and contextual and typological comparisons. The virtual reconstruction goes beyond the purpose of recreating an artifact in its integrity, also aiming to reenact the social and cultural context and narrate a piece of its story.

The virtual reconstruction could be considered an extension of the stylistic restoration concept (see Section 1), and it still needs to respect the scientific criteria declared in the London Charter and in the Seville Principles.

Usually, a virtual reconstruction starts from elements that are still existing on the original site, or that have been moved from the original site to the museum. These elements, once digitized, can be included in the virtual reconstruction as they are, or after an intervention of virtual anastylosis to improve their legibility. They are then integrated in a wider reconstruction also including hypothetical elements coming from iconographic sources, typological similarities, literary sources, architectural rules, and proportion theories.

Thanks to such a virtual reconstruction, it is also possible to contextualize objects coming from the site, if archaeological documentation is available, thus making their function evident for the public.

3.3. Narration

Many academics and museum curators have, for years, belittled the role of virtual reconstructions by relegating them to mere visualization and denying their role in the interpretative and cognitive process. They usually preferred a “neutral” approach with artefacts, avoiding telling or suggesting to visitors anything beyond the pure evidence. However, avoiding interpretations is not a neutral choice, and it does not preserve the objectivity of information: if a visitor is left alone and without any supports, he/she will be free to fall into arbitrary, false, and erroneous deductions. The omission does not produce positive effects, but can cause greater damage to the comprehension.

Narration, as virtual reconstruction, gives form to abstraction and is a fundamental support to raise engagement, motivation, self-identification, and memorization [57]. In the last ten years, several surveys have been carried out, at the international level, on museums visitors experiencing digital applications. Narration always emerges as the most powerful component of the experience, and secondly interaction (the latter especially for young visitors) [34].

A story is something much more powerful and emotional than a simple description. It combines verifiable reality (certain and circumstantial contents and facts regarding a specific artefact) and imagination, especially when dealing with past events. These imaginary parts should take root in the historical background and cultural context related to the artefact. Uncertainty imposes the necessity to make interpretative choices, both in graphic representation and in storytelling (personages and occasional events).

Some strategies and storytelling techniques can be adopted to manage the problem of uncertainty and to make it explicit to the public, expressing some dilemma and bringing “on the stage” the complexity of the interpretative process.

For at least ten years, semantic mapping systems have been introduced in virtual reconstructions to suggest levels of reliability, eventually associated with metadata. For this purpose, a color coding is applied to the model.

However, a mapping of the different levels of reliability of storytelling has never been attempted. It would be very interesting to experiment with such a coding method, to distinguish certain content (based on evidence) from possible content (based on handed down narratives or contextual elements) and completely evocative content (used to create transitions or to enhance characters or facts in the fiction).

In the cinematographic arts, the “color script” is used as a sort of score, following the storyboard, in which the different moments of the story are associated, by means of color coding, to the prevailing emotion aroused by the contents (joy, fear, emotion, terror, sadness, love). In this way, creators have an immediate overview of how the story affects human emotions, and they can verify and maximize the level of user involvement.

Such a coding system could be tested to identify and document the different levels of reliability in the proposed story. This kind of visualization has never been attempted, and we have advanced a concrete proposal in Section 4, referring to the Kunagota Sword case study.

4. Making Scientific Process Explicit to the Audience: Strategies and Case Studies

How can we make the scientific process explicit and at what level of depth?

Who is our communication addressed to? Experts? The general public? In what context and place is it presented? An excursus of our experiences on this topic will be presented in Section 4 to discuss the different strategies experimented with and evolved over almost twenty years of research, to make the interpretation process explicit to the public. A recurring methodological approach has been applied, based on the integration of bottom-up and top-down sources and multi-layered representation. In most cases, the final aim was to create virtual museums.

4.1. Livia’s Villa

The project Livia’s Villa Reloaded was born in 2013. It was a follow-up to the “Virtual Museum of Ancient via Flaminia” developed by CNR ITABC in 2006–2008, and it reused the original digital contents [58]. The purpose was the documentation and valorization of the archaeological landscape along the Flaminia Roman consular road, and Livia’s Villa was one of the sites taken into consideration. It is a very important suburban roman villa that belonged to Livia’s (wife of the emperor Augustus) family since the late republican age; then, it had a new important architectural phase during Livia’s and Augustus’ life (first century BC–first century AD) and survived until the Constantine age (fourth century AD). A virtual reality application was realized for the public of the Roman National Museum, Diocletian’s Baths. Users were able to explore 3D models of (1) the archaeological landscape (as it is today); (2) the reconstructed landscape (as it was in the Augustan age); and (3) the interpreted diachronical landscape, resulting from the transparent overlapping of the archaeological remains and their stylized reconstructions. Every reconstruction was performed starting from the topographical survey of the existing structures. The interpretative process used for reconstructing the villa has been made explicit in the interactive museum application by means of:

• educational movies that can be activated while exploring today’s villa, guiding the users through the visualization of the different chronological phases, archaeological layers, and sources considered in top down processes;
• panels that can be found in the reconstructed rooms and gardens, showing typological comparisons; and
• short captions automatically appearing during the exploration of each room of the villa, regarding the level of reliability of its virtual reconstruction, distinguishing architectures and decorations (Figure 3).

4.2. Teramo Virtual City

In 2010, the “Teramo virtual city” project was presented to the municipality of Teramo (Italy). Its purpose was documentation and valorization of some historical and archaeological sites, urban paths, museum artefacts, and intangible culture through multimedia and virtual reality technologies [59]. Connections were created among different cultural contexts and contents in the urban space, in the archaeological museum, and within virtual web communities. In particular, the S. Anna archaeological site was considered one of the most relevant. Here, the ancient Cathedral of Santa Maria Aprutiensis was built, perhaps in the seventh century, on the remains of an important roman domus dated to the first century BC. Today, just a few remains of the two structures are visible, because the site was destroyed in 1156, when most of the town was burst by the Norman Roberto count of Loretello. The character of a virtual architect Virtuvius (in memory of the famous roman Vitruvius) is the leading soul of the applications: he guides the users and shows them how a monument can be virtually built upon its ruins, creating columns, walls, and roofs. However, he also explains to the public that the reconstruction process is often problematic, and the discussion of the interpretative process is a fundamental issue to let the public understand the context (https://vimeo.com/156160530, https://vimeo.com/156170235, accessed on 8 April 2021).

Regarding the virtual reconstruction of the Roman domus, a comparison between the present condition of the archaeological remains and their past 3D reconstruction has been implemented in a multimedia application. Some significant points of view have been chosen to superimpose the two representations, favoring a better and immediate comprehension of the reconstructive process. Moreover, an accessory visualization level has been included, where the virtual reconstruction appears rendered in false colors, defining the different reliability levels of the proposed reconstruction. Red is used for archaeologically existing and currently visible elements; pink is used for archaeologically existing but not currently visible elements; blue is used for hypothetical elements based on structural data and archaeological records; and light blue is used for hypothetical elements based on comparison with similar structures (Figure 4).

4.3. Giotto’s Colours

The exhibition “Giotto’s colors” was opened to the public in Monte Frumentario Palace in Assisi in 2010 to celebrate the end of the restoration work carried out on Giotto’s frescoes, painted in the Upper Basilica of St. Francis of Assisi at the end of the 13th century. This long restoration followed the earthquake that occurred in 1997. In the exhibition, experts coming from the High Institute for Conservation and Restoration presented 27 panels (1 × 1 m) with scenes as they might have appeared in Giotto’s time, with brilliant colors. The intervention was made painting with material colors on printed photos of the original scenes (in their actual state of preservation) [23].

In the exhibition context, a virtual environment has been created from the scene “The Rule Confirmation”. The scene has been designed in 3D, starting from an accurate study of Giotto’s space, characters, and proportions, and the models have been “mapped” using the restoration of the original artist’s painting. On this basis, two virtual reality installations have been carried out. In the first one, it is possible to virtually enter into the Giotto fresco. The 2D painted scene is brought to life in a 3D environment: characters are animated and represented while performing the action painted by the artist. Visitors can interact within the virtual space just using body movements, and the scene painted by Giotto becomes an “impossible” place of immersive experience, open to multi-sensorial narration and participation.

In contrast, the approach of the second installation is descriptive and interpretive. It focuses on the spatial and perspective investigation of Giotto’s fresco and compares it with the 3D reconstruction based on Renaissance perspective rules. (Figure 5).

The primary objective of this experiment is to involve the observer in the scene painted by Giotto and to enable the visitor to feel and understand the message that Giotto was communicating [60].

4.4. The Regolini Galassi Tomb

Etruscanning is a European project (2011–2013) born from the cooperation of museums and research organizations. It was aimed at the digital acquisition, restoration, and 3D reconstruction of Etruscan tombs and artefacts and their dissemination through innovative VR systems. One of the main objectives was to digitally relocate the artefacts discovered in the tombs (and now preserved in museums) within their funerary context and in their presumed original position. This was the case of the famous Etruscan Regolini Galassi tomb, located in Sorbo necropolis (Cerveteri, Italy), and its precious funerary goods, preserved at the Vatican Museums [61]. Starting from the digital acquisition through range-based and image based techniques, the original context was reconstructed. The reenactment was mainly based on the iconographies by Canina, Grifi, and Hamilton, who were the first to draw the tomb (1838–1841) and its grave goods, as they could have appeared to the discoverers (they made drawings after the removal of the funerary goods, based on recent memory). Besides the virtual reconstruction, a virtual restoration was needed, especially for bronze objects, in order to simulate their ancient appearance. This activity was performed under the scientific supervision of the Vatican Museums. Similarly, engraved and embossed decorations of fragmented objects were integrated where possible, following iconographic comparisons with objects of the same period and cultural context or influence. An interesting example is a silver and gold patera that was 70% intact. Its decoration, in Egyptian style, consists of soldiers with similar shapes, on foot or on horseback in a repetitive sequence, distributed on concentric bands. In this case, the missing parts were recreated by analogy by copy-pasting the existing figures. The remaining missing parts were integrated by comparing this patera to a very similar one in the National Museum of Antiquities, Leiden, Netherlands, coming most probably from the same workshop. The completed line drawing was used to simulate the engraving and embossing processes, generating a “depth map”, represented as a grayscale image in Photoshop (Figure 6).

A virtual installation has been created in the Vatican Museums with such a reconstruction where the public can experience a multisensory experience with powerful storytelling. As the installation is located close to the exhibition room presenting the funerary goods, virtual and real contents are juxtaposed, and they can be easily compared. The alternation and exchange between real and virtual experience translates into a cognitive anakiklosis, where each step produces a difference, strengthening comprehension and learning.

4.5. Lucus Feroniae

In the context of the project Tiber Valley Virtual Museum [62], a virtual reconstruction of the Roman colony of Lucus Feroniae was fulfilled between 2011 and 2015. The result was implemented in a virtual reality installation, placed in the Etruscan National Museum of Villa Giulia (Rome, Italy), and presented to the public. As already mentioned, the reconstruction workflow started from the digital acquisition of the archaeological site and continued through the integration of bottom-up and top-down approaches. The output was an interactive installation composed by three large adjoining screens, arranged in a semicircle in front of the user.

The application, currently still running, narrates the story of the Tiber Valley and its archaeological sites by means of different visualization paradigms.

Indeed, the visualization alternates between a unique panoramic view and split views (on the three screens). When the narration is interpreted by actors playing the role of ancient characters, the visualization is panoramic, to give users a greater sense of immersion and involvement. In contrast, during the guided exploration, an analytic vision prevails, and the contents on the three screens differ to encourage critical interpretation in the visitors: on the left screen, the archaeological site is visible, while on the central one, its virtual reconstruction can be enjoyed. The exploration is in first person, and both visualizations are synchronized thanks to camera tracking techniques. On the right screen, the virtual reconstruction is shown in third person from an aerial point of view. This view allows the user a better overview of the entire context and his/her position within the site. Furthermore, the user can compare in real time the current archaeological site and its possible ancient reconstruction. In such a way, comprehension of the function and meaning of fragmentary archaeological elements is immediate and the principle of recognizability is guaranteed (Figure 7).

4.6. Early Medieval European Collections: The Cases of Mytilene Treasure and Kunagota Sword

As discussed in Section 3, in the creation of a narration and certain and circumstantial contents regarding the artefact are combined with plausible and probable ones. An example will clarify this assumption. In 2015–2019, the European CEMEC project (Connecting Early Medieval European Collections) gave us the opportunity to conceive and realize a holographic showcase that has been shown in the context of an itinerant exhibition among the European museums’ partners of the consortium. This showcase has been designed to integrate the original artefact with virtual animations using Pepper’s Ghost effect [63]. The purpose was to simulate the sensory dimension around this object, showing and telling some meaningful moments of its life and function, using narration. The objects have been digitized in 3D, and virtual restoration has been attempted and shown in the showcase through animated holograms.

In the case of the trulla, a silver basin from the Byzantine Mytilene Treasure, dated to the seventh century AD, a digital restoration aimed at reintegrating its fragmented shape was made. From the digitally restored version, a physical replica was created with 3D printing technologies. The replica was used to shoot a short movie with real actors and show how the trulla should have been used in the past, which was to keep water for bathing, handled by the ancient owner, whose role was played by an actress (Figure 8).

Another artefact from the project was the Kunàgota sword, preserved in the Hungarian National Museum of Budapest, for which we have created a story. In this case, starting points were [64]:

• Certain information related to the objects itself: The sword was discovered in an Avar tomb of the seventh century AD, near the village of Kunàgota (Hungary). The sword was part of the funerary good of an Avar warrior, and its scabbard was adorned with golden sheets showing figures coming from the Byzantine iconography and style. Such kinds of decorations and figures can be often found on precious Byzantine caskets, of which there are many examples. The golden sheets have been removed from the original object and broken, to be used to embellish the scabbard (Figure 9).
• Certain information regarding the general historical context of the Avars: They were a population of warriors, often fighting against the Byzantines; they practiced looting and believed in the afterlife, the Blue Sky of “Tengri” (according to a shaman tradition of central Asia from where they originated).
• Plausible and probable contents: The style and the dimensions of the golden sheets let us suppose that originally they could belong to a precious small Byzantine casket, for instance to contain jewelry. It is historically plausible—even if not certain—that the Avars took the casket during a looting, after a battle against the Byzantines. As the golden sheets were broken with scant attention paid to the integrity of the figures, it is also possible to suppose that this work was made by an Avar goldsmith who did not understand the identity and the meaning of the figures. The sword was really used for war, so it can be assumed that the goldsmith made this work when the Kunàgota chief died, to adorn his sword in gold, before burying him.

Merging these three levels of contents, a visual story has been drawn, with soundscapes and characters. Events have been dramatized and represented in short chapters: the small casket, containing jewelry, is in a rich Byzantine house; the battle is going on and the casket is taken away by the Avars; the Kunàgota chief is dying on his deathbed and he talks to his son, asking him to make the sword adorned in gold and put it in his tomb; the goldsmith in his workshop, with his assistant, cuts the sheets and fits them on the scabbard, trying to understand what kind of figures they are (Figure 10); finally, the user can see the tomb connected to the Blue sky of Tengry (represented according to the original Avar iconography), from where the spirit of the Kunàgota chief speaks to us for the last time (https://vimeo.com/236305120, accessed on 8 April 2021).

The story combines certain and plausible facts with fictional elements. To suggest the reliability of each segment of the audio-visual narration, we have created a colorscript where different colors are used to define the different levels of certainty/uncertainty, according to the available sources (Figure 11). In the reliability colorscript, these symbolic colors are superimposed to the final rendering to give an immediate overview to the scholars. Such a method has never been attempted before, and we propose it here for the first time. We think it could be a good practice, also because it comes up beside the well-known method using colors to put in evidence the reliability of the different elements in an architectural virtual reconstruction.

4.7. The Roman Villa of Aiano

The roman villa of Aiano is an important archaeological site close to San Gimignano (Italy) dated between the end of the third and the seventh century AD. Since 2005, the villa has been excavated by an Italian-Belgian mission coordinated by the UCLouvain as part of the international project “VII Regio. The Elsa Valley during Roman Age and Late Antiquity’’. During the activities, a virtual reconstruction of the so-called trefoil in the fifth century was performed. 3D modeling was used as a tool of investigation and visualization to restore the legibility of the archaeological remains, characterized by monumental architecture and decorations, and, above all, to get a deeper understanding of archaeological sites. During the process, the Extended Matrix, ref. [51] a formal language based on the archaeological stratigraphic approach, was used as a tool to keep track of virtual reconstruction processes and guarantee intellectual transparency. All the sources and paradata led to the archaeological interpretations of the roman villa, which also connected to the 3D model. Three different models have been produced (Figure 12). (1) A digital replica: a 3D model of the hall in its current state of preservation (2018) scanned with image-based modeling techniques. (2) A proxy model: a semantic 3D schematic reconstruction that the querying of information and the sources used in the reconstructive process; it also allows the visualization of levels of reliability using different color coding to distinguish extant structure from virtual anastilosis. This color coding declares, through a graphic expedient, the hermeneutic limit of reconstruction: red identifies extant structures; blue identifies virtual reconstruction based on archaeological evidence found in situ; yellow identifies virtual reconstruction based on archaeological evidence found out of their original context; green identifies virtual reconstruction based on testimonies and comparisons. (3) A realistic virtual reconstruction that simulates the building in its formal unity and in its hypothetical aspect, aimed at improving the legibility of the building, which is unique in its kind, and therefore even more difficult to comprehend, even for experts [65].

4.8. The Forum of Augustus

A rapidly evolving sector, in the field of Virtual Heritage, is represented by video games, as evidenced by the last calls of Creative Europe and Horizon 2020. Video games are oriented toward a greater integration between heritage and creative media to enhance user involvement and entertainment. Between 2018–2019, the CNR ISPC participated as a partner in the project Reveal (revealvr.eu), funded under the European Horizon 2020 program and supported by Sony Interactive Entertainment. The project was created with the aim of using the game platform Sony Play Station (www.playstation.com, accessed on 8 April 2021) as a tool for disseminating knowledge of the historical–artistic and archaeological heritage of Europe. The project developed a series of cognitive strategies that can be used by game developers. Among the demonstrators realized, the case of the video game entitled “A night in the forum” is significant, entirely set in the Forum of Augustus, in Rome, in the first century AD. Interacting in its virtual reconstruction, the user plays the role of an ancient Roman (the guardian of the forum) and can deepen his/her knowledge by exploring digital replicas of cultural assets and performing historically authentic activities, according to the pedagogical paradigms of environmental storytelling and learning-by-doing [66].

Virtual reconstruction here is a great didactic tool, as it improves cognitive processes by making historical and archaeological data easily understandable to the public. However, the virtual reconstructions of the past required a great effort to ensure the consistency and reliability of the reconstructive hypothesis. The reconstruction work followed the same method presented in the above-mentioned project of the Roman villa of Aiano. The reconstruction has been elaborated after a process of analysis and interpretation of the sources, in collaboration with the staff of the Museo dei Fori Imperiali—Mercati di Traiano. The reconstructive model has been designed starting from the three-dimensional survey of the archaeological site and from some architectural fragments that have been relocated with a process of virtual anastylosis (Figure 13 and Figure 14). The missing parts have been completed by referring to the numerous iconographic testimonies, historical–archaeological documents, and stylistic comparisons [67].

Not being able to show the whole scientific process inside the game, principles of recognizability and transparency of the data are guaranteed by using graphic and narrative workarounds. During the exploration within the video game, the player explores the site in its current state of preservation before viewing the reconstruction, to distinguish the real data from the interpreted one. The immersive visit with a VR headset allows the user to have a full-scale perception of the surrounding space. He is thus able to experience and physically perceive the volumetric relationships, distances, and magnificence of the architecture. Within the application, each game task provides the user with information related to the public spaces of the forum, the activities that took place there, and the function of the artefacts found. All the information gathered, together with the artefacts with which the user interacts, is based on real archaeological findings that are narrated by a guiding voice.

4.9. Keys2Rome

Keys to Rome has been an international exhibition on Roman culture held simultaneously in Rome, Sarajevo, Amsterdam, and Alexandria between 2014 and 2015. It was organized by V-MusT, the Virtual Museum Transnational Network, and curated by archaeologists, art historians, architects, computer scientists, and communication experts. The exhibition used immersive technology to present and connect these regional cultures within the Roman Empire. Combining real artefacts, virtually restored 3D objects, and virtual environments, visitors had the possibility to travel back in time to search for lost objects and discover more about the lives of their owners [68]. The multimedia technology infrastructure was developed to improve the legibility of the artefacts exhibited and help visitors to understand their original appearance and functions. Two installations in particular allowed visitors to view the digital restoration of some artefacts in real time.

The former was an augmented reality application developed by the Fraunhofer IGD with the collaboration of CNR. The application, just by pointing at a fragmentary object with a smart device, allowed users to see on the screen the hypothetical virtual reconstruction of the object itself (augmented reality). Furthermore, by clicking on a specific tag, the app also provided storytelling (Figure 15).

The second application, developed by the Institut National de Recherche en Informatique et Automatique (INRIA) [69] and called “revealing flashlight”, was based on a projection mapping technique. The application allowed users to discover the original colors and drawings of fragmented marble slabs, which were part of the decoration of the Colossus Hall in the Forum of Augustus (Rome). The application worked as follows: the user pointed to an area of the marble fragments with his/her finger; an infrared sensor, connected to a PC and a projector, tracked and interpreted the movements of the hand; finally, a beam of light was projected only in the area indicated by the user, reproducing the original colors onto the artefact (Figure 15).

These applications allowed the fragments to be digitally restored in real time, giving back formal unity and legibility to the artefacts, while guaranteeing authenticity and recognizability.

Indeed, projection mapping on the original artefact is a very powerful technique for virtual restoration, based on the only use of light. Starting from a digitization of the material artefact on which the projection is directed, this technique is able to resolve the fragmentary nature of the physical consistency of the artefact, making it legible by restoring missing portions and decorations that are today no longer visible.

5. Authenticity in the Virtual Heritage Era

Cesare Brandi proposed to analyze the artwork on two levels [70]:

(1)

on itself, on its own structure;

(2)

on the moment in which it is accepted in consciousness.

“A historical building is constituted not only by a quantity of matter, but also by an artistic concept that is immaterial. Unlike matter, the artistic concept does not age over time: it is perceived by human consciousness, and this can happen only in the present. Therefore, the artwork always exists in the present. Consequently, recognition needs to be established every time the artwork is contemplated, even in terms of its restoration”.

This concept helps us to understand that perception, recognition, and enjoyment of the artwork are in continuous evolution over time, following the tendency of social cultures and values. The artwork is thus historicized at two separate moments: (1) when it is created by the artist, and (2) when it is recognized by an individual’s consciousness in the present. Undoubtedly, digital practices are changing our relationships with cultural heritage, also in terms of perception, evaluation, and expectation.

Of course, the concept of authenticity of the artwork is fundamental in our tradition; it is the pillar of Brandi’s theory. However, today this concept needs to be again pondered: what is the effect that the virtual intervention impresses on the artwork? What is its limit? Does virtual restoration modify the perception, the imaginary projections, the attribution of meanings, or the expectations related to a cultural heritage? Does virtual restoration effectively save the artwork’s authenticity? How can the conservative approach, related to the authenticity preservation, be combined with the integrative approach, related to the transmission of its aesthetic value and meaning? To what extent and until what limit is such a mixture permissible? Virtual restoration allows an evolution of the discipline in the direction of integration to be pursued. Let us think about Domus Aurea, the house of the roman Emperor Nero, in Rome. Today, it is little more than a dark cellar, while originally it was an architecture that celebrated the triumph of light and brilliance. Water, light, and shiny materials were the design tools of ancient architects. Today, they can be perceived and understood only through a virtual reconstruction recreating a sensory immersion [71]. If we realize such an extended virtual reconstruction, we can transmit the essence of the monument, its deep and authentic meaning, without touching and compromising its material authenticity as it appears today.

The wide diffusion of virtual restoration practices in the communication to the public could progressively reduce the ability to “read” and understand the fragmentation of cultural heritage. Is this a reasonable worry? Or is it completely unfounded? And what about the traditional concept of the “uniqueness” of the artwork? Maybe for young generations, and for some cultures in the world in the globalization era, the distinction between real and virtual is not so clear and sharp as for elder scholars and theorists. The relationship with the material aspects of cultural heritage that traditionally have imposed a very strict and inflexible stance seems to become increasingly blurred. The digital domain is extending the concept of restoration to embrace aspects related to technique, perception, and attribution of meanings/values [72].

Indeed, society is evolving toward a democratization of culture, promoting the accessibility and understanding of heritage to an ever-widening public. A couple of examples will help us to think about these aspects.

In the fall of 2006, the Louvre Museum reached an agreement with the Giorgio Cini Foundation of Venice to grant Adam Lowe’s company Factum Arte the permission to digitize Veronese’s immense painting “Le Nozze di Cana” (67.29 m²), to create a reproduction of extraordinary quality. In fact, since 11 September 1797, the artwork has stood in the same room as the Mona Lisa at the Louvre Museum, seized as war booty from the Venetian monastery of San Giorgio Maggiore by French commissioners of Napoleon’s army. Through a 3D acquisition at very high resolution both of geometry and texture, a perfect copy has been created and returned to the original monastery in Venice, where it was originally located (Figure 16).

The famous French curator and art historian Jean Clair poses an interesting question: is it possible that the Venetian version, even though it clearly proclaims that it is a copy, is actually more original than the Parisian original? He reflects on the importance and role of contemporary copies [73]: “What is better: an original that, once deposited in the Museum, has lost its destination, or a copy that, recovering the destination of the original, finds again its own meaning? What is better: the distorted and deteriorated work, badly illuminated and completely out of context, or the copy, of superior quality to the original, whose re-contextualization restores its reason for being? […] The original possesses the quality of the relic. […] because we know, or we think we know, that it was done by the hand of the same artist. […] Detached from their origin and their function, the works of our museums have become our idols”.

A much more striking case is represented by the 1:1 scale reproduction of the Sistine Chapel realized to be exhibited at Mexico City and later in other Latin American countries, in an itinerant exhibition that started in 2016 [74]. It was placed at Plaza de la República, in Mexico City, not far from the monument celebrating the world’s first socialist revolution. This project has been approved by the Vatican Museums, whose implementation alone cost $2.4 million. This is a vibrant proof of how the reproducibility of artworks allowed by more and more sophisticated digital technologies can enormously increase the art market in the decades to come. The physical reconstruction of the Sistine Chapel was realized by 280 Mexican architects, engineers, set designers, artists, photographers, and construction workers. The director and producer Gabriel Barumen states that it is a faithful and multisensory copy, in which even the aromas, sounds, and lighting are similar to those experienced by visitors of the Vatican Museums. The entire chapel was reproduced on a canvas after scanning through 2.7 million micro-photograms, each measuring 3 cm², digitally assembled into 2800 final photographs. The team of technicians made the acquisition for 170 nights, during the Museums’ closing time. Differently from the previous example of Veronesi’s “Le Nozze di Cana” aiming at a re-contextualization, the purpose here is the opposite: to de-contextualize in order to offer a spectacular version of the masterpiece of the Renaissance.

The examples that we should present are many. Everyone can formulate his/her own opinion about such kinds of interventions, dealing with the meaning and essence of the art. Of course, new economies will move in this direction in the future globally. A fundamental question to which there is not yet a common solution is: to whom belongs the copyright of these reproductions?

The hope is that the industry of facsimiles will follow an ethical approach, going beyond the goal of creating mere spectacles for millions of dollars. Reproductions can represent an opportunity, if accompanied by the necessary educational content and information about how they have been realized. For instance, they can support preservation of the original, especially if the tourist pressure is unsustainable, reducing the time of permanence of visitors in front of the original and offering a good alternative experience. Moreover, they can contribute positively to approaching the experience of art to those people who might otherwise never see the original masterpiece, and this experience can indirectly contribute to bring peoples and cultures closer together.

6. Conclusions

What is discussed in the paper shows how, in processes of virtual restoration and virtual reconstruction, the digital component is almost always connected to the physical one. They exchange information in synchronous or asynchronous mode. Both contribute to creating a perceptive and interpretative space of experience. The crucial question the paper tries to find an answer to is: do the digital multiplications of the original cultural heritage, faithful or re-interpreted, hurt respect for the original and its appreciation? Do they trivialize or enhance its value? We believe that they can assume a relevant role in education, comprehension, and deep contact with cultural heritage, translating in a wider cultural transmission. The condition for this to occur is a good methodology, in the respects of recognizability, transparency, and reliability criteria, taking into consideration the needs of the audience to which they are addressed, Virtual tools can help scholars to analyze, interpret, advance different hypotheses, and make simulations. Additionally, they let the public understand past cultures with an augmented awareness and sensibility, favoring new processes of interaction and critical thinking. Real and virtual should no longer be considered as opposed, but as a “continuum”, bringing values to the cultural and human experience. They include the observer, who becomes an active participant with a leading role.

Therefore, communication is no longer one way (the artwork towards the spectator): it becomes bidirectional or multidirectional. Experts and audiences have improved tools at their disposal to interact with the tangible and intangible artwork’s elements and to give new form to abstractions. Real and virtual together can generate an evolutionary and creative process, social interaction translating into an evolving knowledge.