Bringing the Illusion of Reality Inside Museums—A Methodological Proposal for an Advanced Museology Using Holographic Showcases

Abstract

The basic idea of a hologram is an apparition of something that does not exist but appears as if it was just in front of our eyes. These illusion techniques were invented a long time ago. The philosopher and alchemist Giovanni Battista della Porta invented an effect that was later developed and brought to fame by Prof. J. H. Pepper (1821–1900) and applied in theatrical performances. The innovation nowadays consists in the adopted technology to produce them. Taking advantage of the available digital technologies, the challenge we are going to discuss is using holograms in the museum context, inside showcases, to realize a new form of scenography and dramaturgy around the exhibited objects. Case studies will be presented, with a detailed analysis of the EU project CEMEC (Connecting Early Medieval European Collections), where holographic showcases have been designed, built and experimented in EU museums. In this case, the coexistence in the same space of the real artifact and the virtual contents, and interior setup of the showcase, its dynamic lighting system, the script and the sound, converge to create an expressive unity. The reconstruction of sensory and symbolic dimensions that are ‘beyond’ any museum object can take the visitor in the middle of a lively and powerful experience with such technology, and represents an advancement in the museological sector. User experience results and a list of best practices will be presented in the second part of the paper, out of the tests and research activities conducted in these three years of the project.

1. Introduction

1.1. The Virtual Museum in The Real Museum—Need for a Better Integration

In the last 10–15 years, the domain of virtual museums (VMs) has seen great improvement in terms of technological solutions, graphic implementation and visualization, interaction interfaces, and methodological pipelines [1].

However, especially when VMs are conceived to be included and accessed inside real museums (or exhibitions), digital applications often appear to be insufficiently connected and harmonized with the real collections and the museum’s space.

Multimedia and virtual contents are confined in their own ‘frame’ of exposition, either a screen, a multi-projection set, a head-mounted display, or a tablet. In such cases, the public can live experiences that are more or less immersive and engaging. Nevertheless, these experiences are optional and ancillary in relation to the visiting path along the collections, with whom they are not strictly related. For instance, temporary exhibitions commonly include introductive movies at the beginning of the visit, and sometimes tablets at the end, with the aim of offering additional information about the topics on show. However, the setup of an exhibition and the communicative approach of the collections remain very traditional, supported by printed panels and small captions located beside the objects. In that condition, common visitors can only look at artifacts and admire their aesthetic consistency. They cannot ‘experience’ anything of the objects’ contexts, of the stories and people that are ‘behind’ [2].

Even in the case of more sophisticated installations, like VR applications using novel interaction paradigms or offering truly immersive experiences, the dialogue between digital contents and real artifacts can often be weak, sometimes even hostile [3,4].

The majority of visitors are usually immediately attracted by digital technologies, especially if they are new, because they are able to solicit their senses and curiosity [5]. Playing with them is a challenge because the dynamism of multimedia is perceived as something that interrupts the silence and the repetitiveness of the museum visit experience, making it more pleasant, unpredictable, and fun. This explains why so many visitors, entering in a room with audio-visual/virtual installation on one side and common showcases with real artifacts on the other side, are immediately attracted by the digital rather than by the real contents. This behavior has been very often observed and recorded in occasion of several surveys we have carried on in museums and exhibitions, aimed at evaluating the users’ interaction with real and digital contents [6,7].

On the other side, the attention towards digital contents decreases rapidly if they do not offer a good connection with the real contents of the museum and if the storytelling is not really engaging [8,9]. Technological innovation is not sufficient to hold interest for a long time. A visitor entering a museum is going to live a complex and demanding experience, conceptual and physical (a long path, several rooms, showcases, a huge variety of objects coming from different ages and places), and he does not want to be distracted for too long by an installation that is not completely involving and not coherent with the collection along the way.

In other words, it seems that today more and more museums are embracing the digital challenge, but there is still not a real awareness of how to let virtual and real contents work together to enhance the experience of the visitors in and with the museum’s collection.

Real and virtual continue to be juxtaposed, but not combined together, in order to produce a powerful experience of mixed reality.

Consequently, despite the great contribute that multimedia and VR could offer to the museum’s sector, the exhibited collections still suffer from lack of contextualization, of stories, of sensorial immersion and anthropological dimensions, joining past and present together.

A new conception of the museum’s space is needed, and a new curatorial practice as well (from both sides of real and virtual museums) to create such an integration, where users, collections, and stories are complementary and interconnected. Such interconnection requires the cooperation of different professionals: content experts, designers, developers, creatives, technicians, cognitive scientists, artists, experts of usability [10].

The museum would thus become a place of enjoyment, learning, and social inclusion, reflecting local, global, individual, collective dimensions, and perspectives [11].

Starting from these premises, the intent of creating a ‘dramatic dimension’ inside the museum’s space in order to push the user’s sensory involvement with objects and stories, has brought some researchers and creatives to test and apply an ancient representation technique of being able to interact with real museum’s objects. This technique consists in the Pepper Ghost effect, living inside the showcase, in presence of the artifact, producing an effect of mixed reality. As National Research Council and E.V.O.CA., a company focused on communication products about Cultural Heritage, we are working with this kind of approach in the framework of the European project CEMEC (Connecting Early Medieval European Collections) which is still in progress (Creative Europe programme, 2015–2019).

The selection and use of such an approach derives from considerable experience in the domain of VMs, accompanied by a deep reflection in the museological domain. We have experimented and developed several kinds of digital applications for museums: audio-visual contents visualized on screens along the paths of visit, beside the objects, with a low level of interaction and characterized by an educational approach; VR environments located in secluded and dedicated spaces, in absence of real objects. Especially in this second case, the experimentation has been very long and complex, both with interaction and with interface design and storytelling. Several surveys on audience engagement with digital installations have been carried on in order to understand the impact of such experiences [12,13]. What emerged from these evaluations is that visitors mostly love to be engaged and ‘embodied’ in stories, to play as protagonists, but, also, they love to be guided and not abandoned to themselves. Free and unconditioned interaction in VR is something too complex and frustrating for visitors, especially if they do not have a preliminary knowledge of the subjects, and if they want to follow a narration.

Narration and emotions play a crucial role in the public’s engagement, in cultural transmission and memorization [14,15]. For this reason, we are working on the creation of new narrative ‘forms’ combining different paradigms and techniques, coming from VR, cinema, theatre, serious game, augmented reality, digital art. Each ‘language’ and paradigm can suggest an inspiration, can open a vision, converging in the creation of something new for the next generation of VMs. These experiences led us to experimentation with holographic showcases, where perception, mixed reality, and dramatization are the essential topics.

The basic idea of a hologram is an apparition of something that seems tangible, in an empty space. Something that does not exist but appears as if it was just in front of our eyes. Indeed, this technique of illusion was invented long time ago, in theatres; the innovation of using it today consists in employing technology to produce the same effect. Therefore, taking advantage of the available digital technologies in the current market, the challenge is to realize a new form of scenography and dramaturgy around the museum’s objects, with a calibrated interaction according to the real needs and suitable for all targets, with a simple technology, compatible with the museum’s preservation and maintenance needs.

1.2. The Box of Stories—A New Proposal towards an Improved Museology

In the CEMEC project, we have conceived and experimented holographic showcases, working on the definition of a coherent communicative format, where (a) integration of real and digital contents; (b) structure; (c) materials; (d) interior setup; (e) illumination; (f) perception; (g) audio-visual grammar, (h) narrative approach and dramatization; (i) scalability, are designed as a whole. We have evolved a specific research on the communicative approach, engaging experts in humanities, museology, psychology, curators and artists, engineers, working together and supported by the user experience evaluators This integration and the obtained results that have helped us to define some general guidelines, constitute the innovative value of our proposal.

The so called ‘box of stories’ has been conceived as a small theatrical stage equipped with controls for the direction and synchronization of the individual stage devices: lights, audio speakers, scenography, projections. Everything is managed and controlled by a software specifically developed. As already told, the virtual contents are projected in the same space of the real object and bring it back to life, to its sensory dimension, following a ‘dramaturgy’. Real and symbolic scenarios can be evoked and represented, accompanied by emerging personages, voices, and soundscape.

Why should we use holograms to tell the story of an object, rather than a common 2D screen or even a mobile device with a multimedia? It is not only to surprise and produce an astonishing reaction in the visitors. Through the hologram we change the traditional paradigm: if we include the real artifact inside the holographic showcase, the attention still remains focused on it. It stays as the real protagonist of the story, not only on a conceptual level but also in its physical consistency. Not its virtual replica, but the original itself, is the center of our attention along the time of the whole experience: all the virtual animations, the fragments of stories originate from its real figures and details. We can mount and dismount, virtually restore, move, and transform the object, overlapping virtual projection on the real artifact, and playing with lights, thus creating an experience of mixed reality.

Through holograms we do not immerse ourselves in virtual reality using particular viewers/visitors, but it is the virtual reality that enters our space. Our reality becomes an augmented reality. Because of this integration, the museum’s object becomes alive, enforced and not weakened by multimedia. This is not a marginal difference, because it greatly influences perception, engagement and comprehension, favoring a deep relation between the visitor and the observed object.

After a brief state of the art and a discussion about the utility and potentialities of holograms to improve the museological sector in Section 1, Section 2 of this paper will present some technological solutions for holograms and diverse examples of the related work done in museums and exhibitions in recent years.

In Section 3 the EU CEMEC project will be analyzed in depth, in relation to the creation of a specific holographic showcase, a ‘box of stories’, based on Pepper’s Ghost technique where we propose an innovative communicative approach and a strong technological concept. This installation and the way it fits in relation with the museum space/context/visitors have been carefully evaluated in terms of attractiveness, usability, and educative potential.

The theoretical assumptions and the concrete results obtained from this survey allow us to present and discuss in Section 4, some fundamental guidelines that can be helpful for those ones that want to create holograms in museums.

2. Holography

In recent years, holographic techniques have greatly advanced. Thanks to holography, it is possible to duplicate reality, create characters, objects, or scenes that do not exist.

Holography is an optical technology for recording and storing visual information in the form of a very fine interweaving of interference fringes through the use of coherent laser light, appropriately projected; the image created by the interference fringes is characterized by an illusion of three-dimensionality. It is more properly a parallax effect in the perception of the image: the image is indeed perceived differently, depending on the point of view; in the case of two eyes, each of them perceives the image from a slightly different position with respect to the other. This difference, called ‘parallax’, in the normal vision determines the three-dimensional perception [16,17,18].

The etymology of the term ‘holography’ comes from the ancient Greek ὅλος, holos, (‘everything’), and γραφή, grafè, (‘writing’) and literally means ‘I describe everything’. Holography was theorized in 1947 by the Hungarian scientist Dennis Gabor (1900–1979), honored with the Nobel Prize for the Physic in 1971, for this invention. It started to have meaningful applications with the introduction of laser sources of light, highly coherent, in the 1960s [19,20].

In the registration of a hologram, the light from a laser is divided by a semi-transparent mirror (beam-splitter). The two resulting rays are then expanded and conveyed through appropriate mirrors: one of them goes illuminating the object (wave front of the object), while the second one directly affects a photographic plate (reference wave-front). On it, the two wave fronts interfere, and the recording of the interference fringes is the hologram. The tridimensional effect is produced by diffraction (Figure 1). The plate is developed and fixed as in an ordinary black and white photographic process [21,22,23].

Holography preserves much more information than a common photographic plate or film, because it records not only the light intensity but also the phase of the light wave, thanks to the use of lasers.

However, this holographic technology is still immature to be applied to cinematic and multimedia [17,22]. For instance, it is hardly applicable outdoors, and it is not suitable for large objects. Moreover, the hologram recording and the storage on a photographic plate are obviously not suitable for moving images.

Advanced laboratories in the world are dealing with holography, including the Massachusetts Institute of Technology (MIT), the College of Optical Science OF Arizona University (UA), the Fraunhofer-Gesellschaft in Germany, The Istituto Nazionale di Ottica (INO) of the National Research Council, Italy. Nowadays, the computational power made available by modern computers offers the possibility to generate holograms also through numerical procedures (DH—digital holography; CGH—computer generated holography), able to calculate the interference fringes using mathematical algorithms, starting from a representative model of the object. At MIT, for instance, they use a TOF (Time Of Flight) camera (specifically a Microsoft Kinect) to produce a real time depth map of the shot scene (even in movement); then an algorithm extracts the fringes of interference that compose the digital hologram. Common GPUs (Graphic Processor Unit) are able to calculate the hologram. At the University of Arizona they use, instead, another method, deriving a 3D model of the object taking images by means of a set of video-cameras located all around the object itself. A parallel research is in progress in relation to displays, for which different technologies are experimented (SLM, Spatial Light Modulator, or a plate of special rewritable polymeric material) [24].

However, the holographic images created by mean of the numerical procedures, especially those ones dynamically projected, still have many limits: low quality of the images, stains, noise, low contrast, low resolution or low frame rate, monochromatic aspect, very narrow field of view in the tridimensional perception, lack of interactivity [20]. For such a reason, other techniques have taken place in multimedia. Beyond Gabor’s principles, today the word ‘holography’ is often used in a generalized and not fully pertinent way. It is used to indicate a number of different methods able to produce translucent and tridimensional images, that are typical of the images obtained by means of holography.

The Pepper’s Ghost is one of these techniques, particularly used in performances. It is appropriate and easy to implement also inside museums, in terms of integration with original artifacts, image quality, compatibility with preservation needs, scalability, adaptability, robustness, daily management, costs. In this paper, authors will refer in particular to this technique, discussing how it can be used in the context of museum exhibitions and how it can influence the creation of new narrative and representative approaches, focusing mainly on museological and communicative issues.

2.1. Pepper’s Ghost and the Illusion of Reality

The Pepper’s Ghost effect is an apparition of something that does not effectively exist but appears as if it was just in front of our eyes. It is an illusion of reality [25].

In the XVI century the Neapolitan philosopher and alchemist Giovanni Battista della Porta (1535–1615) invented the camera obscura. In his publication Magia Naturalis (1584), he describes for the first time an optical illusion titled: “How we may see in a Chamber things that are not” [26]. His invention was later developed and experimented, up to be brought into theatrical performances by the English scientist John Henry Pepper (1821–1900). He wanted to create magic effects, fascinating the spectators. The technique became very famous and, in honor of his developer, it was called “Pepper’s Ghost” [27].

The illusion consists in the perception of things, a place or a floating figure in the empty space, in a position where they are not in the reality—given that their real presence is in a secret place, hidden for the observers [28]. Originally, during theatrical performances, a room was created in a position invisible for the public (defined as the ‘blue room’), adjacent and perfectly corresponding to the main stage. This hidden room could be located under the main stage or aside. In this hidden room, real figures (characters, objects) moved in front of a powerful source of light. Once illuminated, these figures were reflected by an oblique transparent surface (typically a mirror) that was positioned with a corner of 45° between the hidden room and the spectators. Because of an optical effect, the reflected image was automatically projected on the stage, in front of the spectators’ eyes, on a determined surface. In this way, it was possible to create ghost effects—thus an illusion of reality.

Only what was illuminated by the source of light could be reflected by the mirror and projected on the stage: if the source of light in the hidden room illuminated only a figure, leaving the surrounding space in the darkness, only the ghost of that figure would have appeared, floating in the empty space of the stage, eventually interacting with real actors and real scenography. Figure 2 shows a typical setup a sheet of glass is hung across the front of the stage so that the image of an actor standing in the orchestra pit appears to float on stage.

John Henry Pepper firstly showed this effect applied to theatrical performances in 1862, in Charles Dickens’ “The Haunted Man”, at the Royal Polytechnic Institution [29]. Several plays were further written around 1863, especially to use such effect, even if its application became limited by the fact that the ghost could not speak.

Today, the ancient ghost technique is still used on the stage of concerts, in theatrical performances (Figure 3), in films, or inside a museum’s showcase, at a smaller scale.

Dead famous artists or singers relive and perform again on the stage thanks to holograms. This was, for instance, the case of Michael Jackson, who performed as a dead man in 2014 on the Billboard Awards stage [30]. The dark room, the light and the real content we want to reflect as illusion are replaced by a monitor (or by a projector depending on the size) and virtually represented; the transparent mirror is usually a special glass or a film or common Plexiglas (Figure 4).

2.2. Pepper’s Ghosts in Museums Today—Some Examples of Use and Technological Solutions

When discussing holographic showcases in the following sections, we shall mostly refer to Pepper’s Ghost technique that is the most used in performing arts and has the greatest potential in museums, for the reasons explained in Section 2 and Section 2.1.

Technological solutions and dimensions of a holographic setup, using Pepper’s Ghost effect, depends essentially on the dimensions expected for the ghost, especially if it must interact with real actors/environments. Nevertheless, the dimensions are just one of the main parameters able to increase the cost of such system. The quality of the infrastructural design and the hardware components can considerably increase the quality of the effect and, consequently, the cost. The final design of a holographic showcase and the final cost can indeed vary considerably—even if it is still an accessible technological solution for museums.

Some materials are fragile, the most delicate component is the mirror placed at 45 degrees. In replacement of the solid mirror a thin and transparent film can be used, especially in the case of very large dimensions of the holographic setup (where the mirror would be impossible to manage). However, the transparent film is even more fragile than the mirror, it is difficult to clean and, above all, not simple to reuse in case of movement and repositioning of the Pepper’s Ghost setup: the film needs indeed to be perfectly stretched to be invisible and to avoid image deformation, and, due to its delicate consistency, it is hard, or impossible, to be stretched more than one time, thus the images will result deformed.

A common type of digital showcase is the holo-pyramid display, a combination of four Pepper’s Ghosts, properly illuminated by a screen, for example a tablet. Usually in this case, the curiosity of the spectator is aroused by the vision of a 3D object, floating and turning in the empty space. Frequently, the holo-pyramid display is quite small, able to visualize a small virtual object with animations and possible accessorial graphic effects. The space available for something else to happen is very limited, so the storytelling potential; therefore, audio contents and interaction between real and virtual contents are usually absent.

Other types of holographic showcases are possible, with different volumes and setup solutions. For the CEMEC project, Connecting Early Medieval European Collections [31,32], a small holographic showcase has been prototyped by the designers of the Allard Pierson Museum (APM), Amsterdam, and presented in the context of the CEMEC itinerant exhibition. It uses a mini-projector located at the bottom of the structure, instead of a TV screen; once projected on the ceiling of the small showcase, images are reflected by a flexible oblique transparent film (instead of a solid mirror) (Figure 5).

Every time the showcase is mounted in a new location this film must be stretched again, until it becomes perfectly tense and invisible. Technically, this solution seems to work fine, even if it is a little bit critical along the time, because the film is not robust and does not have long-lasting mechanical properties. The window on the front is very small (about 40 cm wide × 15 cm high × 15 cm deep) and it obliges visitors (maximum two at a time) to come very close to the glass, as if it was a secret box to peek inside (Figure 6).

This small digital showcase is conceived to contain one small museum artifact (a golden brooch, or a belt buckle), while a ghost animation plays over it. A short audio description accompanies the virtual images, showing details of the golden jewelry in 3D, manufacturing procedures, while simple drawings suggest contextual scenes. The small dimensions of the showcase and the kind of user experience that has been designed do not encourage to create a real dramaturgy that would be unbalanced and too powerful in comparison with the strength of the visualization. This kind of holographic showcase is not very invasive of the museum space and it can easily be multiplied along the path of visit, to offer a series of multimedia events to the visitors.

The case of the big holographic showcase, again created in the context of the CEMEC project by CNR and E.V.O.CA. team (and presented in detail in the following sections), is more demanding in terms of space, and its multimedia potential is greater compared to the smallest [33]. First of all because of its dimensions: the front window is 120 cm wide, the depth is 140 cm, the height 80 cm. A larger audience can see it at the same time and is possible to simulate many more events inside of it. Users can see more than one real object (or one big object), the interior space can host even a simple real scenography, and virtual characters can be projected in real scale. Also, contextualization scenes can be represented more realistically. Figure 7, for instance, shows how the real candle holder exhibited in the showcase is incorporated and contextualized in the virtual scene and thus brought to life. This approach opens exciting perspectives in the experimentation of dramaturgical paradigms, as we are going to discuss in the following sections of this paper.

Another wonderful example of Pepper’s Ghost of big dimensions, with a great dramaturgical power, is the Sarcophagus of the Spouses installation [34], specifically designed by the well-known new media artist Franz Fischnaller, in collaboration with Giosuè Boetto, Cineca and the Superintendence for Archaeological Heritage of Lazio and Southern Etruria. This installation was conceived and realized in occasion of the temporary exhibition Etruscan Journey towards the Afterlife (2014–2015), in the History of Bologna—Museum Genus Bononiae, Bologna, Italy. Here virtual archaeology, 3D video mapping, holographic techniques, audio-visual narration, scenography have been combined together [35]. Sure enough, the virtual show was expected to involve the entire exhibition space of the room (14 × 14 × 15 m high) in the narrative context. Therefore, the installation was based on (1) a three-sided holographic pyramid (4 m wide × 3 m high), to visualize the ghost of the Sarcophagus of the Spouses in the middle of the space and in real scale (1.14 m high × 1.9 m wide), and (2) a 3D mapping video projection system to visualize the narrative context on the surrounding walls, thus creating a dramaturgy in the whole space.

This ambitious installation required a large area and, above all, a huge hardware apparatus that was possible to manage in occasion of a temporary exhibition but that could be hard to include permanently inside a museum in terms of daily maintenance, in absence of specific managerial and economic strategies.

Another interesting example comes from the National Archaeological Museum of Cerveteri. Here, in 2013, holographic showcases have been implemented by Mizar company in collaboration with Soprintendenza Archeologica per l’Etruria Meridionale and DTC Lazio. A simple touch on the external glass of the showcase, in correspondence of the artifact, activates a Pepper’s Ghost in the showcase that seems to ‘wrap’ and animate the artifact. The animation describes the object in its details, contextualizing it with other objects of the tomb in which it was found [36].

In “Welcome to Rome” exhibit, (ex Cinema Augustus in Rome, Mizar, 2017), the story of Rome is told along a path of visit that presents exclusively multimedia installations [37]. In each room, deeply dark, there was nothing except an installation. In this condition, visitors entering the room were isolated and immediately induced to silently pay attention to the digital audio-visual narration. Many of these installations were showcases using Peppers’ Ghost effects (Figure 8), mapped directly on the real models of some important Roman monuments. What was mostly interesting was the interior setup of the holographic showcase: the space was very deep to enhance the 3D effect of the whole show, and the 3D model, materially reconstructed, worked as dynamic scenography of the virtual show.

The “Keys to Rome” exhibition (2014–2015) presented another occasion to work with Pepper’s Ghost. The exhibition, involving at the same time four European Museum and their collections, was an interactive journey to discover Roman culture, starting from the city of Augustus and spreading out over the entire Roman provinces. In the Imperial Fora Museum, we (as CNR team) presented a holobox whose main issue was interaction [38]. It consisted of a Pepper’s Ghost setup connected to a Leap Motion sensor, able to capture the hands’ gestures (https://vimeo.com/109283974) needed to explore and interact with objects. With the holobox, visitors were able to see high resolution 3D virtual models of the objects in all four museums. The visitors had also the opportunity to manipulate them using a natural interaction system (Figure 9): Leap technology allowed them to actively play with the item by personally selecting details, looking at the object from different perspectives, loading other interconnected items. The selection of objects was done through another adjacent VR application, ‘Admotum game’ where the context of the objects could be explored: items collected in Admotum scenarios could be ‘sent’ to the holobox for a detailed exploration.

In conclusion, pure visualization, without additional contents, can be useful to show digital collections, making accessible virtual replicas of real artifacts that are preserved elsewhere. In such visualization, it is of course possible to integrate additional contents, like for instance virtual restoration. However, if the aim is to transmit historical contents, believes, personages, events, contexts of usage, the creation of a story is needed, as some of these examples have shown and as our CEMEC project aims at.

3. The CEMEC Case Study: The Box of Stories

3.1. Purposes, Topics, Steps of Work

The EU project CEMEC—Connecting Early Medieval Age Collections—gave us the opportunity to experiment and test assumptions and ideas about holographic systems in museums. Started in 2016, the project aims to create a collaborative network, and a cost-effective business model, between eight European museum collections, seven universities, and six technical partners. Drawing on objects from participating museum collections, the project has produced a traveling exhibition focusing on connectivity and cultural exchange during the Early Middle Ages (300–1000) in Europe and around the Mediterranean, from Ireland to Egypt and from Spain to Hungary and Greece [31].

Technology has an important role in this exhibition: it serves as a supplementary way of telling the story of the early medieval period to museum audiences. The holographic showcase is indeed the chosen communicative strategy. It has been used here to tell the story of some objects of the VII sec a.C. which are:

• Kunàgota sword. An Avar sword exposed at the National Hungarian Museum of Budapest (NHM), belonged to an Avar chief of the village of Kunàgota. It was never used because it represented a protective object for the afterlife of the buried man.
• Mytilene treasure. A set of Byzantine objects from the Byzantine and Christian Museum of Athens, specifically a golden bracelet, a candlestick and a trulla, a tool for water’s spilling, belonged to a wealthy family that lived on the Mytilene island, in front of Asia Minor coast. These objects were part of the domestic equipment.

In the holographic showcase, each object has been presented alternating (a) a short presentation, with a more descriptive style, to communicate the basic information; (b) a dramaturgy which means a scenario where the object is contextualized in its original environment. The dramatized scenarios are useful to create a magic atmosphere but also to suggest additional information regarding the function of the objects themselves. They are evocative and symbolic and combine 3D graphics and real actors/objects. Digital contents also interact with the real objects.

The purpose of using such type of multimedia inside a museum is the creation of an artificial system that reflects technologically, symbolically, the range of life. It indeed transmits contents that otherwise would not be perceptible, increasing the awareness and understanding of the flow of history by visitors. This flow is no longer far away, neither extraneous, but it becomes part of our present. From a technical point of view, the holographic showcase foresaw precise working steps:

• infrastructure design (the skeleton of the showcase; the choice of the hardware);
• production of 3D and multimedia contents to be harmonized with real objects;
• implementation of a real time rendering platform in VVVV software, able to synchronize audio-video play-out, lights, external devices: along a multi-track timeline all the audio-visual events are organized and managed according to a precise sequence;
• management of Arduino/MIDI controller, to manage real lights along a timeline and to control buttons for language selection;
• light design (seven led lights inside the showcase switch on and off on real objects);
• interaction and user experience design.

Special attention has been dedicated to ensuring the portability, reusability and sustainability of the equipment (Figure 10), as well as ensuring the safety of the display containing the real museum objects. The holographic showcase has been designed to be scalable—depending on the size and quantity of objects displayed inside and the available space in the museum. It has been indeed implemented in two solutions:

• stand-alone showcase (dimensions 1.50 × 1.40 × 2.50 m)
• showcase inserted in a projection wall 4 m wide and 2.50 m high.

In the following sections, we will present the reasons of the adopted design and communicative solution, the technical deployment, and some more meaningful data from the user experience evaluation, relevant to extrapolate some general guidelines about the methodological approach when working with holographic techniques inside museums. The resulting guidelines will be presented in Section 4 of this paper.

3.2. The Kunàgota Sword Tells the Story of the Avars

The first story that has been created for our holographic showcase deals with a sword belonging to an Avar warrior, who lived in the mid-seventh century AD. It is the so-called Kunàgota sword, from the place of discovery, and it is preserved at the Hungarian National Museum in Budapest. The sword was discovered in the steppe-like southeast region of Hungary, inside the tomb of a prestigious Avar chief, as part of his funerary good. His tomb was isolated, just excavated into the soil. Originally, the golden sheets applied on the sword belonged to a Byzantine object, most probably a casket for jewels. An Avar goldsmith took the golden sheets from the casket and applied them to the sword and the scabbard when the Kunàgota chief died.

The entire installation is built around the real object, physically present inside the showcase. Two configurations of the installation have been implemented and presented in museums:

• the holographic showcase integrated in a wider projection wall, presented in the Hungarian National Museum in Budapest, in the context of the exhibition “Avars Revived” (February–May 2017) (Figure 11);
• the holographic showcase in its stand-alone version, without projection wall, presented in the Allard Pierson Museum in Amsterdam, in the context of the “Crossroads” exhibition (September 2017–April 2018) (Figure 12).

In the first configuration, the show is organized as follows: in some moments, the showcase surface becomes part of the overall projection displayed on the wall; in others, the projection wall turns black and the showcase becomes the center of attention, hosting the holograms that appear and move above and around the Kunàgota sword. Thanks to this alternation, the real object is enabled to tell its story—and with it, the life and the spiritual dimension of those who owned it.

During the large projection on the wall, the user receives the basic information necessary to understand and contextualize the story of the Kunàgota sword in a more common style [39]. The real sword is well illuminated, while, on the surrounding wall, short captions, enlargements of details of the sword’s decoration (virtual replicas), images of the other funerary goods coming from the same tomb, and illustrations of Avar warriors are projected, accompanied only by music. Digital contents are not invasive in relation to the real ones (Figure 13). This preliminary phase is called “neutral vision”.

After this ‘neutral’ fruition, the showcase takes control and a dramatization takes place inside, through Pepper’s Ghost illusion. No traditional storytelling is used here, nor a linear narrative strategy; differently, the same object, through its presence, evokes voices, fragments of life, customs, religious and cultural identities. The manufacturing process of the object is represented and so the historical events from which it came from: the looting in battle by Avars of a Byzantine casket covered in golden plates; its spoliation to cover with gold the sword of a dying Avar chief. The figures themselves engraved in the golden plates become the protagonists. The virtual content overlaps and integrates the real object in the same space of the showcase. This phase represents the’dramatization phase’. The holographic projection, in its minimal essentiality, produces finally the magical effect of the illusion of reality: the hands of two Byzantine women appear, stealthily scanning the contents of their master’s casket; the dust rises from the hooves of horses running during the battle; splashes of blood are produced by swordplay; the hand of the dying father raises his sword and offers it to his son asking it be decorated with the Byzantine gold just stolen—accompanying him in the afterlife. Voices and whispers of various characters animate the events.

The last chapter, “the Farewell and the Blue Sky of Tengri”, contextualizes the sword in the Avar chief’s tomb and accompanies the visitors through the vision of the afterlife, according to the belief of the ancient Asiatic shamans. Here starts the ‘vision phase’. The only surviving object, which testifies the cosmic beliefs of Avars, is a rudimentary decoration engraved on a small bone casket found in the Avar cemetery of Mokrin, in Serbia, used to contain an ointment. The drawing shows a tree on a hill, on the sides both the sun and the moon and various stylized animals. It probably represents the Tree of the World: the roots reach the underworld, the trunk represents the earthly world, while the branches reach the sky. The Turkish people believed that the heavens were ruled by the god Tengri and the dead traveled in this world—the same kingdom in which the shamans ascend to create a connection with the spirits.

Starting from this drawing, the scene has been virtually and carefully reconstructed; it has been also animated, as a theatrical scenography in motion: each single element enters the stage, one by one, to better catch the visitor’s attention on its symbolic and metaphysical meaning. The grave is shown in the ground among the roots of the tree; the warrior chief lies dead, near his sacrificed horses, ready to ride for the last time to heaven. At this point, his spirit appears (an actor filmed in green screen and integrated into the virtual scene) and he whispers for the last time, while animals move around him (Figure 14). He evokes his vision, the moment of the burial, and his next and final departure in the blue sky, that is finally accomplished. It is a first-person drama, organized in different scenes, each of them introduced by a short title.

In the second configuration presented in Amsterdam, at APM, the holographic showcase is a stand-alone installation. All narrative phases, including the first one, are played inside it.

The animations of the objects, in this kind of holographic solution, cannot be seen from all around the structure. These objects are shown to us as they relive, under our eyes, the moments that have characterized their history over the centuries—moments that have led to shifts, transformations, passages from one owner to another, and so on. From a narrative point of view, it is an inverted perspective, so that historical events are not telling us the full story of the objects (as usually happens in movies), but the objects themselves tell us about the facts that led them here and now. In this way, user does not only learn facts or interpretation of facts, but lives them again in real time; he participates to them emotionally. The phantasmal vision of the holographic event breaks out from the past in all its power, full of its history.

3.3. The Mytilene Treasure Tells the Story of a Byzantine Family

In the Byzantine and Christian Museum (BCM) in Athens the holographic showcase hosted three Byzantine objects, as part of the Mytilene treasure.

The Mytilene Treasure is a collection of more than 70 precious objects (silver vessels, gold jewelry and coins, a bronze seal) discovered in 1952 in Kratigos, in Mytilene island. They belonged to a wealthy family living there, or maybe in the Asia Minor coast, in the first decades of the VII century, as revealed by the effigies on the coins and the Imperial control stamps impressed into some objects. Sure enough, after the efflorescence under Justinian (VII–VIII c.), the Byzantine empire was exhausted by the assaults of Persians, Avars, Slavs, Arabs. The population was decimated, economy decreased, and urban centers dwindled. A stamp, gold buckles, which were a privilege of high-ranking officials, let us suppose that the treasure belonged to the family of a high official of the Byzantine administration. The sudden danger of an enemy attack could have obliged this family to escape, after having buried the precious objects of the house into the ground, to preserve them from possible looting. Three objects are today exhibited in the holographic showcase: (1) a silver candle holder; (2) a golden bracelet of a child, with a very rare openwork monogram; (3) a silver trulla (a bowl) decorated with marine themes and probably used in bathing.

The holographic showcase has been presented in its standalone configuration, without the projection wall. The narrative approach is partly similar to the one used for the Kunàgota sword, but more classical. Thanks to the virtual projections on and around the objects, the public can experience fragments of their story, meet characters, and relive historical events. Explanatory moments alternate with dramatizations where objects are contextualized and shown during their daily use by the family. These characters are performed by real actors, shot in slow motion (Figure 15).

In order to shoot these scenes, it has been necessary to print the 3D replica of the objects in real scale, paying attention to make them plausible, resistant, and beautiful, similar to the original ones exhibited in the same space. Original artifacts in the showcase enter in the compositing as element of the virtual scene. For instance, the real candle holder appears laying on a virtual table, with a virtual candle fixed and burning on it, close to a virtual character writing a paper. Such integrations seem very powerful. In addition to computer graphics, the language of video art was used too: slow motion, together with the essentiality of the scenes, create an abstract and suspended atmosphere of strong attractiveness. At the end of the story, the burial of the treasure is suggested only by soil falling down in slow motion, from the top to the floor of the showcase, increasing its mass during the animation. Also, the soundscape has a crescendo, until the complete coverage and disappearance of the real artifacts (lights on them switch off). Also, in this case real lights behave as part of the animated compositing.

3.4. Virtual Reconstructions and Visual Mood of Kunàgota Sword and Mytilene Treasure

The most challenging reconstruction work in this project regards the Kunàgota word. Indeed, according to scientific studies, the golden sheets which decorate the sword, originally come from a Byzantine small casket. During the late-Roman period and Early Byzantine Empire, there were furniture pieces such as caskets and cabinets decorated with carved wood or other inlaid panels like ivory or golden sheets. The construction of the original casket was based on archeological and geometric data, analyzed and interpreted with the support of experts from the National Hungarian Museum (NHM) of Budapest. The missing parts of the casket were based on comparative studies and observation of artifacts displaying similar mythological themes (Figure 16).

In order to realistically portray the casket and its appearance in the holograms, the physical reaction to light of golden and ivory materials needed to be studied and simulated in the rendering engine. Different types of maps and shaders, that simulate the color of the artifacts and the main appearance of the material surface, were used to give the museum objects a realistic look in the rendering.

The same workflow has been used also for the objects belonging to the Mytilene treasure. In that case, since the artifacts were almost intact and did not need any virtual restoration, we only optimized the geometry of the scanned objects and we simulated the different materials (the gold of the bracelet and the silver of the chandelier and the trulla).

Light management was another crucial aspect for ensuring a realistic look of the virtual artifacts and create a proper visual mood. The screenplay required that virtual objects were rendered on a black background, enhancing the ghost effect. Rendering such virtual objects, characterized by reflective materials like silver and gold, without any environment and an appropriate multi-point light setup, would have ended in artificial visual results. Whereas, to create a realistic illumination, we adopted a HDRR (High Dynamic Range Rendering) approach for lighting calculation. HDRR consists of using an HDR environment texture as light source to simulate a greater range of details than standard computer graphic light emitters [40] (Figure 17).

For all the renderings, we used different HDR textures. For those intended to illustrate the artifacts and their details, we used a studio HDR which simulates a photographic high-key lighting setup with an emotionally neutral mood. It is composed by several light: main, secondary, back, and fill light with almost the same intensity and fairly soft.

In other cases, where strong emotional lights were needed, we adopted a more intense lighting scheme, changing the balance of highlights and shadow to increase their hardness and obtain a much powerful effect. This kind of mood was especially used in those shots where the virtual objects were integrated in video with actors.

3.5. User eXperience (UX) Evaluation

Conducting inquiries about the requirements useful to understand how the museum visitors ‘perceive’ museums and their digital products is essential to define the guidelines for future museums.

The User eXperience (UX) design is the ‘discipline’ that investigates the intimate relationships that exist between the design features of a product and the emotional and cognitive feedbacks of the user to whom this product is submitted. Although it plays its own role in the plan and development of any digital application, it finds obstacles in structuring a univocal procedural taxonomy to carry out the evaluation activities. It is obvious that there is no a unique investigative methodology to be used, because each survey refers to a specific multimedia project, often very complex and ambitious. Furthermore, one of the difficulties in conducting such evaluations lies in the multidisciplinary nature of the investigative activity.

The user experience, in a practical sense, can be defined as “the perception of a person and the feedback deriving from the use of a product, system or service” [41]. The term is thus applied to the user’s positive, neutral and negative emotions arising from the interaction with computer systems and interfaces, conditioned also by the physical characteristics of the latter or, more, by the context in which they are included and used. Therefore, conducting UX evaluations is not an easy task, but still can be done taking into account what the experience and the literature taught us about background information of users’ behavior inside museums (Section 3.5.1) and following a precise methodology which compares data on multiple levels (Section 3.5.2).

3.5.1. Background Information

Under the CEMEC project, three evaluations have been carried out for the holographic showcase trying to point out (a) the general feedback of users (appreciation and satisfaction); (b) the usefulness of contents told in such a manner (holography upon real museum objects); and (c) feasibility of the system into museum contexts [33,39].

The project of the holographic showcase is indeed conceived to be connected to the museum’s collections and consists in hosting on the same perceptive level the real artifact and the virtual projections on it and around it; the same museum object is brought back to life by means of dramatized and narrative techniques. The visitors’ experience is thus enhanced by means of detailed visualization, virtual restoration, contextualization, storytelling, and dramatization of the object and on the object. No physical interaction has been designed in these expositive occasions, a part from language selection; but surely the narrative approach is the key issue from a cognitive point of view. For the CEMEC holographic showcase, it is clear that:

• Users are the museum visitors, given that the application has been only presented during temporary itinerant exhibitions (five months in each venue) of the CEMEC museum partners. For such a reason, motivation behind the visit can be merely referable to [42]:
  • curiosity towards the subject of the exhibitions (i.e., Avars, Early Medieval Age, Byzantines and Christians);
  • touristic needs if users were not citizens of the city where the CEMEC exhibitions were hosted—museums as landmarks that “MUST be seen” when in a foreign city;
  • professional interest towards the subject and the technology used into the exhibitions, given the previous academic background of some users and their actual professional activity (i.e., researchers, academics, curators, project managers...);
  • educative reasons which could push families and educators (i.e., teachers, students...) to deepen the topic proposed by the CEMEC exhibitions;
  • commercial and promotional plan of project managers, politics, and cultural departments;
  • social interaction between school groups, families, and people sharing the same interests.
• What generally capture users’ attention inside museums and bring them to love or not a digital product is [6,43]:
  • the overall atmosphere—quiet, comfortable, familiar, and clear;
  • the type of interaction—easy to practice, quick to access the related content, and shareable with friends and others;
  • the graphic interface and the soundscape—colors, music, volume of music and visibility of commands highly influence the attractiveness of the digital application and its memorability in the future;
  • the type of story and the way it is told—this datum is relevant to maintain the attention of users during the fruition of the digital application.
• The reliability of the 3D content in correspondence with the real artifact on show stays into [44]:
  • the high quality of the 3D models in terms of shape, colors and visual effects (i.e., materials, reflection, light effects, shadows, aging effects, and so on) which could allow users to have a perception of real, of a ‘credible’ object;
  • the overlapping of real museum object and its 3D model in order to produce a sense of continuity between virtual and real, reconstructed and virtually restored;
  • the characterization of the storytelling which is composed by (a) the type of the cultural notions cited along the story; (b) the level of ‘insights’ intended as curiosities and factual information granted to the users; and (c) the voice who tell the story (Is it a real character or the object itself?).

3.5.2. UX Evaluations—Multi-Partitioned Analysis

Given this background information, User eXperience (UX) evaluations were planned. They took place in Budapest (National Hungarian Museum—NHM), Amsterdam (Allard Pierson Museum—APM) and Athens (Byzantine and Christian Museum—BCM) during the itinerant CEMEC exhibition.

Museum staff and CNR researchers followed the evaluations in each location, using a multi-method approach, which best answered our needs [12]; it indeed addressed to identify and evaluate the impact of storytelling within the holographic showcases by matching and crossing multiple parameters coming from the same source—the users.

The evaluation plan foresaw three different moments of investigation (and related activities):

• A general analysis, at a time prior to the main UX survey, on the general museum environment. This was important in order to study the context of fruition and the exhibition availabilities, how the visitors usually approached the space and the objects exposed, which were the interesting points of the museum visit path and which were less interesting, which was the visitors’ favorite pathway, the affluence of public in museum spaces, and how the institution included any digital and multimedia equipment throughout the exhibition. Statistics, maps, reports, and so on were really useful to study the CEMEC hosting museums.
• The UX survey on public, during the museum visit at the holographic showcase, by means of observations, to verify and analyze the practicability of this expositive solution included into the visit tour, the related effectiveness compared to traditional cultural panels and its relationship with the permanent collection on display.
• After the museum experience, by means of paper-based questionnaires, to verify the users’ feedback in respect to certain notions acquired or not when interacting with the installation, the usability and the interface design.

We called this multi-method approach ‘multi-partitioned analysis’ and it consisted of two evaluative methods:

• Observations. They allowed us to have an overview of the users’ behavior toward the holographic showcase in terms of visibility of the digital product along the museum path, user’s attitude toward it, time of permanence in front of it, type of visit (single or group), attention or distraction while watching the story inside the showcase, need for help to make the story starts. It was mainly composed by three sections: (a) demographic data to be collected in general form (gender, age); (b) users’ behavior to be transcribed in form of written texts and into a grid of values (like actions, face miming, gestures, and comments); (c) context of fruition to be analyzed in relation with the overall environment and the museum room hosting the holographic showcase, and the user social behavior (alone or in group). Open comments for operators were also available at the end of the observation template, allowing extra notes to be included as relevant for the report. Each observed user had a corresponding document, progressively numerated and reporting information about date and period of the observation.
• Direct questionnaires. They gave the chance to gather direct feedback of users to confront to what just observed by operators, finding a common sense and interpreting users’ reactions putting comments into contexts of reference. The template was again composed by three parts: (a) questions about the system’s usability and the visibility of the graphic elements and of the real objects; (b) questions about the storytelling, the clearness of the content proposed and the language understandability; (c) closing questions about the appreciation and the enjoyment of the experience just had with the holographic showcase. All these questions matched and intersected and then related to the observation protocol, revealed a the users’ behavior and activities, their level of reliability, and commitment towards the evaluation (Figure 18).

Observations and questionnaires were made along the day, covering all the time slots of the opening hour of the CEMEC museums. This was necessary in order to grant the most democratic users’ selection, avoiding specific target groups—which would have badly influenced the final result of the evaluations. Users were chosen randomly according to their entrance at the exhibitions (the target was the usual museum target) and to their availability in participating at the evaluation.

Participating to the survey was on voluntary basis and anonymous. Users were informed at the entrance of the CEMEC exhibitions about this activity by an informative panel located at the ticket office, and by a disclaimer placed near the digital applications in the exhibition rooms.

Operators conducting the evaluation were part of the museum staff or dependent of the CNR; both knew very well the CEMEC project and the digital application; the CNR personnel had a specific background in user experience studies while the museum staff were mainly sociologists and anthropologists. Their task was to intercept the user once he/she stepped into the museum rooms (where the digital applications were located) and follow his/her visit. Trying not to be seen by the visitor, the operator observed his/her actions and movements, if he/she was alone or in group, taking constant note of what was going on, by filling in two predetermined templates.

They made observations during the visitor’s permanence in front of the holographic showcase and then they asked the visitor to fill in the questionnaire just after the experience: at NHM it was just after the visitor got out of the dark room; at APM, instead, once he/she got out of the last exhibition room; in BCM was done once he/she finished to watch the story.

For all the venues, almost 150 observations have been collected on the big holographic showcase. This does not apply to questionnaires which have been 142 in Budapest, 70 in Amsterdam and 70 in Athens. This difference was due to the period of inquiry and to the diversity in the visitors flow in each museum. Moreover, for APM and NHM the evaluation was done on the holographic showcase hosting the Kunàgota sword; while at BCM the subject was the Mytilene treasure.

In general, users positively reacted to the evaluations: some of them wanted to give their own contribution at the research spontaneously approaching the operators; others, at direct request, kindly answered to join the study; very few visitors refused to participate. Even if some differences occurred in the users’ attitude towards such kind of activities (due to the cultural predisposition, the city of the evaluation, in the north of Europe or in the south, the background education, etc.) almost all of them fully compiled the questionnaire—leaving no unanswered questions.

3.5.3. Results

The user groups of all the evaluations were mainly composed by women (51% out of the global users) so the separation between genders was not so clear-cut. The medium age registered was around 60 years old (45% out of the global users), followed by adults between 45 and 60 years old (31% out of the global users), young people between 15 and 30 years old (15% out of the global users) and individuals between 30–44 years old (9% out of the global users).

These data must be read in the spectrum of the geography of where such evaluations were conducted: surely in Amsterdam (APM) and in Athens (BCM), adults were the main museum visitors; in the case of APM they were citizens and retired people, who could access the museum freely given the cultural card available in the country; while, at BCM, they were couples on vacation and professionals working in the city interested to the subject of the exhibition. At NHM, the museum visitors were generally younger: students of the nearby university, professionals and experts of the Avar period, young families with children.

The educational background of the user groups was mainly humanistic (35% out of the global users) and related to the field of ICT and engineering (17% out of the global users); they also were interested to the field of social science (10% out of the global users).

Here below, we go in deep with the results, trying to analyze them in comparison with our starting goals.

3.5.4. ‘Taste’ of the Overall Experience

As a matter of fact, UX evaluations confirmed a great interest toward the new narrative approach and the holographic system designed for the Kunàgota sword, in the Hungarian and Dutch venues, and for the Mytilene treasure in the Greek location (Figure 19). In all cases, a marked sense of curiosity toward such multimedia installation has been recorded. Visitors came close to the Kunàgota showcase not really knowing what to expect but, at the end, they seemed satisfied and fascinated from what they have just experienced. The same goes for Mytilene treasure: the drawings and characters captured the attention and interest of all target groups, stimulating a sense of wonder in the eyes of the audience as resulted from the open comments. Questionnaires and observations supported such considerations by registering a high percentage of users approaching the holographic showcase autonomously and remaining in front of it for all the duration of the story (89%).

3.5.5. Storytelling

Regarding the narration, the majority of visitors paid great attention to the story; of the five questions about historical information, visitors reached almost the 80% of correct answers both at APM and HNM while at BCM was registered a 93% (Figure 20). The visitors’ memorization rates were indeed high, especially if we match these results with the ones coming from questions about elaboration and recalling of units of content, again favorable. Nevertheless, going throughout the demographic data, we can observe that given the professional and academic background of the visitors it was quite predictable that they knew already about the subject of the holographic showcase: a great percentage of them, in all museum venues, were indeed in the field of humanities.

Operators also noticed that once they finished the questionnaires, 40% of APM visitors returned to the showcase to verify if they have correctly answered—showing their sincere interest toward this research and a great sense of curiosity. At a direct request, 74% of visitors liked the storytelling at NHM, compared to 97% at APM and 86% at BCM.

3.5.6. Users’ Attitude

Observations revealed that at APM 78% of visitors visited the installation alone; instead, for NHM, nearly half of the visitors were in a group (52%) at BCM, the great majority were in group as well (80%). Operators noticed that the average time spent watching the story into the showcase was around 1–3 min (42%) for APM, out of the global duration of 7 min, while 5–7 min (48%) for NHM (Kunàgota sword story). At BCM, instead, of the 65% of visitors who watched the story (Mitylene treasure), almost the 83% waited until the end of the story. Such data confirmed that the duration was generally appropriate, but it was dependent on the conditions of fruition: noise, crowd, visibility of the holographic showcase in the room, group visit, presence of seats or benches around the installation, just to mention few (Figure 21).

From users’ comments, it was clear that the visibility of the holographic showcase in some cases did not help them to understand what it was about and how to interact with it; It was not perceived along the museum visit path, because in some case it was hidden and not visible from the main pathway. At NHM, the digital installation was more perceived as a cinema-like room and people did not get the ‘dynamicity’ of the experience, with storytelling playing at first on the wall and then in the showcase but, at the same time. They indeed remained seated and in silence—like they were indeed at the cinema; they had some expectations and a sense of wonder raised just a second before opening the door of the room where the holographic showcase was placed. At APM and BCM, instead, the installation was put close to other museum attractors, so the attitude of users remained the same along the museum visit path and all the other usual showcases.

3.5.7. Context of Fruition

The majority of visitors participated to the story silently (84% for NHN, 81% for APM, and 76% for BCM). The whole environment was appropriate for concentration, for the majority of the time, and it favored the vision (98% for NHM, 74% for APM, 68% for BCM); however, in some cases, the room was very crowded (at APM) or, in others, an overlapping of audio coming from more than one multimedia was present in the same area (at BCM) preventing the visitors to fully benefit from the narration.

Some of the users suggested the use of headphones, but still the idea of a collaborative experience, shared with friends and families, was the most relevant thing highlighted by people. Comments like “Beautiful to see the explanation of the museum object appearing so intensively in front of our eyes and have the chance to attend it with all my families” and “Nice to have the possibility to explain to my children what is going on of magic into the showcase” give the feeling of how powerful holography is inside museums.

3.5.8. System and Interface Design

Objects’ visibility was another important issue: the holographic showcase allowed the inclusion of real museum objects inside the structure; due to technical measurements such objects needed to be placed at a certain distance from the front view. This situation in some cases prevented visitors to clearly observe the artifact, provoking a sense of frustration and disappointment. In general, at the NHM, 65% of users considered the visibility as good and also the lighting system (87% said it was working well); at the APM, the percentages were higher with 82% of users appreciating the visibility of the objects and 91% of them noticing the good lighting system. In BCM, due to the different artifacts exposed, the reduced dimensions of the bracelet of the Mytilene treasure really influenced the visitors’ opinion: only 57% of them said that the visibility was good (Figure 22). Therefore, selecting the appropriate object for the holographic showcase means also taking into consideration dimensions and distance from the user’s point of view in the museum room.

A solution to overcome this problem can be a transparent or semitransparent glass at the back of the holographic showcase: this may allow visitors to see closer the museum objects, without any projection on them.

Interaction was not evaluated as crucial in such type of experience. The public did not demand any particular way of make use of the holographic showcase. They only complained about the selection of the language: in each venue, it was in the local language and in English. In Budapest, and Amsterdam, we did not offer the chance to select the language but they were running one after the other—somehow obliging users to follow and wait till their own country’s words. Whereas, in Athens, we did allow the language section, thus overcoming the users’ waiting.

4. Guidelines

The feedback collected out of the CEMEC experience lead us to propose some relevant guidelines useful to obtain efficient and attractive holograms inside museum showcases, in harmony with the surrounding space and the paths of visit.

4.1. Overall Experience: The Showcase in the Museum Space

The success of a digital installation like a holographic showcase is largely dependent on the public’s potential interest and experience. Potential interest relies upon multiple parameters like:

• users’ previous expectations when visiting a cultural place (i.e., learn something new; see something studied; spend time with friends and families…);
• their (mental and emotional) predisposition at multimedia inside museums;
• availability to interact with digital system;
• time at disposal to dedicate to such installation;
• interest and curiosity toward such way of exploring the museum’s content.

Experience, instead, reflects the general conditions of fruition and, for the holographic showcase it needs to take into account [10]:

• The dynamicity of the user visit path. In the majority of the cases, the museum visitors have to follow a precise museum visit path which includes also the digital installations; they have to retrace the curatorial line which blend together all the museum objects in a thematic or chronological manner. When in front of the typical showcase, visitors have always the same behavior: reading, observing, confronting, or listening to the audio-guide; differently, when in front a ‘digital’ showcase, the situation changes and they need to adapt their behavior to what is proposed: interact, watch, follow a precise story, listen to voices and music. If the museum thematic path is coherent, users will not have difficulties in following it.
• The discomfort of the user in front of the system. The holographic showcase ‘obliges’ visitors to stop in front of it in order to listen to the story and watch the image floating around the museum object. This aspect may influence the experience given that a person stands still and at a certain distance from the digital showcase to benefits of the content. The users can be tired so they can decide to leave after a while; or, if they seat, may obstacle the passage of other visitors and the latter can walk across them and hide the visual of who is sitting. Therefore, studying the best solution in terms of comfort and user experience is a positive point.
• The ‘wow’ effect of the holography. The hologram has something magic in its apparition given its game of reflections. Wonder and surprise surely lead visitors towards such digital installation. Attractiveness is thus crucial when planning a type of experience like the holographic one: colors, voices, superimposition, and a good storytelling are the key elements; it is important to work on visitors’ emotions and sensations in order to get the best way of telling what is behind the museum object: ancient visions, evocations, dreams, and beliefs.

4.2. Environmental Conditions, Setup Precaution, Public Management

When it comes to the cognitive aspects that influence the experience of a holographic showcase inside real museums, precise considerations need to be done. The context in which the holographic installation should be included has already a long tradition in terms of exposition methods, visibility of museum objects and logistics for visitors when they are in the rooms.

Generally, curators and museum managers have the duty to plan, arrange, and set up reliable and effective exhibitions, paying attention to:

• the cultural message to be transmitted to the audience;
• the exposition to be correctly addressed to the selected target groups;
• the language, atmosphere, and flavor of the exhibition to be coherent and coordinated;
• the security and the condition of display of objects to be respected;
• the safety of the museum environments for people to be respected.

Nevertheless, just a few pay attention to technology and multimedia supports included into exhibitions. What is important is to ensure that multimedia included in exhibitions does not overshadow the artifacts exposed, by subtracting attention to visitors. On the other side, multimedia has not to be invisible in comparison to the artifacts exposed, otherwise its usefulness is lost; differently, (digital) curators should take care that technology is coherently integrated into the museum visit path, as an important component of the visitors’ experience.

On the visitors’ side, it is required that they have enough space to move comfortably in the museum rooms, either following a precise visit path in the exhibition or going around autonomously. Moreover, they have to see objects well and easily, with no barriers or an excessive distance from the museum piece, which can prevent them from clearly observing it. The visitor’s flow is also conditioned by the time at their disposal and the type of visit they are conducting:

• Are they following a guided visit or are they free to go wherever they prefer?
• Are they using a multimedia tool to support their visit in the museum/exhibition or are they only using the exposition panels only?
• How can the all the different communicative supports work coherently?

Despite these aspects often being taken for granted, it is common to find inappropriate conditions of fruition in museums, especially regarding the integration of real and digital audio-visual contents. Small details in the exhibition design may negatively condition the visitors’ experience, like objects visibility, bad illumination, lack of space to sit down in front of a multimedia content, overlapping audio, and bad volume, unharmonized durations and rhythms of narration in relation to the surrounding museum’s space and dynamics of the public.

All these issues must be considered by (digital) curators and museum managers to figure out how to optimize the exposition space; likely, content developers, designers, and evaluators must also cooperate to create a coherent cultural, social, and physical experience for the public.

Site-specific installations, like the holographic showcase, may influence the visitors’ pathway, obliging them to choose whether or not to see the related content in spite of something else. This choice does not depend only on the quality of the audiovisual content but also on the comfort of the general condition of usage, the location in the space, the interferences with other contents, or environmental elements, or visitors’ groups. The story is definitely not enough: technology and museology need to work together to build up the best setting for museum objects to be narrated, observed, and understood by the majority of visitors.

The evaluation of the impact of museum installations on the public in terms of efficacy, usability, and appreciation, is mandatory if the museum curators want to improve their cultural offer and test their exposition choices. User eXperience (UX) has been defined as “the user’s perceptions and responses resulting from the use of a system or a service” by the ISO 9241-210(2009) norm. This concept really helps evaluators and researchers in the design and study of the state of a person when using a product (such as a digital application), specifically referring to how easy or pleasing the product is to use. The UX design, indeed, explores and investigates the physical and mental roadmap that each person undertakes when facing an object or a situation.

In the digital cultural heritage domain, UX encompasses all aspects of a person’s interaction with digital applications including behavior, actions, perceptions, and satisfaction. Putting an emphasis on UX before the final exposition setup and just after will not only benefit the museum audience, but it will consequently deliver results for who developed such installations. However, only a few UX models have been framed and evaluated as successful for museum applications in the scientific literature [45].

4.3. Perceptual Aspects

In terms of visual perception, the best out of a hologram can be obtained when the three-dimensional depth of the overall effect is pushed at the maximum, but when does this happen, actually? The effectiveness of a hologram does not depend exclusively on the quality of its graphic content, on the beauty of its animations, or on the resolution of the reproduction device. The hologram presents substantial differences compared to other image-fruition systems familiar to us. In the case of these systems, the images are indeed reproduced over two-dimensional flat spaces (just think of a television, or a computer monitor, or a cinema screen) and all the information about the depth of the space that we face are necessarily already incorporated into the visual contents that constitute them. On the other hand, with the hologram, the visualization space has three dimensions and not two, and if we are not able to make perceivable all of them to the user, even the more sophisticated representation will look like a conventional, although beautiful, video. Therefore, the effectiveness of a hologram depends also on its tridimensional effects.

As a matter of fact, with the hologram, our screen is the entire space that we can embrace with our eyes. The extension of this screen along the x and y axes (width and length axes) can be well recognizable for all of us, especially if the hologram is displayed inside a showcase, but it is around the extension (and perception) of the z axis (that is the depth), that we construct the perceptual impact of a hologram. How can this emphasis on the z axis be obtained?

• First of all, the space along the z axis must be recognizable by the user. For this reason, if the holographic showcase has a solid background, it needs to be as deep as possible, so that the projection plane is not too adherent to the background itself.
• Moreover, the solid background must be somehow visible, a bit illuminated, because the depth (that is the distance between the hologram’s projection plane and the background) must be perceivable, otherwise the effect will look like a movie on a normal screen.
• Visual interferences are a positive factor for a rich perception of a hologram. For this reason, if we leave the background transparent, it is much better to locate a holographic showcase in the middle of a space, rather than close to a wall (on the back). In such a condition, the floating effect of the images will become much more evident (Figure 23). Obviously, in the case of a transparent background, random interferences will occur, and this is not a problem at all. Nevertheless, a very interesting research field is actually the creation of increasingly effective setups, in order to obtain specifically designed interferences in the holographic spectacle. That is to say that a conscious use of visual interferences between real world and holograms is contemplated.

Therefore, the showcase, in its entire depth, can rise to the role of a real theatrical space, conceived with real objects, painted backgrounds, scenography, different levels of depth, all capable of making an ideal complement for a holographic representation. In this way, the aim of a holographic apparition is not to materialize itself in a generic empty space, but to contextualize itself in a precise environment, where it can give the illusion of interacting with things. That is why it may be strongly enhanced in terms of narration and dramatization, maintaining at the same time all its strength on a strictly perceptive level.

4.4. Setup of the Showcase

The holographic showcase requires a specific design for hardware setup and concealment, interior illumination, and overall efficacy of the Pepper’s Ghost effect.

Visibility of the real objects inside the showcase plays a crucial role. The holographic showcase stands as an opportunity to visualize the symbolic and sensory dimensions of the exhibited objects beyond their formal aspect, but without overshadowing the perception of their physical consistency. For this reason, having a good view of the real artifacts is essential.

The holographic projection inside the showcase must have a millimeter accuracy and need to follow precise visual rules in order to produce the expected tridimensional effects, the superimposition and matching with the real objects. Therefore, the position of real pieces is forced in a precise space, that is the ghost projection plane. In the classical Pepper’s Ghost setup with an oblique mirror, the projection plane is not close to the showcase’s frontal window, because the depth of the showcase is an essential factor to enhance the holographic effect: the deeper the showcase is, the more the ghost seems to float magically in the empty space. Thus, real objects remain closer to the background rather than to the front side. This distance can produce negative feedback on visitors because an optimal perception from the front side is precluded: very small details can be hardly appreciated. Nevertheless, the 3D effect helps a lot in this situation, and overcome such visibility problems because digital contents work exactly on these aspects, animating and magnifying object details and moving them to the foreground (Figure 24). Visibility can be solved through:

• A good lighting system upon the museum objects inside the showcase;
• Avoiding as much as possible reflections on the frontal glass closing the window;
• A transparent background. Instead of closing the back of the structure with an opaque panel, a glass or a plexiglass panel can be used, so that visitors can walk to the back and look at the objects from a very short distance, admiring every detail. From the back side, the ghost is not perceivable, because it can be perceived only looking from the front side. Therefore, the holographic showcase can work as a traditional showcase if accessed from the back, and as an ‘augmented’ one if accessed from the front. Of course, in the general environment of the room, the light must not be too intense (a semi-dark condition is ideal), otherwise the light could invade the showcase and the ghost could become not well perceivable in its details and brightness.

4.5. Interaction

Interaction with virtual contents can be a problem in museum environments, especially along the path of visit and especially when it requires complex actions or the understanding of not immediate interfaces. Interaction must:

• Be not an expedient itself, only because it is associated with the concept of technological innovation. Interaction can be an element of technological innovation, but it is not the only one, if misused it becomes a barrier for the public. Technological innovation also lies in the visualization and dramatization of a scenic space through digital technologies;
• Really create added value for the public, that is, be a structural part of the idea of fruition and experience and not an occasional or pretentious element;
• Be supported by the museum staff, who are able to observe the visitors and help them in case they do not understand how to act. Sometimes, this is a problematic aspect in museums because there is not always a dedicated staff inside of them.

The limited interaction can be rewarded with the wonder of narration and vision and with a perfectly calibrated rhythm of the story. Besides, interaction has not to be intended only as a physical action, but also as a cognitive action, generated by the multiplication of the perceptive layers, as it happens in the case of holograms, where visitors can experience augmented and mixed reality.

Surely enough, a low level of interaction is needed with multimedia located along the path of the visit, allowing, for instance, language selection. Even in the case of brief contents, people are not so patient to wait until the beginning of their language: they want to choose immediately, otherwise the risk is they could skip the content. If, on the contrary, interaction is required in a holographic showcase, it will be most probably used to manipulate virtual objects. In this case, specific devices can be used, for instance Leap Motion for gesture-based interaction (as in “Keys2Rome” exhibition mentioned in Section 2.2). Users’ engagement is mostly generated by active exploration, rather than by dramatized stories.

Until now, surveys on the public of holographic showcase developing a story in museums, have not revealed that interactivity, in terms of manipulation and selection, is considered as an attractive element by the public (see Section 3.5.3).

4.6. Storytelling, Cultural Transmission, and Dramaturgy of the Object

If we want to transmit meanings, beliefs, values, personages, events, contexts of usage, we need to create a story around the exhibited object. The integration and interaction between real and virtual contents greatly enhances this experience.

Cultural transmission can be approached in two ways: with a simple description or with a dramaturgy or evocation. In the first case, the style is more traditional, impersonal and ‘flat’, lacking in emotional power and personal perspectives, while the rhythm is regular. The impression is impartial, of something familiar and already seen in museums, even if applied to something new—like the holographic showcase. The magic power of the Pepper’s Ghost exists in itself, and is not enhanced by the pathos of a story.

In the second case, objects become the occasional points where history ‘coagulates’ and a dramatization takes place. The story is built to create an expectation and brings the visitors in the middle of an emotional experience [33].

Some basic rules are used in tales. Usually, the story develops throughout three phases in order to catch the attention and curiosity of the public: (a) order, (b) transgression, (c) restored order and security. The classic tale contains a dialectic between law and transgression. In every story, the moment of transgression is always important, evil is always more fascinating than right. The tale is developed by including different perspectives, different rhythms, and draws up a space-time system in which the viewer/visitor is always in the right place at the right time; such a condition is reassuring and rewarding, allowing visitors to trust again and having the ‘illusion’ of being inside the story. Everything contributes to create an expressive unity: oral performance/recitation, visual mood, soundscape, lighting, rhythms, ‘atmosphere’. The object is the protagonist, the starting and the ending point of the tale. It also opens visions on several meanings and dimensions.

The environmental context in which the showcase is located, along the museum’s paths, assumes a fundamental importance and must be considered by authors, especially in relation to the duration of the story, the rhythms, the sound, in order to be coherent with the environment of the exhibition. Reciprocal cooperation with museum’s curators is mandatory to make good and plausible stories that can be welcomed by all the public and by the experts.

Some starting questions to the museum curators can help digital creatives to collect information upon which they will draft the story. For instance:

• Where was the object found? Was it the original place in which it was used?
• Which period is the object referred to?
• Which was the context of use of the object?
• Do we know something about the identity of the owners? What about their social status and job?
• How was the life in the place where the object was used at that time? Who lived there? Which job or activities did those people carry on?
• Where was the object preserved/used during its daily life? In a house, in a palace, in a religious environment, in a shop, in a tomb….
• Was the object used in the daily life or was it exhibited by the owners just for its symbolic value?
• Did the object belong to a woman or a man?
• Which historical events can be connected to the object’s life? Is it possible to mention any episode in particular?
• Was the object belonging to a standard typology or was it unique and special? Was it a cultural transmitter for successive generations/cultural patterns?
• Which materials was it made of? What about the manufacturing process?
• Which is the provenience of the object’s materials?
• Which main characters can be associated to the story of the object? Warriors, merchants, politics, artists, goldsmiths, emperors…?
• Is the object connected by a common significance to the others exhibited beside (in the same showcase or in the showcases nearby)? If yes which are the most notable relations?

There are also some useful questions in order to know the curators’ opinion about digital restoration:

• What about digital restoration of this object? Which are the limits according to you?
• Do you agree with virtual reconstruction of shape and decoration? Do you agree with simulation of the original aspect of the materials?
• Do you think that the same criteria of real restoration should be respected also in the case of virtual restoration, (distinction between original and restored/ different layers of visualization)?

Storytelling definitely becomes the means to recreate the context, to penetrate the form and the significance of things, opening a vision on the lives of populations and historical processes. The reconstruction of sensory and of symbolic dimensions that are ‘beyond’ the object’s appearance can thus take the visitor in the middle of a lively and powerful experience. On this, we can report that when the museum’s visitors have been asked about what they liked mostly about such a “box of stories”, the most common answers have been:“How the story is told”, “The combination of information, lights and sounds”, “The chance of not reading the usual museum panel to understand the object”, “History becoming alive”, “The modularity of the story: it changed every time” (referring to the phases of the story).

4.7. Audio-Visual Grammar to Creating the Illusion

Given that the holographic effect is an illusion of reality, we need to simulate something that seems to be happening under the eyes of the spectators. Thus, in the conception of a showcase using Pepper’s Ghost effect in a museum, there are specific rules and constraints that need to be respected, especially in the visual grammar (Figure 25). The show, in fact, is much closer to a theatrical dramatization rather than to a movie or a generic multimedia animation.

Audio-visual grammar includes some of the following rules.

4.7.1. Coexistence of Real and Virtual

The most powerful impact, in terms of surprise and enhanced experience, derives from the coexistence of the real objects and the virtual projections (ghost) on the stage. There are many substantial conceptual and technical differences between a showcase without or with the real object inside. In the presence of real objects, the designer of virtual contents has much more constraints to respect to create a convincing integration: scale factor, point of view, camera FOV, calibration, matching of lights and colors, shadows coherence, etc.

4.7.2. 3D Graphics

If the ‘ghost’ must be credible and realistic, 3D graphics are needed. Under this condition, the illusion of reality becomes convincing: the stage is a 3D space where events are happening. 2D graphic can be used as well, but in this case the difference in terms of consistency and style will be evident and the Pepper’s Ghost effect will be similar to an illustration.

4.7.3. Background of the Ghost

Virtual contents must be like apparitions in the empty space; usually a black background in the digital animation is suggested. If the showcase has glasses both on the front and on the back and it is possible to watch through the space from one side to the other, the black background of the animation will not be visible; it will appear as transparent because black means no light reflected in the ghost.

4.7.4. The Whole Image in the Frame

The illusion of reality imposes that virtual contents should be entirely contained in the frame, without cuts on their borders. For example, the ghost can detach from the real artifact and then floating in the darkness, or it can emerge from a distance, always in the darkness, coming in foreground and positioning on the real object. It can even enter in the frame from a side but, once it is inside, it must be completely included in the real space of the showcase (as it happens in the stage of a theatre but differently from a cinema movie).

4.7.5. Scale Factor

In absence of real objects in the showcase, the scale of representation is not so much important, because there are no dimensional references on the stage. On the contrary, when a real object is inside, the scale of the holographic projection needs to be real, correctly proportioned to create a precise correspondence between real and virtual contents, especially if the two interact each other. This does not mean that the virtual content cannot zoom in and out. Images floating backward and forward in the empty space can be useful in the story and can contribute to the magic effect; the same occurs for very small details detaching from the real object and coming in the foreground, to be better perceived. Nevertheless, the original proportion and correspondence with the dimensions of the real artifact must be visually declared at the beginning or during the animation, in order to be kept by the visitor.

4.7.6. Position of the Real Object on the Stage

One more constraint is the position of the real object on the stage. The reflected images (ghost) lay on a well-defined plane of projection that is unique. Only the perfect overlapping of the real object with this plane can ensure the possibility to perceive the integration of reality and illusion. By means of virtual masks, it is possible to create the illusion that virtual contents are behind the real object. If we have an animation of these virtual images moving behind, the mask must be painted frame by frame with rotoscoping techniques.

4.7.7. Still Camera

Due to the illusion of reality, a unique camera position (coincident with the visitor’s eyes) is needed and must be kept. Multi-camera is not allowed, because this would immediately bring us into the cinematographic domain rather than in the theatrical one.

4.7.8. Camera Point of View and FOV

In the presence of real objects in the showcase it is important to keep a coherence between the point of view of real and virtual contents, as they must interact with each other. During animations, the virtual objects can move and rotate, but a correspondence must be visually suggested in some moments of the animation, as in the case of scale factor. It is also very important to render 3D graphic contents using a camera with a field of view coherent with the perception of the real object (i.e., 50 mm full frame).

4.7.9. Camera Depth of Field

Everything in the scene must be in focus, because of the illusion of reality. For the same reason also motion blur should be avoided. HFR (High Frame Rate) is recommended to escape jittering (48 or 50 or 60 frame/second, progressive video).

4.7.10. Light and Color Matching

In presence of real objects inside the showcase, real lights are directed on them to make them visible. It is very important that virtual lights, set for the ghost, match with the real ones in terms of intensity and color, in order to create a plausible compositing.

4.7.11. Shadow Coherence

If the real object inside the showcase is contextualized in a virtual scene that emerges all around (as in a chroma key scene) it is important that the shadow of the real object is simulated and rendered in the virtual frame, to avoid a floating and incoherent effect.

4.7.12. Charm

The magic effect can be enormously favored if the rhythm of the show is correctly calibrated. Essentiality and slowness can be very useful because they are able to ‘suspend’ the events in a metaphysical dimension, thus enhancing the significance and the preciousness of each detail. In the holograms, less is more.

4.7.13. Sounds

Sounds are fundamental to suggest the sensorial dimension beyond the exhibited object, creating an engaging experience for the public. Unfortunately, the integration of sounds inside a museum’s space is usually very critical. Immersiveness would require a wide sound projection, in a large space and with a good volume. Wifi earphones could be a good alternative, but they have some critical aspects (hygiene, protection, inclusion, and social relation of many users at the same time). To reduce the noise, some museums prefer to use sound showers but they are mono and usually cut off many frequencies, compromising the perception of a sophisticated sound design.

4.7.14. Sound Spatialization

Stereo sound spatialization (sounds entering from one side and moving towards the opposite direction) is possible if the dimension of the holographic showcase is large enough and the space of the sound projection is wide.

Dolby surround 5.1 can increase even more the immersiveness of the experience but, to be enjoyed by visitors, it requires a dedicated secluded space where the installation should be located—as the audio speakers need to be located all around the public, taking the installation as central reference point.

4.8. Overlapping of Real and Virtual

A mixed reality environment, where real and virtual objects are overlapped and constantly interconnected, is created inside the holographic showcase. Mismatching or shape differences between real and virtual objects could tear-down the illusion of reality and nullify the sensory involvement of the visitors brought by the realism of the holograms. An accurate and meticulous work of 3D graphic is mandatory for a successful integration of virtual contents, tailored to this new communicative visual language.

For instance, the 3D graphics produced for our case studies (see the Kunàgota sword and Mitylene treasure case studies), is created by mixing and integrating 3D scanning and 3D modeling techniques. The former is a process of capturing, through active (laser scanner) or passive devices (cameras), shape and appearance of physical objects and processing them into digital 3D replicas. The latter, instead, is the process of designing a 3D representation of an object through computer graphics software, in a simulated tridimensional environment.

4.9. Virtual Replica and Virtual Reconstructions

During animations, the artifact shown in the showcase ‘comes up to life’. When the story starts, images start moving, floating, zooming in and out highlighting details and reliefs. Of course, it is not the real object moving but the virtual one—which is perfectly overlapped thanks to an accurate projection mapping. To guarantee this illusion of reality, the correspondence of the point of view and the visual coherence between the real artifact and its digital replica are essential. The 3D models representing the virtual replica of the authentic objects are created using 3D scanning technology.

According to the scale of projection, desired level of accuracy and budget, a Structure From Motion (SFM) approach can be chosen to obtain quickly an accurate 3D model reconstruction with color information. SFM is a photogrammetric range imaging technique which allows three-dimensional models to be obtained from two-dimensional image sequences [46,47] (Figure 26). According to our experience and the most recent literature, this technique can be successfully adopted for digitizing museum objects even in case of artifacts with complex geometry and complex material properties, like absorptivity, reflectivity, and scattering [48]. It indeed offers multiple advantages: not destructive impact on materials, short time of acquisition and post-processing of the data, very detailed textured models as output, economical and portable equipment. Moreover, the acquisition can be done directly in museums, even supported by internal staff of photographers.

In order to obtain 3D digital replicas of museum artifacts we adopted the following workflow:

• At the beginning, some precautions are kept during the acquisition, like (a) the positioning of the museum artifacts to be digitized; (b) the number and point of view of the shoots; and (c) lighting setup [49].
• When the acquisition has finished, the entire dataset of images is imported within Agisoft Photoscan, SFM software, to find correspondences between images and perform the photogrammetric model. During the image alignment, the software automatically finds tie points and camera parameters. Subsequently, dense cloud is computed.
• After the optimization of the dense cloud, the mesh is calculated using the Poisson surface reconstruction algorithm which allows to reconstruct a triangle mesh from the point cloud. The last step is the parameterization of the model and the generation of textures. As texture generation parameters, a combination that uses the weighted average value of all pixels from all images of the dataset to blend them in a single atlas texture was used to map the model (Figure 27).

Virtual reconstructions play an important role in the digital archaeology domain and the evolution of the 3D technology. Theoretical and methodological discussions around the definition of virtual archaeology have significantly increased its use and application for research activities and museum applications [50,51,52].

A virtual reconstruction is the act of rebuilding a destroyed or lost heritage (monument, artifact, or architecture) to its entirety, at the time of its creation or at a specific moment of its past life. In our case study, we reconstructed not only the original appearance of the museum objects (Kunàgota sword and Mytilene treasure) but also their usage in order narrate their history and recall their evolution through the time.

Reconstructing the past is a complex task which requires a big work of survey, source analysis, discussion and interpretation, before proposing a hypothesis. Furthermore, in order to guarantee scientific consistency and clarity, the virtual reconstruction workflow should follow theoretical principles, as fixed for instance in the London Charter and in the Principle of Seville [53], whose purpose is supporting interpretation and simulation of the past basing on a scientific approach.

4.10. Monitoring and Recording Users’ Behavior

By monitoring and recording users’ behavior, researchers (and the same museum staff) can obtain a crucial understanding of what museum visitors are actually doing once in front of the holographic showcase [54]. It allows us to know exactly what people are looking at, what they like or dislike, and how they are using the installation, what they are learning, giving the unique opportunity to make modifications that will guarantee a better usability and satisfaction in the next future. Moreover, knowing the public creates an opportunity to identify ideal areas for further development, giving to researchers and developers the chance to craft something tailored to the final customers. The information acquired by monitoring user experience is an invaluable asset to any digital cultural heritage project to improve the effectiveness and efficacy of multimedia inside museums.

A great variety of methods are available to conduct UX research on public. Observations, direct questionnaires, focus groups, driven-scenarios, simulations, and quizzes. Not all of these are really valuable for UX evaluation of digital installations inside museums. Some can be employed singularly, others work better if mixed and confronted with others. Nevertheless, it is not automatic and easy to assess user experience since the latter is subjective, context-dependent and dynamic over time. Moreover, it is conducted by humans so it can be partially influenced by operators. The researcher has thus to select the right methods, target, tools, and choose a specific area of interest such as game or digital application.

4.11. Sustainability

According to the V-Must classification of virtual museum (VM) [38], sustainability is one of the key aspects since one of the negative experiences reported in the creation of VMs is the lack of policy and strategy regarding the future persistence and sustainability of applications [55].

The holographic showcase setup complies with this important aspect since it is fully reusable from different point of view:

○

Maintenance—The holographic structure is characterized by a reusable and long-lasting setup. The hardware is made with aluminum sections bars and panel that can be adapted to every location and can be easily replaced; it is easy to realize and stable in the everyday management.

The software to manage the entire application, is completely customizable to control different type of media: video, sound, lights, touch and physical interfaces, sensors for environmental monitoring. The computing and networking infrastructure is robust and can be continuously monitored and updated via web using a remote controlling software.

○

Compatibility—Among the holographic techniques, the Pepper’s Ghost effect is not only able to produce a high-quality 3D perception, it is also fully compatible with the conservation needs and the museographic constraints, as it does not damage the museum’s artifact in the showcase. Furthermore, the objects are completely inaccessible and protected within a secure holographic showcase.

○

Exchangeability—The structure can be easily mounted and dismounted in a couple of days and can easily travel together with museum objects for temporary exhibitions, as experimented in the CEMEC project (https://cemec-eu.net) where the Kunàgota sword was shown in different locations (Budapest, Amsterdam, and Bonn). Another advantage of this kind of installation is that it can work also with physical replica of the museum artifacts. The recent evolution of 3D printing technologies allows 3D scanned models of objects, originally digitized to produced virtual animation, and then to be also used for printing physical copies.

○

Scalability—The creative workflow and the overall setup is completely adaptable and scalable since it can be adjusted to different museum contents. The holographic structure (dimension, setup, multimedia duration, type of visualization, integrations of projection wall,….) can be thus tailored on diverse situations we can encounter inside museums.

Furthermore, the use of 3D printing opens up new possibilities: museum objects that cannot be included in the holographic showcase, can be whereas printed with 3D machines; this allows the museum pieces to be as well exhibited, in case of impossibility to expose the original one for (a) security reasons, (b) conservation issues, or (c) in case of exchanges with other museums for itinerary exhibitions or simultaneous ones (Figure 28).

5. Conclusions

Holographic showcases inside museums represent a great opportunity to evolve towards a better and harmonic integration between collections and multimedia and to solve more effectively the traditional lack of communication of the museum’s artifacts. A methodological proposal about how to use holograms in the best way in museums has been presented in this paper. A wide set of precautions and processes have been described that should be taken in consideration to build a proper visual grammar and powerful stories using this very specific technique. The purpose of using holograms is to create an illusion of reality in front of the visitors’ eyes, through the apparition of tridimensional images floating in the empty space and giving the impression to be interacting with our physical world.

Dealing with holographic projections, however, the authors do not refer to real ‘holography’, as invented by Gabor in 1947 and further developed in the 1960s. That optical technology records a visual information in the form of a very fine interweaving of interference fringes using coherent laser light, appropriately projected. Even if in these years the holographic technologies have greatly advanced, they are still immature to be applied to cinematic and multimedia. In fact, the obtained images are not satisfying in terms of resolution, details, colors, field of view, frame rate, also because viewers are not ready yet. For this reason, the most used technique in the field of performative arts and museums is the Pepper’s Ghost, that became very famous in the second half of the XIX century inside theaters. Today, digital technology allows artists and researchers to easily use this technique in a scalable way, with new improvements and automations. What is really challenging is the communicative paradigm, the new kind of narration, more oriented towards simulation and dramaturgy.

Several examples of installations using Pepper’s Ghost technique have been presented, together with a more detailed case study. The latter, in the last part of the paper, is referred to the European project CEMEC, Connecting Early Medieval European Collections, still in progress (2015–2019), that has brought the authors to experiment this technique in the context of an itinerant exhibition hosted inside important museums in different parts of Europe. Here, the Pepper’s Ghost effect has been used inside showcases, in presence of the real artifacts, in order to produce an effect of mixed reality and a dramaturgy beyond the museum objects. Not only a technical know-how has been acquired during the four years of work, but also many useful data about the impact of such kind of installations on the public and curators, under specific conditions.

The design of a good user experience for holographic showcase is a great challenge both for curators and developers and it passes through the reflection on certain key factors:

• the context of use
• the environmental conditions (silent or noisy, dark or illuminated, secluded or crowded)
• the target age
• the conditions of content accessibility
• the time of usage
• the possibility of intervention on the physical space hosting the installation
• the expectations of curators and visitors.

All of these aspects must influence the conceptual structure of any digital application inside museums, especially the ones which play with holography. It is no more feasible in the 21st century to design and develop digital applications without taking into account the final users and their emotional involvement—either common people or experts and professionals. Consequently, the selection and choice of the style of storytelling, the interaction interface, the sense of embodiment, the choice of technology and the appropriate software architecture derive from the above-mentioned issues.

As presented in Section 1, it is common to find an occasional static touch screen in a corner of a cultural heritage site, which contains pages of dry information about certain artifacts and collections, perhaps accompanied by a couple of images; likely, you can find a site-specific immersive application set aside from the museum visit path rather than a mobile application not really integrating the real museum objects with their digital replica. This is hardly inspiring, particularly in the eyes of common visitors, who are not expert in this specific field or belong to younger generations—for whom technology has become the very center of their daily lives.

It is obvious that museum managers and curators need to take seriously the interest and retention of these target groups to secure an intense and lively experience of the cultural heritage venues. Incorporating digital technology ‘within’ the physical space is one such way to do this, providing more linked and deep occasions of fruition and understanding.

When visiting museums, three conditions are still missing today:

• No stories are told about each single museum object about the owners, the place of belonging and manufacturing and the period of its persistence. Only a cold list of facts and factual information are often presented in captions or panels inside museums. The practice of storytelling is yet not fully exploited inside cultural venues.
• No relations are revealed and brought to light between objects of the same collection; museums address their exposition ordering the pieces by age or provenance and not by theme or subject.
• No contextual description about museum objects is presented. Specifically, reconstructions about the function, the usage, the environment in which it was included, the atmosphere of that specific period in history and so on. A long tradition of physical paper-based setting reconstructions can be today retractable in museums [56]; however, with the development of technology and the advancement in multimedia techniques, the same reconstructions can be easily translated in 3D and made interactive, allowing more involvement and immersiveness for the final users.

Why emotional involvement is so relevant for experiencing cultural goods in museum spaces? Technology is a mean which help us shaping the sense of involvement in the digitally augmented reality. The public is usually attracted by the technological solutions.

However, the several surveys we carried out observing the people’s behaviors, showed very clearly that the attention towards technology in museums is not long-lasting; it rapidly decreases if the cultural contents are not able to keep it alive and, similarly, if the interaction is difficult and not natural.

For such a reason, it is necessary to rethink technologies at a deeper level of usage. But how? By working on a cognitive and emotional level. The emotional component is fundamental in any learning experience because it is the ‘irrational’ part of human being and it generates motivation—that is the first aspect that pushes people to face technological solutions and lets the learning process take place [57].

If we think about the holographic showcase, the processes of creating emotions, raising curiosity, and stimulating a sense of wonder do work the same way. This innovative solution can be really an answer to the newly born necessities of museum curators to listen to visitors of all target groups to foster the culture.

Reactions towards holographic showcases have been investigated through specific surveys, carried on in museums of different countries [39]. It was interesting to discover that the holographic techniques usually meet the expectations of peoples of different cultures and provenance, and the needs of the curators of different museums. What is undeniable is the attractiveness and usability of such installations, and the naturalness of their integration inside the museum’s space along which the visitor’s experience develops.

Bringing the virtual inside these showcases, interacting with the real artifacts in a harmonic way, creating a dramatization, is the most useful and pleasant experience that we can imagine with digital technologies inside museums, for a large audience of different ages and provenience. The first aspect of innovation that usually is appreciated refers to the different way of presenting the museum pieces: they are not only well illuminated and supported by a close caption describing few features of them (as in traditional showcases); they are rather ‘duplicated’ in their tridimensional shapes, they are ‘augmented’, virtually reconstructed and restored in some cases; in some others, embraced into an evocative atmosphere; words, images, and sounds fluctuate around the objects inside the holographic showcase, highlighting details or adding contextual parts.

The communicative style can vary to meet the expectations of curators and the cultural patterns of the public of different regions, without giving up the main guidelines and criteria that have been presented above. Some curators may encourage an amazing and bizarre approach in the use of virtual technology in their museums, others prefer a more classical style. The augmented experience of mixed reality offered by holographic showcase is cognitively interactive in terms of intellectual engagement of the visitors, even where no or few choices or actions are required, because different levels of perceptions, interpretation, and suggestions are created in the showcases.