Evaluating the Usability and the Personal and Social Acceptance of a Participatory Digital Platform for Cultural Heritage

Abstract

Nowadays, through the Internet, cultural heritage reaches broader audiences via digital platforms that manage and disseminate cultural content formulating a common sense in societies and supporting society’s cohesion. Various cultural heritage digital platforms embraced the idea of user participation in contributing cultural content. In this work, we evaluate the usability and the personal and social acceptance of a specific participatory platform (Culture Gate) that manages, disseminates and exploits cultural content and services. We associate platform usability with the level of engagement platform’s content and services bring on users. Personal acceptance is related to specific factors like the users’ disposal to disseminate and exploit platform content and services or to actively participate in a digital community. We relate social acceptance to specific factors like the promotion of users’ responsibility towards the online community and the society, the support of intergenerational dialogue and the amplification of a user’s sense of belonging to a society. Evaluation methodology applies a research model based on hypotheses, deriving from the ideas of widely accepted sociological and economic theories, which reflect each of the aforementioned factors. Evaluation results suggest that the test-bed platform is considered usable and acceptable, in a personal and social level, by the users.

1. Introduction

Nowadays, a plethora of technological breakthroughs appears daily in our lives shaping the communities of today and tomorrow. The development and domination of technologies in various areas, from energy and housing to entertainment and interpersonal communication, threatens to leave behind other important aspects of human life like active social participation. Cultural heritage as an aggregation of tangible (buildings, monuments, landscapes, books, works of art and artefacts) and intangible elements (folklore, traditions, language and knowledge), comes in aid of contemporary people as units and societies as groups [1]. Usually, cultural heritage gives individuals and societies a sense of connection and cohesion with the past and reshapes strong bonds between individuals of various ages and the society they belong to [1,2], even though considerations have been recorded about social exclusion phenomena deriving from the misconception of the meaning and scope of cultural heritage [3].

In order to adjust in a volatile environment, cultural heritage has incorporated throughout the last decades various technological applications as a means to document, preserve, store, disseminate and exploit cultural heritage content [4,5,6,7,8]. Digitization and storing of cultural heritage content, creation of online collections, dissemination of cultural content through the Internet, eMuseums, interactive storytelling, cultural heritage mobile applications, augmented and virtual reality applications, 3D reconstructions of cultural heritage elements and serious games are only some of the aspects of digital cultural heritage.

As cultural heritage and technology are becoming more and more intertwined, new research questions arise. How can we measure the personal and social acceptance or the usability of a platform that manages digital cultural heritage resources? Which axes should we use to extract a user’s opinion about acceptance and usability? Could we formulate a model that explains and predicts the adoption and success of a participatory digital platform for cultural heritage?

The term acceptance describes the users’ attitude towards a digital platform, whether the users would adopt the platform and use it or reject it as irrelevant. If a user accepts a platform then she/he is more likely to use the platform now or in the future. Andrew Dillon in Reference [9] defines user acceptance of information technology as “the demonstrable willingness within a user group to employ information technology for the tasks it is designed to support.” Dillon states that “acceptance theorists are less concerned with unintended uses or non-discretionary use of technologies and more interested in understanding the factors influencing the adoption of technologies as planned by users who have some degree of choice. By developing and testing models of the forces shaping user acceptance, human factors researchers seek to influence the process of design and implementation in a manner that will minimize the risk of resistance or rejection by users.”

The idea of active user participation in generating, collecting and contributing personal content is not new to digital cultural heritage [10,11,12,13]. Mia Ridge in Reference [14] states that “crowdsourcing in cultural heritage is more than a framework for creating content: as a form of engagement with the collections and research of memory institutions, it benefits both audiences and institutions.” In a participatory digital environment, many users contribute small portions of information, thus resulting in a system that stores vast amounts of content. The idea is extremely successful in another popular application type of the modern era: social media. In participatory platforms, users generate their own content, share it with the community or read and review other users’ contributions. This interaction leads to the formation of a vivid online community that, in the case of cultural heritage digital platforms, focuses on cultural heritage. Online communities share a series of common characteristics like the increased user engagement and the increased number of content contributions, characteristics that are desirable in any modern online platform.

Cultural Heritage as a sector that comes from and shapes humans and human societies has always been open in adopting and adjusting ideas from proven theories of other scientific fields like economic studies or social studies. Previous research efforts have shown that the successful use of active user participation in digital cultural heritage platforms depends on the psychological factors that influence users’ to use such information tools for their benefit and for contributing in the highlight of the society (local, national, global) they belong to [15,16]. Within the context of active user participation in cultural heritage digital platforms, various well-known theories from economic, social and information technology studies can help us to examine the factors that influence end users personally and socially. Examples of such theories are the Social Identity Theory (SIT) [17,18,19], the Customer Loyalty Theory (CLT) [20,21,22], the Social Cognitive Theory (SCT) [23,24,25] and the Radical Collaborative Approach (RCA) [26]. However, research regarding the factors which contribute to personal and social acceptance from using participatory platforms in cultural heritage domain is still limited. One such example is the study introduced in Reference [27]. Authors investigated the social and personal impact and effect of a digital platform called CLIO in fields like participation and engagement, belonging and social interaction and intergenerational dialogue but they did not investigate the evaluation of the users’ personal and social responsibility or the dissemination and exploitation potential of their platform from its users.

In this work, we propose a new research model that gives a first answer to the aforementioned research questions and a mixed type evaluation methodology. The proposed research model develops a series of research hypotheses that derive from the key features of the SIT, the CLT, the SCT and the RCA. More specifically, in our research model we propose a set of factors that influence user acceptance of a participatory digital platform for cultural heritage. We categorize those factors as personal and social. A user accepts the platform’s existence and is willing to use it for personal reasons because she/he can exploit its content and services (exploitation), she/he wants to participate actively in a digital community (participation) and she/he wants to convince other people (peers or others) to become a part of that digital community (dissemination). On the other hand, a user accepts the platform’s existence and is willing to use it for social reasons, concerning her/his relationship with the society she/he lives in, because through the platform the user can get in touch with her/his roots (belonging), can help in the preservation and promotion of the cultural heritage capital of the society she/he lives in (responsibility) and can associate with the older generations (intergenerational dialogue). The proposed research model is applied in a specific participatory cultural heritage management platform to measure its usability along with personal and social acceptance. We believe that this research model is a first step towards the development of an evaluation framework of the acceptance of participatory digital platforms for cultural heritage. The evaluation methodology uses a combination of scientifically accepted assessment tools (questionnaires, personal interviews, usage statistics) in order to assess the usability and personal and social acceptance of a participatory cultural heritage digital platform. For our experiments, we use as a testbed Culture Gate1 [28,29], an online trustworthy participatory platform for cultural heritage that manages cultural heritage digital content in various ways, designed and implemented by the authors of this paper. The platform invites its users (scientists, artists, the public etc.) to contribute small portions of cultural information in digital form. User-generated cultural content is stored, organized and then presented to platform visitors in user-friendly ways. Evaluation procedure takes place in three different stages and focuses on seven axes: user participation in an online community via contributing cultural content and forming an online community, likelihood of platform dissemination to peers and others, user engagement with the platform, platform exploitation (content and services additional value), emotional impact and sense of belonging to a society, promotion of personal and social responsibility and supporting intergenerational dialogue deriving from platform usage. Google Analytics2 (GA) web metrics tool is used throughout the evaluation process in order to collect and analyse usage statistics in each of the three experimental stages. Since the tracking is collected by Google rather than the individual website, this gives us a completer and more objective picture of user behaviour [30].

The analysis of the results from questionnaires, personal interviews and platform usage statistics suggested that the platform could promote active user participation and lead to the formation of an active online community of individuals that work on or love cultural heritage. Moreover, users show great interest in disseminating the platform not only to their peers but also to the broad public. User engagement to the platform is rather satisfying, an indication that platform content and services could be interesting, modern and up-to-date. Furthermore, evaluation results suggest that the platform offers a user-friendly, attractive and easy-to-use environment for users to spend their time searching for or contributing cultural content. Results analysis suggested that users could use the platform in their professional life (since all users where current or future professionals of cultural heritage) generating value from both the platform content and specific services. Additionally, the platform contributes in instilling into a person a sense of belonging to a society. Alongside, analysis of the results suggests that the platform boosts user intention to be responsible towards the platform’s online community and the society in matters of cultural heritage preservation. Finally, our results provide a first glimpse about the positive influence that the platform in particular and cultural heritage in general could have on society’s continuity and cohesion by promoting the intergenerational dialogue bringing together different generations of users, thus helping in the bridging of the unavoidable generation gap.

2. Related Work

Cultural heritage has always been a pioneer field in incorporating new technologies and capabilities. Therefore, many excellent digital platforms, repositories, systems, environments and applications that offer services and manage content related to cultural heritage have been introduced to the scientific community and the public until today [4,5,6,7,8]. Each platform has its own unique impact on both the local, national and international society and on each individual that loves or works within the field of cultural heritage. EUROPEANA3, a flagship project funded by the European Union, is an online cultural heritage platform that interconnects more than 3700 libraries, archives, museums, galleries and audio-visual collections across Europe providing access to over 53 million cultural heritage items like image, text, sound, video and 3D material. The KORA platform is an open source cultural heritage digital repository addressing mainly to archaeologists and museum curators [4]. The CHESS is a research prototype system that offers personalized, interactive digital storytelling experiences to enhance museum visits, demonstrating the authoring and visiting experiences [5]. The “Gossip at Palace” application is a location-based mobile game that integrates a storytelling approach used primarily in museums [6]. The “Ghost Detector” mobile application is an educational location-based museum game for children [7]. SNOPS is a smart city environment for cultural heritage applications [8].

Several cultural heritage digital platforms, systems and applications have included the idea of active user participation as a core or complementary service in their design and implementation [10,11,12,13]. Participatory cultural heritage digital platforms offer individuals the capability to become active parts in generating, collecting, exploring and notifying cultural heritage knowledge. The “stedr” cultural heritage mobile application is a social storytelling application that narrates stories about the history of an outdoor location [10]. 1001 Stories of Denmark4 is a participatory cultural heritage digital platform dedicated to storytelling in Denmark. The Megalithic Portal5 application allows users to contribute photos of megalithic monuments in Europe. The Ancient Lives cultural heritage application asks volunteers to transcribe ancient Greek text on fragments from the Oxyrhynchus Papyri collection [11]. The Know Your Place cultural heritage digital application engages local communities in shaping the stories of their neighbourhoods by allowing contributors to add media and metadata, thus producing archaeologically relevant information [12]. The Heritage Together platform allows users to upload their own photographs of heritage assets, which are then processed into 3D models using an automated photogrammetry workflow [13].

The evaluation of such platforms, during the design phase or even after their launch to the broad public, plays a crucial role in their usability and acceptance. Nina Simon in Reference [31] argues that the “lack of good evaluation of participatory projects is probably the greatest contributing factor to their slow acceptance and use in the museum field.” The author continues by stating that there are 4 considerations to have in mind when evaluating a participatory platform for cultural heritage information management: (i) “evaluation must focus on participant behaviour and the impact of participatory actions,” (ii) “evaluators should articulate goals for the participants who actively collaborate with the institution, for the staff members who manage the process and for the audience that consumes the participatory product,” (iii) “participatory projects often benefit from incremental and adaptive measurement techniques” and (iv) “it is beneficial to make the evaluative process participatory in itself.” In the first consideration Simon states that the evaluators should measure “what participants do and describe what happens as a result of participation” because it is not useful to just present the services of participatory platforms and continues by arguing that the “participatory outcomes may be external, like increased incidence of conversation among visitors and internal, such as development of new skills or enhanced relationships.” The second consideration argues that the evaluation should set a clear goal in all stakeholders. The third and fourth considerations suggest that an evaluation procedure should integrate various methods or tools adapting to the changing environment of the platform and should involve the users of the platform and the moderators that perform the everyday maintenance tasks.

Many research teams have performed usability evaluation studies on participatory digital platforms [32,33,34]. Authors in Reference [32] present a usability evaluation for a web-based public participatory GIS platform, concluding that users with familiarity in GIS, higher education and web surfing experience tend to use more easily the platform than other users who do not have those characteristics. Paper [33] presents the usability evaluation of a participatory platform for young people empowerment in the digital society, concluding that the users should accept the platform in order for the system to fulfil its goal. Authors in Reference [34] propose a conceptual model based on the DeLone & McLean success model [35], that “assesses the citizen satisfaction regarding the perception of the e-participation system quality; the expectation-confirmation model for the continued intention to use, which evaluates satisfaction based on the confirmation of ex-post experience on e-participation use and the perceived usefulness; and the dimensions of sense of place, which play a moderator role between the citizen satisfaction and the e-participation use.” Other theories or models, in scientific literature, that examine the usability and user acceptance of a digital platform, in general and can be applied in participatory designs are the Theory of Reasoned Action (TRA) [36] and its extension the Theory of Planned Behaviour (TPB) [37,38] or the Technology Acceptance Model (TAM) and its extensions [39,40,41].

Cultural heritage capital is apparent in various aspects of human life, influencing mainly positively individuals and societies [42,43,44,45]. Cultural heritage platforms, systems and applications, as carriers and disseminators of cultural heritage information, occupy an important position in today’s world. Many researchers have dedicated their time in examining whether modern cultural heritage platforms fulfil their destiny and at what extent [16,46,47,48,49]. Authors in Reference [47] evaluate the capability of the “Living Avatars Network”, a platform that urges people to experience the cultural heritage of Singapore, to promote intergenerational dialogue. A qualitative evaluation of the benefits that the participatory platform “Stories for Hope–Rwanda” offers to intergenerational dialogue in a post-war Rwanda, is presented in Reference [48]. A cultural heritage platform is evaluated in Reference [49] using qualitative and quantitative tools in order to explore how online visitors connect with virtual heritage. An evaluation concerning the valorisation of GeoPan Atl@s, a cultural heritage application that hosts information about the history and historical changes that took place in the cultural landscape of the Lombardy region in Italy, is presented in Reference [16]. Authors attempt to reveal the impact that the aforementioned application has on activating user participation and on raising awareness about the status of cultural heritage landscape in the Lombardy region. The social impact of MOSAICA, a system that urges users to engage actively in preserving cultural heritage via activities such as investigation, exploration and storytelling, is evaluated in Reference [46]. MOSAICA is also evaluated for the quality of its content, its attractiveness and user-friendliness. Author in Reference [45] examines the personal impact of a well-designed crowdsourcing cultural heritage application on individuals and performs an evaluation of audience engagement and user participation in the crowdsourcing procedure.

Several methodologies have been proposed by the scientific community to evaluate a cultural heritage digital platform, system, application or tool [27,46]. The evaluation methodology of MOSAICA [46] follows the mixed methods research model [50] that combines both quantitative (through questionnaires) and qualitative (through personal interviews) evaluation tools. Moreover, the evaluation process consisted of three stages and it was conducted on a group of 20 individuals (amateurs and professionals) with strong interest for MOSAICA’s content and services. The evaluation methodology of CLIO, an urban computing application that allows forming and interacting with the collective city memory, dictated the application’s installation on the streets of Corfu, Greece and Oulu, Finland [27]. Evaluation process in Corfu involved 30 students and 27 adults which were asked to use the application in real-time. The experiment team observed and recorded user reactions, took personal interviews and distributed questionnaires to the users. Evaluation process in Oulu was conducted in the streets of the city were individuals were asked to use CLIO to explore collective city memory and share their own memories via smart personal devices. The experiment team observed and took notes over the behaviour of random users and also formulated a group of 12 users with equal distribution among genders and ages from teenage up to sixties which were asked to complete specific tasks, answer a questionnaire (to assess user-friendliness) and give a personal interview (to extract people’s views of the system). During the experiment, usage statistics were recorded for further analysis.

A review of the evaluation literature pointed out various kinds of evaluation tools (methods and instruments) for collecting evaluative information in order to examine the personal and social impact of a cultural heritage digital platform. Three of the most popular tools are the questionnaires, the personal interviews and the analysis of usage statistics [51]. Questionnaires are used as an evaluation tool since the early days of software applications [52,53] because they are inexpensive, quick and easy to implement and analyse. On the other hand, questionnaires could be confusing for the interviewee or they could provide a limited number of possible answers leading interviewees to insincere responses, jeopardizing their reliability [54]. In order to ensure that a questionnaire is reliable, several measures have been introduced to the scientific community most common of which is Cronbach’s alpha [55]. Cronbach’s alpha determines the internal consistency or average correlation of items in a survey instrument to gauge its reliability [56]. Personal interviews are a qualitative research tool that involves conducting individual interviews with a small number of respondents to explore their perspectives on a software application. Interviews are useful for extracting detailed information about a person’s thoughts and behaviours or want to explore new issues in depth. Interviews are often used to provide context to other data (such as outcome data), offering a more complete picture of a software application. On the other hand, personal interviews have some limitations like biased answers, excessive time load in implementation and analysis and conclusions that are not easily generalizable [57].

In questionnaires and interviews, users say what they would do during a website visit. On the other hand, usage statistics monitor how users actually work while visiting a website [58]. Google Analytics (GA) web service, is a web analytics tool offered by Google, that tracks and reports website traffic. Launched in 2005, GA has become the most popular tool for viewing website usage statistics, according to web portal w3techs.com6. The last decade, many researchers have used GA in order to collect access data and examine user behaviour during a website visit [30,59,60,61,62]. Author in Reference [59] highlights the reasons why choose GA to measure access data in cultural tourism websites like the fact that GA provides time series data, it is free and it is a user-friendly application with the guarantee of Google technology. The importance of time series data is also acknowledged in Reference [60], where the author explains that “the analysis of the effectiveness of a site’s traffic source lies necessarily in the use of time series analysis.” Authors in Reference [30] state the superiority of GA as a usage statistics collector in comparison to traditional server logs and the tool’s effectiveness in understanding the behaviour of users in the virtual environment. Besides measuring user behaviour, GA is an excellent tool for evaluating the overall usability of websites, improve the design and content of web sites and to identify potential usability problem areas [61,62]. Authors in Reference [61] discover specific GA metrics for evaluating and improving content, improving navigation, evaluating accessibility and improving design of a website. Authors in Reference [62] use the GA service to evaluate and improve the design and content of an online library website [62].

3. Materials and Methods

3.1. Culture Gate—Cultural Heritage Participatory Digital Platform

Culture Gate [28,29] is an online participatory platform that collects, manages and disseminates cultural heritage content (Figure 1). The platform architecture is modular for security and scalability reasons. As a participatory platform, Culture Gate asks users to contribute cultural heritage digital content. The platform collects user contributions, organizes each contribution in cultural items and displays them on the public in user-friendly ways (thematic lists, geographical cultural map). A user is allowed to contribute content only if she/he registers an account on the platform and thus becomes a member of an online community that focuses on cultural heritage. The platform uses experts to author user contributions in order to increase the quality of contributed content, after the user-contributor initiates the authoring procedure by marking her/his contribution as “Needs Authoring.” This mechanism does not check all users’ contributions but rather distinguishes and highlights the ones that are marked as “Authored.” Platform users can search for content associated with all major cultural disciplines (Archaeology, Architecture, Theatre, Music, Museums, Libraries, Folklore, Literature). Additionally, users have the opportunity to exchange opinions about cultural heritage topics in platform’s discussion forum or be informed about upcoming cultural events in their area from the cultural events calendar service. The platform can be accessed via desktops and laptops, tablets and smartphones from indoor and outdoor locations. Moreover, users can download and install the Culture Gate Collector, a dedicated mobile application, which is used for capturing cultural content in real-time, annotating it and uploading it directly to the platform from everywhere, at any time. Culture Gate also provides a friendly environment for professionals of cultural heritage to exploit its content and services. Culture Gate characteristics, services and content make this platform an ideal candidate for evaluating the personal and social impact of a cultural heritage participatory digital platform.

3.2. Research Model

We next develop our research model and hypotheses by re-synthesizing the key elements of various established theories in economic and social studies [17,20,23,26]. We are interested in demonstrating the level of usability and personal and social acceptability of a participatory cultural heritage digital platform, for its users. The Social Identity Theory [17] and the Social Cognitive Theory [23] are considered fundamental theories in social sciences. The Customer Loyalty Theory [20] is also a fundamental theory in economical and marketing studies. The aforementioned theories have been used a basic research tool in numerous scientific publications the last 30 years [18,19,21,22,24,25]. The Radical Collaborative Approach [26] on the other hand is one, maybe the first, theory that provides a theoretical basis in the field of online participatory platforms. The impact of the Radical Collaborative Approach is already considered significant in the scientific community of participatory digital platforms. The SCT comes from psychology and explains that individuals tend to be influenced by the actions of other people. A person can be triggered to perform an action if she/he observes other people to perform the same action and the consequences of that action. More specifically, the SCT considers the unique way in which individuals acquire and maintain behaviour, while also considering the social environment in which individuals perform the behaviour. The theory takes into account a person’s past experiences, which factor into whether behavioural action will occur. These past experiences influences reinforcements, expectations and expectancies, all of which shape whether a person will engage in a specific behaviour and the reasons why a person engages in that behaviour. All evaluated factors in our research model have a clear connection with the aforementioned key feature of the SCT. The SIT comes from social psychology and states that an individual’s actions and behaviours derive from that person’s perceived membership in a social group. More specifically, the SIT argues that groups/communities give individuals a sense of social identity: a sense of belonging to the social world. A person tries to increase her/his self-image by enhancing the status of the group to which she/he belongs. Based on the aforementioned human characteristic, if a person becomes a part of a digital community then she/he will try to perform actions that will promote the digital community to the broad public in order for her/him to gain positive “reputation.” The evaluated factors of active participation, dissemination, responsibility and belonging are deriving from the characteristics of Social Identity Theory. The CLT comes from Marketing and explains that satisfied customers of a business tend to be more conscious about the success of the business and they are uncomfortable to buy similar products/services from competitors. The satisfied customer will use the brand’s products or services vividly and will try to spread her/his good word to other people. In some occasions the customer could take a step further and use the brand’s services for professional reasons in order to produce personal revenue. Adjusting the above characteristic to a digital platform, we can conclude that a happy user would disseminate the platform positively to the broad public, would use the platform content and services excessively and could exploit the platform’s content and services. The evaluated factors of exploitation, dissemination and participation are deriving from the characteristics of the CLT. The RCA is considered one of the first attempts to provide a theoretical basis in the field of online participatory platforms. The foundation of the theory is that the idea of active user participation should not be limited in content collection but it should be present in every stage of the evolution of an online platform. The users should participate in the design, development, content collection, maintenance and upgrade of the platform. Engaging the user in every step of the evolution of a digital platform is a very important factor that contributes to user satisfaction and acceptance since the user believes that she/he has played and plays a key role in the platform’s existence. The evaluated factor of active participation is derived from the RCA in a more obvious way than the other theories, as the theory refers to online participatory platforms like Culture Gate.

3.2.1. Usability

The term usability of a digital platform has a proven connection to the environment’s perceived usefulness and ease-of-use [63]. Both factors are related to the notion of engagement. A platform that offers modern, interesting, interactive, easy-to-use, attractive, personalized services and qualitative content tends to engage users for longer sessions. Digicult 2002, a European report devoted to the use of ICT in cultural heritage, identifies specific user expectations from a cultural heritage digital platform like user-friendliness, simple and accessible information, qualitative content, interactivity, engaging visualization and presentation of information and personalization [64].

Engagement. We are interested in finding out whether Culture Gate creates an engaging environment for its users. [46] evaluates the engagement level of a cultural heritage platform using as measures the quality of its content, its attractiveness and user-friendliness and concludes that those factors play a key role in user engagement. Thus, we propose the following hypothesis.

H1. 

the platform creates an engaging environment for its users.

3.2.2. Personal Acceptance

In this work, we relate personal acceptance with the notions of active user participation, dissemination and exploitation. A user that accepts that a digital platform fulfils its scope, that user is ready to use its content and services, to disseminate the platform to her/his peers or even to the broad public and also try to exploit platform’s material. In the case of a participatory platform, if the user accepts its scope them, he/she is ready to actively participate in the contribution of personally created digital content.

Participation. We are interested in finding out whether users comprehend the participatory nature of the platform, whether they associate active participation with content contribution and whether they feel that by using the platform they become a member of an online community. Moreover, we need a solid indication that users are eager to participate actively in the platform by contributing their own cultural heritage content. We investigate how open each user is to contribute her/his own cultural heritage content or review another user’s contribution to the platform thus fulfilling in action the goal of active participation. Furthermore, we want to examine if a user would be intrigued to further study a cultural heritage topic. Additionally, we are interested in finding out whether users would be open to enrich a cultural heritage topic by contributing content that is related to an existing user contribution thus creating a thematic collection. Finally, we make a first study of the impact of the platform’s mobile applications usage in triggering users to make contributions. How eager are users to download, install and use the mobile applications to contribute cultural content in real-time? Is this intention of usage influenced by external conditions like the lack of qualitative and costless wireless internet access? RCA suggests that anyone, from anywhere and at any time can actively participate in creating knowledge online and Constructionist Learning states that individuals become participants if they are driven by meaningful outcomes. Author in Reference [65] takes a step forward and identifies four categories concerning motivation in crowdsourcing environments: (a) direct and competitive, (b) direct and non-competitive, (c) indirect and competitive and (d) indirect and non-competitive. Culture Gate as a multipurpose platform addressing to various user types and stakeholders aims at triggering active user participation in any of the four aforementioned contexts. [16] evaluates user participation in crowdsourcing systems concluding that such platforms can activate user participation in cultural heritage projects. In addition, [15] assesses the personal impact of a crowdsourcing cultural heritage application on individuals, suggesting that audience engagement and user participation are closely connected in the crowdsourcing procedure. Thus, we propose the following hypothesis.

H2. 

The platform promotes active user participation.

Dissemination. We are interested in finding out whether users would actively help in the expansion of the platform by spreading their good work to their peers (friends, relatives) or even to others. Efficient dissemination by the users is a key point for a platform’s success. SIT dictates that a positive dissemination action could be initiated by an individual’s perceived membership in a relevant social group [18]. A vital factor for a user to disseminate a digital platform constantly is to feel that she/he is a member of a community. Thus, we propose the following hypothesis.

H3. 

Users would disseminate the platform to their peers or even to the broad public.

Exploitation. We are interested in finding out whether users would use platform content and services for commercial purposes. The platform addresses not only to individuals that care about cultural heritage but also to professionals of that domain. Museum curators, excavation archaeologists, cultural events organizers and tour operators are some of the potential stakeholders that could exploit platform content and services to access targeted audiences and increase their clientele. Authors in Reference [66] argue that every modern participatory digital platform has an economic and social impact. Individuals could generate additional sources of revenue and local or national communities could stimulate their economy through the rise in tourist numbers. Concerning the platform’s well-being, the exploitation expectancy creates loyal and active platform users. This notion derives directly from the CLT which suggests that loyal customers, in our case platform users, are more interested about the businesses,’ in our case the platform, success and sustainability. Thus, we propose the following hypothesis.

H4. 

Users would exploit platform’s content and services to produce personal revenue.

3.2.3. Social Acceptance

In this work, we relate social acceptance with the notions of connecting with origins (belonging), personal and social responsibility and intergenerational dialogue. In many cases, individuals are unaware of the history or the cultural heritage elements of the area they live, act or come from. Authors in Reference [67] argue that belongingness is such a fundamental human motivation that we feel severe consequences of not belonging. Individuals appreciate their birth/living/origination place by learning information about its cultural heritage; categorize themselves as members of a group with common principles which leads to the development of a stronger bond with society, as stated by the SIT. Following the ideas of SCT, we try to establish a meaningful relationship between the usage of a participatory digital platform for cultural heritage and a user’s responsibility. SCT states that a person adopts a behaviour simply by following the paradigm of other people. Each behaviour witnessed can change a person’s way of thinking. Responsibility towards the online community is strongly connected with the users’ feeling of being an important member of that community. A responsible user is more likely to search in depth a cultural topic before making a contribution in order to contribute sound and qualitative content. As human lifespan increases, more generations live in the same time and space. With the generational gap as a given, societies are trying to find new ways in order to retain their internal cohesion. Since cultural heritage is a factor that, in most of the cases, unites people, a cultural heritage platform should be able to promote intergenerational dialogue effectively. Following the ideas of SCT, we try to establish a meaningful relationship between the usage of a participatory digital platform for cultural heritage and intergenerational dialogue in a society. According to the SCT, individuals can get in touch with elements of cultural heritage in their living areas and appreciate more the previous generations that preserved and highlighted them. Authors in Reference [68] state that “By providing an arena for intergenerational sharing, the format encouraged cross-age connections and in doing so, supported, in a broad sense, the transmission of cultural knowledge.”

Connecting with Origins. We are interested in finding out whether the platform usage leads an individual to reconnect with her/his roots and strengthens that person’s feeling of belonging to a society. [49] evaluates a cultural heritage platform in order to explore how online visitors connect with virtual heritage, concluding that such a bond can be reinforced by cultural heritage digital platforms. Thus, we propose the following hypothesis.

H5. 

The platform supports forging a strong bond between its users and their place of birth/living/origination.

Responsibility. We are interested in finding out whether users have the intention to be responsible towards the platform’s online community when making contributions and the society in matters of cultural heritage preservation. Does the platform lead its users to be more aware about the cultural heritage of their birth/living/origination places through the process of knowing more about the cultural heritage elements of that place? According to the SCT, an individual’s empathy towards the preservation of cultural heritage can motivate her/him to be more responsible concerning cultural heritage in her/his area. [16] evaluates the level of responsibility towards society in a cultural heritage platform concluding that such platforms can raise awareness about the status of cultural heritage landscape. Thus, we propose the following hypothesis.

H6. 

The platform cultivates social responsibility to its users.

Intergenerational Dialogue. We are interested in detecting a first trace concerning the platform’s potential to promote intergenerational dialogue, an important aspect of modern societies. [47] and [48] evaluate intergenerational dialogue in a cultural heritage platform suggesting that such a platform supports intergenerational dialogue. Thus, we propose the following hypothesis.

H7. 

The platform promotes intergenerational dialogue.

3.3. Evaluation Methodology

3.3.1. Survey Instruments

In order to study the validity of the aforementioned hypotheses, we use a range of qualitative and quantitative evaluation tools: questionnaires, personal interviews and usage statistics. Questionnaires were used in all research hypotheses while personal interviews provided information for the first (engagement) and fourth hypothesis (exploitation) and usage statistics concentrated in research hypotheses 1, 2 and 5 (engagement, participation, belonging).

3.3.2. Sample

This study used a non-random sampling technique (i.e., convenience sampling) to collect data. The experiment team formulated a group of individuals that specialize in the fields of cultural heritage management and cultural informatics. Chosen individuals possess two desirable characteristics for the assessment: an obvious interest in using a platform that manages cultural heritage content and an adequate level of understanding cultural technology (individuals stated of having averagely 8 years of computer experience and that they all own a personal computer). The sample involved two categories of individuals that are students of the Department of Cultural Heritage Management and New Technologies of the University of Patras in Greece: 56 undergraduate students (24 men and 32 women) between the ages of 18 and 25 and 12 post-graduate students (5 men and 7 women) between the ages of 25 and 50 that follow the M.Sc. program “Networked Cities and Representations.” All individuals participated voluntarily in the experiment after responding to a public notification issued by the experiment team. Each person’s participation was not in any case linked with a specific course in his or her studies to avoid biased results. In order to guarantee complete anonymity for the evaluation procedure, questionnaire and personal interview answer sheets were anonymous (the usage statistics did not record the IP of the users).

3.3.3. Evaluation Procedure

The evaluation procedure was divided in three stages (Figure 2). Experimental stage 1 took place in a controlled environment (computer laboratory), supervised by the members of the platform’s design and development team. At the beginning of stage 1, users were given an oral presentation explaining platform scope, services and content. Afterwards, users accessed the platform from their computers viewing a video tutorial that explained in details the platform use. Then the users were guided to perform a series of assignments in order to get familiar with the basic system services. Each individual was given an adequate amount of time to complete all scenarios and then supervisors distributed an evaluation questionnaire. The questionnaire was separated in various parts and covered the seven research hypotheses. Each hypothesis was approached by a series of statements. Users were asked to state their agreement or disagreement with each statement. Moreover, the experiment supervisors interviewed each individual personally to extract information concerning their opinion about the platform services. We performed structured interviews with specific questions and predetermined responses but allowed the interviewees to explain thoroughly their statements in order to extract a better understanding about their opinion towards the platform. Interviews were conducted face-to-face with the interviewee in the computer laboratory. Each interview lasted 10 minutes and the most important points of each interview were noted in hard copies. Users were asked to state their favourite service, which service there is a greater chance to use in their professional life and which additional services would be an improvement for the platform. During the first stage of the experiment, supervisors were recording the corresponding usage statistics with the use of the GA tool.

Assignment 1. Content search (Thematic lists)—Search the platform to find content related to the cultural discipline “Theatre.” Click on the “Theatre” menu item located on the main menu found on the platform’s homepage. Search for information concerning ancient theatres. Choose a random ancient theatre and visit the dedicated webpage by clicking on the corresponding image.

Assignment 2. Content search (Cultural map)—Search the platform to find content related to the famous writer of theatre plays Bertolt Brecht. Click on the “Cultural map” menu item located on the main menu found on the platform’s homepage. Submit the search term “Brecht” on the form located on the cultural map. Choose a random cultural point related to Brecht and visit the dedicated webpage by clicking on the corresponding image.

Assignment 3. Register, Login/Logout—Create an account and proceed to login and logout. Click on the “Join” menu item located on the main menu found on the platform’s homepage to register. Submit personal information and click the “Register” button. In order to login press the “Login” link. In order to logout click on the “Logout” link.

Assignment 4. Contribution—Submit cultural content of your choice. Login to the platform and click on the “Add Listing” menu item located on the main menu found on the platform’s homepage. Submit cultural information of your choice.

Assignment 5. Cultural events calendar—Contribute an event. Login to the platform and click on the “Cultural Events Calendar” menu item located on the main menu found on the platform’s homepage. Submit information related to a cultural event in your area and click the “Submit” button. In order to view your event access the “Cultural events calendar” page.

Assignment 6. Forum—Join a thread on the community forum and post a comment. Login to the platform and click on the “Discussion Forum” menu item located on the main menu found on the platform’s homepage. Choose a random thread and post a comment.

Stage 2 was conducted in the form of homework for the participants. Experiment supervisors asked participants to use platform services and content at will. The users had 5 days to perform the task. During this time period supervisors were recording the corresponding usage statistics. Second stage’s goal was the quantitative measurement of user experience with the platform in uncontrolled conditions (engagement, participation, connection with origins - belonging).

Stage 3 was a repetition of stage 1 since users were asked to perform the same assignments on the platform, while supervisors recorded the task completion times. The goal was to find out how easy it is for a user to become familiar with the platform’s use. Furthermore, we wanted to discover which services presented a growing interest among users and which were not very important for them.

4. Results

In this section, we present all the experimental results that were extracted from users’ answers and actions throughout the three stages of the evaluation procedure.

4.1. Experimental Stage 1

Questionnaire results. Users received a dichotomous questionnaire, which asked them to agree or disagree with specific statements related to the platform’s usage. The questionnaire was organized in 7 subsections associated to the 7 hypotheses: participation, dissemination, exploitation, connecting with origins (belonging), engagement, responsibility and intergenerational dialogue. Questionnaire’s reliability was confirmed by estimating the Cronbach’s Alpha measure to 0.8104. This measure indicates that the questionnaire results present a high level of reliability.

Participation. The first set of statements investigates how the platform scores in the field of active user participation. Questionnaire analysis results for participation are shown in

Table 1. Participation Questionnaire Results.

Statement	User Agreement
S1. The platform promotes the idea of participation among its users.	97.06%
S2. The platform promotes a sense of belonging to a community among its users.	98.53%
S3. The platform allows users to upload their own cultural heritage content.	92.42%
S4. The user would contribute cultural heritage content to the platform.	88.24%
S5. The user would contribute a review to another user’s contribution.	66.67%
S6. The user would contribute information about a cultural event to the platform.	87.88%
S7. The platform intrigues users to further study a cultural heritage topic/issue/ event/object.	67.16%
S8. Specific information provided by the platform could trigger users to contribute related content and form a thematic collection.	63.08%
S9. The user would download platform’s mobile applications in order to contribute content in real-time.	97.01%
S10. The user would contribute cultural heritage content with the use of CGC while participating in a cultural event.	59.70%
S11. The intended use of platform’s mobile applications is influenced by the lack of free wireless internet access in their living area.	65.67%
S12. The user would use the platform’s mobile applications to search or contribute cultural heritage content if she/he could connect to the platform reliable, fast and free of charge regardless of the time and space of the connection.	94.03%

.

Dissemination. The second set of questions investigates how willing the users are to disseminate the platform to their peers or to the broad public. Questionnaire analysis results for dissemination are shown in

Table 2. Dissemination Questionnaire Results.

Statement	User Agreement
S13. The user would motivate her/his peers or others to visit and use the platform.	94.12%
S14. The user would motivate her/his peers or others to contribute content to the platform.	65.67%

.

Exploitation. The third set of questions investigates whether users are open to exploit platform services and content. Questionnaire analysis results for exploitation are shown in

Table 3. Exploitation Questionnaire Results.

Statement	User Agreement
S15. The user would use the platform to promote/advertise her/his artwork, technological application or cultural event.	73.53%
S16. The user would use the platform in her/his work.	89.39%

.

Connecting with Origins. The fourth set of questions investigates whether the platform use reinforces the sense of belonging among its users, to their place of living/birth/origination, establishing a stronger bond. Questionnaire analysis results for reconnecting with individual origins are shown in

Table 4. Connecting with Origins Questionnaire Results.

Statement	User Agreement
S17. The user would be interested in searching content (cultural point, cultural event) related to her/his birthplace, living area or origination.	92.54%
S18. The platform helps the user to come in touch with her/his birthplace, living area or origination.	91.04%
S19. The user is interested in contributing cultural content related to her/his birthplace, living area or origination.	80.60%
S20. The user’s opinion about her/his birthplace, living area or origination has been improved after finding cultural heritage information related to those places.	67.69%
S21. The user learned interesting information about her/his birthplace, living area or origination.	82.09%
S22. Platform use would urge the user to visit specific locations within her/his birthplace, living area or origination.	80.60%

.

Engagement. The fifth set of questions studies whether the platform content and basic services are important, attractive, useful, easy-to-use, intriguing and thus engaging for the users. Questionnaire analysis results for user engagement are shown in

Table 5. Engagement Questionnaire Results.

Statement	User Agreement
S23. The user likes the platform.	100%
S24. The platform’s use created a specific emotion to the user.	58.21%
S25. The user is indifferent to the platform use.	5.97%
S26. The user finds platform content intriguing.	100%
S27. The user finds platform services intriguing.	98.50%
S28. The user finds the cultural map service intriguing.	95.59%
S29. The user finds the cultural points list service intriguing.	94.03%
S30. The user finds the calendar of cultural events service intriguing.	92.53%
S31. The user finds the discussion forum service intriguing.	86.57%
S32. The user prefers the map service better than the list service.	74.24%
S33. The user finds the FAQ service helpful in learning how to use the platform.	76.12%
S34. The user finds the video tutorial service helpful in learning how to use the platform.	85.07%
S35. The user believes that the registration service is easy-to-use.	100%
S36. The user believes that the login/logout service is easy-to-use.	94.03%
S37. The user would inform platform administrators if she/he acknowledged a technical issue.	46.27%

.

Responsibility. The sixth set of questions investigates user intention to be responsible towards the platform’s online community and the society in matters of cultural heritage preservation. Questionnaire analysis results for user responsibility are shown in

Table 6. Responsibility Questionnaire Results.

Statement	User Agreement
S38. The user would dedicate a significant amount of time to prepare and upload scientifically sound content.	72.31%
S39. The user would inform platform administrators for scientifically inaccurate content.	67.16%
S40. The platform use would sensitize the user to engage in the rescue of a cultural heritage element (object, building etc.) at the place of their birth/living/origination.	86.36%

.

Intergenerational Dialogue. The seventh set of questions investigates whether the platform promotes intergenerational dialogue, an aspect of human life that affects the cohesion of modern societies. Questionnaire analysis results for intergenerational dialogue are shown in

Table 7. Intergenerational Dialogue Questionnaire Results.

Statement	User Agreement
S41. The user would urge her/his parents and grandparents to visit and use the platform.	74.63%
S42. The user would help her/his parents and grandparents to visit and use the platform.	86.36%
S43. The user would ask her/his parents and grandparents to provide information about cultural heritage issues linked to the place of their birth/living/origination and the user would contribute that content on the platform.	79.10%

.

Personal Interviews. During the first stage of the experiment, users gave personal interviews stating their opinion about platform services. Our purpose was to investigate if users would consider incorporating a cultural heritage digital platform to their professional life and thus exploiting its services to produce revenue. The results from the personal interviews concerning the exploitation potential of Culture Gate produced 64 responses from 52 individuals. Users were allowed to provide more than one answer to each interview question. The question addressed to the participants referred to which specific platform service there is a greater probability to use in their professional life, now or in the future. Most of the individuals gave one answer while some gave two or three different answers concerning the specific services that they would be willing to use during a professional task or project and 16 participants could not envision a specific service that would be exploitable by them. Users considered more likely to use the cultural map service (20 answers) and the platform’s cultural heritage content (32 answers) in their work. 8 users responded that they could use the cultural events service to advertise or promote a future event of their own. Thematic lists service (2 answers), discussion forum service (1 answer) and Artists Corner service (1 answer) were also envisioned as potential revenue generators for a number of users. The second interview question tries to reveal users’ favourite services. The results from the personal interviews concerning the most preferred and interesting services of Culture Gate produced 70 responses from 52 individuals. Users were allowed to provide more than one answer to each interview question. The question addressed to the participants referred to which specific platform service they consider more interesting and why. Most of the individuals gave one answer while some gave two, three or four different answers and 16 participants could not highlight a specific service as the most interesting to them. As expected, the most popular services were the interactive cultural map service and the cultural heritage content with 24 answers. 10 users favoured the cultural events service the most. 4 users considered as the most important service the content contributions service, while 3 of them enjoyed the thematic lists service the most. Discussion forum service (2 answers), artists corner service (1 answer), personalized content viewing service (1 answer) and educational videos content (1 answer) appeared also in user answers as their favourite services. Finally, users were asked to state their opinion about what could improve platform content quality, make the platform friendlier and more attractive to the public. The most popular answer was the lack of a platform version to the local language (Greek). Other answers had to do with graphics design issues in the front end of the platform like a different colours set and better audio-visual content.

Usage Statistics. Usage statistics were collected with the help of the GA tool. Results appear in

Table 8. Access Log Data Comparison Analysis Throughout Experiment Stages.

Service	Measurement	1st Stage	2nd Stage	3rd Stage
Overall	M1. Number of sessions.	68	188	68
Overall	M2. Bounce rate.	6.35%	14.75%	8.75%
Overall	M3. Pages per Session.	26.22	19.22	18.3
Overall	M4. Session average duration.	46.15 min	29.38 min	18.24 min
Contributions	M5. Number of successful contributions.	9	21	17
Contributions	M6. Successful contribution average duration.	4.17 min	7.27 min	1.43 min
Content Search	M7. Number of content searches.	99	546	366
Content Search	M8. Content searches average duration.	0.59 min	0.43 min	0.44 min
Content Search	M9. Number of local searches.	38	261	285
Content Search	M10. Local searches average duration.	0.41 min	0.49 min	0.42 min
Thematic Lists	M11. Number of thematic lists accesses.	280	244	121
Thematic Lists	M12. Thematic lists accesses average duration.	0.50 min	0.46 min	0.34 min
Cultural Map	M13. Number of cultural map accesses.	161	399	374
Cultural Map	M14. Cultural map accesses average duration.	2.09 min	1.33 min	0.56 min
View Content	M15. Number of cultural points visited.	120	472	114
View Content	M16. Avg time spent on cultural point.	1.38 min	2.33 min	1.28 min
Discussion Forum	M17. Number of discussion forum accesses.	38	213	80
Discussion Forum	M18. Forum accesses average duration.	0.16 min	3.51 min	1.31 min
Events Calendar	M19. Avg time spent on cultural events calendar.	0	1.48 min	1.07 min
Events Calendar	M20. Number of successful event contributions	0	3	1
Events Calendar	M21. Number of cultural events accesses	0	139	32
FAQ	M22. Number of FAQ service accesses.	38	51	14
FAQ	M23. Average time spent on FAQ service.	2.14 min	2.01 min	2.10 min
Registration	M24. Avg time spent on registration procedure.	2.37 min	1.12 min	0.44 min
Login/Logout	M25. Avg time spent on login procedure.	0.46 min	1.04 min	0.52 min
Platform	M26. Number of accesses on platform information pages.	235	212	36
Platform	M27. Avg time spent on platform information pages.	1.32 min	1.11 min	0.44 min

. Usage measurements are classified in categories based on the platform service they associate to: contributions, content search, content viewing (Thematic Lists, Cultural Map, Cultural Points), discussion forum, events calendar, frequently asked questions, registration and login/logout.

4.2. Experimental Stages 2 and 3

The second stage of the experiment comprised of a homework for the users and the recording of platform usage statistics for a period of five days. Comparison results from the usage statistics analysis in all experimental stages appear on Table 8. The first set of measurements was associated to the platform usage at each experimental stage. The number of sessions measurement was the same in both stages (1 and 3) since the experimental conditions and the requested tasks where similar. Bounce rate (the percentage of sessions that left the platform without performing any task) was similar in stages 1 and 3 due to the similar experimental conditions, while stage 2 recorded a slightly higher bounce rate explained by the wider time period of the experiment (M2). In stages 2 and 3, users visited almost the same number of pages per session, while in stage 1 users visited about 7 pages more (M3). The higher number of pages per session in stage 1 indicates that users, who accessed the platform for the first time, wanted to view as much content as possible or spend more time to find the content or service they need. On the other hand, during the second and third stage of the experience users became more familiar with the platform content and services so they visited only the pages they needed or liked. Finally, the platform was used by users for almost half an hour (29.38 minutes) during the second stage of the experiment, an excellent time period which indicates that the platform offers engaging content and services for its users (M4). Session average duration presents great variations for each stage due to the conditions of the experiment. In stage 1, users were trying to know the platform so they spent a lot of time to search its content in order to perform the requested tasks. Users in stage 3 performed the requested tasks quickly and easily since they were familiar with its content and services. Stage 2 was not subjected to time limits so users were free to use the platform at will.

5. Discussion

5.1. Engagement

We investigated the hypothesis that the platform offers an interesting and engaging environment for its users (H1). Questionnaire results reveal that all users liked the platform (S23). This is an index that the basic idea of the platform is engaging for the users and it is up to the platform services to prolong the time period that a user will spend on the platform. More than half of the users (58%) answered that when using the platform they experienced specific emotions (S24). Only 6% of the users felt indifferent towards the platform (S25). The next set of statements tries to reveal user attitude towards platform content and basic services, which constitute the basic fabric that makes a digital platform engaging. Given their background, (cultural heritage students) all users naturally responded that they are interested in the platform content (S26). Moreover, platform basic services (cultural map, cultural points list, calendar of cultural events, discussion forum) were very interesting for almost all users (S27–S31) with the discussion forum service scoring lesser results. However, results from the usage statistics analysis revealed that the discussion forum was one of the most visited areas of the platform during the second stage of the experiment. This trend indicates that users started to realize forum’s importance over time since the discussion forum is the prime point where a user can share thoughts with the other community members about topics of common interest. Furthermore, we asked users to state their preference between two services that serve the same purpose in order to find out which of the two services is considered the most engaging: the interactive cultural map service and the traditional cultural points list service. The cultural map service displays all cultural points as pins of distinct colour and marker icon, based on the discipline they belong to, on a geographical cultural map. The cultural points list service displays cultural points in lists per discipline. More than 7 out of 10 users (72%) confirmed that they preferred the interactive map service better (S32). This result is confirmed by the examination of user answers in private interviews. Users were asked to state their favourite service of the platform. More than 34% of the answers, about user favourite service, related to the cultural map service while 4.29% of the answers referred to the thematic lists service. Additionally, the platform offers two basic services to help users learn how to use the platform: the Frequently Asked Questions (FAQ) service and the video tutorial service. As expected, both services were considered rather helpful but the users favoured the video tutorial service the most with 85% of acceptance (S34) in comparison to the 76% score of the FAQ service (S33). This difference is another indication that the interactive services, like the video tutorial are more engaging for the users. Almost all users responded that the registration and login services are considered very easy-to-use (S35-S36). Statement 37 revealed that users are not very open to provide feedback to platform administrators if they spot a technical problem yet (46%). Providing feedback for technical problems is a classic index that indicates how aware a user is about the success of a digital platform. Users rarely spend their time to improve an application if they do not care about it. Moreover, the platform offers only one way to allow users send technical feedback (direct email). Nowadays users prefer to communicate with an institution or a company through social media.

We extract a conclusion for hypothesis H1 by comparing usage statistics among stages 1, 2 and 3. The comparison between stages 1 and 3 indicates that users were more familiar with the contributions process and performed the same task at one third of the time for the double amount of successful contributions (M6). Measurements 7 to 10 examine the content search service. M7 reveals that users performed 3.5 times more content searches in stage 3 than in stage 1. M8 indicates that users find the content they need, during a search, 15 seconds faster on average in third stage than in first stage. Both measurements indicate that users become familiar with the platform over time and perform the same tasks faster. Measurements 11 and 13 reveal a decreasing interest in using the thematic lists service and an increasing interest in using the cultural map service for finding cultural content. Those measurements confirm yet again the aforementioned preference of users for the modern interactive map service in comparison to the traditional thematic lists service. Moreover, measurement 14 shows a decrease in the average duration that users spend on searching information on the cultural map, which is explained by the fact that users become more familiar with the map and find the information they need faster over time. Measurement 15 shows a balance in user accesses on specific cultural point, both in first and third stage, due to the similar experiment conditions. Furthermore, measurement 16 reveals a balance in time spent on each cultural point, in both the first and third stage, indicating that users give the same attention to the cultural content. Measurements 17 and 18 reveal that users accessed the discussion forum 2 times more and spent 6 times more time on average in the third stage than in the first stage. The cultural events calendar service is one of the most interesting services for the users (questionnaire answers). Access log analysis confirms the growing users’ interest in accessing the content pages of current and future cultural events (M19-M21). During the first stage of the experiment, users did not visit the service at all. But during stage two, users unveiled their interest in the service with 139 accesses and an average duration of almost 2 minutes spent on the service. This trend continued during the third stage of the experiment where users visited the service 32 times and even contributed one event successfully in a limited amount of time. The FAQ service, quite naturally, was visited by users more times during stage one than stage three (M22), indicating that users needed fewer help after using the platform for one week. Moreover, users that seek for help devoted a significant amount of time to read usage instructions (about 2.10 minutes) in both stages (M23). Measurement 24 indicates that users found registration procedure rather easy, since they managed to register an account on the platform, 4 times faster in the third stage than in the first experimental stage. Furthermore, users spent similar amounts of time to login to the system in each experimental stage (M25). Login and logout procedure is similar in the majority of online applications so users where familiar with the process from the first stage of the experiment. Finally, measurements 26 and 27 referred to the number and average duration for accesses of pages that hosted platform’s informative content (About us, Contributors, Authors, Artists Corner, Features etc.). Analysis results show that users visited such pages more and spent more time on average to read their content in the first stage than in the third stage because they wanted to learn more about the platform.

Personal interview answers identified the platform services that are considered the most interesting and therefore engaging for the users. The most popular answers were the interactive cultural map and the platform’s cultural content confirming related findings from the questionnaires and the usage statistics analysis. Other services that are considered rather interesting are the calendar of cultural events and the contributions service. Evaluation results concerning engagement in cultural heritage digital platforms, positively support related findings presented in Reference [15,46].

5.2. Participation

We investigated the hypothesis that users intend to participate actively in contributing content to the platform (H2). Results reveal that almost all users acknowledge the platform’s participatory nature (S1) and almost all users accept that the platform promotes a sense of belonging to a society among its users (S2). Almost all users understand that the platform allows them to upload cultural heritage content (S3). More than 87% of users are open to contribute cultural heritage content (S4) or a cultural event (S6) to the platform. 2 out of 3 users (66.67%) are eager to leave a review on another user’s contribution (S5). More than 67% of the users responded that they are intrigued to further study a cultural heritage topic/issue/event/object after finding related information to the platform (S7). More than 63% of the users responded that specific information provided by the platform could trigger them into contributing related content and form a thematic collection (S8). Almost all users answered that they would download platform’s mobile applications to contribute cultural content in real-time (S9) and about 60% of them would make a contribution while witnessing a cultural event (S10). The lack of reliable and costless infrastructure influences negatively user intention of 2 out of 3 users to use the mobile applications (S11) but almost all of the users stated their intention to use platform’s mobile applications if they could connect to the platform reliably, fast and free of charge regardless of the time and space of the connection (S12). Recorded usage statistics in stage 2, show that users spent an average of 7.27 minutes per contribution for 21 contributions, indicating that users devoted an adequate amount of time to make a notable contribution (M5, M6). Moreover, users needed significantly fewer time to make a contribution in stage 3 (1.43 min) than in stage 1 (4.17 min).

Questionnaire results suggest that the users are willing to participate actively in an online community focused around cultural heritage. Almost all users comprehended that the platform allows them to upload their own cultural heritage content and the wide majority of the users would be eager to contribute content to the platform. The platform also provides a potential starting point for users to investigate a cultural topic further. Users seem to be comfortable in enriching existing content in order to form a thematic collection, an important feature since users tend to seek information regarding a specific topic in each session thus improving user engagement. Usage statistics results in stage 2 indicate that users devoted a significant amount of time to make a sound contribution of cultural content to the platform suggesting that they wanted to offer rich scientifically correct information. Comparing the average contribution time in stages 1 and 3 (similar experimental conditions, limited amount of time to perform a contribution) we can conclude that as users become more familiar with the platform they spend fewer time to make a contribution. The main platform area that promotes the online community and the exchange of opinions on cultural topics is the discussion forum. Measurements 17 and 18 reveal that users accessed the discussion forum 2 times more and spent 6 times more time on average in the third stage than in the first stage. This increase in user interest could be an indication that the discussion forum service has the potential to support a vivid online community of users that share a passion for cultural heritage. Evaluation results concerning user participation in cultural heritage digital platforms, positively support related findings presented in Reference [15,16].

5.3. Dissemination

We investigated the hypothesis that users have the intention to disseminate the platform to their peers or even to the broad public (H3). Results reveal that almost all users would motivate their peers or others to visit the platform, view its content and use its services (S13). On the other hand, about 2 out of 3 (65.67%), answered that they would urge their peers or others to contribute content to the platform (S14). This measurement suggests that users comprehend the content viewing process as distinct from the content contributing process.

5.4. Exploitation

We investigated the hypothesis that users have the intention to exploit platform services and content in order to generate revenue (H4). Results reveal that the majority of users (above 73%) would be willing to use the platform in order to advertise their creations (artwork, cultural digital application) or a cultural event that they are organizing to platform’s audience (S15). Furthermore, the great majority of users (above 89%) answered that they would be open to use platform services and content in a project at work (S16). The results from the personal interviews, concerning the exploitation potential of Culture Gate, identified the platform services that are considered more likely to be exploited by them in their professional life. The platform’s collected digital cultural content was the most popular service with the greatest exploitation potential. Notable services with exploitation potential according to the participants were the interactive cultural map service and the calendar of cultural events. The above results suggest that the platform touches the goal of creating an exploitable environment for users to promote their work or use its content and services in their professional life, now and in the near future.

5.5. Belonging

We investigated the hypothesis that the platform usage reinforces a user’s sense of belonging to her/his place of living/birth/origination (H5). Questionnaire results reveal that almost all users would be interested in searching and discovering content related to their place of living/birth/origination (S17). Furthermore, almost all users agree that the platform helps them link to their origins (S18). The great majority of users (80%) stated that they would be interested in contributing content related to their origins (S19). More than 2 out of 3 users responded that their opinion about their place of birth/living/origination has been improved when they found cultural heritage information related to that place (S20). Most of the users (above 82%) answered that they learned new and interesting cultural information related to their place of birth/living/origination (S21) and 80% of the users stated that the use of Culture Gate would motivate them to visit specific locations within that place (S22). The study of user access logs strongly confirms user interest towards information related to their roots. During the experiment, users, without guidance by the supervisors, performed their own searches of cultural content. 54.87% of user searches were related to cultural content within or near their roots. Moreover, users spent over 71 seconds to view information related to their roots in comparison to the 63 seconds that they spent on viewing other information. This result highlights the increased interest of users in cultural content related to their roots since the experiment. Usage statistics results reveal an increasing interest in local searches (M9). In stage 3, 77% of the searches had local characteristics in contrast to 48,7% of stage 2 and 38% of stage 1 (M9). This measurement reveals that users, at least at the initial stages of their contact with the platform, care about content related to their origins. Furthermore, local searches occupied users, more time on average (0.49 min) than total searches (0.43 min) or thematic collections searches (0.38 min) (M10). Evaluation results concerning the sense of belonging supported by a cultural heritage digital platform, confirm related findings presented in Reference [49].

5.6. Responsibility

We investigated the hypothesis that the platform promotes personal and social responsibility among its users (H6). Results reveal that about 72% of the users would be willing to dedicate their time in contributing scientifically correct content to the platform (S38) and more than 67% answered that they would inform platform staff in they found scientifically inaccurate content (S39). The aforementioned data reveal that the users possess an adequate level of responsibility towards the other users since they are more likely to contribute content when they have done a full research on a cultural heritage topic and they are sure that their contribution is scientifically sound. A very high percentage of users (86%) answered that the platform could motivate them in getting involved with the preservation and rescue of cultural heritage property within the area they live or come from. The above evidence suggests that the platform instils a sense of responsibility to its users towards the society and the online community.

5.7. Intergenerational Dialogue

We investigated the hypothesis that the platform promotes intergenerational dialogue (H7). Results reveal that more than 74% of the users would present the platform to their parents and grandparents and would urge them to visit and use platform services (S41). Moreover, 86% of the users would help their relatives to search and view cultural content (S42). Furthermore, 79% of the users answered that they would use their parents’ and grandparents’ experiences, memories and knowledge to derive information about cultural heritage issues related to the area they live or come from. The above results provide a first glimpse about the positive impact a cultural heritage participatory digital platform could have on strengthening the bond between consecutive generations and promoting the peaceful continuity of modern societies, thus supporting related findings presented in Reference [47,48].

5.8. Limitations and Future Work

This study has various limitations. At first, the examined sample consisted of only one category of potential stakeholders. Chosen undergraduate and postgraduate students are future or current professionals of cultural heritage with familiarity in using IT platforms. The sample is intentionally biased in terms of skills, age and educational levels. The sample could be enriched with other categories of potential users like public authorities’ officials, tour operators, school teachers or parents, in order to extract a more general view of the platform’s personal and social impact. Moreover, the limited time period of usage statistics collection did not allow us to explore the potential interactions that could have been developed among the personal and social impact factors (participation, dissemination, exploitation, belonging, engagement, responsibility, intergenerational dialogue). Different results may be obtained if this study measured personal and social impact for a longer period of time. Future studies can reveal potential relationships among participation, dissemination, exploitation, belonging, engagement, responsibility and intergenerational dialogue. Another important limitation is the real life truth that there is a massive potential gap between people’s attitudes and actions. This fact inserts a component of uncertainty in linking the evaluation results with our conclusions. Finally, we chose to guide the evaluation procedure through well-structured experiments and not organically, in order to capture the first impressions of cultural heritage experts towards the impact of using a participatory cultural heritage platform.

6. Conclusions

This study gives a first answer to a series of research questions concerning how usable and acceptable, in a personally and social level, is a specific participatory cultural heritage digital platform (Culture Gate) considered by its users. Through a series of experiments and utilizing popular evaluation instruments, individuals, future experts and professionals of cultural heritage, were guided to provide answers concerning specific areas that reflect on usability and acceptability: active participation, dissemination, exploitation, belonging, engagement, responsibility and intergenerational dialogue. Moreover, user answers were accompanied by the collection of the corresponding usage statistics, in order to extract a more spherical view about the evaluation results. Questionnaire answers, interviews and usage statistics suggested that the platform is considered both usable and acceptable by its users, as suggested by all evaluated factors.

In the area of active user participation, evaluation results suggested that the platform provides the necessary background (tools and services) to its users to participate actively in an online community focused around cultural heritage. Although, there is a controversy among users around the idea of using platform’s mobile applications to contribute content in real-time, such an intention is clearly influenced by the lack or presence of qualitative and costless mobile internet access infrastructure in their living area.

The dissemination potential of Culture Gate was backed by the evaluation results since users stated their intention to spread their good word to peers and to the broad public about the merits of the platform, urging them to visit Culture Gate or download the platform mobile applications. On the other hand, users believe that it would be easier to convince another user to access the platform to view content than convince her/him to upload content. Furthermore, such an attitude towards content contribution could mean a lack of trust in the content submission process that requires further investigation.

In the area of exploitation, evaluation results suggested that users would advertise their creations or cultural events to platform’s audience. As Culture Gate is a platform dedicated solely to cultural heritage, its targeted audience towards cultural heritage presents a great advantage for promotion actions. For that reason, there is a greater likelihood that platform visitors would be interested in such cultural heritage creations or events. Moreover, users would be open to exploit the platform services and content in the course of a professional project or task.

Based on the evaluation results, Culture Gate seems to have a positive impact on a user’s sense of belonging to her/his place of living/birth/origination. Evaluation results support a characteristic of participatory platforms that manage and disseminate cultural heritage digital content: such platforms motivate users to come in touch with their origins, helping individuals build a strong connection with their place of birth/living/origination and improving users’ opinion about those places.

Culture Gate aspires to be an engaging and interesting environment for its users. Questionnaire results suggest that the platform provides an engaging environment based on the interesting content and the modern, interactive and informative services it offers. Usage statistics measurements reveal that users managed to become familiar with the platform after using it for only a week (each user accessed the platform 3-4 times in 7 days) suggesting that the platform is easy to use and user-friendly. Moreover, the platform seems to offer engaging content and services since users spend a significant amount of time on the platform to access or contribute cultural heritage content.

Evaluation results suggest that the platform has a positive influence on individuals concerning the cultivation of personal and social responsibility towards the society and the online community. A significant percentage of the users would not remain abstinent towards inaccurate content. This attitude could be strengthened as users become more aware about the online community. Furthermore, users answered that the platform could urge them to help in preserving or rescuing cultural heritage property within their living areas.

The evaluation procedure suggested that Culture Gate could have a positive influence on supporting intergenerational dialogue, managing to trace a first footprint of the impact that digital cultural heritage platforms could have on an otherwise complex and difficult issue to approach. Further studies are needed in order to establish a strong connection between the usage of participatory cultural heritage platforms and intergenerational dialogue.

Personal interviews results also indicate that the platform offers various interesting services covering a wide spectrum of user needs and aspirations from modern ones that seek interactive and personalized services to more traditional users that want to view cultural content in thematic lists and engage in discussions about their favourite cultural heritage topics.

This paper is a first step towards the formulation of a model that explains the adoption and success of a participatory digital platform for cultural heritage. As a first step we used as a test-bed for the application of the model we propose, a specific participatory platform for cultural heritage. In the future we plan to extend our study investigating the application of our research model to other participatory digital platforms in the field of cultural heritage. Various stakeholders that wish to implement a similar platform could exploit Culture Gate’s evaluation results on personal and social acceptance and usability. Finally, evaluation results suggest that sustaining an engaging, user-friendly and attractive environment for users, plays a key part in the popularity of cultural heritage participatory digital platforms.