Next Article in Journal
Interactive Landscape Design and Flood Visualisation in Augmented Reality
Next Article in Special Issue
Observing Collaboration in Small-Group Interaction
Previous Article in Journal
Guest Editors’ Introduction: Multimodal Technologies and Interaction in the Era of Automated Driving
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Websites with Multimedia Content: A Heuristic Evaluation of the Medical/Anatomical Museums

by
Matina Kiourexidou
1,*,
Nikos Antonopoulos
2,
Eugenia Kiourexidou
3,
Maria Piagkou
4,
Rigas Kotsakis
5 and
Konstantinos Natsis
1
1
School of Medicine, Faculty of Health Sciences, Aristotle University, Thessaloniki 54124, Greece
2
Digital Media and Communication, Ionian University, Argostoli 28100, Greece
3
Faculty of History and Archaeology, Aristotle University, Thessaloniki 54124, Greece
4
Medical School, Faculty of Health and Sciences, National and Kapodistrian University of Athens, Athens 11527, Greece
5
School of Journalism and Mass Communications, Aristotle University, Thessaloniki 54124, Greece
*
Author to whom correspondence should be addressed.
Multimodal Technol. Interact. 2019, 3(2), 42; https://doi.org/10.3390/mti3020042
Submission received: 11 May 2019 / Revised: 4 June 2019 / Accepted: 10 June 2019 / Published: 12 June 2019
(This article belongs to the Special Issue Multimodal Conversational Interaction and Interfaces)

Abstract

:
The internet and web technologies have radically changed the way that users interact with museum exhibits. The websites and their related services play an important role in accessibility and interaction with the multimedia content of museums. The aim of the current research is to present a heuristic evaluation of forty-seven medical and anatomy museum websites from usability experts, for the determination of the principal/key characteristics and issues towards the effective design of a museum website. For homogeneity and comparison purposes, the websites of museums with no support of English language were not included in the evaluation process. In the present paper, the methodology was structured with the assessment of the technologies and services of anatomy museum websites in mind. The results of the current statistical examination are subsequently analyzed and discussed.

1. Introduction

The field of art/culture and specifically museums has undoubtedly been influenced by the rapid development of new technologies and interaction mechanisms [1], particularly via web services and social media. In this regard, websites play an important role in promoting comprehensively the ideas, notions and education elements of cultural heritage [2]. The traditional model of a museum was based on a physical collection of objects, publicly funded, exhibited and therefore accessible to people who express cultural interests [3]. While embracing the constantly evolving creation and distribution potential of digital environments and multimedia assets, museums are inevitably focused on the digitization of their collections not only for cultural heritage preservation reasons, but also to serve effective content management and description processes that could facilitate suitable storage, indexing and retrieval mechanisms, and for the attractive and efficient presentation of content [4]. Furthermore, it has to be mentioned that one of the major advantages of careful and innovative designing of digital media is the enhancement of user interaction, which leads to an improved quality of experience [5]. In recent years, many museums have been engaged in creating an online footprint via websites, in order to exploit the aforementioned advantages [6]. In addition, anatomy museums, despite their specialized aim and related exhibits, have created websites with a plethora of digital images, including preserved corpses of humans, as well as medical tools. These museums aim to enrich knowledge in the field of anatomy through the dissection of cadavers, while their collection of medical artefacts are important recordings for historical and state of the art evidence [7].
Creating a website is by definition an easy way to access and attract remote users worldwide, and specifically, recent studies have shown a huge increase of online visitors to museums’ web services [8]. However, as the outmost target of websites is to maintain and subsequently increase the traffic on them, an evaluation process seems rather necessary, especially for understanding usefulness and functionality factors, which reveal design problems and erroneous structural elements [5]. The evaluation procedure should consider basic prerequisites, with the most important being that visitors to museum websites expect them to be comprehensible, easily navigable, visually pleasing and enjoyable [9,10,11]. In this context, many researches are focused in the direction of creating assessment metrics specifically adapted to museum websites [12]. Fantoni, Stein and Bowman [8] examined people’s motivation and preferences for visiting a website and furthermore, they investigated a correlation between users’ incentives and the Internet involvement. Another scientific research [13] tried to capture/monitor the profiling and special characteristics of the visitors of museum websites in order to determine their needs and expectations during navigation. A user survey, conducted for the National Museum of Liverpool website, collected scalable data right after the users’ visit, in an effort to comprehend and analyze the key factors of motivations, tasks, engagement and domain knowledge [14]. Monistrol et. al. [8] presented the results of their comparative study performed on 68 Catalonian museums’ websites, while Theocharidis et al. [15] after analyzing 53 museums’ websites developed the following six element classification scheme: contact-communication, visit the museum, the museum, education, website features and use of social media. Moreover, another research work formulated an evaluation methodology/strategy to assess technologies and services of Greek e-tourism and e-cultural websites [16], while the user experience was thoroughly assessed for 180 museum visitors of all ages in [17]. Kabassi’s review project [5] presented and contributed to the most common and effective criteria that can be utilized towards the multifactor assessment of museum websites, which can either be common or expert user evaluations.
The aim of the current research is to present and analyze the results of a heuristic comparative evaluation of forty-seven medical and anatomy museum websites. The evaluation process was based on the recorded opinions of usability experts, answering to specific assessment questions/metrics, while it has to be mentioned that museum websites with no English language support were excluded to avoid heterogeneity problems.
It is worth mentioning that similar evaluation studies for anatomy museums were not pinpointed while examining the contemporary literature, consequently, the current work tried to contribute to enhancing state-of-the-art work regarding the assessment and usability of cultural websites. Moreover, this attempt could be regarded as the preliminary research step, oriented towards the formulation and subsequent structuring of a generic assessment model/implementation for examining the multi-factor usability properties of the anatomy museums or even cultural heritage websites in general.

2. Materials and Methods

The conducted research was launched on February 2019, while the first step was the formulation of the museum websites collection, which derived from: a) the European Association of Museums of the History of Medical Sciences [18], b) TripAdvisor and c) Wikipedia. Thereafter, a filtering process had to be imposed in order to identify and create a subset of museums related to human anatomy interest. In this way, the initial number of 82 museums was restricted to 47 by eliminating those websites that were out of function/service and those that did not support English language, as mentioned before. Thus, the final dataset of museum websites enumerated 47 instances of international and national museums, which were subsequently implicated in the comparative experiments.
At the beginning of the research, three general-scope questions were considered as evaluation metrics, concerning their own domain name, if they have virtual museums services and three-dimensional modeling support. However, in the scientific field of human-computer interaction, the factor of website usability was determined via specialized assessment/inspection methods [19], while the methodology of heuristic evaluation is a mainstream, easy-to-follow and inexpensive procedure [20] that guides evaluators to search shortcomings and problems [21] in technology-oriented assessment tasks. Furthermore, the usability tests were able to reveal and quantify the effectiveness, accuracy and completeness of the user experience of the design choices of a website/application, while they also examined the efficiency of the offered resources/content, towards meeting the users’ needs and expectations, thus, providing a sense of the satisfaction, comfort and acceptability of use/interaction.
The main task of the usability evaluation operation refers to the investigation of the website appropriateness/suitability for its targeted/intended users [22]. In the current work the anatomy museums websites were assessed by a skilled team consisting of five field experts. Specifically, three of the evaluators had an academic background in computer science graduate and master’s studies, while their expertise and research interests are mainly focused in graphic design and human-computer mechanisms. The remaining two members of the expert group, who also have a background in computer science, have participated in the past in multivariate usability evaluation procedures of web services and applications, thus they hold extensive knowledge and experience in heuristic assessment tasks. The followed evaluation process was an inexpensive and straightforward operation, requiring the engagement of only one user-expert at a time in the role of evaluator. It has to be noted that throughout the assessment duration, the evaluators worked on their own without any intercommunication, ensuring in this way the independent nature of their answers in all registered metrics. The heuristic evaluation process was deployed in two phases: a) during the first phase, the experts navigated in the provided collection of websites in order to judge the flow, sense and dynamics of the interaction; b) in the second phase, the evaluators visited again the websites several times in order to assess and provide answers related to validation metrics and qualitive factors that were asked for [20,23]. Based on the expert responses, the respective descriptive statistics (percentage and average values) were computed, for analyzing the research findings.
As regarding the evaluation metrics, the following factors were selected empirically, based on usability heuristics introduced by Nielsen [24,25]:
  • H1: Visibility of system status—search availability. The search option within the website is a fundamental property, usually expected, for users’ convenience, saving visitors’ time by helping them to quickly access the information that they are looking for [26].
  • H2: Match between system and the real world—magnifying glass in the search. During effective design of differentiated sites and applications a reliable and easy-to-find search field is incorporated, serving also interoperability purposes by associating it with the image of a magnifying glass [27].
  • H3: User control and freedom—the website logo link to the homepage. During interaction and navigation in online services, users are accustomed to the standardized interconnection of the website logo with the home page, reducing this way confusion in the site map [28].
  • H4.1: Consistency and standards w3c—all text gets larger according to user preferences. This key feature offers adaptivity potentials to users’ needs and expectations, since some people may prefer to enlarge websites’ content, and especially textual information, for effective material projection and easier reading [29].
  • H4.2: Consistency and standards w3c—logo for Web Content Accessibility Guidelines (WCAG) for the levels A, AA, AAA. W3C provides conformance logos for Web Content Accessibility Guidelines (WCAG) 2.0, therefore, content providers could exploit these logos in their site design to indicate a claim of conformance, conveying in this way a sense of reliability [30].
  • H5: Error prevention—broken links (internal or external). These types of unpleasant and unprofessional errors/indicators deteriorate the website appearance and make it look like it stands in a state of maintenance or that it is sometimes out of service. Specifically, for this factor, the evaluators utilized a dedicated web tool called the Online Broken Link Checker [31], in order to determine if there are these kinds of errors in the websites [26].
  • H6: Recognition rather than recall—showing users where they are with the breadcrumb. These graphical control properties are very important navigational elements, facilitating users’ location awareness, reducing disorientation problems that could lead to unpleasant feelings during interaction [26,32,33].
  • H7: Flexibility and efficiency of use—there is a sitemap. The main benefit of the site map existence is that it informs the users about the website pages, internal and external links etc., providing an overview of the site areas in a single glance [34].
  • H8: Aesthetic and minimalist design—colors must have a sufficient contrast between text color and its background. This structural element was investigated by the experts via a specialized online tool, named Colorable [35], that checks whether there is an adequate color contrast between text and background [36]. The utilized color palette is a powerful design factor that could attract or push away the visitors if they feel unpleasant and inconvenienced during navigation, reading, or generally interacting with the web content.
  • H9: Help users recognize, diagnose, and recover from errors—if the well-known “404-page” error exists. This message initially states that the page is not found [37], while during the evaluation process the experts were checking if the “page 404” indication is accompanied by a feedback explanation to the user why the URL could not be found.
  • H10: Help and documentation—the website has Frequently Asked Questions (FAQ) to answer users’ questions, solve specific problems etc. For this reason, customer service on the website is very important to support the visitors’ navigational problems and fulfill their needs and expectations [23,38].

3. Results

As mentioned above, the initial set of museum websites that was formulated was constituted by eighty-seven (87) websites, but forty of them did not have an English version, concluding in the final subset of forty-seven (47) websites, that were finally implicated in the evaluation experiments, deriving from (in decreasing frequency ranking): seven from USA, six from United Kingdom, four from Netherlands and Italy, three from Germany and Greece, two from Australia, Latvia, Portugal and Switzerland, and one from Austria, Azerbaijan, Belgium, Denmark, France, Hong Kong, Lithuania, Osaka, Scotland, Spain, Sweden and Ukraine. It is worth mentioning that among the 47 websites, twenty of them had their own domain name, while nine museums had an online e-shop, either for objects or for tickets. The detailed properties of the museums concerning their names, location (country) and the respective links are presented in Table 1.
During the analysis of the descriptive statistics of the aforementioned assessment factors, for the question "Search availability" H1, all evaluators agreed that 32% of museum websites did not have “search availability”, except the second evaluator, who found a differentiated incidence of 34%. The search option has been associated with the magnifying glass image (H2) according to Nielsen Norman Group [27], and all five experts agreed on their responses to this element, independently of the presence of a magnifying glass in the website search.
For H3 “the website logo link to the homepage”, the evaluators concluded an average value of 18.7%, while increased attention was paid by museums (an incidence of 89.4%) for the design factor “Text gets larger according to user preferences” (H4.1). Concerning the existence of Logo W3C for the levels AA, AAA on museum websites, all experts agreed in total that there was no appearance of these indicators on any website (H4.2.).
During the evaluation of the error problems, the assessment group concluded that 12.8% of the 47 websites did not have a broken link (H5). Furthermore, it was determined that 50% of the websites did not have a breadcrumb (H6), while only 22.52% of them offered a sitemap element (H7), according to the evaluators. The text color was in efficient contrast with the background color (H8) in the first page of the anatomy museums websites, a fact that was validated for all websites. For the assessment factor H9 the experts concluded that 40% did not have the erroneous page 404, while it is worth mentioning that almost none of the museums (in 90% percentage) had Frequently Asked Questions embedded support (H10). The aforementioned results are synopsized in the detailed Table 2, where the percentage values depict the rate of websites with absence (nonexistence) of the corresponding property. Moreover, as Table 2 presents, the evaluators opinions were in total agreement for five out of 10 tasks, specifically for H2, H4.1, H4.2, H5 and H8.
Besides the calculation of the presented average percentages for usability metrics, a cross-correlation analysis was conducted for these properties, aiming at detecting the dependencies between the respective variables. More specifically, the Pearson coefficient was computed for each pair of usability factors, with the exception of H4.2 and H8 properties, which involved in all cases percentages of 100% of the museums for absence of w3c logo and 0% for text color contrast with the background. It has to be mentioned that Pearson coefficient withholds values from zero, for totally uncorrelated data, to one for absolute correlation. In this direction, Table 3 depicts the computed cross-correlation matrix for the usability metrics, while Figure 1 is a visual and more comprehensible representation of it.
Based on the above coefficient values, it is concluded that there was strong dependency (0.762) between H1 and H2 properties, indicating that, when a search field option existed in a website (H1), usually the magnifying glass (H2) was also utilized. Moreover, medium correlations (around 0.3–0.4) were computed for the pairs H1-H6, H1-H7, H1-H9, H2-H6, H2-H9, H3-H6 and H6-H7, revealing that, in most cases, the usability modality of breadcrumb inclusion (or not) in the website (H6) is in accordance with the existence (or not respectively) of search filed (H1), magnifying glass (H2), link between logo and homepage (H3) and sitemap (H7) usability elements. The rest of the paired correlation values were below 0.3, revealing the independency of these factors, meaning that the state of appearance of one design property did not prerequisite/relate to the other one.

4. Discussion

The current study was focused on the evaluation of anatomy museums websites based on the thoroughly described usability metrics [24,25]. An initial detailed research was conducted for the formulation of the collection of museum websites, before developing a filtered subset without language heterogeneity problems, which was implicated in the subsequent assessment procedure for the investigation and comparison of key design properties and structural elements. The heuristic evaluation of the current dataset was performed by five experts, who concluded that for the final sample of 47 anatomy museums websites from different countries, a common base of insufficiency existed, since most of the websites lacked of effective accessibility, missing logo for disabilities, they had broken links and no Frequently Asked Questions section.
ISO 9241-151 [26] provides detailed guidance rules that aim on increasing the users’ usability in terms of navigation and search operations. Among the current research findings (Table 2), it is recorded that 68% of the websites had a search service/option (H1), which, according to Sherwin [27], is usually connected to a magnifying glass icon for an easier and standardized recognition during users’ navigation. In this context, the assessment by the five evaluators revealed that 45% of the subjects following the website use the magnifying glass in the search (H2).
Many users, while they surf the web, are accustomed to the functionality whereby clicking on the main website logo will take them to the website home page. An evaluation on this design factor was conducted by Gupta [39], who recorded that 89.8% of the National Libraries websites provided an active link “to homepage”, which was also validated for the case of anatomy museums in a percentage of 81.27% (H3). Furthermore, Gupta [39] pinpointed that the “Text resizing option” did not exist at 89.8% of his sample, which is also in agreement to the findings of the current research for factor H4.1 in a rate of 89.40% of the museum websites.
As Table 2 exhibits, the W3C logo for the levels AA, AAA (H4.2) was totally absent, leading to the vital need to trigger and increase web developers’ awareness and knowledge of the importance of designing according to established accessibility standards and guidelines. Manzoor et al. [40] found that 100% of the utilized sample of university websites suffered from broken links (H5), while this percentage remained also high (87.23%) for museum websites. Carvalho et al. (2016) [41] highlighted the importance of embedding a breadcrumb at the top of the screen to track user’s actions and show their exact location, as well as the way to get there, however, in the sample of anatomy museums only 50% had this integrated feature (H6). Moreover, a sitemap (H7) functionality was present in only 22.52% of the museums’ websites, while in the Manzoor et al. [40] survey it is recorded that only 30% of university websites in Europe, less in Canada (25%) and even less in USA (20%) involved this key property. It has to be noted that Internet accessibility should be offered to everyone, without any discriminations, without excluding/forgetting people with physical disabilities. For this reason, it is welcomed that the assessment team observed that all museum websites had a sufficient contrast between their text color and background (H8) in the home page [36], promoting effective comprehensibility without visual disorientation. As Table 2 presents, the experts’ responses in H4.2 and H8 research items are in total agreement, because the respective control tests were performed with the support of dedicated online tools, as mentioned before.
According to ISO 9241-151 and Nielsen’s usability evaluation approach [37], it is considered really important to check if the error messages (the URL was not found) are accompanied by a feedback explanation to the user why it was not possible to access the requested content/information. In the H9 metric, the experts examined whether the page 404 erroneous indication was justified to users, resulting in a rate of 40% of the museum websites where no further explanation was given. Moreover, despite the crucial impact of FAQ (Frequently Asked Questions) services for informative problem-solving operations during users’ navigation and interaction [26,38,42], in 90% of the cases this vital element was missing (H10). In addition, it is worth mentioning that during the qualitive assessment of museums websites, the expert group noticed the absence of virtual and three-dimensional modeling features, that could be particularly useful for educational and explanatory purposes. However, the Museum of Contemporary Art (in the district of North Miami) has invested resources in the improvement and optimization of its website, aiming to be more accessible for people with disabilities [43]. Finally, a very important and critical conclusion deriving from the evaluation process, refers to the out-of-date appearance of the websites, since it was observed that in many cases the graphics, the content and even the structure was the same without the integration of innovative features and improved functional elements.
While proceeding into more specific results, it has to be noted that none of the museums were positively evaluated on all of the ten usability aspects. However, during the assessment operation, the experts highlighted a subset of museum websites, which outperformed in several usability factors, presenting a more efficient design layout. In this context, the Harry Brookes Allen Museum of Anatomy and Pathology, in Australia, was considered in the highest order position, with positive evaluation from all the experts in eight usability metrics: H1, H2, H3, H7, H8, H9, H10, H11. Furthermore, Pauls Stradins Museum for history of medicine, in Latvia fulfilled satisfactory aspects of seven usability factors: H1, H2, H3, H4, H7, H9, H10, while the National Museum of Health and Medicine, in the USA, was positively evaluated also in seven properties: H1, H2, H3, H7, H8, H9, H11, with slight differentiations to the previous ones. The rest of the museums covered three to six usability variables, pointing to the direction of a more problematic user experience design.
A useful remark that was extracted during the experimentation refers to the correlation between the usability elements, as Table 3 and Figure 1 exhibit. Specifically, some properties were engaged to their common existence such as H1 and H2 (search field and magnifying glass respectively), while more restricted dependencies were pinpointed for other pairs of usability metrics such as H6 (breadcrumb modality) to H1 (search field), H2 (magnifying glass), H3 (link between website logo to the homepage), H7 (sitemap support). Furthermore, the rest of the evaluation factors were uncorrelated, pointing out that they are stand-alone choices during the design/development of the website.
While synopsizing the results of the conducted operational assessment, the deficiencies of the museum websites were highlighted, towards the formulation of initiatives for optimization potentials and sometimes, even radical changes, which would promote/favor the creation of a pleasant and attractive environment, with convenient navigation and effective interaction mechanisms, fulfilling the visitors’/users’ demands and expectations.

5. Conclusions

The current work examined the usability properties of 47 anatomy museum websites based on a well-defined heuristic model that involved several differentiated usability metrics (H1–H10). The evaluation process was conducted from five field experts, who responded on specific questions related to the usability assessment factors as proposed by Nielsen [24,25]. Based on the experimental results, it can be concluded that several features can be included/optimized for improving the layout and presentation of the websites, and enhancing user experience fulfillment during navigation. In this direction, basic modalities such as search fields, breadcrumbs, sitemaps as wells as w3c logo provisions and Frequently Asked Questions sections are some of the most fundamental elements that must be incorporated in anatomy museum websites towards the guidance, effective interaction, confusion avoidance, pleasant and convenient navigation of the visitors, improving also the attractiveness of these services.
It is worth mentioning that similar researches for anatomy museums were not retrieved during the literature study, thus, the current work tried to fill the corresponding state-of-the-art gap. However, this is considered as only the first scientific step towards the proposal and development of a generic assessment framework for examining the multi-factor usability properties of anatomy museums or even cultural heritage websites in general.
In the future, the creation and integration of more specialized assessment metrics in each category are intended, in order to be studied and compared, facilitating more thorough evaluation and conclusion extraction operations. Furthermore, a crucial aspect for validating the impact and contribution of this research refers to the repeatability potentials of the evaluation process, even with the same metrics but in different time periods, aiming to track possible changes and write down the noticed improvements in the same websites or even to different/new ones. These findings will inevitably lead into the launch of further studies, aiming to discover/justify the reasons for the accessibility problems and deficiencies of the implicated websites. Finally, the outmost target of the current work was to contribute to the development of a qualitive anatomy museum website, with the elimination of the aforementioned structural problems, while attempting an effective, up-to-date and innovative design that will be subjected to user assessment.

Author Contributions

Conceptualization, M.K.; Data curation, M.K., N.A., E.K., M.P., and R.K.; Formal analysis, M.K.; Investigation, M.K., N.A., E.K., M.P., and R.K.; Methodology, M.K., N.A., E.K., and R.K.; Project administration, M.K. and K.N.; Supervision, M.K. and K.N.; Validation, M.K.; Writing—original draft, M.K., N.A., E.K., M.P., R.K., and K.N.; Writing—review & editing, M.K., N.A., E.K., M.P., R.K., and K.N.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chu, J.H.; Mazalek, A. Embodied Engagement with Narrative: A Design Framework for Presenting Cultural Heritage Artifacts. Multimodal Technol. Interact. 2019, 3, 1. [Google Scholar] [CrossRef]
  2. Kabassi, K.; Botonis, A.; Karydis, C. Evaluating the websites of the museums, conservation labs: The hidden heroes. In Proceedings of the 9th International Conference on Information, Intelligence, Systems and Applications, Zakynthos, Greece, 23–25 July 2018. [Google Scholar]
  3. Bertacchini, E.; Morando, F. The future of museums in the digital age: New models for access to and use of digital collections. Int. J. Arts Manag. 2013, 15, 60–72. [Google Scholar]
  4. Sylaiou, S.; Liarokapis, F.; Kotsakis, K.; Patias, P. Virtual museums, a survey and some issues for consideration. J. Cult. Herit. 2009, 10, 520–528. [Google Scholar]
  5. Kabassi, K. Evaluating websites of museums: State of the art. J. Cult. Herit. 2017, 24, 184–196. [Google Scholar] [CrossRef]
  6. Monistrol, R.; Rovira, C.; Codina, L. Catalonia’s Museums Websites: Analysis and Evaluation Proposal. Hipertext.Net. Available online: https://www.upf.edu/hipertextnet/en/numero-4/museos.html (accessed on 10 May 2019).
  7. Alberti, S.; Hallam, E. Medical Museums: Past, Present; The Royal College of Surgeons of England: London, UK, 2013. [Google Scholar]
  8. Fantoni, S.F.; Stein, R.; Bowman, G. Exploring the Relationship between Visitor Motivation and Engagement in Online Museum Audiences. Museums and the Web. Available online: https://www.museumsandtheweb.com/mw2012/papers/exploring_the_relationship_between_visitor_mot (accessed on 10 May 2019).
  9. Tractinsky, N.; Katz, A.S.; Ikar, D. What is beautiful is usable. Interact. Comput. 2000, 13, 127–145. [Google Scholar] [CrossRef] [Green Version]
  10. Lin, A.C.H.; Fernandez, W.D.; Gregor, S. Understanding web enjoyment experiences and informal learning: a study in a museum context. Decis. Support Syst. 2012, 53, 846–858. [Google Scholar] [CrossRef]
  11. Van Dijk, E.; Lingnau, A.; Kockelkorn, H. Measuring enjoyment of an interactive museum experience. In Proceedings of the 14th ACM International Conference on Multimodal Interaction, Santa Monica, CA, USA, 22–26 October 2012; ACM Press: New York, NY, USA, 2012. [Google Scholar]
  12. Cunnliffe, D.; Kritou, E.; Tudhope, D. Usability evaluation for museum web sites. Mus. Manag. Curatorship 2001, 19, 229–252. [Google Scholar] [CrossRef]
  13. Marty, P.F. Museum websites and museum visitors: Digital museum resources and their use. Mus. Manag. Curatorship 2008, 23, 81–99. [Google Scholar] [CrossRef]
  14. Walsh, D.; Hall, M.; Clough, P.; Foster, J. Characterising online museum users: A study of the national museums liverpool museum website. Int. J. Digit. Libr. 2018, 1–13. [Google Scholar] [CrossRef]
  15. Theocharidis, A.I.; Nerantzaki, D.M.; Vrana, V.; Paschaloudis, D. Use of the web and social media by Greek museums. Int. J. Cult. Digit. Tour. 2014, 1, 8–22. [Google Scholar]
  16. Lazarinis, F.; Kanellopoulos, D.; Lalos, P. Heuristically Evaluating Greek e-Tourism and e-Museum Websites. Electron. J. Inf. Syst. Eval. 2018, 11, 17–26. [Google Scholar]
  17. Roussou, M.; Katifori, A. Flow, staging, wayfinding, personalization: Evaluating user experience with mobile museum narratives. Multimodal Technol. Interact. 2018, 2, 32. [Google Scholar] [CrossRef]
  18. European Association of Museums of the History of Medical Sciences. Museums & Collections. Available online: http://www.eamhms.org/about-us/museums_collections/ (accessed on 10 May 2019).
  19. Yeratziotis, A.; Pottas, D.; Van Greunen, D. A usable security heuristic evaluation for the online health social networking paradigm. Int. J. Hum.-Comput. Interact. 2012, 28, 678–694. [Google Scholar] [CrossRef]
  20. Alonso-Ríos, D.; Mosqueira-Rey, D.; Moret-Bonillo, V. A systematic and generalizable approach to the heuristic evaluation of user interfaces. Int. J. Hum.-Comput. Interact. 2018, 34, 1169–1182. [Google Scholar] [CrossRef]
  21. Bader, F.; Schön, E.-M.; Thomaschewski, J. Heuristics considering UX and quality criteria for heuristics. Int. J. Interact. Multimed. Artif. Intell. 2017, 4, 48–53. [Google Scholar] [CrossRef]
  22. McClellan, M.A.; Karumur, R.P.; Vogel, R.I.; Petzel, S.V.; Cragg, J.; Chan, D.; Jacko, J.A.; Sainfort, F.; Geller, M.A. Designing an educational website to improve quality of supportive oncology care for women with ovarian cancer: An expert usability review and analysis. Int. J. Hum.-Comput. Interact. 2016, 32, 297–307. [Google Scholar] [CrossRef]
  23. Kostaras, N.; Xenos, M.N. Assessing educational web-site usability using heuristic evaluation rules. In Proceedings of the 11th Panhellenic Conference in Informatics, Patras, Greece, 18–20 May 2007. [Google Scholar]
  24. Nielsen, J. Usability Engineering; Academic Press: San Francisco, CA, USA, 1993. [Google Scholar]
  25. Nielsen, J. 10 Usability Heuristics for User Interface Design. Available online: https://www.nngroup.com/articles/ten-usability-heuristics/?lm=usability-engineering&pt=book (accessed on 10 May 2019).
  26. Bevan, N. Guidelines and standards for web usability. In Proceedings of the HCI International, Las Vegas, NV, USA, 22–27 July 2005; Lawrence Erlbaum: Mahwah, NJ, USA, 2005. [Google Scholar]
  27. Sherwin, K. The Magnifying-Glass Icon in Search Design: Pros and Cons. Available online: https://www.nngroup.com/articles/magnifying-glass-icon/ (accessed on 10 May 2019).
  28. Loranger, H. Homepage Links Remain a Necessity. Available online: https://www.nngroup.com/articles/homepage-links (accessed on 10 May 2019).
  29. W3C Web Accessibility Initiative. Resize Text. Available online: https://www.w3.org/WAI/test-evaluate/preliminary/#resize (accessed on 10 May 2019).
  30. W3C Web Accessibility Initiative. W3C Web Content Accessibility Guidelines 2 Conformance Logos. Available online: https://www.w3.org/WAI/standards-guidelines/wcag/conformance-logos (accessed on 10 May 2019).
  31. Online Broken Link Checker. Broken link. Available online: https://www.brokenlinkcheck.com (accessed on 10 May 2019).
  32. Farrell, S. Navigation: You Are Here. Available online: https://www.nngroup.com/articles/navigation-you-are-here (accessed on 10 May 2019).
  33. Indiana University. Recognition Rather than Recall. Available online: http://www.mattsoave.com/old/cogs187a/iu_site_eval/6recognition.html (accessed on 10 May 2019).
  34. Nielsen, J. Site Map Usability. Available online: https://www.nngroup.com/articles/site-map-usability (accessed on 10 May 2019).
  35. Jackson, B. Color. Available online: https://colorable.jxnblk.com (accessed on 10 May 2019).
  36. W3C Web Accessibility Initiative. Colors with Good Contrast. Available online: https://www.w3.org/WAI/perspective-videos/contrast/ (accessed on 10 May 2019).
  37. Nielsen, J. Improving the Dreaded 404 Error Message. Available online: https://www.nngroup.com/articles/improving-dreaded-404-error-message (accessed on 10 May 2019).
  38. Ssemugabi, S.; De Villiers, R. Effectiveness of heuristic evaluation in usability evaluation of e-learning applications in higher education. S. Afr. Comput. J. 2010, 45, 26–39. [Google Scholar]
  39. Gupta, M. Structure Analysis of the National Libraries’ Websites of the World. Libr. Philos. Pract. 2017, 2017, 1–21. [Google Scholar]
  40. Manzoor, M.; Hussain, W.; Sohaib, O.; Hussain, F.K.; Alkhalaf, S. Methodological investigation for enhancing the usability of university websites. J. Ambient Intell. Hum. Comput. 2018, 10, 531–549. [Google Scholar] [CrossRef]
  41. Carvalho, L.R.; Évora, Y.D.; Zem-Mascarenhas, S.H. Assessment of the usability of a digital learning technology prototype for monitoring intracranial pressure. Revis. Lat.-Am. Enferm. 2016, 24. [Google Scholar] [CrossRef] [PubMed]
  42. Nielsen Norman Group. Strategic Design for Frequently Asked Questions; Nielsen Norman Group: Fremont, CA, USA, 2018. [Google Scholar]
  43. The Museum of Contemporary Art, North Miami. Web Accessibility Statement. Available online: https://mocanomi.org/web-accessibility-statement (accessed on 10 May 2019).
Figure 1. Cross–Correlation Coefficients.
Figure 1. Cross–Correlation Coefficients.
Mti 03 00042 g001
Table 1. List of Museums.
Table 1. List of Museums.
Museum NameCountryAssociated Link
Western Australian Medical MuseumAustraliahttp://www.wamedicalmuseum.org.au/p/home.html
Harry Brookes Allen Museum of Anatomy and PathologyAustraliahttps://harrybrookesallenmuseum.mdhs.unimelb.edu.au
Das anatomische Museum InnsbruckAustriahttp://www.anatomie-innsbruck.at/museum
Anatomical Educational MuseumAzerbaijanhttp://www.amu.edu.az/en/pages/2/2774
Hopital Notre Dame a la roseBelgiumhttps://www.notredamealarose.be/en/
Medical MuseionDenmarkhttp://www.museion.ku.dk/
Ecorché d’anatomie MuseumFrancehttp://www.musee-anatomie.fr/indexUS.htm
Medical-Historic MuseumGermanyhttps://www.uke.de/english/departments-institutes/institutes/history-and-ethics-of-medicine/medical-historic-museum/index.html
Berlin Museum of Medical HistoryGermanyhttps://www.bmm-charite.de/en/index.html
Deutsches Hygiene MuseumGermanyhttps://www.dhmd.de/
Medical Anatomy MuseumGreecehttps://www.uoa.gr/to_panepistimio/moyseia/anatomias
Anatomy MuseumGreecehttp://anatomy.gr
Medical MuseumGreecehttp://museum.med.uoc.gr/
Hong Kong Museum of Medical SciencesHong Konghttps://www.hkmms.org.hk/en/home/
Museum and Historical Library of BiomedicineItaliahttp://www.anfamedmuseo.unipr.it
Collezione anatomica "PaoloGorini"Italiahttp://www.museogorini.com/
Museum of Human AnatomyItaliahttp://museoanatomia.unito.it/index.php/en
Wax Anatomical ModelsItaliahttp://pacs.unica.it/oldmedicina/cere/mono01_en.htm
Pauls Stradins Museum for history of medicineLatviahttp://www.mvm.lv/en/
RSU Anatomy MuseumLatviahttps://www.rsu.lv/en/rsu-anatomy-museum
Museum of the History of Lithuania Medicine and PharmacyLithouaniahttps://www.muziejai.lt/Muziejus.en.asp?Db_kodas=2719&meniu=32
Museum VrolikNetherlandshttps://www.amc.nl/web/museum-vrolik-english.htm
CorpusNetherlandshttps://corpusexperience.nl/en/#!
Rijksmuseum BoerhaaveNetherlandshttps://rijksmuseumboerhaave.nl/
Anatomical MuseumNetherlandshttps://www.lumc.nl/onderwijs/faciliteiten/anatomisch-museum/?setlanguage=English&setcountry=en
Museum of Traditional MedicineJapanhttps://www.morinomiya.ac.jp/harikyumuseum/index_e.html
Museu de Historia da MedicinaPortugalhttps://museumaximianolemos.med.up.pt/index.php
Museu de MedicinaPortugalhttp://www.museudemedicina.fm.ul.pt/DesktopDefault.aspx
Anatomical MuseumScotlandhttps://www.ed.ac.uk/biomedical-sciences/anatomy/anatomical-museum
Basque Museum of the History of MedicineSpainhttps://www.ehu.eus/eu/web/basque-museum-medicine
Medicinhistoriska museetSwedenhttp://medicinhistoriska.sahlgrenska.se
Anatomisches Museum BaselSwitzerlandhttps://anatomie.unibas.ch/museum/en/index.html
Anatomische StudiensammlungSwitzerlandhttps://www.uzh.ch/cmsssl/de/outreach/museums/anatomische-sammlung.html
National Museum of Medicine of UkraineUkrainehttp://nmmu.org/
The Welsh Museum of Health and MedicineUKhttp://www.wmhm.org.uk/
The Wohl Pathology MuseumUKhttps://museum.rcsed.ac.uk/wohl-pathology-museum
Thackray Medical MuseumUKhttps://www.thackraymedicalmuseum.co.uk/
Hunterian MuseumUKhttps://www.rcseng.ac.uk/museums-and-archives/hunterian-museum/
National MuseumUKhttps://www.rcplondon.ac.uk/about-us/what-we-do/museum
Worcester Medical MuseumsUKhttps://medicalmuseum.org.uk/
Warren Anatomical MuseumUSAhttps://www.countway.harvard.edu/center-history-medicine/warren-anatomical-museum
Mütter MuseumUSAhttp://muttermuseum.org/
Mobile Medical MuseumUSAhttps://www.mobilemedicalmuseum.org/
International Museum of Surgical ScienceUSAhttps://imss.org/
National Museum of Health and MedicineUSAhttps://www.medicalmuseum.mil/
The Health MuseumUSAhttp://www.thehealthmuseum.org/
Museum of Medical HistoryUSAhttp://www.ssvms.org/museum.aspx
Table 2. Usability evaluation for website museums by the five experts.
Table 2. Usability evaluation for website museums by the five experts.
Heuristic violatedUsability Factors1st2nd3rd4th5th
H1. Visibility of system statusAvailability of search32%34.04%32%32%32%
H2. Match between system and the real worldMagnifying glass in the search45%45%45%45%45%
H3. User control and freedomThe website logo link to the homepage21.27%17.02%19.15%17.02%19.15%
H4.1. Consistency and standards w3cText gets larger according to user preferences89.40%89.40%89.40%89.40%89.40%
H4.2. Consistency and standards w3cLogo W3C for the level AA, AAA100%100%100%100%100%
H5. Error preventionError page (broken link)12.80%12.80%12.80%12.80%12.80%
H6. Recognition rather than recallBreadcrumb53.20%46.81%53.20%53.20%53.20%
H7. Flexibility and efficiency of useSitemap76.60%76.60%76.60%81%76.60%
H8. Aesthetic and minimalist designContrast of text color to background in the first page0%0%0%0%0%
H9. Help users recognize, diagnose, and recover from errors404—page34.04%42.55%34.04%46.8%42.55%
H10. Help and documentationFAQ (Frequently Asked Questions)89.36%83%91.50%93.62%91.50%
* The percentage values refer to websites with absence (nonexistence) of the corresponding property.
Table 3. Cross–Correlation Matrix for Usability Metrics.
Table 3. Cross–Correlation Matrix for Usability Metrics.
H1H2H3H4.1H5H6H7H9
H20.762
H30.2020.160
H4.10.0880.1060.179
H50.1480.0410.1130.075
H60.3680.4140.3830.2300.103
H70.3780.1940.2870.2980.0610.489
H90.3750.3480.2850.1020.2630.1340.079
H100.0880.0320.1790.1190.0750.2300.2980.102

Share and Cite

MDPI and ACS Style

Kiourexidou, M.; Antonopoulos, N.; Kiourexidou, E.; Piagkou, M.; Kotsakis, R.; Natsis, K. Websites with Multimedia Content: A Heuristic Evaluation of the Medical/Anatomical Museums. Multimodal Technol. Interact. 2019, 3, 42. https://doi.org/10.3390/mti3020042

AMA Style

Kiourexidou M, Antonopoulos N, Kiourexidou E, Piagkou M, Kotsakis R, Natsis K. Websites with Multimedia Content: A Heuristic Evaluation of the Medical/Anatomical Museums. Multimodal Technologies and Interaction. 2019; 3(2):42. https://doi.org/10.3390/mti3020042

Chicago/Turabian Style

Kiourexidou, Matina, Nikos Antonopoulos, Eugenia Kiourexidou, Maria Piagkou, Rigas Kotsakis, and Konstantinos Natsis. 2019. "Websites with Multimedia Content: A Heuristic Evaluation of the Medical/Anatomical Museums" Multimodal Technologies and Interaction 3, no. 2: 42. https://doi.org/10.3390/mti3020042

Article Metrics

Back to TopTop