A Scientometric Study of Usability and User Experience Evaluation: Evolution, Context and Trends

Abstract

Usability and user experience (UX) evaluation frameworks are well established within human–computer interaction and human factors research, yet they are typically applied as general-purpose models that do not fully capture the specific cognitive and perceptual demands of open data visualization systems (ODVS), such as visual reasoning and sense-making. This study examines the evolution of usability and UX evaluation research to identify dominant methods and emerging trends informing future methodological developments for this domain. A scientometric analysis used bibliographic metadata from Scopus, applying a structured search combining usability, UX and evaluation-related terms to titles and author keywords for publications from 1982 to early 2025. After manual screening, 9346 documents were analyzed through bibliometric indicators and keyword co-occurrence techniques using Tableau Public (2025.2) and VOSviewer 1.6.20. The analysis reveals sustained growth in usability and UX evaluation research: while the System Usability Scale and heuristic evaluation remain central, empirical and technology-assisted approaches have gained relevance, alongside increased attention to accessibility, human factors, and complex interactive systems. These findings expose limitations in existing evaluation models when applied to ODVS and support adapting current frameworks or developing complementary attributes addressing the human factors specific to data interpretation and sense-making.

1. Introduction

Usability has been defined over the years by various authors, including Jakob Nielsen (widely known as the father of usability), Nigel Bevan [1], Ben Shneiderman [2], and Jennifer Preece [3,4], among others [5,6], as well as several institutions and organizations related to technology, standards, and quality frameworks, such as ISO and [7,8,9].

The review of these definitions highlights that usability should be seen as a multidimensional concept rather than just as an intrinsic attribute of objects or systems, since, according to various authors and quality models analyzed, usability encompasses a set of characteristics that, depending on the degree of their implementation and fulfillment, determine whether a system can be considered usable or not. Nielsen defines usability as a “quality attribute that assesses how easy user interfaces are to use” [10]. He also conceptualizes usability in terms of five key attributes or components: learnability, memorability, efficiency, low error rate with rapid recovery (error tolerance and prevention), and satisfaction [11]. This latter definition is more appropriate and aligns more closely with other influential authors and quality models in which usability is defined or incorporated as a measurable construct.

Usability is often confused with another closely related concept, namely User Experience (UX). Although usability and UX are related, they differ in both meaning and scope. Like usability, user experience does not have a single, universally accepted definition and can be interpreted in multiple ways. One of the most classical definitions describes UX as the experience resulting from the interaction between a human being and a system [12]. However, it is important to clarify that over the years UX has not been limited to the domain of information systems, websites, or digital media, but also applies to the use of physical products and services [13]. Other definitions of UX describe it as the outcome of the user’s mental state, considering their thoughts, emotions, feelings, perceptions, and mood at the moment of interaction with a product, system, or service [14]. Furthermore, UX can be regarded as an umbrella term encompassing not only attributes and concepts, but also and more importantly, techniques, tools, and research fields such as information architecture and interaction design, including usability itself, which constitutes a key factor to be evaluated within the overall user experience (Figure 1).

Figure 1. Venn diagram originally created by Dan Saffer and edited by Thomas Gläser, illustrating the different disciplines that comprise user experience design [15].

Knowing the definitions of each term, the relationships between them and their differences, it is common to observe that, when conducting online searches related to the ease of use of a product or service and the impact it generates on users, these terms are often used interchangeably without affecting the overall search results.

As mentioned earlier, user experience encompasses a broad set of techniques, attributes, tools, and even professional roles, all of which are aimed at understanding and improving the process involved in using a product, a system, or accessing a service, whether physical/on-site or digital.

Information systems constitute a resource that is highly susceptible to analysis, design, and evaluation from the perspectives of user experience and usability. This is because they are not merely means for content consultation or publication but often require greater skills or knowledge for their management and use. Therefore, it is necessary to adopt a user-centered design approach, seeking to minimize design, logic, interaction, and content errors that may negatively affect the use of such products.

Nowadays, there is a large number of methods [16,17], models [18,19,20,21], as well as standards and guidelines [8,9,13] for evaluating the usability of websites in general. Moreover, usability evaluation approaches have been proposed for specific contexts or domains, such as education [22,23], banking [24], and healthcare [25,26], among others.

Despite the existence of a large number of models, techniques, standards, and methodologies for evaluating the usability of websites, such as those mentioned above, there is little to almost no research specifically focused on evaluating this particular type of website, namely open data visualization systems. Consequently, although such sites may comply with certain usability or user experience-related attributes, characteristics, or quality indicators (e.g., learnability, memorability, understandability, efficiency of use, satisfaction, consistency, accessibility, and navigability), they fall short in capturing and addressing intrinsic features that are essential to this type of website. These include aspects such as data representation, types of visualizations, the message conveyed, relevance, relationships among data, the variables employed, and the application of perceptual principles.

For this reason, this study seeks to explore how the concepts of usability and user experience have been addressed in evaluations, measurements, and analyses within the scientific literature through a scientometric analysis of research trends on these topics. The aim is to identify existing evaluation models, as well as the attributes and characteristics proposed in such models, which may be applicable to ODVS, and to formulate potential new attributes.

This article presents the results of a scientometric analysis of research trends in the context of usability and user experience, using metadata from journal articles published since the emergence of these research streams with an emphasis on evaluations, measurements, estimations, analyzes, and similar approaches in information systems contexts.

2. Methods

The process for analyzing research trends related to methodologies, procedures, and methods for evaluating usability and user experience was based on two main stages: data collection and subsequent analysis. The methodology used to gather the most relevant metadata for the research, as well as the techniques and tools used for visualizing and analyzing the resulting data, is described below.

The tool used as the data source was Scopus, a bibliographic database widely recognized as one of the largest and most reliable repositories of abstracts and citations, and it has been regarded as one of the most prominent databases worldwide since 2004 [27].

Temporal attributes such as publication dates or years were not included in the search query; therefore, all documents published between 1982 and 2025 were considered. It is worth noting that usability and user experience began to emerge as significant research topics in the 1990s, a decade during which studies related to the themes addressed in this article started to be published with greater frequency.

For the construction of this query, the two main terms were included along with their abbreviations, as well as different verbs related to evaluation, testing, assessment, measurement, and similar activities. The asterisk (*) was used as a wildcard operator to include all variations of a word root (e.g., evaluat* for evaluated, evaluation, evaluate, etc.). These terms were combined using logical operators and proximity operators provided by Scopus, and the search was restricted to two fields: Title (TITLE) and Author Keywords (AUTHKEY).

As the final outcome of the first step or stage of the methodology described above, a search query was obtained that aimed to retrieve the largest possible number of published documents explicitly related to usability and user experience evaluation methods, metrics, methodologies, and processes, as presented in Table 1.

Table 1. Search expression.

Id	Expression
1	(usability OR “user experience” OR ux) W/1 (evaluat* OR assess* OR measur* OR test* OR apprais* OR metric* OR indicat* OR index* OR scale* OR gauge* OR valid* OR verificat* OR judg* OR benchmark* OR examination* OR estimat* OR analy*)
(TITLE(1) OR AUTHKEY(1))
TITLE ((usability OR “user experience” OR ux) W/1 (evaluat* OR assess* OR measur* OR test* OR apprais* OR metric* OR indicat* OR index* OR scale* OR gauge* OR valid* OR ver-ificat* OR judg* OR benchmark* OR examination* OR estimat* OR analy*)) OR AUTHKEY ((usability OR “user experience” OR ux) W/1 (evaluat* OR assess* OR measur* OR test* OR apprais* OR metric* OR indicat* OR index* OR scale* OR gauge* OR valid* OR verificat* OR judg* OR benchmark* OR examination* OR estimat* OR analy*))

The search and retrieval of bibliographic records were conducted on 17 January 2025 exclusively on records indexed in Scopus, and did not include additional filters; however, to ensure reproducibility, it would be advisable to add a date filter up to the date previously indicated. Applying the search query through the Scopus web interface yielded a total of 9668 results.

Using the tools available in the Scopus web interface, the bibliographic records were downloaded in order to obtain information on authors, title, year of publication, source name, abstract, and, most importantly, the EID (Electronic Identifier of the document).

Subsequently, a manual screening process was carried out by examining the titles of the retrieved records to determine whether they were aligned with the research topic and the inclusion criteria defined by the search strategy. When the title did not provide sufficient information to determine the relevance of a record, its abstract was also examined. During this screening process, records were excluded when the terms usability or user experience did not refer to the evaluation of the ease, effectiveness, efficiency, satisfaction, or overall quality of interaction with a digital or interactive system. In particular, studies were excluded when usability was used in the sense of the possibility, suitability, applicability, or feasibility of using something rather than as a human–computer interaction or user experience construct; when the study focused on physical products, materials, or non-digital services without software or digital-system usability as its primary focus; when robotics studies addressed the usability or performance of robotic hardware or physical robotic systems rather than their software, interfaces, or human–computer interaction; or when medical and biomedical studies concerned the use or application of equipment, devices, or procedures in areas such as bones, knees, musculoskeletal conditions, or osteoarthritis without evaluating the usability or user experience of a digital system. Records were also excluded when their research objectives were unrelated to the evaluation, assessment, measurement, testing, or analysis of usability or user experience in digital systems, even when the corresponding terms appeared in the title, abstract, or author keywords and therefore satisfied the search expression.

The screening was initially performed by one researcher following the predefined inclusion and exclusion criteria. Subsequently, the other authors conducted random checks of the screening decisions and the resulting dataset to assess the consistency of the selection process. Disagreements or cases requiring clarification were jointly reviewed by all authors, discussed, and resolved by consensus. This audit process helped reduce the potential influence of individual judgement on the final corpus and strengthened the consistency and transparency of the selection process.

No additional databases were used, no language restrictions were applied, and the search strategy was designed around terms associated with usability and user experience evaluation, including their commonly used methodological and measurement variants. This choice may introduce database and terminology-related biases, as relevant studies not indexed in Scopus or using different terminology may not have been retrieved. In addition, the manual screening process involved researcher judgement when determining the conceptual relevance of records, particularly in cases where the term usability was used outside the human–computer interaction perspective adopted in this study. These considerations, together with differences in author keyword practices and Scopus indexing coverage, should therefore be taken into account when interpreting the results and their generalizability.

Although the corpus was not restricted to open data visualization systems, the identified research landscape provides a basis for examining how this specific application context is represented within the broader usability and user experience evaluation literature.

In some cases, and given that the titles were not clear enough, the abstracts were also reviewed, trying to not reduce but instead make sure that the research objective and the document itself were in accordance with the research topic.

After this review, a total of 322 documents were excluded. Although these records met the search criteria, they were found to address different approaches or contextual perspectives. As a result of this filtering process, 96.6% of the documents were preserved, suggesting that the search expression had been appropriately defined.

Content analysis was performed using Tableau Public (2025.2) [28] and VOSviewer 1.6.20 [29]. The bibliometric analysis was performed in Tableau using tables and different types of visualizations. Publications by period, most productive authors, most cited articles, and other relevant factors were some of the analyses conducted.

This first analysis involved prioritizing author keywords based on the number of associated documents, but before this, a homogenization process was carried out to unify terms with the same meaning and to correct inconsistencies or variations in capitalization. For example, (UX, User Experience (UX), User experiences, User-experience, User’s experience) were replaced with user experience; (Eye movement tracking, eye-tracking, eyetracking, eye movement analysis) were replaced with eye tracking; and (children, child, early childhood, infant, pre-school children, child participants) were replaced with children. It is important to note that this procedure was applied to all terms with more than two occurrences, as well as to some single-occurrence terms whose replacement term was considered “strong” in terms of frequency of appearance. Finally, all terms were converted to lowercase.

After this, the VOSviewer 1.6.20 tool was used to execute a co-occurrence analysis and identify possible trends, generating a general map based on keywords defined by the author. A minimum occurrence threshold of 15 was applied, reducing the initial set of 12,533 keywords to 283 terms included in the co-occurrence analysis. VOSviewer 1.6.20’s default normalization and design parameters were used, generating a map with eight clusters for this specific analysis. For all VOSviewer 1.6.20 maps, node size represents occurrence frequency, link strength represents the strength of co-occurrence relationships, and cluster colors represent thematic groupings.

3. Results and Discussion

Table 2 presents the top 15 terms or author keywords with the highest number of occurrences. From this list, the term “NULL” (indicating the absence of author keywords in a document) was excluded, as well as those terms that were part of the search query, whose high frequency was expected, such as usability, UX, user experience, among others. The resulting set of keywords provides a meaningful representation of the main thematic research lines currently shaping the field of usability and user experience (UX) evaluation. First, the term “human-computer interaction” stands out above the others due to the size and prominence of the node, confirming that the discipline maintains a structural focus on understanding and optimizing the interfaces between people and technology. A high presence of classic evaluation methods such as the system usability scale, heuristic evaluation, think aloud and questionnaire were also found. Due to the prominence of those nodes on map, it can be seen that despite the emergence of advanced technologies, methodological approaches are essential for establishing the quality of the user experience.

Table 2. Top 15 Author keywords.

Rank	Keyword	Documents	Rank	Keyword	Documents
1	human–computer interaction	550	9	user interface design	168
2	system usability scale	470	10	older adults	167
3	mobile application	386	11	ehealth	157
4	user-centered design	361	12	augmented reality	142
5	heuristic evaluation	322	13	e-learning	135
6	virtual reality	205	14	think aloud	123
7	eye tracking	205	15	questionnaire	119
8	mhealth	193

From a more contemporary and modern point of view, terms such as mobile application, virtual reality, augmented reality, and eye tracking reflect how usability evaluation has had to adapt to new devices (and therefore, ways to interact with them), immersive environments, and context-aware interfaces, requiring the development of specific metrics and tools to accurately capture user interaction. Likewise, the emphasis on user-centered design and user interface design suggests that designing around users’ real needs continues to be a foundational principle, now strengthened by the integration of empirical data and iterative techniques.

The appearance of terms such as eHealth, mHealth, older adults, and e-learning highlights an expansion of the field toward critical social domains such as healthcare and education, introducing new ethical, accessibility, and personalization challenges. However, there is no “strong” representation of visualization systems, data visualization, or any terms directly associated with that context, suggesting that research into the usability and UX evaluation of ODVS remains quite limited to date. Overall, this table not only synthesizes current trends but also reveals a field that preserves its traditional evaluative foundations while dynamically adapting to emerging technological and demographic realities.

Additionally, author keywords strictly related to usability and/or user experience analysis, design, and evaluation methods were examined and are detailed in Table 3. In some cases, such as heuristic evaluation (ranked #2), several related terms (e.g., heuristic evaluation, heuristics, usability heuristics) were grouped together, and their occurrences were therefore aggregated in the count.

Table 3. Top 20 Methods.

Rank	Keywords (Grouped)	Documents	Rank	Keywords (Grouped)	Documents
1	system usability scale	470	11	focus groups	37
2	heuristic evaluation, heuristics, usability heuristics	422	12	design thinking	37
3	questionnaire; user experience questionnaire	213	13	benchmarking	22
4	eye movement and eye tracking	215	14	iso 9241	24
5	think aloud	123	15	interviews	27
6	testing, user experience testing, user testing	147	16	mixed methods	14
7	cognitive walkthrough	75	17	online survey	17
8	remote testing; remote usability testing	87	18	sentiment analysis	18
9	usability evaluation (automated and web)	85	19	personas	20
10	qualitative research	41	20	scenarios	17

Table 3 shows a clear overview of the current state of methodological practices in the field. The method known as System Usability Scale (SUS) is highly prominent due to its 470 mentions in documents. This largely confirms its position as a standard tool for getting fast and comparable evaluations, especially when you face contexts where efficiency and quantitative validation are required. Secondly, there is heuristic evaluation, an expert inspection technique which is widely used due to its low cost, and it can be applied in early design processes or stages. The prominence of the term could be interpreted as a preference for methods that allows evaluate and identify usability issues without the involvement of end users. The terms questionnaire (or related) also are highly mentioned and it is very frequent the combination of questionnaires with techniques such as think aloud suggesting an intentional effort to balance reported experiences with observed behavior, considering that as the anthropologist Margaret Mead says: “What people say, what people do, and what they say they do are entirely different things” [30]. On the other hand, there are contexts where perception and human psychology influence the resulting experience of using a product or service, such as neuromarketing [31,32,33]; the growing interest in technologies such as eye tracking points to a shift toward more sophisticated and objective methods that support the analysis of visual and cognitive patterns in complex environments, especially in advanced interactive systems.

Additionally, the grouping of terms such as testing, remote usability testing, and automated evaluation indicates an evolution toward flexible and scalable methods adapted to distributed contexts, such as web applications or remote testing scenarios. In some contexts or scenarios, conducting on-site testing is unlikely or inconvenient due to limited resources or the context per se. Remote testing allows you to gather users feedback when travel budgets are limited, deadlines are tight, or test participants are difficult to find meanwhile automated methods are highly efficient because they could be executed very fast and the results always come from the same parameters, without subjective assessments. The main problem with these methods is that they are always based on almost finished or finished systems and cannot be used in the early stages of the development life cycle.

Methods such as the cognitive walkthrough are less frequent but continue to play a relevant role in validating step-by-step processes from an expert’s perspective. Finally, the inclusion of qualitative research reinforces the importance of understanding users’ contexts, emotions, and motivations beyond numerical metrics, particularly in sensitive domains such as healthcare and education. Qualitative research aims to answer questions such as “what?”, “why?”, and “how?” to extract and analyze irrational and unconscious aspects, and requires small samples, which is an advantage in many cases due to resource issues (budget, time, and personnel). In general, this table shows that while traditional evaluation methods remain prevalent, the field is increasingly incorporating more hybrid, technology-driven, and context-aware approaches in response to the growing complexity of systems and the diversity of their users.

The same analysis was performed by selecting, from the top 100 most frequent keywords, those characteristics, dimensions, or attributes that set the direction of current research. Accessibility was ranked first, and this is not surprising, considering that accessibility is recognized as an essential attribute of usability according to standards such as ISO 9241-11:2018 (Ergonomics of human–system interaction—Part 11: Usability: Definitions and concepts) [9]. This standard establishes that systems must be effective, efficient, and satisfactory for all users, including those with disabilities. Consequently, accessibility has become a transversal dimension within user experience, particularly in inclusive or universal design contexts. The growing prominence of accessibility as a keyword partly reflects increasing regulatory and legal pressure (e.g., WCAG, UNE-EN 301549:2022 [34] in the European Union, Section 508 in the United States, among others). All these regulatory frameworks seek to guarantee digital accessibility for people with disabilities. In addition, there is a strengthening ethical commitment to digital inclusion. Considering population aging and functional diversity, there is a growing demand for interfaces adapted to diverse user needs, highlighting the importance of research in this area. Not too far in position number two, human factors reveal the need from study and understand the “environmental, organizational and job factors, and human and individual characteristics, which influence behavior at work in a way which can affect health and safety” [35]. Table 4 presents the top ten terms directly related to usability characteristics, dimensions, or attributes.

Table 4. Characteristics, dimensions or attributes.

Rank	Keywords (Grouped)	Documents	Rank	Keywords (Grouped)	Documents
1	accessibility	92	6	efficiency	48
2	human factors	86	7	perceived usability	38
3	information visualization; visualization	80	8	security	34
4	satisfaction	60	9	quality	26
5	effectiveness	49	10	validity	25

On the other hand, considering the temporal occurrence of keywords, an analysis was conducted to identify how research themes have evolved over the past four decades. Given the relatively small number of documents identified in the early years (1982–1994), the first era was defined as a 12-year period. Subsequently, the timeline was divided into five-year intervals (lustrums) up to the most recent period (2020–2025), which includes five years and the first 17 days of January.

The analysis shows that during the last five years (2020–2025), corresponding to the post-COVID era, there has been a high incidence of publications related to the medical domain. This period is also characterized by a strong focus on older adults, driven by the need for health applications that can be used autonomously.

None of the stages revealed terminology linked to ODVS evaluation, due to the constraint of filtering by the top ten terms per period. This total absence reflects a broader thematic landscape across the analyzed dataset and shows how literature remains heavily dominated by other domains like medical and educational contexts, as well as emerging technologies. This profound lack of research not only underscores a critical literature gap regarding ODVS as a whole but also provides a powerful empirical justification for the necessity and novelty of this study. Table 5 presents the categorization of research themes over time based on keyword occurrence. It should be noted that the top ten positions (in terms of number of documents) were selected for each era, grouping some related keywords and analyzing the research topics of its.

Table 5. Themes evolution over time.

Time Period	Keywords (Documents)	Research Topics
1982–1994	guidelines (5) people with disabilities (4) analysis and evaluation techniques (4) user interface design; user interface evaluation (grouped 4) ergonomics; software ergonomics (grouped 4) walkthroughs (2) user-centered design (2) tools (2) style guide (2) product development (2) internationalization (2) graphical user interface design (2)	Guidelines Usability evaluation across different types of interfaces and interactive systems User-friendly documentation Technical considerations in usability evaluation Cognitive walkthrough Expert-based evaluation methodologies User-centered design Product development from an ergonomics perspective
1995–1999	Web; web design; web usability; website (grouped 12) user interface design (10) user-centered design (9) task analysis (8) product development (8) human–computer interaction; human factors (grouped 8) usability engineering (5) style guide (5) prototyping (5) practical techniques (5)	Web-focused content, encompassing both design and evaluation User-centered product design and development Reduced emphasis on theory and greater focus on practice in user-centered design Human factors and their relationship and interaction with technology
2000–2004	Web; web design; web usability; website (grouped 26) user-centered design (15) user interface design (12) human–computer interaction (11) usability engineering (10) heuristic evaluation (10) human factors (9) user studies (8) children (8) design (7) automated usability evaluation (7)	Web-focused content, addressing both design and evaluation User interface design centered on users, taking human factors into account User-based and expert-based evaluation techniques Interfaces for children and usability testing with children
2005–2009	human–computer interaction (70) web; web design; web usability; website (grouped 68) user-centered design (49) heuristic evaluation (31) user interface design (24) think aloud (20) e-learning (20) user studies (19) usability engineering (16) internet (16) digital libraries (16)	The web and the Internet as the primary technological context, with an emphasis on improving user experience Education and digital learning driven by computing and technology User-centered interface design
2010–2014	human–computer interaction (119) user-centered design (61) web; web design; web usability; website (grouped 60) heuristic evaluation (59) eye tracking (46) mobile application (32) user interface design (30) electronic health records (27) e-learning (27) usability engineering (26) think aloud (26)	Human–computer interaction in different contexts After the mobile technology boom, increased focus on improving interfaces and user experience Migration of analog processes to electronic systems in the healthcare domain Technology-driven learning Web design adapted to the technological context of the era New usability measurement techniques
2015–2019	human–computer interaction (145) mobile application (127) system usability scale (114) heuristic evaluation (100) user-centered design (96) eye tracking (73) web; web design; web usability; website (grouped 69) older adults (51) mhealth (51) virtual reality (47)	Study of human–computer interaction, driven by the evolution and emergence of innovative technologies and interfaces Mobile applications across different contexts Evaluation of user experience in diverse contexts using multiple types of techniques Eye tracking and virtual reality as immersive techniques for experience analysis and evaluation Digital health and the use of Information and Communication Technologies in healthcare, such as electronic medical records, with a particular emphasis on older adults Design and evaluation of websites from usability and user experience perspective
2020–2025	system usability scale (325) mobile application (225) human–computer interaction (200) virtual reality (138) mhealth (136) user-centered design (129) heuristic evaluation (121) ehealth (103) augmented reality (84) older adults (83)	Usability evaluation using the System Usability Scale Mobile applications aligned with contemporary technological contexts Human–computer interaction, driven by the wide variety of available interface types Virtual reality and augmented reality as innovative technological strategies User-centered digital health Design and evaluation methods aimed at improving user experience Mobile health applications, particularly for older adults

Figure 2 shows a keyword map generated with the VOSviewer 1.6.20 tool, illustrating the presence of the research themes published between 1982 and 2025. It is important to highlight that terms included in the search equation, such as usability, user experience, or usability evaluation, were excluded from the analysis because the aim was to identify specific topics associated with the main theme or current research trends. Including such general terms, and in this case, the main terms of the search equation, could minimize the presence of other, more relevant terms for the analysis.

Figure 2. Keyword map of author’s published documents between 1982 and 2025.

Broadly speaking, several clearly defined clusters can be identified, grouping related terms such as human–computer interaction, system usability scale, mobile application, and heuristic evaluation, among others. The central node (larger in size and visually prominent) corresponds to human–computer interaction, highlighting its role as an articulating concept connecting different secondary areas or sub-themes such as emerging technologies (virtual reality, augmented reality), application contexts (mHealth, telemedicine, education), and evaluation methods (heuristic evaluation, user testing, eye tracking). Other terms associated with accessibility, digital health, and mobile technology can also be observed, indicating a trend toward inclusive design and the use of interactive technologies in sensitive contexts. Overall, the map reveals a thematic network that integrates diverse methodological, technological, and contextual approaches to addressing usability and user experience.

After the general analysis, each cluster was studied individually. In the first cluster, shown in Figure 3, it can be observed a focus on human–computer interaction, this is the main node with a size much higher than others. As a whole, all of it associated terms show an approach on contextual, methodological and technical aspects of usability research, centered on evaluation, automation, accessibility (as a design factor), and user-centered design. Specifically, the presence of terms associated with formal and well-known usability evaluation methods such as usability metrics, usability heuristics, evaluation framework, validation, empirical study, ISO 9241, cognitive walkthrough, log file analysis, inspection methods, and model-based usability evaluation is evident, reflecting the classic methodological framework, focusing on how the system is evaluated. The priority is not to introduce new evaluation methods, but to adapt and validate existing approaches to meet the demands of increasingly complex interactive systems.

Figure 3. Cluster 1.

The cluster also includes applied tools, environments, and terms directly used in real-world design and interaction contexts, such as smartphone app, video games, automated teller machine, navigation, remote usability testing, and usability testing tools. This is complemented by an inclusive and universal design perspective, as reflected in terms such as accessibility, visually impaired, universal design, and human-centered design.

The coexistence of terms related to accessibility, user diversity, and emerging technologies points to a discipline in continuous evolution, in which human–computer interaction addresses not only efficiency and usability, but also inclusion and adaptation to changing technological contexts. Finally, this cluster represents the intersection between theoretical rigor and practical application, revealing how human–computer interaction extends toward emerging technologies and diverse populations while maintaining a strong commitment to the continuous improvement of user experience.

The cluster presented in Figure 4 focuses on the evaluation of usability and user experience in emerging technologies applied to educational and mobile contexts, primarily through standardized tools such as the System Usability Scale (SUS—previously identified as the most frequently occurring method in the analyzed documents) and models such as the Technology Acceptance Model (TAM). Both the system usability scale and user experience evaluation function as central axes that structure the research within this cluster.

Figure 4. Cluster 2.

In the educational context, there is an increasing need to study and apply methodologies focused on learning through emerging technologies, such as augmented reality, virtual reality, and mixed reality, introduced into classroom settings, particularly at lower educational levels (children), with the aim of making learning processes more meaningful, dynamic, and engaging for students (game-based learning, serious games, games, and gamification). At the same time, these learning processes are closely linked to mobile technologies and interfaces (mobile devices, mobile usability testing, smartphone, tablet, mobile learning), as well as to how acceptance, quality of experience, and interaction are measured in modern interactive systems for educational purposes (user experience evaluation, user experience questionnaire, quality of experience, interface, interaction, communication, collaboration, learning engagement).

This cluster shows a strong interconnection between immersive technologies (augmented reality, virtual reality), mobile devices, and educational contexts (education, higher education), illustrating how usability evaluation adapts to variable and always available environments not only by integrating technical dimensions, but also aspects of social interaction, collaboration, and user engagement. Moreover, the presence of terms related to mobile banking and social networks indicates that these methodologies extend beyond educational settings into diverse everyday contexts. In conclusion, this cluster brings together usability evaluation methods with immersive, mobile, and educational technologies to optimize user experience in these environments. Regarding ODVS, no direct or strong relationship was found between the use of standardized evaluations such as the SUS (the central node of this cluster) and ODVS; this suggests a lack of research (or a complete absence thereof) into the application of such usability and/or user experience evaluation methodologies in this context. This may be partly explained by the fact that the questions included in this questionnaire do not fully capture the intrinsic nature of ODVS. Consequently, it may be considered an inadequate tool for evaluating usability and UX, or for subsequently improving their quality, usability, and UX levels.

The cluster in Figure 5 shows a strong emphasis on the digital health context, with terms like mobile application, mHealth, mHealth applications, and telemedicine, with user-centered design serving as a main component. In this regard, and as reflected by the prominence of the node sizes, user-centered design acts as the main methodological axis, while the other terms represent the most recurrent application domains within digital health.

Figure 5. Cluster 3.

User-centered design is defined as “an iterative design process in which designers focus on users and their needs in each phase of the design process” [36], more specifically, it focuses on the design of physical products, digital products, and services centered on users. Within this process, phases of analysis, design, and evaluation are incorporated. Accordingly, the figure reveals a set of techniques, methodologies, and attributes associated with each of these phases, such as personas, interviews, co-design, and user requirements.

All of this translates into products and services that are increasingly easy for users to adopt, which is essential in applications aimed at supporting self-managed healthcare, particularly for older adults (older adults, self-management, decision aids, self-care). The most prominent diseases addressed in the literature include COVID, dementia, stroke, cancer, mental health, and diabetes mellitus.

The cluster shown in Figure 6 is primarily focused on heuristic evaluation (the dominant node), which is an inspection method in which an expert examines and evaluates the usability of an interface based on a set of heuristics or rules. This method is closely connected to other techniques such as eye tracking, usability analysis, user testing, and think aloud. All of these reinforce the empirical nature of the analysis and represent methodologies based on direct observation of user behavior as a way to understand users and their context. These approaches enable the evaluation of usability and user experience in websites and information systems, particularly in contexts such as digital libraries (digital libraries, academic libraries, library website), electronic commerce (e-commerce, e-commerce websites), and governmental platforms (e-government), represented in related nodes.

Figure 6. Cluster 4.

Additionally, a strong relationship can be observed among three terms—effectiveness, efficiency, and satisfaction—which are part of one of the most widely accepted definitions of usability: “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” [9]. The presence of metrics such as learnability and user satisfaction (key factors in Nielsen’s [11] definition of usability) along with statistical techniques such as factor analysis, reinforces the pursuit of objectivity in measurement.

Likewise, the inclusion of terms such as web accessibility and information architecture highlights concerns related to structural and inclusion aspects, while the use of standardized questionnaires such as the SUS strengthens the comparability and validity of results.

This cluster integrates inspection methods, observational techniques, and quantitative measurements, forming a robust framework for usability evaluation in web-based environments, including and adapting to the needs of a diverse population in terms of abilities. However, as was the case with the SUS, no direct link was found to terms associated with data or information visualization, open data systems, and the like. Traditional usability frameworks and heuristics focus on general interaction patterns—such as feedback or error prevention—which are necessary but insufficient for ODVS. These systems demand domain-specific evaluation criteria that directly address core data UX dimensions, including visual encoding accuracy, interactive exploration, cognitive load, and the user’s ability to derive meaningful insights.

Evaluation methods have progressively evolved to incorporate more advanced technologies, particularly artificial intelligence, as part of their assessment strategies, integrating techniques such as machine learning and natural language processing to support the analysis of user behavior, perceptions, and responses. Several of these techniques have been used in different contexts, such as healthcare (machine learning) [37], tourism (sentiment analysis) [38], marketing (electroencephalography—EEG) [39], development (brain–computer interface—BCI) [40], among others. All these methods seek to evaluate usability, emotions, perceptions, feelings, and the overall user experience from more organic contexts, interpreting what humans say, feel, think, and the direct reflection of their body signals such as heartbeat, body movements, and eye movements, as well as their brain’s responses to different types of stimuli. This is the only cluster directly related to terms such as data visualization and information visualization. In the case of data visualization, it appears as a step within a data analysis process, associated with terms like data analysis, data mining and dashboard. Information visualization, on the other hand, is interpreted more as a context of use for analyzing the information presented. Terms such as ontology, semantic web, information retrieval, and sentiment analysis underscore this point. Overall, Figure 7 indicates a shift toward evaluation practices that seek to address the growing complexity of contemporary interfaces by combining established human factors methods with computational and data-driven techniques.

Figure 7. Cluster 5.

The next cluster, presented in Figure 8, primarily focuses on user interface design and interaction with interfaces (interaction design, adaptive interfaces, multimodal interfaces, and medical devices). It reflects a strong orientation toward empirical research (empirical methods), encompassing both contextual approaches (ethnography, contextual inquiry, user research, case study, user studies) and participatory methods (questionnaire, online survey), as well as comparative and trend-based analyses (benchmarking). In addition, this cluster is associated with research and techniques that extend beyond the analysis phase, including design-related activities (design principles, user experience design, prototyping) and evaluation (interface evaluation).

Figure 8. Cluster 6.

Altogether, this cluster highlights the prominent role of user research as a strategy for improving interfaces in technologically complex and critical environments (such as ubiquitous computing and healthcare) throughout the entire process, from design to the final user experience. ODVS as visualization interfaces do not appear in this cluster, unlike multimodal, wearable, or adaptive interfaces. The only specific context they encompass is that of healthcare, through medical devices and wearable interfaces.

The cluster shown in Figure 9 emphasizes the relationship between usability evaluation and health information systems, particularly electronic health records (EHRs), commonly known as medical records, where all the information of the patients and their treatments is stored. The graphic shows a scenario where terms like health information systems, hospital information systems, clinical decision support, and medical informatics define the context of use and, therefore, the usability evaluation of different types of interfaces. Currently, and especially in a post-COVID era, where the growth of health applications has increased exponentially [41,42,43,44] the design and evaluation of user-centered systems and applications that are easy to use for young people, older adults, and people without technological knowledge is really important.

Figure 9. Cluster 7.

Key terms such as usability assessment, assessment, user–computer interface, and heuristics points to the use of evaluation methods that seek to guarantee that clinical interfaces are effective and safe. Additionally, remarks on the importance of validity, reliability, human factors and user feedback, which are fundamental aspects of formal evaluations, especially in critical systems.

Moreover, this cluster incorporates a strong human and cognitive dimension, with terms such as human factors, ergonomics, cognitive load, mental workload, and simulation, which demonstrate a focus on reducing mental workload for both healthcare professionals and patients, as well as on improving patient safety. This is achieved through the design of user-friendly interfaces based on principles of ergonomics and human factors, which constitute the foundations and origins of usability [2,45].

Patient safety and clinical decision support are terms with a high presence, reinforcing the idea that proper usability evaluation of electronic health records not only optimizes user experience but also has a direct impact on the quality of care and patient safety. All of this shows that usability in health information systems is not limited to the technical evaluation of interfaces but represents an interdisciplinary challenge in which medical informatics, human factors, ergonomics, and cognitive psychology converge.

There is a significant difference between EHRs and ODVS. EHRs are private and intended for use by medical personnel, whereas ODVSs (as the name implies) are for public use. Because they serve a broader, more diverse audience (who may or may not be familiar with the context or the tool itself), they must be as usable and understandable as possible; consequently, user-centered design and evaluation are key factors in the success of such systems.

Figure 10 shows a set of terms that focuses on the evaluation of digital technologies (software, mobile technology, interactive digital television) considering dimensions like security, privacy, quality, and performance, which are fundamental aspects for functionality and to assure the operational continuity and market success of any software product according to [46,47]. Other concepts such as prototype, testing, biometrics, authentication, methodology, and standards appear, reflecting also an interest in ensuring that systems (particularly those based on mobile and interactive interfaces) are reliable and secure, through user’s perception, biological characteristics as a security method and guidelines.

Figure 10. Cluster 8.

Evaluation methodology is also linked to highly sensitive contexts, such as autism spectrum disorder, where data protection and accuracy are critical, highlighting the need to carefully assess the risks and benefits of digital technologies before their implementation. In conclusion, this cluster puts a strong emphasis on methodological rigor and the development of normative frameworks that ensure not only system quality and performance, but also user security and privacy, especially in critical scenarios where the protection of personal data and the inclusion of vulnerable populations are essential.

Although the bibliometric analysis was conducted on the broader literature on usability and user experience evaluation, the results were examined specifically to identify the extent to which open data visualization systems (ODVS) are represented within this research landscape. Terms directly associated with open data visualization showed limited representation across the identified clusters, whereas other application contexts, such as digital health, education, and immersive technologies, were more strongly represented. This finding does not imply an absence of research on open data visualization but rather suggests that its integration with usability and user experience evaluation remains less consolidated within the broader research landscape.

It is important to note that the evaluation attributes and methods identified in Table 4 and the bibliometric clusters represent complementary but independent views of the existing knowledge on usability evaluation and were not derived specifically for ODVS. Although these attributes and methods can be applied to such systems, they do not necessarily capture all the characteristics and evaluation requirements associated with this type of platform. ODVS combines usability concerns with specific characteristics related to data accessibility, exploration, interpretation, transparency, interaction with heterogeneous datasets, and the communication of data-driven information. Accordingly, the contribution of the bibliometric analysis is not to establish a direct mapping between clusters and operational evaluation criteria, but to characterize the thematic scope of usability evaluation research and reveal the relatively limited representation of ODVS within it. When considered together with the evaluation attributes summarized in Table 4, these findings provide a basis for identifying which existing approaches can be transferred to ODVS and, more importantly, where they may leave relevant characteristics of these systems insufficiently addressed. This gap motivates further investigation of evaluation approaches specifically tailored to ODVS.

4. Conclusions

The study covered more than four decades, from 1982 until early 2025, during which the evolution (size), consolidation (behavior and growth pattern), and diversification (tools, topics, and context) of research in usability and user experience were observed. An exponential growth in the volume of publications was noted, along with a significant thematic and methodological expansion. This thematic expansion has been product of several reasons such as the technological advances (from web and mobile technology to artificial intelligence and brain–computer interfaces), also emerging social needs (such as population aging, digital health, and the challenges of education and healthcare in a post-COVID era), and the emergence of thematic axes such as user-centered design, which changed the process of design and evaluating products and services.

The System Usability Scale is one of the most widely used methodologies, acting as a central focus of a large number of investigations, mainly due to its ease of use and rapid implementation in almost every context. Other methods such as heuristic evaluation and empirical testing remain highly relevant and demonstrate their great ability to adapt to different contexts such as education, medicine, e-commerce, and artificial intelligence, combining and complementing with emerging methodologies and techniques such as eye tracking, think-aloud protocols, sentiment analysis, and the use of biometrics in critical contexts.

The analysis also shows how usability and user experience function as transversal axes across multiple disciplines and contexts, with applications ranging from video games to electronic health records and highlights the need for robust methodological and normative frameworks to ensure system quality, privacy, and security, not just from the evaluation of ended products but from the first stages of design.

Additionally, the study revealed a growing concern for design attributes and factors such as accessibility, efficiency, and satisfaction (these last two are two of the key factors that define the usability of a product according to [9]), reinforcing the need to design digital products that are not only easy to use but also ethically inclusive, considering human and cognitive factors such as ergonomics and mental workload. In the specific case of ODVS, while these attributes are important, they are insufficient to fully capture the demands of exploratory, data intensive environments, and it is also necessary to consider other characteristics such as visual clarity, accuracy, interactivity, user-centered design, storytelling, fast performance, high-quality content, understandable, and in open formats.

With this in mind (the thematic patterns identified in the bibliometric analysis and the specific characteristics of ODVS), we propose a preliminary evaluation framework that brings together what should be evaluated and how these aspects could be assessed. The framework is organized into three complementary levels. General usability and UX attributes (Level 1) that include effectiveness, efficiency, satisfaction, accessibility, acceptability, and perceived usability, which represent established dimensions consistently associated with usability and user experience evaluation. A second level based on human and cognitive factors (Level 2) that incorporates human factors, cognitive workload, mental workload, comprehension, and decision support, reflecting the increasing presence of these concepts in the identified thematic clusters and their relevance to interaction with complex digital systems. And finally, an ODVS specific attributes (Level 3) extends these dimensions to address the particular demands of data-intensive and exploratory environments, including visual clarity, visualization adequacy, data accuracy, interactivity, data comprehension, sense-making, storytelling, data quality, performance, and openness/reusability.

The first two levels are supported by the attributes, methods, and thematic patterns identified in the analyzed corpus, while the third responds to the specific research gap identified for ODVS. To make these dimensions more actionable, existing evaluation approaches could be adapted to include tasks related to data exploration, visualization interpretation, sense-making, and decision-making. This would allow observable aspects such as task completion, interaction time, interpretation accuracy, perceived workload, decision accuracy, data verification, and successful use of interactive features to be examined. Rather than introducing entirely new evaluation methods, the proposed framework suggests adapting and combining established usability and UX methods to better address the particular ways users interact with and interpret open-data visualizations. For example, usability testing could be adapted to examine data exploration and interpretation tasks, workload measures could be incorporated when evaluating cognitively demanding interactions, and visualization or accessibility inspections could be combined with user-based evaluation. These examples are intended as potential operationalizations of the proposed dimensions rather than as definitive or validated measurement procedures. Therefore, the framework is intended as a starting point for future research to further develop and validate these dimensions and their potential operationalization across different ODVS contexts.

Finally, this study provides a preliminary conceptual basis for addressing the identified research gap in the evaluation of specific platforms such as ODVS, which aim to provide transparent, accessible, and interactive insights into public datasets for audiences with different levels of education, abilities, and knowledge. The proposed framework connects the broader research trends identified in the corpus with the specific evaluation needs of open-data visualization environments, while providing a basis for future research to further develop and validate these evaluation dimensions and their potential operationalization. It also offers a starting point for exploring how established usability and UX methods can be adapted and combined to address the particular characteristics of ODVS, particularly the human and cognitive factors involved in data interpretation, sense-making, and decision-making. In this way, the framework establishes a research direction for human–computer interaction in increasingly dynamic and multidimensional data environments.