How Well Do Current Geoportals Support Geodata Discovery? An Empirical Study

Abstract

Implementing effective geospatial data discovery mechanisms in geoportals is crucial for facilitating easy access to geospatial data and services. Despite existing efforts to formulate geoportal design requirements, understanding end-user issues beyond a single geoportal in the context of geodata discovery is still lacking. To address this gap, this study reports on a usability study conducted in Germany and Namibia, with the aim of examining issues faced by users during geodata search and discovery. The study employed a mixed-method approach combining Retrospective Think-Aloud (RTA) interviews and structured questionnaires. The results reveal key usability issues, including inefficient search mechanisms, inefficient presentation of search results, lack of user guidance, inefficient map interactions, and inefficient metadata descriptions. Additionally, the study revealed a difference in user perceptions regarding user experience aspects between the two user groups. The findings are of interest to the designers of geoportals in the context of open data reuse and spatial data infrastructure.

1. Introduction

The increasing availability of online geospatial information resources and the ever-increasing need to facilitate their discovery has led to a proliferation of geoportals, which have become a key component of the spatial data infrastructure landscape. A geoportal has been defined as an entry point for geospatial data and service discovery. It refers to a web application that acts as a one-stop-shop to access geospatial services from multiple sources over a network [1,2]. Typical functionalities offered by geoportals include: data search, cartographic visualizations (to inspect published geoinformation), metadata publication, and quality assurance (review/approval of metadata submitted for publication) [3].

Geodata search and discovery involve locating, accessing, and retrieving geospatial datasets through digital platforms such as geoportals. These processes rely on metadata, search filters, and visualization tools to help users find relevant spatial data efficiently [4]. Ensuring that geoportals implement effective and user-friendly data discovery and retrieval mechanisms is essential for facilitating seamless access to geospatial data and geoinformation (GI) services. In GIScience, efficient geodata discovery is critical for researchers, policymakers, and decision-makers interested in the reuse of open (geographic) data. Also, geoportals are used by a wide range of users with diverse levels of domain and technical expertise, experience and familiarity, expectations and possibly diverse cultural backgrounds, which should be taken into consideration during their design and evaluation. Empirical studies about geoportals are useful for providing a snapshot of key issues faced by users during geodata search. Hence, these studies are the first step in synthesizing the key needs of geodata users (see e.g., [5]) and in formulating evidence-based guidelines for the design of geoportals. These guidelines, in turn, are vital to increase the chances of portals’ adoption after deployment (benefit for data providers), as well as their efficiency and enjoyability during use (benefit for data users).

There have been attempts to synthesize the literature on geoportals [4,6], formulate requirements for the design of user experiences on geoportals (e.g., [7,8]), and evaluate the usability and user experience of geoportals [9,10]. Nonetheless, an understanding of typical end-user issues beyond single geoportals in the context of geospatial data discovery is still lacking. The current article explores these issues to provide broader insights into geodata discovery practices and enhance the understanding of user experience during geodata search in geoportals. Evaluating multiple geoportals across diverse user groups is important for at least two reasons. First, it captures variations in user experience and system performance beyond the limitations of a single geoportal or user group. Second, a cross-geoportal usability study enhances the generalizability of usability findings, making them more applicable to the broader field of GIScience.

This work conducted a usability evaluation study in Germany and Namibia that examines how existing geoportals facilitate geodata search. Contrary to previous work that tested the usability of single geoportals (e.g., [11,12,13]), our goal is to empirically learn about usability issues across geoportals and between user groups. The research question examined is: How well do geoportals support geodata discovery? The study was carried out using a combination of usability evaluation methods, namely, reflective think-aloud (RTA) interviews and structured to semi-structured questionnaires. RTA is a method used to identify usability problems where participants verbalize their thoughts about the interaction experience after the completion of a task [14]. It provides the moderator with the ability to ask and obtain more information regarding the thoughts and actions of the users during the task. The questionnaires were used to collect user ratings about different aspects of the user experience (UX) of geoportals: perceptions of search effectiveness, complexity of the user interface, and pragmatic and hedonic user experience, as described by [15]. Overall, the study looked into the following four questions:

• Q1: What are the usability problems currently experienced by users during geodata discovery and access in geoportals?
• Q2: What is the performance of LB and LBM displays, respectively? (LB: list-based only display of search results; LBM: list-based+map display of results)
• Q3: Does performance and the perceptions of user experience change according to the type of query scenario, i.e., ST vs. STT? (ST: spatio-thematic queries; STT: spatio-temporal-thematic queries)
• Q4: What are the differences in user experience perception between the two user groups examined (i.e., Namibian users vs. German users)?

The following section provides more details on the state of the research about the usability of geodata discovery in geoportals (Section 2), followed by the methods section outlining the experiment design and materials used (Section 3). Thereafter, Section 4 summarizes the study results. Section 5 synthesizes the main findings and research implications, while Section 6 concludes the article.

2. Background

Since this study aims to assess the usability of geoportals in the context of geodata discovery, this section provides a brief review of previous work on geoportals and geoportal usability.

2.1. Geoportals in SDIs: Importance and Usability Considerations

Geoportals are essential components of spatial data infrastructures (SDIs), serving as central hubs for accessing, discovering, and sharing geospatial data. Ensuring that geoportals provide intuitive navigation, efficient search mechanisms, and meaningful data visualization is crucial to maximize the impact of SDIs and to promote open geospatial data use [2,16]. A well-designed geoportal facilitates seamless data accessibility, usability, and interoperability, enabling researchers, government agencies, businesses, and the public to make informed decisions based on geospatial data [1,16]. However, poor usability, inefficient search functions, complex user interfaces, and inadequate metadata presentation can hinder data discovery [11] and discourage adoption. Though there are various definitions of usability (for examples, see [17,18]), there is general consensus that ‘usability’ has been often used to refer to instrumental aspects of the interaction with products (e.g., efficiency, effectiveness), whereas the cognate notion of user experience is a broader concept that covers both instrumental and non-instrumental aspects (e.g., stimulation, appeal, enjoyment, aesthetics, …), see [19,20]. In this regard, Resch and Zimmer [8] went beyond usability in their synthesis of requirements for map-based geo-portals, emphasizing key user experience (UX) design principles. These principles highlight essential UI components such as search functions, map interfaces, symbolization, interaction mechanisms, and UI aesthetics. A preliminary framework for assessing the user experience of geoportals was suggested in [21]. In line with these ideas, the current work examines the user experience during the discovery task in geoportals (see Section 3).

We are witnessing a rise in geospatial data discovery applications on the web, with differences in scope, functionalities, appearances, and usability status. Previous research highlights these disparities, often attributed to the lack of a formal definition of geoportal [4,8,22] and the absence of standardized design guidelines for such applications [23]. Different geoportal providers have unique requirements that contribute to this heterogeneity. Furthermore, research about geoportals still lacks a standardized classification system along with proper formalizations of the distinguishing characteristics of portals’ types. Various studies have proposed different classification methods. For instance, Maguire and Longley [24] categorized geoportals into catalog portals and application portals, while Jie [25] extended this classification to include enterprise portals. Various taxonomies may offer a variety of perspectives on geoportals. Nonetheless, the absence of formal operationalisations of the nuances of the concept may hinder the accumulation of knowledge across studies in GIScience, as that accumulation needs similar conceptualizations of the ‘geoportal’ concept as a starting point.

Kellenberger et al. [26] defined a guiding functional requirement for an effective geodata search process in a geoportal, namely that a user should visually be able to navigate the geodata spatially, thematically, and temporally and be able to select datasets and area, as well as download their preferred data directly. These functionalities are integral to geodata discovery, and our study adopts these requirements as a baseline for reviewing and selecting geoportal interfaces to include in the study. Furthermore, from a functional perspective, effective geodata discovery relies on more than just UI design; metadata quality, search functionality, and user interaction mechanisms also play a crucial role. Maguire and Longley [24] emphasize that geoportals facilitate querying metadata records, while Kalantari et al. [27] highlight how metadata design directly impacts data discovery outcomes. Interactive web maps, in particular, improve geodata exploration by supporting visual thinking of the user, aiding in the interpretation of large data sets, and improving real-time data integration in SDIs [28,29,30]. However, many geoportals still lack narrative, visual, and interactive elements, which limits user engagement and comprehension during geodata searches [30,31].

2.2. Usability Evaluation of Geoportals

Usability evaluation studies have consistently identified design flaws in geoportal interfaces as reported in [23]. However, most of these studies focus on single interfaces within specific user groups, while relying on traditional usability metrics such as task completion rates (e.g., [11,13,32]). These studies arguably provide insight into how the geoportals assessed meet the needs of their target users (e.g., [13,33]), but say little about the extent to which these insights apply to other geoportals. Our research diverges from this norm by evaluating multiple interfaces in one study and employing qualitative and quantitative methods, including Retrospective Think-Aloud (RTA) protocols and questionnaires [13].

Additionally, notable work has been made to advance the methods of evaluating usability in geoportals and other web map interfaces [34], and developing frameworks to provide generic solutions to usability problems [22]. Even so, usability challenges in geoportals are reportedly coming from the continuity of data updating, methods of data harmonization, and multiple functionalities for professional users [4]. A study by Vahidnia [35] revealed that the integration of task-oriented chatbots in geoportals can significantly improve geodata retrieval by reducing the time required to access the data. Quiñones et al. [7] have developed heuristics for evaluating the usability of geoportals. Still, a gap often remains between user skills, needs, and expectations, and the functionalities offered by geoportals. Empirical studies that examine this gap are needed to advance our understanding of the nature of the problems and represent the first step in formulating design guidelines for geoportals. However, such studies are still largely lacking. Put differently, a comprehensive examination of geoportal search functionality has not been conducted between user groups [36]. Our study conducts a cross-portal investigation to address this gap.

2.3. Summary

In summary, though previous studies have offered valuable insight through the evaluation of single geoportals, these come with the limitation that generalization beyond the settings of the portal assessed is limited. Empirically derived guidelines for geoportal design should be informed by issues that are not unique to a single portal but are common across multiple portals. A few studies using several open data portals and geoportals to compile issues faced by users during open data reuse exist. Examples include Degbelo et al. [37], who used a brainstorming workshop to collect insights about interaction with multiple thematic layers in geoportals, and Benitez-Paez et al. [38], who collected general issues about data reuse in geoportals through participatory workshops. What we currently lack, though, is an in-depth analysis of specific issues encountered during geodata search tasks. Since one of the goals of geoportals is to provide one-stop access to geographic information resources, understanding search-related issues is particularly important to uncover (current) impediments to reaching that goal. The current study contributes to shedding light on these issues through an examination across diverse geoportals.

3. Materials and Methods

The study consists of two evaluation studies. The first was conducted in Dresden, Germany, in July 2023 and was subsequently replicated in Windhoek, Namibia, in September 2023. As mentioned in Section 1, the work aims to empirically assess how well geoportals currently support discovery, with a focus on user experience aspects. At the most abstract level, the theoretical assumption underlying this work is a hypothesis (H) about three constructs, namely that user characteristics may affect their cognitive style, which in turn may affect their experience of the geoportal (see Figure 1). Several aspects of user characteristics were considered in this work: country of location/origin, computer skills, spatial analysis skills, programming skills, map-making skills, geodata search experience, UI familiarity/experience and web-map use skills. The concepts of user experience and geoportal can be conceptualized in various ways, hence we provide here a brief clarification of the conceptual stance adopted for this study. The concept of cognitive style is also briefly introduced below.

User experience has several dimensions, five of which are considered during the work (Figure 1): performance (i.e., search effectiveness), usability problems, user interface complexity, pragmatic user experience, and hedonic user experience. Additionally, we have mentioned above that the geoportal can be conceptualized in many ways. We used the following criteria during this study: websites that (1) act as an entrance point or gateway to a collection of geographic information resources; (2) support text-based search, and (3) present search results either using a list-based format or a list-based format along with a map. Criterion (1) is in line with definitions suggested previously in the literature (e.g., [3,8,24,39]); criteria (2) and (3) are in line with the concrete characteristics of interest in the present study. We have explicitly controlled for three independent variables to nuance the results according to these dimensions (Figure 1): geodata search tasks, the types of displays used to visualize the search results, and the participant’s country of origin. We also collected data about seven subject variables as shown in Figure 1.

Throughout the work, we focus on user perception while conducting geodata search tasks. Perception is only one aspect of the broader notion of ‘cognitive style’, defined as “self-consistent characteristic modes of perceiving, remembering, thinking, and problem-solving” [40] or “consistent individual differences in ways of organizing and processing information and experience” [40]. In the long run, shedding light on these ‘consistent individual differences’ would be particularly useful for informing the development of guidelines for geoportal design. By focusing on perception, our work provides a starting point to build upon. Cognitive style has the status of a latent variable [41] because it may not be directly measured but can be examined indirectly through related observable variables (e.g., perception, memorability, problem-solving ability). The dependent and independent variables shown in Figure 1 were selected to keep a manageable complexity for the study during this first exploration. More variables, for instance, more aspects of the users’ background (e.g., demographic details) and more cognitive style dimensions (e.g., information processing, information organization, problem-solving) could be included in subsequent studies. The research began by reviewing and selecting suitable geoportals for the study, followed by the definition of user tasks, as discussed in the following sections.

3.1. Geoportal Selection

To ensure a diverse selection of geoportal interfaces for use as stimuli throughout the study, an exploratory analysis was conducted by examining openly accessible geoportals on the internet. A total of 50 geoportals were randomly selected and reviewed. Of these, five were discarded due to inaccessibility and log-in restrictions. The remaining 45 geoportals were further surveyed according to the dimension of the type of informational features used to display search results (i.e., list-based display and list-based + map display), language, and spatial coverage. The list-based displays (LB) present the search results in a list form, while list-based + map (LBM) displays present the results in list form along with an additional map interface to browse through the results. In the case of LBM, some lists are interlinked with the map interface, whereby when the user hovers or clicks on a result on the list, the map display interactively changes to the result’s location or simply provides the result preview. To ensure a balanced representation of LB and LBM portals during the study, we aimed for 4 × LB and 4 × LBM displays. We also considered only the geoportals in English or provided an English version. Similarly, the spatial coverage was important during the selection as we aimed to ensure diversity of geographic scope in the sample. Eight geoportals were selected from the survey list to meet both the objectives of balanced representation of display types and diversity (see

Table 1. A list of geoportals selected for the study and their classification accordingly. All portals were last accessed in June 2025.

Geoportal	Query Type	Display Type	Coverage
US Data.gov [42]	ST	List-based	Global
Irish Spatial Data Exchange [43]	ST	List-based	Local
SASSCAL Data and Information Portal [44]	ST	List-based + map	Regional
Humanitarian Data Exchange [45]	ST	List-based + map	Global
GEOSS [46]	SST	List-based + map	Global
BTAA [47]	SST	List-based + map	Global
Lithuania Geoportal [48]	SST	List-based	Local
European Geological Data Infrastructure [49]	SST	List-based	Regional

). Figure 2 shows selected screenshots of user interfaces of the geoportals at the time of the study.

3.2. User Task Creation

The study design is based on a task-oriented approach. Participants received a scenario and a geodata-search task to perform on a geoportal and achieve specified goals. The study was designed and conducted in English in both setups.

We created four types of geodata search tasks by varying attributes of the search process along two dimensions: the type of primitive geodata search queries (i.e., ‘datasets about space-theme (ST)’ vs. ‘datasets about space-time-theme (STT)’) and the informational features used to display the search results (i.e., list-based only vs. list-based + map-based). The two dimensions were taken from the literature on geodata reuse [50] and search user interface design [51], respectively. An example of a search task is shown below.

3.3. Participant Recruitment

According to [8], a typical geoportal user has considerable experience and understanding in using maps and geodata search functionalities. Also, the user knows how to transfer geodata from one system to another to support their tasks. With this understanding, the targeted participants were required to have a background in Geographic Information Science (GIS), Cartography, Remote Sensing, or related fields to make sure the participants understood basic geoinformation terminologies. Most importantly, they understand what geodata is and how it is used. Due to this, we targeted students and professionals within these disciplines. Participants were required to be of German origin (study in Dresden) or Namibian origin (study in Windhoek), respectively. Moreover, the participants were to be 18 and above. The recruitment was done through advertisement at TU Dresden and the Namibia University of Science and Technology, and on a referral basis. An invitation with the specified requirements was extended to the students and potential participants by email. A total of 32 participants took part (16x Germans and 16x Namibians).

In the first study, there were a total of 8 students, 5 researchers, and 3 professional employees from private companies. The age ranges from 18 to 44 years where 13 are male and 3 females. Most of them come from GIS and Remote Sensing, and Forestry backgrounds. While in the second study, 12 students, 2 professional employees from a private company, 1 researcher, and 1 government authority employee. Their age ranges from 18 to 34 years, with a domain background in GIS and Remote Sensing, Urban and Regional Planning, and Geography. Each participant received an incentive of EUR 10 in cash after completion of the task.

3.4. Study Procedure

The usability experiment was conducted in a lab setup on campus in both studies. Each study session consisted of an interviewer and one participant and the study was conducted in the following order:

• Introduction: Firstly, the researcher welcomed the participants and introduced them to the experiment, giving them instructions before they could provide their consent to participate in the study. They were then given a brief introduction to familiarize themselves with the requirements of the search task on a geoportal interface. Thereafter, they were given the consent and user background forms to fill out and begin the task.
• Task: As illustrated in Figure 1, each participant began with the task by searching for geodata. After the participants performed the search task, they provided feedback on search effectiveness (relevance rating of the first 5 search results on a scale from 1 to 10), perceived user interface complexity (on a scale from 1 to 10), and overall user experience (through items inspired from the short version of the user experience questionnaire; see [52]). We only asked participants to rate the first 5 search results to minimize the cognitive load during the task and participants’ fatigue during the study overall. Each participant was randomly allocated one search task on one geoportal.
• Interview: Subsequently, the screen recording of their search task was played retrospectively to collect qualitative feedback regarding their actions in a Retrospective Think Aloud (RTA) session. When necessary, follow up questions were asked according to the participants’ responses.

One participant session lasted for approximately 45 min.

3.5. Data Analysis

Our research study is exploratory in nature, employing a mixture of qualitative and quantitative analysis methods. The quantitative data was analyzed through correlation analysis, linear modeling, and bootstrapping. We computed both the discounted cumulative gain (DCG and the normalized discounted cumulative gain (nDCG, [53]) at positions 3 and 5 respectively, as measures of ranking quality.

The qualitative data analysis began with data pre-processing by transcribing the audio recording into textual transcripts. The next step was holistic coding (see [54]), which was performed manually by one researcher through an iterative review of all transcripts. In the first round of coding, broader themes were identified to capture common and recurring usability-related issues. In the subsequent rounds, the transcripts were reviewed again, themes were refined, merged, or reconstructed accordingly. This iterative, multi-round process served as a procedure to ensure coding consistency. The themes for usability problems and the classes were inspired by [23,51,55]. A usability problem in this study is defined as any challenge, obstruction, or confusion experienced by participants while performing search tasks. Here, we draw a basic distinction between usability challenges related to the search user interface, and usability challenges related to the content (data and metadata). The usability problems were then grouped according to the following classes:

Search User Interface

• Input features: Usability issues related to how users formulate and enter queries in the search tool.
• Informational features: Usability problems relating to how search results are organized and displayed to support relevance evaluation.
• Control features: Usability problems relating to how the search continues after the formulation of the search query. They may arise at query refinements, filtering, corrections, etc.
• Search effectiveness: Search effectiveness is a dependent variable being examined in the study. Usability problems under this theme pertain to the search process not yielding a satisfactory search experience.
• User guidance: Usability problems relating to the lack of or ineffective system feedback and user guidance. Problems relating to UI documentation and help features were also assigned to this category.
• Intuitiveness and design: Refers to usability problems relating to UI design and ease of recognition of needed features.
• Navigation: usability problems relating to navigation and interaction design aspects.
• Map: Usability problems relating to the use of map interface during the search and visualization of geodata.

Content

• (Meta)data quality: Refers to usability problems about the quality, currency, and language of metadata content that is hindering the effective identification of geodata.
• (Meta)data quantity: usability problems related to content insufficiency.

Although this study is not explicitly designed as a cross-cultural evaluation, it examines usability across two diverse user groups, allowing us to explore regional differences in user experience perception. In addition to factors such as domain knowledge, technical skills, and familiarity with geoportal interfaces, we briefly consider cultural theories to uncover further insights. Specifically, we draw from Hofstede’s cultural theory [56] of power distance (PD) and Marcus and Gould [57]’s framework, which translates cultural dimensions into user interface design implications. Power distance in this context denotes ‘the extent to which the less powerful members of organizations and institutions accept and expect that power is distributed unequally’ [56].

Marcus [58]’s model of user interfaces comprises five UI design dimensions: Metaphor, Mental Model, Navigation, Interaction, and Presentation of Information. According to Hofstede’s 2023 Index [59], Germany is classified as a low-power-distance country (score: 35), while Namibia is categorized as a high-power-distance country (score: 65) Our study is interested in the dimension of the mental model, which concerns how data, functions, and tasks are organized in the user interface. According to [57,60], high-PD cultures would tend to favor complex, structured, and hierarchical information organization, while low-PD cultures would prefer simpler, informally structured, and relevance-based presentations. What makes these groups cultural is that their members share common values, beliefs, histories, and meaningful activities [56,61]. Based on this, we hypothesized that Namibian participants would show a stronger preference for complex and highly structured information compared to their German counterparts (H1).

4. Results

We now present the study results according to the five dependent variables in turn. Since we are interested in how well geoportals support search in general, we first document the results according to the results display type (list-based vs. list-based+map) and the query scenario (spatio-thematic vs. spatio-temporal-thematic). The confidence interval (CI) values were computed using the bootES package in R [62] and stand for 95% confidence intervals. Confidence intervals for the difference of two means that do not contain zero indicate statistical significance. The narrower the interval, the stronger the evidence.

4.1. Search Effectiveness

Here we report on the DCG and nDCG values, with both playing distinct roles in the study. The nDCG values are easier to interpret and will be used as proxies to discuss the quality of the order in which the search results are presented. We document the DCG values because they can serve as a baseline for assessing efficient ranking algorithms in the context of geoportals in future studies.

The average DCG@3 value across the 32 participants was 12.3 (CI: [10.3, 14.1]) while the average DCG@5 value was 16.8 (CI: 14.4, 19.7). The values for the nDCG@3 and nDCG@5, averaged across the 32 participants were 0.94 (CI: [0.89, 0.96]) and 0.90 (CI: [0.85, 0.93]), respectively. The differences between the display types, the query scenarios, and across user groups were not significant as well (

Table 2. List of all ratings obtained during the experiment. Significant differences are highlighted in green in the table. User interface (UI) complexity ratings are on a Likert scale from 1 to 10. Pragmatic and Hedonic values of user experience (UX) ratings are on a Likert scale from −3 to 3 (−3 to −1 = negative ratings, 0 = neutral, and 1 to 3 = positive ratings). LB: list-based displays; LBM: list-based + map displays; ST: space-theme query; STT: space-time-theme query.

	Display Type		Query Scenario		User Groups
	LB	LBM	ST	STT	Namibian	German
Relevance Ratings
DCG@3	12.1; [9.8, 14.2]	12.5; [9.0, 15.5]	12.1; [8.8, 15.3]	12.5; [10.2, 14.6]	13.4; [10.5, 16.1]	11.2; [8.4, 13.7]
DCG@5	16.7; [14.4, 19.6]	16.3; [12.2, 20.1]	16.9; [13.0, 20.5]	16.3; [13.4, 19.1]	18.2; [14.4, 21.1]	14.9; [12.0, 18.1]
nDCG@3	0.93; [0.86, 0.97]	0.93; [0.88, 0.98]	0.95; [0.85, 0.98]	0.93; [0.87, 0.97]	0.94; [0.89, 0.97]	0.93; [0.85, 0.97]
nDCG@5	0.90; [0.82, 0.94]	0.90; [0.84, 0.94]	0.89; [0.79, 0.94]	0.91; [0.87, 0.94]	0.91; [0.84, 0.95]	0.89; [0.81, 0.94]
UI Complexity	5.5; [4.1, 6.5]	4.6; [3.6, 5.8]	5.7; [4.4, 7.0]	4.4; [3.4, 5.4]	5.9; [4.8, 7.0]	4.2; [3.1, 5.3]
UX-Pragmatic
SupportiveObstructive	1.4; [0.3, 2.1]	0.44; [−0.6, 1.3]	0.9; [−0.3, 1.7]	0.9; [−0.1, 1.7]	1.1; [0.4, 1.6]	0.7; [−0.6, 1.7]
EasyComplicated	−0.1; [−1.1, 0.7]	0.8; [−0.5, 1.8]	0.1; [−1.1, 0.9]	0.5; [−0.6, 1.5]	0.4; [−0.6, 1.3]	0.2; [−1.0, 1.5]
EfficientInefficient	0.9; [−0.1, 1.6]	0.7; [−0.3, 1.6]	0.7; [−0.3, 1.5]	0.9; [−0.1, 1.7]	1.1; [0.1, 1.8]	0.5; [−0.5, 1.3]
ClearConfusing	0.9; [−0.3, 1.8]	0.9; [−0.3, 1.9]	0.7; [−0.5, 1.7]	1.1; [−0.1, 1.9]	0.7; [−0.4, 1.5]	1.1; [−0.2, 2.1]
UX-Hedonic
ExcitingBoring	0.6; [−0.1, 1.3]	0.3; [−0.7, 1.2]	0.0; [−0.7, 0.7]	0.9; [−0.1, 1.6]	0.9; [0.1, 1.6]	0.0; [−0.8, 0.9]
InterestingNotinteresting	1.3; [0.4, 1.9]	0.5; [−0.4, 1.3]	0.9; [0.1, 1.5]	0.9; [0.0, 1.6]	1.1; [0.6, 1.6]	0.7; [−0.3, 1.6]
InventiveConventional	0.2; [−0.8, 1.0]	−0.4; [−1.3, 0.4]	−0.4; [−1.3, 0.4]	0.2; [−0.6, 0.9]	−0.2; [−0.9, 0.5]	−0.1; [−0.9, 0.9]
LeadingedgeStandard	−0.8; [−1.8, 0.3]	−1.9; [−2.5, −1.3]	−1.5; [−2.3, −0.5]	−1.2; [−2.1, −0.2]	−0.7; [−1.5, 0.3]	−2.0; [−2.7, −1.1]

). The nDCG scores are close to 1, which indicates that the ordering of the results in the geoportals examined were close to the ideal ordering, overall. To put the DCG values into context, we can compare them to DCG@3 and DCG@5 values computed from a previous study [63] on the effectiveness of search strategies in the context of open government data because the authors used a similar scale (1–10) to ours to collect the relevance ratings. The focus of their study was on ST queries, and the DCG@3 values were between 14.4 and 17.6, while the DCG@5 values were between 18.6 and 22.6. Our DCG values fall slightly lower at 12.1 (DCG@3) and 16.9 (DCG@5), respectively, which could hint at room for improvement in the portals investigated. The differences in DCG values may also be because the framing of the query scenarios handed over to participants was different in the two studies.

4.2. Perceived User Interface Complexity

UI complexity refers to how difficult or intricate a UI feels to the end user. It is a subjective measure influenced by factors such as layout, navigation, visual clutter, and the cognitive effort required to complete tasks [64]. An interface may be technically simple, meaning that it has a straightforward structure or minimal functionality but still be perceived as complex if it is poorly designed, confusing, or unintuitive. Across the two studies, the average complexity rating was 5.0 (CI: [4.2, 5.8]), indicating that all participants perceived the geoportals as neither too easy nor too complex. The ratings did not differ significantly across display types and query scenarios (Table 2). However, there was a significant difference between the two user groups (mean difference: −1.7, CI: [−3.3, −0.2]), which indicates that the Germans perceived the geoportal UIs as slightly less complex than the Namibians (Figure 3a). Also, the data suggests a significant relationship (Correlation coefficient: −0.48, p = 0.005) between web map use skills and UI complexity ratings (Figure 4).

4.3. User Experience

Here, we considered the ratings provided for all eight dimensions proposed by the short version of the user experience questionnaire. The first four dimensions touch upon pragmatic user experience, whereas the last four touch upon hedonic user experience. As discussed in [20], a product can be perceived as pragmatic because it provides effective and efficient ways to achieve behavioral goals (usability); it can be perceived as hedonic because it is engaging, aesthetically pleasing and stimulating due to its challenging and novel character (stimulation function); or it can provide identification by communicating important personal values to others (social function).

Pragmatic user experience: across all 32 participants, the average ratings obtained were 0.9 (CI: [0.2, 1.5]; SupportiveObstructive), 0.3 (CI: [−0.4, 1.0]; EasyComplicated), 0.8 (CI: [0.1, 1.3]; EfficientInefficient), and 0.9 (CI: [0.1, 1.6]; ClearConfusing). All these values may be seen as neutral values, which means that the geoportals selected were seen as neither too effective nor too ineffective for the completion of the tasks. The differences across the conditions were not significant (Table 2).

Hedonic user experience: The average scores of the 32 participants were 0.6 (CI: [−0.2, 1.0]; ExcitingBoring), 0.9 (CI: [0.3, 1.4]; InterestingNotinteresting), −0.1 (CI: [−0.7, 0.4]; InventiveConventional) and −1.3 (CI: [−1.9, −0.7]; LeadingedgeStandard). That is, the geoportals were perceived by the participants as rather novel, even though they did not raise any particular excitement. Significant differences were observed for the LeadingEdge vs. standard dimension; LB vs. LBM (−1.188, CI: [−2.5, −0.1]), which suggests that interfaces, where maps were used to display the results, were perceived as more ‘novel’ than those were maps were not used (Figure 3c); and German vs. Namibian (−1.312, CI: [−2.5, −0.2]), which suggests that the German participants perceived the interfaces as more ‘novel’ than the Namibian participants overall (Figure 3b).

The interfaces were generally rated as supportive, efficient, and clear since the CI values are mostly positive. However, there were mixed responses on ease of use (EasyComplicated), as the confidence interval included both positive and negative values. The relatively small CI ranges suggest moderate confidence in these findings, although variability in responses (especially in the “EasyComplicated” scale) should be noted.

4.4. Interaction Effects

We also examined interaction effects between the type of interface display and the user groups, the type of display and the query scenario, and the query scenario and user groups for all the dependent variables listed above. There were no simultaneous effects of the independent variables in most cases, except for the following three instances.

• EasyComplicated: Evidence of an interaction between user groups and query scenario (significance level: 0.05, p = 0.03), while the Namibian participants found the STT query scenarios to be more complicated than the ST queries, the reverse was the case for the German participants (Figure 5a).
• SupportiveObstructive: Weak evidence of an interaction between user groups and display type (significance level: 0.10, p = 0.085); while the Germans found LB displays to be more obstructive than supportive for the tasks, the Namibians rated LBM displays as slightly more obstructive than supportive (Figure 5b).
• LeadingEdgeStandard: Weak evidence of an interaction between display type and query scenario (significance level: 0.10, p = 0.058); in both query scenarios, LBM displays were rated as more leading edge than standard. However, the difference in ratings by participants who completed STT scenarios is much higher than the difference in ratings by participants performing ST query scenarios (Figure 5c).

4.5. Usability Problems

A total of 92 usability problems were extracted from the RTA interview data. The majority (88 percent) of these issues were related to the search UI, while 12 percent were associated with content-related aspects (see

Table 3. Number of usability problems per category. Search user interface issues relate to design aspects of the geoportals’ user interfaces, while content issues relate to the characteristics of the data/metadata displayed.

Categories	Number of Problems	Percentage
Search User Interface	80	86
Input features	22
Informational features	10
Control features	9
Search effectiveness	8
User guidance	9
Intuitiveness and design	8
Navigation	3
Map	10
Content	12	14
Meta(data) quality	8
Meta(data) quantity	4
Total	92	100

). A list of all usability problems can be found in Appendix A. The table below provides a breakdown of the number of usability problems per category as explained in methods section.

The usability problems reported the most frequently in both studies were related to the input characteristics (22), the information characteristics (10), and the map (10). Subsequently, categories of control features, user guidance, and meta(data) quality took the second rank of frequent usability problems recorded as depicted in Table 3. Below, we describe the usability problems according to the top 3 categories with the most frequent problems:

• Input Features: Participants struggled with search query formulation (P5, P6, P10, P17, P20, P21, P25, P27, P30, P31, P32). The search interface was difficult to locate at times (P10, P17, P30, P32), with poor positioning and small input fields reducing accessibility in some geoportals. Date-based filtering was unreliable (P17, P30), often failing to update results correctly (P27) or showing inconsistencies between calendar dates and timeline information (P5). Searches returned no results without clear explanations at times (P20, P21, P25, P27, P31), frustrating participants and making them question whether the issue lay with their queries or the data itself (P25). Searching by defining locations using bounding boxes was also confusing (P17, P27) and further complicated the search process. The complexity of the search interfaces and lack of intuitive guidance (P31) also made it challenging to formulate effective search queries.
• Informational Features: One of the main difficulties was assessing the relevance of search results due to unclear metadata descriptions and insufficient search result details (P2, P10, P12, P23, P31). Participants were required to manually sift through multiple results to find relevant data in some geoportals, as filtering and ranking mechanisms were inadequate. Additionally, the participants encountered further challenges, such as search results frequently refreshing during the search process or search history interfering with their queries (P24, P26). Usability problems relating to the presentation of search results hindered the user experience, making geodata discovery time-consuming and cumbersome.
• Map: participants faced several challenges while interacting with the map, including the inability to enlarge it for better navigation (P3). The maps lacked essential functionalities such as user-controlled visualization (a lack of control over how participants interact with the map) and a lack of varying base map options (P26). Data visualization was problematic in some geoportals (P21, P24, P26), and unclear instructions made the map search functionality difficult to use (P26). Additionally, participants struggled to understand the purpose of the map in the search process and were uncertain about how to use it (P2, P10). These usability issues led to confusion and an overall ineffective user experience.

Usability problems of concern were also recorded for additional categories, such as user guidance issues during searching (P2, P19, P27) and navigating the geoportal interfaces (P7, P27, P30). Additionally, participants expressed dissatisfaction due to poor search effectiveness (P1, P7, P21, P25, P26, P31). They found the geodata search process too difficult and complex. Finally, meta(data) quality and quantity problems were reported (P1, P3, P6, P10, P11, P12, P16, P21, P24), especially concerning vague dataset titles, inconsistent data descriptions, or insufficient information, making it difficult for the participants to assess the relevance of the provided data confidently. This section has presented a summary of the issues, referencing the users who mentioned them. A summary referencing the geoportals instead is available in Appendix B.

5. Discussion

5.1. Takeaways

In this section, we synthesize the main takeaways from the study results in the context of the research questions as follows:

5.1.1. Q1: Common Usability Problems

The findings indicate that usability problems are still persistent in geoportals, most especially at the Search UI level. In many cases, participants struggled to recognize the provided search functionalities and also to figure out how to formulate an effective query, especially for spatio-temporal queries. To say the least, search user guidance is lacking in geoportals. Additionally, search effectiveness is a concern in regards to the performance of search functions in retrieving relevant data as hinted in Section 4.1. Therefore, geoportals need to improve how search results are presented to allow easy evaluation of the results by users. Also, usability problems relating to meta(data) quality are persistent in geoportals, where language jargon, inefficient description of data or titles make it difficult for the users to understand the data. Some usability problems arise because users are not familiar with the interfaces; this means that geoportals are still lacking intuitiveness.

Overall, participants rated the interfaces as supportive, efficient, and clear with mixed responses in Section 4.2. However, the qualitative data indicate that participants found the search functionalities unintuitive, difficult to navigate, and lacking guidance, highlighting room for improvement regarding the interface designs and the search mechanisms to enhance usability.

5.1.2. Q2: Types of Geoportal Displays

The informational features used to present search results are crucial for evaluating their fitness for use by users. The study did not find a significant quantitative difference in most of the measured variables between the two display types. For instance, the UI complexity ratings suggest that list-based displays may be perceived as more complex than list-based+map displays, with a mean difference of 0.9 (see Table 2). This suggests that the perceived complexity may vary among users, requiring a larger sample size to draw definitive conclusions. Additionally, list-based+map displays exhibited more usability issues related to input features, user guidance, and map interactions. This suggests that while users struggled with search functionality in both display types, interaction-related difficulties were more pronounced in LBM displays (Appendix A).

Finally, all usability problems related to metadata quantity were observed in list-based displays, while both display types recorded issues related to metadata quality.

5.1.3. Q3: Types of Query Scenarios

We evaluated how geoportals support queries along two search query dimensions: Spatio-Thematic (ST) and Spatio-Temporal–Thematic (STT) queries. The quantitative measures did not reveal statistically significant differences between the two query types. However, qualitative insights suggest that geoportals still lack the capabilities to effectively handle spatio-temporal queries. Many participants struggled to define the temporal term in STT queries, leading to usability issues. While some geoportals provided time sliders for refining searches within a specified period, these were often unresponsive or ineffective, contributing to user frustration. The absence of clear user guidance further increased confusion during STT queries. Despite these challenges, STT queries were perceived as slightly more “Leading Edge” than ST queries, though this difference was not statistically significant (see Table 2). In contrast, ST queries were associated with fewer usability problems, particularly regarding input features such as locating the search functions. The proportion and nature of usability problems suggest that ST queries seemed to have posed more issues to users than STT queries. However, unexpectedly, geoportals evaluated with STT queries were rated as simpler than those evaluated with ST queries.

Ultimately, query type influenced usability outcomes differently across measures. Although effectiveness and usability problems suggested that ST queries were easier to handle, perceptions of UI complexity and efficiency ratings varied between user groups. This highlights the need for improved interface support for spatio-temporal queries, particularly regarding user guidance and functional time-based search tools.

5.1.4. Q4: Differences in User Groups

Finding 1—Similar problems, different emphases. Although the usability problems encountered in the two studies do not differ much (Section 4.5), the overall usability concerns and emphasis were slightly different. The German participants were mainly concerned about the availability and quality of relevant data, as well as the possibility of evaluating the search results for fitness for use. By contrast, the Namibian participants were mostly concerned about user guidance and support features and familiarity with the UI designs and features. Specifically, in the input feature category we observed that the Namibian participants were more concerned about getting support on defining search queries or assurance that they are defining the queries as expected, while the German participants were concerned about the efficiency of the search functions.

Finding 2—Diverging preferences about how information should be presented. Across the two studies, the ratings obtained for list-based vs. list-based+map were quite similar (Table 2). Nonetheless, the Namibian participants found the list-based displays to be more supportive than the list-based+map displays (Figure 5b), while the German users found list-based displays to be less supportive than the list-based+map displays. In essence, the difference between list-based and list-based+map geoportals lies in the way information is presented and visualized. Here, we observed diverging preferences on how geodata should be presented, visualized, and structured in a geoportal according to the two user groups.

Finding 3—Diverging relative perceptions of task complexity. Here also, ST vs. STT query scenarios obtained similar ratings overall (Table 2). However, the Germans perceived STT tasks to be easier than the ST tasks, while the Namibians perceived ST tasks to be easier than STT tasks (Figure 5a). We lack elements in this work to speculate about the reasons for these differences, but these observations suggest that the relative perception of task complexity within geoportals should be generalized with caution across diverse user groups and possibly diverse user group settings.

Finding 4—Notable differences in perceived usability and novelty between user groups. We observed notable differences between the two user groups in their perceptions of user interface complexity (Figure 3a) and perceived ease of use depending on the type of search tasks (Figure 5a). We also found moderate differences in how the groups rated task support based on the type of display (Figure 5b). Additionally, perceptions of ‘novelty’ varied between the groups (Table 2), suggesting distinct preferences or expectations.

Finding 5—Participants’ expectations in relation to Hofstede’s cultural theory. Our study suggests that some user expectations align with Hofstede’s cultural dimensions. For instance, Namibian participants, associated with higher power distance (PD) values, showed a preference for hierarchical structures and clearly defined navigation paths, as described in [60]. This was reflected in their preference for categorically arranged information, a pattern not observed among the German participants. Additionally, Namibian participants expressed a stronger preference for predefined search cues and guided navigation, which helped them efficiently complete search tasks. In contrast, German participants emphasized flexibility in search options and placed greater importance on the relevance of retrieved information. These findings appear to be consistent with cultural theory, supporting our hypothesis (H1), and call for further work (e.g., using participants’ self-reported data about qualitative preferences about UI organization) to confirm the evidence. Also, the underlying causes for these observations remain unclear. Hence, further research would be needed to explore the specific influences shaping these differences.

5.1.5. Additional Observations

Finding 6—Prior web map use skills influence the perception of UI complexity ratings. The data show that participants with self-reported high competency skills in web map use found interfaces less complex compared to those with low levels of competency (Figure 4).

5.2. Theoretical and Practical Implications

As mentioned in Section 3, the key theoretical assumption underlying this work is an hypothesis (H) about three constructs, namely that user characteristics may affect their cognitive style, which in turn may affect their experience of the geoportal (see Figure 1). We have operationalized user characteristics through several dimensions (e.g., location/country of origin, web mapping skills, programming skills, …), limited the assessment of cognitive style to user perception only, and operationalized user experience through dimensions such as perception of usability problems, self-reported user experience, perceived user interface complexity, and perceived search effectiveness. Though we did not find evidence that all user characteristics lead to differences in all dimensions of user experience, there was evidence that some do. Hence, we learned that the scope of validity of H is limited to some user characteristics and some dimensions of user experience. That is, users’ location/country of origin may affect perception, which in turn affects (aspects of) user experience; users’ web mapping skills may affect perception, which in turn affects (aspects of) user experience. Given the current lack of theories about the user experience of geoportals, these two statements provide working hypotheses [65] to advance such theoretical work in GIScience. Furthermore, considering additional dimensions of cognitive style left out in the present study would help clarify whether a reflective model [66] (in which case memorability and problem-solving ability would covary with perception ratings) or a formative model [66] provides a more suitable representation of the cognitive style construct in the context of geodata discovery.

From the practical point of view, the work has highlighted the need for better user interfaces during discovery. Since we noticed that several recommendations from the literature on search user interface design were not (yet) implemented in some of the geoportals considered, we now link the issues mentioned in Section 4.5 to these recommendations.

Design of input features: challenges related to participants getting confused by queries returning no result for their search or struggling with the positioning of the user interface components were mentioned.

• Recommendation 1: “Help searchers to create useful queries whenever possible” [51]. This may happen, for example, through the use of autocomplete (i.e., giving people guidance towards queries that are likely to work [51]) or interactive query expansion (i.e., the suggestion of additional, or replacement words to the searcher that might help the system return more precise results [51]).
• Recommendation 2: Conduct (formative) usability evaluation, whenever possible, to get feedback on the appropriate placement of user interface components. (No general recommendation is possible here, as the design of each geoportal is unique).

Design of informational features: challenges related to participants not understanding jargon, unclear metadata descriptions, and insufficient search result details were mentioned.

• Recommendation 3: Introduce the possibility of generating laymen’s descriptions for information resources (e.g., these descriptions may be generated on-demand during the search process using current and future language models).
• Recommendation 4: Design text snippets that preview the results with care. As mentioned in [51]: “Informally, two lines are typically chosen as the optimal balance of communicating useful information and including as many results as possible above the first-scroll point”.
• Recommendation 5: Highlight query terms within text snippets to facilitate relevance assessment. (Note that this basic recommendation could already be useful for the current context, as we have observed that several geoportals do not have a mechanism for highlighting salient terms during results presentation. Nonetheless, a purely text-based approach—where boldface is used in snippets to highlight query terms and enhance their salience—may be insufficient as mentioned in [67]. Visual snippets and thumbnails, for example, could be considered as useful complements. Also, snippets are only one example of search support tools. We refer the reader to [68] for additional information on possible options for designing tools for search support more broadly.)

Design of map components: as mentioned in Section 4.5, some users struggled to understand the purpose of the map and were uncertain about how to use it during the search process.

• Recommendation 6: “Make sure the dimensions and layout of [the map] visualization are clear and intuitive to the searcher” [51]. It is worth noting that maps serve a dual role here, both as an input component and a display component, which may also contribute to the ambiguity. Hence, empirical studies are still needed to inform about best design practices for maps as input and display components during the search for geodata.

5.3. Limitations

As mentioned in Section 3.3, we targeted students and professionals within Geographic Information Science and related fields as participants, in line with findings from the literature that a typical geoportal user has considerable experience and understanding in using maps and geodata search. Hence, the findings cannot be claimed to apply to general users of geoportals. Furthermore, while the participant pool ensured relevance to the study’s research question, the study did not examine differences in user expertise levels, such as novice versus expert users. As Section 4.5 illustrated, issues about input features, informational features, map interaction, (meta)data quality, and user guidance during search tasks on geoportals extend beyond the scope of a single portal and single user. However, since usability perceptions and interaction patterns can vary based on experience, future studies could consider including a broader range of user expertise levels and larger samples of participants to provide a more nuanced understanding of profile-specific usability challenges. Another limitation of this study is that geoportal selection did not account for factors such as the number of available datasets, specific focal disciplines, or data update frequency. These aspects can influence usability and user experience, and their omission means that findings may not fully capture how such variations impact geoportal interactions.

From a methodical point of view, the work has performed a purely qualitative assessment of usability issues. Qualitative assessments rely on participants’ self-reports and hence are subject to recall bias. Evidence from the literature suggests that Retrospective Think-Aloud is insensitive to memory decay issues for both informational websites [69] and search tasks in an online catalog [70]. Also, Ohnemus and Biers [71] reported that up to a 24-hour delay does not impact the value of users’ verbalizations when using the RTA method. While these studies are not specific to geoportals, the study has assumed a minimal memory decay bias because search tasks were given to users as done in [70]. The current experimental design, where a participant collects data about a single portal, was chosen to limit the effect of participant fatigue: each portal evaluation took approximately 45 min, and this time commitment would increase proportionally with the number of portals assessed. While the design allowed us to collect four data points per portal (two in Germany and two in Namibia), which were then averaged to derive insights relevant to the aspects addressed in the research questions, we acknowledge that having a single participant evaluate multiple portals is a viable alternative study design that could be considered in future work.

Additionally, given that assessing the quality of results returned by geoportals was the main objective, the work focused on the outcomes of the search task (assessed by the DCG), without examining the details of the users’ search strategies. Hence, aspects such as the number of trial-and-error operations performed by users and time measurement were not in the scope of this work. These could be relevant for future work investigating how users’ search strategies in geoportals impact their overall user experience.

At last, usability issues were assessed based on the geoportals’ versions available at the time of the study (Table 1). Since geoportals continuously evolve through redesigns, feature updates, and changes in functionality, the study cannot claim that the identified usability issues will persist over time. However, the findings remain valuable as they highlight critical areas that require continued research and development to enhance geoportal usability and user experience. Finally, saving the exact version of portals tested in the interest of scientific reproducibility is desirable but not trivial. For want of anything better, links to the web archives of the portals, and short video recordings to give an impression of user interaction of the participants with the interfaces are available at https://doi.org/10.6084/m9.figshare.31061875.

6. Conclusions and Future Work

This work has presented two studies that explored issues faced by participants during the search for geographic data in geoportals. The findings from both studies reveal challenges in evaluating geodata relevance, providing user guidance, and effectively presenting search results in geoportals. Usability problems remain a key concern, indicating that search user interfaces require significant improvements to better support effective data discovery. Additionally, issues with metadata quality and quantity directly impact how users retrieve geodata, highlighting the need for further work to ensure easy and efficient data discovery. While participants’ perceptions of UI complexity were influenced by their experience with web maps, pragmatic user experience perceptions remained relatively consistent across users. A key takeaway is that usability findings for geoportals cannot be universally generalized, as individual and group-level differences shape user experiences.

Future work: It has become clear from the study that the values of DCG cannot be meaningfully compared to those from previous work due to a difference in the data collection procedures. For example, Mai et al. [72] asked users to provide relevance judgment using an ordinal scale with labels ranging from ‘bad’ (0) to ‘perfect’ (4), with the numbers in brackets corresponding to the relevance scores. Jiang et al. [73] used relevance labels from 0 = ‘bad’ to 7 = ‘excellent’. Future work will benefit from an assessment of the effect of different relevance scales on the one hand and the use of similar/standardized collection procedures as input for review studies on the other hand. Standardized collection procedures, along with open data from diverse studies, can open up new possibilities for knowledge accumulation across geoportal evaluation (e.g., enable assessments such as longitudinal evaluations or inter-rater reliability evaluations about the analysis of the qualitative data collected). Additionally, the current study did not discuss with users why they rated the items the way they did, and this could be the subject of a follow-up investigation. Relevance scores can be computed from query-dependent and query-independent features (see e.g., [73]), and it would be interesting to get some insights into which of these features mattered to the user and to which extent. Ultimately, that understanding could provide input for ‘learning to rank’ [74] approaches for information search in geoportals. Finally, this study underscores the need for further research to distinguish usability aspects that are culturally invariant from those influenced by cultural factors, particularly in contexts where power distance varies between user groups. These insights are critical for advancing open data reuse and spatial data infrastructure initiatives, as geoportals are central to these domains. Moving forward, a more nuanced investigation is necessary into how both cultural and individual factors shape user experiences in GIScience applications. Additionally, enhancing metadata quality, refining search UIs, and addressing usability challenges should be prioritized to improve data discovery and accessibility.