Usability Assessment Framework for Crowdsensing Data and the Implicit Spatiotemporal Information

Abstract

Crowdsensing data serves as a crucial resource for supporting spatiotemporal applications and services. However, its inherent heterogeneity and quality uncertainty present significant challenges for data usability assessment: the evaluation methods are difficult to standardize due to the diverse types of data; assessment dimensions are predominantly confined to internal quality attributes; and a comprehensive framework for data usability evaluation remains lacking. To address these challenges, this study proposes an innovative, multi-layered usability assessment framework applicable to six major categories of crowdsensing data: specialized spatial data, Internet of Things (IoT) sensing data, trajectory data, geographic semantic web, scientific literature, and web texts. Building upon a systematic review of existing research on data quality and usability, our framework conducts a comprehensive evaluation of data efficiency, effectiveness, and satisfaction from dual perspectives—data sources and content. We present a complete system comprising primary and secondary indicators and elaborate on their computation and aggregation methods. Indicator weights are determined through the Analytic Hierarchy Process (AHP) and expert consultations, with sensitivity analysis performed to validate the robustness of the framework. The practical applicability of the framework is demonstrated through a case study of constructing a spatiotemporal knowledge graph, where we assess all six types of data. The results indicate that the framework generates distinguishable usability scores and provides actionable insights for improvement. This framework offers a universal standard for selecting high-quality data in complex decision-making scenarios and facilitates the development of reliable spatiotemporal knowledge services.

1. Introduction

Spatiotemporal information [1,2], derived from real-world perception and human activities [3], has become a foundational resource for high-quality development and intelligent decision-making. The proliferation of the Internet of Things (IoT) and ubiquitous sensing technologies has led to crowdsensing data, which collects large-scale, multi-source, and multi-type spatiotemporal information across geographic space, the natural environment, traffic navigation, and human activities [4,5]. Over time, crowdsensing data have become a fundamental resource for empowering high-quality national and local development with spatiotemporal information [6]. Crowdsensing data, gathered through various sensing devices, sensors, smart terminals, or participants, offer broad coverage at low cost but are inherently heterogeneous and uncertain in the big data era.

Current research on data usability remains fragmented, with evaluation methods often limited to specific data types (e.g., geospatial vs. trajectory data), impeding unified assessment in integrated applications, such as spatiotemporal knowledge graphs. Moreover, most frameworks emphasize intrinsic data quality (e.g., accuracy, completeness) while neglecting extrinsic usability dimensions essential for decision support, such as accessibility, cost-effectiveness, and user satisfaction [7,8]. There is also a lack of structured methodologies to quantify how intrinsic quality translates into contextual usability.

To address these gaps, this paper proposes a novel, multi-layered usability assessment framework for heterogeneous crowdsensing data. The main contributions are:

• Unified, Cross-Category Evaluation Framework: We break down methodological barriers by establishing a consistent indicator system applicable to six major crowdsensing data categories: specialized spatial data, IoT sensing data, trajectory data, geographic semantic web, scientific literature, and web texts. This enables comparable usability scoring across diverse data types in a unified application scenario, which prior work lacks.
• Comprehensive Usability Dimensions: The framework integrates the ISO-defined usability dimensions of efficiency, effectiveness, and satisfaction. It operationalizes these through a dual-perspective structure evaluating both data source (e.g., authority, accessibility) and data content (e.g., richness, accuracy), capturing intrinsic state and extrinsic utility.
• Transparent and Operational Methodology: We provide a complete system with primary and secondary indicators, detailed computation methods, and a weighted aggregation mechanism. Indicator weights are derived using the Analytic Hierarchy Process (AHP) and expert consultation, with sensitivity analysis validating robustness. A case study on spatiotemporal knowledge graph construction illustrates practical applicability for data selection and improvement.

This framework provides a standardized, quantitative tool for selecting high-quality crowdsensing data in complex decision-making scenarios, facilitating the development of more reliable spatiotemporal knowledge services.

2. Related Work

2.1. Data Quality and Usability

Data quality refers to the degree to which the data is suited for the intended purpose [9,10], and is a multi-dimensional concept [11]. It includes both the subjective perceptions of the individuals involved with the data, and the objective measurements based on the data itself. Subjective assessments reflect the needs and experiences of stakeholders. Objective assessments can be either task-independent metrics that reflect states of the data, or task-dependent metrics constrained and influenced by specific application contexts [12]. Data quality typically encompasses dimensions such as accuracy, completeness, consistency, timeliness, and reliability [13].

Usability was originally derived from the manufacturing and production domain, referring mainly to the ease of use and acceptability of a product or system for users to complete designated tasks [14]. Drawing upon the ISO 9241-11 standard [15], we define data usability as “the extent to which data can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” This concept extends beyond intrinsic quality to encompass the fitness for use. For data, usability refers to the appropriateness and acceptability of data for users to complete tasks in specific application contexts. It requires data to not only meet quality requirements, but also be easily accessible, understandable, and usable to accomplish the intended tasks. For example, data must be technically accessible and align with the requirements of decision-making contexts.

Data quality has often been associated with the “fitness for use” principle from the outset of research, which refers to the subjectivity and context dependency [16]. Data quality assessment primarily focuses on the intrinsic attributes of data, such as accuracy and consistency [17,18], while data usability assessment emphasizes the applicability and operability of data in specific application scenarios. In essence, data usability builds upon data quality, extending it to practical considerations such as accessibility, relevance, and user experience. Thus, the key distinction lies in their focus: data quality is concerned with the intrinsic state of the data itself, whereas data usability is concerned with its extrinsic value and practical application from a user’s perspective within a specific context.

Data quality directly impacts its usability. Quality characteristics such as accuracy, completeness, and consistency determine whether data can effectively support practical applications. However, even if data performs well in terms of quality, its usability will be questioned if it fails to meet user needs or support decision-making in practice effectively. Therefore, data quality and usability are not merely two independent evaluation processes; they are interdependent. Data quality provides the foundational support for usability, while data usability extends the concept of quality, validating the practical utility and effectiveness of the data.

With the advent of the big data era, data quality assessment has gradually integrated with data usability evaluation, offering a more comprehensive analytical framework to ensure that data achieves the desired outcomes in complex application environments. Our framework, structured around the ISO-defined dimensions of effectiveness, efficiency, and satisfaction, is designed to capture this comprehensive view.

2.2. Research on Data Usability

With the increasing data complexity and diverse application scenarios, the research on data usability has evolved beyond traditional quality metrics to encompass user-centered fitness-for-use, integrating technical, contextual, privacy, and security aspects. To establish a solid theoretical foundation for our framework, this section reviews existing studies categorized by three dominant methodological paradigms: spatial analytical methods, IoT and sensing technologies, and artificial intelligence (AI) techniques.

2.2.1. Spatial Analytical Methods

In Geographic Information Science (GIS), usability research has shifted from pure quality assessment to contextual suitability. Early work proposed multidimensional metrics emphasizing user-centered concepts [4,7]. With the rise of open spatiotemporal data and crowdsourced platforms like Volunteered Geographic Information (VGI) [19] and OpenStreetMap (OSM) [8,20], quality uncertainties from heterogeneous contributors have driven multidimensional frameworks combining quantitative and qualitative criteria [21,22,23].

Research in this field has largely established two dominant paradigms. The first is an indirect evaluation based on provenance and trustworthiness, inferring credibility from edit histories, contributor credibility models [24,25], or participatory assessments [26,27]. The core idea here is to convert the uncertainty in data quality into a quantifiable issue regarding the reliability of sources. The second is a direct evaluation based on authoritative comparison, which assesses location accuracy and completeness by spatially overlaying data [26,27,28,29,30]. This method offers objective, interpretable results but depends on benchmark availability and quality. These approaches highlight that, without full production control, trust can be built through provenance or gold-standard comparisons—a principle reinforced in geographic citizen science, where map usability directly influences participation and output quality [31].

Overall, spatial analytical methods provide structured, interpretable frameworks, balancing between inferring trust from provenance and validating against authoritative sources, though constrained by the availability of high-quality benchmarks.

2.2.2. IoT and Sensing Technology Methods

The integration of IoT and sensing technologies has led to an explosion in the volume and variety of sensing data [32,33,34] and trajectory data [35,36,37]. This is particularly relevant to mobile crowdsensing, where data is often collected via pervasive but heterogeneous and low-cost devices. Evaluation methods illustrate a significant shift from traditional error metrics to perceptual utility and fitness for use. However, the basic quality assessment methods, comparing data against benchmark datasets [38,39] to analyze quality through discrepancies with true values [35,36,37], are costly or impractical, especially for large-scale crowdsensed datasets.

For complex sensing data [40,41], such as videos [38,39], point clouds, and trajectories, “usability” encompasses not only numerical accuracy but also adherence to human cognitive norms [42]. For instance, trajectory data should be highly consistent with human visual perception [35,36,37], leading to the development of perceptually validated metrics for crowd trajectory data. The visual quality of point cloud data should also align with human visual perception characteristics [43]. The core evaluation criteria are no longer limited to how closely data approximates “truth” but rather how well it aligns with “user perception” or “downstream model requirements.”

The contextual dependence is evident in domains [44] like environmental monitoring, where highlighting the extreme importance of data reliability and usability in supporting critical decision-making [45,46]. Thus, methods for IoT and sensing data prioritize contextual fitness and perceptual validity, moving beyond pure accuracy metrics, yet they face challenges in establishing universal benchmarks and managing the inherent noise of crowdsourced sensing streams.

2.2.3. Artificial Intelligence (AI) Techniques

Artificial intelligence (AI) technologies are driving a profound revolution in usability assessment methods. On one hand, data fusion and machine learning correction methods address noise in low-cost sensor data, enhancing reliability cost-effectively [32,33]. On the other hand, research focus has shifted from evaluating the “data itself” to assessing the “quality of datasets used for training machine learning models.” Some studies assert that dataset quality evaluation serves as a preliminary control over model performance, thereby influencing ultimate usability [47]. This is crucial for crowdsensing data, where label noise and bias are prevalent.

Furthermore, AI techniques, particularly Natural Language Processing (NLP), have become indispensable for assessing the usability of unstructured or semi-structured crowdsensed data, such as scientific literature and web texts. They enable the automated evaluation of dimensions like information credibility, thematic relevance, and spatiotemporal information extraction, which are central to the usability of textual sources [48,49].

In the realms of knowledge graphs and semantic data, evaluation dimensions have transcended traditional accuracy metrics to encompass deeper attributes such as semantic richness, logical consistency, and pattern coverage [50,51], leveraging AI for more nuanced quality assessments. In the AI era, data evaluation has transitioned from “static quality” assessment to evaluating its logical coherence as a “knowledge carrier” and its applicability as “fuel for models.”

In contemporary data applications, interoperability, security, and usability are viewed as interrelated and indispensable core attributes [52]. Systematic explorations of privacy and security concerns within software and data usability examine the challenges of incorporating security dimensions into comprehensive usability evaluation frameworks [53]. AI and emerging techniques offer powerful tools for enhancing and assessing data usability, such as enabling sophisticated noise handling, bias detection, and privacy preservation [54,55].

As shown in

Table 1. A synthesis of data quality and usability assessment metrics from existing literature.

Dimension	Metrics	Data Categories	Assessment Methods	Citation
Accuracy	positional accuracy, measurement error, deviation from ground truth, proportion of anomalous records, planar error, elevation error, labeling accuracy, dataset bias	geospatial data, sensing data, trajectory data, machine learning training datasets	Spatial overlay analysis with authoritative datasets, comparison with high-precision benchmark equipment, data cleaning, bias detection	[9,17,33,37,47,56,57]
Reliability /Credibility	data source authority, contributor reputation, user trust, platform credibility	VGI, web texts, scientific data, open government data	contributor reputation models, analysis of historical edit records, assessment of website nature, authority certification	[19,24,25,27]
Completeness	attribute missing rate, record missing rate, data missing ratio, feature completeness, attribute completeness	structured data, Internet of Things (IoT) sensing data, databases, geographic information data	statistical calculation, check for mandatory fields	[12,13,58]
Consistency	logical consistency, format consistency, semantic consistency, topological consistency, schema consistency, logical contradiction rate, consistency with external knowledge bases	databases, knowledge graphs, semantic webs, geographic information data	rule checking, ontological reasoning, schema matching, topological rule validation, reasoning verification, external knowledge base comparison, consistency checking algorithms	[50,51,58,59,60]
Timeliness	update frequency, release time, data latency, temporal error, data currency	streaming data, news, social media data, sensor data, emergency response data	check timestamps, evaluate update cycles	[10,32,61]
Fitness	thematic relevance, spatial extent matching, temporal coverage, information richness, decision-support capability	scientific data, environmental data, domain-specific data, emergency data	domain expert review, application scenario validation, content analysis	[7,45,62]
Usability	accessibility, understandability, cost-effectiveness, user satisfaction, ease of operation, applicability	open government data, data portals, software systems, scientific data	User-Centered Design (UCD), heuristic evaluation, user interviews	[14,15,22,63]
Normativity	standards compliance, data compliance, format standardization, coordinate system uniformity	professional surveying and mapping products, geographic information data, vehicle-borne mobile measurement data	check compliance with national standards (GB/T), industry standards (CH/T)	[56,64,65,66]
Privacy and Security	privacy protection strength, utility loss, encryption strength, response time, data security	trajectory data, healthcare data, digital repositories	trajectory synthesis, differential privacy, searchable encryption, blockchain technology, security protocol evaluation	[52,54,55]
Visualization and Perceptual Quality	visual quality, perceptual consistency, user experience, visualization effectiveness	point cloud data, video data, geovisualization, open data portals	subjective user scoring, perceptually-validated metrics, eye-tracking, visualization effectiveness evaluation	[19,31,38,43]

, our system summarizes the key dimensions, specific indicators, applicable data categories, and commonly used evaluation methods from previous research. Significant progress has been made in defining core data attributes in these studies. However, these advancements are fragmented across different data types, with a notable lack of a unified framework capable of integrating diverse indicators and suitable for cross-category usability evaluation. This deficiency hinders consistent evaluation of heterogeneous, multi-source data in integrated applications, such as spatiotemporal knowledge graphs.

2.3. Limitations and Contributions

Research on data usability assessment across various domains has provided a wealth of methodologies and profound insights [67,68,69,70]. However, when considering the integrated use of heterogeneous crowdsensing data in complex applications (e.g., spatiotemporal knowledge graphs), three significant limitations persist in the current literature.

First, there is a notable methodological isolation among different data types. Although comprehensive quality frameworks exist for individual data types (e.g., for Linked Data [71] and Volunteered Geographic Information [72]), these paradigms remain incompatible [73,74,75]. This methodological fragmentation obstructs the unified assessment of data usability across diverse types in application contexts.

Second, existing frameworks frequently exhibit indicator incompleteness. While classic dimensions like accuracy and credibility have been explored, other dimensions, such as authority and timeliness, have also received some investigation [76]. However, these still tend to adopt a “data user” perspective. They have often failed to fully capture its value in real-world decision-making processes.

Third, there is a lack of structured approaches that integrate data quality and usability. Most frameworks do not clearly delineate the interactions between quality attributes (such as accuracy and completeness) and usability, resulting in a gap in understanding how quality metrics quantitatively transform into usability scores under specific application constraints.

To address these challenges, this study proposes an integrated usability assessment framework specifically designed for heterogeneous crowdsensing data. This study makes the following key contributions:

• Cross-Domain Unified Assessment Framework: We bridge the methodological isolation by establishing a multi-layered, multi-granular indicator system that coordinates evaluation standards across six categories of crowdsensing data. This system enables consistent and comparable evaluation of usability for different data types within a unified application context.
• Comprehensive Usability Dimensions: Our framework integrates a broad range of usability dimensions. It provides a structured synthesis that integrates both traditional quality and decision-centric usability dimensions, and emphasizes usability relevant to decision support and practical applications. This reflects both the intrinsic data quality and its extrinsic utility.
• Integrated Quality and Usability Evaluation Framework: Our framework explicitly captures the transition from data quality assessment to usability evaluation. By dividing the assessment into the dual aspects of “data content” (intrinsic quality) and “data source” (contextual factors), and by integrating usability criteria such as accessibility and understandability into the calculation process, the framework provides a structured and operational workflow for deriving usability scores.

This integrated assessment framework provides a standardized tool for selecting heterogeneous crowdsensing data in complex decision-making scenarios, facilitating the construction of high-quality spatiotemporal knowledge services and supporting more reliable data-driven decisions.

3. Assessment Objects and Basic Idea

3.1. Basic Idea

The assessment of data usability is the process of evaluating the acceptable or appropriate degree of data science usage, which is conducted using either qualitative or quantitative methods. Data, users, and applications are the three fundamental elements in constructing a usability assessment framework [77,78]. Crowdsensing data is the assessment object, focusing on the spatiotemporal attributes. Users are the primary agents whose needs dictate requirements beyond intrinsic quality, such as accessibility and source of authority. Applications refer to specific scenarios and goals of data usage.

Figure 1 presents a conceptual model illustrating these relationships. Data quality is a necessary but insufficient condition for usability. While high-quality data are essential for usability, other factors, such as user needs and application requirements, must also be evaluated to ensure the overall usability of the data.

The framework is guided by the ISO 9241-11 standard, which defines usability through three core dimensions: efficiency, effectiveness, and satisfaction [7,15,79]. These dimensions directly translate the fundamental elements into measurable concepts. Specifically, efficiency translates to the resources required from the user to acquire and use the data. Effectiveness translates to the accuracy and completeness with which the data supports goal achievement in the application. Satisfaction refers to the degree of acceptance and value perceived by the user regarding the data within the application context. The framework considers how both data quality and usability influence the user experience in real-world applications of crowdsensing data.

To operationalize three core dimensions into measurable criteria, our assessment framework is structured around two essential aspects: data source (origins and acquisition channels, affecting initial trust and access effort) and data content (structural characteristics and latent information, determining applicability and value). The application of the ISO 9241-11 standard’s three usability dimensions to our framework is not only conceptually aligned but also operationalized through specific evaluation indicators. This operational logic is detailed in Figure 2.

• Efficiency is assessed via accessibility, which gauges the ease and feasibility of obtaining data. For the data content, it is measured by information richness and resolution, reflecting the resources consumed to extract sufficient value from the data.
• Effectiveness is evaluated through authority, accuracy, and timeliness, which reflect the trustworthiness and intrinsic quality.
• Satisfaction is inferred from user engagement, response efficiency, impact, and reliability, which measure user acceptance and trust.

This structured approach allows us to deconstruct the abstract concept of usability into a concrete set of assessable indicators. The subsequent sections will detail the specific indicators for each category of crowdsensing data based on this conceptual framework.

3.2. Assessment Objects

The usability evaluation framework focuses on crowdsensing data containing rich spatiotemporal information, characterized by multiple sources, diverse types, and significant differences in data structure, presentation forms, and information content. It primarily includes six categories:

Specialized spatial data are commonly obtained from geography and surveys using specific information acquisition technologies. It represents the shape, size, location, and other characteristics of spatial entities, such as vectors and map data. Examples include administrative boundary datasets, digital elevation models (DEMs), and land-use/land-cover (LULC) maps. These data often serve as the foundational geospatial framework in applications like urban planning and environmental monitoring.

IoT sensing data refer to the state data of the spatiotemporal information of the environment collected through sensing devices, aggregated, and integrated via the IoT. This includes monitoring videos, photos, and geological monitoring data, such as measurements of subsidence and deformation, and meteorological data, such as rainfall and temperature. For instance, hourly readings from a national air quality monitoring network or real-time video feeds from traffic cameras are typical IoT sensing data used in environmental analysis and smart city operations.

Trajectory data refers to data obtained by sampling the movement process of one or more moving objects in a spatiotemporal space. The key information includes the location, sampling time, speed, and direction at each sample point [80,81,82]. The trajectory data generally includes data on vehicle activity, human movement, animal migration, and natural phenomenon trajectories. Examples include GPS tracks from shared bicycles, mobile phone signaling data, or ship automatic identification system (AIS) records.

Geographic semantic webs (GSW) organize concepts, attributes, and relationship models within a geographic domain using intelligent web technology. GSW are composed of structured, machine-readable knowledge graphs, where entities and their relationships are explicitly defined using formal ontologies (e.g., RDF, OWL), which are the focus of the evaluation. Prominent examples include GeoNames, Wikidata-Geo, and the OSM Semantic Network, which link places with rich attributes and relationships.

Scientific literature comprises textual materials published based on systematic research and experiments, such as academic journal articles, conference papers, technical reports, and patents. Such literature forms the foundation for knowledge accumulation and academic discourse within specific domains. This study focuses on evaluating the usability of scientific literature, analyzing its textual content to assess information quality, dissemination efficiency, and reliability, rather than directly evaluating the data contained within the literature. For example, a review article on climate change impacts or a dataset description paper in a scientific journal would fall into this category.

Web texts refer to the text on web pages found on various websites that contain spatiotemporal descriptions and geographic variations. Web text is a significant type of textual data on the Internet. Web texts can be categorized into two types: professional-generated content (PGC) and user-generated content (UGC) [83,84,85]. PGC refers to official publications, such as online encyclopedia entries (e.g., Baidu Baike, Wikipedia), news from media outlets, and government documents. In contrast, UGC refers to content produced by individuals, including posts, comments, and reviews on social media platforms (e.g., Weibo, Facebook). These texts often contain implicit spatiotemporal cues like location check-ins or event dates.

These six categories encompass a comprehensive range of spatiotemporal data types applied typically in spatiotemporal information services. The data sources include both professional datasets and crowdsourced data. The data is presented in structured, semi-structured, or unstructured formats, including numerical and textual information. These datasets contain rich explicit or implicit spatiotemporal information. This paper focuses on conducting usability evaluations of these six categories of crowdsensing data and the implicit spatiotemporal information.

4. Analyses of Usability Assessment Framework

4.1. Indicators

This study proposes a hierarchical indicator system with two levels: primary indicators outline broad evaluation dimensions, and secondary indicators specify measurable quantities with defined acquisition and calculation methods. The latter is a component of the former.

The framework covers the six data categories defined in Section 3.2, with a total of 14 primary and 99 secondary indicators. It employs a hierarchical, modular design, where each category uses a tailored subset of indicators to ensure comprehensive coverage and application-specific flexibility.

Table 2. Primary indicators of crowdsensing data usability assessment.

Categories	Aspects	Indicators
Specialized spatial data	Data source	Authority; Accessibility
	Data content	Spatiotemporal Information Richness; Spatial Resolution; Temporal Resolution; Timeliness; Completeness; Accuracy and Reliability
IoT sensing data	Data source	Authority; Accessibility
	Data content	Spatiotemporal Information Richness; Spatial Resolution; Temporal Resolution; Timeliness; Completeness; Accuracy and Reliability
Trajectory data	Data source	Authority; Accessibility; Mobile Entity
	Data content	Spatiotemporal Information Richness; Spatial Resolution; Temporal Resolution; Timeliness; Completeness; Accuracy and Reliability
Geographic semantic web	Data source	Authority; Accessibility
	Data content	Spatiotemporal Information Richness; Accuracy and Reliability; Timeliness; Completeness
Scientific literature	Data source	Authority; Accessibility
	Data content	Spatiotemporal Information Richness; Impact; Openness; Timeliness
Web texts	Data source	Authority; Accessibility
	Data content	Spatiotemporal Information Richness; Information Density; Spatiotemporal Information Existence; Spatiotemporal Information Granularity; Accuracy and Reliability; Impact; Timeliness

summarizes the primary indicators for each category, structured around the dual aspects of data source and data content.

The proposed indicators include both generic and specific types. Generic indicators apply to all six data categories, addressing common user concerns, and are derived from established core dimensions (Section 2.3). They enable cross-category comparability. For example, authority is a primary generic indicator for data source, but its secondary indicators manifest differently per category (e.g., entity nature for spatial data, journal impact factor for scientific literature).

Specific indicators are tailored to individual data categories, justified by the data’s organizational structure, characteristics, and domain-specific requirements. Their selection is justified by a detailed analysis of the data’s organizational structure, the characteristics of the data structure, and domain-specific requirements. Examples include spatiotemporal resolution for sensing data and impact for web texts. Introducing these indicators represents a key innovation, ensuring comprehensive, fit-for-purpose evaluation.

To enhance practical guidance, each secondary indicator is categorized as “Core” or “Optional” in the following subsections. Core indicators are essential for most scenarios; optional indicators address specific attributes or advanced analyses for flexible selection based on application needs.

4.1.1. Specialized Spatial Data Indicators

The indicators for specialized spatial data are grounded in authoritative geospatial standards (e.g., ISO 19157, GB/T 18316, GB/T 24356) [56,57,58], focusing on data conformity and accuracy. And indicators are extended with usability dimensions from the AGILE usability framework and spatiotemporal data reliability research [17].

Intrinsic quality indicators are designed to enhance the evaluation of the fundamental quality of spatial data, including spatial resolution, temporal resolution, timeliness, completeness, and accuracy and reliability. Spatial and temporal resolution reflect the detail level and richness of the spatiotemporal information. Timeliness is assessed by the publish date of data, which indicates the degree to which information is delivered promptly. Completeness is evaluated by the schema and records completeness. Accuracy and reliability evaluate compliance with application accuracy and precision requirements.

Extrinsic usability indicators emphasize the practical applicability of the data, including authority, accessibility, and spatiotemporal information richness. Authority reflects the credibility and the user engagement of the data source. Accessibility refers to the feasibility and convenience for users to obtain data, including the stability and cost to acquire the data. Spatiotemporal information richness is measured by attribute type diversity, data value richness, observational dimension, temporal range, and spatial range. Attribute type diversity assesses the structural variety of the data schema. A dataset with rich types typically supports more comprehensive and multi-dimensional analysis. For datasets where richness derives from multiple dimensions of the same data type (e.g., multi-band remote sensing imagery), this is complemented by the observational dimension metric. Data value richness refers to the extent of variability within the values of each attribute.

The indicators and their value acquisition methods are listed in

Table 3. Indicators and acquisition methods of specialized spatial data usability assessment.

Primary Indicators	Secondary Indicators	Definition	Calculation Methods
authority	entity nature (Critical)	the nature of data production entity	multi-level judgment based on entity nature ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{g}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{a}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{s}\\ \begin{array}{c}100,\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\\ 70,\mathrm{b}\mathrm{u}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s} \mathrm{o}\mathrm{r}\mathrm{g}\mathrm{a}\mathrm{n}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{d}\mathrm{e}\mathrm{m}\mathrm{i}\mathrm{c} \mathrm{t}\mathrm{e}\mathrm{a}\mathrm{m} \mathrm{o}\mathrm{r} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{c}\\ 25,\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{o}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	user count (Optional)	the number of users from data provider	logarithmic scaling of user count (r) from API ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,25\times {\log }_{10}\left(\mathrm{r}+1\right))$
	download volume (Optional)	the total amount of data that has been downloaded	logarithmic scaling to download volume (r) from API ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}(\mathrm{100,15}\times {\log }_{10}\left(\mathrm{r}+1\right)),\mathrm{d}\mathrm{o}\mathrm{w}\mathrm{n}\mathrm{l}\mathrm{o}\mathrm{a}\mathrm{d}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\\ 20,\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{b}\mathrm{l}\mathrm{e} \mathrm{o}\mathrm{n}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{e}\\ 0,\mathrm{u}\mathrm{n}\mathrm{a}\mathrm{v}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\end{array}$
accessibility	release frequency (Critical)	the regularity with which data is updated, distributed, or published	multi-level judgment based on data release regularity ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{l}-\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\\ 80,\mathrm{h}\mathrm{i}\mathrm{g}\mathrm{h}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{d}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{y}/\mathrm{w}\mathrm{e}\mathrm{e}\mathrm{k}\mathrm{l}\mathrm{y})\\ \begin{array}{c}60,\mathrm{l}\mathrm{o}\mathrm{w}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}/\mathrm{y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{l}\mathrm{y})\\ 30,\mathrm{b}\mathrm{e}\mathrm{y}\mathrm{o}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{n}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{s}\\ 0,\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{u}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	acquisition method (Critical)	the cost and convenience of the data acquisition process	multi-level judgment based on acquisition method ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{o}\mathrm{p}\mathrm{e}\mathrm{n} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ \begin{array}{c}90,\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ 70,\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{p}\mathrm{a}\mathrm{i}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ 0,\mathrm{n}\mathrm{o} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\end{array}\end{array}$
spatiotemporal information richness	attribute type diversity (Critical)	the variability in format, such as: numerical, categorical, textual, temporal, spatial, and binary	entropy formula of data patterns diversity
	observational dimension (Optional)	the number of observational dimensions or thematic layers	normalization based on the dimensions or layers count, max count is the benchmark number of the specific type data 1 ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}\left(\mathrm{100,100}\times {\displaystyle \frac{\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s} \mathrm{o}\mathrm{r} \mathrm{l}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}\mathrm{s} \mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}}{\mathrm{m}\mathrm{a}\mathrm{x} \mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}}}\right)$
	data value richness (Optional)	the variety and distribution of the actual values within each attribute	entropy formula of data values diversity
	temporal range (Critical)	the time over the dataset	binary judgment based on meta-data
	spatial range (Critical)	the spatial over the dataset	binary judgment based on meta-data
spatial resolution	spatial resolution (Critical)	the pixel sizes of remote sensing images	binary judgment based on meta-data
	scale (Optional)	the scale of geographic information	binary judgment based on meta-data
temporal resolution	temporal resolution (Critical)	the time interval between consecutive observations	binary judgment based on meta-data
timeliness	publish date (Critical)	the degree of promptness of information	linear decay model based on publish time and update cycle, k is the decay value per cycle (default value is 10) ${\mathrm{I}}_{\mathrm{s}}=100-\mathrm{k}\times \lfloor {\displaystyle \frac{∆\mathrm{t}}{\mathrm{u}\mathrm{p}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e} \mathrm{c}\mathrm{y}\mathrm{c}\mathrm{l}\mathrm{e}}}\rfloor$ $∆\mathrm{t}=\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}-\mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}$
completeness	attribute items completeness (Critical)	whether the attributes included in the data comprehensively cover the necessary content	multi-level judgment based on attributes 2 ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}\begin{array}{c}100,\mathrm{n}\mathrm{o} \mathrm{m}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{g} \mathrm{a}\mathrm{t}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{b}\mathrm{u}\mathrm{t}\mathrm{e}\\ 50,\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{g} \mathrm{u}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t} \mathrm{a}\mathrm{t}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{b}\mathrm{u}\mathrm{t}\mathrm{e}\end{array}\\ 0,\mathrm{missing} \mathrm{important} \mathrm{attribute}\end{array}$
	records completeness (Critical)	missing ratio of data records	ratio calculation based on data records ${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{v}\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{t} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}})$
accuracy and reliability	planar errors (Critical)	deviations in the horizontal plane	binary judgment based on planar errors
	elevation errors (Optional)	deviations in the vertical dimension	binary judgment based on elevation errors
	temporal errors (Optional)	deviations in the time stamps or temporal measurements	binary judgment based on temporal errors

¹ The value of max count is determined by domain standards (e.g., 11 for Landsat-8 bands) or expert consultation for the target application. ² $\mathrm{I}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}$ attributes is defined by the relevant data standards. “GB 21139-2007 Basic requirements for standard data of fundamental geographic information” [86], “GB/T 37119-2018 Specifications for Collecting and Processing volunteered Geographic Information” [87].

.

4.1.2. Internet of Things Sensing Data Indicators

IoT sensing data indicators are informed by principles of usability for scientific and environmental data [19,62,63,88] and consider the data’s unique production chain involving sensing instruments and transmission networks.

Intrinsic quality indicators focus on core attributes, including spatial resolution, temporal resolution, timeliness, completeness, and accuracy & reliability. Spatial resolution is ideally defined by the sensor’s sampling footprint or positional accuracy. In practice, for sparse networks or unspecified deployments, it may be inferred from the characteristic distance between sensing points or conservatively estimated based on application requirements. The temporal resolution is typically obtained from the sensor’s sampling configuration or metadata. When not explicitly stated, it can be derived from the statistical mode of the observed time intervals between consecutive records. Timeliness is critical for real-time applications. Completeness is evaluated by the missing ratio of recorded results. Accuracy & reliability are measured based on the anomalous records proportion, measurement errors, and temporal errors. In high-precision ground truth absence, practical assessment relies on proxy indicators such as manufacturer-specified sensor accuracy, calibration records, internal consistency checks (for measurement errors), and network latency or timestamp coherence logs (for temporal errors).

Extrinsic usability indicators, derived from usability principles for scientific data, include authority, accessibility, and spatiotemporal information richness. Authority and accessibility are conceptually aligned with their definitions for specialized spatial data. Based on the characteristics of IoT data, spatiotemporal information richness is operationalized through secondary indicators: sensor component (reflecting the diversity of sensing sources), data value richness, temporal range, and spatial range.

The indicators and their value-acquisition methods are listed in

Table 4. Indicators and acquisition methods of IoT sensing data usability assessment.

Primary Indicators	Secondary Indicators	Definition	Calculation Methods
authority	entity nature (Critical)	the nature of data production entity	multi-level judgment based on entity nature ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{g}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{a}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{s}\\ \begin{array}{c}100,\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\\ 70,\mathrm{b}\mathrm{u}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s} \mathrm{o}\mathrm{r}\mathrm{g}\mathrm{a}\mathrm{n}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{d}\mathrm{e}\mathrm{m}\mathrm{i}\mathrm{c} \mathrm{t}\mathrm{e}\mathrm{a}\mathrm{m} \mathrm{o}\mathrm{r} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{c}\\ 25,\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{o}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	user count (Optional)	the number of users from data provider	logarithmic scaling of user count (r) from API ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,25\times {\log }_{10}\left(\mathrm{r}+1\right))$
	download volume (Optional)	the total amount of data that has been downloaded	logarithmic scaling to download volume (r) from API ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}(\mathrm{100,15}\times {\log }_{10}\left(\mathrm{r}+1\right)),\mathrm{d}\mathrm{o}\mathrm{w}\mathrm{n}\mathrm{l}\mathrm{o}\mathrm{a}\mathrm{d}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\\ 20,\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{b}\mathrm{l}\mathrm{e} \mathrm{o}\mathrm{n}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{e}\\ 0,\mathrm{u}\mathrm{n}\mathrm{a}\mathrm{v}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\end{array}$
accessibility	release frequency (Critical)	the regularity with which data is updated, distributed, or published	multi-level judgment based on data release regularity ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{l}-\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\\ 80,\mathrm{h}\mathrm{i}\mathrm{g}\mathrm{h}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{d}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{y}/\mathrm{w}\mathrm{e}\mathrm{e}\mathrm{k}\mathrm{l}\mathrm{y})\\ \begin{array}{c}60,\mathrm{l}\mathrm{o}\mathrm{w}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}/\mathrm{y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{l}\mathrm{y})\\ 30,\mathrm{b}\mathrm{e}\mathrm{y}\mathrm{o}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{n}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{s}\\ 0,\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{u}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	acquisition method (Critical)	the cost and convenience of the data acquisition process	multi-level judgment based on acquisition method ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{o}\mathrm{p}\mathrm{e}\mathrm{n} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ \begin{array}{c}90,\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ 70,\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{p}\mathrm{a}\mathrm{i}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ 0,\mathrm{n}\mathrm{o} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\end{array}\end{array}$
spatiotemporal information richness	sensor component (Optional)	the number of sensors reflecting the diversity of the sensing information in the data	normalization based on the sensor count, max count is the benchmark number of a complete suite of sensor types deemed necessary for the specific application 1 ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}\left(\mathrm{100,100}\times {\displaystyle \frac{\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{o}\mathrm{r} \mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}}{\mathrm{m}\mathrm{a}\mathrm{x} \mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}}}\right)$
	data value richness (Critical)	the variety and distribution of the actual values	entropy formula of data values diversity
	temporal range (Critical)	the time over the dataset	binary judgment based on meta-data
	spatial range (Critical)	the spatial over the dataset	binary judgment based on meta-data
spatial resolution	spatial resolution (Optional)	the smallest distinguishable spatial unit represented in datasets.	binary judgment based on meta-data
temporal resolution	temporal resolution (Critical)	the time interval between two consecutive measurements,	binary judgment based on meta-data
timeliness	publish date (Critical)	the degree of promptness of information	exponential decay based on data publish time ${\mathrm{I}}_{\mathrm{s}}=100\times {\mathrm{e}}^{-\mathsf{\lambda }·∆\mathrm{t}}$ $∆\mathrm{t}=\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}-\mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}$ $\mathrm{real}\text{-}\mathrm{time}: ∆\mathrm{t}$$\mathrm{unit}: \mathrm{hours}, \mathsf{\lambda }$ default value: 0.1 $\mathrm{general}: ∆\mathrm{t}$$\mathrm{unit}: \mathrm{daily}, \mathsf{\lambda }$ default value: 0.01 $\mathrm{long}\text{-}\mathrm{term}: ∆\mathrm{t}$$\mathrm{unit}: \mathrm{years}, \mathsf{\lambda }$ default value: 0.01
completeness	records completeness (Critical)	missing ratio of records	ratio calculation based on data records ${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{v}\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{t} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}})$
accuracy and reliability	anomalous proportion (Critical)	the ratio of anomalous records in the data	ratio calculation based on data records ${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{a}\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{o}\mathrm{u}\mathrm{s} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}})$
	measurement errors (Critical)	the discrepancies between the observed value and the true value	binary judgment based on measurement errors
	temporal errors (Optional)	deviations in the time stamps or temporal measurements	binary judgment based on temporal errors

¹ The value of max count is determined objectively by referring to domain-specific standards (e.g., national air quality monitoring standards mandate max count = 6 for pollutants) or through expert consultation.

.

4.1.3. Trajectory Data Indicators

Trajectory data indicators are based on established research and standards (e.g., CH/T 6003-2016, CH/T 6004-2016) [64,65], including the quality control of the measurement process and data. The assessment considers influences from the collection process (equipment, sampling frequency, and storage).

Intrinsic quality indicators encompass spatial resolution, temporal resolution, timeliness, completeness, and accuracy and reliability. Spatial resolution is described by sampling point density, and temporal resolution by the sampling interval. These are crucial for data efficiency. For applications requiring real-time data (e.g., navigation), timeliness is a critical efficiency metric. Completeness is measured by the loss rate against the theoretical sampling interval. For data without a predefined interval, this metric is handled per the method for missing data. Accuracy and reliability are assessed by positioning accuracy, typically calculated as the proportion of anomalous points identified by drift detection algorithms (e.g., Kalman filter).

Extrinsic usability indicators include authority, accessibility, mobile entity, and spatiotemporal information richness. Authority and accessibility are conceptually aligned with prior categories. Mobile entity is a key effectiveness indicator. The type of mobile entities (e.g., pedestrian, car, ship) directly determines movement patterns and sampling capabilities. Spatiotemporal information richness includes semantic richness (diversity of high-level semantic attributes like transportation mode and activity type), temporal range, and spatial range.

The indicators and their value acquisition methods are listed in

Table 5. Indicators and acquisition methods of trajectory data usability assessment.

Primary Indicators	Secondary Indicators	Definition	Calculation Methods
authority	entity nature (Critical)	the nature of data production entity	multi-level judgment based on entity nature ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{g}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{a}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{s}\\ \begin{array}{c}100,\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\\ 70,\mathrm{b}\mathrm{u}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s} \mathrm{o}\mathrm{r}\mathrm{g}\mathrm{a}\mathrm{n}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{d}\mathrm{e}\mathrm{m}\mathrm{i}\mathrm{c} \mathrm{t}\mathrm{e}\mathrm{a}\mathrm{m} \mathrm{o}\mathrm{r} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{c}\\ 25,\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{o}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	user count (Optional)	the number of users from data provider	logarithmic scaling of user count (r) from API ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,25\times {\log }_{10}\left(\mathrm{r}+1\right))$
	download volume (Optional)	the total amount of data that has been downloaded	logarithmic scaling to download volume (r) from API ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}(\mathrm{100,15}\times {\log }_{10}\left(\mathrm{r}+1\right)),\mathrm{d}\mathrm{o}\mathrm{w}\mathrm{n}\mathrm{l}\mathrm{o}\mathrm{a}\mathrm{d}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\\ 20,\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{b}\mathrm{l}\mathrm{e} \mathrm{o}\mathrm{n}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{e}\\ 0,\mathrm{u}\mathrm{n}\mathrm{a}\mathrm{v}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\end{array}$
accessibility	release frequency (Critical)	the regularity with which data is updated, distributed, or published	multi-level judgment based on data release regularity ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{l}-\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\\ 80,\mathrm{h}\mathrm{i}\mathrm{g}\mathrm{h}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{d}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{y}/\mathrm{w}\mathrm{e}\mathrm{e}\mathrm{k}\mathrm{l}\mathrm{y})\\ \begin{array}{c}60,\mathrm{l}\mathrm{o}\mathrm{w}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}/\mathrm{y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{l}\mathrm{y})\\ 30,\mathrm{b}\mathrm{e}\mathrm{y}\mathrm{o}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{n}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{s}\\ 0,\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{u}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	acquisition method (Critical)	the cost and convenience of the data acquisition process	multi-level judgment based on acquisition method ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{o}\mathrm{p}\mathrm{e}\mathrm{n} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ \begin{array}{c}90,\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ 70,\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{p}\mathrm{a}\mathrm{i}\mathrm{d} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\\ 0,\mathrm{n}\mathrm{o} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\end{array}\end{array}$
mobile entity	mobile entity (Critical)	the object, device, or individual that moves in the environment to obtain data	multi-level judgment based on mobile entity ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{m}\mathrm{o}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{e} \mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{y}\\ 50,\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{y} \mathrm{t}\mathrm{y}\mathrm{p}\mathrm{e}\\ 25,\mathrm{u}\mathrm{n}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}\end{array}$
spatiotemporal information richness	semantic richness (Optional)	the variability in semantics, such as spatial, temporal, behavioral, user, contextual, traffic flow, social, and intentional	entropy formula of data patterns diversity
	temporal range (Critical)	the time over the dataset,	binary judgment based on meta-data
	spatial range (Critical)	the spatial over the dataset	binary judgment based on meta-data
spatial resolution	sampling point density (Critical)	the number of data points collected per unit of space	binary judgment based on meta-data
temporal resolution	sampling interval (Critical)	the time interval between consecutive data points in the sampling process	binary judgment based on meta-data
timeliness	publish date (Critical)	the degree of promptness of information	exponential decay based on data publish time ${\mathrm{I}}_{\mathrm{s}}=100\times {\mathrm{e}}^{-\mathsf{\lambda }·∆\mathrm{t}}$ $∆\mathrm{t}=\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}-\mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}$ $\mathrm{real}\text{-}\mathrm{time}: ∆\mathrm{t}$$\mathrm{unit}: \mathrm{hours}, \mathsf{\lambda }$ default value: 0.1 $\mathrm{general}: ∆\mathrm{t}$$\mathrm{unit}: \mathrm{daily}, \mathsf{\lambda }$ default value: 0.01 $\mathrm{long}\text{-}\mathrm{term}: ∆\mathrm{t}$$\mathrm{unit}: \mathrm{years}, \mathsf{\lambda }$ default value: 0.01
completeness	loss rate (Critical)	the number of missing trajectory points is calculated based on the theoretical sampling interval	ratio calculation based on data records ${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{v}\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{t} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}})$
accuracy and reliability	positioning accuracy (Critical)	the proportion of anomalous points identified using drift detection algorithms	ratio calculation based on data records, anomalous records are identified by the positioning drift detection algorithm, such as a filtering algorithm ${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{a}\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{o}\mathrm{u}\mathrm{s} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}})$

.

4.1.4. Geographic Semantic Web Indicators

The quality assessment of the geospatial semantic web is an integrated field that integrates standards from three primary dimensions: geospatial information, semantic web, and data quality. The assessment indicators for the geographic semantic web are proposed based on the standards for geospatial information (e.g., ISO 19157, ISO 19115, GB/T 42986.1-2023) [58,89,90] and linked data quality frameworks [59,60,75,91,92].

Intrinsic quality indicators include timeliness, completeness, and accuracy and reliability. Timeliness is evaluated using the most recent update date. Completeness is assessed from two perspectives: data instance completeness (loss rate of mandatory attribute values) and schema completeness (coverage of target domain concepts). Target domain concepts are the core entity types essential for describing the geographic phenomena within the evaluation scope. In general-purpose evaluation, the concepts are derived by extracting geographic entity classes (e.g., gn:Feature) from established cross-domain knowledge graph schemas like GeoNames and Wikidata-Geo, offering a comprehensive, community-vetted foundation. In a domain-specific application (e.g., land survey, emergency response), this baseline is superseded or refined by concepts from relevant national standards, industry classifications, or official regulatory schemas, such as the geographic feature classification in GB/T 13923 [93]. Accuracy and reliability are evaluated on two levels: semantic accuracy (correctness of factual content, e.g., by detecting abnormal triples) and logical consistency (adherence to predefined logical constraints, e.g., absence of property value conflicts).

Extrinsic usability indicators include spatiotemporal information richness, authority, and accessibility. Spatiotemporal information richness is operationalized through entity count (the scale), proportion of spatiotemporal tuples, attribute type diversity, structural richness (denseness of the semantic network), and temporal/spatial range. Authority reflects the trustworthiness of the GSW’s sources (entity nature, user count). Accessibility is assessed by query response efficiency and update release frequency.

The indicators and their value acquisition methods are listed in

Table 6. Indicators and acquisition methods of geographic semantic web usability assessment.

Primary Indicators	Secondary Indicators	Definition	Calculation Methods
authority	entity nature (Critical)	the nature of data production entity	multi-level judgment based on entity nature ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{g}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{a}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{s}\\ \begin{array}{c}100,\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\\ 70,\mathrm{b}\mathrm{u}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s} \mathrm{o}\mathrm{r}\mathrm{g}\mathrm{a}\mathrm{n}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{d}\mathrm{e}\mathrm{m}\mathrm{i}\mathrm{c} \mathrm{t}\mathrm{e}\mathrm{a}\mathrm{m} \mathrm{o}\mathrm{r} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{c}\\ 25,\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{o}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	user count (Optional)	the number of users from data provider	logarithmic scaling of user count (r) from API ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,25\times {\log }_{10}\left(\mathrm{r}+1\right))$
accessibility	release frequency (Optional)	the regularity with which the semantic web is updated	multi-level judgment based on data release regularity ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{h}\mathrm{i}\mathrm{g}\mathrm{h}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{d}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{y}/\mathrm{w}\mathrm{e}\mathrm{e}\mathrm{k}\mathrm{l}\mathrm{y})\\ 80,\mathrm{l}\mathrm{o}\mathrm{w}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}/\mathrm{y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{l}\mathrm{y})\\ \begin{array}{c}60,\mathrm{b}\mathrm{e}\mathrm{y}\mathrm{o}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{n}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{s}\\ 0,\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{u}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	response efficiency (Critical)	the query response speed of the semantic web endpoint	linear decay based on average response time (r) of a suite of standardized queries, which included simple, complex, and aggregate types (in seconds) 1 ${\mathrm{I}}_{\mathrm{s}}=100\times \left(1-{\displaystyle \frac{\mathrm{r}}{10}}\right)$
spatiotemporal information richness	entity count (Critical)	the total number of distinct entities in the semantic web	logarithmic scaling of tuple count (r) ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,14.29\times {\log }_{10}\left(\mathrm{r}+1\right))$
	proportion of spatiotemporal tuples (Optional)	the percentage of tuples that involve both spatial and temporal components	logarithmic scaling of spatiotemporal tuples ratio ${\mathrm{I}}_{\mathrm{s}}=50\times {\log }_{10}\left(1+{\displaystyle \frac{\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l} \mathrm{t}\mathrm{u}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{s} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}}{\mathrm{t}\mathrm{u}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{s} \mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}}}\right)$
	attribute type diversity (Optional)	the variability in the format of tuple attribute, such as numerical, categorical, textual, temporal, spatial, and binary	entropy formula of data patterns diversity
	structural richness (Optional)	the sparseness (or sparsity) in graph theory is used to indirectly characterize the denseness of the semantic network and the richness of semantic connections	exponential decay based on sparseness of graph theory calculating by structure of the semantic network ${\mathrm{I}}_{\mathrm{s}}=100\times {\mathrm{e}}^{-0.51\times {\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{n}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{s}}{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}\mathrm{s}}}}$
	temporal range (Critical)	the time over the semantic network	binary judgment based on meta-data
	spatial range (Critical)	the spatial over the semantic network	binary judgment based on meta-data
accuracy and reliability	semantic accuracy (Critical)	the correctness of knowledge content in the semantic web	ratio calculation based on tuples, anomalous tuples is identified by the Isolation Forest algorithm ${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{a}\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{o}\mathrm{u}\mathrm{s} \mathrm{t}\mathrm{u}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{t}\mathrm{u}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{s}}})$
	logical consistency (Optional)	the proportion of knowledge in the semantic web that conforms to logical rules	ratio calculation based on tuples, consistent tuple is identified by rule that no instance has conflicting values for the same property ${\mathrm{I}}_{\mathrm{s}}=100\times {\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{u}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{t}\mathrm{u}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{s}}}$
timeliness	update date (Critical)	the most recent update date	exponential decay based on data update date ${\mathrm{I}}_{\mathrm{s}}=100\times {\mathrm{e}}^{-\mathsf{\lambda }·∆\mathrm{t}}$ $∆\mathrm{t}=\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}-\mathrm{u}\mathrm{p}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}$ $∆\mathrm{t}$$\mathrm{unit}: \mathrm{years}, \mathsf{\lambda }$ default value: 0.23
completeness	schema completeness (Optional)	the degree of coverage of target domain concepts	ratio calculation based on concepts in the semantic web 2 ${\mathrm{I}}_{\mathrm{s}}=100\times {\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{p}\mathrm{t}\mathrm{s} \mathrm{i}\mathrm{n} \mathrm{t}\mathrm{h}\mathrm{e} \mathrm{s}\mathrm{e}\mathrm{m}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{c} \mathrm{w}\mathrm{e}\mathrm{b}}{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{p}\mathrm{t}\mathrm{s} \mathrm{i}\mathrm{n} \mathrm{t}\mathrm{h}\mathrm{e} \mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t} \mathrm{d}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{i}\mathrm{n}}}$
	loss rate (Critical)	the completeness of the data values in the semantic web	ratio calculation based on tuples values ${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{v}\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{t} \mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{p}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{s}}})$

¹ According to research by Google, a website response time of 10 s is the threshold for user attention and patience. ² The target domain concept set is constructed via a defined methodology (see Section 4.1.4).

.

4.1.5. Scientific Literature Indicators

Unlike the structured data mentioned earlier, scientific literature is typically presented as multimodal, semi-structured, and complex textual data. The assessment of scientific literature usability presents a unique challenge, as its ‘data’ is the published knowledge itself, rather than a direct measurement of phenomena. Therefore, the quality of scientific literature should be evaluated in terms of academic impact, reliability, and reproducibility. In practice, the evaluation often relies on proxies that signal credibility and impact.

Intrinsic quality indicators are designed to reflect the quality of the publication, including spatiotemporal information richness, impact, openness, and timeliness. Spatiotemporal information richness assesses applicability to user objectives via subject richness (diversity of key terms), proportion of spatiotemporal information, and temporal/spatial range. The evaluation of indicators involves textual content, which relies on the Natural Language Processing (NLP) techniques (e.g., Named Entity Recognition (NER) models). Impact includes download count (immediate interest), citation count (community endorsement over time), and social impact (e.g., altmetrics score, Douban score). For academic journals and their articles, indicators such as the journal impact factor (JIF) and citation frequency are widely recognized and easily accessible proxies for journal prestige in academic practice [94], although these indicators for assessing literature quality are controversial [95]. For academic books, the ideal metrics are library holding counts and media mention rates, often extremely difficult to obtain systematically. Therefore, in our framework, we employ more readily accessible proxies, such as the Douban score. Openness refers to adherence to open science practices, specifically whether data/code within the publication provides accessible links, improving reproducibility and dissemination [96,97]. Timeliness depends on the publication date, with newer publications generally holding higher value in fast-evolving fields.

Extrinsic indicators analyze the publishing platform, including authority and accessibility. Authority serves as a proxy for academic impact and user trust. For journals, the Journal Impact Factor (JIF) is a widely recognized, quantifiable proxy for prestige, despite its controversies. For publishers, the nature/level of the institution serves a comparable role. When selecting scientific literature, users typically prioritize sources from high-prestige journals, as this influences their initial judgments regarding the authority and credibility of the literature sources. Total downloads and citations of the platform reflect academic engagement. Accessibility evaluates the ease of access, including the acquisition method (e.g., open access status) and publication frequency.

The indicators and their value acquisition methods are listed in

Table 7. Indicators and acquisition methods of scientific literature usability assessment.

Primary Indicators	Secondary Indicators	Definition	Calculation Methods
authority	academic prestige (Critical)	the initial reference for prestige and influence of the journal. journal: journal impact factors serve as a proxy of journal reputation; publisher: the nature of publisher serves as a proxy of publisher reputation.	multi-level judgment based on impact factor (r) from API ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{r}\ge 5\\ \begin{array}{c}80,3\le \mathrm{r}<5\\ 60,1\le \mathrm{r}<3\end{array}\\ 40,\mathrm{r}<1\end{array}$ multi-level judgment based on publisher nature ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{n}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{a}\mathrm{u}\mathrm{t}\mathrm{h}\mathrm{o}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{s}\\ 80,\mathrm{t}\mathrm{o}\mathrm{p}-\mathrm{t}\mathrm{i}\mathrm{e}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y} \mathrm{a}\mathrm{n}\mathrm{d} \mathrm{p}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{s}\\ \begin{array}{c}60,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y} \mathrm{o}\mathrm{r} \mathrm{c}\mathrm{o}\mathrm{m}\mathrm{m}\mathrm{e}\mathrm{r}\mathrm{c}\mathrm{i}\mathrm{a}\mathrm{l} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{s}\\ 40,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{r}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{r}\mathrm{y} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{s}\end{array}\end{array}$
	total downloads (Optional)	the number of downloads for all articles in the journal and publisher	logarithmic scaling of downloads count (r) from API ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,15\times {\log }_{10}\left(\mathrm{r}+1\right))$
	total citations (Optional)	the number of times all articles in the journal and publisher have been cited	logarithmic scaling of downloads count (r) from API ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,18\times {\log }_{10}\left(\mathrm{r}+1\right))$
accessibility	acquisition method (Critical)	journal: whether the journal is open access; publisher: whether the publisher sells e-books.	binary judgment based on journal ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{o}\mathrm{p}\mathrm{e}\mathrm{n} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s},\mathrm{e}-\mathrm{b}\mathrm{o}\mathrm{o}\mathrm{k}\mathrm{s}\\ \begin{array}{c}50,\mathrm{n}\mathrm{o} \mathrm{o}\mathrm{p}\mathrm{e}\mathrm{n} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s},\mathrm{t}\mathrm{h}\mathrm{i}\mathrm{r}\mathrm{d} \mathrm{p}\mathrm{a}\mathrm{r}\mathrm{t}\mathrm{y}\mathrm{e}-\mathrm{b}\mathrm{o}\mathrm{o}\mathrm{k}\mathrm{s}\\ 0,\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{n}\mathrm{t} \mathrm{b}\mathrm{o}\mathrm{o}\mathrm{k}\mathrm{s}\end{array}\end{array}$
	publication frequency (Optional)	journal: how often a journal publishes new issues or volumes within a given time period publisher: the titles published annually	multi-level judgment based on publication regularity ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{s}\mathrm{e}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}/\mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}\\ 90,\mathrm{b}\mathrm{i}\mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}/\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{r}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{l}\mathrm{y}\\ \begin{array}{c}70,\mathrm{w}\mathrm{e}\mathrm{e}\mathrm{k}\mathrm{l}\mathrm{y}/\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{w}\mathrm{e}\mathrm{e}\mathrm{k}\mathrm{l}\mathrm{y}\\ 60,\mathrm{s}\mathrm{e}\mathrm{m}\mathrm{i}\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l}/\mathrm{a}\mathrm{n}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l}\end{array}\end{array}$ logarithmic scaling of titles published annually ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,20\times {\log }_{10}\left(\mathrm{r}+1\right)+30)$
spatiotemporal information richness	subject richness (Optional)	the variety of subject words in the literature or book	entropy formula of data patterns diversity
	proportion of spatiotemporal information (Critical)	the percentage of keywords that involve both spatial and temporal information	ratio calculation based on words in the literature, spatiotemporal words is identified by named entity recognition (NER) ${\mathrm{I}}_{\mathrm{s}}=100\times {\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l} \mathrm{w}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{w}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}}$
	spatial range (Critical)	the spatial extent in the literature or book	binary judgment based on spatial information extracted from the text content using NER
	temporal range (Critical)	the time scale in the literature or book	binary judgment based on temporal information extracted from the text content using NER
impact	download count (Optional)	the number of times articles, papers, or book are downloaded by users	standardization based on download count from API
	citation count (Critical)	the number of times an article, paper, or book has been cited by other academic publications	standardization based on citation count from API
	social impact (Optional)	literature: the altmetrics score of the literature book: Douban score of the book	literature: standardization based on altmetrics score, that is obtained from the Altmetric official website by searching literature book: standardization based on Douban score, that is obtained from the Douban website by searching book
openness	openness (Optional)	whether the data and code within the literature or book provide links for access	binary judgment based on whether the literature or book provide links for access
timeliness	publish date (Critical)	the publication date of literature or book	exponential decay based on publish date ${\mathrm{I}}_{\mathrm{s}}=100\times {\mathrm{e}}^{-\mathsf{\lambda }·∆\mathrm{t}}$ $∆\mathrm{t}=\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}-\mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}$ $∆\mathrm{t}$ unit: years $\mathrm{dynamic} \mathrm{research} \mathrm{fields}: \mathsf{\lambda }$ default value: 0.35 $\mathrm{broad} \mathrm{research} \mathrm{fields}: \mathsf{\lambda }$ default value: 0.15 $\mathrm{stable} \mathrm{research} \mathrm{fields}: \mathsf{\lambda }$ default value: 0.1

.

4.1.6. Web Texts Indicators

Web texts, as the primary textual data on the Internet, are characterized by volume, velocity, informality, and interactivity. The quality assessment focuses on dimensions such as accuracy, timeliness, and credibility, involving perspectives from information content, user relationships, and data sources [48,49]. In the specific implementation of indicators, we divide texts into two categories: PGC and UGC. There could be different evaluation methods for different types of texts.

Intrinsic quality indicators include accuracy and reliability, impact, timeliness, information density, the spatiotemporal information existence, and spatiotemporal information granularity. Accuracy and reliability measure publisher credibility through publisher identity and follower count. Information published by verified official accounts is generally accorded higher reliability. Impact reflects dissemination level and interactivity via clicks, shares, likes, and comments. Timeliness is important for rapidly changing internet information, including publication time and valid status (e.g., for policies).

Given the sparsity and noise of web texts, information density (effective general information per unit text) is essential, and relies on simple pre-extraction of text. From a spatiotemporal perspective, web texts are social perception records attached to spatiotemporal signals with different confidence levels. Its quality assessment needs to consider the existence and explicitness. Spatiotemporal information existence measures whether any time/space information is present, using API or NER models, and determines its suitability for spatial-temporal analysis. Spatiotemporal information granularity assesses the detail level of time and space, determining support for fine-grained analysis. The level of detail is categorized as high granularity, medium granularity, or low granularity (or ultra-low granularity).

Extrinsic indicators provide an initial source evaluation of the website platform, including authority, accessibility, and spatiotemporal information richness. Authority is assessed by website type (e.g., .gov, .edu), daily average page views (PV), and daily average user visits (UV), reflecting user engagement. Accessibility is measured by update frequency, response time, and domain duration, indicating service continuity and stability. Industry research often uses 20 years of operation as a threshold for authoritative sites. Based on WebArchive data, which shows such sites have 30% higher content authenticity and 45% greater data continuity, a scoring gradient is applied: sites accrue 5 points per year, capping at 100 points for 20 years to denote established authority. Spatiotemporal information richness includes the proportion of spatiotemporal information and its spatial/temporal range. For PGC (e.g., news, policies), ranges refer to applicable regions/periods or event locations/times. For UGC, they are represented by location tags and timestamps.

The indicators and their value acquisition methods are listed in

Table 8. Indicators and acquisition methods of web texts’ usability assessment.

Primary Indicators	Secondary Indicators	Definition	Calculation Methods
authority	website type (Critical)	the type of website	multi-level judgment based on website ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{g}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{w}\mathrm{e}\mathrm{b}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{e}\\ \begin{array}{c}100,\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{w}\mathrm{e}\mathrm{b}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{e}\\ 70,\mathrm{s}\mathrm{o}\mathrm{c}\mathrm{i}\mathrm{a}\mathrm{l} \mathrm{w}\mathrm{e}\mathrm{b}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{e}\end{array}\\ \begin{array}{c}50,\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{r}\mathrm{y} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{c} \mathrm{w}\mathrm{e}\mathrm{b}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{e}\\ 25,\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{o}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	daily average user visits (UV) (Optional)	the average of users who visit the website per day	log-logistic regression model based on UV (r) from API 1 ${\mathrm{I}}_{\mathrm{s}}=100/\left(1+{\mathrm{e}}^{\left(-0.9\times \left({\log }_{10}\left(\mathrm{r}+1\right)-4.5\right)\right)}\right)$
	daily average page views (PV) (Optional)	the average of times the webpage is viewed per day	log-logistic regression model based on PV (r) from API 1 ${\mathrm{I}}_{\mathrm{s}}=100/\left(1+{\mathrm{e}}^{\left(-0.9\times \left({\log }_{10}\left(\mathrm{r}+1\right)-4.5\right)\right)}\right)$
accessibility	release frequency (Optional)	how frequently the website is updated or refreshed with new information	multi-level judgment based on data release regularity ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{h}\mathrm{i}\mathrm{g}\mathrm{h}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{d}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{y}/\mathrm{w}\mathrm{e}\mathrm{e}\mathrm{k}\mathrm{l}\mathrm{y})\\ 80,\mathrm{l}\mathrm{o}\mathrm{w}-\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}(\mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{y}/\mathrm{y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{l}\mathrm{y})\\ \begin{array}{c}60,\mathrm{b}\mathrm{e}\mathrm{y}\mathrm{o}\mathrm{n}\mathrm{d} \mathrm{a}\mathrm{n}\mathrm{n}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{s}\\ 0,\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{u}\mathrm{e}\mathrm{d}\end{array}\end{array}$
	response efficiency (Critical)	the average time it takes for the website to respond to users’ requests	linear decay based on average response time (r) of website(in seconds) 2 ${\mathrm{I}}_{\mathrm{s}}=100\times \left(1-{\displaystyle \frac{\mathrm{r}}{10}}\right)$
	domain duration (Optional)	the length of time for which a domain name is registered	calculate based on domain duration (r) ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,\mathrm{r}\times 5)$
spatiotemporal information richness	subject richness (Optional)	the variety of subject words in the web texts	entropy formula of data values diversity
	proportion of spatiotemporal information (Critical)	the percentage of keywords that involve both spatial and temporal information	ratio calculation based on words in the text, spatiotemporal words is identified by named entity recognition (NER) ${\mathrm{I}}_{\mathrm{s}}=100\times {\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l} \mathrm{w}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{w}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}}$
	spatial range (Critical)	professionally generated content (PGC): applicable regions of policies and regulations, occurrence location of News events via NER user-generated content (UGC): address tag	binary judgment based on spatial information
	temporal range (Critical)	PGC: effective time of policies and regulations, occurrence time of News events via NER UGC: timestamp	binary judgment based on temporal information
information density	information density (Optional)	the number of key information points contained per unit (100 words) of web texts	ratio calculation based on words in the text, keywords is identified by simple pre-processing 3 ${\mathrm{I}}_{\mathrm{s}}=100\times {\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{k}\mathrm{e}\mathrm{y}\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s} \mathrm{p}\mathrm{e}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{i}\mathrm{t}}{25}}$
spatiotemporal information existence	spatiotemporal information existence (Critical)	whether the web texts contain any form of time or space information	multi-level judgment based on information form ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}(\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{m}\mathrm{p}/\mathrm{t}\mathrm{a}\mathrm{g})\\ 50,\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}(\mathrm{i}\mathrm{n} \mathrm{t}\mathrm{h}\mathrm{e} \mathrm{t}\mathrm{e}\mathrm{x}\mathrm{t}\mathrm{s})\\ 0,\mathrm{a}\mathrm{b}\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{e}\end{array}$
spatiotemporal information granularity	temporal granularity (Critical)	the level of time detail	multi-level judgment based on temporal information 4 ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{h}\mathrm{i}\mathrm{g}\mathrm{h} \mathrm{g}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\\ 60,\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{u}\mathrm{m} \mathrm{g}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\\ 20,\mathrm{l}\mathrm{o}\mathrm{w} \mathrm{g}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\end{array}$
	spatial granularity (Critical)	the level of space detail	multi-level judgment based on spatial information 5 ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{h}\mathrm{i}\mathrm{g}\mathrm{h} \mathrm{g}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\\ 70,\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{u}\mathrm{m} \mathrm{g}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\\ \begin{array}{c}50,\mathrm{l}\mathrm{o}\mathrm{w} \mathrm{g}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\\ 20,\mathrm{u}\mathrm{l}\mathrm{t}\mathrm{r}\mathrm{a}-\mathrm{l}\mathrm{o}\mathrm{w} \mathrm{g}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\end{array}\end{array}$
accuracy and reliability	user followers count (Optional)	the number of followers	logarithmic scaling of followers count (r) ${\mathrm{I}}_{\mathrm{s}}=100/\left(1+{\mathrm{e}}^{\left(-0.9\times \left({\log }_{10}\left(\mathrm{r}+1\right)-4.5\right)\right)}\right)$
	publisher nature (Critical)	the nature of text’s publisher	multi-level judgment based on publisher ${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{g}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\\ \begin{array}{c}100,\mathrm{V}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{o}\mathrm{r}\mathrm{g}\mathrm{a}\mathrm{n}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\\ 70,\mathrm{V}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{p}/\mathrm{r}\mathrm{e}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\end{array}\\ \begin{array}{c}50,\mathrm{V}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l}\\ 25,\mathrm{O}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r} \mathrm{a}\mathrm{c}\mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\end{array}\end{array}$
impact	Clicks (Optional)	the total number of times users click on web texts	logarithmic scaling of clicks count (r) ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,15\times {\log }_{10}\left(\mathrm{r}+1\right))$
	Shares (Optional)	the total number of times users have shared web texts with others	logarithmic scaling of shares count (r) ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,10\times {\log }_{10}\left(\mathrm{r}+1\right))$
	likes and comments (Optional)	the total number of likes and comments web texts receive	logarithmic scaling of likes count (r) ${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,10\times {\log }_{10}\left(\mathrm{r}+1\right))$
timeliness	publish date (Critical)	the publication time of web texts	exponential decay based on publish time ${\mathrm{I}}_{\mathrm{s}}=100\times {\mathrm{e}}^{-\mathsf{\lambda }·∆\mathrm{t}}$ $∆\mathrm{t}=\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}-\mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}$ $∆\mathrm{t}$$\mathrm{unit}: \mathrm{daily}, \mathsf{\lambda }$ default value: 0.05
	valid status (Critical)	whether the policy is in effect, suspended, expired, or under revision	binary judgment based on valid status

¹ The metric is calculated using a log-logistic regression model. This approach incorporates a logarithmic transformation of the raw input value, which is then mapped through a sigmoid (S-curve) function. The model elegantly captures the characteristic growth dynamics typical of website development. ² According to research by Google, a website response time of 10 s is the threshold for user attention and patience. ³ Linguistic theory establishes 25 as the theoretical maximum for information density. ⁴ Temporal Granularity: high granularity is precise timestamps (second/minute); medium granularity is dates and time periods; low granularity is relative time (e.g., recently) and seasons. ⁵ Spatial Granularity: high granularity is precise coordinates and specific addresses; medium granularity is districts, counties, and towns; low granularity is cities and provinces; ultra-low granularity is regions.

.

4.2. Methods

In the previous discussion, crowdsensing data was categorized into six types, each corresponding to a specific set of assessment indicators. The usability assessment indicators for each type were organized into two levels: the first level consists of several primary indicators (${\mathrm{I}}_{\mathrm{f}}$), and each primary indicator includes multiple secondary indicators (${\mathrm{I}}_{\mathrm{s}}$). Taking specialized spatial data as an example, Figure 3 illustrates the structure of usability assessment indicators, with orange denoting data sources, blue representing data content, solid boxes indicating primary indicators, and dotted boxes indicating secondary indicators. This study provides detailed definitions of the indicators and related descriptions in the previous section. The following research methods are discussed from two perspectives, indicator calculation and indicator synthesis, considering the computational requirements involved in the usability assessment process.

4.2.1. Indicator Calculation Methods

The indicators calculation transforms multi-source heterogeneous raw data into standardized measurable values, forming the basis for comprehensive assessment. This study adopts the following four types of calculation methods based on the core mathematical and logical connotations of the indicators (details of the calculation methods for each data indicator can be found in Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8).

• Judgment-based Methods;

Judgment-based methods rely on logical reasoning or rule matching to map qualitative information or data states to discrete rating levels, suitable for determining compliance or category type. It depends on the domain knowledge to establish judgment rules, with a simple and highly interpretable calculation process.

Binary judgment assigns 100 for compliance and 0 otherwise, determining whether the data state meets preset requirements. This method originates from the concept of “consistency” in international geographic information standards (e.g., ISO 19157). It is used for rigid constraints like spatiotemporal range, accuracy errors, data openness, or validity status.

Multi-level judgment maps qualitative descriptions to quantitative scores based on preset grading standards and rules, which are based on domain research and industry consensus. It is used by indicators, such as entity nature, data acquisition method, release frequency, mobile entity, spatiotemporal information existence/granularity, and publisher nature. For example, the entity nature of the data provider is scored as: government agencies = 100, professional institutions = 100, business organizations = 70, academic/public teams = 50, individuals/unlabeled = 25. The government agencies are entities with official designations with terms such as bureau, ministry, or .gov domain suffix. The professional institutions are recognized national or international professional bodies, or research academies, typically identified via inclusion in authoritative official registries. The business organizations are commercial entities indicated by .com or .co domain suffixes, or keywords like Inc., Ltd., or company. The academic or public teams are groups affiliated with .edu/.org domains or public collaborations. The individual/unlabeled are sources lacking a clear institutional affiliation. The formula is as follows:

$${\mathrm{I}}_{\mathrm{s}}=\{\begin{array}{c}100,\mathrm{g}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{a}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{s}\\ \begin{array}{c}100,\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\\ 70,\mathrm{b}\mathrm{u}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{s}\mathrm{s} \mathrm{o}\mathrm{r}\mathrm{g}\mathrm{a}\mathrm{n}\mathrm{i}\mathrm{z}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}\end{array}\\ \begin{array}{c}50,\mathrm{a}\mathrm{c}\mathrm{a}\mathrm{d}\mathrm{e}\mathrm{m}\mathrm{i}\mathrm{c} \mathrm{t}\mathrm{e}\mathrm{a}\mathrm{m} \mathrm{o}\mathrm{r} \mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{c}\\ 25,\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l} \mathrm{o}\mathrm{r} \mathrm{u}\mathrm{n}\mathrm{l}\mathrm{a}\mathrm{b}\mathrm{e}\mathrm{l}\mathrm{e}\mathrm{d}\end{array}\end{array},$$

(1)

2.

Statistical Methods;

Statistical methods are based on probability theory and mathematical statistics, extracting quantitative characteristics from data value distributions. The core concepts include counting, proportion, and distribution measurements, suitable for measuring intrinsic state attributes.

Ratios and proportions quantify data status by calculating the proportion of defects or specific components, which is a common method for measuring completeness and error rates in standards such as ISO 19157 [58]. For example, the accuracy and reliability of IoT sensing data is described by the anomaly ratio:

$${\mathrm{I}}_{\mathrm{s}}=100\times (1-{\displaystyle \frac{\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{a}\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{o}\mathrm{u}\mathrm{s} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{s}}}),$$

(2)

Distribution measurement uses information entropy to measure the diversity and uncertainty of data values. A higher entropy value indicates that the data values are more evenly distributed and unpredictable, implying richer information. Applicable indicators include the diversity of data patterns, calculated based on the frequency of occurrence of each data type. For example, the attribute type diversity of specialized spatial data. The formula for entropy is as follows, where H(T) represents the entropy of attribute type T, p(t) is the probability of attribute type t, m is the count of all attribute type T, and H′(T) is the standardization entropy.

$$\mathrm{H}\left(\mathrm{T}\right)=-\sum \left(\mathrm{p}\left(\mathrm{t}\right)\ast {\log }_2\left(\mathrm{p}\left(\mathrm{t}\right)\right)\right).$$

(3)

$${\mathrm{H}}^{\prime }\left(\mathrm{T}\right)=100\times \mathrm{H}\left(\mathrm{T}\right)/{\log }_2\mathrm{m}$$

(4)

Another type of indicator focuses on the richness of internal data values. The average entropy of each attribute’s values reflects the overall information content of the data. For discrete attributes (including categorical and discrete numerical values), entropy is computed directly on the observed values. For continuous numerical attributes, the values are first discretized into bins using equal-width binning, with the number of bins determined by Sturges’ rule ($k=\lceil {\log }_{2}N\rceil +1$, where N is the number of distinct values), to enable probability estimation. For example, the data value richness of IoT sensing data. The formula for this is as follows, where H(X) is the entropy of the value of attribute X, p(x) is the probability of a specific value x of attribute X, and m is the count of possible values of attribute X. F represents the average standardization entropy of all attributes X.

$$\mathrm{H}\left(\mathrm{X}\right)=-\sum \left(\mathrm{p}\left(\mathrm{x}\right)\ast {\log }_2\left(\mathrm{p}\left(\mathrm{x}\right)\right)\right),$$

(5)

$${\mathrm{H}}^{\prime }\left(\mathrm{X}\right)=100\times \mathrm{H}\left(\mathrm{X}\right)/{\log }_2\mathrm{m}$$

(6)

$$\mathrm{F}={\displaystyle \frac{1}{\mathrm{n}}}\ast \sum {\mathrm{H}}^{\prime }\left(\mathrm{X}\right)$$

(7)

3.

Transformation Methods;

Transformation methods map and transform raw data using mathematical functions to eliminate dimensional differences, adjust scales, and simulate patterns. It is suitable for situations where raw data exhibit large differences in dimensionality and numerical ranges, necessitating mapping them to a unified, dimensionless scoring range. The key parameters within these methods (e.g., scaling factors in logarithmic functions) were calibrated based on statistical benchmarking from representative datasets, ensuring scientific validity and cross-dataset comparability.

Logarithmic scaling uses logarithmic functions to non-linearly transform raw scale indicators to simulate the diminishing marginal utility pattern, which aligns better with human cognition. Indicators suitable for this method usually present extreme skewness (e.g., power-law distribution) in their absolute scale values. For instance, the user count and download counts of data sources, as represented in the formula:

$${\mathrm{I}}_{\mathrm{s}}=\mathrm{m}\mathrm{i}\mathrm{n}(100,25\times {\log }_{10}\left(\mathrm{u}\mathrm{s}\mathrm{e}\mathrm{r} \mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}+1\right)),$$

(8)

For certain indicators that model growth with saturation, such as daily average page views and daily average user visits for websites, a log-logistic function is employed. This sigmoidal model effectively maps the logarithmically transformed raw data into a saturated scoring range (0–100). The calculation follows the formula:

$${\mathrm{I}}_{\mathrm{s}}=100/\left(1+{\mathrm{e}}^{\left(-0.9\times \left({\log }_{10}\left(\mathrm{r}\mathrm{a}\mathrm{w} \mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}+1\right)-4.5\right)\right)}\right),$$

(9)

Exponential decay models simulate the decrease in data value over time, and are mainly used to calculate data timeliness. The formula is as follows, taking the timeliness of trajectory data as an example. $∆\mathrm{t}$ is the time difference between the current time and the data’s publish time. When the application is real-time, the unit of $∆\mathrm{t}$ is hours, and the default value of $\mathsf{\lambda }$ is 0.1. When the application is general, the unit of $∆\mathrm{t}$ is daily, and the default value of $\mathsf{\lambda }$ is 0.01.

$${\mathrm{I}}_{\mathrm{s}}=100\times {\mathrm{e}}^{-\mathsf{\lambda }·∆\mathrm{t}},$$

(10)

$$∆\mathrm{t}=\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}-\mathrm{p}\mathrm{u}\mathrm{b}\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{h} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}$$

(11)

Additionally, there is a specialized indicator for website response timeliness, focusing on the satisfaction of user experience with page response speed. According to research by Google, a website response time of 10 s is the threshold for user attention and patience. Based on this, the linear decay model is used to quantify this indicator. The specific formula is as follows:

$${\mathrm{I}}_{\mathrm{s}}=100\times \left(1-{\displaystyle \frac{\mathrm{a}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e} \mathrm{r}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{e} \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}(\mathrm{i}\mathrm{n} \mathrm{s}\mathrm{e}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{s})}{10}}\right),$$

(12)

Using the concept of z-scores in statistics, the relative position of raw data within a dataset is calculated. Standardization eliminates the influence of overall distribution, reflecting the “relative quality” of individual data in the population, making it especially suitable for indicators such as citation counts or download counts that show significant variation within a discipline. The specific formula is as follows:

$${\mathrm{I}}_{\mathrm{s}}=50+10\ast {\displaystyle \frac{\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}-\mathsf{\mu }}{\mathsf{\sigma }}},\mathsf{\mu } \mathrm{i}\mathrm{s} \mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n},\mathsf{\sigma } \mathrm{i}\mathrm{s} \mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{r}\mathrm{d} \mathrm{d}\mathrm{e}\mathrm{v}\mathrm{i}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n},$$

(13)

4.

Algorithmic Model Methods;

Algorithmic model methods rely on specialized algorithmic models to extract features or perform complex judgments from raw data. These methods are designed to solve problems that traditional statistical methods cannot directly handle, such as semantic understanding, pattern recognition, and deep anomaly detection. The use of these models represents a necessary and optimized practice for specific technical tasks, drawing on validated tools from computer science.

For trajectory data, the calculation of positioning accuracy relies on drift detection algorithms (e.g., Kalman filters) to identify anomalous points. The response efficiency of the GSW is measured through standardized query tests to obtain the average response time. To assess the semantic accuracy of a GSW, the Isolation Forest algorithm is utilized for its efficiency and linear time complexity in detecting anomalous tuples within the knowledge graph. For scientific literature and web texts, spatiotemporal words are recognized by NER in the calculation of the proportion of spatiotemporal knowledge. Furthermore, for text data’s spatiotemporal extents, a binary judgment is made by evaluating whether the identified spatiotemporal information or words are relevant to the target application’s spatiotemporal scope.

5.

Handling Calculation Issues;

In practical applications, the following critical issues must be noted during the calculation process.

Unified Quantification of Indicators.

This study addresses the quantification of indicators through the methods discussed above. Qualitative descriptions are transformed into quantitative values based on rule-based assignments, while the raw objective data of quantitative indicators are converted into evaluation values through various calculations. However, the value ranges of different indicators may vary significantly. For instance, the range of “response time” for a website typically spans from 0 to 10 s, whereas “daily average user visits” is usually measured in tens of thousands. Therefore, the diverse secondary indicators are mapped to a unified interval of [0, 100]. The normalization methods include Min-Max normalization, standardization, and other methods (as described in the previous section).

Semantic Differences in Indicator Value Trends.

The framework mandates that a higher score always indicates better quality. For indicators where the raw value is negatively correlated with quality (e.g., accuracy and reliability, response time), mathematical transformations (e.g., subtraction from 1, decay functions) are applied to invert the trend direction prior to integration. This approach eliminates integration bias caused by semantic differences, ensuring that indicators align in terms of trends.

Handling Missing Data in Indicators.

Practical evaluation must contend with missing values. To ensure the scientific validity and robustness of the evaluation results, data missing for different reasons are treated distinctly.

If an indicator is logically inapplicable to a specific data instance, it is excluded, and its weight is proportionally redistributed among other secondary indicators within the same primary category. For example, evaluating “spatial resolution” in vector data or assessing “temporal resolution” in user-uploaded trajectory data. If there are n secondary indicators under a primary indicator, and the k-th indicator is missing with weight w_k, the new weight for the other indicators w_i′ is calculated as:

$${\mathrm{w}}_{\mathrm{i}}^{\prime }={\mathrm{w}}_{\mathrm{i}}+{\mathrm{w}}_{\mathrm{i}}\times {\displaystyle \frac{{\mathrm{w}}_{\mathrm{k}}}{{\sum }_{\mathrm{j}=1,\mathrm{j}\ne \mathrm{k}}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{j}}}},$$

(14)

This method prevents assigning a value of zero to an attribute that logically does not exist, ensuring the practical applicability and fairness of the evaluation.

If an indicator is logically applicable but its value is missing, statistical imputation methods are used. Specifically, for numerical indicators, methods like mean, median, or regression interpolation are employed, while for categorical indicators, mode interpolation is applied. Interpolation retains the maximum amount of information and statistical power in the dataset, mitigating potential biases during the analysis.

For binary-judgment indicators with missing metadata, a tiered strategy is proposed: first attempting inference from available data content (e.g., inferring a coordinate system from data values); if impossible, assigning a conservative default score (e.g., 50) to signify “unverified,” accompanied by an uncertainty flag to maintain transparency. The choice of strategy depends on the indicator’s criticality and the application’s risk tolerance.

4.2.2. Indicator Synthesis Methods

Indicator synthesis methods consolidate the unified quantified scores of various fundamental indicators, generating a comprehensive usability assessment. The weighted evaluation method used in this study is a widely adopted multi-criteria decision-making technique [98,99]. This method is computationally simple, easy to understand, and provides an intuitive representation of how each indicator influences the final evaluation result, thus enhancing interpretability. The weighted evaluation method accounts for the diversity between indicators while avoiding biases that may arise from simple summation. Furthermore, the indicator hierarchy in the proposed evaluation framework aligns well with the layered weighting and aggregation process of the weighted evaluation method, making it suitable for implementation.

Based on the indicator structure presented in this study, the comprehensive assessment results were calculated through weighted aggregation at each level after determining the weights for each indicator. In addition, the computation of the weighted evaluation requires assigning a weight ($\mathsf{\theta }$) to each indicator to determine its relative importance (as detailed in Section 4.2.3). The following formulas were used for the calculations. Formula (15) describes the calculation of primary indicators ${\mathrm{I}}_{\mathrm{f}}$. In this formula, ${\mathrm{I}}_{{\mathrm{s}}_{\mathrm{j}}}$ is the score for each secondary indicator associated with the primary indicator ${\mathrm{I}}_{\mathrm{f}}$, and ${\mathsf{\theta }}_{{\mathrm{s}}_{\mathrm{j}}}$ denotes the weight corresponding to the secondary indicator. Formula (16) describes the calculation of the overall assessment result ${\mathrm{Q}}_{\mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}}$, where ${\mathrm{I}}_{{\mathrm{f}}_{\mathrm{i}}}$ is the score for each primary indicator calculated using Formula (15), and ${\mathsf{\theta }}_{{\mathrm{f}}_{\mathrm{i}}}$ denotes the weight corresponding to the primary indicator. The scores are aggregated to assess this category of data.

$$I_f=\sum \left({\theta }_{s_j}· I_{s_j}\right), j\in \left\{1, 2,\dots \right\} ,$$

(15)

$$Q_{Data}=\sum \left( {100·I}_{f_i}·{\theta }_{f_i}\right), i\in \left\{1, 2,\dots \right\}$$

(16)

To facilitate intuitive interpretation and decision-making, the continuous usability score Q is mapped to a discrete usability grade. When 90 ≤ ${\mathrm{Q}}_{\mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}}$ ≤ 100, the usability level is classified as “Excellent”; when 70 ≤ ${\mathrm{Q}}_{\mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}}$ < 90, it is “Moderate”; when 60 ≤ ${\mathrm{Q}}_{\mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}}$ < 70, it is “Fair”; when 30 ≤ ${\mathrm{Q}}_{\mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}}$ < 60, it is “Poor”; and when 0 ≤ ${\mathrm{Q}}_{\mathrm{D}\mathrm{a}\mathrm{t}\mathrm{a}}$ < 30, it is “Very Poor.”

This scoring scheme is based on common practices used in educational scoring and system evaluation scales, which are widely understood and accepted [100,101]. The thresholds, particularly the critical scores of 60 for “Pass/Fail” (representing “Fair” and “Poor”) and 90 for “Excellent,” are designed to provide clear and actionable benchmarks for data selection and filtering. While these thresholds are based on conventional practices, their practical effectiveness and sensitivity are further validated through sensitivity analysis in Section 6.2.

4.2.3. Weight Determination

The determination of indicator weights is a critical step to ensure the composite evaluation accurately reflects their relative importance in the application context. Following the “Application-Oriented” principle, the weights were designed to mirror the significance of each indicator within its specific application context, such as constructing a spatiotemporal knowledge graph.

In weight determination, several alternative approaches (e.g., entropy method, Bayesian weighting) exist but rely heavily on the availability, scale, and distribution of the underlying data. Given the heterogeneous nature of crowdsensing data, the indicators vary greatly in terms of nature, scale, and semantic meaning. Hence, purely data-driven approaches are not suitable for uniform application across all indicators. Furthermore, some indicators are difficult to quantify or lack sufficient historical data for reliable estimation. Therefore, we employed the AHP combined with the Delphi method for expert consultation. This hybrid approach integrates expert knowledge to systematically analyze heterogeneous indicators, providing a weight scheme that balances theoretical rigor with practical applicability.

The combination of the AHP and the Delphi method incorporates both quantitative calculations and subjective judgments, achieving a scientifically rigorous and objective decision-making process [102,103]. AHP effectively converts qualitative expert assessments of indicator importance into quantitative weights, while consistency checks verify the logical coherence of judgments, reducing subjective arbitrariness.

The evaluation aims to measure the applicability of crowdsensing data for spatiotemporal knowledge graph construction. Based on the evaluation framework, a hierarchical model was built: the primary indicators form the criterion layer, serving as the foundational factors for achieving the evaluation objectives; and the secondary indicators constitute the sub-criterion layer, providing specific quantitative measures.

Ten experts, all holding senior professional titles (≥10 years of experience) in geographic information science, data quality, or related fields, were invited. Their collective expertise comprehensively covers all six data categories assessed by the framework. The panel comprised three GIS researchers from Geographic Information Science from research institutions, three senior engineers from the National Quality Inspection and Testing Center for Surveying and Mapping Products, two researchers from the Moganshan Geospatial Information Laboratory, and two university professors in surveying engineering. During the consultation, experts first reviewed the indicator system for theoretical soundness, then judged the relative importance of indicators within the hierarchical model using a 1–9 scale method, following the classic two-round Delphi procedure with feedback. The resulting judgment matrices were aggregated, and the final weight assignments showed low standard deviations (primarily 0.02–0.18), indicating high expert consensus on the weighting scheme.

To avoid inconsistencies resulting from misjudgments, we conducted rigorous consistency checks on the judgment matrices at each level. First, the Consistency Index (CI) was calculated using Formula (17). Then, the Consistency Ratio (CR) was obtained by comparing the CI with the Random Index (RI). The specific formulas are as follows:

$$\mathrm{C}\mathrm{I}=\left({\mathsf{\lambda }}_{\mathrm{m}\mathrm{a}\mathrm{x}}-\mathrm{n}\right)/\left(\mathrm{n}-1\right),$$

(17)

$$CR=CI/RI$$

(18)

where λ_max is the maximum eigenvalue of the judgment matrix; n is the number of evaluation criteria; and RI is the random consistency index, which can be obtained from

Table 9. Random Consistency Index (RI) Table.

n	1	2	3	4	5	6	7	8	9
RI	0	0	0.58	0.90	1.12	1.24	1.32	1.41	1.45

based on the matrix order.

A CR value of less than 0.1 was adopted as the threshold for acceptable consistency. In this study, the CR values of all judgment matrices were found to be less than 0.1, indicating that the expert judgments were logically consistent and the results valid. For instance, the primary indicator matrix for specialized spatial data is presented in

Table 10. The specialized spatial data primary indicator matrix.

	Authority	Accessibility	Spatiotemporal Information Richness	Spatial Resolution	Temporal Resolution	Timeliness	Completeness	Accuracy & Reliability
authority	1	5	2	2	4	4	3	1/2
accessibility	1/5	1	1/4	1/4	1	1	1/3	1/8
spatiotemporal information richness	1/2	4	1	1	3	3	2	1/5
spatial resolution	1/2	4	1	1	3	3	2	1/5
temporal resolution	1/4	1	1/3	1/3	1	1	1/2	1/7
timeliness	1/4	1	1/3	1/3	1	1	1/2	1/7
completeness	1/3	3	1/2	1/2	2	2	1	1/6
accuracy and reliability	4	8	5	5	7	7	6	1

, with a CR value of 0.0276. This matrix is employed to determine the relative importance of the eight primary indicators under the “specialized spatial data” category. The geometric mean of the expert judgments was taken to form a comprehensive judgment matrix, and the normalized weights were calculated using the eigenvalue method. Ultimately, the weight scheme for this framework was derived (details in

Table 11. A reference scheme for the weight of Indicators.

Categories	Primary Indicators	$\mathbf{Total} \mathbf{Weights} \left({\mathit{\theta }}_{\mathbf{f}}\right)$	Secondary Indicators	$\mathbf{Weight} \left({\mathit{\theta }}_{\mathbf{s}}\right)$	Categories	Primary Indicators	$\mathbf{Total} \mathbf{Weights} \left({\mathit{\theta }}_{\mathbf{f}}\right)$	Secondary Indicators	$\mathbf{Weight} \left({\mathit{\theta }}_{\mathbf{s}}\right)$
specialized spatial data	authority	0.2044	entity nature	0.6033	geographic semantic web	authority	0.1915	entity nature	0.7750
			user count	0.1649				user count	0.2250
			download volume	0.2318		accessibility	0.0872	release frequency	0.2500
	accessibility	0.0817	release frequency	0.5416				response efficiency	0.7500
			acquisition method	0.4584		spatiotemporal information richness	0.1631	tuple count	0.1805
	spatiotemporal information richness	0.0927	attribute type diversity	0.2589				proportion of spatiotemporal tuples	0.0937
			data value richness	0.2313				attribute type diversity	0.1352
			temporal range	0.2549				structural richness	0.2032
			spatial range	0.2549				temporal range	0.1937
	spatial resolution	0.1192	spatial resolution	0.5000				spatial range	0.1937
			scale	0.5000		accuracy and reliability	0.2963	semantic accuracy	0.3750
	temporal resolution	0.0845	temporal resolution	1.0000				logical consistency	0.6250
	timeliness	0.0364	publish date	1.0000		timeliness	0.0591	update date	1.0000
	completeness	0.0588	attribute items completeness	0.3750		completeness	0.2028	schema completeness	0.3750
			records completeness	0.6250				loss rate	0.6250
	accuracy and reliability	0.3223	planar errors	0.3889	scientific literature	authority	0.3382	journal impact factors	0.5000
			elevation errors	0.3888				total journal downloads	0.2500
			temporal errors	0.2223				total journal citations	0.2500
IoT sensing data	authority	0.2133	entity nature	0.6031		accessibility	0.0818	acquisition method	0.7500
			user count	0.1651				publication frequency	0.2500
			download volume	0.2318		spatiotemporal information richness	0.1629	subject richness	0.2495
	accessibility	0.0960	release frequency	0.7500				proportion of spatiotemporal knowledge	0.5595
			acquisition method	0.2500				spatial range	0.0955
	spatiotemporal information richness	0.0987	sensor component	0.2821				temporal range	0.0955
			data value richness	0.2421		impact	0.1625	download count	0.1365
			temporal range	0.2379				citation count	0.2385
			spatial range	0.2379				altmetrics	0.6250
	spatial resolution	0.0861	spatial resolution	1.0000		openness	0.0971	openness	1.0000
	temporal resolution	0.0861	temporal resolution	1.0000		timeliness	0.1575	publish date	1.0000
	timeliness	0.0309	publish date	1.0000	web texts	authority	0.1384	website nature	0.5921
	completeness	0.0848	records completeness	1.0000				daily average user visits	0.2404
	accuracy and reliability	0.3041	anomalous proportion	0.4485				daily average page views	0.1675
			measurement errors	0.3698		accessibility	0.0501	release frequency	0.4107
			temporal errors	0.1817				response efficiency	0.3928
trajectory data	authority	0.2054	entity nature	0.6031				domain duration	0.1965
			user count	0.1651		spatiotemporal information richness	0.0459	subject richness	0.3083
			download volume	0.2318				proportion of spatiotemporal information	0.3709
	accessibility	0.0954	release frequency	0.7916				spatial range	0.1604
			acquisition method	0.2084				temporal range	0.1604
	mobile entity	0.0523	mobile entity	1.0000		information density	0.0449	information density	1.0000
	spatiotemporal information richness	0.1258	semantic richness	0.3817		spatiotemporal information existence	0.2241	spatiotemporal information existence	1.0000
			temporal range	0.3698		spatiotemporal information granularity	0.2241	temporal granularity	0.3333
			spatial range	0.2485				spatial granularity	0.6667
	spatial resolution	0.0999	sampling point density	1.0000		accuracy and reliability	0.0564	user followers count	0.2084
	temporal resolution	0.0518	sampling interval	1.0000				publisher nature	0.7916
	timeliness	0.0413	publish date	1.0000		impact	0.0659	clicks	0.3023
	completeness	0.0957	loss rate	1.0000				shares	0.4435
	accuracy and reliability	0.2324	positioning accuracy	1.0000				likes and comments	0.2542
						timeliness	0.1502	publish time	0.3750
								valid status	0.6250

).

5. Application

To systematically validate the effectiveness of the proposed framework in a real-world scenario, this study conducted usability evaluation cases on six categories of crowdsensing data, based on the project “Collaborative Spatiotemporal Knowledge Graph and Knowledge Services”. It should be emphasized that the presented case study serves primarily as a proof-of-concept demonstration of the framework’s operability and its capacity to generate differentiated, interpretable assessments. Its goal is to illustrate practical utility, not to provide statistical inference, which would require evaluation on a large, homogeneous dataset—a valuable direction for future work. The cases were centered on the core application scenario of constructing a spatiotemporal knowledge graph to serve land use planning in the Qinling region of Shaanxi Province. Six representative real-world datasets were selected as evaluation objects (details in

Table 12. Data Usability Assessment Results.

Data Type	Dataset	Usability Score	Grade	Data Characteristics
Specialized Spatial Data	2023 County-level Administrative Division Data	92.45	Excellent	Extremely high authority and good accuracy, making it ideal data for constructing the spatial benchmark of the knowledge graph.
IoT Sensing Data	2023 Hourly National Air Quality Station Monitoring Data	90.46	Excellent	High spatiotemporal resolution, good timeliness, low accuracy and reliability, and very suitable for dynamic environmental monitoring.
Trajectory Data	Shanghai Mobike Shared Bicycle Trajectory Data (August 2016)	68.31	Fair	The data quality itself is acceptable, but its spatial scope completely mismatches the application requirements, resulting in the lowest comprehensive score.
Geographic Semantic Web	Entity Linking Dataset from OSM and Wikidata (China Region)	80.14	Moderate	Large entity scale, high degree of structure, but timeliness is its main shortcoming.
Scientific Literature	2023 China County Statistical Yearbook	75.61	Moderate	Authoritative source, comprehensive content coverage, but the information is unstructured.
Web Texts	Baidu Baike Webpages for County-level Divisions in China (3215 pages)	87.58	Moderate	Wide information coverage, reliable sources, explicit spatiotemporal information with moderate granularity, and serving as an important semantic information source.

). Given the small number of heterogeneous datasets, formal statistical significance tests are not applicable; instead, the practical discriminability and categorical separation are emphasized. The cases aimed to test whether the framework can achieve unified measurement across different data types and accurately identify the core strengths and key weaknesses of each data type in supporting knowledge graph construction. The evaluation process strictly followed the indicator system and calculation methods established in Chapter 4. The full information of the indicator is provided in Supplementary Tables S1–S6.

To deeply reveal the intrinsic characteristics of different data types, we created radar charts (Figure 4) of the evaluation results based on the primary indicators for each category. These charts clearly show the differentiated performance of various data types across key dimensions. For example, specialized spatial data performed outstandingly in the “Authority” and “Accuracy” dimensions, while the geographic semantic web held a clear disadvantage in the “Timeliness” dimension. The intuitive comparison provided by the radar charts serves as a powerful supplement to the comprehensive scores, making the strengths and weaknesses of the data immediately apparent.

The comprehensive quantitative evaluation results (summarized in Table 12) indicate that the framework effectively generates distinguishable and practically meaningful usability scores. The analysis reveals distinct data profiles. Specialized spatial data is ideal for the knowledge graph’s spatial base layer due to its high authority and accuracy. IoT sensing data is highly suitable for environmental monitoring, given its high spatiotemporal resolution and low error rate. Web texts serve as a crucial semantic source, characterized by reliable information and explicit spatiotemporal tags. In contrast, the geographic semantic web and scientific literature are constrained by timeliness and unstructured content, respectively, despite their value. Trajectory data received the lowest score, primarily due to a critical spatiotemporal mismatch with the application context. It underscores that the framework effectively captures the critical dimension of “fitness-for-use,” a core aspect of usability that transcends intrinsic data quality. This outcome validates the framework’s design premise: to evaluate data not in a vacuum, but within a specific application context.

This case study successfully verifies the practical value of the proposed evaluation framework. The results not only provide clear prioritization for data selection in spatiotemporal knowledge graph construction but also, through visual charts, indicate precise directions for subsequent data preprocessing and targeted enhancement. This framework elevates data selection from qualitative, experience-based decision-making to a systematic, quantitative evidence-based analysis process, significantly enhancing the scientific nature of data resource management and utilization.

6. Discussion

6.1. Applicability and Considerations

The proposed framework provides a standardized, multi-layered methodology for assessing the usability of heterogeneous crowdsensing data, ensuring broad applicability across domains such as natural resource management and smart city development. Its core design integrates the ISO-defined dimensions of efficiency, effectiveness, and satisfaction, facilitating comprehensive evaluation across diverse data types and application scenarios.

A key strength of the framework is its inherent customizability, which supports its role as a “meta-framework” or template. On the one hand, by considering the specific data requirements of the users and applications in the target domain, the assessment criteria can be further expanded. For example, in scenarios with higher privacy protection demands, an indicator for “privacy protection intensity” can be added. On the other hand, the framework supports the customization of indicators and weights based on different application contexts. For instance, in scenarios focused on near-real-time availability evaluation, the framework should emphasize metrics that can be rapidly computed and reflect the real-time nature of the data, such as timeliness and response efficiency. Through such extensions and adaptations, the framework not only meets existing needs, but can also be flexibly adjusted to accommodate evolving application scenarios. Therefore, this framework is not just a specific evaluation tool, but also a “meta-framework” or “template.”

In implementing any usability assessment, the balance between data usability and security presents a critical consideration. This evaluation framework primarily promotes secure and trustworthy data usage by indirectly assessing the sources and acquisition processes of the data. The “authority” metric favors data from regulated sources, such as government agencies and professional organizations. The “accessibility” metric for data acquisition methods that require applications, approvals, or usage agreements may score lower than fully open data, but this “inconvenience” serves as an important security control mechanism, ensuring traceability and compliance in data usage. Therefore, high-availability data, as evaluated by this framework, also tends to be data that is more likely to adhere to basic security and ethical standards. Future research could explore incorporating the strength of privacy protection as an explicit, quantifiable evaluation metric.

Despite its practical applicability, the framework has certain limitations. The framework lacks research on other scientific data standards and non-Chinese/English languages, which may limit its applicability in different cultural and technological environments. This limitation is especially prominent in global applications, and in the future, more international experts could be invited to further refine the framework to ensure its global applicability. Furthermore, usability assessments must simultaneously balance comprehensiveness, security, and feasibility. Users are encouraged to judiciously select and adjust indicators and weights based on their specific resource constraints and application goals, ensuring the assessment process itself remains efficient and actionable.

6.2. Sensitivity Analysis of Weights

A sensitivity analysis was conducted to assess the robustness of the proposed evaluation framework against potential subjectivity in expert-assigned weights. In this study, we employed the widely used One-at-a-time (OAT) method, where, under constant conditions, the weights of specific indicators were perturbed one at a time, and the changes in the output results were observed [104].

We selected representative core primary indicators from six categories of data for testing. These include “Authority” and “Accuracy & Reliability” for specialized spatial data and IoT sensing data, “Spatial Resolution” and “Temporal Resolution” for trajectory data, “Spatiotemporal Information Richness” for the geographic semantic web, “Attention” for scientific literature, and “Timeliness” for web texts. For each selected indicator, perturbations of ±10% and ±20% were applied to its original weight to simulate potential reasonable deviations in expert opinions. When the weight of an indicator was increased or decreased, the weights of other indicators within the same level were proportionally adjusted to ensure the total weight remained 1. Under each weight perturbation scenario, the comprehensive usability score (Q) of the dataset was recalculated. The Mean Absolute Percentage Change (MAPC) was used as a quantitative measure of stability, with the following formula:

$$\mathrm{M}\mathrm{C}={\displaystyle \frac{1}{\mathrm{n}}}\sum {\displaystyle \frac{\left|{\mathrm{Q}}_{\mathrm{n}\mathrm{e}\mathrm{w}}-{\mathrm{Q}}_{\mathrm{o}\mathrm{r}\mathrm{i}\mathrm{g}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}}\right|}{{\mathrm{Q}}_{\mathrm{o}\mathrm{r}\mathrm{i}\mathrm{g}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}}}}\times 100\mathrm{\%},$$

(19)

where n is the number of test datasets, and ${\mathrm{Q}}_{\mathrm{o}\mathrm{r}\mathrm{i}\mathrm{g}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l}}$ and ${\mathrm{Q}}_{\mathrm{n}\mathrm{e}\mathrm{w}}$ represent the usability scores before and after the weight perturbation, respectively.

Additionally, we evaluated the validity and sensitivity of the ranking thresholds based on the changes in rankings. The robustness of the evaluation framework was analyzed from both qualitative and quantitative perspectives.

The results of the sensitivity test for key indicator weights are shown in Figure 5. Across all tested datasets and perturbation scenarios, the MAPC in usability scores was consistently below 1%. Crucially, not a single perturbation altered the final usability ranking or grade classification of any dataset. This strongly demonstrates that, although the determination of weights incorporates subjective expert judgments, the evaluation framework proposed in this study is not sensitive to minor changes in weights and exhibits good robustness. This significantly strengthens the credibility and practical reliability of the assessment results.

7. Conclusions and Perspectives

To address the challenges of standardizing usability assessment for heterogeneous crowdsensing data and bridging the gap between intrinsic quality and contextual usability, this study proposed a novel, multi-layered evaluation framework. The framework establishes a unified metric system applicable across six major data categories, breaking methodological silos. The evaluation dimensions are expanded from the static intrinsic data quality to encompass comprehensive usability, including efficiency, effectiveness, and satisfaction, better aligning with the practical needs of decision-making scenarios. Through a dual-perspective structure of “data source” and “data content,” we clarify the conversion path from quality fundamentals to usability ratings.

The framework’s scientific rigor and practicality were validated through sensitivity analysis, which confirmed the robustness of evaluation results against weight variations, and a case study involving six real-world datasets. The case study demonstrated the framework’s capacity to generate distinguishable usability scores and actionable insights, successfully highlighting the differentiated performance profiles of various data types.

This framework elevates data selection and evaluation from qualitative, experience-based decision-making to a systematic, explainable scientific process based on quantitative evidence. Its evaluation results not only provide a reliable data foundation and preprocessing guidance for building high-quality spatiotemporal knowledge services but also offer an actionable, structured theoretical model for the complex concept of data usability, which holds reference value for fields such as geographic information science.

We acknowledge certain limitations, which point to valuable future research directions: (1) enhancing the framework’s global applicability by validating and refining the weight set with a more internationally diverse expert panel; (2) conducting task-based downstream validation (e.g., comparing knowledge graph construction performance using datasets of different usability scores); and (3) explicitly integrating indicators for data security and privacy protection to enable a comprehensive assessment of “secure usability.” Through continuous refinement, we envision this framework providing core methodological support for reliable, efficient, and responsible data utilization in an increasingly complex data environment.