Incremental Data Cube Architecture for Sentinel-2 Time Series: Multi-Cube Approaches to Dynamic Baseline Construction

Highlights

What are the main findings?

• We introduce a Multi-Cube architecture for Sentinel-2 time series that integrates spatial hashing, dynamic baseline policies, intelligent parallelism and automated subdivision of large areas into independent, temporally consistent cubes using Zarr-based storage. This design enables fully incremental and scalable data cube construction.
• Applied to 83,755 Sentinel-2 Level-2A images (2016–2024), the framework achieves 5.4× faster end-to-end processing and over two orders of magnitude less disk Input/Output than a conventional sequential pipeline, thanks to its incremental update engine and optimized multi-tile handling strategy.

What are the implications of the main finding?

• The framework provides interoperable cubes that grow incrementally and remain independent of analytical methods, allowing flexible integration with diverse workflows. This design offers a future oriented backbone for cloud-native EO systems, enabling analysts to adopt or replace algorithms without restructuring the data architecture and supporting reproducibility, long-term maintainability and seamless integration while also benefiting from shorter processing times.
• Beyond incremental ingestion, the architecture addresses the challenge of preserving temporal consistency across multi-tile areas by automatically generating stable and hash indexed units with coherent temporal baselines. This reduces mosaicking artifacts and configuration overhead, enabling reliable multi-year monitoring with minimal manual intervention.

Abstract

Incremental computing is becoming increasingly important for processing large-scale datasets. In satellite imagery, spatial resolution, temporal depth, and large files pose significant computational challenges, requiring efficient architectures to manage processing time and resource usage. Accordingly, in this study, we propose a dynamic architecture, termed Multi-Cube, for optical satellite time series. The framework introduces a modular and baseline-aware approach that enables scalable subdivision, incremental growth, and consistent management of spatiotemporal data. Built on NetCDF, xarray, and Zarr, Multi-Cube automatically constructs stable multidimensional data cubes while minimizing redundant reprocessing, formalizing automated internal decisions governing cube subdivision, baseline reuse, and incremental updates to support recurrent monitoring workflows. Its performance was evaluated using more than 83,000 Sentinel-2 images (covering 2016–2024) across multiple areas of interest. The proposed approach achieved a 5.4× reduction in end-to-end runtime, decreasing execution time from 53 h to 9 h, while disk I/O requirements were reduced by more than two orders of magnitude compared with a traditional sequential reprocessing pipeline. The framework supports parallel execution and on-demand sub-cube extraction for responsive large-area monitoring while internally handling incremental updates and adaptive cube management without requiring manual intervention. The results demonstrate that the Multi-Cube architecture provides a decision-driven foundation for integrating dynamic Earth observation workflows with analytical modules.

1. Introduction

Over the past few years, the availability of high-resolution and high-frequency Earth observation (EO) data has transformed the monitoring and analysis of dynamic surface processes. Optical satellite constellations such as Sentinel-2 provide global coverage at 10–60 m spatial resolution and frequent revisit cycles, enabling the construction of multiyear archives essential for environmental monitoring, disaster assessment, and change detection. However, the volume, growth rate, and variety of these datasets pose significant computational challenges, particularly when handling multi-format storage, heterogeneous temporal coverage, and large-scale multidimensional analyses [1,2,3,4].

Recent advances in scalable geospatial analytics, including high-performance computing, cloud native processing, and distributed storage, have improved our ability to address these challenges. Cloud-based formats such as NetCDF (Network Common Data Form), Zarr (a chunked cloud-native format for multidimensional arrays wherein data are divided into small blocks or “chunks” that can be accessed independently), and Cloud-Optimized GeoTIFF (COG, a version of GeoTIFF designed for efficient web access) enable direct, on-demand access to large EO datasets without requiring full local downloads. In parallel, frameworks such as Dask (a parallel computing library) and xarray (a library for labeled multidimensional arrays) support memory-aware and parallel computation across long satellite image time series [1,5]. Together, these approaches mitigate common bottlenecks in conventional workflows, including repeated disk input/output (I/O), redundant recalculations, and monolithic file handling, which otherwise lead to increased computational costs and processing latency.

In parallel, the analysis of satellite image time series has become a cornerstone for understanding environmental change [2,6], combining statistical trend detection with increasingly advanced machine learning (ML) and deep learning (DL) models. These approaches support a wide range of applications, including land cover dynamics, vegetation phenology, and the detection of natural hazards. Optical time series are particularly effective in capturing surface variations since spectral indices derived from visible, near-infrared (NIR), and shortwave infrared (SWIR) bands respond sensitively to vegetation stress, soil moisture, and hydrological fluctuations [7,8], providing a consistent view of a landscape’s evolution over time. Complementary approaches based on Sentinel-1 SAR time series further illustrate how multiyear satellite observations can strengthen environmental monitoring capabilities [6].

Several recent studies have highlighted the convergence of methodological and computational advances. Kganyago and Shikwambana [3] integrated multi-source MODIS, VIIRS, and land cover data to characterize wildfire dynamics across continents, revealing the operational importance of multi-sensor time series. Zhao et al. [9] applied persistent scatterer and differential InSAR techniques to long-term Sentinel-1 and ALOS-2 archives, extracting deformation fields at high temporal resolution and illustrating how architectural designs must accommodate heterogeneous sensors. Fang et al. [10] proposed a cross-attention framework for fusing unaligned hyperspectral and RGB data, a relevant strategy for integrating multimodal EO datasets with differing spatial alignment.

At the data and infrastructure level, Selea [4] introduced AgriSen-COG, a multicountry Sentinel-2 benchmark dataset distributed as a COG and SpatioTemporal Asset Catalog (STAC), demonstrating the value of FAIR (findable, accessible, interoperable, and reusable) EO data for large-scale ML applications in agriculture. Complementarily, Sun et al. [11] provided a comprehensive review of cyberinfrastructure for ML and big data in geosciences, highlighting the critical role of distributed computing, workflow orchestration, and cloud native frameworks in enabling scalable operational geospatial Artificial Intelligence (AI).

Recent advances increasingly address the need for scalable processing architectures to handle large-scale EO datasets. Dritsas and Trigka [12] note persistent challenges in data integration, computational efficiency, and model interpretability, even as ML and distributed processing gain traction. They emphasize the need for flexible architectures capable of operating across heterogeneous environments while supporting rapid data ingestion and low latency inference. Burgueño et al. [13] demonstrated a distributed AI-powered pipeline for land cover mapping over the Mediterranean Basin, processing multisource satellite data through a scalable cloud-based infrastructure. The system integrates automated feature extraction and grid-based processing, illustrating the potential of hybrid workflows that adapt to varying spatial and temporal resolutions. Complementary approaches, such as serverless computing, further illustrate how event-driven architectures can support scalable EO processing without persistent infrastructure [14].

In terms of high-performance fusion middleware (i.e., software layers that integrate heterogeneous data, models, and distributed computing resources), the CyberGIS-Compute platform [15] provides a reproducible and scalable environment for integrating multi-sensor remote sensing datasets in high-performance computing infrastructures. Built on XSEDE resources (the former U.S. national supercomputing network), the platform enables users to configure fusion workflows through Jupyter-based notebooks, supporting computational efficiency and workflow transparency in large-scale geospatial analysis.

Statistical modeling has also benefited from scalable satellite data platforms. Xie et al. [16] combined dynamic statistical models with Landsat archives to map rangeland condition changes across Mongolia at 30 m resolution. They incorporated pixel wise uncertainty and data-driven priors to detect subtle environmental shifts over time, validating the results against field observations and confirming the benefits of integrating long time series into environmental forecasting.

Regarding model complexity, Adegun et al. [17] compared Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) on standard satellite imagery datasets like EuroSAT and NWPU-RESISC45 and found that while ViTs achieve higher classification accuracy in heterogeneous environments, they require substantially more computational resources, thus hindering their scalability in real-time or large-area monitoring tasks.

These examples illustrate the current momentum toward efficient, modular, and intelligent architectures capable of processing vast and heterogeneous satellite datasets for acquiring near-real-time geospatial insights. Despite these advances, most existing EO-processing frameworks remain optimized for single-event detection or fixed temporal extents and lack robust mechanisms for incremental expansion of time series without full reprocessing. Very few solutions can dynamically ingest and integrate new observations while maintaining consistency with historical baselines, particularly when data arrive in multi-format or multi-tile structures [9,10]. This limitation is critical in operational-monitoring scenarios, where real-time responsiveness and long-term continuity must coexist. Bridging this gap therefore requires architectures that minimize computational overhead while supporting automated and adaptive referencing mechanisms that evolve with each new observation—precisely the challenge addressed in this work.

In this study, we present a cloud-native architecture for efficient long-term management of Sentinel-2 time series (2016–2024). Beyond enabling efficient data organization, the framework operates as an intelligent system that internally analyses processing context and automatically decides how data should be subdivided into cubes, how existing baselines should be reused, and how incremental updates should be applied. This decision-driven logic supports partial analyses and recurrent monitoring without full reprocessing, optimized for repeated analyses over the same areas as new observations become available. Execution strategies are adapted automatically to the available hardware. As a result, the proposed framework is more than just a multidimensional data repository based on Zarr, providing a scalable, near-real-time, and intelligent foundation for evolving EO analyses.

2. Materials and Methods

This section describes the data sources, architectural design, and experimental configuration used to implement and evaluate the proposed Multi-Cube framework. It first introduces the Sentinel-2 datasets and the reference-processing workflow and then details the core components of the Multi-Cube architecture, including its dynamic baseline strategy and internal decision logic. Finally, the study areas and experimental setup used for validation are presented, providing the methodological context for the results reported in the subsequent section.

2.1. Image Description

Sentinel-2 is an optical-imaging mission of the European Union’s Copernicus EO program providing multispectral 10 m spatial-resolution imagery of the whole planet. Since the launch of Sentinel-2A in 2015 and Sentinel-2B in 2017, the mission has delivered global coverage every five days at the equator through its MultiSpectral Instrument (MSI) and covers 13 spectral bands (Table 1) from visible to shortwave infrared with 10 m, 20 m, and 60 m resolutions, enabling detailed mapping of vegetation, water bodies, urban areas, and land cover change over seasons and years [18,19].

Table 1. Sentinel-2 spectral bands.

Band Name	Resolution (m)	Central Wavelength (nm)	Bandwidth (nm)	Description
B01	60	443	20	Coastal Aerosol
B02	10	490	65	Blue
B03	10	560	35	Green
B04	10	665	30	Red
B05	20	705	15	Red Edge
B06	20	740	15	Red Edge
B07	20	783	20	Red Edge
B08	10	842	115	Near Infrared
B08A	20	865	20	Red Edge
B09	60	945	20	Water vapor
B10	60	1375	30	Cirrus
B11	20	1610	90	Short-Wave Infrared
B12	20	2190	180	Short-Wave Infrared

The mission delivers two key data products: Level-1C products are top-of-atmosphere reflectance images that are orthorectified and organized into UTM/WGS84 tiles of roughly 100 × 100 km and distributed in the SAFE format with JPEG2000 bands and rich metadata. Subsequently, Level-2A products apply atmospheric correction and generate bottom-of-atmosphere reflectance alongside a scene classification layer (SCL) using the Sen2Cor processor, enabling cloud, water, vegetation, and shadow masking [20,21]. These well-engineered data streams make the mission ideal for time series analysis, trend monitoring, and change detection.

Researchers and practitioners worldwide have harnessed Sentinel-2 data across diverse domains. In agricultural monitoring, red-edge metrics and spectral indices are exploited to assess crop phenology, stress, and yield [22]. Water quality specialists use this dataset’s spectral reach to map turbidity and chlorophyll in inland and coastal waters through tailored correction algorithms [23]. At regional to global scales, analyses track deforestation, land use change, and urban expansion through multitemporal classification models [24,25]. Emergency response teams apply spectral indices and SCL masking to identify burned areas, floods, and damage after natural disasters [26].

Overall, Sentinel-2 combines fine spatial detail with frequent revisit cycles, providing a robust observational baseline for Earth monitoring. At the same time, its rich data delivery capacity challenges analysts to adopt efficient, incremental, and cloud-enabled workflows to fully exploit its potential for long-term landscape change analysis.

2.2. Reference-Processing Workflow

In practice, the classical Sentinel-2 workflow is a locally executed file-centric process (Figure 1) in which users select an area of interest (AOI) and date range, download Level-1C or Level-2A tiles in SAFE format, and (optionally) run Sen2Cor to determine surface reflectance and the SCL. A key limitation of this traditional approach is that ESA periodically updates the Processing Baseline (PB), and changes such as the radiometric offset introduced in PB 04.00 on 25 January 2022 alter product values over time. If these differences are not addressed at ingestion, multiyear analyses can become inconsistent, requiring harmonization across products processed under different baselines [27,28,29,30,31].

Figure 1. Traditional optical satellite image-processing workflow (file-centric and sequential). The figure summarizes a conventional pipeline in which each user request independently triggers the same end-to-end stages: (a) user request definition, including the area of interest, temporal range, and processing parameters; (b) data discovery through a web interface or API via catalog queries and scene selection; (c) downloading and local storage of selected products; (d) calibration and preprocessing steps, such as atmospheric correction, masking, and spatial cropping; (e) an analysis stage, including time series construction and feature or index computation; and (f) generation of final outputs, such as analytical results, visualizations, and reports.

The sequence shown in Figure 1 also highlights that the division of responsibilities across modules is intentional. Interactive discovery is handled by the WebInterface/API (Figure 1b), while CPU-intensive and repetitive disk operations are isolated within the ImageProcessor (Figure 1d). Stateful and versioned time series assembly is performed by the DataAnalyzer (Figure 1e), and final-product generation is confined to the OutputGenerator (Figure 1f).

The main performance bottlenecks occur at module boundaries [29]:

• WebInterface/API → Storage is bandwidth-bound (large SAFE granules);
• Storage → ImageProcessor is dominated by JPEG2000 decoding and repeated reads/writes during masking, resampling, and clipping;
• DataAnalyzer → OutputGenerator turns multi-tile and multiyear stacks into sizable artifacts and noticeable latencies whenever re-runs are triggered by new acquisitions or method updates.

Traditional Sentinel-2 processing pipelines typically operate in a strictly sequential manner, triggering full preprocessing chains for each request even when data have already been processed. This approach leads to repeated downloads, clipping, masking, and reprojection operations, introducing substantial redundancy as archives grow. At the same time, Sentinel-2 Level-2A tiles are large, often hundreds of megabytes, and multiyear collections can easily reach the terabyte scale, while the computational cost of JPEG2000 decoding further motivates one to use cloud-optimized and chunked formats such as COG, Zarr, or NetCDF for efficient spatial and temporal access [32,33].

These limitations become increasingly evident in long-term time series analyses: repeatedly reprocessing full acquisition sequences rapidly increases computational costs, inflates storage requirements, and complicates temporal consistency when mosaicking tiles acquired on different sensing dates. At larger scales, this local-file-centric workflow, where each product is handled as an independent file, becomes brittle. Repeated downloads, JP2 decompression, atmospheric correction, SAFE archive bookkeeping, and cross-baseline harmonization significantly increase turnaround time and reduce reproducibility. Recent surveys show that the growing spatial and temporal depth of Sentinel-2 archives pushes local workflows near their I/O and memory limits, motivating the development of approaches that minimize data movement and standardize preprocessing across evolving processing baselines such as PB 04.00 [28,31,34].

Given these constraints, the community is increasingly shifting toward cloud-native patterns such as analysis-ready and cloud-optimized data, STACs, and COG/Zarr assets employed closer to storage, often through xarray/Dask or serverless backends. Examples include national-scale EO data cubes, large-area Sentinel-2 dynamic cubes, and harmonization pipelines such as HLS v2.0, which reduce data movement and wall-clock time by avoiding bulk downloads and exploiting in cloud parallel execution [35,36,37,38,39,40,41].

However, most existing approaches operate as monolithic cubes or fixed pipelines optimized for specific regions or products, lacking robust mechanisms for incremental ingestion, automated subdivision into multiple multidimensional cubes, and efficient partial analyses without reprocessing entire time series. These limitations highlight the need for dynamic, federated, and adaptive frameworks capable of supporting long-term, multi-regional time series processing, a challenge directly addressed by the Multi-Cube architecture proposed herein.

2.3. Overview of the Multi-Cube Framework

While Section 2.2 describes the reference Sentinel-2 processing pipeline typically used in operational workflows, this subsection presents the proposed Multi-Cube framework, an incremental and dynamic architecture specifically designed for Sentinel-2 imagery to overcome the limitations of standard sequential pipelines.

Data cubes now represent one of the most efficient paradigms for handling satellite-acquired big data. Conceptually, a data cube is a multidimensional array in which spatial, temporal, and spectral dimensions are integrated into a coherent structure that enables rapid querying, scalable analytics, and reproducible workflows [42,43]. Unlike traditional file-centric repositories, cubes support direct subsetting and computation across time and space, reducing I/O overhead and avoiding redundant preprocessing. Frameworks such as the Open Data Cube (ODC) [44] and Sentinel Hub’s EO Browser [45] illustrate the transition towards cloud-native cube infrastructures that allow efficient exploration of multiyear and multi-sensor archives. In this sense, data cubes represent not only a technical solution but also a conceptual shift in EO infrastructures towards FAIR-compliant, analysis-ready data ecosystems [46,47,48].

Building upon these advances, the proposed architecture introduces a Multi-Cube framework tailored to large-scale Sentinel-2 time series. Unlike monolithic cubes, this approach generates independent yet interlinked stable cubes whenever a time series must be subdivided due to tiling boundaries or heterogeneous temporal coverage. A stable cube corresponds to a fixed spatial unit whose extent does not change after creation. It is stored in Zarr for efficient cloud-native access and indexed through a spatial hash, allowing it to evolve only temporally through incremental updates.

Each cube operates as a self-contained and persistent object that is reused over time for recurrent monitoring and analysis rather than being created per query. Cubes remain interlinked through shared metadata and hashes, enabling federated querying while preserving internal consistency. A dynamic baseline strategy binds user-defined update policies to the orchestration logic so that only previously unseen dates are incorporated from new NetCDF exports. Each stable cube maintains a local manifest describing its own spatial footprint, temporal coverage, projection, and resolution, avoiding the accumulation of a centralized or global metadata catalog. As a result, metadata growth remains partitioned across independent cubes, while baselines evolve incrementally without full archive regeneration [35,49].

User interactions are therefore resolved against existing cubes whenever possible. Requests targeting spatial or temporal subsets already contained within a stable cube are handled through direct access and local subsetting, without triggering cross-cube matching or metadata scans. New cubes are created only when no existing cubes satisfy the requested spatial or temporal extent or when required by explicit update policies. This design ensures bounded retrieval complexity and is particularly well suited to continuous monitoring and recurrent analyses over predefined areas.

The Multi-Cube framework follows the modular architecture illustrated in Figure 2, which represents a continuous workflow from user-defined inputs to analytical outputs. The Inputs stage defines the AOI, temporal range, variables, and update policy, while the system queries a STAC to identify all Sentinel-2 Level-2A items contributing to the requested time series.

Figure 2. Conceptual overview of the proposed Multi-Cube architecture for incremental satellite time series processing. The figure illustrates the organization of the framework into interconnected functional layers, ranging from user-defined inputs and data discovery to preprocessing, cube orchestration, and analytical outputs. Unlike traditional monolithic data-cube designs, the Multi-Cube architecture adheres to an incremental and modular strategy, in which stable data cubes are created, extended, or queried without reprocessing the full archive. Visually, the architecture is represented as a collection of hash-indexed stable sub-cubes that evolve independently over time while remaining linked through shared metadata, enabling scalable access, partial updating, and flexible spatiotemporal analysis. A detailed description of the architecture and its components is provided in Section 2.3, Section 2.4, Section 2.5 and Section 2.6.

The request is then handled by the Orchestrator, who resolves both spatial and temporal planning. Each AOI is converted into a persistent spatial hash, enabling fast retrieval of an existing stable cube or the creation of a new one when required. Temporal completeness is evaluated by comparing requested dates with those already stored. When an AOI intersects with multiple Sentinel-2 tiles across different acquisition dates, the framework activates a subdivision mechanism that splits the region into coherent sub-areas, preserving temporal consistency and avoiding the mosaicking of scenes acquired on different days. This process results in a precise execution plan specifying which spatial units, dates, and update policies apply.

In the Preprocess module, all observations required by the execution plan are standardized. For each date, the tile with the greatest spatial overlap is selected and processed through band reading, reprojection, clipping, and variable computation. The resulting per-date NetCDF files act as reusable intermediate layers, preventing redundant preprocessing and enabling selective reconstruction of temporal subsets.

These standardized layers are then integrated into the Incremental Baseline, which maintains the stable Zarr-based data cubes. Depending on the policy selected, the framework either appends only new dates, fills missing temporal gaps, rebuilds a cube, or performs a recompute-window operation—the last of which, crucially, is a non-mutating path in which the framework resolves the existing stable cube(s) matching the requested spatial and temporal scope and reads only the required chunks to construct a temporary in-memory sub-dataset. No writes are performed for the Zarr baseline, whose layout and temporal chunking are preserved, enabling fast selective access, efficient parallel I/O, and long-term consistency without full cube regeneration.

In the Analysis stage, computations are performed either on an updated stable cube or on a temporary in-memory sub-cube, depending on the selected policy, while results are ultimately materialized in the Outputs stage as raster products, vector layers, metrics, or visualizations. Through this sequence of tightly integrated operations (illustrated in Figure 2), the Multi-Cube framework provides a scalable and reproducible architecture in which ingestion, preprocessing, baseline maintenance, and analysis are coherently linked. This design minimizes redundant processing and enables fast, incremental updating across long-term Sentinel-2 time series, explicitly supporting recurrent and multi-regional monitoring workflows.

2.4. Dynamic Baseline and Multi-Cube Construction

A central contribution of this framework is the implementation of a dynamic baseline strategy that governs how new observations are integrated into existing time series archives with different dimensions. In current EO systems, update strategies vary considerably. For example, some STAC-enabled cloud-native repositories and data platforms allow indexing and appending of new scenes without rebuilding the entire archive. In other cases, especially when processing algorithms or baseline versions change, operational collections are typically reprocessed to ensure temporal consistency and cross-sensor comparability.

In this context, the Multi-Cube framework formalizes explicit update policies and binds them to an incremental orchestration logic that minimizes unnecessary recomputation while preserving temporal continuity and reducing computational and storage costs.

The Multi-Cube framework relies on the Zarr data structure (Figure 3), a cloud-native format designed for storing and managing large multidimensional arrays. Unlike traditional file-centric formats such as NetCDF or HDF5, which package variables into monolithic files, Zarr organizes data into small, addressable chunks that can be independently stored in local file systems or as objects via cloud storage. This chunk-based organization enables selective access to spatial or temporal subsets without loading entire datasets into memory while supporting efficient parallel I/O through libraries such as xarray and Dask.

Figure 3. Regular Zarr single structure.

Consequently, Zarr significantly improves scalability and performance when handling large satellite archives and has rapidly started being adopted in EO workflows as a foundation for analysis-ready and cloud-optimized data storage.

Before applying any baseline update policy, the framework first identifies which stable cube should receive the new observations. This task is achieved through spatial hashing, a mechanism that converts each AOI or sub-geometry into a deterministic unique identifier, ensuring consistent mapping between geometries and their corresponding cubes. As shown in Equation (1), the AOI is serialized into Well-Known Binary (WKB), a standard format for encoding vector geometries defined by the Open Geospatial Consortium (OGC) Simple Feature Access specification, and then hashed using a cryptographic digest. The resulting truncated hexadecimal prefix is used as an area key, allowing direct access to the corresponding cube directory without scanning global catalogs. This guarantees reproducibility and fast lookups even when managing thousands of AOIs.

This design is explicitly aligned with the primary objective of the Multi-Cube framework: efficient recurrent monitoring over fixed AOIs. By associating each recurrent AOI with a stable hash-based identifier, the framework enables rapid and deterministic access to previously constructed cubes, minimizing overhead when the same regions are queried repeatedly as new observations become available.

While the current implementation relies on exact geometry hashing, the same principle can be extended to alternative spatial indexing strategies. For example, grid-based deployments can employ space-filling curve encodings such as Morton (Z-order) indexing (Equation (3)), which quantize longitude, latitude, and (optionally) time or spectral dimensions into a single location-aware key. Both approaches ensure that incremental ingestion consistently targets the correct baseline cube while preserving flexibility for future extensions.

Formally, for exact geometry hashing, which we use in Multi-Cube, we set

$$H=\mathrm{SHA}1\left(\mathrm{WKB}\left(\mathcal{G}\right)\right),\mathrm{area}\_\mathrm{id}={\mathrm{prefix}}_L\left(H_{\mathrm{hex}}\right)$$

(1)

Here, SHA1 is a cryptographic hash function that converts any input into a fixed 160-bit hexadecimal digest, $\mathcal{G}$ is the AOI geometry, H is the digest, and ${\mathrm{prefix}}_L$ extracts the first L hexadecimal characters to create a unique key.

For grid-based indexing, spatial coordinates can be quantized as follows:

$$i_x=⌊{\displaystyle \frac{\lambda -{\lambda }_{min}}{\Delta \lambda }}⌋,i_y=⌊{\displaystyle \frac{\varphi -{\varphi }_{min}}{\Delta \varphi }}⌋,$$

(2)

with longitude $\lambda$ and latitude $\varphi$. A Morton (Z-order) key is then obtained by interleaving the binary digits of $i_x$ and $i_y$:

$$\mathrm{morton}(i_x,i_y)=\mathrm{interleave}\_\mathrm{bits}(i_x,i_y).$$

(3)

Here, interleave_bits simply denotes the bit-alternation operation that constructs the Morton key. This operation produces a compact integer index that preserves spatial locality and can serve as the cube identifier in large-scale tiled deployments.

In the Multi-Cube approach, the dynamic baseline evolution is controlled by four policies:

• auto-append, which extends the cube by appending only unseen dates after the last stored timestamp;
• append-from-missing, which reprocesses only those dates within the defined interval that were not successfully ingested, thereby filling temporal gaps;
• recompute-window, which isolates a specific temporal or spatial window and reconstructs it independently in memory, without affecting the stable baseline;
• force-rebuild, which creates or regenerates the entire cube when required.

This policy-driven design is explicitly aligned with the primary objective of the framework: ensuring efficient recurrent monitoring over stable and repeatedly queried AOIs. In such scenarios, the same AOIs are revisited continuously as new satellite acquisitions become available, and rapid access to their historical baselines is essential. By combining deterministic spatial hashing with flexible baseline-updating policies, the framework enables direct and constant access to these recurrent AOIs while ensuring that only strictly necessary recomputation is performed. As a result, long-term monitoring workflows can evolve incrementally, preserving historical consistency and minimizing both computational overhead and data movement.

Figure 4 illustrates the internal Multi-Cube construction workflow, in which each new ingestion request is handled through a policy-driven sequence of coordinated stages. After input validation and stable cube identification via spatial hashing, the framework computes the effective temporal window required by the request and resolves the corresponding update policy. Depending on this policy, the engine either appends new per-date NetCDF layers to an existing baseline, selectively rebuilds specific temporal segments, or performs a read-only recompute-window operation that assembles only the required spatiotemporal slices in memory without modifying the stored cube.

Figure 4. Internal workflow of the Multi-Cube incremental engine illustrating how a new ingestion request is resolved through policy-driven decision paths. The diagram shows the sequence of validation, baseline planning, and policy selection that determines whether data will be appended to an existing stable cube, selectively rebuilt, or extracted as a temporary in-memory sub-cube. The workflow explicitly distinguishes mutating operations, which update the persistent Zarr-based baseline and its associated manifest, from non-mutating recompute-window operations, which assemble read-only sub-cubes without altering the stored data. Concurrency management, validation, monitoring, and metadata updates are conducted across multiple branches of the workflow, ensuring consistent and efficient handling of incremental updates and on-demand analyses.

Execution then proceeds by loading and normalizing the necessary files, applying parallel processing through the concurrency manager and writing or assembling the resulting dataset according to the selected policy branch. When baseline mutations occur, metadata are consolidated, and the cube manifest is updated accordingly, whereas non-mutating operations produce temporary in-memory sub-cubes for on-demand analysis. This modular workflow enables efficient handling of entire AOIs or spatially subdivided tiles while preserving temporal integrity and avoiding unnecessary mosaicking and redundant computation.

A central aspect of the framework is that the dynamic baseline manages not only temporal updates but also spatial subdivision. When an AOI intersects multiple tiles across different dates, the system automatically generates new hashed cubes, allowing each baseline to evolve modularly and expanding the cube library only when required. This design preserves spatial consistency and temporal integrity while avoiding the systematic mosaicking typical of monolithic cube implementations [50]. By combining incremental ingestion with Multi-Cube construction, each stable cube, indexed by its spatial footprint, can evolve independently while remaining accessible within a federated archive. This modular organization enhances reproducibility and scalability, addressing challenges also highlighted in regard to Earth System Data Cubes [51] and in studies on interoperability across distributed EO cube instances [52].

In contrast, when a request overlaps multiple existing cubes with heterogeneous spatial or temporal coverage, the framework resolves the situation through an ordered, policy-driven process. Temporal containment is evaluated first to determine whether existing cubes can fully satisfy the requested date range. Spatial overlap is then optimized by prioritizing cubes that provide the largest spatial coverage, minimizing unnecessary subdivision. Only when no existing cube satisfies both spatial and temporal constraints does the framework activate controlled subdivision or trigger new ingestion according to the selected update policy. In such cases, existing stable cubes are treated as independent baselines and are not recombined to form a new cube. This design preserves temporal consistency and avoids implicit mosaicking, while allowing joint interpretation across multiple cubes at the analysis or output stage.

Building on this structure, the framework also accommodates the demands of complex algorithms that function based on long and heterogeneous time series. Incremental baseline management and independent cube evolution provide a flexible computational backend that supports a wide range of analytical workflows without requiring full cube reconstruction. This approach aligns with recent developments in semantic EO data cubes, where scalable structures and enriched metadata enable more advanced and context-aware interpretations [52]. The following section illustrates the practical implications of this design by showing how different baseline configurations and update policies lead to distinct performance behaviors when applied to large-scale remote sensing time series.

2.5. Cloud and Parallel Processing

The Multi-Cube framework integrates cloud-native access and parallel execution to address the computational demands of multiyear and multi-tile satellite time series. Given the spatial resolution and acquisition frequency of missions such as Sentinel-2, distributed processing is required to outperform traditional sequential pipelines [33,53,54]. Accordingly, the framework employs Dask-based parallelization, enabling concurrent execution of band reading, reprojection, clipping, and variable computation across spatial subdivisions defined by the orchestrator while maintaining stable memory usage.

Cloud infrastructures further enhance scalability by providing STAC-enabled, analysis-ready assets through Copernicus cloud services and similar platforms, enabling on-demand access to optimized Sentinel data while reducing I/O overhead by avoiding full local downloads. Recent advances in cloud-native EO ecosystems show that hybrid on-premise and cloud processing can sustain long-term analytical workloads [55,56], while access to distributed holdings across providers improves operational flexibility and data throughput [57]. Within this context, the framework leverages chunked processing and lazy evaluation to ensure that only relevant portions of the archive are accessed, further improving efficiency.

In addition, the framework incorporates an adaptive execution layer that automatically inspects the operating system and available computational resources during runtime. Based on this information, the orchestrator dynamically configures process pools and concurrency levels across CPU cores, allowing parallel execution to be adjusted to the characteristics of the host environment without requiring manual tuning. This adaptive strategy prioritizes stable throughput and controlled resource usage rather than maximal parallelism.

Parallelization and cloud-native access operate jointly with the incremental ingestion strategy of the dynamic baseline. Spatial subdivision is processed independently, daily NetCDF files are generated or skipped depending on the update policy, and only the required temporal slices are appended to the stable Zarr store. This modular form of execution avoids full pipeline regeneration and supports scalable long-term monitoring across multiple regions, aligning the Multi-Cube framework with recent advances in adaptive and semantic EO data cubes [54,55,56,57].

2.6. Decision-Driven Orchestrator for Incremental Workflows

Rather than being a mere multidimensional repository, the proposed Multi-Cube framework is designed to be a decision-driven methodology for incremental EO workflows under heterogeneous usage scenarios. Its contribution lies in an explicit reasoning layer that is intrinsic to the framework and automatically governs how data are organized, subdivided, updated, and executed during runtime, without requiring users to manually define spatial partitioning strategies, baseline management rules, or execution parameters. In this sense, Multi-Cube emphasizes operational intelligence—how and when to construct, reuse, and recompute beyond the underlying storage format.

To situate the contribution of the framework within the existing ecosystem, it is useful to contrast its design philosophy with that of established data cube systems. General-purpose open-source cubes such as ODC [44], gdalcubes [43] and other cloud-native implementations primarily address the challenges of data standardization, centralized cataloguing, and unified access to long-term EO archives. Their core value lies in transforming disparate satellite products into analysis-ready, multidimensional arrays that can be queried spatially and temporally through standardized APIs, making large archives FAIR.

In contrast, the Multi-Cube framework was designed to be an operational methodology that embeds decision logic to support dynamic and recurrent monitoring workflows, where continuous data arrival and repeated requests over similar regions make “how to update and reuse” as important as “how to store and access”. In this respect, decision-making in Multi-Cube is an integral part of the core architecture. This decision layer governs, for example, when automatic spatial subdivision into independent cubes is required and how execution parameters are adapted to the user’s hardware environment. In comparison, in ODC, such choices are typically delegated to user-defined ingestion pipelines, configuration files, or deployment-specific conventions rather than being resolved automatically by the framework itself.

The Multi-Cube framework therefore operates at a distinct architectural level. Rather than prioritizing the storage or uniform querying of monolithic data structures, it formalizes a policy-driven orchestration layer that governs how incremental EO workflows should evolve over time. By automating operational decisions that are commonly externalized to manual workflow design or deployment-specific configurations, Multi-Cube provides a flexible and reusable methodological backbone for recurrent monitoring scenarios.

The following subsection presents the study areas selected to validate the proposed decision-driven framework, while Section 3 reports the corresponding application and performance results.

2.7. Study Area

We selected study areas suitable for demonstrating the effectiveness of the Multi-Cube framework in handling multiple polygons of varying extents, each associated with distinct time series and parallel analytical processes. Several polygons were deliberately located at Sentinel-2 tile boundaries, reflecting common operational scenarios where consecutive satellite orbits do not provide identical sensing dates and partial tile overlap occurs.

Figure 5 illustrates the global Sentinel-2 tiling grid and representative examples of tile overlap. Consequently, some study areas span multiple tiles or require more than one tile for complete coverage. In such cases, the framework activates the subdivision mechanism described in the previous section, splitting the AOI into independent sub-cubes that preserve temporal consistency without mosaicking during ingestion time.

Figure 5. Global Sentinel-2 tiling grid covering the entire Earth’s surface, shown over a Google Earth basemap (© Google Earth). Each grid cell, referred to as a tile, represents a fixed spatial unit of approximately 110 × 110 km.

Three study areas with large spatial extents were selected, with Areas A, B, and C covering 258 km², 317 km², and 343 km², respectively (Figure 6). These areas were designed to validate the decision-making capacity of the framework under different spatial configurations. Area A is fully contained within a single Sentinel-2 tile, representing the ideal case in which no spatial subdivision is required. Area B intersects two tiles; however, one tile provides complete coverage, allowing the framework to apply a single-tile policy and avoid unnecessary subdivision. In contrast, Area C overlaps two tiles without full coverage from either, triggering the splitting mechanism that divides the study area into two independent sub-areas, each associated with its own time series cube.

Figure 6. Study areas selected to test the framework over a Google Earth basemap (© Google Earth).

In addition, the input AOI consists of a multi-polygon with spatially independent regions. The framework automatically identifies and resolves each polygon independently during cube construction, generating separate stable cubes without user intervention and applying the same decision logic used for tile-based subdivision.

The areas illustrated in Figure 6 correspond to the original input regions A, B, and C provided to the framework. During cube construction, area C is automatically subdivided into two stable cubes (C-1 and C2) because of incomplete tile coverage, resulting in the four experimental cubes later visualized in the performance analysis. Table 2 summarizes the main spatial, temporal, and data-volume characteristics of each constructed cube used in the experimental evaluation.

Table 2. Characteristics of the stable cubes generated for the experimental evaluation. All cubes share the same temporal coverage (2016–2024) and were constructed from Sentinel-2 imagery at 10 m spatial resolution using five spectral bands and seven derived indices.

Attribute	Cube A	Cube B	Cube C-1	Cube C-2
AOI Coverage (%)	100	100	27	73
Spatial extent WGS84	[28.515, −1.427] [28.709, −1.319]	[28.852, −1.413] [29.085, −1.303]	[28.734, −1.640] [28.798, −1.525]	[28.797, −1.640] [28.974, −1.525]
Dimensions (X × Y)	2153 × 1190	2589 × 1218	710 × 1276	1962 × 1276
Area (km2)	258	317	91	252
Sentinel-2 images	598	598	599	598
Total image data	20,930	20,930	20,965	20,930

Building on this experimental setup, Section 3 analyzes the results obtained under different baseline and execution policies, focusing on the behavior of the automated decision engine and the resulting optimization of computational execution times across the three defined areas and their corresponding stable cubes showed in Table 2.

3. Results

The proposed framework was applied to three AOIs located in the central rainforest of the Congo Basin (Table 2, Figure 6) to demonstrate its adaptability to external analytical methodologies operating over massive satellite image time series. For this experiment, the framework ingested and processed Sentinel-2 Level-2A imagery from 2016 to 2024, constructing stable Multi-Cubes that store harmonized, temporally consistent, and spatially standardized multivariate time series. Each cube encapsulates both the original optical bands and derived spectral indices, resulting in a multidimensional representation suitable for downstream analysis. The Multi-Cube logic automatically handles heterogeneity in tile coverage by generating sub-cubes when AOIs intersect multiple Sentinel-2 footprints while preserving temporal consistency even under irregular acquisition patterns.

In the construction of each cube, multiple categories of inputs that are harmonized during preprocessing are integrated:

• Sentinel-2 Level 2A bands;
• Quality and SCL for masking clouds and shadows;
• Derived spectral indices;
• Metadata attributes standardized from the STAC, including acquisition date, tile ID, and quality indicators.

Within this experimental setting, we integrated the semi-supervised and multitemporal event detection algorithm developed by Deijns et al. [7]. Their method, originally designed for unexplored regions, detects geomorphic hazard impacts such as landslides and flash floods in Sentinel-2 imagery by computing the cumulative difference with respect to the mean curves for multiple spectral indices. Sudden disturbances produce a distinctive triangular signal whose peak indicates the timing of the event. Identifying these peaks requires completing mathematically intensive signal-processing steps including optimized peak-finding algorithms applied over massive multidimensional arrays supported by operations such as array transposition and stride reduction.

This approach is well suited to large-scale time series analysis in regions for which there are few ground-based observations, making it an appropriate use case for evaluating the Multi-Cube framework. The workflow relies on long, multivariate, and temporally aligned series, which directly benefit from the framework’s ability to standardize inputs, parallelize preprocessing, and manage multiple AOIs with heterogeneous spatial and temporal characteristics. This enables methods such as Deijns et al.’s [7] to be reapplied efficiently over the same stable cubes without modifying the underlying baseline, supporting recurrent monitoring over fixed areas, which is a core objective of the proposed event detection methodology.

In this context, the algorithm is integrated as an external analysis stage that directly consumes the stable Multi-Cubes, reinforcing the generality of the proposed architecture. Any method requiring spatially consistent and temporally aligned variables can be applied to the cube as a standardized analytical interface. This integration exercise demonstrates that the methodology developed by Deijns et al. [7] integrates naturally within the Multi-Cube environment and that the framework can support multi-AOI monitoring workflows while remaining extensible to additional dimensions in future applications.

3.1. Performance Results

The performance evaluation focused on contrasting full cube reconstruction against reuse-based execution over pre-existing stable cubes. In this context, the reported metrics reflect not only computational efficiency but also the effect of the framework’s policy-driven spatial and temporal resolution mechanisms, including automatic evaluation of tile coverage, subdivision of partially covered areas, resolution of multi-tile configurations, and selective recomputation over stable cubes.

To evaluate the performance of the proposed Multi-Cube framework, a series of experiments were conducted using the Sentinel-2 dataset and AOIs described in the previous section. In these tests, we quantified computational costs under different configurations and assessed the efficiency gains enabled by the proposed architecture. All experiments were executed on a dedicated high-performance workstation to ensure there was a stable and reproducible benchmarking environment. The main hardware characteristics are summarized in Table 3.

Table 3. Hardware specifications.

Component	Specification
Operating System	Linux Mint 22 (Ubuntu 24.04 base), Kernel 6.8.0-64, x86_64 architecture
CPU	AMD Ryzen 9 5900X, 12 cores/24 threads, base 2.2 GHz, boost up to 3.7 GHz
RAM	128 GB DDR4
GPU	NVIDIA GeForce GTX 1080, driver 570.169
Storage	NVMe SSD 1 TB + SSD 4 TB + 3 × 10 TB HDD
Total Storage	~31.8 TiB
Motherboard	ASUS PRIME X570-PRO, UEFI BIOS v3801
Network	Intel Gigabit Ethernet (I211 and 82574L controllers)

To evaluate the framework’s performance under different computational conditions, two distinct experiments were conducted:

• The first experiment compared a sequential baseline mode, in which parallelization is deliberately suppressed to simulate the lower-bound computational cost, against the default configuration, where the Multi-Cube autonomously allocates workers and exploits multi-threading to optimize pipeline execution;
• The second experiment evaluated the framework’s ability to provide immediate access to sub-cubes in both temporal and spatial dimensions, enabling automated extraction of partial series or subsets, thereby contrasting with conventional approaches that require recalculating the full cube even for small AOIs.

These experiments quantified the computational demands of large-scale satellite time series processing, and the results illustrate the efficiency gains enabled by the proposed architecture. The performance evaluation was conducted using the full data volume summarized in Table 2, comprising a total of 83,755 processed and stored images under identical experimental conditions.

3.1.1. Sequential Emulation vs. Adaptive Parallelization

Using the machine reported in Table 3 and the dataset described in Table 2, we executed two end-to-end runs of the Multi-Cube base pipeline. The first run was a controlled sequential baseline in which the same scientific routines as the full pipeline were executed but with all parallelism disabled:

• Execution is restricted to a single CPU core;
• Numerical and I/O libraries are limited to one processing thread;
• The is no subdivision of the time series into smaller sub-areas (tiling disabled);
• A single worker handles all stages (data loading, transformation, and computation), ensuring a strictly serial process.

The second run leveraged the framework’s full parallel potential:

• The framework automatically adjusts the number of workers assigned to each processing stage according to system capacity;
• Time series are divided into smaller independent spatial–temporal blocks that can be processed simultaneously;
• External numerical libraries are constrained to a controlled number of threads to prevent excessive contention for CPU resources;
• Data are read and processed in spatial and temporal chunks so that I/O and computation can proceed concurrently.

As discussed above, the optimized Multi-Cube framework enables scalable execution by orchestrating small, independent tasks and adapting concurrency to the available hardware, whereas the sequential baseline disables these mechanisms and serves as a clean reference in which performance differences arise solely from orchestration rather than from changes in the analytical code.

Under these conditions, end-to-end runtime was reduced from approximately 59 h in the sequential mode to about 9 h in the fully optimized Multi-Cube run. System monitoring indicated that the level of CPU usage for sequential execution remained close to that for single-core CPU usage, whereas the Multi-Cube configuration sustained high overall CPU utilization with workers working concurrently. Given comparable I/O volumes, the performance gain mainly reflects an acceleration of CPU-bound stages such as reprojection, resampling, and per-pixel time series analysis enabled by multilayer parallel scheduling. The principal results are summarized in Table 4.

Table 4. Main performance summary (sequential vs. parallel).

Metric	Sequential	Parallel	Ratio
Runtime (hh:mm:ss)	52:48:20	09:40:18	5.46×
Mean CPU (%)	9.5	27.3
Peak CPU (%)	102.6	530.8
Main-process RAM p90/peak (GB)	37.42/54.53	38.91/54.96
Disk read (GB)	6.58	6.04
Disk write (GB)	157.90	109.91
Mean concurrency	4.02	14.23	3.54×
Peak concurrency	10	24	2.40×

The sequential configuration showed near-serial execution with minimal concurrency and low multi-core utilization, resulting in extended idle periods. In contrast, the parallel configuration sustained concurrent task execution, achieving greater aggregate CPU usage and much shorter runtimes. These dynamics explain the performance gap observed (Figure 7).

Figure 7. Worker lifetimes for the sequential (a) and parallel (b) configurations under identical runtime conditions, illustrating how process overlap and CPU load evolve over time. The y-axis is categorical and represents the set of independent worker processes, used solely to visually separate concurrent processes rather than encode a quantitative variable, while the x-axis shows wall-clock time in minutes from the start of the pipeline. Each colored segment corresponds to the period during which a worker was active, with color encoding its average CPU load (dark-purple/blue = low, green = medium, and yellow = high, with the same normalization in both panels). The long bar at the bottom represents the main controller process, which remains active throughout the execution, whereas the shorter bars correspond to transient workers spawned to execute tasks. In the sequential configuration, workers appear in short, mostly non-overlapping bursts, resulting in extended periods with few active processes. In contrast, the parallel configuration exhibits dense overlap and stable plateaus of concurrent workers, with diagonal patterns indicating continuous replacement of completed tasks. The dense overlapping color bars in (b) make individual segments harder to distinguish, constituting clear evidence of continuous task overlap and short handoffs, whereas the sequential panel displays distinct, well-separated segments indicative of limited temporal overlap and near-serial execution.

Resource utilization and execution dynamics are shown in Figure 8 for the comparative analysis between the sequential emulation (Figure 8a–c) and the adaptive parallel execution (Figure 8b–d), highlighting clear differences in these aspects. Figure 8a,b illustrate worker concurrency. In the parallel configuration, more workers remain active simultaneously, with pronounced overlapping lifetimes that reflect an efficient distribution of tasks across multiple processes. In contrast, the sequential emulation maintains a nearly constant and limited number of workers producing long plateaus of near-serial execution with minimal concurrency and limited exploitation of available CPU resources.

Figure 8. Sequential vs. adaptive parallel performance. (a,b) Worker concurrency for sequential and parallel modes. (c,d) CPU and memory usage for sequential and parallel configurations. Solid blue lines show raw one-second measurements, while orange dashed lines indicate 15-s moving averages.

In parallel mode, CPU usage exhibited sustained peaks reaching several hundred percentage points, evidencing effective multi-core utilization, while memory usage remained stable within expected thresholds, indicating that overhead was well-managed despite higher concurrency (Figure 8c,d). In sequential mode, CPU consumption stayed near the single-core baseline, with long inactive periods, and although memory usage was stable, the lack of concurrency restricted throughput. Each of these panels displays two curves for each metric: a solid blue line showing the raw one-second measurements and an orange dashed line representing a 15-s moving average that filters short-term fluctuations. This dual representation provides both instantaneous variations and smoothed trends, helping distinguish structural load patterns from transient spikes.

CPU and memory exhibit similar temporal shapes because both metrics respond directly to worker activity. As workers start, finish, or overlap, they jointly drive increases or decreases in CPU load and memory usage, resulting in curves with comparable forms. The difference lies in magnitude: CPU utilization is mapped to the left y-axis, and memory is mapped to the right y-axis. The colors do not distinguish between CPU and memory; instead, blue and orange encode raw and smoothed values, respectively, while the axes uniquely identify which metric is being displayed.

In parallel mode, workers become active immediately due to the initialization of process pools and task allocation structures, as reflected in Figure 8b. However, this startup phase produces only minimal CPU and memory activity, and therefore the raw and smoothed traces in Figure 8d remain close to zero during the first few minutes. This does not represent missing data or misalignment; it simply reflects the low-cost initialization phase before sustained parallel computation begins.

Taken together, these figures provide a comprehensive overview of the framework’s performance. Concurrency plots (Figure 8a,b) reveal differences in orchestration efficiency, while CPU and memory plots (Figure 8c,d) quantify computational load and resource balance. The integration of these perspectives confirms that the parallel configuration not only reduces runtime but also maintains stable system performance, validating the design of the Multi-Cube framework for scalable, large-scale time series processing.

Finally, in both the sequential emulation and the adaptive parallel execution modes, the framework successfully processed the three AOIs introduced in Figure 6, validating its robustness under heterogeneous tile coverage scenarios. Figure 9 presents the final outputs produced by the external algorithm within the Multi-Cube framework. Figure 9a,b display the three processed areas. The area with incomplete spatial coverage corresponds to Area C introduced in Figure 6. This apparent incompleteness is not due to missing data; rather, it reflects the automatic cube subdivision mechanism activated when none of the Sentinel-2 tiles provide full coverage of the original AOI, resulting in independent sub-cubes that together represent Area C. In this case, the Multi-Cube architecture automatically subdivided the AOI into two independent sub-series (C-1 and C-2), which were processed consistently in both sequential and parallel configurations, yielding independent results without user intervention.

Figure 9. (a) Automatic subdivision of Area C into a spatially coherent sub-area (C-1), aligned with the native Sentinel-2 grid, as resolved by the framework due to incomplete single-tile coverage. (b) Complementary sub-area (C-2) processed in parallel, ensuring full AOI coverage. Both panels are shown superimposed over a Google Earth basemap (© Google Earth).

In contrast, for Areas A and B introduced in Figure 6, which were intersected by more than one tile on the same acquisition date, the framework systematically selected a single optimal tile, thereby preserving temporal integrity and avoiding redundant mosaicking. The results shown correspond to the application of the change detection algorithm by Deijns et al. [7]. Although a detailed evaluation of these outputs is outside the scope of this paper, the results are consistent with the expected behavior of Deijns et al.’s method. Figure 9 illustrates how the automatic selection and subdivision logic of the Multi-Cube framework consistently produces coherent and reproducible time series suitable for large-scale analyses with algorithms operating within this framework.

3.1.2. On-Demand Sub-Cube Access vs. Full-Cube Recalculation

In this experiment, we compared on-demand extraction of a spatiotemporal sub-cube with a full reconstruction of a data cube for a newly requested area. The requested region corresponds to a small spatial subset (Figure 10) that is fully contained within Study Area A introduced in Figure 6. In addition, the request targets a restricted temporal window of one calendar year, covering the 12 months of 2023. As this temporal range is already included in the existing stable cube, the corresponding time series is readily available.

Figure 10. The white polygon corresponds to Study Area A, whose full spatial footprint is shown to contextualize the location of the extracted subset. The yellow inner extent represents the sub-cube extracted by the framework. It is fully contained within the spatial limits of Study Area A and was used for the temporal analysis window. The background imagery corresponds to a Google Earth basemap (© Google Earth).

The marked performance advantage observed for the on-demand sub-cube extraction is a direct consequence of the framework’s baseline-aware design. When both the requested spatial extent and temporal interval are already present in a stable cube, the framework bypasses the ingestion and preprocessing stages and accesses only the minimal set of required spatiotemporal chunks. This approach avoids unnecessary cube reconstruction and substantially reduces computational overhead.

As a result, on-demand sub-cube extraction significantly lowers both I/O operations and CPU usage relative to a full cube rebuild, explaining the pronounced performance gap observed in this experiment.

For this case, we evaluated three configurations:

• Full-Cube Recalculation (Sequential Emulation), which rebuilds the entire pipeline for the AOI from ingestion to analysis, assuming no prior data exist;
• Full-Cube Recalculation (Optimized Parallel Execution), which rebuilds the entire AOI from scratch under the framework’s optimized parallel configuration;
• On-Demand Sub-Cube Access, which avoids cube reconstruction by materializing an in-memory spatiotemporal sub-cube directly from the stable cube containing the requested area, writing only the analysis artifacts and leaving the dynamic baseline unchanged.

The framework automatically resolves the source data through spatiotemporal intersection, requiring only a target sub-area and temporal range from the user. When the requested region is already contained within an existing stable cube, the system restricts I/O to the corresponding spatial and temporal chunks instead of accessing the full dataset. This selective access avoids the expensive stages associated with monolithic rebuilds, reducing latency, read/write volume, and CPU usage. The objective of this experiment was to quantify the resulting time until a result was acquired and resource savings relative to full cube reconstruction.

Table 5 compares the three configurations—Sequential Rebuild, Parallel Rebuild, and On-Demand Sub-Cube—and highlights the clear performance gains obtained when the requested sub-area is fully contained, both spatially and temporally, within an existing stable cube. The On-Demand configuration achieved the shortest wall-clock time (approximately 16.2 min), outperforming the Parallel Rebuild (approximately 22.9 min) and the Sequential Rebuild (approximately 135 min). This result corresponds to a speed-up of about 8.3× relative to the Sequential Rebuild and 1.4× relative to the Parallel Rebuild. These runtime improvements were accompanied by a drastic reduction in write I/O, with a value of only 9.3 MB for the On-Demand Sub-Cube, while those for the Sequential and Parallel Rebuilds were 2.3 GB and 2.1 GB, respectively, representing reductions of roughly 248× and 221×, reflecting the fact that the On-Demand strategy reuses the existing stable cube and, rather than rebuilding the baseline, retains only the analysis artifacts.

Table 5. Comparative summary of performance.

Metric	Sequential Rebuild	Parallel Rebuild	On Demand Sub-Cube
Elapsed (H:M:S)	02:14:57	00:22:56	00:16:14
Mean CPU (%)	7.65	45.79	69.61
RAM peak (GB) (%)	3.35	4.09	3.69
Disk read (MB, cum.)	216.1	21.6	1501.2
Disk write (MB, cum.)	2304.0	2057.0	9.3

CPU utilization also reflects this efficiency: the On-Demand Sub-Cube maintained the highest mean CPU usage (approximately 69.6%), which can be compared with the values for Parallel Rebuild (approximately 45.8%) and Sequential Rebuild (approximately 7.7%), while peak RAM usage remained comparable across runs (approximately 3.3–4.1 GB). Overall, when the analysis targets a sub-area already contained spatiotemporally within a stable cube, on-demand sub-cube extraction avoids repeated reconstruction, drastically reduces write I/O, and shortens the time until a result is obtained without increasing memory requirements.

Across the three configurations, the Disk I/O profiles in Figure 11 reveal a clear discontinuity between the rebuild-based workflows and the On-Demand Sub-Cube strategy. Both the Sequential Figure 11b and Parallel Figure 11a rebuild configurations generate 2.1–2.3 GB of cumulative disk writing, meaning that the entire cube is rewritten regardless of the requested extent. The Parallel mode improves runtime but not efficiency, as the I/O footprint remains unchanged. Conversely, the On-Demand Sub-Cube (Figure 11c) collapses the write curve to almost zero (~9 MB, a ≈248× reduction) because it reuses existing baseline cubes and writes only localized outputs instead of rebuilding the full archive. This transforms the process from a heavy storage-bound workflow into a lightweight and near-interactive extraction operation suitable for iterative or localized analyses.

Figure 11. (a) Parallel Rebuild—Full-cube recomputation with a high I/O footprint. (b) Sequential Rebuild—Full-cube recomputation with similar disk-writing accumulation over a longer runtime. (c) On Demand Sub-Cube—Minimal writing activity restricted to the requested spatial and temporal subset.

Finally, Figure 12 is the worker-level CPU heatmap for the On-Demand Sub-Cube configuration; it shows a distinctive execution pattern characterized by the selective activation of worker processes (PIDs) rather than continuous load distribution. Each PID represents a separate worker process, and only a single high-intensity burst of CPU activity can be observed, likely corresponding to the precise moment when the sub-cube extraction or analysis detection stage was triggered. Aside from this isolated event, the remaining time bins exhibit near-idle CPU levels, indicating that no persistent background workers were maintained. This behavior confirms that unlike full-rebuild workflows that continuously engage in multiple processes throughout the execution window, the On-Demand strategy allocates computational resources only at the exact moment of need, avoiding redundant cube-wide reconstruction and resulting in a highly efficient demand-driven processing profile. Together with the I/O and runtime results, this finding confirms that on-demand sub-cube access fundamentally shifts the execution model from archive-wide processing to targeted, responsive analysis.

Figure 12. CPU heatmap for the On-Demand Sub-Cube configuration. A brief, localized activation of worker PIDs occurs during the sub-cube request, showing the demand-driven nature of this mode, which is unlike the continuous load observed in full-rebuild workflows.

In summary, the performance evaluation demonstrates that the proposed Multi-Cube framework not only scales efficiently under full reconstruction scenarios but can also transition to a lightweight, demand-driven execution mode when prior baselines are available. By combining incremental storage logic with selective sub-cube access, the architecture moves away from a monolithic rebuild paradigm toward an adaptive execution model that optimizes both turnaround time and resource utilization.

4. Discussion

Recent advances in EO data infrastructures indicate a move toward cloud-native cube architectures built on STAC/COG indexing and Zarr-based multidimensional backends, as seen in platforms such as Google Earth Engine [58] and the Open Data Cube initiative [59]. These systems emphasize scalable catalog access and server-side processing, while FAIR-aligned and federated STAC initiatives [50] and tools like gdalcubes [43] highlight the benefits of lazy evaluation and chunk-level processing. However, most of these approaches prioritize virtual cube generation over the explicit preservation of incremental baseline states for reuse across executions. In this context, our contribution is complementary: we formalize a baseline-aware, incremental Multi-Cube approach in which consolidated cubes are not regenerated but rather efficiently extended or selectively accessed through on demand sub-cube materialization.

Within this broader ecosystem, the proposed Multi-Cube framework complements existing open-source cube platforms such as ODC [59] and other geospatial data cube implementations [60,61] from an architectural and operational perspective. From a practical standpoint, Multi-Cube instances may coexist with established cube infrastructures by ingesting analysis-ready products [62] generated by ODC-based pipelines, or by operating as a lightweight backend for localized or application-driven monitoring scenarios alongside larger national or continental cubes. Exploring deeper technical interoperability with existing cube frameworks is considered a valuable direction for future work, but is beyond the scope of the present study.

The Multi-Cube framework demonstrates how the combination of chunk-based storage reuse and adaptive task-level scheduling can significantly improve execution efficiency across a wide range of analytical methodologies. While other scalable EO systems have explored distributed I/O optimization [63], our approach extends these ideas by incorporating automatic spatiotemporal intersection and selective sub-cube reuse. As a result, the specific analytical methodology has limited impact on overall performance when its computations can be decomposed locally.

A key strength of the proposed system is its modular design, which enables seamless integration of external analytical algorithms without modifying the cube infrastructure. While existing EO frameworks provide extensible architectures for large-scale time series analysis [64,65], they generally focus on catalog-driven access and platform-specific pipelines rather than reusable data baselines that external algorithms can directly consume. Recent reviews highlight these interoperability gaps and limited analytical coupling in systems such as Open-Data-Cube-style stacks, on-demand cubes, and Google Earth Engine, underlining the need for architectures that decouple storage from analytical engines [43,62].

This architectural decoupling is increasingly relevant given the recent introduction of task-specific and learning-based EO methods, such as fast region-merging strategies and deep segmentation approaches based on superpixel representations and graph-based models applied to SAR and optical imagery [66,67], creating a need for flexible and reusable data backends. In contrast, the Multi-Cube framework provides standardized time series interfaces that allow third-party models to connect directly while maintaining incremental baseline updates.

Although the performance benchmarks presented in this study were achieved on a high-end workstation, the observed performance trends are primarily driven by architectural design choices rather than hardware-specific optimizations. The main efficiency gains arise from incremental baseline reuse, chunk-level data access, and the avoidance of full archive reprocessing, approaches that remain effective across different computational environments. On more-standard workstations with fewer cores or limited memory, absolute runtimes would increase; however, the relative benefits of selective execution and on-demand sub-cube materialization are expected to persist, as I/O reduction and demand-driven computation remain dominant factors. In cloud-based environments, the Multi-Cube design aligns naturally with object storage and Elastic Compute models, where chunked Zarr storage and task-level parallelism enable horizontal scaling while minimizing data movement and recomputation costs.

The performance results presented in Section 3 indicate both a conceptual and practical shift from rebuild-centric workflows—in which full cubes are reconstructed regardless of the requested spatiotemporal extent—toward a reuse-first execution model that prioritizes minimal I/O and selective computation. This behavior aligns with recent discussions on Earth System Data Cubes, modular EO backends, and cloud-native pipelines, which emphasize semantic, request-driven access to subsets and the importance of controlling chunk locality and avoiding redundant writes for cost-efficient infrastructures [33,34,60]. In our experiments, the On-Demand Sub-Cube mode reduced cumulative writing operations from gigabytes to megabytes by reusing stable temporal baselines and triggering computation only at the sub-cube level. This selective execution strategy avoids redundant data duplication, improves turnaround time, and mitigates memory pressure and CPU saturation, making the architecture particularly well suited to iterative and exploratory analyses over localized regions and evolving temporal windows.

Although the framework was validated in this study using Sentinel-2 optical imagery, its design is not mission-specific. The Multi-Cube architecture explicitly separates sensor-dependent preprocessing steps from the core incremental logic, which relies on standardized spatiotemporal metadata, explicit grid definitions, and stable baseline management. Consequently, extending the framework to other medium-resolution optical missions, such as Landsat, which is commonly adopted in large-scale EO data management programs [61], primarily requires adapting radiometric normalization, cloud-masking strategies, and grid alignment, while the incremental cube construction, hashing strategy, and update policies remain unchanged. Lower-resolution missions, such as MODIS, introduce additional considerations related to spatial aggregation and temporal compositing, but these can be accommodated within the same modular architecture.

Finally, future work will focus on extending the Multi-Cube framework toward more adaptive and intelligent execution strategies. This will include exploring adaptive worker throttling and memory-aware scheduling within the analysis modules as well as expanding support for multi-sensor workflows and additional classes of analytical algorithms that can be coupled with the incremental execution model. These endeavors aim to further refine the balance between localized bursts of parallel computation and controlled resource usage while broadening the applicability of the framework to more diverse EO analytics and monitoring scenarios.

Limitations and Edge Cases

While the proposed Multi-Cube provides clear advantages for incremental data management and on-demand sub-cube extraction, some limitations and edge cases should be acknowledged. The framework was validated in this study using optical satellite imagery and cloud-oriented data access patterns; extending it to other sensing modalities, such as SAR or local-file-based workflows (large collections of GeoTIFFs), may require complementary preprocessing and ingestion strategies beyond the scope of this work.

The framework is primarily designed for large-scale satellite-data-processing workflows, where parallel execution is central. Its core efficiency gains such as adaptive sub-cube partitioning, incremental baseline reuse, chunk-level access, and avoidance of full archive reprocessing remain effective across different computational environments. However, some parallelization-oriented optimizations may yield smaller absolute speedups for resource-constrained systems, including with respect to memory availability. In such cases, performance is bounded by available hardware resources, even though the relative benefits are preserved.

Although the architecture itself is algorithm-independent, the experimental validation presented here focuses on analytical workflows based on satellite time series. Other analytical pipelines were not evaluated, reflecting the scope of the current validation rather than a limitation of the framework. Future evaluations may explore a broader range of analytical strategies within the same architectural design.

In this context, methods with strong global dependencies may exhibit smaller relative performance gains when incomplete coverage requires subdividing the requested area into multiple sub-cubes. This behavior reflects the inherent difficulty of decomposing globally coupled computations rather than a limitation of the proposed architecture.

Finally, the proposed architecture is primarily intended for recurrent monitoring over well defined AOIs, where incremental updates and repeated analytical queries are expected. This design emphasis reflects the framework’s focus on efficient reuse and stable behavior across recurring analytical requests.

5. Conclusions

This article details an incremental Multi-Cube architecture designed to optimize the management and analysis of satellite image time series. The proposed framework departs from conventional full-rebuild or catalog-centric cube paradigms by introducing the baseline preservation of on-demand sub-cube access, allowing computational processes to reuse pre-existing temporal baselines rather than regenerate the entire cube for each request. This approach enables efficient temporal extension and localized analysis without compromising spatial or temporal consistency.

At the core of this architecture lies an automated decision-making engine that orchestrates how analytical requests are executed. Rather than relying on static ingestion or processing workflows, this engine dynamically partitions areas of interest into spatiotemporal sub-cubes, evaluates data availability and coverage, and coordinates parallel task execution based on predefined policies and available computational resources. By embedding this orchestration logic within the framework, computation is aligned with data locality, reuse, and parallelism, forming the basis for the observed efficiency gains.

Through comparative performance evaluations, the architecture demonstrated that reusing existing baselines and materializing only the requested sub-cubes drastically reduces I/O overhead and processing time. The On-Demand mode achieved reductions of more than two orders of magnitude in cumulative writing operations compared to the full-rebuild workflows while maintaining consistent data integrity. These results confirm that selective execution and incremental storage can allow practically real-time responsiveness in analytical workflows involving high-dimensional EO data.

Beyond computational gains, the architecture establishes a generalizable, mission-agnostic model for scalable and adaptive cube infrastructures capable of accommodating dynamic multi-sensor datasets and future satellite missions with varying spatial, spectral, and temporal resolutions. By combining chunk-aligned data partitioning, baseline reuse, and demand-driven orchestration, the framework advances the notion of computational frugality in cloud-native EO systems by prioritizing selective computation over exhaustive recomputation. This design principle represents a paradigm shift toward more sustainable and interactive modes of EO analysis.

In addition, the system’s architecture supports the direct incorporation of independently developed analytical algorithms, allowing researchers to connect their own models that operate on satellite time series. This plug-and-play capability broadens the applicability of the framework beyond its native modules, providing an open and extensible platform that can accommodate evolving methodologies and diverse analytical engines.

Future research will focus on adaptive parallelism and memory-aware scheduling, extending the framework to hybrid optical radar environments and AI-assisted change detection pipelines. The goal of integrating these strategies is to consolidate the proposed system into a fully responsive backend for large-scale multitemporal environmental monitoring, supporting both research and operational deployments.