Abstract
One of the main problems faced by database administrators for optimizing analytic workloads is fragmentation. Therefore, in recent decades, several fragmentation methods for analytical platforms have been proposed because this technique is able to improve the performance of OLAP (Online Analytical Processing) queries. In this study, we conducted an exploratory review of horizontal fragmentation methods for analytical repositories such as data warehouses, data lakes, and data lakehouses. This study presents a scoping review conducted using Arksey and O’Malley’s methodological framework and reported according to the PRISMA guidelines, covering 58 primary studies on horizontal fragmentation published from 2015 to 2025. Our analysis focuses on five aspects: (1) determining the main techniques used in horizontal fragmentation works for analytical repositories, (2) the classification of these studies, (3) the performance metrics considered when evaluating the horizontal fragmentation scheme, (4) the information type indexed by the repositories, and (5) the technologies most used by the approaches. Our findings suggest that horizontal fragmentation is a good opportunity to improve the performance of analytical workloads in most cases. The results of this scoping review will provide important guidelines for future research on horizontal fragmentation methods. In addition, the results will provide clues about the use of OLAP technologies for professionals and academics considering future directions.
1. Introduction
First introduced in the 1980s, data warehousing provides the architecture and tools necessary for business executives to systematically organize, understand, and use data to make strategic decisions. Data warehousing systems have become valuable tools in today’s competitive and rapidly changing world. In recent years, numerous companies have invested millions of dollars in developing enterprise-wide data warehouses, driven by the fact that data warehousing enables distributed access to up-to-date business knowledge across departments and functions, thereby improving business efficiency and customer service [1,2,3]. Originally, data warehouses focused on structured data to support decision-making; however, the growing volume of data and new data sources, such as Internet of Things (IoT) devices and social media, have put the original data storage infrastructure to the test. Therefore, alternative approaches have been developed, such as data lakes and later data lakehouses, to address these challenges [4].
Data lakes emerged as a flexible, scalable, and low-cost alternative to overcome the structural and economic constraints of traditional data warehouses, marking a shift toward Big Data-oriented analytical infrastructure. Instead of imposing strict structural limitations on data, data lakes simply store all data, whether structured or unstructured, in a central location. The data lake promised to be a democratizing force, freeing the enterprise to drink from a fountain of unlimited data. Data lakes started with the HDFS (Hadoop Distributed File System). As cloud adoption grew, these data lakes migrated to cloud-based object storage, offering extremely low storage costs and unlimited capacity.
Despite the promise and hype, data lake 1.0 had serious shortcomings. Data grew to unmanageable sizes, with few tools for schema management, data cataloging, and discovery. Data processing was also challenging. Mundane data transformations, such as joins, were a huge headache to code as MapReduce jobs. In response to the limitations of first-generation data lakes, several players have sought to improve the concept to fully realize its promise. For example, Databricks introduced the concept of a data lakehouse. The lakehouse combines the controls, data management, and data structures of a data warehouse while still hosting data in object storage and supporting a variety of query and transformation engines. In particular, the data lakehouse supports ACID (Atomicity, Consistency, Isolation, and Durability) transactions, a significant departure from the original data lake, where data is dumped and never updated or deleted. The term data lakehouse suggests a convergence between data lakes and data warehouses [5].
In recent decades, several fragmentation (a.k.a. partitioning) methods for data warehouses have been proposed because this technique can reduce the response time of OLAP (Online Analytical Processing) queries and offers significant advantages during table loading and maintenance operations [6]. Fragmenting a data warehouse (DW) is more complex and challenging than fragmenting a relational database due to the variety of options for fragmenting a star schema. In the DW, either the dimension tables or the fact table, or both, can be fragmented [7].
Most of the fragmentation methods for data warehouses focus on horizontal fragmentation because fact tables typically have a high data volume, i.e., there are millions of tuples [8] and queries usually only require a subset of them. Horizontal fragmentation partitions a relation along its tuples. Thus, each fragment has a subset of the tuples of the relation. There are two versions of horizontal partitioning: primary and derived. Primary horizontal fragmentation of a relation is performed using predicates that are defined on that relation. Derived horizontal fragmentation, on the other hand, is the partitioning of a relation that results from predicates being defined on another relation [9]. Instead of splitting a fact table based on all the select predicates used by the queries, it is recommended that the Database Administration (DBA) first identify the relevant dimension table(s), fragment them, and propagate their fragmentation schemes to the fact table. Therefore, several methods consider the primary horizontal fragmentation of the dimension tables and the derived horizontal fragmentation of the fact table. This procedure could reduce the complexity of the partitioning process by discarding the selection predicates corresponding to the undesirable dimension tables [10].
Horizontal fragmentation methods can also be static or dynamic; the former obtain a horizontal fragmentation scheme (HFS) optimized for a given workload, but they do not adapt the HFS when access patterns change. In contrast, the latter modifies the HFS when it degrades to maintain its quality.
Horizontal data partitioning has also been applied in data lakes to divide large datasets into manageable, discrete partitions. Each partition is managed as an individual data entity. Using this technique improves query performance, optimizes data management, and is compatible with data lake architecture. Therefore, horizontal data partitioning is an integral aspect of data lakehouse (DLH) architecture as well, since DLH combines the best aspects of a data lake and a data warehouse, and horizontal partitioning is considered essential in this configuration, as it enables efficient management and faster access to data [11].
This paper presents a scoping review on fragmentation methods for data warehouses, data lakes, and data lakehouses from 2015 to 2025. This review aims to provide an evaluative perspective on the application of horizontal fragmentation, the challenges addressed by authors, and the results obtained. In addition, we analyzed the cost models used by the methods and their characteristics and identified the most relevant performance metrics the authors used to evaluate their fragmentation scheme. Furthermore, this work has also identified the most used technologies, benchmarks, and database management systems for fragmentation.
We identified some related reviews that examine and compare data management architectures, particularly data warehouses, data lakes, and data lakehouses, across different domains. These works investigated how platforms influenced scalability, governance, data quality, analytical efficiency, and their integration into machine learning and Big Data workflows. Most studies emphasized the constraints of data warehouses in processing unstructured data, the governance and consistency issues in data lakes, and the advantages of data lakehouses in combining the strengths of both systems. In Jayabalan [4] the analysis examined how data warehousing has evolved alongside emerging approaches, including cloud solutions, data lakehouses, and the data mesh model. Based on a review of the literature, the paper identified areas that still require advances or improvements within this technological field. The results indicated that current data warehousing architectures should be characterized by scalability, agility, and interoperability. Finally, they highlighted various opportunities for future development that support sustainable and ethical data management practices. In the same context, Ait Errami et al. [12] found that traditional methods of spatial data management in spatial Big Data environments proved insufficient to meet new technological requirements. The review analyzed these approaches and described how their evolution resulted in the emergence of the data lakehouse model. It then described how this paradigm could be adapted and extended to provide efficient support for large-scale spatial data, offering suggestions for the components and best practices necessary to design a Spatial Data Lakehouse architecture optimized for storing and processing this data. Likewise, Harby and Zulkernine [13] conducted a comparative analysis between data lake, data warehouse, and data lakehouse architectures to identify their strengths, limitations, and current requirements for managing and analyzing large volumes of data. Based on this assessment, the authors proposed an improved data lakehouse design aimed at greater efficiency and data unification. In addition, they performed experimental validation using the HDFS as a data lake, Hive as a data warehouse, and Delta Lake as a lakehouse to run analytical queries on a real dataset, demonstrating that the lakehouse architecture can provide better performance and support for modern analytical workloads.
The main difference between our scoping review and existing reviews on analytics platforms is that our review specifically focuses on the role of horizontal fragmentation across data warehouses, data lakes, and data lakehouses. Our scoping review aims at describing the current state of research on horizontal fragmentation by analyzing five aspects: (1) the main techniques used in horizontal fragmentation methods on analytical platforms, (2) the classification of horizontal fragmentation methods, (3) the performance metrics most commonly considered, (4) the application domains of methods, and (5) the most frequent technologies used by authors to develop their methods. The review covers the scientific literature published between 2015 and 2025.
2. Materials and Methods
Our review draws upon Arksey and O’Malley’s [14] methodological framework for conducting scoping studies, complemented by Levac’s [15] recommendations for refining this approach. In addition, we adopted the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines proposed by Moher et al. [16] for reporting systematic reviews and meta-analyses, along with the PRISMA-ScR extension. To guide the organization and presentation of the results, we followed the recommendations of Tricco et al. [17]. The scoping review process was structured into five stages: (1) defining the research questions, (2) identifying relevant studies, (3) selecting eligible studies, (4) charting the data, and (5) collating, summarizing, and reporting the findings.
2.1. Research Questions
We established five research questions that structured the scope of our review, facilitated the attainment of the study’s objectives, and provided guidance throughout the review process.
- RQ1. What are the main characteristics of horizontal fragmentation methods across data warehouses, data lakes, and data lakehouses?
- RQ2. How are the horizontal fragmentation methods classified?
- RQ3. What performance metrics are commonly considered to evaluate horizontal fragmentation schemes?
- RQ4. What type of information has been indexed within the horizontally fragmented data warehouses, data lakes, and data lakehouses?
- RQ5. What are the main technologies used by horizontal fragmentation methods in analytical platforms?
2.2. Inclusion and Exclusion Criteria
For this scoping review, an initial search was conducted across major electronic journal databases to compile an exhaustive bibliography of relevant research on fragmentation. The digital libraries considered were: (1) ACM Digital Library, (2) IEEE Xplore Digital Library, (3) Science Direct (Elsevier), (4) Springer Link, and (5) Google Scholar. In addition, a keyword-based search strategy was applied to select the most relevant articles published between 2015 and 2025. Table 1 presents the set of keywords considered, which were applied both individually and in combination through the logical operators “AND” and “OR” to expand the scope of the search results. Based on these terms, the subsequent queries were constructed to locate primary research within the chosen repositories.
Table 1.
Keywords and related concepts.
- ‘Horizontal fragmentation’ AND (‘Data warehouse’ OR ‘Data lake’ OR ‘Data lakehouse’ OR ‘OLAP’ OR ‘ETL’)
The preliminary analysis of this query revealed additional relevant terms associated with different database architectures. To capture these, a second query was constructed.
- 2.
- (‘Horizontal Fragmentation’ OR ‘Partitioning’ OR ‘Partition pruning’ OR ‘sharding’ OR ‘Query pruning’ OR ‘Bucketing’ OR ‘Hashing’ OR ‘Data Compaction’) AND (‘Data Warehouse’ OR ‘Data Lake’ OR ‘Data Lakehouse’ OR ‘OLAP’ OR ‘ETL’ OR ‘Extract-Transform-Loading’ OR ‘On-Line Analytical Processing’)
Ultimately, the PRISMA model served as the basis for structuring, organizing, and presenting the findings of this review.
2.3. Study Selection and Eligibility
Once the search process was completed, 1979 relevant results were identified: 1456 from ACM Digital Library, 230 from IEEE Xplore Digital Library, 75 from ScienceDirect, 60 from SpringerLink and 158 from Google Scholar. After removing duplicates, 1889 articles were retained for the initial analysis, which required classification based on their title and abstract. From this set, a full-text reading was conducted for 350 papers, of which 58 were ultimately included in the scoping review (see Figure 1).
Figure 1.
Study selection process—PRISMA diagram flow.
From the initial set of 1979 studies, we retained only those whose abstracts contained at least two of the keywords listed in Table 1. Studies not related to horizontal fragmentation in OLAP platforms were excluded, resulting in a refined pool of 350 studies: 120 from IEEE Xplore Digital Library, 140 from ACM Digital Library, 40 from ScienceDirect (Elsevier), 20 from SpringerLink, and 30 from Google Scholar.
These works were subsequently evaluated according to our predefined exclusion criteria: studies that did not propose horizontal fragmentation or were not relevant to the research scope were excluded. Additional reasons for exclusion included papers written in languages other than English, unpublished working papers, textbooks, and master’s or doctoral dissertations. Ultimately, 58 primary studies met all eligibility requirements and were included in the scoping review. Full-text versions of each study were downloaded to ensure a thorough analysis. As illustrated in Figure 2, most of these studies (89.7%) were published between 2018 and 2025.
Figure 2.
Annual distribution of primary studies.
With respect to the source, most of the primary studies were obtained from ACM (17) and Google Scholar (17), followed by IEEE Xplore (10) and Springer Link (9). Research articles retrieved from Science Direct (5) were less frequent. The selected studies were classified according to the quality of their publication media. Specifically, journal articles were classified by quartile ranking in the Journal Citation Reports (JCR) according to Web of Science (WOS), while conference papers were classified using the Computing Research and Education (CORE) classification system. Figure 3 shows the classification of the selected works. This additional analysis provides a clearer picture of the scientific impact and prestige of the publication media included in our review.
Figure 3.
Primary studies by JCR and CORE ranking.
Figure 4 illustrates the geographical distribution of our primary studies. As can be seen, most were conducted in the United States, followed by Algeria and Germany.
Figure 4.
Geographical distribution of primary studies.
2.4. Data Collection and Analysis
Once the primary studies included in the review had been selected, their bibliographic and content data were extracted. The bibliographic data included the research title, authors, the type of platform considered, the consulted database, and the type of publication, classified in WoS and CORE, when applicable. The content data corresponded to the information extracted from each study that contributed to answering the research questions defined in Section 2.1.
3. Results
The selected research papers were classified according to the following seven criteria: (1) classification of method; (2) cost model; (3) dynamic; (4) restriction of the number of fragments; (5) validation; (6) type of storage; and (7) type of indexed information. Table 2, Table 3, Table 4, Table 5 and Table 6 show the main results for the analyzed methods for data warehouses, data lakes, and data lakehouses, respectively. NS means not specified.
Table 2.
Comparison of the main characteristics of horizontal partitioning works for data warehouses.
Table 3.
Comparison of key features of machine learning-based horizontal fragmentation methods in data warehouses.
Table 4.
Comparison of key features of methods based on other horizontal fragmentation approaches in data warehouses.
Table 5.
Comparison of the main characteristics of horizontal fragmentation methods in data lakes.
Table 6.
Comparison of the main characteristics of horizontal fragmentation methods in data lakehouses.
- Classification of HF method: Classification according to the HF technique used to obtain a horizontal fragmentation scheme.
- Cost model: We determined if the method uses a cost model to provide an estimate of resource consumption according to the frequently used query load. Cost models generally consider factors that reduce total cost, including Central Processing Unit (CPU), Input/Output (I/O), and communication costs.
- Dynamic: This aspect determines if the horizontal methods consider adapting the current fragmentation scheme to deal with the evolution of the workload.
- Restriction on the number of fragments in the fact table: It considers whether a maximum number of fragments was established by the DBA. This restriction is important to avoid excessive fragmentation that may affect query optimization, storage management, or system maintenance.
- Validation: This criterion indicates whether synthetic data (benchmark) or real data were used to validate the fragmentation method. Some benchmarks utilized are SSB, TPC-H, and TPC-DS; on the other hand, real data examples are experimental workloads, real-world datasets, and sales records generated for evaluation purposes.
- Type of storage: This aspect considers the data model and Database Management System (DBMS) used (e.g., relational, distributed or parallel) by each horizontal fragmentation technique, and it also reflects the architecture of the analytical platform where the fragmentation method was evaluated.
- Type of indexed information: It specifies if the horizontally fragmented repository indexed information that was structured, unstructured, or semi-structured. To complement this section, information has been added about the areas in which the proposed fragmentation methods have been applied, such as business, finance, geospatial, or telecommunications.
The papers presented in Table 2 are those partition-based works; they consider the common techniques provided by the DBMSs like range, hash, list, and round-robin, but also modern approaches such as micro-partitions. Partition-based approaches are strategies that divide a data warehouse table into fragments according to predefined partitioning attributes, such as time or region, without explicitly considering the query workload. The usefulness of machine learning in the horizontal fragmentation of a data warehouse lies in its ability to analyze query patterns and relationships among predicates or attributes to design more efficient HF schemes. Through techniques such as clustering (e.g., k-means or hierarchical clustering) and optimization algorithms, it is possible to group queries or predicates with similar behavior, facilitating the generation of fragments that better match the workload. Table 3 lists ML-based approaches. Table 4 lists the main characteristics of each horizontal fragmentation method that is not based on partitioning or ML, and includes approaches based on cost, graph, hierarchy, and heuristics, among others.
Table 5 contemplates the horizontal fragmentation approaches developed in data lakes. These works, although minor in the literature, show that the partition-based approach has been most widely used in this type of repository, followed by heuristics. Finally, Table 6 summarizes the main aspects of studies addressing horizontal fragmentation in lakehouse repositories, highlighting the main techniques, validation methods, and storage used, as well as the type of information indexed. As in the previous tables, partition-based approaches emerge as the predominant strategies for improving the performance of these analytical platforms.
Analysis of the articles included in this scoping review indicates that static fragmentation schemes predominate in data warehouse platforms (31), while a smaller number of studies incorporate cost models (22) and restrict the number of fragments generated (17). In these environments, validation is mainly performed using benchmarks, and approaches are mostly based on range, list, and hash partitioning. In contrast, in data lake platforms, few studies consider cost models (2) and focus on dynamic schemes; the evaluation using benchmarks is geared toward large data volumes, such as Table Union and Yahoo Streaming Benchmarks. Finally, in data lakehouse architectures, there is a slight increase in the use of dynamic approaches. In addition, validation is mainly based on benchmarks such as TPC-H, TPC-DS, OpenMessaging Benchmark, and synthetic datasets based on a set of wind farms, rather than standardized benchmarks, favoring partitioning strategies such as range and hash. The following section describes the methods analyzed in greater detail.
4. Discussion
This section describes our findings, which are categorized according to our research questions.
4.1. RQ1. What Are the Main Characteristics of Horizontal Fragmentation Methods Across Data Warehouses, Data Lakes, and Data Lakehouses?
Horizontal fragmentation (HF) is an optimization technique widely used to reduce the execution cost of star–join queries in relational DWs. These queries contain multiple joins and complex select operations involving fact tables and multiple dimension tables. Star–join queries are the most expensive OLAP queries and often imply a high level of communication [42]. In this section, we present the main characteristics of the 58 HF techniques analyzed.
- Cost model: After analyzing the papers, it was found that 23 of them contained cost models for horizontal fragmentation approaches. According to [9], in a distributed database system, the total cost to be reduced includes the Central Processing Unit (CPU), Input/Output (I/O), and communication costs. The CPU cost is incurred when executing operators on data in memory. The I/O cost is the time required to access the disk. This cost can be minimized by reducing disk access through fast data access methods and the efficient use of main memory (buffer management). The communication cost is the time required to exchange data between the sites involved in executing the query. This cost occurs in message processing and data transmission over the communication network. The first two components measure the cost of local processing. Network topology significantly influences the relationships among these components. In a wide-area network such as the Internet, communication costs are often the dominant factor due to the greater distances over which data is retrieved (or sent). Of the cost models found, 20 were proposed for data warehouses (e.g., [54,55,57,59]), two for data lakes [68,69] and one was considered in a data lakehouse [74]).
- Dynamic: 58 research articles published between 2015 and 2025 were analyzed, and it was observed that they mainly focus on static fragmentation; only 20 articles consider adapting the current fragmentation scheme to cope with the evolution of the workload and are considered dynamic. For example, in [41], the authors stated that most of the strategies for DW fragmentation focus on relational DW fragmentation and ignore OLAP cubes, although they are the first affected by multidimensional user queries. To address this problem, a dynamic fragmentation strategy for OLAP cubes using association rule mining was proposed. This strategy first analyzes queries from the log files utilizing the Apriori algorithm to identify the most frequent predicate itemsets. Then, the partitioning algorithm obtains the best partitioning criteria by the itemsets. The partitioning algorithm is executed whenever the OLAP cubes suffer performance degradation. Other representative dynamic works for data warehouses are [18,21,22,23,31,32,33,44], while in data lakes, there is a limited number of works [68,69]. In Ref. [69], the authors proposed an approach that continuously adapts partitioning and storage decisions based on access patterns and constantly evolving workload characteristics, periodically updating predictions and reorganizing data partitions when new query behaviors are observed. In the case of lakehouses, three works are considered dynamic [70,72,74].
- Restriction of the number of fragments: Furthermore, 17 of the 58 approaches (29%) considered that the analytical platform administrator should provide the maximum number of HF fragments; therefore, they are classified as works with restrictions or limitations. There are 15 works that consider the restriction of the number of fragments in data warehouses, for example, in [36], where the authors proposed a methodology based on statistics (Jaccard index), data mining (hierarchical clustering), and meta-heuristics (particle swarm optimization) to solve the problem of horizontal fragmentation in data warehouses using a relatively large workload. First, it calculated the attraction between the predicates using the Jaccard index, followed by a hierarchical grouping of the predicate set with the Ward algorithm. In the second step, it used DPSO (Discrete Particle Swarm Optimization) to select the best HFS. Other works considered with restrictions on the number of fragments in data warehouses are [25,27,37,45,49,54,58], while only [68,71] consider this aspect in the context of a data lake and lakehouse, respectively, because in these architectures, recent fragmentation approaches tend to avoid restricting the number of partitions, as modern distributed engines, scalable object storage, and advanced table formats enable more efficient management and parallel processing of large numbers of fragments.
- Validation: 19 benchmarks were used to evaluate horizontal partitioning methods on the three analytical platforms. Figure 5 shows that the most used benchmark in data warehouses was TPC-DS, which in [39] was used to generate the data warehouse and the analytical query workload to evaluate the performance of the proposed partitioning and bucketing strategy. Similarly, in [43] SSB was used alongside TPC-DS and TPC-CH to evaluate the effectiveness of the proposed Deep Reinforcement Learning (DRL)-based partition advisor by comparing query execution times with traditional partitioning heuristics. For data lakes, the benchmarks used were Yahoo! Streaming Benchmark (YSB), ESP Bench, Table Union Benchmark, TPC-DS, and TPC-H, the latter being the most widely used. Finally, the data lakehouse approaches used the OpeningMessaging benchmark, which, in [73], was used to generate event streams and evaluate Ursa’s streaming performance, measuring throughput and latency under different workloads. TPC-DS [72] and TPC-H [72,74] were also used in these repositories. In [72], TPC-DS and TPC-H benchmarks were used to evaluate the performance of the BigLake system and the optimizations implemented in BigQuery and Spark. TPC-H is the only benchmark utilized on three platforms [18,29,32,56,65,69,72,74]. On the other hand, horizontal fragmentation studies that do not use a benchmark to evaluate their scheme use both real data (e.g., [19,57,61] with CRM and ERP datasets of a cellular company, tables with real production data and sales data, respectively, and [38] with real datasets CovType and SEP85L, STCO-MR2010 AL MO, and OnlineRetail), and synthetic datasets from different domains; for instance, Bakken and Soylu [70] refer to a set of wind farms. Figure 6 shows the main forms of validation when no benchmark was available.
Figure 5.
Benchmarks used for horizontal fragmentation method validation.
Figure 6.
Validation of the methods without a benchmark.
- Data model and data storage used: Figure 7 shows that the data model most utilized for the implementation of the methods was the Relational one, followed by Distributed. Some HF works do not specify the kind of data storage used, but the most widely employed DBMS for data warehouses were PostgreSQL [11,24,30,42,53,56], followed by the Distributed DW Hive [23,27,39,42,53] and Oracle [19,36,51]. In the case of repositories as data lakes, the most widely used data model was Cloud Object Storage, with S3 and Azure Data Lake Gen 2, while for data lakehouses, Cloud Object Storage also prevailed with Google Cloud Storage and S3. However, these types of repositories are trending toward the use of agentic data platforms (Dremio), which organize information around intelligent agents, their memory, status, and actions, allowing Artificial Intelligence (AI) systems to make decisions and execute tasks autonomously, and In-Memory Hybrid Transactional/Analytical Processing (HTAP), which is a database architecture that allows transactional processing (OLTP) and analytical processing (OLAP) to run simultaneously on the same system and the same data, using main memory (RAM) to achieve very high performance. Figure 8 shows the classification of the DBMS by analytical platform. PostgreSQL was used in most studies because it provides native support for horizontal fragmentation via declarative partitioning, thereby improving query performance and data lifecycle management. Its range, list, and hash-based models enable optimizations such as partition pruning and sequential scans, although its benefits must be empirically validated using performance metrics (PostgreSQL: Documentation: 18: 5.12. Table Partitioning Available online: https://www.postgresql.org/docs/18/ddl-partitioning.html#ddl-partitioning-declarative (accessed on 5 January 2026)). In addition, Hive provides native mechanisms for optimizing data ingestion and query execution in Hadoop environments through partitioning and clustering (bucketing). Partitioning logically decomposes a table into subsets based on the values of one or more attributes (e.g., temporal or spatial dimensions), enabling queries to scan only the relevant partitions. In turn, clustering organizes data within partitions into buckets based on hash functions applied to specific columns, significantly enhancing join operation efficiency and improving overall query performance in large-scale analytical workloads (IBM Integrated Analytics System Available online: https://www.ibm.com/docs/en/ias?topic=movement-importing-data-partitioned-clustered-hive-tables (accessed on 19 January 2026)). Finally, Oracle offers a great evolution of horizontal fragmentation, as it supports various fragmentation methods: composite fragmentation methods include all possible combinations of basic methods (range, hash, and list) and support column fragmentation, where a table is decomposed using a virtual attribute defined by an expression, using one or more existing columns of a table, and storing this expression as metadata; it was also the first capable manager to perform referential fragmentation to divide a table using a parent–child relationship, which is similar to derived fragmentation (Oracle Partitioning Extreme Data Management and Performance, 2023).
Figure 7.
Data model used by the methods.
Figure 8.
DBMS used by the methods.
4.2. RQ2. How Are the Horizontal Fragmentation Methods Classified?
In the literature, various algorithms have been proposed to obtain a horizontal fragmentation scheme for a data repository. After analyzing 58 papers on horizontal fragmentation, it was determined that there are several main groups of data warehouse, data lake, and data lakehouse approaches. One way to classify the HF works is according to their capability to adapt the HFS to the changes in the workload into static, e.g., [35,38,40,46,50,51,52,63,65,70,73], or dynamic-like [18,41,44,56,68,69,70,72,74]. There are also two types of algorithms: unconstrained algorithms, for example [18,41,43,44,69], and threshold-based algorithms, for instance [25,36,39,45,47,49,55,56,57,59,68,71]. Unconstrained algorithms select a fragmentation scheme for a database without concern for the number of fragments generated. Without control over this number, a given fragmentation algorithm can generate a very large number of fragments that become difficult to manage [58].
Figure 9 shows the main classification of horizontal fragmentation methods in data warehouses, which are divided into eight groups: (1) Partitioning [25], which includes hash [65], list [24], and range [24] approaches, to mention a few. These works take advantage of the horizontal partitioning techniques provided by DBMSs. For example, Romero et al. [18] applied the range partitioning of HBase, called “regions”, to develop their approach for analytics in Hadoop. Likewise, in [21], users can apply round-robin or hash partitioning in Amazon Redshift. In contrast, in [24], the user specifies the key and the partitioning strategy list or range to fragment a table. (2) Machine learning-based works that apply association [38,39], clustering [42,46,64], classification [40], and reinforcement learning [43,44] algorithms to obtain an HF. For instance, Ramdane et al. [39] applied the frequent itemset mining technique to select the more frequent attributes to partition and distribute the data warehouse. (3) Cost-based methods like [18,49] that focus on minimizing the number of inputs/outputs required for executing the workload. Similarly, Kechar and Nait-Bahloul [58] proposed selecting the partitioning predicates to fragment the fact table, considering their access frequency and selectivity. The HF method developed evaluated the partitioning scheme using a cost model that also measures the expected number of disk I/Os implied by query processing. (4) Graph, for example [60], that builds a Join Multi-Graph to represent the workload history of a customer for data distribution. (5) Meta-heuristic [54,62], which considers genetic algorithms [47] and particle swarm optimization [36] to select the best HF scheme. (6) Predicates, for instance [51], that consists of three steps: 1. Obtaining a predicate and query usage matrix, 2. Selecting the fragmentation predicates based on the number of occurrences of selection predicates and their access frequency, and 3. Forming partitions according to the fragmentation predicates. (7) Heuristic; one example is [11], which proposed partitioning a global schema obtained integrating heterogenous data sources. And (8) Others, which include the HF works based on hierarchy [50], Map-Reduce [53], and Tree [61].
Figure 9.
Classification of horizontal fragmentation methods for data warehouse.
Figure 10 shows that for the data lakes, there are six main groups: (1) Heuristic HF approaches [63,67]; (2) Machine learning-based methods that apply clustering techniques [64]; (3) Partitioning proposals, including hash and range [65,68] methods; (4) Cost-based works [69]; (5) Graph-driven strategies [69]; and (6) Timestamp [66] partitioning, which is applied to reduce data redundancy. For data lakehouses, the approaches are based on partitioning conforming to range and hash [70,71,72,73,74] techniques.
Figure 10.
Classification of horizontal fragmentation methods for Data Lakes.
4.3. RQ3. What Performance Metrics Are Commonly Considered to Evaluate a Horizontal Fragmentation Scheme?
The capacity to efficiently handle load reflects the performance of an application. Anything that consumes the resources of the system, e.g., the CPU, memory, and network bandwidth, is load. The main metrics for the performance of distributed systems are throughput and response time. The former measures the number of requests the application processes per second, and the latter is the time elapsed between a client sending a request to the application and receiving a response [75]. Another performance metric is latency, i.e., the duration that a request waits to be handled [76]. The total cost of processing queries is also considered in the HF works. Total cost is the sum of the time taken to process the operators of the query at several sites and in communication among sites [9]. An application is scalable if an increase in load does not degrade its performance. Several experiments were reported in the reviewed works on the application of horizontal fragmentation in OLAP repositories to demonstrate the scalability of the proposals. Also, machine learning performance metrics are used to compare HF schemes.
In the studies analyzed, 26 evaluate the performance of horizontal fragmentation through query response time, which remains the predominant metric in data warehouse, data lake, and data lakehouse contexts. This metric captures the improvement in query efficiency after applying fragmentation or partitioning techniques. In contrast, execution cost is used in a smaller subset of studies, typically alongside response time, to provide a cost–benefit perspective, especially in cost-based approaches integrated into optimizers. The main metrics considered by the methods are shown in Table 7.
Table 7.
Performance metrics used by the analyzed HF works.
4.4. RQ4. What Type of Information Has Been Indexed Within Data Warehouses, Data Lakes, and Data Lakehouses?
Most of the analyzed works focused on HF of data warehouses’ indexed structured information about the business domain. For example, in [18,36,49] the repository used for validation included business operations, specifically the benchmarks APB-1, SSB, and TPC-H. Also, Marcus et al. [56,57] developed NashDB, a method capable of adjusting the fragmentation, replication, and cluster sizing according to the workload. The authors validated NashDB using the TPC-H benchmark and two datasets provided by a large corporation. Regarding semi-structured and unstructured HF proposals for data warehouses indexed data in entertainment; for instance, in [55,59], the authors used the IMDB benchmark to validate their HF techniques for optimizing OLAP query processing. Likewise, Xu, Gao, and Davidson [28] demonstrated the benefits of their strategy for achieving eventual consistency of data warehouses with the YouTube benchmark.
A similar pattern was found for HF approaches to data lakes, as the indexed information in these repositories was predominantly from the business [65,66,68,69] domain for structured data and from the entertainment [63,64,65] industry for semi-structured and unstructured information. The data lake fragmented in [65], for example, included 466 Kafka topics from sources like IoT streams and user interactions. In contrast, Shah et al. [66] proposed R2D2 (Reducing Redundancy and Duplication in Data Lakes), which considers data partitioning via timestamp. The authors used enterprise data partitioned and stored in Parquet format for the evaluation of R2D2. In [63], the IMDB benchmark was also used to evaluate the approach to integrating data lake tables, where partitioning is applied in their Full Disjunction algorithm.
The proposals for fragmenting data lakehouses mainly indexed structured, semi-structured, and unstructured information [71,72,73]. Chattopadhyay et al. [71] highlighted the availability of near-real-time data for interactive analytics as a relevant aspect of storage in Meta’s lakehouse. Therefore, they introduced a partition state in Metastore to register them when data arrives. In [72], authors describe three features of BigLake: (1) BigLake tables which enhanced components of BigQuery storage and metadata infrastructure to improve the performance of the data lake tables, (2) BigLake Object tables to analyze unstructured data using artificial intelligence services and widely known SQL commands, and (3) Omni, a multi-cloud lakehouse. BigLake tables support metadata caching and Big Metadata to store partitioning information and file statistics in a columnar cache to achieve high performance partition and file pruning. Likewise, Merli et al. [73] designed Ursa motivated by the disadvantages of Kafka related to high complexity and costs for real-time data streaming. The performance evaluation of Ursa was accomplished with event streams generated by the OpenMessaging benchmark. Figure 11 shows the type of information indexed by the fragmented repositories.
Figure 11.
Type of indexed information by the fragmented repositories.
4.5. RQ5. What Technologies Are Most Used by Authors to Implement Horizontal Partitioning Methods in Analytical Platforms?
After analyzing different horizontal partitioning methods in analytical platforms, it was found that the most used technologies were Spark (11) [66,72], Java (10) [11,36,42,45], and Hadoop (9) [18,25,53,59], followed by Parquet (9) [35,66,67,72], and Python (6) [52,60,63,70]. This shows that technologies stand out for their fundamental role in the Big Data ecosystem, providing a mature, scalable, and widely adopted infrastructure for distributed data processing [25]. For its part, Parquet has established itself as a widely used columnar storage format, optimizing analytical query performance through techniques such as efficient compression and reduced I/O operations. It also allows data to be consumed by multiple engines for various types of workload [52] for adaptive, workload-sensitive partitioning, making it a standard choice in data lake and data lakehouse architectures. Figure 12 shows the main technologies used by the methods.
Figure 12.
Technologies used by the methods of horizontal fragmentation in analytical platforms.
5. Trends and Challenges of Horizontal Fragmentation in Analytical Platforms
After conducting a thorough analysis of the most outstanding work on horizontal fragmentation in analytical platforms, a remarkable transformation in large-scale data repositories has been identified, characterized by a shift from traditional data warehouses, which rely on schemas, to dynamic, cloud-native architectures that integrate transactional and analytical workloads in a unified environment. This technological convergence gave rise to the data lakehouse model, which combines the scalability of data lakes with the consistency and high query performance of data warehouses. There are some identified trends. Horizontal fragmentation has evolved into:
- AI-assisted fragmentation: The recent literature describes a transition to fragmentation methods that leverage artificial intelligence and machine learning techniques [38,39,41,45,46] to decide how and when data should be partitioned. These approaches identify access patterns, query frequencies, and attribute correlations to automatically recompose fragments. Instead of relying on manually determined rules, systems learn from how actual workloads behave and adjust partitions to improve query response time and make storage more efficient. This adaptive design increases query performance while optimizing resource allocation and storage utilization in real time [64].
- Dynamic, workload-sensitive fragmentation: Another trend is the implementation of adaptive partitioning mechanisms that respond to variations in queries and workload types. Studies indicate that dynamic partitioning schemes perform better in real-time analytical environments [68,69,70,74] or hybrid architectures such as Hybrid Transactional/Analytical Processing [24]. Instead of maintaining a fixed fragmentation scheme, these systems use adaptive policies that redistribute fragments whenever the system detects workload fluctuations.
- Integration into cloud-native ecosystems: Most modern analytics platforms, such as Snowflake [35] and Amazon Redshift [60], have adopted partitioning approaches that leverage cloud-native environments, providing elasticity, separation of storage and processing, and automated resource management. Integration enables sharding processes to be carried out seamlessly within distributed query engines, reducing manual configuration. In addition, multi-cloud ecosystems introduced complexity and opportunities. Currently, shards are distributed across different cloud providers or regions, improving scalability, availability, and load balancing.
- Convergence of transactional and analytical processing (HTAP): One of the most important trends in recent studies is the combination of transactional (OLTP) and analytical (OLAP) systems into a single architecture capable of handling both types of work at the same time [20,24,33]. In this type of hybrid system, horizontal sharding is key: it allows transactions to access only the data they need, while analytical queries can access organized shards for faster calculations or reviews. Thanks to this union, it is no longer necessary to separate operational storage from analytical storage, making it easier to obtain real-time information more quickly and consistently.
- Automation of the fragmentation lifecycle: Several recent studies emphasize the need to automate the entire fragmentation lifecycle, from workload detection to fragment reconfiguration and metadata updates [19,23,32]. This involves feedback loops driven by performance metrics, enabling the system to evaluate the impact of changes and automatically readjust. Fragmentation thus becomes a cyclic, self-regulating process rather than a one-time design decision—aligning with the principles of modern data engineering and agile analytics.
- Cost-aware and sustainable fragmentation: With the increasing adoption of cloud infrastructure, several studies [21,37,68,69] have begun incorporating cost and energy efficiency considerations into fragmentation decisions. Systems aim to minimize unnecessary data movement and duplication, balancing query performance with long-term operational expenses. This emerging focus on sustainability promotes cost-efficient and environmentally conscious fragmentation, where performance optimization coexists with resource efficiency in Distributed and Multitenant environments.
Despite significant progress and the increasing number of studies addressing fragmentation in analytical ecosystems, five major challenges persist and require further attention:
- Architectural integration: Some studies [20,21,24,33] highlight ongoing difficulties in achieving seamless interoperability between OLTP and OLAP systems. While hybrid architecture (HTAP) and shared storage approaches attempt to reduce redundancy, synchronization overhead remains high. Several works (e.g., in Spark SQL, Snowflake, and Redshift environments) report performance degradation when transactional consistency and analytical throughput must coexist, suggesting the need for adaptive middleware and unified data access layers.
- Dynamic cost modeling: Most cost models analyzed still assume static workloads or homogeneous fragment distributions. However, recent studies propose adaptive and workload-aware models that dynamically adjust fragmentation boundaries based on query frequency, access patterns, or data skew. Yet these approaches often lack the predictive capabilities needed to balance optimization costs with query benefits. The literature emphasizes the need for multi-objective cost models that jointly consider query response time [49,50,52,53,55,58,59], data movement cost [20,21,58], and storage elasticity [26,33,56,57,69] in cloud settings.
- Benchmarking and evaluation: Current benchmarking frameworks, such as TPC-H and TPC-DS, are insufficient for evaluating real-time, hybrid, or AI-driven workloads. Recent proposals, including synthetic or workload-driven benchmarks [66,67,73], begin to reflect emerging use cases, but lack standardization and reproducibility. The literature highlights the need for new evaluation frameworks that incorporate adaptive fragmentation behavior, distributed query planning, and machine learning-assisted partitioning.
- Redundancy and data duplication: Data lakes often accumulate redundant and duplicated datasets, leading to increased storage costs, slower query execution, and higher maintenance complexity. Detecting and managing redundancy at scale is computationally challenging and closely tied to fragmentation, partitioning, and metadata quality [63,66,69].
Overall, the analyzed body of research indicates a clear orientation toward AI-assisted, self-optimizing, and cloud-native fragmentation strategies, while underscoring the need for unified frameworks that ensure integration, governance, and efficiency across next-generation analytical platforms.
Limitations
In this subsection, we list the main limitations of this scoping review:
- Study limitations: One of the main limitations of this scoping review lies in its defined scope, which focuses exclusively on horizontal fragmentation methods applied to modern analytical platforms (data warehouses, data lakes, and data lakehouses). Consequently, this study does not include techniques related to other types of fragmentation, such as vertical or hybrid fragmentation, which could provide complementary perspectives on storage and query optimization.
- Excluded research contexts: The review excludes approaches oriented toward Big Data, particularly those centered on distributed processing without a direct relationship to data fragmentation, as well as studies focused on traditional relational or transactional databases (OLTP), whose contexts and objectives differ substantially from those addressed in the analytical domain.
- Limited technical analysis about fragmentation techniques: Another limitation of this study is that it does not provide an in-depth technical evaluation of fragmentation techniques, such as algorithmic complexity, parallelism, maintenance costs associated with insertion and update operations, or the balance of fragments in the data structure. These aspects require experimental or analytical evaluation and provide an opportunity to extend the study in future research.
- Some high-quality search sources not considered: A partial limitation of this work is that we did not consider some high-quality search sources, like WOS and Scopus, although one of the digital libraries used was ScienceDirect, where we initially found 75 works on horizontal fragmentation for analytical repositories. Also, the majority of the works found in the libraries used are indexed in WOS.
- Language restrictions: Papers in languages other than English are not included in this review, but their consideration in future research could improve the investigation.
Although the methodological decisions described limit the scope, they ensure that the review remains focused on recent advances in horizontal fragmentation within modern analytical data architectures.
6. Conclusions
Horizontal fragmentation minimizes the response time of OLAP queries on analytical platforms such as traditional data warehouses, data lakes, and data lakehouses, which has motivated various authors to develop HF methods based on different approaches to address the challenges of the Big Data era.
This comprehensive review includes scientific articles published between 2015 and 2025. The main objective was to identify the predominant approaches followed by authors when developing horizontal fragmentation methods and to analyze their characteristics. To this end, the current state of horizontal fragmentation methods in analytical platforms was reviewed, considering five aspects: (1) the identification of the main techniques used in horizontal fragmentation methods in analytical platforms, (2) the classification of these methods, (3) the performance metrics considered to evaluate fragmentation schemes, (4) the domains of application of the methods, and (5) the technologies most used in their implementation.
The results are presented as answers to the research questions posed. In the case of data warehouses, it was determined that the most used benchmarks for validating methods were TPC-DS and SSB. Similarly, the implementation of fragmentation schemes was mainly carried out in PostgreSQL and Oracle, which are Object-Relational and Relational database management systems, respectively, with special emphasis on the latency costs, specifically query response time, involved in the execution of OLAP queries.
Extending this analysis to data lakes and data lakehouses reveals a clear paradigm shift. Partitioning has evolved from a static process to a dynamic, workload-driven mechanism, often reinforced by advanced machine learning techniques such as hidden or context-aware partitioning. Unlike traditional data warehouses, these systems prioritize adaptability, multi-cloud storage, and read-schema architectures, enabling real-time analytical performance across heterogeneous data sources.
Most studies reviewed in the context of data lakes report their validation using Yahoo! Streaming Benchmark, ESP Bench, and Table Union Benchmark, but, as with data warehouses, they also use widely tested benchmarks such as TPC-DS and TPC-H. In this context, performance metrics go beyond query time to incorporate method effectiveness, resource efficiency, and system scalability, consistent with the hybrid nature of lakehouse environments.
The area in which horizontal fragmentation methods were most developed was business within data warehouse-centric approaches; for platforms such as data lakes, business domain knowledge remains essential, but there has been notable growth in sectors such as entertainment, sports, and transportation. As for data lakehouses, other sectors such as energy and telecommunications are being studied. This conclusion was based on the type of benchmark or evaluation used to validate the approach. In addition, modern frameworks such as Apache Hive, Amazon S3, Dremio, and Digazu are replacing traditional DBMSs such as Oracle—evidence of a transition to distributed and cloud-native architectures. Finally, the most widely used technologies for implementing horizontal fragmentation methods were Hadoop, Java, and Spark for data warehouses, and Parquet, Spark, and Python for data lakes and data lakehouses. The results of this research provide an update on horizontal fragmentation approaches for analytical platforms and will serve as a starting point for researchers in computing, databases, and Big Data. The future work involves two steps: (1) provide guidelines to select the fragmentation method that best aligns system requirements and workload characteristics and (2) establish how fragmentation behaves according to its application in data warehouses, data lakes, and data lakehouses to improve performance, reduce costs, and improve scalability.
Author Contributions
Conceptualization, G.A.-H., N.R.-M. and L.R.-M.; data curation, N.R.-M.; formal analysis, N.R.-M.; funding acquisition, G.A.-H. and L.R.-M.; investigation, N.R.-M. and G.A.-H.; methodology, L.R.-M., G.A.-H. and N.R.-M.; project administration, L.R.-M., J.C. and N.R.-M.; resources, G.A.-H. and L.R.-M.; supervision, G.A.-H. and L.R.-M.; validation, L.R.-M. and F.C.-M.; visualization, N.R.-M.; writing—original draft, N.R.-M.; writing—review and editing, L.R.-M., J.C., F.C.-M., G.A.-H. and N.R.-M. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by Mexico’s Secretariat of Science, Humanities, Technology and Innovation (SECIHTI) for the scholarship awarded from 01 September 2026 to 31 August 2027 by participating in the call for POST-DOCTORAL STAYS FOR MEXICO MODE 1, application number 10618225, to develop the project titled “Advanced Analytical Models for the Design of Auxiliary Treatments for the Remission of Type 2 Diabetes Mellitus in High-Risk Populations Using Artificial Intelligence Techniques”.
Data Availability Statement
No new data were created or analyzed in this study.
Acknowledgments
This work was supported by Mexico’s National Technological Institute (TecNM) and sponsored by both Mexico’s Secretariat of Science, Humanities, Technology and Innovation (SECIHTI) and the Secretariat of Public Education (SEP) through the PRODEP project (Programa para el Desarrollo Profesional Docente).
Conflicts of Interest
The authors have no conflicts of interest to declare. They have seen and agreed with the contents of the manuscript, and there is no financial interest in reporting. We certify that the submission is original work and is not under review at any other publication.
References
- Furtado, P. Experimental Evidence on Partitioning in Parallel Data Warehouses. In Proceedings of the CIKM04: Conference on Information and Knowledge Management, Washington, DC, USA, 12–13 November 2004; pp. 23–30. [Google Scholar]
- Han, J.; Pei, J.; Tong, H. Data Mining: Concepts and Techniques, 4th ed.; Elsevier: Amsterdam, The Netherlands, 2022; pp. 1–752. [Google Scholar] [CrossRef] [Scilit]
- Maheshwari, A.K. Business Intelligence and Data Mining, 1st ed.; Ferguson, M., Ed.; Business Expert Press: New York, NY, USA, 2015. [Google Scholar]
- Jayabalan, D. Evolutionary Trends in Data Warehousing: Progress, Challenges and Future Directions. Int. J. Sci. Res. 2024, 13, 225–229. [Google Scholar] [CrossRef] [Scilit]
- Reis, J.; Housley, M. Fundamentals of Data Engineering Plan and Build Robust Data Systems; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2022; 446p. [Google Scholar]
- Kimball, R.; Ross, M.; Thornthwaite, W.; Mundy, J.; Becker, B. The Data Warehouse Lifecycle Toolkit, 2nd ed.; Wiley Publishing, Inc.: Indianapolis, IN, USA, 2008. [Google Scholar]
- Bellatreche, L.; Mohania, M.; Karlapalem, K.; Schneider, M. What Can Partitioning Do for Your Data Warehouses and Data Marts? In Proceedings of the International Database Engineering and Applications Symposium, Yokohama, Japan, 18–20 September 2000; pp. 18–20. [Google Scholar] [CrossRef] [Scilit]
- Noaman, A.Y.; Barker, K. A Horizontal Fragmentation Algorithm for the Fact Relation in a Distributed Data Warehouse. In Proceedings of the Eighth International Conference on Information and Knowledge Management, Kansas City, MO, USA, 2–6 November 1999; pp. 154–161. [Google Scholar] [CrossRef] [Scilit]
- Ozsu, M.T.; Valduriez, P. Principles of Distributed Database Systems, 4th ed.; Springer Nature: Cham, Switzerland, 2020. [Google Scholar]
- Bellatreche, L.; Woameno, K.J. Dimension table driven approach to referential partition relational data warehouses. In Proceedings of the ACM Twelfth International Workshop on Data Warehousing and OLAP, Hong Kong, China, 6 November 2009; pp. 9–16. [Google Scholar] [CrossRef] [Scilit]
- Silva, B.; Moreira, J. EasyBDI: Near Real-Time Data Analytics over Heterogeneous Data Sources. In Proceedings of the 24th International Conference on Extending Database Technology (EDBT), Nicosia, Cyprus, 23–26 March 2021; 4p. [Google Scholar] [CrossRef]
- Ait Errami, S.; Hajji, H.; Ait El Kadi, K.; Badir, H. Spatial Big Data Architecture: From Data Warehouses and Data Lakes to the LakeHouse. J. Parallel Distrib. Comput. 2023, 176, 70–79. [Google Scholar] [CrossRef] [Scilit]
- Harby, A.A.; Zulkernine, F. Data Lakehouse: A Survey and Experimental Study. Inf. Syst. 2025, 127, 102460. [Google Scholar] [CrossRef] [Scilit]
- Arksey, H.; O’Malley, L. Scoping Studies: Towards a Methodological Framework. Int. J. Soc. Res. Methodol. 2005, 8, 19–32. [Google Scholar] [CrossRef] [Scilit]
- Levac, D.; Colquhoun, H.; O’Brien, K.K. Scoping Studies: Advancing the Methodology. Implement. Sci. 2010, 5, 69. [Google Scholar] [CrossRef] [Scilit]
- Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Antes, G.; Atkins, D.; Barbour, V.; Barrowman, N.; Berlin, J.A.; Clark, J.; et al. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [Scilit]
- Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.J.; Horsley, T.; Weeks, L.; et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med. 2018, 169, 467–473. [Google Scholar] [CrossRef] [Scilit]
- Romero, O.; Herrero, V.; Abelló, A.; Ferrarons, J. Tuning Small Analytics on Big Data: Data Partitioning and Secondary Indexes in the Hadoop Ecosystem. Inf. Syst. 2015, 54, 336–356. [Google Scholar] [CrossRef] [Scilit]
- Khan, F.A.; Ahmad, A.; Imran, M.; Alharbi, M.; Mujeeb-ur-rehman; Jan, B. Efficient Data Access and Performance Improvement Model for Virtual Data Warehouse. Sustain. Cities Soc. 2017, 35, 232–240. [Google Scholar] [CrossRef] [Scilit]
- Boissier, M.; Kurzynski, D. Workload-Driven Horizontal Partitioning and Pruning for Large HTAP Systems. In Proceedings of the 2018 IEEE 34th International Conference on Data Engineering Workshops (ICDEW), Paris, France, 16–20 April 2018; pp. 116–121. [Google Scholar]
- Armenatzoglou, N.; Basu, S.; Bhanoori, N.; Cai, M.; Chainani, N.; Chinta, K.; Govindaraju, V.; Green, T.J.; Gupta, M.; Hillig, S.; et al. Amazon Redshift Re-Invented. In Proceedings of the ACM SIGMOD International Conference on Management of Data, Philadelphia, PA, USA, 12–17 June 2022; pp. 2205–2217. [Google Scholar] [CrossRef] [Scilit]
- Pedreira, P.; Croswhite, C.; Bona, L. Cubrick: Indexing Millions of Records per Second for Interactive Analytics. Proc. VLDB Endow. 2015, 9, 1305–1316. [Google Scholar] [CrossRef] [Scilit]
- Sebaa, A.; Chikh, F.; Nouicer, A.; Tari, A.K. Medical Big Data Warehouse: Architecture and System Design, a Case Study: Improving Healthcare Resources Distribution. J. Med. Syst. 2018, 42, 59. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Lyu, Z.; Zhang, H.H.; Xiong, G.; Guo, G.; Wang, H.; Chen, J.; Praveen, A.; Yang, Y.; Gao, X.; Wang, A.; et al. Greenplum: A Hybrid Database for Transactional and Analytical Workloads. In Proceedings of the ACM SIGMOD International Conference on Management of Data, Virtual Event, China, 20–25 June 2021; pp. 2530–2542. [Google Scholar] [CrossRef] [Scilit]
- Khalil, M.Y.; Hamad, M.M. Big Data Management Using Hadoop. J. Phys. Conf. Ser. 2021, 1804, 012109. [Google Scholar] [CrossRef] [Scilit]
- Potharaju, R.; Kim, T.; Wu, W.; Acharya, V.; Suh, S.; Fogarty, A.; Dave, A.; Ramanujam, S.; Talius, T.; Novik, L.; et al. Helios: Hyperscale Indexing for the Cloud & Edge. Proc. VLDB Endow. 2020, 13, 3231–3244. [Google Scholar] [CrossRef] [Scilit]
- Costa, E.; Costa, C.; Santos, M.Y. Evaluating Partitioning and Bucketing Strategies for Hive-Based Big Data Warehousing Systems. J. Big Data 2019, 6, 34. [Google Scholar] [CrossRef] [Scilit]
- Xu, Z.; Gao, Y.; Davidson, A. Keep Your Distributed Data Warehouse Consistent at a Minimal Cost. Proc. ACM Manag. Data 2023, 1, 1–25. [Google Scholar] [CrossRef] [Scilit]
- Bruno, N.; Galindo-Legaria, C.; Joshi, M.; Calvo Vargas, E.; Mahapatra, K.; Ravindran, S.; Chen, G.; Cervantes Juárez, E.; Sezgin, B. Unified Query Optimization in the Fabric Data Warehouse. In Proceedings of the ACM SIGMOD International Conference on Management of Data, Santiago, Chile, 9–15 June 2024; pp. 18–30. [Google Scholar] [CrossRef] [Scilit]
- Cusack, M.; Adamson, J.; Brinicombe, M.; Carson, N.; Kejser, T.; Peterson, J.; Vasudev, A.; Westerfeld, K.; Wipfel, R. Yellowbrick: An Elastic Data Warehouse on Kubernetes. In Proceedings of the 14th Annual Conference on Innovative Data Systems Research (CIDR ’24), Chaminade, HI, USA, 14–17 January 2024. [Google Scholar]
- Pan, S.; Zhang, Y.; Sikaria, A.; Zakharov, P.; Sharma, A.; Shankar, S.; Shuey, M.; Wareing, R.; Gangapuram, M.; Cao, G.; et al. Facebook’s Tectonic Filesystem: Efficiency from Exascale. In Proceedings of the 19th USENIX Conference on File and Storage Technologies (FAST 21); USENIX: Santa Clara, CA, USA, 2021; pp. 217–231. [Google Scholar]
- Arnold, J.; Glavic, B.; Raicu, I. A High-Performance Distributed Relational Database System for Scalable OLAP Processing. In Proceedings of the 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019, Rio de Janeiro, Brazil, 20–24 May 2019; pp. 738–748. [Google Scholar] [CrossRef] [Scilit]
- Cao, W.; Li, F.; Huang, G.; Lou, J.; Zhao, J.; He, D.; Sun, M.; Zhang, Y.; Wang, S.; Wu, X.; et al. PolarDB-X: An Elastic Distributed Relational Database for Cloud-Native Applications. In Proceedings of the IEEE 38th International Conference on Data Engineering, Kuala Lumpur, Malaysia, 9–12 May 2022; pp. 2859–2872. [Google Scholar] [CrossRef] [Scilit]
- Sivov, V.V.; Bogatyrev, V.A. Data Warehouse Failover Cluster for Analytical Queries in Banking. Adv. Eng. Res. 2023, 23, 76–84. [Google Scholar] [CrossRef] [Scilit]
- Zimmerer, A.; Dam, D.; Kossmann, J.; Waack, J.; Oukid, I.; Kipf, A. Pruning in Snowflake: Working Smarter, Not Harder. In Proceedings of the ACM SIGMOD International Conference on Management of Data, Berlin, Germany, 22–27 June 2025; pp. 757–770. [Google Scholar]
- Toumi, L.; Moussaoui, A.; Ugur, A. EMeD-Part: An Efficient Methodology for Horizontal Partitioning in Data Warehouses. In Proceedings of the International Conference on Intelligent Information Processing, Security and Advanced Communication, Batna, Algeria, 23–25 November 2015; pp. 1–7. [Google Scholar]
- Rodríguez-Mazahua, N.; Rodríguez-Mazahua, L.; López-Chau, A.; Alor-Hernández, G.; Machorro-Cano, I. Decision-Tree-Based Horizontal Fragmentation Method for Data Warehouses. Appl. Sci. 2022, 12, 10942. [Google Scholar] [CrossRef] [Scilit]
- Phan-Luong, V. Searching Data Cube for Submerging and Emerging Cuboids. In Proceedings of the 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), Taipei, Taiwan, 27–29 March 2017; pp. 586–593. [Google Scholar]
- Ramdane, Y.; Boussaid, O.; Kabachi, N.; Bentayeb, F. Partitioning and Bucketing Techniques to Speed up Query Processing in Spark-SQL. In Proceedings of the 2018 IEEE 24th International Conference on Parallel and Distributed Systems (ICPADS), Singapore, 11–13 December 2018; pp. 142–151. [Google Scholar]
- Khaleel, M.Y.; Hamad, M.M. Data Quality Management for Big Data Applications. In Proceedings of the 2019 12th International Conference on Developments in eSystems Engineering (DeSE), Kazan, Russia, 7–10 October 2019; pp. 357–362. [Google Scholar]
- Letrache, K.; El Beggar, O.; Ramdani, M. OLAP cube partitioning based on association rules method. Appl. Intell. 2019, 49, 420–434. [Google Scholar] [CrossRef] [Scilit]
- Ramdane, Y.; Kabachi, N.; Boussaid, O.; Bentayeb, F. SDWP: A New Data Placement Strategy for Distributed Big Data Warehouses in Hadoop. In Proceedings of the Big Data Analytics and Knowledge Discovery; Ordonez, C., Song, I.-Y., Anderst-Kotsis, G., Tjoa, A.M., Khalil, I., Eds.; Springer International Publishing: Linz, Austria, 2019; pp. 189–205. [Google Scholar]
- Hilprecht, B.; Binnig, C.; Röhm, U. Towards Learning a Partitioning Advisor with Deep Reinforcement Learning. In Proceedings of the Second International Workshop on Exploiting Artificial Intelligence Techniques for Data Management, Amsterdam, The Netherlands, 5 July 2019; pp. 1–4. [Google Scholar]
- Hilprecht, B.; Binnig, C.; Röhm, U. Learning a Partitioning Advisor for Cloud Databases. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, Portland, OR, USA, 14–19 June 2020; pp. 143–157. [Google Scholar]
- Benkrid, S.; Bellatreche, L. A Framework for Designing Autonomous Parallel Data Warehouses. In Proceedings of the Algorithms and Architectures for Parallel Processing, International Conference on Algorithms and Architectures for Parallel Processing; Springer International Publishing: Cham, Switzerland, 2020; Volume 11945, pp. 97–104. [Google Scholar]
- Hamdi, W.; Faiz, S. Distributing Data in Real Time Spatial Data Warehouse. In Proceedings of the Algorithms and Architectures for Parallel Processing; Qiu, M., Ed.; Springer International Publishing: Berlin/Heidelberg, Germany, 2020; pp. 3–13. [Google Scholar]
- Benkrid, S.; Mestoui, Y.; Bellatreche, L.; Ordonez, C. A Genetic Optimization Physical Planner for Big Data Warehouses. In Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 10–13 December 2020; pp. 406–412. [Google Scholar]
- Ramdane, Y.; Kabachi, N.; Boussaid, O.; Bentayeb, F. A Data Mining Approach to Guide the Physical Design of Distributed Big Data Warehouses. In Advances in Knowledge Discovery and Management: Volume 9; Jaziri, R., Martin, A., Rousset, M.-C., Boudjeloud-Assala, L., Guillet, F., Eds.; Studies in Computational Intelligence; Springer International Publishing: Cham, Switzerland, 2022; pp. 107–125. ISBN 978-3-030-90287-2. [Google Scholar]
- Boukorca, A.; Bellatreche, L.; Benkrid, S. HYPAD: Hyper-Graph-Driven Approach for Parallel Data Warehouse Design. In Algorithms and Architectures for Parallel Processing. ICA3PP 2015: Volume 9531; Wang, G., Zomaya, A., Martinez, G., Li, K., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Stwitzerland, 2015; pp. 770–783. [Google Scholar] [CrossRef] [Scilit]
- Mateus, R.C.; Siqueira, T.L.L.; Times, V.C.; Ciferri, R.R.; de Aguiar Ciferri, C.D. Spatial Data Warehouses and Spatial OLAP Come towards the Cloud: Design and Performance. Distrib. Parallel Databases 2016, 34, 425–461. [Google Scholar] [CrossRef] [Scilit]
- Kechar, M.; Nait-Bahloul, S. Performance optimisation of the decision-support queries by the horizontal fragmentation of the data warehouse. Int. J. Bus. Inf. Syst. 2017, 26, 506. [Google Scholar] [CrossRef] [Scilit]
- Ferreira, L.; Alves-Souza, S.; Maria Da Silva, L. Startable: Multidimensional Modelling for Column-Oriented NoSQL. In Proceedings of the 11th International Conference on Data Science, Technology and Applications, Lisbon, Portugal, 11–13 July 2022; pp. 21–30. [Google Scholar]
- Barkhordari, M.; Niamanesh, M. Chabok: A Map-Reduce based method to solve data warehouse problems. J. Big Data 2018, 5, 40. [Google Scholar] [CrossRef] [Scilit]
- Barr, M.; Boukhalfa, K.; Bouibede, K. Bi-Objective Optimization Method for Horizontal Fragmentation Problem in Relational Data Warehouses as a Linear Programming Problem. Appl. Artif. Intell. 2018, 32, 907–923. [Google Scholar] [CrossRef] [Scilit]
- Nam, Y.-M.; Kim, M.-S.; Han, D. A Graph-Based Database Partitioning Method for Parallel OLAP Query Processing. In Proceedings of the 2018 IEEE 34th International Conference on Data Engineering (ICDE), Paris, France, 16–19 April 2018; pp. 1025–1036. [Google Scholar]
- Marcus, R.; Papaemmanouil, O.; Semenova, S.; Garber, S. NashDB: An End-to-End Economic Method for Elastic Database Fragmentation, Replication, and Provisioning. In Proceedings of the 2018 International Conference on Management of Data, Houston, TX, USA, 10–15 June 2018; pp. 1253–1267. [Google Scholar]
- Marcus, R.; Zhang, C.; Yu, S.; Kao, G.; Papaemmanouil, O. NashDB: Fragmentation, Replication, and Provisioning Using Economic Methods. Proc. VLDB Endow. 2019, 12, 1830–1833. [Google Scholar] [CrossRef] [Scilit]
- Kechar, M.; Nait-Bahloul, S. Bringing Together Physical Design and Fast Querying of Large Data Warehouses: A New Data Partitioning Strategy. In Proceedings of the 4th International Conference on Big Data and Internet of Things, Rabat, Morocco, 23–24 October 2019; pp. 1–8. [Google Scholar]
- Nam, Y.-M.; Han, D.; Kim, M.-S. A parallel query processing system based on graph-based database partitioning. Inf. Sci. 2019, 480, 237–260. [Google Scholar] [CrossRef] [Scilit]
- Parchas, P.; Naamad, Y.; Van Bouwel, P.; Faloutsos, C.; Petropoulos, M. Fast and Effective Distribution-Key Recommendation for Amazon Redshift. In Proceedings of the VLDB Endowment; Balazinska Magdalena, X.Z., Ed.; ACM: New York, NY, USA, 2020. [Google Scholar]
- Tatemura, J.; Zou, T.; Sankaranarayanan, J.; Huang, Y.; Chen, J.; Zhang, Y.; Lai, K.; Zhang, H.; Manoharan, G.N.B.; Graefe, G.; et al. Progressive Partitioning for Parallelized Query Execution in Google’s Napa. Proc. VLDB Endow. 2023, 16, 3475–3487. [Google Scholar] [CrossRef] [Scilit]
- Benkrid, S.; Bellatreche, L.; Mestoui, Y.; Ordonez, C. PROADAPT: Proactive Framework for Adaptive Partitioning for Big Data Warehouses. Data Knowl. Eng. 2022, 142, 102102. [Google Scholar] [CrossRef] [Scilit]
- Khatiwada, A.; Shraga, R.; Gatterbauer, W.; Miller, R.J. Integrating Data Lake Tables. Proc. VLDB Endow. 2022, 16, 932–945. [Google Scholar] [CrossRef] [Scilit]
- Naeem, Z.A.; Ahmad, M.S.; Eltabakh, M.; Ouzzani, M.; Tang, N. RetClean: Retrieval-Based Data Cleaning Using LLMs and Data Lakes. Proc. VLDB Endow. 2024, 17, 4421–4424. [Google Scholar] [CrossRef] [Scilit]
- Schmitz, D.; Berrewaerts, L.; Rosinosky, G.; Skhiri, S.; Rivière, E. A Tale of Many Streams: Characterizing a Hybrid Batch-Stream Production Workload in Digazu, a Data Lake Supported by Apache Kafka and Flink. In Proceedings of the 19th ACM International Conference on Distributed and Event-Based Systems (DEBS ’25), Gothenburg, Sweden, 10–13 June 2025; pp. 188–198. [Google Scholar] [CrossRef] [Scilit]
- Shah, R.; Mukherjee, K.; Tyagi, A.; Karnam, S.K.; Joshi, D.; Bhosale, S.P.; Mitra, S. R2D2: Reducing Redundancy and Duplication in Data Lakes. Proc. ACM Manag. Data 2023, 1, 1–25. [Google Scholar] [CrossRef] [Scilit]
- Prammer, M.; Zeng, X.; Meng, R.; Mckinney, W.; Zhang, H.; Pavlo, A.; Patel, J.M. Towards Functional Decomposition of Storage Formats. In Proceedings of the 15th Annual Conference on Innovative Data Systems Research (CIDR ’25), Amsterdam, The Netherlands, 19–22 January 2025. [Google Scholar]
- Weintraub, G.; Gudes, E.; Dolev, S. Needle in a Haystack Queries in Cloud Data Lakes. In Proceedings of the Workshop Proceedings of the EDBT/ICDT 2021, Nicosia, Cyprus, 23–26 March 2021; pp. 1–8. [Google Scholar]
- Mukherjee, K.; Shah, R.; Saini, S.K.; Singh, K.; Khushi; Kesarwani, H.; Barnwal, K.; Chauhan, A. Towards Optimizing Storage Costs on the Cloud. In Proceedings of the 2023 IEEE 39th International Conference on Data Engineering (ICDE), Anaheim, CA, USA, 3–7 April 2023; pp. 2919–2932. [Google Scholar]
- Bakken, M.; Soylu, A. Chrontext: Portable SPARQL Queries over Contextualised Time Series Data in Industrial Settings. Expert Syst. Appl. 2023, 226, 120149. [Google Scholar] [CrossRef] [Scilit]
- Chattopadhyay, B.; Pedreira, P.; Agarwal, S.; Vakharia, S.; Li, P.; Liu, W.; Narayanan, S. Shared Foundations: Modernizing Meta’s Data Lakehouse. In Proceedings of the 12th Annual Conference on Innovative Data Systems Research (CIDR ’23), Amsterdam, The Netherlands, 8–11 January 2023. [Google Scholar]
- Levandoski, J.; Casto, G.; Deng, M.; Desai, R.; Edara, P.; Hottelier, T.; Hormati, A.; Johnson, A.; Johnson, J.; Kurzyniec, D.; et al. BigLake: BigQuery’s Evolution toward a Multi-Cloud Lakehouse. In Proceedings of the ACM SIGMOD International Conference on Management of Data, Santiago, Chile, 9–15 June 2024; pp. 334–346. [Google Scholar]
- Merli, M.; Guo, S.; Li, P.; Chen, H.; Lu, N. Ursa: A Lakehouse-Native Data Streaming Engine for Kafka. Proc. VLDB Endow. 2025, 18, 5184–5196. [Google Scholar] [CrossRef] [Scilit]
- Ritter, D.; Andrei, M.; Cho, S.; Görgens, M.; Lee, T.; May, N.; Pathak, A.; Willems, P.R. The HANA Native Query Engine for Lakehouse Systems. Proc. VLDB Endow. 2025, 18, 4831–4845. [Google Scholar] [CrossRef] [Scilit]
- Vitillo, R. Understanding Distributed Systems, 2nd ed.; Roberto Vitillo: Monee, IL, USA, 2022; ISBN 1838430210. [Google Scholar]
- Kleppmann, M. Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems; O’Reilly: Springfield, MO, USA, 2017; ISBN 9781491903063. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.











