A Survey on Big Data Technologies and Their Applications to the Metaverse: Past, Current and Future

Abstract

The development of big data technologies, which have been applied extensively in various areas, has become one of the key factors affecting modern society, especially in the virtual reality environment. This paper provides a comprehensive survey of the recent developments in big data technologies, and their applications to virtual reality worlds, such as the Metaverse, virtual humans, and digital twins. The purpose of this survey was to explore several cutting-edge big data and virtual human modelling technologies, and to raise the issue of future trends in big data technologies and the Metaverse. This survey investigated the applications of big data technologies in several key areas—including e-health, transportation, and business and finance—and the main technologies adopted in the fast-growing virtual world sector, i.e., the Metaverse.

1. Introduction

In modern society, digital applications have been extensively deployed in numerous areas. These applications can generate enormously large data, which provide abundant resources for data analytics, prediction, and decision making. The rapid growth of heterogeneous datasets demands new big data technologies for more efficient data processing. Previous research studies have provided various reviews and surveys on big data technologies. This paper reviews recent developments in big data technologies, and their applications to a virtual reality world, e.g., the Metaverse.

In the big data environment, large amounts of datasets are created, and spread rapidly, which incurs extra computational overheads [1]. Conventional data processing technologies face the challenges of data explosion, increasing data variety, limited real-time data processing efficiency and growing demands for more accurate analytical methods. The growth of data complexity and volume is hindering the application of big data analytics in the real world. In particular, e-business companies have relied excessively on online data collection and analytics: consequently, these e-business organizations seek to promote business and increase profits; nevertheless, of the e-business companies that have applied big data technologies to their business processes, only 37 percent have proven to be successful [2]. Information overload in the big data environment has increased the complexity and difficulty of the e-business decision-making process [1,2,3]. There is a crucial need to understand the development of big data technologies, in order to deploy more efficient new technologies to various applications.

In recent years, virtual reality applications have attracted much attention. The Metaverse is one of the most up-to-date virtual reality concepts: it is a shared three-dimensional virtual platform, which creates a mirror image of the real world based on digital twin and related technologies [4]. The use of big data technologies will unavoidably become an important issue in various virtual reality worlds, including the Metaverse. One of the main research objectives of this study was to investigate the cutting-edge big data technologies used in the Metaverse, in order to assist the development of virtual reality applications. This paper firstly explores the respective applications of big data and the Metaverse, and then connects them, in Section 4.

This paper is organized as follows: the next section provides a comprehensive survey of big data applications in different areas; Section 2 investigates the current trends in big data technologies; Section 3 reviews the development of virtual reality platforms, e.g., the Metaverse and its correlation with the big data technologies; Section 4 discusses the future application of big data technologies and the Metaverse, and their roles, and it explains the literature review methods. The last section concludes the research findings.

2. Applications of Big Data Technologies in Different Areas

‘Big data’ is defined as a data source that has the characteristics of large volume, high velocity, a wide range of variety, and veracity [5]. This definition is known as ‘the 4Vs of big data’. In many cases, big data technology implies analytics, storage, and rapid processing in a dynamic environment. In [5], a review on medical big data applications was conducted systematically. The review summarized various applications that had been developed, based on big data technologies. This section provides a systematic review of the applications of big data technologies in several key areas.

2.1. Big Data Technologies in the e-Health Sector

The four characteristics of big data, i.e., the 4Vs, have enormous impact on medical data processing systems. Various e-health systems have been developed to tackle the problems occurring in the complex and dynamic big data environment.

Statistics indicate that the current worldwide data volume is set to more than double each year. The global healthcare data storage market is predicted to grow from USD 3.08 billion in 2020 to USD 6.12 billion by 2027 [6]. Medical data systems are facing challenges presented by the rapid growth of data volume due to the increasing use of magnetic resonance imaging (MRI), computed tomography (CT) scans, and rising patient numbers. The development of an efficient large data processing system has become a critical requirement for e-health systems.

In recent years, big-data-driven platforms for personalized healthcare have been developed, to reduce readmission rates and accelerate real-time response [7]. The extensive applications of the Clinical Data Warehouse (CDW) database have incorporated online analytical processing and sophisticated network analysis, to discover new clinical findings. In [8], a big-data-driven system was developed, to reveal and analyze hospital information system user behaviors, and potential features based on behavior analysis.

The system incorporated the extract, transform and load (ETL) module, which can process data and store it in the big data warehouse, in parallel. Like conventional large data systems, clinical systems have incorporated structure data, semi-structure data and unstructured data into their data warehouses. Most medical health systems now have the capacity to store metadata, protected health information and heterogeneous e-health data. Existing medical health systems offer cloud-based and hybrid storage solutions. Figure 1 shows the architecture of a typical health data warehouse: the medical data have structured and unstructured sources, which include patients’ medical records, health monitoring signals, genome data, MRI and CT images, medical analytical reports, and laboratory testing results. It is crucial to efficiently integrate the various medical data sources in the dynamic big data environment. In [9], a patient-centric healthcare application, Health-CPS, was developed for the unification of various medical data sources. The Health-CPS system deploys data-oriented service and data collection layers with a united standard, which can efficiently integrate various medical data sources: this system demonstrates that the implementation of cloud and big data technologies in healthcare systems can enhance their system performance [5,9,10].

From the velocity perspective, the extensive applications of wearable and sensory health monitoring devices demand a swift response, to fulfill the requirements of real-time medical data processing. Under the circumstances, the velocity of medical big data is vital to many life-saving health systems. In [11], a wearable medical emergency response system was implemented, based on a large number of medical sensors. In this system, patients’ clinical data were obtained and analyzed, based on numerous wearable sensors for real-time clinical monitoring, diagnosis and treatment therapies [12].

The rapid development of wearable and sensor technologies has accelerated the applications of real-time multi-sensor wearable healthcare devices in the healthcare sector. In particular, wearable medical devices have simplified the processes of health monitoring, as they can efficiently monitor, analyze and diagnose patients’ daily health conditions. In [13], a flow acceleration measurement model was developed, to assess users’ posture and movement during long-term measurements in daily life: this model has been applied extensively in rehabilitation, psychophysiology and cardiology areas [5,13]. Multi-sensor medical models have focused, in recent decades, on monitoring chronic diseases. The sensor network for assessment of patients (SNAP) was developed for real-time response to patients with chronic diseases [14]. Figure 2 shows the framework of the medical SNAP model. The velocity issue is crucial in these multi-sensor wearable medical devices; therefore, more sophisticated real-time medical systems and devices have been developed, to fulfill the velocity requirement in the big data environment [15,16,17,18,19].

Veracity is one of the most important characteristics, because in many cases medical data sources can be incomplete, biased and inaccurate; therefore, it is crucial to ensure data quality, in order to minimize bias, duplication, abnormalities, inconsistencies and volatility during big data analytical processes [5,21]. Research studies have shown that one of the key factors in a successful healthcare system is the implementation of efficient algorithms and data analysis technologies, to handle large amounts of heterogeneous data, so as to produce clinical results with sufficient veracity [22,23,24].

Previous research work has paid much attention to improving the accuracy, reliability and efficiency of medical data processing systems. Several methods have been adopted for improving the veracity of medical data sources, including data cleaning, data normalization and data fusion [25]. A new data cleaning approach has been applied for replacement of missing text, and to improve the number of relevant cases retrieved by search queries in clinical systems [26]. Some conventional methods have been used for medical data cleaning, which include [26]:

(1)

Replacing missing categories, and standardizing contents in clinical reports;

(2)

Abbreviations substitution, through medical dictionaries and ontologies;

(3)

Filtering and eliminating data noise, errors and inconsistency, by using Natural Language Processing (NLP) methods.

In brief, veracity is undoubtedly important in medical and healthcare systems, and the development of big data technologies is improving veracity in healthcare systems.

2.2. Big Data Technologies in the Transportation Sector

Fast-growing transport systems in modern society are generating a large amount of data, including transportation trajectory data, GPS data, traffic management data, and transportation network data. Due to the increasing demands for efficient transportation systems in the big data era, various intelligent transportation systems (ITS) have been developed, to accommodate the requirements of processing and analyzing the growing amount of transport data [27,28,29,30]. Like the medical sector, transportation systems are facing the challenges of volume, velocity, variety and veracity in the dynamic big data environment. ITS applications are focusing on tackling big data problems by adopting advanced big data technologies. Research studies have shown that big data analytics can improve ITS data processing capacity, operation efficiency, and safety levels [29]. Figure 3 shows the architecture of a big data analytics framework in ITS.

The major big data analytics technologies are applied in ITS applications as follows:

• Supervised learning methods: the major data analytic and machine learning methods used in ITS include regression, decision tree, Artificial Neural Network (ANN) and Support Vector Machine (SVM) [29] (Figure 4). Linear regression is one of the most efficient methods for classification, and it has been applied extensively in ITS, for traffic route analysis and traffic flow prediction [31,32,33]. The decision tree method has been applied to ITS applications, such as traffic accident detection, traffic congestion prediction and accident severity prediction [34,35,36]. In [29], the SVM classifier with the kernel function K (x, x’) could derive the support vector α_i:

$$\underset{a_i}{\max }-\frac{1}{2}{{\displaystyle \sum }}_{i=1}^l{y}_i{y}_j{\alpha }_i{\alpha }_{j}K\left(x_i, x_j\right)+{{\displaystyle \sum }}_{i=1}^l{a}_i$$

$$s.t. {{\displaystyle \sum }}_{i=1}^l{y}_i{a}_i=0$$

(1)

where the decision function g(x) to compute the label for the sample x was:

$$g\left(x\right)=sgn\left({{\displaystyle \sum }}_{i=1}^l{y}_i{a}_i\ast K\left(x_i,x\right)+b\right)$$

(2)

If x is an incident sample, then g(x) = 1; otherwise, g(x) = −1 [29].

• Unsupervised learning and ontology-based methods: the conventional unsupervised learning method adopted in ITS is K-means, which has been applied to travel time prediction, travel path planning, etc., [29,37,38]. Ontology-based methods deploy data semantics that can efficiently associate data semantic relations, which are extensively applied in the ITS field for semantic traffic data processing [39,40,41].
• Deep learning and reinforced learning methods: the application of reinforced learning in ITS is to reduce the computational overhead through exploring and learning the optimal policy, based on ITS data [42]. Reinforcement learning is feasible in traffic signal control in ITS, as it incorporates supervised and unsupervised methods [43,44]. The Q-learning in reinforced learning modeling is the value iteration update, which is listed as follows:

$$Q\left(s_1,a_t\right)=Q\left(s_1,a_t\right)+\alpha \left(r_{t+1}+\gamma \underset{A}{\max }Q\left(S_{t+1}, a_t\right)-Q\left(S_t, a_t\right)\right)$$

(3)

The abovementioned data analytical and machine learning methods have been used widely in the ITS area, with the support of big data platforms, such as Hadoop, Hbase, Spark, etc., [29,45].

Recently, several ITS domain-oriented big data platforms have been developed, to accommodate the rising demands for more effective data processing in ITS. A big data platform was developed in [46], with multiple engines to support heterogeneous traffic data analytics. Several other data processing platforms and frameworks have been applied to ITS, such as Godzilla [47], K-Feed [48], Sipresk [49] and the ITS big data simulation platforms [50].

In brief, big data technologies can benefit ITS applications, by enabling the highly efficient data processing and analytical capacities that contribute to the solutions of modern ITS applications, including traffic management, transportation infrastructure, transportation logistics and smart connectivity [29,51].

2.3. Big Data Technologies in the Business and Financial Sectors

Our society is entering the digital era, because of the rapid development of digital technologies. Online activities have played an important role in in our daily lives, which continuously generate large amounts of data. These data sources are generated in various sectors, including healthcare, transportation, manufacturing, finance, business, and social activities. The use of big data technologies has accelerated digital transformation processes in all sectors, enabling adaptation to market changes and the rise of the digital economy. Most modern businesses are unable to survive without the support of digital platforms [52].

The extensive application of data analytics in the commerce and finance sectors has improved efficiency in financial decision-making processes. Financial and commercial applications deploy big data technologies to perform various financial and commercial analyses, which eventually generate more accurate and comprehensive decision-making models [53,54,55]; however, digital applications have increased the financial risk of cyberattack and online fraud, as personal data are rapidly becoming the new currency in the digital economy [56,57]. Technological innovation is one of the most important driving forces in financial markets, for pursuing high profits—especially digital technologies, which are applied extensively in the financial sector, and have great impact on various financial applications [54,58,59,60,61].

Modern financial systems collect accessible transparent data, to improve financial data analysis and risk control, particularly with regard to the applications of big data technologies. Big data technologies can improve the performance of both enterprise and individual-level financial systems, as they are based on large data analytics. The application of cloud computing technology in financial organizations improves data security, and reduces the costs of big data management and analysis, which facilitates the deployment of big data technologies in the business and financial sectors: for instance, Amazon Web Services (AWS) provides a scalable, cost-effective cloud platform for various businesses, including financial and commercial analytics, around the world. Figure 5 shows the big data analytics in financial services.

Big data technologies play a vital role in numerous online commercial activities in almost all sectors: for instance, Netflix applies big data technologies to analyzing customers’ viewing behaviors, so as to provide customers with more accurate recommendations; mobile and car-based GPS applications feature location-based services such as Google Maps, which heavily rely on big data technologies for rapid customer service response [56,63,64]. Business companies utilize cloud computing to facilitate applications of big data technologies, in order to improve customer satisfaction [65]. In recent years, conventional Business-to-Consumer (B2C) companies and organizations have extended their operations to the Business-to-Business (B2B) marketing sector [63]. Consequently, more sophisticated big data technologies have been deployed in large commercial data analytics.

Fast growth, in online businesses, financial and social activities, leads to accumulation of enormously large amounts of data, which facilitates data-driven business innovation, through the deployment of proper big data technologies [66]. Information technology-based firms, such as Amazon, Google, eBay, TikTok, Twitter, etc., constantly store and analyze customer service data, including customer and product details, transaction times and other service-related data. These data sources provide companies with the basis for effective decision making and efficient strategic business plans, to enhance their customer service performance and business innovation [54,67,68]. Research studies have shown that big data technologies are becoming essential for business projects with high technological novelty or radical innovation, especially in a big data environment, as many researchers believe that customers are sources of information and knowledge [69,70]. In [70], four potential key success factors were identified for organizations to integrate big data technologies, so as to accelerate their product innovation processes [71].

A comprehensive investigation has been conducted on the big data technologies that have been applied in the business and financial sectors, including the major methods, technologies and models used for business/data analytics, as shown in

Table 1. Big data technologies applied in the business and financial sectors [72,73,74,75,76,77,78,79].

Big Data Technologies	Business/Financial Activities	Industrial Applications/Platforms
Advanced message queuing protocol (AMQP), XMPP, Extract, Transform, Load (ETL)—NoSQL, etc.	Business data acquisition, data cleaning, data pre-processing	Seon’s fraud data detection (Weld ETL tool) Lenovo’s data cleansing (Talend Data Fabric), etc.
Hadoop, Hive, Hydra, Pig, Spark, Mapreduce, Storm, Segmentation (NAD, Bootstrapping), etc.	Data storage, data management, data infrastructure, data migration	eBay (Apache Spark, Storm, Kafka) Netflix’s media data (AWS s3 & EC2) Uber (Schemaless, Hadoop), etc.
Collaborative filtering (recommender), linear regression, K-means clustering, apriori association rule, C4.5 (Decision Tree), SVM, etc.	Business analytics, sale prediction, market prediction, financial investment trends analysis	Walmart (marketing, promotion, personalized customer service) Uber (Horovod-distributed deep learning framework), etc.
Attribute-based encryption, 3KDEC, storage path encryption, differential privacy, fast anonymization of big data streams, top-down specialization, etc.	Business data privacy, data security, data recovery, big data encryption	Netflix (released personally identifiable information, removed movie ratings) IBM-developed ethical framework (for legitimate big data collection), etc.

. A number of major big data technologies are listed in Table 1, in accordance with their relevant business and financial activities, as well as the real-life industrial applications or platforms where big data technologies are implemented. Our study discovered that big data technologies have been applied in almost all major activities in the business and financial sectors, as shown in Table 1 [72,73,74,75,76,77,78,79].

3. Trends in Big Data Technologies

Big data technologies are now critical to the success of organizations and companies. Numerous big data technologies have been developed and applied to almost every aspect of our daily lives. Online shopping companies, such as Amazon, eBay, Alibaba, Walmart, etc., utilize advanced data mining methods, to provide customers with personalized shopping services and recommendations. Logistics and transportation companies, such as Fedex, DHL, UPS, CSX Transportation, etc., have deployed sensor-based big data technologies and machine-learning methods to improve the efficiency of “last mile delivery”.

Big data is a technology with a very broad landscape [80]: it covers various domains and fields, including infrastructure, analytics, applications, data resources, data sources, APIs and open sources. According to the big data technologies statistics in [80], the most influential big data technologies have been adopted in the health sector, particularly in cancer classification, in terms of academic citations [80]. With regard to industrial applications, online shopping and e-commerce are the domains that rely heavily on big data technologies.

In this paper, big data technologies are grouped into four categories: big data acquisition and pre-processing; big data storage and infrastructure; big data analytics; and big data privacy and security.

Table 2. An overview of current major big data technologies.

Categories	Current Big Data Technologies	Future Trends
Big data acquisition and pre-processing	Advanced message queuing protocol (AMQP)—acquisition protocol [75,81]; Extensible messaging and presence protocol (XMPP)—acquisition protocol [82]; Java Message Service (JMS)—acquisition protocol [75]; Extract, Transform, Load (ETL/ELT)—data integration [75]; Crowdsourcing—data curation [75]; NoSQL—data structure [75,83]; Kafka, Flume—platforms [75]; Crawling—acquisition tool [75,77];	Future big data acquisition and pre-processing technologies should be able to deal efficiently with more unstructured, high dimensional data; several techniques are suggested below: Blockchain—acquisition protocol, integration and curation [84]; Data fabric—data acquisition and integration [85]; Natural Language Processing (NLP) pipelines—data structure and integration [86].
Big data storage and data infrastructure	Distributed file systems: HDFS, GFS; Data query analytical tool: Hive QL; Pig Latin; JAQL, etc., [87]. Storage reduction and optimization: Mapreduce (Hadoop); Spark; Hydra; Storm, etc., [88]; Data segmentation: (NAD [74], bootstrapping [89]), etc.;	Future trends in big data storage methods will focus on more elastic and cloud-based solutions: Fog-to-Hybrid and Multicloud [90]; Data as a Service Model [91]; Blockchain data storage [92]; Non-Volatile Memory Express (NVMe) [93] and tensor networks; Covariance/matrix-based high-dimensional data segmentation and tensor networks [94].
Big data analytics	Recommendation: collaborative filtering; apriori association rule and FP growth tree, etc., [95,96]. Unsupervised: K-means clustering; DBSCAN clustering; OPTICS (Ordering points to identify clustering structure); SOM (self-organizing map), etc., [72,97,98]; Supervised: linear regression; C4.5 (Decision Tree); SVM; CNN; LSTM, etc., [72,99].	The trends in big data analytics are in the following areas: Natural language processing and sentiment analysis (unstructured data or semi-structured data) [100]; High-performance, real-time analysis models and virtual reality/Metaverse analytics models [101]; Data-centric AI analytics and augmented data analytics [102].
Big Data privacy and security	Data encryption: attribute-based encryption; public key encryption; storage path encryption [78,103]; Data privacy: privacy-preserving machine learning algorithms [104]; differential privacy [78,105]. Big data storage privacy and security: Apache Rhino; Sentry; Ranger [106].	The trends in big data privacy and security will mainly focus on cloud- and blockchain-related areas: Cloud Security—combination of centralized cloud services with fog computing [107]; Blockchain secure data privacy and security [90,107].

provides an overview of current major big data technologies and their trends. Future big data technologies will focus on providing advance solutions, so as to ensure that they can handle more diverse data sources, process heterogeneous data storage, provide real-time analytical solutions and protect the security and privacy of user data more efficiently.

The tensor network is regarded as an efficient solution for future trends in big data storage [108]. The Tensor Train (TT) decomposition, described in [109], can be executed in a simple non-recursive form. A dth-order tensor is defined to be the TT-format, if it satisfies the following format [108]:

$$A\left(i_1,i_2,\cdots ,i_d\right)=A_1\left(i_1\right)A_2\left(i_2\right)\cdots A_d\left(i_d\right)$$

(4)

where $A_{\left(k\right)}\left(i_k\right)\in R^{r_{k-1}\times r_k},r_0=r_d=1, A_{\left(k\right)}\left(i_k\right)$ is referred to as the core tensor. The TT format is based on a low-rank approximation, through singular value decomposition of auxiliary unfolding matrices [108].

Efficient data segmentation methods have been suggested in various studies [74,93]. A recent study showed that future trends in big data storage are focusing on converting very large datasets to smaller segments without information losses [74]. The Normal Distribution Approximation (NDA) method is adopted for fast large data segmentation, which can be split with minimum data information loss; however, the NDA or the Poisson Distribution Approximation (PDA) methods can only process a one-dimensional data source. Future trends will focus on high-dimensional datasets that take covariance into consideration [94], defining an f ([x₁, x₂…x_i]) function, to judge how close the mean of the subdataset is to the mean of the original dataset. The f function’s expression is:

$$\left(\left[x_1, x_2\dots x_i\right]\right)={\left(X_1-x_1\right)}^2+{\left(X_2-x_2\right)}^2+\dots +{\left(X_i-x_i\right)}^2$$

(5)

where i is the size of the dataset’s dimension, [x₁, x₂…x_i] is the subdataset’s mean, and [X₁, X₂…X_i] is the original dataset’s mean. This process involves the calculation of the covariance matrix Σ_A (n × n) of the original dataset A and the covariance matrix Σ_B (n × n) of the data subset B. A positive definite matrix Σ can be decomposed into Σ = U^TΛ U, where U is the upper triangular matrix, and Λ is the diagonal matrix, in which the diagonal elements are non-negative [94]:

$${\displaystyle \sum }=U^T\Lambda U=\left[U^T{\Lambda }^{\frac{1}{2}}\right]\left[{\Lambda }^{\frac{1}{2}}U\right]={\left[{\Lambda }^{\frac{1}{2}}U\right]}^T\left[{\Lambda }^{\frac{1}{2}}U\right]$$

(6)

and, therefore, the matrix Σ = C^TC, where C = Λ^1/2U [94].

The optimized bootstrap algorithms could affect the future trends for big data storage [89]. The bootstrap weights algorithm can be applied to estimate the variance of smooth and non-smooth parameters [89]. The bootstrap weight is defined as:

$$w_{\mathit{ik}}^{\ast }=\left\{1+{\left(\frac{n^{\prime }}{n-1}\right)}^{1/2}\left(\frac{{\mathit{nn}}_i^{\ast }}{n^{\prime }}-1\right)\right\}{\pi }_i^{-1}{\pi }_{k|i}^{-1}$$

(7)

where n is the sample size of the primary sample units (psu), and $n_i^{*}$ denotes the number of times the ith psu is selected in the bootstrap sample; $\widehat{{\theta }^{*}}$ is then computed, using the formulae that were used to obtain the original point estimator, with the original weights replaced by the bootstrap weight, $w_{ik}^{*}$ [94]. The bootstrap variance estimator, $va{r}^{*}(\widehat{{\theta }^{*}})$, can be applied when all $\widehat{{\theta }^{*}}$ are obtained. The Monte Carlo approximation of $va{r}^{*}(\widehat{{\theta }^{*}})$ is adapted:

$$\widehat{{\mathit{var}}^{*}}=\frac{1}{B-1}{{\displaystyle \sum }}_{b-1}^B\left(\widehat{{\theta }_b^{*}}-\widehat{\overline{{\theta }^{*}}}\right)$$

(8)

In the future big data environment, unstructured data sources will grow exponentially, and the trends in efficient and real-time data storage solutions will be crucial. The fast-growing Virtual Reality (VR), Augmented Reality (AR), Extended Reality (XR) and Metaverse applications are becoming an increasingly significant issue for big data models. Research studies indicate that the VR and Metaverse will replace the current web-based online businesses, and become the major areas of generating large data volume in the near future [101,110,111,112,113]. The following section investigates the current development of the Metaverse and the digital human.

4. Metaverse-Related Technologies and Applications

The definition of the Metaverse in [114] is a virtual space where users can interact with one other, and with their environment, via 3D digital objects and virtual avatars, in a complex manner that mimics the real world, holding things developed using artificial intelligence techniques; therefore, creating digital humans is essential to the development of the Metaverse and other AR/VR/XR applications.

4.1. Digital Human Reconstruction

How to create digital humans has been a much-studied subject recently, due to the rising demand for virtual reality applications, including the Metaverse. One of the core drivers of mathematical progress is the discovery of objects, patterns and ultimately their formulaic representations; in the course of such progress, scientists often need to leverage a variety of tools and data to help them cultivate ideas, propose a conjecture, and eventually prove/disprove with experiments and evidence, where possible. There is no doubt that the evolution of computational methodology has not only changed the way scientists conduct their studies, but has also accelerated the life cycle of scientific research, leading to profound impacts on people’s daily lives—including, for example, the early hand-calculated prime number tables used by Gauss (which led to the prime number theorem) [114], the RSA public key algorithm [115] inspired by prime number theory, and our modern blockchain infrastructure.

The introduction of computational methodology has given scientists an understanding of problems previously incomprehensible; however, while previous computational methodologies have proven effective in certain scientific problems or domains, they are not easily generalized to other domains. Big data technologies, especially the field of deep learning that has emerged in recent years, offer a range of techniques capable of effectively detecting patterns in data, and are increasingly proving their utility in scientific disciplines. A specific case of virtual human reconstruction in the Metaverse will serve as an example, to illustrate how deep learning can be used to solve mathematical problems in practical settings.

Virtual human reconstruction is one of the essential tasks in various Metaverse applications: it aims to utilize sensory data to recover the three-dimensional geometry and appearance of humans, achieving accurate photorealistic reconstructions, and ultimately producing compact 3D representations that can be ported to a variety of devices. This problem involves many practical facets that require sophisticated engineering; however, its core challenges lie in deep learning modeling and mathematical optimization, as shown in Figure 6.

Various techniques have been applied to recreate human models in the Metaverse. Many studies start from simple image-based 2D feature detection, such as key points [117], silhouettes [118] and limb segments [119]. It seems that simple movements can be represented relatively clearly by two-dimensional contents; however, it is becoming clear that complex human behaviors, which often occur in practical settings, do not fit the simple assumptions imposed by two-dimensional models, and that more descriptive models with finer granularity are desirable; consequently, more studies [120,121,122] have turned to exploring more complex human pose modeling in three dimensions. Recently, researchers have noticed that body shapes, contacts, gestures and expressions which directly interact with the world are much easier to measure and evaluate; consequently, the focus of researchers has shifted towards three-dimensional mesh recovery of the human body [123,124]. Human body modeling is then further extended by face and hands support [125,126,127,128]. Meanwhile, similar techniques have also facilitated downstream tasks, such as clothed human reconstruction [129,130,131], volume rendering [132], virtual try-on [133], the computer-assistant system [134] and many more Metaverse applications. There are two common paradigms for dealing with virtual human reconstruction: the optimization-based paradigm (described in Section 4.3) and the regression-based paradigm (described in Section 4.4).

Although these two paradigms may have different advantages/disadvantages, and address different aspects, both paradigms can share similar human body modeling techniques. Figure 7 shows an interesting possible way of integrating both paradigms into one coherent framework. The next section will review the existing approaches, in terms of human body modeling.

4.2. Review of Human Body Modeling

Early human body modeling started with the study of articulated geometric primitives, including line segments [135], cylinders [136], planar rectangles [137] and ellipsoids [138]. As three-dimensional full-body scanners became accessible, more detailed measurements of body surfaces could be accurately recorded, such as the CAESAR (Civilian American and European Surface Anthropometry Resource) [139] dataset. The availability of large amounts of body scan data has given rise to a powerful representation: the statistical body model, which factors body deformations into identity-dependent and pose-dependent components. Among the statistical body models, SCAPE [140], SMPL [141], SMPL-X [126], SMPL+H [142], 3DMM [143] and STAR [144] are popular ones, which are not only capable of effectively modeling both shape and pose deformations, but are also highly compatible with existing graphics rendering engines, benefiting from the explicit mesh model. This family of explicit approaches first learns shape deformations through principal component analysis of body scans, and then combines them with skeletal pose-driven deformations (so-called linear blend skinning in traditional skeletal animation), to construct a shape-and-pose parametric human body model. Despite the popularity of explicit approaches, they still have their limitations: firstly, global blend shapes may capture spurious long-range correlations [144], resulting in non-local deformation artifacts; secondly, correlations between body shape and pose-dependent shape deformation may be ignored; furthermore, due to the linear nature of principal component analysis, it can be difficult to reproduce the highly nonlinear deformations of body soft tissue.

In order to overcome the limitations of explicit approaches, instead of explicitly defining the human body as mesh vertices and edges or other elements, implicit approaches try to define surfaces as level sets of continuous functions. Due to these continuous properties, this implicit representation has a better chance of being elegantly optimized and integrated with deep learning frameworks: it is continuous across the spatial domain, and thus theoretically has infinite resolution, and it can easily handle highly nonlinear deformations, and even topological changes, which are not possible with explicit approaches. Study [145,146] estimated implicit surface functions, by aligning image pixels with the global three-dimensional shape or texture of the photographed object, and then using a dedicated multi-level network to refine the resulting geometry. The flexibility of implicit approaches enabled it to handle intricate surfaces and topological changes with ease, but there was one drawback, which was that topologically distinct human representations can exist across time: in other words, implicit human representations may not be topologically consistent in time.

4.3. Optimization-Based Paradigm

In this paradigm, the human body model is explicitly optimized, by minimizing an objective function that fits the model to the observations in an iterative manner. The objective function typically consists of two parts: (1) the data term is a measure of the alignment between the extracted observation features and the transformed human body features; (2) the regularization term is added, to constrain the convergence that preserves a physically plausible body model. In earlier work, the silhouette feature played a crucial role in fitting the body model to the image, as it was used to penalize pixels in non-overlapping regions [147,148].

With the emergence of deep learning, many studies have utilized it to calibrate the optimization initial conditions. SMPLify [123] adopts off-the-shelf neural networks [149] to detect two-dimensional key points, and then iteratively fits a SMPL model, to detect the key points of an unconstrained image. While SMPLify produces relatively well-aligned results, sparse key points do not offer sufficient constraints for body shape optimization. To improve geometric details, [150,151,152] combined key points, silhouettes and part segments, to further constrain the optimization process. Moreover, [153,154] have shown that deep learning techniques can learn local landscapes and decent directions of optimization from training data, and then use them to guide the gradient-based optimization process: in this way, traditional problem-independent optimization schemes can be endowed with the ability to adaptively learn problem-specific convergence schemes. Image-based key point regression was performed by [155,156], to obtain three-dimensional body key points, then solve the inverse kinematics based on the key points and the skeletal structure, so as to calculate the accurate joint rotations, ultimately estimating the parameters of a SMPL model.

Although the optimization-based paradigm can faithfully reconstruct the human body when high quality data is available, it performs poorly in situations where data is scarce and useful information is latent; furthermore, as the optimization-based paradigm intrinsically tries to solve complex non-convex optimization problems in high-dimensional spaces, its outcomes are susceptible to initialization and prone to falling into spurious local minima.

4.4. Regression-Based Paradigm

Alternatively, the regression-based paradigm exploits the powerful learning and approximation capabilities of neural networks, to recover model parameters directly from sensory data. To achieve better performance, researchers have explored a wide variety of network architectures and regression objectives—for example, [125] was one of the pioneering efforts to incorporate the SMPL model into an end-to-end network architecture that minimized the reprojection errors between manually annotated and estimated key points. An end-to-end adversarial learning framework was proposed by [124], which used a discriminator to supervise the training process, so as to exclude anthropometrically implausible or self-intersecting body structures. A top-down framework was proposed by [157], to simultaneously regress SMPL parameters of multiple people in a coherent manner, where depth ordering was consistent, and no interpenetration occurred among reconstructed people. Instead of regressing the SMPL parameters, [158] opted to directly regress the mesh vertices using a Graph Convolutional network, thus allowing the template mesh structure to be explicitly encoded within the network, easily exploiting the mesh spatial locality. Inspired by [124], VIBE [159] went a step further, to estimate dynamic motion sequence from videos. By replacing the regression network with a temporal generative network, and changing the three-dimensional supervision dataset to a motion capture dataset, AMASS [160], VIBE empowered an adversarial learning framework with temporal information, enabling motion sequence estimation as a whole.

To leverage expressive human models and paired data, [127,161,162] adopted a divide-and-conquer strategy, by breaking down the human reconstruction problem into part-specific estimation subproblems, where body, hand and face estimates were performed using the respective part-specific models. The final expressive model was obtained by assembling the individual results of the subproblems into the corresponding body template layers. ExPose [127] directly regressed hands, face and body parameters in the SMPL-X format, and utilized body-driven attention to localize the face and hands regions for refinement, using part-specific knowledge learned from existing face- and hand-only datasets. A real-time method was introduced by [163], to capture body, hands and face with competitive accuracy, by exploiting correlations between body and hands. Pose2Pose [164] extracted joint-specific local and global features, to train a graph convolutional neural network, and regress body/hand joint rotations from it. PIXIE [161] first fused the features from body, face and hand experts, according to their part-specific confidences, and then fed these features into the part-specific networks, for robust regression.

4.5. Technologies in AR/VR/XR Platforms and the Metaverse: Future Trends

In our opinion, AR/VR/XR applications will undoubtedly, in the near future, become the ultimate customer service platforms. In other words, AR/VR/XR applications will at least become the dominant platforms, if they do not completely wipe out the current mobile and computer platforms. Consequently, a big data surge will very soon occur in the virtual world. The Metaverse is likely to be the front platform to face the data surge challenge, due to its rapid growth in recent years. The following figure shows our recently developed VR-based shopping platform.

The authors observed that two extreme situations would occur in the Metaverse, while conducting user recommendation and data analysis: (1) The cold start problem. This situation often occurs when too little data is available for data analysis, due to the VR platforms being new to users, and to not much information having been generated and accumulated for analysis, a common situation in the big data environment, when new platforms are released for users; (2) The virtual data explosion problem. This situation occurs when the Metaverse or VR platforms generate too much data, including user interaction data, wearable sensor data, eye tracking data, location trajectory data, brain EEG data, and business transaction data. Figure 8 shows the data sources of the Metaverse and its architecture [165], which indicates that the Metaverse consists of various data sources from physical, social and digital worlds.

Several methods have been suggested for solving the abovementioned problems. In [166], a position-based VR online shopping recommendation system was developed, to solve the cold start problem in VR platforms. In such a system, the cold start problem is tackled by analyzing new users’ interaction and behaviors within the virtual world. For instance, the position-based VR online shopping system acquires new users’ trajectories in the virtual world, and conducts analysis based on their movements, to generate user recommendations, as shown in Figure 9.

Future trends in solving the cold start problem in the Metaverse will further utilize users’ behavior and sentiment data, including user eye tracking data, user movement trajectory, wearable user device data, and user sentiment data. In particular, human brain data analysis will likely become an essential technology for user analysis in VR platforms, such as the Metaverse.

The cold start problem is not a persistent problem in VR platforms, as it can be solved automatically when data accumulation reaches a certain quantity, whereas the virtual data explosion problem is a persistent challenge to VR platforms like the Metaverse. The wide range of data sources in the Metaverse will grow exponentially, due to its digitization in nature. Some research studies have suggested adopting the Data as a Service (DaaS) framework [91], as the solution to the data explosion problem in the digital world, including the Metaverse. Several other solutions, including tensor networks and sentiment analysis, have been proposed, to solve this problem. The future trends of technical development in the Metaverse and other VR platforms can be summarized as follows:

(1)

Digital human reconstruction is becoming a crucial area for the Metaverse and other VR platforms: this is a core technology that can accelerate the development of the Metaverse, so as to truly realize human–machine interaction in virtual worlds, as mentioned in Section 4.1, Section 4.2 and Section 4.3;

(2)

Digital Twin-related methods are the foundation for creating digital worlds that can mimic the physical world. The digital twin is defined as the effortless integration of data between a physical and virtual environment, in either direction [167]. VR-developing tools, such as Unreal Engine, Unity, 3DS Max & Maya, SketchUp, etc., will be the major developer’s toolkits for digital twin models in the coming decades. The future trends in digital twin will focus on the following: enabling a conformance relationship between digital twin and the real world; digital world autonomy, runtime self-adaptation and self-management; and integration and cooperation, to achieve common goals or provide services [168]. A number of digital twin applications have been developed, based on Microsoft Kinect sensors and the Oculus VR headset.

(3)

Brain–Computer Interface (BCI) technology will become a very important area for the Metaverse and for VR platforms. Previous research indicates that non-invasive BCI technology has been applied extensively in various areas in recent years, because of its minimal potential risks and time precision [169]. Figure 10 shows the high-performance EEG BCI method (left), and EEG BCI experiments (right) [169,170].

The NDA/PDA-based methods are adopted, to enhance EEG data analytical efficiency, in order to accommodate the real-time interaction in the Metaverse and VR platforms [74]. The definition for the NDA method is as follows: if S [a, b] ⊆ A [1, k], if x ∈ [a, b] satisfies:

$$f(A(x),\mu ,\sigma )=\frac{1}{\sigma }\Phi (\frac{A(x)-\mu }{\sigma })\underset{\begin{array}{l}a\le x\le b\\ a\ne b\end{array}}{\to }S[a,b]\subseteq ND$$

(9)

$$\Phi (S)\ge (1-m_r)\times \frac{1}{\sigma \sqrt{2\pi }}{\displaystyle {\int }_a^b\exp (-\frac{c^2}{2})dx}$$

(10)

where m_r is the adjusting parameter, and S [a, b] is an NDA set. The ND-based method derives the data values using ksdensity function, to generate a probability distribution [170]. The definition for the PDA method is as follows: the PDA model takes one of the calculated σ and λ values as λ × t, as indicated in the following equations, 11 and 12. Assuming the original data set has σ, then Mean (λ) is the event rate. If Mean (λ) − λ = ∆, then λ × t is lying between Mean (λ) and λ. With |y − λ × t| = a, a^1/2+a = ∆ is satisfied.

$$P(k events in fix time)=e^{-\lambda }\frac{{\lambda }^k}{k!}$$

(11)

$$P(N(t)=n)=\frac{{(\lambda t)}^n{e}^{-\lambda t}}{n!}$$

(12)

where N(t) is the sample data in the t time window. The Gamma function is utilized in the PDA method for processing complex numbers, which is expressed in (13) below [171]:

$$\Gamma (z)={\displaystyle {\int }_0^{\infty }{x}^{z-1}{e}^{-x}dx}$$

(13)

The ∆ parameter is used to regulate the size of the sample data sets, to get the nearest λ and σ values. The ∆ parameter in the PDA plays the same role that it plays in the NDA method. The PDA model employs a PDA benchmark point selection method [169,170,171].

(1)

Blockchain technology is an efficient and secure solution for digital worlds, such as the Metaverse. In the blockchain model, a new transaction can be verified and added to existing records, i.e., blocks, through linking the new transaction to previous ones, by cryptographic hash operation [172]. Each block contains a cryptographic hash of the previous block, a timestamp, and transaction data [173]. The main characteristics of blockchain technology are that it is secure, decentralized, digitized, collaborative and immutable: these characteristics make blockchain technology a perfect solution for digital virtual worlds, such as the Metaverse. Currently, the most successful security technology for blockchain employs the Public Key Infrastructure (PKI)-based blockchain methods [174]. Researchers in the field have started to search for more efficient solutions. The future trends in blockchain technology development in the Metaverse intend to focus on more autonomous, intelligent and scalable models, such as intelligence-agent-based blockchain [175], Self-Sovereign Identity (SSI) blockchain [176], non-fungible tokens (NFTs) [177] and bio-identity-based blockchain.

(2)

Artificial intelligence (AI) is a discipline essential to almost all areas in our modern world, particularly for future virtual worlds such as the Metaverse. AI can accelerate analytical efficiency, enhance security and privacy, improve interoperability, and provide better solutions for human–machine interaction and collaboration. The increase in applications of Natural Language Processing (NLP), sentiment analysis and brain informatics technologies to digital worlds is stimulating the development of AI in these areas. The successful stories of AI implementation in image recognition, voice recognition, human–machine interaction and intuition, reveal the promising future of AI in the Metaverse and other virtual worlds. A recent survey showed that a majority of studies had focused on exploring efficient integration and collaboration between Edge AI architecture and the Metaverse [178].

5. Discussion

Virtual platforms, such as the Metaverse, and big data technologies are becoming a part of our daily lives. The development of these two different but closely related technologies has accelerated in recent years. In this section, the authors would like to discuss the role of big data in the development of the Metaverse, by reviewing a chronicle of the Metaverse. This section provides the literature review methods used in this survey.

5.1. A Chronicle of the Metaverse and the Role of Big Data in the Metaverse

Figure 11 demonstrates how the Metaverse and its related technologies, which include big data, have evolved and developed [178].

Data sources in the Metaverse and other virtual platforms are growing exponentially; therefore, big data technologies are crucial for the Metaverse, if it is to efficiently manage its digital world, and provide users with real-time analytical services. Big data technologies are fundamental tools for rendering virtual platforms, such as the Metaverse, feasible for users. In other words, big data is a fundamental component in the Metaverse; and the Metaverse accelerates the development of big data technologies; however, big data is not only crucial in the virtual world—it is also an important component of our real physical world, as evidenced in various areas, including the fields indicated in Section 1. Figure 12 shows the relationship between big data and the Metaverse.

The current definitions of the Metaverse vary according to different studies; however, many researchers share a common view that the Metaverse is imitating our physical world. In this survey, the authors believe that future virtual worlds, including the Metaverse, will develop to be totally different world from our physical world: these virtual worlds will go beyond our current social structure and civil life.

Table 3. A brief review of example applications of big data and the Metaverse in major sectors.

Sectors	Big Data	Metaverse
Healthcare	Real-time big data analytical models (Health-CPS); Data as a Service e-health systems, etc., [179].	Metaverse hospital (Thumbay, Davita); Interactive diagnosis platforms, etc., [180].
Finance and Economy	Big data finance and business analytics (Splashback); Online business decision support, etc., [72,73,74,75,76,77,78,79,181].	Metaverse banks (Onyx, ZELF); NFTs, Bitcoins, VR-funds, etc., [182,183].
Education	Learning performance analysis and customization [184]; Education data warehouse, BD curriculum, etc., [185].	Metaversity (Novartis, King’s InterHigh); Immersive realistic learning scene [186].
Entertainment and Social	User behavior and opinion analysis, social trends [187]; Game data monitoring, sentiment analysis [188].	Metaverse games (Roblox, Sandbox) [189]; Virtual social (Meta, Altspace VR) [190].

shows the example applications of the Metaverse and big data in several key sectors.

5.2. Literature Review Methods

This research adopted comprehensive and systematic literature review methods, which consisted of the background and concepts review, survey data retrieval tool selection, and survey data extraction and analysis, summarizing future trends based on analysis and comparison.

This research followed the principles of the literature review outlined in [191], and conducted three types of review methods in the background and concepts review process, including narrative review, systematic review and integrative review [192]. The review of big data and Metaverse applications in different sectors was based on the narrative and systematic review methods. The review of the trends in big data and the Metaverse adopted the integrative review method. The purpose of the big data and Metaverse applications review was to explore the latest developments in big data and Metaverse technologies in the industrial sectors; therefore, a large number of literature and online resources were obtained for narrative review in this process. The purpose of investigating the trends in big data and Metaverse development was to provide readers with a future perspective on big data and Metaverse technologies, and to outline the emerging new methods and the potential paradigm shift. The survey data sources and retrieval tools used in this paper are listed in

Table 4. Literature review methodologies.

Review Content	Review Approach	Retrieval Tools	Data Sources
Big data applications in different sectors (Section 1)	Narrative and systematic review	Digital library Workshop discussion Online survey	IEEE/ACM online IDP2022 forum WJX survey
Metaverse technologies and applications (Section 4.1, Section 4.2, Section 4.3 and Section 4.4)	Narrative and systematic review	Digital library Online search engine Workshop discussion	IEEE/ACM online Google search IDP2022 forum
Future trends in big data and Metaverse technologies	Integrative review	Online search engine Digital library CiteSpace	Google search IEEE/ACM online Web of Science
Correlation of big data and Metaverse and future development	Systematics and integrative review	Digital library CiteSpace Online search engine Workshop discussion	IEEE/ACM online Web of Science Google search IDP 2022 forum

.

The analytical methods of the literature review consisted of CiteSpace analytics [193], Google Scholar [194], Semantic Scholar [195] and the IEEE digital library: these tools provided the authors with the literature chronicle review, citation and impact factor analyses, and the trends analysis. Figure 13 shows the CiteSpace tool that was used in our previous survey [196] for BCI analysis associated with this research.

6. Conclusions

This survey provides a comprehensive investigation of the development of big data technologies and virtual platforms, i.e., the Metaverse. Undoubtedly, the Metaverse and big data technologies will become the most influential areas in the following decades. Big data is an essential component of the Metaverse, and it also plays a crucial role in the parallel real world: these two areas will eventually merge into the virtual world, and advance mutual development.

This research survey grouped the big data technologies into four categories: big data acquisition and pre-processing; big data storage and data infrastructure; big data analytics; and big data privacy and security. The authors further investigated the future trends in each category, and obtained the following findings: (1) future big data acquisition and pre-processing technologies will aim to deal efficiently with more unstructured, high-dimensional data. Among various data acquisition methods, blockchain, data fabric and NLP pipelines demonstrate their potential for future big data acquisition; (2) future trends in big data storage methods intend to focus on more elastic and cloud-based solutions. Fog-to-Hybrid and multicloud, Data as a Service model, blockchain data storage, and high-dimensional data segmentation and tensor networks are the major future trends for big data storage; (3) the trends in big data analytics will focus on natural language processing, sentiment analysis, real-time analytical models and virtual reality/Metaverse analytics models, and data-centric AI analytics; (4) the trends in big data privacy and security will mainly focus on cloud, blockchain-related areas, such as cloud security (combination of centralized cloud services with fog computing), blockchain secure data privacy and security.

The Metaverse and other virtual platforms have grown rapidly in recent years. The PwC Co. predicts that VR and AR platforms will boost global GDP by USD 1.5 trillion by 2030 [197]. To date, applications of the Metaverse have included online shopping, virtual social media, video games, virtual tours, and online museums and arts [111,112,198]. Many large technology companies have announced plans to launch their Metaverse products, such as Facebook Horizon, Nvidia Omniverse, and Amazon Metaverse. The future trends in technical development in the Metaverse and other VR platforms can be grouped into five main areas: digital human; digital twin; brain–computer interface (BCI), blockchain and artificial intelligence. Notably, brain–computer interface technologies have become increasingly important to Metaverse development in recent years, as immersive interactions provided by BCI can enhance user experience [196,199,200,201,202].

In summary, our society is becoming more digitized and virtualized. A virtual world, e.g., the Metaverse, can bring our society some overwhelming benefits, including convenience, energy saving, time efficiency, creativity, and being environmentally friendly. Big data technologies, as a fundamental component of the Metaverse, offer methods and algorithms to solve the data explosion and analytical bottleneck problems in the digital world.

The Metaverse is currently imitating our physical real world as a parallel virtual world; however, the Metaverse will become very different from our physical world. The authors believe that virtual worlds, with the help of big data and AI technologies, will become more advanced than our physical world, in many aspects. Big data is undoubtedly one of the most crucial areas for virtual worlds, such as the Metaverse, because virtual worlds are digital data-based, and will accelerate data explosion; therefore, it is crucial to understand the development of big data and of the Metaverse, respectively, and then to find out the common areas in which they would facilitate each other, as shown in Figure 12. Future research will further identify the research problems in the Metaverse and other virtual worlds, in terms of dealing with big data issues in complete virtual or hybrid virtual environments.