Integrating Machine Learning with Non-Fungible Tokens

Abstract

In this paper, we undertake a thorough comparative examination of data resources pertinent to Non-Fungible Tokens (NFTs) within the framework of Machine Learning (ML). The core research question of the present work is how the integration of ML techniques and NFTs manifests across various domains. Our primary contribution lies in proposing a structured perspective for this analysis, encompassing a comprehensive array of criteria that collectively span the entire spectrum of NFT-related data. To demonstrate the application of the proposed perspective, we systematically survey a selection of indicative research works, drawing insights from diverse sources. By evaluating these data resources against established criteria, we aim to provide a nuanced understanding of their respective strengths, limitations, and potential applications within the intersection of NFTs and ML.

1. Introduction

The advent of NFTs has marked a significant milestone in the digital age, introducing a new paradigm of “proof-of-ownership” for digital assets. Rooted in blockchain technology [1], NFTs have emerged as a powerful tool for asserting the uniqueness, provenance, and ownership of digital assets. While cryptocurrencies, in general, enable instantaneous proof of ownership by signing a transaction, NFTs extend this concept by providing unique and verifiable proof of ownership for digital items. NFTs are distinguished by their inability to be replicated, offering a digital counterpart to the uniqueness found in physical collectibles and artworks [2,3]. The concept of fungibility, where items can be exchanged on a one-to-one basis, like currencies, contrasts sharply with the non-fungible nature of assets like real estate, art, music, and unique digital-based art. This distinction underpins the value and appeal of NFTs, enabling them to represent ownership of unique digital items—ranging from art and music to virtual real estate and beyond. Each NFT points to a distinct entity on the blockchain, providing a transparent and immutable record of ownership and transaction history [2,3]. The rise of NFTs has been propelled by the Ethereum blockchain, initially through the development of the ERC-721 standard [4], which laid the groundwork for creating and trading unique digital assets.

However, the first-ever NFT on the blockchain was “Quantum” on Namecoin blockchain [5]. This innovation not only revolutionized the way we conceive digital ownership but also created new streams for artists, creators, and collectors to monetize and exchange digital works in ways previously unimaginable. The explosion of interest in NFTs between 2020 and 2022 highlighted their potential to transform digital commerce and ownership. High-profile sales, such as Beeple’s “Everydays: The First 5000 Days”, which fetched $69.3 million at auction, underscored the market’s capacity for the valuation of digital art and collectibles. Despite experiencing a period of volatility and a subsequent “crypto winter”, the evolving landscape of NFTs continues to show signs of maturation and innovation, with new use cases emerging beyond mere speculation. As NFTs carve out their role in the digital economy, they also intersect with advancements in ML, offering novel approaches to verifying authenticity, enhancing valuation models, and curating personalized experiences in the metaverse [6]. Taking into consideration the discussions around the emergence of blockchain-based metaverse(s), which is thought of as the next evolution of the web [6], often referred to as web3, NFTs have the potential to innovate almost every sector, such as education [7], gaming [8], real estate [9], etc.

2. Literature Review

The synergy between NFTs and ML is paving the way for more secure, efficient, and meaningful interactions in the digital realm, heralding a new era of digital asset management and ownership. Moreover, several startups have emerged that leverage non-fungible tokens to generate revenue. For instance, NFTValuations [10] employs advanced methodologies integrating ML techniques to analyze and predict the value of NFTs. NFTValuations [10] continuously fetches transactions and events from supported marketplaces and collections, aiming to provide accurate valuations and market insights. Such initiatives highlight the innovative approaches startups are taking to capitalize on the NFT market [3].

Furthermore, the recent advancements in AI, exemplified by the advent of high-caliber interactive platforms powered by Large Language Models (LLMs) like ChatGPT, mark a pivotal transformation in technology exploration. This evolution underscores the potential for synergy among cutting-edge technologies such as blockchain, AI, and the Internet of Things (IoT), fostering new opportunities and applications.

In this study, we aim to investigate how ML, a critical subset of AI, influences NFTs across multiple dimensions. Our exploration encompasses the use of ML in creating multimodal objects for NFT deployment on the blockchain, and extends to indexing, searching, managing, and recommending NFTs. Furthermore, we delve into ML’s direct applications in fraud detection on the NFT ecosystem, NFT valuation, and its role in supporting indirect functions such as the development of market-focused business intelligence tasks on the NFT ecosystem.

Taking things from the very beginning, the history of NFTs from their very early stages is related to various notable NFT collections created using generative art algorithms, which denotes the usage of ML for the creation of data-driven aesthetics. The broader generative art [11] theory, which is out of the scope of this work, refers to generative art as the kind of art created by using external systems such as computer machines, programs, sets of natural language rules, or other kinds of systems which is capable of producing a completed work of art, where the artists contribution differs in the essence of controlling the control of these rules. In this work, we aim at a subset of generative art and more specifically the generative art NFTs. To briefly explain, generative art NFTs are NFTs created by a code where the artist creates a set of visual objects (re-images) and then adds certain rules to the code like attributes of varying degrees of rarity and then the code combines the rules with the set of virtual objects to produce an output usually minted into NFTs on the blockchain network.

In the work [12], there is a discussion around the concept of Programmatic NFTs (pNFTs) also referred to in the literature as dynamic NFTs [13]. The concept of pNFTs/dNFTs refers to the ability of the attributes of NFTs, also known as metadata, to not be static but able to interact with their environment and change based on various factors such as ownership or time, through smart contracts [12,13]. The concept of pNFTs/dNFTs was introduced as a groundbreaking form of artistic expression, where AI capabilities will enable the creation of intricate artworks from diverse inputs, utilizing ML algorithms to interpret and innovate upon existing art trends. Ref. [12] envisions an AI-driven art ecosystem where all functions traditionally held by humans will be driven by AI entities/agents. This will include artists generating art based on programmed inspirations, AI curators organizing exhibitions with thematic coherence, AI galleries showcasing these works in digital or physical spaces, and AI collectors engaging in the buying and selling of artworks in a completely autonomous manner. As the research work [12] focuses on the theoretical aspects behind the concept of pNFTs/dNFTs, it could not be classified under the comparative analysis of the respective research works but works as an insightful step for the authors of this research work towards the identification of relevant research works attempting to implement the concept of pNFTs/dNFTs using ML in various sectors.

The AI field can be approached from two perspectives: top-down and bottom-up, which directly influence the design and development of computational models. The former relies on a rule-driven framework where human experts typically craft specific rules of interest, which are then utilized by algorithmic mechanisms to derive knowledge. In contrast, the latter perspective operates without assuming preexisting rules or knowledge, instead extracting target knowledge algorithmically through data analysis. The bottom-up perspective forms the basis of ML, where data serves as input and is processed to extract actionable insights using various computational tasks such as classification, regression, or generation, depending on the specific problem at hand. Given the vast scope of the AI/ML field, this work focuses on a specific aspect: proposing a methodological framework to aid understanding of its convergence with NFTs. Throughout this work, AI and ML are used interchangeably to denote the data-driven (bottom-up) paradigm, unless otherwise specified.

The core research question of the present work is as follows: How does the integration of ML techniques and NFTs manifest across various domains? Specifically, this research question explores the core ML modeling aspects utilized in various NFT use cases, from data and feature engineering to evaluation metrics, as well as other related matters. The objective of this question is to provide a comprehensive overview that bridges both scientific and engineering perspectives by offering critical information at each stage of the ML pipeline. It addresses a common issue in existing literature where such crucial information is often presented in a fragmented manner. By examining the entire process holistically—from data and feature levels to evaluation metrics—we aim to foster a more integrated understanding of ML applications in the NFT domain. This is motivated by the fact that the integration of ML and NFTs constitutes a symbiotic relationship grounded in several inherent advantages. Firstly, ML exhibits natural application within the realm of NFTs due to its capability to derive patterns from both the content encapsulated within NFTs and their associated metadata. Moreover, ML extends its utility beyond the NFT’s content and metadata, permeating into peripheral aspects critical to the ecosystem. These include core blockchain aspects, community dynamics, as well as the respective economic dynamics. Within this multifaceted context, ML leverages various feature types and employs diverse methodological approaches encompassing both unsupervised and supervised learning paradigms. Such versatility empowers ML to serve as a transformative toolset for comprehensively harnessing the potential of NFTs across their entire spectrum.

Our contribution to addressing the aforementioned research question is a proposal of a methodological framework that consists of two pillars. The first pillar deals with the identification of a series of technical parameters that are essential for the holistic characterization of a model, while the second pillar takes the form of a spectrum of application domains. The combination of the two pillars enables a comprehensive and unbiased perspective by addressing key technical and application-specific criteria. This approach helps to filter out any biases towards specific domains and ensures that all relevant aspects of the research works are considered. The application of the proposed framework is demonstrated across various research works.

The salient system parameters refer to the key technical criteria such as the type of input and output data, the computational tasks, the use of supervised or unsupervised learning, the application of deep neural networks, and the evaluation metrics. These parameters are essential for a holistic understanding of the research works and help to identify strengths, limitations, and potential applications within the intersection of ML and NFTs.

In the previous paragraphs, we provided a relatively high-level overview of the field to set the background. Deeper scientific and engineering aspects are discussed in the sections that follow, where our framework is presented (first and second pillar). Furthermore, it should be noted that this framework is applied to a number of indicative research works that characterize the spectrum of the application areas. The rest of the paper is organized as follows. In Section 2, the proposed methodology is presented. The research findings are reported in Section 3, while the discussion and conclusion are provided in Section 4.

3. Methods

Next, we present the first pillar of the proposed methodological framework which adopts a technical point of view. Specifically, eleven parameters (also referred to as criteria) have been identified for characterizing the research works under consideration as those demonstrated in Figure 1. The proposed set of parameters provides a holistic perspective in the sense that enables the reader to comprehensively understand all the key factors. It should be noted that this pillar is based on a recent work (co-authored by one of the co-authors) dealing with the application of artificial intelligence in pharmaceutical development [14].

• SP: Supervised vs unsupervised;
• CT: Computation task;
• IT: Type of input;
• OT: Type of output;
• ST: Structured vs unstructured input;
• DN: Deep neural networks;
• ND: Non deep neural networks;
• FS: One or multiple models;
• EV: Evaluation metrics;
• DT: Use of accessible data;
• SW: Software.

For each criterion, an abbreviation was utilized, such as “IT” for input type. These parameters encapsulate the essential aspects that an ML expert would typically seek when encountering a method for the first time. Understanding the type of input and output stands out as the foremost consideration. Notably, certain studies lack some of these aspects due to reasons like omission, implication, or assumption of reader expertise in ML, which poses challenges, especially for experts coming from related fields. The SP parameter denotes a crucial yet high-level characteristic determining whether the models under consideration necessitate training (supervised) or not (unsupervised). This specification significantly influences the choice of algorithms and associated costs. Supervised approaches demand annotated data that require a substantial number of expert annotators and a time-intensive process. In contrast, unsupervised methods operate directly on raw (non-annotated) data. The second parameter, CT, pertains to the computational task, which may involve regression, classification, as well as generative models (a widely used family of models for NFTs) determining the model’s output. Regression-based models yield numerical outputs, while classification-based ones produce symbolic descriptions corresponding to classes of interest. Generative models constitute a different approach since they focus on the learning of the distributions that underlie the data of interest. The fifth parameter, ST, concerns the structure of input data. Knowledge of structured or unstructured data is crucial, as the unstructured case demands additional preprocessing to shape data for model input. Parameters DN and ND indicate the application or absence of deep learning approaches, reflecting recent advancements in representational learning. FS represents an engineering parameter involving the combination (also referred to as fusion) of multiple models to enhance accuracy and robustness. Evaluation metrics (EV) are pivotal in ML, guiding model training and tuning based on measurable aspects. Understanding these metrics facilitates comparative analysis across approaches, particularly when applied to common benchmark datasets, closely tied to the DT parameter denoting used datasets. SW denotes software utilized in studies, contributing to transparency and suggesting applicable computational tools.

The second pillar of our proposed framework, which is presented in Figure 2, takes the form of several areas where ML and NFTs converge. These areas are as follows:

• Area 1: NFT generation and enhancement;
• Area 2: Content analysis and classification;
• Area 3: Marketplace dynamics;
• Area 4: Fraud detection;
• Area 5: Ownership and rights management;
• Area 6: Market prediction and investment strategies;
• Area 7: Personalization and recommendation systems.

Figure 2. Visualization of Pillar 2.

Figure 2. Visualization of Pillar 2.

The identification of the above areas aims to smooth a bias that is clearly visible in the respective literature being the focus on the sixth area which includes topics around market valuation and price prediction. While we acknowledge the importance of this area, at the same time we recognize the value of a broader view. Overall, the combination of the aforementioned pillars creates a methodological framework for grounding the analysis and understanding of the research works under investigation.

4. Results

During our literature review, we attempted to identify and export indicative works from the literature that demonstrated the influence of ML techniques on NFTs from different perspectives. These works were classified into seven areas. The classification of the papers derived from the focus of each research work analyzed and described below:

• NFT Generation and Enhancement: During the literature review, we found a significant number of works researching how ML can create or even enhance multimodal objects (i.e., artworks, videos, etc.) that can be minted as NFTs. In this category, we include works focusing on how ML can automate the generation of unique digital assets, keeping the aesthetic and artistic value at the same time.
• Content Analysis and Classification: Research works put under this category examine the application of ML for analyzing and classifying the content of NFTs, including visual and textual data.
• Marketplace Dynamics: This area investigates how ML can be applied to understand and predict the dynamics of NFT marketplaces.
• Fraud Detection: An interesting area where we have the utilization of ML models to detect fraudulent activities within the NFT ecosystem, such as wash trading and anomalous transactions.
• Ownership and Rights Management: In this area, we consider works that focus on identifying plagiarism in NFTs.
• Market Prediction and Investment Strategies: Research works in this area focus on the usage of ML to predict NFT pricing and market trends aiding stakeholders and collectors in making correct decisions.
• Personalization and Recommendation Systems: This area focuses on how ML can be used to personalize user experiences and provide recommendation systems within NFT platforms. This is made through the analysis of user preferences and behaviors through the utilization of ML models which can help in providing accurate buying or investing recommendations on NFTs.

In the following paragraphs, we apply the proposed framework for the identified research works. An overview is provided in

Table 1. Characterization of research works with respect to core ML aspects (supervised/unsupervised, computation task, input/output type, neural networks type, one/more models, evaluation, datasets, software) in association with different application areas.

Work: Reference (1st Author, Year)	Area	Supervised/ Unsupervised	Computational Task	In/Out Type/	Structured	Non Deep/Deep Neural Nets /ND	Fusion	Evaluation Metric	Dataset	Software
[15] (Pungila, 2022)	5: ownership/rights	unsupervised	statistical	text descriptors	yes (conditional)	non deep neural nets	no	yes	yes	no
[16] (Sharma, 2022)	1: generation/enhancement	supervised	generative	images	yes	deep neural nets	no	no	yes	no
[17] (Shahriar, 2022)	1: generation/enhancement	supervised	generative	images	yes	deep neural nets	no	yes (multiple)	yes	yes
[18] (Martelée, 2022)	6: prediction/investments	supervised	various	text	yes	deep neural nets	no	yes	yes	yes
[19] (Wang, 2024)	2: content analysis/classification	supervised	classification	images, text	yes	deep neural nets	no	yes	yes	yes (indirectly)
[20] (Ghosh, 2023)	6: prediction/investments	supervised	various	text	yes	non deep neural nets	yes	yes (multiple)	no (conditionally)	no
[21] (Zhang, 2024)	3: market dynamics	supervised	prediction	graph	yes	non deep neural nets	no	yes (multiple)	yes	yes
[22] (Kapoor, 2022)	6: prediction/investments	supervised	regression, classification	images, text	yes	deep neural nets	yes	yes (multiple)	yes	no
[23] (Pelechrinis, 2023)	4: fraud detection	supervised	regression, classification	text	yes	non deep neural nets	no	no	yes	yes
[24] (Li, 2023)	7: personalization/recommendations	supervised	regression, classification	categorical, numerical data	yes	deep neural nets	yes	yes (multiple)	yes	yes
[25] (Chen, 2023)	2: content analysis/classification	unsupervised	prediction	images	yes	non deep neural nets	no	yes	yes	no

.

In [15] an unsupervised approach is proposed for identifying plagiarism in NFTs. This is motivated by the explosive growth of the respective field, which unfortunately triggers incidents of plagiarism. In this context, CT is straightforward: estimating similarities between text fragments associated with NFTs (referred to as metadata in the context of NFT collections), which can serve as indicators for plagiarism. These fragments constitute the IT, which typically aims to describe the corresponding artwork. Depending on the exact format of IT, we may encounter various cases falling within the structured/unstructured spectrum. OT consists of an indicator signaling the potential occurrence of plagiarism. A non-parametric model is utilized (non-DNN), which is based on non-deterministic finite automata (NDFA). Within this framework, several widely used textual metrics are employed, namely, Euclidean, Hamming, and Levenshtein distance. The NDFA used is not fused with any other models, while the achieved accuracy is evaluated in a very intuitive way based on false positives and false negatives. This work mentions two evaluation datasets reported in the literature; however, they are not specific to NFTs. Regarding software, no specific tools/libraries are reported.

The work reported in [16] utilizes supervised learning for the task of NFT generation. The computation task cannot be regarded as a regression of classification; on the contrary, the distribution that underlies the training data is learned. The input as well as the output are images, which, by definition, follow a structured format. A DNN architecture is used based on Generative Adversarial Networks (GANs) focusing on the following activation functions: GCU and ReLU. In this context, a single model was used rather than a fusion of different models. The evaluation metric for the generated NFTs is not explicitly mentioned. However, it is briefly implied that a discrimination model (real vs. non-real images) can be used for evaluation-related purposes. The widely known Bored Apes Yacht Club Dataset was used.

The research work in [17] explores the application of (GANs) for the automatic generation of digital art that could be minted as NFTs. Similarly to the work in [16], the authors [17] implemented a specific DNN) architecture based on GANs, but they used a different GAN model, specifically StyleGAN2 [26]. The selection of StyleGAN2 [26] was based on this model’s capability to generate high-resolution and stylistically diverse images. For their research, the authors utilized a Kaggle dataset containing 3021 NFT-style art images resized to 512×512 for GAN compatibility. After removing all non-RGB images, 2283 images remained for model training. The training was conducted on Google Colab Pro, leveraging the computational power of an NVIDIA Tesla P100 GPU to generate the output images. Therefore, both the input and the output were images, denoting a structured data format. In general, GANs, fall under the generative models which are designed to generate new data instances that resemble the provided training data. Likewise, in this research work [17] the implemented StyleGAN2 [26], learns from a dataset of NFT-style art images and generates new artworks that mimic the style and content of the training dataset. This approach is fundamentally different from regression and classification tasks; thus, this work cannot be classified under either of the two. Regarding the evaluation metrics incorporated in this research study, they were both quantitative and qualitative. In the quantitative metrics, the Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) metrics were used to evaluate the quality of the generated images relative to the original dataset. On the qualitative side, a case study involving 26 participants was conducted to assess the perceived quality, innovation, and artistic value of the generated images. The results of the research work in [17] showcased the ability of the GAN-based model to generate a high degree of similarity to NFT artworks as this was also supported by the qualitative case study where participants rated GAN-generated artworks closely to real samples in terms of interests, inspiration, and overall quality. Specifically, the generated images were perceived as more innovative than their real counterparts.

Research work in [18] attempts to predict the NFT market using deep learning techniques. To achieve this the authors [18] combined traditional market analysis with ML techniques. During their research various challenges faced by the authors, including the complexities of collecting and preprocessing heterogeneous data from blockchain transactions and social media, managing the high dimensionality and variability inherent in NFT attributes, and the intricacies of designing and training deep learning models to accurately forecast NFT market trends. Therefore, they provide useful insights about the usage of ML as a forecasting tool for NFT market trends. The data utilized for this research are from the Ethereum blockchain and more specifically from OpenSea’s API focusing on prominent NFT collections such as CryptoPunks, Ape Yacht Club, Decentraland, and Rumble Kong League. For the data collection, the authors used (i) OpenSea’s API to fetch transaction details including sale prices, transaction dates, and NFT metadata; (ii) Web Scraping employing Python libraries such as Beautiful Soup [27] to scrape data from NFT marketplaces, and additional insights into market trends; and (iii) Twitter API, focusing on tweets related to the NFT market and specific collections. Following this was the data transformation where authors converted timestamps into more manageable formats normalizing the data, and cleaned tweets’ text by removing URLs, mentions, and non-ASCII characters. NFT attributes were encoded using One-Hot Encoding and Embedding [28]. For the prediction of the NFT market trends the authors [18] explored several deep learning architectures for their wide array of input features. Prediction of NFT prices and sentiment analysis of tweets in this work are classified under supervised learning techniques as there we have a continuous value and regarding the computation task, we have both regression for the NFT prices and classification for the sentiment analysis of tweets. Referring to the input and output of the ML models they include text (tweets), numerical (transactions, dates), and categorical data, while the authors deal with both structured (re numerical, categorical) and unstructured input (tweets). In this research work, various models were implemented. Dense Neural Networks were used for price prediction with ReLu activation and dropout layers for regularization the Long Short-Term Memory (LSTM) was used for sentiment analysis on the tweets’ text. In terms of evaluation metrics introduced to validate the result of their prediction model, the Mean Squared Error (MSE) and R² were used for regression tasks, where “accuracy” was used for classification tasks.

The work described in [19] is motivated by the lack of a research-oriented NFT dataset meant for the field of computer vision. In this context, the authors follow an approach for supervised learning for the task of image-text retrieval formulated as matching images with textual descriptions. The input type (IT) consists of both images and text, while the output type (OT) is the retrieval of images given textual queries (i.e., the descriptions of the desired images). Both structured and unstructured input is present, for images and natural language text, respectively. A pre-trained DNN model is used, namely, the OpenAI’s Contrastive Language-Image Pretraining (CLIP) model [29]. Furthermore, several variants of CLIP are mentioned; however, no fusion approaches are reported, that is, a single model is utilized. A task-specific metric, named the Comprehensive Variance Index, is used for assessing the performance of the retrieval-based image-text similarity. A customized dataset is constructed by compiling the top 1000 PFP (profile pictures) projects following Ethereum’s ERC-721 standard. Specifically, the dataset includes 7.56 million image-text pairs requiring 1.75 TB of storage space. Although not mentioned, it is likely that the OpenAI (https://platform.openai.com/docs/libraries/, accessed 15 April 2024) library is used for the core experiments mentioned in this paper.

In their research work, the authors of [20] present a comprehensive study on the application of ensemble ML techniques and AI, with the outcome of their research to be a multivariate framework that attempts to predict and interpret the price dynamics of NFTs and DeFi assets. The data collection included daily prices of NFTs among four blockchains as those were Enjin, MANA, Tezos, and Theta and DeFi assets on which we will not deepen as they are outside of the scope of this research work. The data collection period spread between January 2020 and July 2022, and variables included technical indicators, prices of Ether, Bitcoin, economic political uncertainty (EPU) [30], prices of crude oil and gold, and several media chatter indices related to COVID-19 which are conditionally accessible. For their prediction model, Isometric Mapping (ISOMAP-GBR) and Uniform Manifold Approximation and Projection (UMAP-RF) techniques were selected or feature engineering due to their efficiency in comparison to other models like support vector regression (SVR), ARIMA, and SARIMA. In terms of the ML models the authors [20], employed Gradient Boosting Regression (GBR) and Random Forest (RF) which demonstrated high predictive accuracy for the NFTs and DeFi assets under study. Regarding the evaluation metrics used to assess the performance of their prediction models, these include (i) Nash–Sutcliffe Efficiency (NSE), (ii) Theil’s Inequality Coefficient (TI), Index of Agreement (IA), and Directional Accuracy (DA). The outcomes of this research are multifaceted and include the performance of the predictive model to forecast market volatility in NFTs and DeFi prices. In addition, the influence of technical indicators along with the prices of Ether and Bitcoin works as a significant indicator for the NFT prices. However, EPU, oil, and gold prices’ impacts were not so significant as predictions were not as strong as those derived from technical indicators and major cryptocurrency prices (re Ether, Bitcoin). Likewise, macroeconomic and media factors were less important indicators for NFT price movements.

The research conducted in [21] is motivated by the need to study the NFT space from the angle of temporal graph analysis, given the respective lack in the literature. The CT of interest deals with link prediction across time, that is, based on transactional history, to predict the formulation of links between nodes. The input data are of a highly structured format, as the typical set of nodes and edges is used for the formal definition of the underlying graph. In this formalism, another set augments the sets of nodes and edges, being a set of timestamps, which adds a temporal dimension to this graph representation. The output takes a straightforward format as it is the aforementioned predictions regarding the possible links between the nodes. A DNN architecture is used, Graph Neural Networks (GNNs), which constitute a broad family of models used for graph-specific tasks including node/graph classification and link prediction. In addition to GNNs, statistical models are utilized to represent the structural properties and dynamic behaviors of the graph. Also, a single-model approach is adopted, that is, no multiple models are fused. Regarding DT, the graph under investigation is built using publicly accessible data (Etherscan) for a period of approximately five years. The acquired data is further filtered according to criteria directly related to NFT standards, such as the EIP-721 standard. Widely used evaluation metrics are used, including Area Under the Curve (AUC) and Mean Reciprocal Rank (MRR). The authors report the use of specific tools like the popular Geth interface and Ethereum ETL that can be used for transforming blockchain data into generic formats like CSV.

The research work in [22] explores the valuation of NFTs and how social media and Twitter in particular influence their price and market trends. As part of their research, the authors [22] collected data from Twitter and OpenSea creating a dataset consisting of 245,159 tweets from 17,155 unique users which included direct links to 62,997 NFT assets. Regarding OpenSea, they collected many valuable features related to each asset through the OpenSea API such as the media file associated with the asset (e.g., video or GIF), totaling a dataset with 62,997 images linked to NFT assets. The total worth of NFT assets at the time of data collection was USD 19 M. The Dataset created for this research conforms to the FAIR dataset principles (i.e., Findable, Accessible, Interoperable, and Reusable) and it is publicly available. As for the diversity of data collected, those retrieved from Twitter included metrics like the number of followers, whether NFT is in the username, number of likes, and replies can be characterized as highly structured data. Furthermore, those derived from OpenSea API included the number of bids, the presale status, the number of transfers, and the associated media file (i.e., video or GIF) were of a structured form too. In their attempt to address the NFT valuation challenge based on the influence of Twitter, they have employed a mix of ML and deep learning techniques including both binary and ordinal classification. Binary classification was used to determine if the sale of an NFT was profitable or not. In the corresponding dataset, 78% of the assets were classified as unprofitable (i.e., unsold or sold for less than USD 10). Furthermore, the authors categorized the unprofitable assets class derived from the binary classification into further subclasses using multiclass ordinal classification. As mentioned above, a mix of ML and deep learning models was used during this research. The ML models included logistic regression, SVM, Random Forest, LightGBM, and XG Boost applied on features retrieved from Twitter and OpenSea. The deep learning models incorporated in their research include Convolutional Neural Network (CNNs) architectures for image-based predictions and in particular ResNet-101 and DenseNet-121. XG Boost ML model showed the best performance across various setups. While evaluation metrics were not directly referred to in the work of [22], binary accuracy, binary F1, ordinal accuracy, and ordinal index were used to determine the accuracy of the deployed models. The findings concluded that the influence of social media and Twitter when combined with market activity is significant to the NFT valuations. Therefore, the incorporation of ML models, which use combined features from social media and market activity can lead to the development of profitable strategies for NFT assets. The authors also mentioned future actions to focus on addressing fraudulent transactions to better handle outlier transactions and artificial market signals [22].

The research work in [23] falls under the fraud detection area as they cope with spotting anomalous trades in NFT markets that could potentially indicate illicit activities such as money laundering. For their research, they focused on a specific marketplace namely NBA TopShot, which facilitates the trading of sports collectibles including NBA moments on Flow blockchain network. Their dataset consists of transactions retrieved from NBA TopShot between July 2020 and March 2021, including among others features like unique identifiers for NBA moments and transactions, player IDs, set IDs, and transaction times, which are structured data. The outcome of their research was the development of a framework that combines ML techniques and network analysis to categorize transactions as anomalous. As part of their developed framework, they use linear regression as part of their developed Profit Model (MPE) to predict the profit for each transaction. In addition, they build a second model called MCDErs for anomaly detection where actual profits significantly deviate from predicted profits. For this model, they incorporated Random Forest for Conditional Density Estimation (RFCDE) [31]. During their research, they managed to identify 2767 transactions as abnormal out of 1,025,728 analyzed transactions, through their proposed framework. Their study concluded several findings with the most significant including the need for more actions towards the security and regulation of NFT marketplaces. Their proposed framework aims to identify abnormal transactions with high accuracy, but due to its focus on a specific marketplace (i.e., NBA TopShot) and the lack of a definitive ground truth for determining whether an abnormal transaction is at the same time an illicit transaction poses a challenge. Moreover, the usage of ML to identify fraudulent actions within the NFT market can lead to significant improvements in the prevention mechanisms (i.e., law enforcement and financial intelligence units) while at the same time, the generalization of such frameworks in more than one NFT marketplaces could enhance the synergy between ML and NFTs in the identification of fraudulent actions in the space.

The research work in [24] proposes a recommender system referred to as NFT.mine [32], which is based on advanced ML techniques and contributes to a deeper understanding of NFT market dynamics. However, the main contribution of this work is a recommender system, we classify it under “Personalization and Recommendation Systems” area. Exploring the study of [24], we identified that their proposed system provides recommendations based on real-time data collection. However, the dataset used in this paper [24] contains 396,707 NFT transactions obtained from OpenSea API including 185 features between 12 and 27 April 2022. In terms of the ML techniques employed in this paper, those are primarily supervised learning methods. Logistic regression (LR) was used to determine the probability of a user preferring a specific NFT. Naive Bayes (NB) was used to estimate the probability of user preference based on independent assumptions among features. Random Forest (RF) was also used to predict user preferences for NFTs based on a subset of data with known outcomes. All the above ML techniques were employed for performance comparison between them and the proposed in-house developed xDeepFM model of NFT.mine. Moreover, the NFT.mine recommender system includes five modules as those are Python scrapper, the Exploratory Data Analysis (EDA) module, the dataset module, the server module, and the xDeepFM model. Examining the components the xDeepFM model consists of these include a Compressed Interaction Network (CIN), a Deep Neural Network (DNN), and a Linear Network, among others, to derive its predictions. The input of the XDeepFM model is in Field-aware Factorization Machine (FFM) format, encompassing a wide range of categorical and numerical features. Based on the FFM input, it is evident that the data input into the xDeepFM model is structured. Furthermore, two evaluation metrics were used to measure the performance of baseline models (i.e., LR, NB, RF) in comparison to the NFT.mine model. These evaluation metrics were the Area under the ROC Curve (AUC) and Cross Entropy Loss (Logloss). The outcome of this research showed that the proposed the NFT.mine model outperformed all baseline models in prediction accuracy and reliability in generating recommendations, demonstrating at the same time its ability to handle complex real-time data. In conclusion, this work demonstrates the potential of AI and ML in developing recommender systems for NFT buyers with significant accuracy in their recommendations.

The work in [25] comprises two main branches: the extraction of visual features, also referred to as aesthetic features, and an investigation of the development of predictive models for NFTs pricing utilizing the extracted features. For this purpose, unsupervised models were used for extracting different feature types, such as color, composition, edges, and entropy related. The computation task is of a statistical nature, while the output takes the form of correlation maps. Inputs comprise image data, yielding a pool of visual features represented by numerical values. A statistical model was used (i.e., a single model without fusion). Regarding the aforementioned computational task (CT), the evaluation uses Pearson’s correlation coefficient. The input data were sourced from two prominent collections on OpenSea, namely, CoolCat and Doodles. The paper does not report any software tools or libraries.

Furthermore, we have observed that there is a number of relevant works/projects that provide sporadic information with respect to the proposed systematic framework. Some of them, especially projects, do not have a presence in the literature. However, we acknowledge their contribution to the overall field. A brief presentation of several indicative works is presented below.

Another study that explores the usage of ML techniques for identifying fraudulent actions related to the NFT market is presented in [33]. Specifically, in their study, the authors in [33] did not explicitly utilize ML techniques but developed transaction graphs for each NFT. In their graphs nodes represent the address while the edges represent the transactions. Their dataset contains open data from Ethereum for 52 prominent NFT collections based on the ERC721 contract between January 2018 and November 2021. It includes 21,310,982 transactions, associated with 459,954 addresses, totaling USD 6.9 million in trading volume. They utilize an adjusted version of the Deep-First-Search-Algorithm [34] to identify closed cycles within the dataset, which denoted possible suspicious transactions. Their findings indicate that wash trading significantly inflates trading volumes, although to a reduced size than previously expected. This is supported by the fact that 2.04% of all transactions and 3.93% of addresses from the dataset were flagged as suspicious [33]. Our literature review on the synergy between ML techniques and NFTs from different perspectives as those classified in our identified areas supports evidence of an increasing demand for ML techniques to address different challenges and enhance current practices on NFTs. Likewise, a notable initiative attempting to establish a methodology for accurate valuation of NFTs is the NFTValuations project [10]. NFTValuations employs an advanced methodology that integrates ML techniques to analyze and predict the value of NFTs. While specific details regarding the ML models used by the project are not publicly available, they refer to a fusion of ML models, both in-house developed and others used in several stages of the process. Those ML models include pre-processing, feature engineering, and model optimization. Their model is based on the continuous fetching of transactions and events from supported marketplaces and collections, primarily via Etherscan since January 2021. Among the data types collected these include transactions, current prices, and NFT attributes. Their valuation for the NFT market is represented by two metrics they have developed, the NFTi Market Capitalization (NFTi Markt Cap) and Adjusted Floor Market Capitalization (AF Market Cap). Briefly explain what each of these evaluation metrics represent. Starting from the NFTi Market Cap, is their main metric deriving from their ML models and estimates the value of the NFT market. NFTi Market Cap is updated on a bi-hourly basis filtering low trading activity collections regarding the total volume traded. Following, AF Market Cap is the NFT market valuation derived from current floor prices [10,35]. So far, NFTValuations have achieved accurate valuations and predicted the price for specific collections providing great accuracy. A recent example is CloneX #16472 NFT (https://app.nftvaluations.com/tokens/ethereum/0x49cf6f5d44e70224e2e23fdcdd2c053f30ada28b/7807, accessed on 31 May 2024) which was sold for Ξ3 ($11,335), while the valuation of NFTValuations at the time of sale was Ξ2.775, demonstrating a 92.5% accuracy (https://x.com/nftvaluations/status/1796002262700949713?t=UOQfe4QQca07CGhLLMyKqA&s=19, accessed on 31 May 2024).

In conclusion, there is significant work out there focusing on addressing the valuation of NFTs challenges using ML techniques. We are confident that the synergy between AI and projects aiming to valuate the NFT market will increase significantly. This derives from the potential that AI can add to such efforts in analyzing complex and high volumes of NFT data. The authors of [36] explore the integration of social media trends into the recommendation system for NFTs. Through their study, they propose a recommendation system architecture that enhances NFT recommendations titled NFT-Trends-RecSys. In doing so, they collected data using OpenSea API and Twitter API, fetching 3872 randomly fetched NFT asset data and 677 randomly fetched trends, spanning 14 different date times. Despite the fact that they refer to the retrieval of open source data, their dataset is not available publicly, and thus, we cannot determine the exact period for the data collected other than those demonstrated in the diagrams in their research article [36]. As part of their proposed recommendation system, they have developed an algorithm to calculate the impact of a trend. In terms of software utilized through their work, the RAKE Vectorizer of the NTLK python Natural Language Processing (NLP) library was used to extract keywords from NFT asset data (re NFT name, description, collection name, etc.). In terms of the sentiment analysis model tested in their research, those were the (i) SpacyTextBlob, (ii) HappyTransformer, and (iii) Twitter-roBERTa-base for Sentiment Analysis. This research work [36], aimed to address the lack of recommendation systems for NFTs. Therefore, the synergy between ML techniques and NFTs in providing accurate recommendations can become an important tool for NFT stakeholders.

5. Discussion and Conclusions

In this article, we propose a framework for assessing research works positioned at the intersection of ML and NFTs. This framework consists of two dimensions and demonstrates a clear technological orientation. The first dimension deals with the specification of a series of ML-related parameters that help interested professionals gain a complete understanding. The second dimension maps the reviewed works to a series of domains that extend beyond the typical NFT valuation use case. It is important to clarify that the key contribution of this work is the aforementioned two-dimensional framework, which can be easily adapted to future states of the respective landscape, rather than an exhaustive coverage of the literature. The reviewed works stand as an indicative survey that adequately reflects the research spectrum of the ML and NFTs intersection, while simultaneously demonstrating the application of the proposed framework.

First, most of the reviewed research is observed to follow supervised approaches instead of unsupervised ones. This high-level differentiation can be attributed to various factors directly linked to the target computational task. The unsupervised paradigm seems to be closer to analytics-related tasks where less sophisticated models are utilized, in contrast to the supervised paradigm for which more complex (trainable) models are employed. Another interesting finding is that tasks such as NFT valuation and price prediction do not constitute the only active research area despite their thematic dominance. We have seen that several diverse areas have attracted the interest of the research community. In our opinion, this is quite encouraging as the monopoly of valuation and price prediction narrows research diversity. Regarding the input type, most of the research works deal with visual features, which are in many cases combined with textual features. This is expected as images stand as the typical modality of NFTs, which are augmented with textual descriptors (serving as metadata). As a result, regarding visual data, the input is highly structured, while for textual data, several structure variants can be encountered spanning from semi-structured to structured. Most approaches employ deep learning models, without excluding the presence of other approaches that rely on more traditional models. This can be attributed to the primary modality of the input, that is, visual features. Deep learning has been proven to be very effective in capturing the underlying patterns of this modality. In this context, another notable observation is the utilization of pre-trained models, which exhibit several advantages including rapid prototyping and lowering the barriers for smaller research teams. Regarding modeling, there is a tendency towards ‘singleton’ approaches, meaning the exploitation of single models instead of combining multiple (and different) models. This is related to the computational tasks of interest. In general, the fusion of models is used more frequently in classification-related tasks (as well as in regression). However, fusion is not so common in the case of generative models where the goal is to learn the distribution underlying the data of interest. Of course, in principle, this is possible; for example, one can fuse different feature spaces and then train a model. Such low-level (feature-level) fusion can be conducted when trying to build joint representations across different modalities. Regarding evaluation, there is a clear use of quantitative metrics. This is, of course, a positive characteristic as it enables the development of common benchmarks in the literature, which is particularly important for comparative analyses. Along the same lines, we have observed the use of known NFT collections. This availability facilitates the creation of datasets based on open and accessible data. This is consistent with the nature of NFTs and, in general, with the overall philosophy of blockchain transparency. Furthermore, we have noticed basic reporting regarding the software used, which deals with core ML models and related techniques. Of course, we should note that these ML models were developed independently of NFTs.

Next, we provide a series of remarks based on the above considerations. These remarks can be viewed from different angles, ranging from critical review to recommendations for future research in this very specific field. First, we acknowledge the importance of information completeness, which should be present in the respective publications. In the present work, we have proposed a series of parameters that provide a holistic understanding when properly addressed. We emphasize this aspect because it is not uncommon to encounter research works where critical information is missing. The strong presence of complex ML models raises the need for adequate computational infrastructure. This becomes of greater importance when deep learning architectures are utilized. In this context, the availability of pre-trained models offers a beneficial convenience, especially when complex and costly models (for example, large language models) are needed. However, this convenience may trigger some drawbacks such as the lack of transparency in the form of explainability and the lack of deep critical analysis, which requires substantial core expertise, for example, core models of computer vision and natural language processing. Such drawbacks are possible when the utilization of pre-trained models is limited to shallow use without fully understanding the nature of the underlying models. Furthermore, we clearly see a gap regarding task-specific (NFT-specific) evaluation metrics and benchmarks in general. The metrics that appear in the literature can serve as an excellent starting point; however, we argue that they should be extended towards the specific needs that characterize the NFT space. This remark can be coupled with similar efforts on the dataset front, that is, the development of datasets with additional annotations that can be used for NFT-specific tasks. Regarding software/tools, another gap can be identified—it would be very helpful to have a suite of tools that are by design tailored for the intersection of ML and NFTs. Such suites are not supposed to reinvent the wheel; instead, they can use existing models in a way that covers all major needs of this specialized intersection, e.g., from data annotation and feature extraction to model evaluation and model sustainability. Last but not least, we conclude with a special remark on the need for developing a multidisciplinary mindset. It is evident that different scientific fields are brought together in the present intersection, and this should be considered in all phases, spanning vertical feasibility studies up to further R&D phases.