Skip to Content
You are currently on the new version of our website. Access the old version .
  • Editor’s Choice
  • Review
  • Open Access

20 March 2026

46 Pages

Generative AI and the Foundation Model Era: A Comprehensive Review

,
,
,
,
and
1
Dipartimento Ingegneria dell’Informazione, University of Pisa, 56126 Pisa, Italy
2
Multidisciplinary Faculty of Nador, Mohammed Premier University, Oujda 60000, Morocco
3
School of Intelligent Engineering, Shandong Management University, Jinan 250357, China
*
Author to whom correspondence should be addressed.

Abstract

Generative artificial intelligence and foundation models have changed machine learning by allowing systems to produce readable text, realistic images, and other multimodal content with little direct input from a user. Foundation models are large neural networks trained on very large and varied datasets, and they form the core of many current generative AI (GenAI) systems. Their rapid development has led to major advances in areas like natural language processing, computer vision, multimodal learning, and robotics. Examples include GPT, LLaMA, and diffusion-based architectures, such as models often used for image generation. Systems such as Stable Diffusion show this shift by illustrating how AI can interpret information, draw basic inferences, and produce new outputs using more than one type of data. This review surveys common foundation model architectures and examines what they can do in generative tasks. It reviews Transformer, diffusion, and multimodal architectures, focusing on methods that support scaling and transfer across domains. The paper also reviews key approaches to pretraining and fine-tuning, including self-supervised learning, instruction tuning, and parameter-efficient adaptation, which support these systems’ ability to generalize across tasks. In addition to the technical details, this review discusses how GenAI is being used for text generation, image synthesis, robotics, and biomedical research. The study also notes continuing challenges, such as the high computing and energy demands of large models, ethical concerns about data bias and misinformation, and worries about privacy, reliability, and responsible use of AI in real settings. This review brings together ideas about model design, training methods, and social implications to point future research toward GenAI systems that are efficient, easy to interpret, and reliable, while supporting scientific progress and ethical responsibility.

1. Introduction

Research in artificial intelligence shifted with the emergence of large generative models, and this shift became clearer after the release of GPT-3 and GPT-4. These developments indicate a shift toward AI systems that can independently produce coherent text, realistic images, and multimodal outputs, reshaping machine learning research and applied work [1,2,3].
This shift is largely driven by foundation models, which are large, pretrained neural networks trained on extensive and diverse datasets. These models form the core of many current generative AI (GenAI) systems and differ in important ways from traditional models built for a single task. Foundation models are not built for a single task; they aim to generalize across many tasks and data types, which allows zero-shot and few-shot learning in different domains and modalities [4]. Because they scale well and can be adapted to many tasks, these models have shaped research in natural language processing, computer vision, robotics, and biomedical fields [2].
Foundation models typically learn from very large datasets using self-supervised or unsupervised objectives, which enables them to form representations that can transfer across tasks [5]. These representations can be adapted to downstream tasks with little extra data, which supports behaviors such as in-context learning, cross-modal reasoning, and creative content generation [6]. Earlier generations of machine learning models could not support these functions to the same extent.
GenAI and foundation models matter not only for new technical methods but also for how research, teaching, and decision making are organized across many fields. Their broad use in research and industry has sped up scientific work and product development, but it has also raised concerns about ethics, safety, bias, privacy, and environmental sustainability [7,8,9]. Widely used systems such as GPT-4, DALL·E, and diffusion-based image generators have drawn sustained attention in research and public discussion, which motivates an integrative survey of current work in the field [10,11].
This paper aims to provide an up-to-date, cross-domain, and methodologically grounded review of GenAI and foundation models, addressing both technical foundations and practical deployment considerations. Unlike prior surveys which focus on isolated domains or single architectural paradigms, this work integrates architectural analysis, training strategies, evaluation practices, and application-level insights within a unified analytical framework. The main contributions of this survey are summarized below:
1.
A unified and architecture-integrated analysis of Transformer-based, diffusion-based, and multimodal foundation models, supported by formal mathematical formulations and comparative structural discussion across paradigms.
2.
A systematic examination of large-scale pretraining, scaling laws, and fine-tuning methodologies—including self-supervised learning, instruction tuning, parameter-efficient adaptation, and alignment strategies—highlighting their interdependencies and cross-domain transferability.
3.
A cross-domain synthesis covering ten major application areas (natural language processing, computer vision, multimodal systems and robotics, biomedical and healthcare, digital twins, education, finance, smart cities, transportation, and virtual reality), enabling horizontal comparison rarely addressed in prior surveys.
4.
The introduction of a unified cross-domain classification framework that organizes GenAI research according to task type, data modality, model paradigm, and evaluation dimensions, facilitating structured comparison across otherwise disconnected fields.
5.
A critical methodological discussion of reproducibility, statistical comparability, dataset bias, sustainability, ethical risks, and the limitations of closed-source foundation models—addressing gaps often underexplored in earlier comprehensive reviews.
6.
A forward-looking perspective outlining research directions toward efficient, interpretable, multimodal, and responsibly governed foundation models, with attention to democratization and sustainable deployment.
This research study shows that organizations today fully adopt generative artificial intelligence technologies across all business areas. The paper presents a unified assessment method which enables researchers to compare GenAI technologies across different fields of study. The review presents a complete framework which shows how different systems approach three main aspects of their operations throughout their various applications. The approach uncovers research methods that universities share which researchers customize to address their own study objectives while crossing different academic fields. The manuscript establishes comparison dimensions which create a meta-analytical framework that enables researchers to understand GenAI studies better across different research areas. The rest of this paper is organized as follows (see Figure 1). Section 2 reviews earlier survey studies and summarizes the main application areas of generative artificial intelligence and LLMs. Section 3 describes the systematic review methodology and literature selection process. Section 4 introduces the core concepts and historical development of GenAI. Section 5 discusses the architectural foundations of Transformer, diffusion, and multimodal models. Section 6 examines pretraining strategies and scaling laws, while Section 7 presents fine-tuning methods and domain adaptation techniques. Section 8 provides an in-depth technical analysis of generative architectures. Section 9 reviews major application areas of GenAI. Section 10 introduces a unified classification framework and cross-field comparison dimensions. Section 11 discusses unresolved challenges and open issues, Section 12 outlines future research directions, and Section 13 closes the paper with a summary of the main findings and a discussion of responsible and ethical deployment of GenAI systems.
Figure 1. Survey structure.

2. Related Reviews

Recent review studies have analyzed the development and application of generative artificial intelligence and large language models (LLMs) in many fields. These surveys offer field-specific insight into new methods, common uses, and emerging challenges, but they are often limited to one discipline. Table 1 provides an overview of selected review studies, describing their main application areas, the methods they use, and their key contributions. The surveys were selected based on their close alignment with the scope and objectives of this review. Priority was given to studies addressing similar architectural paradigms, task categories, or domain-specific applications in order to position the present work within the existing literature and to clearly demonstrate its cross-domain significance. The selection is, therefore, relevance-driven and comparative in nature rather than exhaustive.
A comprehensive survey of ten core natural language processing tasks enabled by generative models is provided in [12]. This review groups prior work into three main approaches: pretraining with later fine-tuning, prompt-based learning, and treating NLP tasks as text generation. The results suggest improved outcomes in summarization, translation, reasoning, and open-ended text generation, while ongoing issues remain in evaluation consistency and in how well the methods hold up and transfer across different tasks. In computer vision, recent surveys of generative vision models focus on generative adversarial networks and diffusion-based architectures [13]. The survey proposes a structured taxonomy that covers image synthesis, enhancement, editing, and semantic understanding. It also examines the move from adversarial learning to diffusion-based generation and notes open issues such as computational cost, limited control over outputs, and bias in training datasets.
The application of generative and multimodal foundation models to robotics and human–robot interaction is systematically examined in [14]. This review examines how large-scale multimodal models combine language grounding, visual perception, and action modeling to enable instruction following, task planning, and robotic control. This section also examines simulation-based learning environments, safety issues, and the current limits of embodied AI systems. A detailed overview of GenAI applications in biomedical and healthcare settings is provided in [15]. The survey summarizes earlier studies on automated clinical documentation, diagnostic support, synthetic medical image generation, and the design of clinical decision support systems. This discussion reviews regulatory and ethical boundaries, along with practical limits on implementation, that may slow adoption in real-world settings. The text argues that systems used in high-stakes clinical settings should be reliable, interpretable, and supported by clinical validation before they are introduced into routine practice.
The role of GenAI in digital twin systems is reviewed with a focus on network simulation, synthetic data generation, and predictive optimization in [16]. This study assesses whether foundation models can raise simulation accuracy and support data-driven system management. It also examines limits related to scaling, interoperability across systems, and integration with cyberphysical infrastructure. A survey of GenAI applications in education reviews studies on personalized learning environments, adaptive assessment approaches, and tools that support collaborative learning [17]. These changes should be read in the context of the broader shift toward Education 5.0 and should include a discussion of emerging concerns related to academic integrity, learner dependency, and ethical governance.In [18], the authors survey how GenAI is used in finance, with examples such as fraud detection, risk assessment, financial forecasting, and systems that make decisions on their own. The review combines a methods-focused analysis with applied case studies and examines current issues in explainability, bias, data security, and regulatory compliance. In [19], the authors study GenAI in smart city settings, focusing on services for residents, better infrastructure performance, and support for daily operations. This survey examines how LLMs are applied in conversational interfaces and urban digital twins, and it considers issues related to governance, user experience, and data integration across departments. In [20], the authors report a systematic review and bibliometric analysis of applications of LLMs in transportation. Previous studies generally fall into three categories: traffic management, autonomous driving, and safety assessment. This review identifies remaining gaps, especially in real-time deployment, system scalability, model interpretability, and cybersecurity for intelligent transportation systems. A survey of prior work on integrating GenAI with virtual reality systems is provided in [21]. This review surveys applications in education, healthcare, entertainment, and industrial training, focusing on how generative models support adaptive settings, immersive interactions, and dynamic content generation. It also addresses issues of computational efficiency, model transparency, and ethics in generative virtual environments.
While existing reviews provide in-depth analyses within individual domains, they largely adopt isolated perspectives that limit cross-domain comparison. By contrast, this study presents a cross-domain survey covering ten application areas: natural language processing, computer vision, multimodal systems and robotics, biomedical and healthcare, digital twins, education, finance, smart cities, transportation, and virtual reality. This survey combines findings from these areas to compare methods, common applications, evaluation approaches, and recurring problems, offering a single view of the changing field of GenAI.
Table 1. Summary of related reviews on generative artificial intelligence and LLMs across application domains.

3. Systematic Review Methodology

In order to maintain complete transparency, accurate repeatability, and high degree of methodological rigor, the present review utilizes a formalized structured systematic review protocol consistent with PRISMA guidelines. A broad search of the literature was conducted across multiple prominent academic databases including IEEE Xplore, ACM Digital Library, Scopus, Web of Science, and arXiv from 2017 through to 2025, using Boolean search logic to combine keywords, including “Generative AI”, “Foundation Models”, “Large Language Models”, “Diffusion Models”, “Multimodal Learning”, “Pretraining”, and “Evaluation”. The studies selected for inclusion in the present review were required to be peer-reviewed or published in high-impact-tier preprint journals, to demonstrate architectural advancements, new training strategies, new evaluation frameworks, and/or applications in the literature from the areas identified to be within scope of this research project. In total, the review screening process resulted in a structured distribution of studies across ten primary application domains. Natural language processing (NLP) represents the largest share, with 30 studies, followed by biomedical and healthcare (13), multimodal and robotics (8), digital twins (7), and computer vision (6). Emerging application areas such as smart cities and transportation are represented by 3 studies each, while finance includes 2 studies, and education and virtual reality are represented by 1 study each, as shown in Figure 2. This distribution highlights the current research concentration in language-centric foundation models while also reflecting the expanding diversification of GenAI applications across interdisciplinary domains.
Figure 2. Distribution of publications across application domains included in this survey.
Exclusion criteria for the screening process of the studies included elimination of duplicate entries and articles that were not technical opinion pieces about the subject matter. The methodology section of the screening process included identification of all studies that matched inclusion/exclusion criteria by assessing the date of database query (i.e., date from which a study would have been able to be included), eliminating duplicate entries from the screening process, filtering titles and abstracts, and performing full-text eligibility assessments for each article remaining after filtering on title and abstract.
As a result, the current survey provides a systematic review, as opposed to simply a narrative, and a detailed description of each of the studies will be able to be independently verified by other researchers. This research study demonstrates its main findings through different observation methods which characterize the expanding use of generative artificial intelligence across different fields.
The framework developed in this paper provides researchers with a standardized method to evaluate GenAI systems across multiple domains. The review presents a comprehensive overview of multiple application domains by displaying their architectural designs, training methodologies, evaluation methods and operational restrictions through a shared analytical framework. The method shows how research methods become shared between different fields while researchers adjust their work to meet specific needs of different domains. The comparison axes established in this research work provide researchers with a meta-analytical framework which supports both GenAI research and testing of research findings through better understanding and clearer research patterns.

4. Overview of Generative AI

GenAI is a branch of artificial intelligence that produces new content, such as text, images, and other mixed-format outputs. Its main aim is to generate coherent material rather than only analyze, label, or sort existing data. This section reviews GenAI and outlines its development from early probabilistic approaches to current large-scale foundation models. This paper defines the key concepts, traces the field’s development through major model architectures, and reviews the current set of widely used models and their applications. This background provides the context needed for the architectural, methodological, and ethical discussions in the next sections.

4.1. Definition and Concepts

Generative artificial intelligence (AI) represents a paradigm shift within machine learning, defined by systems capable of synthesizing novel, plausible data, including text, images, audio, code, and multimodal content, that reflect the underlying distribution of their training data. Unlike discriminative models, which focus on learning the conditional probability P ( y ∣ x ) to map inputs to labels or categories, generative models learn the joint probability distribution P ( x ) of the data itself, enabling them to generate new samples [22]. This distinction is not only technical; it also matters in practice. Discriminative models are commonly used for classification and regression because they learn to map inputs to target outputs. Generative models learn the data distribution, which supports tasks such as producing new content, augmenting training data, and simulating complex settings. The recent rise of GenAI is closely tied to the development of foundation models, which are large models pretrained on broad, often multimodal, datasets and can later be adapted for many downstream tasks [22]. These models are trained with self-supervised methods on very large text collections, which helps them learn patterns and internal representations that transfer across different tasks and fields. These models can generate new outputs and are not restricted to copying existing material. They show emergent properties, including coherence within context, adaptation to different styles, and reasoning across modalities [23]. The move from narrow, task-bound generators to general generative systems points to a basic change in AI’s role, shifting from analytic tools to collaborators in producing new work.

4.2. Historical Evolution

The evolution of GenAI has been driven by successive architectural paradigms that progressively improved generation quality, stability, and scalability, enabled by advances in algorithms, data availability, and computational resources.
  • Generative adversarial networks (GANs): Introduced by Goodfellow [24], GANs enable high-quality images through adversarial training, but training often becomes unstable and can lead to mode collapse, which limits how well the approach scales to diverse, multimodal data.
  • Variational autoencoders (VAEs): VAEs offer a stable probabilistic approach to learning latent representations, but their samples are often less sharp and detailed than those produced by GANs, which limits their use in settings that required high-fidelity image generation [25].
  • Autoregressive large language models (LLMs): Transformer-based architectures [26] have shown that training large models for next-token prediction at scale can yield strong generative outputs and improved reasoning, which has made scalability a central design principle.
  • Diffusion models: Diffusion models generate data by starting from random noise and repeatedly removing it [27]. This process is often stable, yields diverse samples, and supports controllable image synthesis, which is why it is widely used in text-to-image generation [28].
  • Multimodal foundation models: Recent multimodal models combine text and images using contrastive learning and cross-attention [29]. This approach supports tasks ranging from zero-shot image recognition to few-shot multimodal reasoning, as shown in systems such as Flamingo and GPT-4V.
Research has moved from adversarial and variational approaches to scalable autoregressive and diffusion models, and it now focuses on unified multimodal foundation models that reflect current best performance in the field. As illustrated in Figure 3, the historical evolution of GenAI highlights a clear progression from probabilistic and adversarial modeling paradigms toward large-scale Transformer-based systems, diffusion-based generators, and, ultimately, integrated multimodal foundation models.
Figure 3. Timeline summarizing the historical evolution of GenAI architectures from variational autoencoders to multimodal foundation models.

4.3. Current Landscape

The current GenAI landscape is dominated by large-scale foundation models, defined by their scalability, versatility, and increasingly multimodal capabilities. Although models are often grouped by their main output type, the lines between these categories are becoming less distinct over time.
  • Text-only foundation models: Recent LLMs, including GPT-4, LLaMA 3 [30], Claude 3 [31], and Gemini Pro [32], represent current leading performance in reasoning, instruction adherence, and long-context processing, and they underpin many present-day chat-based assistants and code generation tools.
  • Image generation models: Diffusion-based models such as Stable Diffusion, DALL-E 3, and Midjourney are widely used for text-to-image generation. Recent research reports that they often produce images with high visual quality and allow detailed steering through prompts, which supports both creative work and commercial design.
  • Video generation models: Recent systems such as Sora [33], Runway Gen-2, and Pika Labs support short text-to-video generation, but limitations remain in keeping motion consistent over time, matching real-world physics, and producing coherent videos over longer durations.
  • Multimodal and embodied models: Models such as GPT-4V and Gemini Ultra combine language and visual input to support tasks like answering questions about images and interpreting documents. Current research also studies embodied agents that pair perception with action in physical settings.
Autoregressive Transformer-based models have dominated large-scale language modeling from 2020 to 2025. Trained via next-token prediction with self-attention, models such as GPT-4, LLaMA 3, and Gemini exhibit strong reasoning, dialogue, and code synthesis capabilities, supported by in-context learning and predictable scaling behavior [34]. Sequential generation can increase latency, raise the risk of hallucinations, increase inference costs, and create alignment issues, especially in safety-critical settings [34]. Even with these constraints, autoregressive Transformers continue to serve as the main architecture behind most language systems in practical use.
Diffusion models such as Stable Diffusion, DALL-E 3, and Sora generate images and videos by repeatedly denoising a sample over many steps, which supports stable training and high-quality outputs [35]. Diffusion models are widely used for text-to-image generation, image editing, and super-resolution because they offer good control over composition and reduce mode collapse, even though they often require slow inference, high memory use, and still struggle with temporal consistency [35].
Multimodal foundation models such as GPT-4V, Flamingo, and Gemini 1.5 integrate text, vision, audio, and video through cross-attention or fusion mechanisms, enabling strong zero-shot and few-shot cross-modal transfer [36]. Although these models allow more natural interaction and embodied reasoning, they still depend on very large datasets and high compute, raise open questions about how to evaluate them, and may carry forward biases across modalities [36].
Generative adversarial networks, represented by StyleGAN 3 and BigGAN-deep, remain effective for high-resolution image synthesis with fast inference [37]. In practice, training instability, limited controllability, and weak scalability to multimodal data have kept GANs largely confined to niche uses such as portrait generation and artistic style transfer [37].
In addition to the models discussed above, recent developments have introduced GPT-5 as the next-generation large-scale foundation model in the GPT series. GPT-5 extends the autoregressive Transformer paradigm with enhanced multimodal integration, improved long-context handling, and strengthened reasoning capabilities. While official technical documentation remains limited due to its proprietary nature, early reports indicate improved performance in complex reasoning, multimodal understanding, and instruction-following compared to GPT-4-class systems. As with other closed-source models, full architectural details, training datasets, and alignment methodologies are not publicly disclosed, which constrains independent empirical verification. Nevertheless, GPT-5 represents a continuation of the scaling and integration trends that characterize the foundation model era.
Table 2 provides a comparative summary of the dominant GenAI paradigms discussed in this subsection.
Table 2. Strategic comparison of dominant GenAI paradigms (2020–2026).

5. Foundation Model Architectures

Foundation models are characterized by architectural paradigms that enable generalization and cross-domain reasoning. Modern GenAI primarily relies on Transformer, diffusion-based, and multimodal architectures, which together define the core design space of contemporary large-scale generative models.
To ensure conceptual rigor and facilitate technical clarity, this section presents formal mathematical formulations of the core mechanisms underlying Transformer and diffusion architectures. These equations summarize canonical definitions widely adopted in the literature and are included to support reproducibility at the architectural level rather than to introduce novel derivations. While later sections—particularly those covering applications—emphasize cross-domain synthesis and comparative analysis, architectural sections require greater technical specificity to clearly articulate the foundational computational principles that underpin GenAI systems.

5.1. Transformer-Based Architectures

The Transformer architecture represents a significant paradigm shift in deep learning by discarding the use of recurrence and convolution in favor of fully attention-based mechanisms for modeling sequential data. In contrast to recurrent neural networks (RNNs) and their gated extensions, such as long short-term memory (LSTM) and gated recurrent unit (GRU), which process input tokens sequentially, the Transformer enables parallel processing of all sequence elements. This design substantially enhances training efficiency and scalability. Central to the architecture is the self-attention mechanism, which enables each token to dynamically attend to all other tokens in the sequence, thereby facilitating the modeling of both local and long-range dependencies within the data.

5.1.1. Self-Attention Mechanism

Given an input sequence represented as X ∈ R n × d , the Transformer projects it into three learned subspaces—queries (Q), keys (K), and values (V)—through corresponding weight matrices W Q , W K , and W V .
Q = X W Q , K = X W K , V = X W V .
The attention mechanism is formulated through the scaled dot-product operation:
Attention ( Q , K , V ) = softmax Q K ⊤ d k V ,
where d k represents the dimensionality of the key vectors used for normalization. The softmax function converts the resulting similarity scores into a probability distribution, allowing the model to assign greater weight to tokens that are more contextually relevant for a given query.
Multihead Self-Attention
To improve the model’s ability to capture complex relationships, the Transformer generalizes self-attention into several parallel attention heads, where each head learns unique patterns of interaction among tokens. The multihead formulation is given by
MultiHead ( Q , K , V ) = Concat ( head 1 , … , head h ) W O ,
with each individual head defined as follows:
head i = Attention ( Q W Q ( i ) , K W K ( i ) , V W V ( i ) ) ,
where W O is the output projection matrix. This multihead design enables the model to simultaneously capture information from different representational subspaces, enhancing its ability to interpret both semantic meaning and syntactic structure.

5.1.2. Encoder–Decoder Paradigm

The Transformer framework is commonly organized around an encoder–decoder architecture. In this structure, the encoder processes the input sequence and maps it into a high-dimensional latent representation, while the decoder generates the output sequence in an autoregressive manner. Depending on the model’s design, three main configurations are typically distinguished:
  • Encoder-only models: Primarily optimized for understanding and classification tasks (e.g., BERT), these models rely on bidirectional context encoding to capture semantic relationships within the input.
  • Decoder-only models: Intended for generative or autoregressive applications such as text completion (e.g., the GPT family), these architectures predict the next token sequentially based on preceding context.
  • Encoder–decoder models: Applied to sequence-to-sequence problems like translation and summarization (e.g., T5 and BART), these models integrate the strengths of both components to align input and output representations.

5.1.3. Scaling Innovations in Transformer Models

Recent research has shown that the performance of Transformer models follows a predictable trend as model size, dataset scale, and computational capacity increase. This relationship is commonly characterized by the following scaling law:
L ( N , D , C ) ∝ N − α N + D − α D + C − α C ,
where L denotes the model loss, N represents the number of parameters, D the dataset size, and C the available compute budget. Large-scale Transformer architectures such as GPT-3, PaLM, and LLaMA have capitalized on these scaling dynamics by employing hundreds of billions of parameters. To mitigate the resulting computational overhead, these systems adopt architectural refinements like sparse activation schemes and mixture-of-experts (MoE) layers, which enable selective computation and greater efficiency at scale. Collectively, these advances have solidified the Transformer’s role as the cornerstone of contemporary GenAI, supporting diverse applications across natural language processing, computer vision, and multimodal reasoning.

5.2. Diffusion-Based Generative Models

Diffusion-based generative models have become a widely used approach in GenAI, with strong results on many synthesis tasks such as creating images, videos, and audio. Autoregressive and adversarial methods usually generate samples by mapping random noise straight into the data space, while diffusion models learn the reverse of a step-by-step noising process. In this approach, unstructured noise is gradually refined into coherent, high-quality outputs through repeated denoising steps. These models build on non-equilibrium thermodynamics and probabilistic modeling, where data generation is treated as a parameterized diffusion denoising process that the system learns during training.

5.2.1. The Forward and Reverse Diffusion Processes

A diffusion model defines a Markov chain over latent variables { x t } t = 0 T , where an initial data sample x 0 is progressively corrupted into pure noise x T through the application of Gaussian noise at each timestep. The forward (or diffusion) process is modeled as follows:
q ( x t ∣ x t − 1 ) = N ( x t ; 1 − β t x t − 1 , β t I ) ,
where β t ∈ ( 0 , 1 ) represents the variance schedule at time t, determining how quickly information is lost during the noise injection process. This forward process also admits a closed-form marginal distribution:
q ( x t ∣ x 0 ) = N ( x t ; α ¯ t x 0 , ( 1 − α ¯ t ) I ) ,
with α ¯ t = ∏ s = 1 t ( 1 − β s ) capturing the accumulated effect of the noise schedule up to timestep t.
The reverse process seeks to recover the original sample x 0 by iteratively denoising the noisy observation x T . This backward trajectory is learned by a neural network that parameterizes the conditional distribution:
p θ ( x t − 1 ∣ x t ) = N ( x t − 1 ; μ θ ( x t , t ) , Σ θ ( x t , t ) ) ,
where μ θ and Σ θ denote the predicted mean and variance at each step. Training involves minimizing the variational lower bound (VLB) on the negative log-likelihood of the data:
L VLB = E q − log p θ ( x 0 : T ) q ( x 1 : T ∣ x 0 ) .
In practice, this objective is often simplified to a denoising loss, where the model is trained to predict the noise ϵ that was added during the forward process:
L simple = E t , x 0 , ϵ ϵ − ϵ θ ( α ¯ t x 0 + 1 − α ¯ t ϵ , t ) 2 .

5.2.2. Sampling and Generation Procedure

The sampling procedure in diffusion models unfolds in reverse, beginning with a sample of pure Gaussian noise, x T ∼ N ( 0 , I ) . The trained denoising network is then applied iteratively to reconstruct the original data distribution:
x t − 1 = μ θ ( x t , t ) + Σ θ 1 / 2 ( x t , t ) ϵ , ϵ ∼ N ( 0 , I ) .
Through this iterative reverse diffusion process, unstructured noise is gradually transformed into coherent and semantically meaningful samples. Several prominent variants—such as denoising diffusion probabilistic models (DDPMs), denoising diffusion implicit models (DDIMs), and more recent stochastic samplers—introduce both deterministic and accelerated sampling strategies. These innovations seek to enhance the efficiency and output fidelity of diffusion-based generation, thereby improving their viability for large-scale and real-time applications.

5.2.3. Architectural Advancements and Applications

Recent diffusion models such as latent diffusion models (LDMs) generate images in a lower-dimensional latent space learned by an autoencoder, which lowers compute cost while preserving fine detail in the results. Text conditioning through cross-attention, as used in DALL·E 3, Imagen, and Stable Diffusion, lets researchers steer the generative process with natural language prompts and achieve detailed control over the outputs. These models are used in many research and applied settings, such as photorealistic image and video generation, molecular and protein structure generation, audio improvement, and three-dimensional scene reconstruction. Because they are interpretable, produce stable samples, and allow control over the generation process, diffusion models are now a core approach in GenAI and serve as a practical counterpart to Transformer-based and energy-based generative methods.

5.3. Multimodal Architectures

Multimodal architectures are an important step in the design of foundation models because they learn shared representations from different data types, such as text, images, audio, and video. By combining and aligning data from several sensory channels, these models aim to approximate human perception, which allows context-based reasoning, meaning that is grounded in observed inputs, and the creation of content across different modalities. This approach supports many applications that need data from different sources to be handled together so that artificial intelligence systems can remain reliable and adaptable.

5.3.1. Cross-Attention and Multimodal Fusion

At the heart of multimodal architectures lies the cross-attention mechanism, which enables one modality to refine its representation based on contextual cues from another. Let T and V represent the token embeddings corresponding to textual and visual inputs, respectively. The cross-attention operation captures inter-modal dependencies as follows:
CrossAttn ( T , V ) = softmax ( T W Q ) ( V W K ) ⊤ d k V W V ,
where W Q , W K , and W V are learnable projection matrices specific to each modality. This mechanism allows features from one stream—such as text—to be aligned with and enriched by complementary visual information, producing joint embeddings that integrate both semantic meaning and spatial structure. Additional techniques, including late fusion, token-level alignment, and hierarchical attention, further expand the capacity of these models to effectively combine information across modalities.

5.3.2. Representative Multimodal Foundation Models

Early breakthroughs in multimodal learning were driven by models like CLIP (Contrastive Language–Image Pretraining), which established a foundational approach by jointly training a vision encoder and a text encoder to align image–text pairs within a shared embedding space. This alignment is guided by a contrastive loss function, defined as follows:
L CLIP = − 1 N ∑ i = 1 N log exp ( ⟨ t i , v i ⟩ / τ ) ∑ j = 1 N exp ( ⟨ t i , v j ⟩ / τ ) ,
where t i and v i denote the normalized text and visual embeddings, and τ is a temperature parameter that scales the similarity scores.
Building upon this framework, subsequent models such as Flamingo and BLIP-2 introduced cross-modal Transformer architectures that incorporate visual tokens into LLMs via adapter layers, facilitating bidirectional reasoning and few-shot learning across modalities. More recent systems—such as GPT-4V, Gemini 1.5, and Kosmos-2—extend this approach further by integrating textual, visual, and auditory inputs within unified architectures. These models leverage shared token embeddings and multimodal encoders to support generalized perception and generation across diverse tasks.

5.3.3. Applications and Emerging Directions

Multimodal architectures support many tasks in research and practice, such as image captioning, visual question answering (VQA), visual grounding, and generating images or videos from text. In robotics, multimodal Transformers can support embodied agents in interpreting visual scenes and carrying out natural language instructions. In healthcare, they can combine clinical text with medical images to support diagnostic reasoning. Current research is moving toward universal multimodality, where vision, language, audio, tactile input, and symbolic representations are brought together in a single integrated system. This shift is largely due to recent work on cross-modal pretraining, joint embedding methods, and diffusion-based architectures for feature fusion. Work that combines multimodal learning with generative models is expected to support general-purpose foundation models that can interpret the world across data types, respond to context in interaction, and make independent choices using information from different sensory inputs. A more thorough technical comparison will illustrate a clarifying understanding of the computation-efficiency and functionality differences between Transformer architectures, diffusion architectures, and multimodal architectures. For example, Transformer-based self-attention mechanisms exhibit quadratic time complexity O(n2) with respect to a sequence length, which creates a bottleneck on long-context scalability and increases the memory overhead for large-scale applications. Autoregressive models such as GPT-4 and LLaMA 3 generate compelling reasoning and coherent generation outputs but demonstrate inference latency in addition to the risk of hallucination. While diffusion-based architectures such as Stable Diffusion provide stable training dynamics and produce very high-fidelity images, they do so at the expense of cost associated with multiple iterations of denoising needed to reach the level of detail in the resulting images. In addition, multimodal architectures will have added computational overhead as a result of cross-attention fusion and heterogeneous token alignment. In addition, the training objectives applied, the scale of the dataset used, and the compute budgets applied across different architectures significantly affect model performance; there is confirmed evidence supporting empirical scaling laws for model performance that link the size of the model (on a parameter basis), the amount of training data, and the amount of loss reduction through model training. Establishing a structured understanding of these architectural differences will provide a better foundation to determine applicability and inform future research for the purposes of developing more efficient models.

6. Pretraining Strategies

Pretraining is a core stage in current foundation models, giving them broad knowledge that supports generalization and later adaptation to many downstream tasks. It refers to large-scale self-supervised learning methods, data collection and filtering pipelines, and planned scaling approaches that give GenAI systems broad background knowledge and reusable representations. In academic research, pretraining methods are central to reproducible experiments and to faster adaptation across language, vision, and multimodal tasks. This supports wider access to GenAI in both research and teaching settings.

6.1. Self-Supervised Learning (SSL)

SSL is now the main approach used to pretrain foundation models because it can learn useful representations from data without relying on human-labeled annotations. Self-supervised learning methods use patterns already present in the data to create pseudo-labels, so a model can train by predicting masked, hidden, or missing parts of an input sequence.
In natural language processing (NLP), common SSL objectives include masked language modeling (MLM) and next-token prediction (NTP). MLM, as implemented in models such as BERT and RoBERTa, involves randomly masking tokens within a sentence and training the model to recover the original tokens using information from the surrounding bidirectional context:
L MLM = − ∑ i ∈ M log P θ ( x i ∣ x ∖ i ) ,
where M represents the set of masked token indices, and x ∖ i refers to the sequence of unmasked tokens used for prediction.
In contrast, autoregressive models such as GPT are trained using a next-token prediction (NTP) objective, where the model learns to predict each word based on the preceding context:
L NTP = − ∑ t = 1 T log P θ ( x t ∣ x < t ) ,
a formulation that supports strong generative performance and captures temporal dependencies within the input sequence.
Self-supervised learning techniques have also been successfully adapted beyond textual data, particularly in computer vision and multimodal representation learning. In visual pretraining, contrastive learning objectives—used in methods such as SimCLR and CLIP—are designed to align feature representations across different augmented views or modalities. These approaches train the model to bring together positive sample pairs (e.g., different views of the same image or paired image–text inputs) while distinguishing them from unrelated negative examples in the embedding space. A typical contrastive loss is defined as follows:
L contrastive = − log exp ( sim ( z i , z j ) / τ ) ∑ k = 1 N exp ( sim ( z i , z k ) / τ ) ,
where sim ( · ) denotes cosine similarity, τ is a temperature scaling parameter, and z i , z j are embeddings corresponding to a positive sample pair.
In generative methods such as diffusion models and autoencoders, self-supervised objectives are used to learn the data distribution by training the model to recover the original input from a noisy or otherwise corrupted version. Reconstruction-based strategies are central to many current GenAI systems because they help models learn general representations that can transfer across different data types and support later tasks.

6.2. Dataset Curation and Scaling Laws

The effectiveness of pretraining is critically dependent on the quality, diversity, and representativeness of the underlying datasets. Foundation models are trained on very large, mixed datasets that include scientific articles, web pages, images, and text in many languages so that the model covers many topics, treats groups more evenly, and can perform well on new tasks. In research settings, curating a dataset usually follows three main steps. In the first stage, large datasets are gathered and screened, including The Pile, LAION-5B, and Common Crawl. The screening step removes duplicate records, explicit material, and low-quality samples to keep the dataset consistent and suitable for training. The second stage focuses on normalization and tokenization: inputs are standardized, and a method such as byte-pair encoding (BPE) or SentencePiece is applied to produce a compact, stable vocabulary. In the final stage, sampling probabilities are tuned through balancing and weighting so that languages, domains, and modalities are represented more evenly, which reduces bias and supports better generalization.
Empirical research has consistently shown that model performance scales predictably with increases in model parameters, dataset size, and computational resources. As expressed by the general scaling relationship in Equation (5), training loss decreases as a power-law function of parameter count, data volume, and compute allocation.
In the context of pretraining, this relationship emphasizes the trade-offs between expanding dataset size, increasing model capacity, and optimizing compute efficiency. Understanding these scaling dynamics is essential for designing sustainable and cost-effective large-scale training pipelines in GenAI.
Within the domain of GenAI, a strong grasp of scaling behavior enables researchers to design foundation models that grow in capability while maintaining stable performance trends. In turn, these insights support more sustainable and reproducible practices in the development of large-scale generative systems.

6.3. Model Scaling, Efficiency, and Adaptation

As foundation models increase in size and complexity, researchers are placing more emphasis on lowering compute costs and on methods that allow these systems to be adapted to new tasks and settings with less effort. Many current training pipelines combine mixed-precision training, gradient checkpointing, and adaptive optimizers such as AdamW and Lion to reduce memory use, lower energy use, and limit numerical instability. Distributed frameworks such as DeepSpeed and Megatron-LM enable large-scale parallel training by dividing model computation and data across multiple GPUs within a cluster. In sparse modular models such as mixture-of-experts (MoE), the routing step sends each input to only a small subset of the model’s parameters instead of all parameters. This reduces the computation required for each example. Studies of models such as Switch Transformer and GLaM suggest that sending each input through only a selected subset of parameters can increase overall capacity without a similar increase in computation, which allows a practical trade-off between accuracy and inference time.
Improving model adaptability has become a key focus in recent work, particularly through parameter-efficient fine-tuning (PEFT) techniques such as LoRA, prefix-tuning, and adapter-based approaches. These methods aim to reduce the computational cost of fine-tuning by updating only a small subset of the model’s parameters, thereby retaining most of the pretrained knowledge. In the case of LoRA, parameter updates are expressed as a low-rank decomposition:
Δ W = A B ⊤ , where A , B ∈ R d × r , r ≪ d ,
allowing efficient adaptation to new domains with minimal computational overhead.
In academic research, sustainability and accessibility may be improved by using knowledge distillation, multitask and curriculum learning, and synthetic data augmentation based on generative models such as GPT and Stable Diffusion. Open-source projects such as HuggingFace, EleutherAI, and BigScience support reproducible research by giving researchers public access to large foundation models and the methods used to develop them, which improves transparency and access. Recent studies on improving training efficiency, using modular model structures, and adapting models with fewer trainable parameters are shaping how the next generation of foundation models is being developed. These guidelines aim to keep accuracy high while reducing costs and allowing research and industry work to scale, with careful attention to energy use and long-term maintenance.

6.4. Toward Sustainable and Adaptive Pretraining

Recent work on GenAI is increasingly examining pretraining approaches that can be maintained over time and adapted across settings, with the aim of keeping model performance strong while reducing energy demand and keeping access open to many users. Recent studies often combine self-supervised learning, synthetic data generation, and reinforcement learning from human feedback (RLHF) to align model outputs with ethical, social, and environmental aims. In academic research, these methods make it possible to create generative systems for a specific field with modest computing resources, while supporting open and reproducible scientific work. In the next phase of foundation model research, work is likely to focus on efficient architectures, adaptive approaches to fine-tuning, and datasets curated with clear ethical standards. These systems should be assessed not only on performance and how widely they are used, but also on how clear and inclusive they are, as well as their impact on the environment.

7. Fine-Tuning and Domain Adaptation

Fine-tuning and domain adaptation are important steps when applying pretrained foundation models to specialized tasks, because they can improve task performance while preserving broader generalization. Recent studies often combine instruction tuning, few-shot learning, and parameter-efficient methods to improve accuracy while reducing computational demands and making deployment more feasible across a wide range of applications.

7.1. Instruction Tuning

Instruction tuning is now a central method for aligning LLMs and multimodal systems with human preferences and with directions tied to specific tasks. Compared with standard supervised fine-tuning, instruction tuning trains models to respond to natural language instructions that are written in many different task formats. This method supports generalization across many tasks by training on instruction–response pairs that represent user intent, ethical constraints, and the surrounding context.
Instruction-tuning datasets—such as FLAN, Super-NaturalInstructions, and Dolly—provide broad benchmarks that cover a diverse range of linguistic tasks, reasoning challenges, and domain-specific scenarios. The fine-tuning process is typically formulated as a conditional text generation objective:
L instr = − ∑ t = 1 T log P θ ( y t ∣ y < t , x , I ) ,
where x denotes the input prompt, I is the corresponding instruction, and y t represents the target token at timestep t. Through this objective, the model learns not only how to perform tasks but also how to align its outputs with human-like reasoning and communication styles.
When paired with reinforcement learning from human feedback (RLHF), instruction tuning has shown considerable success in improving model safety, coherence, and factual reliability. This combined approach underpins the development of aligned generative systems such as ChatGPT, InstructGPT, and Gemini, which are designed to respond effectively to open-ended prompts while maintaining ethical behavior and contextual relevance.

7.2. Few-Shot and Zero-Shot Learning

Few-shot and zero-shot learning aim to support model generalization when a new task must be handled with very little, or no, task-specific training data. Instead of full retraining, models rely on representations learned during pretraining to carry out new tasks through prompting or small, task-specific updates. In few-shot learning, a model is given only a small number of labeled examples, often placed directly in the prompt, so it can infer the task from those examples at run time. In zero-shot learning, the model relies only on written instructions or contextual cues, instead of learning from labeled examples.
Prompt engineering plays a crucial role in guiding model behavior, with thoughtfully designed input templates shaping outputs toward desired objectives. To further improve adaptability, parameter-efficient fine-tuning techniques—such as adapter layers and Low-Rank Adaptation (LoRA)—enable targeted updates to a small subset of model parameters, while leaving the pretrained backbone intact. In the case of LoRA, weight updates are captured through a low-rank decomposition:
Δ W = A B ⊤ , where A , B ∈ R d × r , r ≪ d ,
allowing the model to efficiently adapt to new tasks without requiring full retraining.
These techniques have seen widespread adoption in GenAI research and academic applications, where they support efficient model customization across a range of domains—including scientific text summarization, multilingual translation, and image–text reasoning. By significantly lowering computational demands while maintaining strong performance, they provide a scalable and practical pathway for domain-specific fine-tuning.

7.3. Task-Specific Fine-Tuning

Task-specific fine-tuning is often the last step in adapting a foundation model, where a pretrained architecture is trained again on domain data to meet the needs of areas such as computer vision, robotics, biomedicine, and industrial automation. In practice, this usually means retraining the model on data from the target field, using a chosen loss function and regularization methods that keep its ability to generalize while improving accuracy on the specific task.
In computer vision research, models such as Vision Transformers (ViT) and CLIP are often fine-tuned for downstream tasks like object detection, image captioning, and visual grounding, using benchmark datasets such as COCO and ImageNet. In robotics, fine-tuned multimodal Transformer models combine sensor inputs with natural language commands to support manipulation and motion planning that fit the current setting. In biomedical research, domain adaptation often involves retraining models on carefully selected datasets such as medical images, genomic sequences, or clinical records. This approach can improve diagnostic prediction and support drug discovery. In industrial and engineering settings, fine-tuned generative models are used for predictive maintenance, anomaly detection, and improving process control, which suggests that foundation models can support practical work in high-impact sectors.
In task-specific fine-tuning, it is common to combine supervised training with self-supervised updates and reinforcement-based feedback so that the model can adjust its behavior to complex conditions in real-world settings. Taken together, these strategies reduce the distance between general-purpose foundation models and domain-specific knowledge, making fine-tuning and adaptation central steps in current GenAI workflows.

8. In-Depth Technical Analysis of Generative Architectures

This section intentionally includes formal mathematical formulations to provide technical clarity and conceptual rigor regarding the core mechanisms underlying generative architectures. While earlier sections emphasize cross-domain synthesis and comparative analysis, the present section focuses specifically on foundational computational principles. The inclusion of equations is, therefore, deliberate, aiming to ensure architectural transparency, reproducibility at the conceptual level, and consistency with canonical definitions in the literature rather than to introduce novel derivations.
While the previous sections introduced core architectures and training strategies, this section provides a deeper technical examination of generative model mechanisms, computational complexity, training objectives, and practical applicability.

8.1. Transformer-Based Models

Transformer-based models, particularly LLMs, rely on the self-attention mechanism to model long-range dependencies.
Given an input sequence X = ( x 1 , … , x n ) , the self-attention operation is defined as follows:
Attention ( Q , K , V ) = softmax Q K T d k V
where Q, K, and V represent query, key, and value matrices.
Advantages:
  • Strong contextual modeling capability;
  • Scalability with large datasets;
  • Transferability across domains.
Limitations:
  • Quadratic computational complexity O ( n 2 ) with respect to sequence length;
  • High memory consumption;
  • Hallucination and factual inconsistency.
Training objective: Most LLMs optimize next-token prediction:
L = − ∑ t = 1 T log P ( x t | x < t )

8.2. Diffusion Models

Diffusion models generate data by progressively denoising a random noise distribution.
The forward process gradually adds noise:
q ( x t | x t − 1 ) = N ( x t ; 1 − β t x t − 1 , β t I )
The model learns to reverse this process.
Advantages:
  • High-quality image synthesis;
  • Stable training compared to GANs.
Limitations:
  • Slow sampling process;
  • High computational requirements.

8.3. GAN-Based Architectures

Generative adversarial networks consist of a generator G and discriminator D trained in a minimax game:
min G max D E x ∼ p d a t a [ log D ( x ) ] + E z ∼ p z [ log ( 1 − D ( G ( z ) ) ) ]
Advantages:
  • Fast inference after training;
  • High realism in image synthesis.
Limitations:
  • Training instability;
  • Mode collapse.

8.4. Training Data Requirements and Scaling Laws

Generative models exhibit scaling behavior where performance improves predictably with model size N, dataset size D, and compute C.
Empirical scaling laws suggest the following:
L ( N , D , C ) ∝ N − α + D − β
Large-scale pretraining requires the following:
  • Massive datasets (billions of tokens/images);
  • Distributed GPU/TPU clusters;
  • High energy consumption.

8.5. Computational Complexity and Practical Deployment

Transformer-based models have training complexity proportional to
O ( L · n 2 · d )
where L is number of layers, n is sequence length, and d is hidden dimension.
Diffusion models require multiple denoising steps (typically 50–1000), increasing inference latency.
In practical applications, trade-offs arise between
  • Model size and latency;
  • Accuracy and energy consumption;
  • Generalization and domain specialization.
These trade-offs significantly influence real-world deployment in healthcare, finance, robotics, and digital twin systems.

9. Applications of Generative AI

GenAI and foundation models have shown wide-ranging influence across many application areas. Their capacity to learn general-purpose representations and produce high-quality synthetic content has supported advances in language understanding, visual synthesis, multimodal reasoning, and scientific research. This section surveys major application areas of GenAI, with brief examples of typical use cases and current research trends.
For clarity and structured comparison, the surveyed applications are systematically summarized in Table 3, Table 4, Table 5 and Table 6. These tables consolidate key information regarding tasks, model paradigms, learning strategies, and reported performance metrics across domains. It is important to note that the reported results are obtained under heterogeneous experimental settings, including variations in datasets, preprocessing pipelines, training protocols, evaluation splits, and model configurations. Consequently, the metrics are not directly comparable across studies and should be interpreted within the context of their original experimental frameworks. The tables are intended to provide a structured overview of representative performance trends rather than to establish strict quantitative rankings.
Figure 4 provides a consolidated overview of the primary application landscape covered in this section, grouping representative tasks and use cases across ten domains.
Figure 4. Overview of representative application areas of GenAI and LLMs across ten domains, summarizing key task categories in (1) natural language processing, (2) computer vision, (3) multimodal and robotics, (4) biomedical and healthcare, (5) digital twins, (6) education, (7) finance, (8) smart cities, (9) transportation, and (10) virtual reality.

9.1. Natural Language Processing

GenAI has become a foundational approach across major natural language processing tasks, as summarized in Table 3. In information retrieval, LLMs are increasingly combined with classical methods to improve accuracy and factual consistency. Fine-tuned Transformer ensembles and reinforcement learning enhance retrieval performance [38,39], while retrieval-augmented generation frameworks integrating symbolic methods such as BM25 reduce hallucinations and improve relevance [40].
Named entity recognition benefits from generative and foundation models, particularly in low-resource settings. Few-shot and zero-shot prompting achieve competitive recall and partial match scores [41,42], while weak supervision using model-generated labels improves biomedical F1 scores [43]. However, supervised encoder-based models still outperform purely generative approaches in structured prediction accuracy [44].
In text classification, generative models serve as alternatives or complements to traditional classifiers. Zero-shot and few-shot prompting can outperform human annotators in some tasks and improve trait coverage [45,46], while supervised fine-tuning achieves consistently high F1 scores [47]. Prompt-based evaluations also reveal demographic and social biases, emphasizing the need for mitigation strategies [48].
Machine translation studies show that in-context learning enables strong performance in low-resource language pairs without parallel data, achieving competitive BLEU scores [49]. Text summarization remains a mature application area, with fine-tuned and hybrid generative models improving ROUGE scores, coherence, and factuality across domains [50,51,52,53,54].
Generative models are also effective for question and answer generation, where parameter-efficient fine-tuning such as QLoRA improves quality and efficiency [55], and multilingual fine-tuning outperforms traditional baselines in low-resource settings [56]. For misinformation and harmful content detection, prompt engineering, debate-based prompting, and retrieval augmentation improve detection accuracy and robustness [57,58,59], with similar trends observed in hate speech detection across multilingual and code-mixed data [60,61,62].
Finally, GenAI supports text generation and sentiment analysis. Supervised fine-tuning and moderation improve generation quality and safety [63], interpretability-focused probing enhances faithfulness [64], and hybrid Transformer architectures achieve state-of-the-art sentiment classification performance and real-world correlation [65,66,67]. Overall, these studies demonstrate the central role of GenAI in advancing scalable and adaptable NLP systems.
Table 3. Summary of studies applying GenAI and foundation models in NLP.

9.2. Computer Vision

Recent work in GenAI has improved computer vision, and diffusion models are now a main approach for image synthesis. Text-to-image generation has developed into a stable area of research and practice, and synthetic image generation can lower costs and provide more flexibility than traditional visual content production in areas such as advertising [68]. Adapter-based conditional generation methods that add LLMs can improve control and output fidelity by strengthening the semantic match between text prompts and images, and they report higher benchmark scores on COCO 2017 [69].
Generative models can support domain transfer and adaptation when training data are limited and when the data distribution shifts across settings. Diffusion models with added control modules, including ControlNet-style pipelines, allow detailed conditioning on structural information and support transfer from synthetic images to real photographs, which is useful for domain-specific tasks such as construction site analysis [70]. Studies of domain shift in medical imaging note that differences across institutions and imaging protocols can change model performance, which is important when deploying adaptive vision systems [71].
Table 4. Summary of studies applying GenAI and foundation models in computer vision.
Beyond image synthesis, generative and foundation models support multimodal vision–language alignment. Temporal vision–language models incorporate time-dependent visual information, improving alignment between visual observations and language in sequential settings and benefiting applications such as video understanding [72]. GenAI also underpins synthetic environment and simulation generation for autonomous systems, enabling scalable and controllable training data for perception and decision making in rare or hazardous scenarios [73]. Overall, these studies demonstrate that GenAI is reshaping computer vision through advances in synthesis, adaptation, multimodal reasoning, and simulation.

9.3. Multimodal and Robotics

Generative artificial intelligence plays a central role in multimodal and robotic systems by enabling integrated reasoning across vision, language, and action. Vision–language–action (VLA) models transfer web-scale multimodal knowledge to real-world robot control and demonstrate strong generalization across large-scale physical evaluations [74]. Hierarchical VLA architectures further improve instruction following by separating high-level reasoning from low-level control, achieving higher task success than flat policy designs [75]. Speech-integrated VLA systems extend these capabilities by combining vision–language models, LLMs, and speech recognition, enabling more effective execution of spoken instructions [76]. Efforts to unify perception, language, and action within a single framework further support multimodal interaction, although existing evaluations remain largely qualitative [77].
Generative models also contribute to robotic planning and environment understanding. Language-model-driven visual token generation offers an alternative to diffusion-based synthesis for embodied planning and achieves strong image generation performance [78]. Three-dimensional generative world models integrate vision, language, and action to predict future states, improving long-horizon planning consistency [79]. Multimodal alignment challenges are addressed through coarse-to-fine vision–language strategies that enhance cross-modal reasoning [80]. Generative multimodal models further support safety-related perception tasks, such as explainable hateful meme detection, improving both detection performance and interpretability through vision–language fusion [81].
Table 5. Applications of GenAI and foundation models in robotics.

9.4. Biomedical and Healthcare

Recent advances in large-scale foundation models have enabled diverse biomedical and healthcare applications. Speech-driven and retrieval-augmented language model systems significantly reduce documentation time and improve electronic medical record accuracy, while usability studies highlight efficiency gains alongside ongoing concerns about safety, trust, and appropriate automation levels [82,83]. Language models have also improved patient-facing communication by simplifying complex pathology reports and, in controlled evaluations, producing responses to rare disease queries that outperform physician-written answers in clarity and correctness [84,85].
Beyond documentation, language models support clinical decision making and care quality analysis. Free-text health record analysis enables large-scale auditing of guideline adherence [86], while real-world emergency department studies show that assistive language-model workflows can improve triage and diagnostic support and reduce under-triage risk [87]. In medication safety, retrieval-augmented co-pilot systems improve prescribing error detection when used alongside clinicians [88]. Prompt-engineered models further enable clinical risk prediction from tabular data with improved accuracy and interpretability, without task-specific fine-tuning [89].
Language-model-based systems can support medical education and can also be used in patient-facing decision support tools. Recent advanced models achieve passing scores on national medical licensing examinations [90], increase referral accuracy in symptom assessment tasks [91], and expand guideline coverage in educational patient simulations [92]. Clinical report generation within a specific medical specialty can improve when the tokenization scheme matches the field’s vocabulary and the Transformer model is fine-tuned on in-domain clinical notes, which can increase textual consistency and reduce mistakes in clinical terminology [93]. Recent review articles describe how foundation models can help digitize medical knowledge and support more inclusive workflows for drug discovery and biomedical innovation [94].
It is important to emphasize that many of these comparative findings are derived from retrospective analyses, simulated clinical scenarios, or benchmark-style evaluations rather than large-scale prospective deployment studies. Performance differences observed in controlled settings may not directly translate to routine clinical environments, where patient heterogeneity, workflow integration, time constraints, and institutional regulations introduce additional complexity. Despite these promising results, several limitations remain critical. Generative models may produce hallucinated or overconfident outputs, which, in high-stakes clinical contexts, can pose patient safety risks. Data privacy concerns, regulatory approval requirements, liability considerations, and the need for longitudinal validation under real-world conditions all constrain immediate large-scale deployment. Accordingly, current evidence supports the role of GenAI systems as assistive and augmentative technologies rather than replacements for clinical expertise. Continued prospective evaluation, bias assessment, and safety auditing are necessary before widespread integration into regulated healthcare environments.
Table 6. Applications of GenAI and foundation models in healthcare and clinical practice.

9.5. Other Domains

Beyond core technical fields, large-scale generative and language models are being used in a range of applied settings, including digital twins, education, finance, smart-city systems, transportation, and virtual reality as shown in Table 7. In digital twin systems, methods based on language models can support interactive use, basic reasoning, and ongoing management of virtual assets across their full life cycle. Blockchain-enabled and federated architectures can support secure governance of data, controlled sharing, and consistent labeling. Retrieval-augmented generation, when grounded in standardized asset representations, can improve semantic interoperability and support tasks that require reasoning over shared data [95,96,97,98]. Survey papers report that generative models are applied to fault diagnosis, predictive maintenance, synthetic data generation, and human-centered digital twin design [99,100], and they also discuss qualitative assessments of virtual reality interfaces [101].
Table 7. Applications of GenAI and LLMs in digital twins and related domains.
In educational settings, language models can estimate student outcomes and flag early risk using structured academic records through few-shot prompting and parameter-efficient fine-tuning, suggesting that these models can also support tasks that are not limited to natural language [102]. In financial settings, embeddings from language models are used for fraud detection, anomaly detection, and risk management. These embeddings are combined with varied transaction records and then passed to reinforcement learning methods or standard classifiers that support audit, compliance, and related review processes [103,104].
In smart cities, language models can support research on cybersecurity usability, help forecast urban energy demand, and aid data integration across municipal departments. Hybrid frameworks that combine semantic text representations, time-series embeddings, and knowledge graphs can support more accurate load prediction, human-centered security design, and natural language access to diverse urban databases [105,106,107]. In transportation systems [112], language-model-based architectures are used for spatio-temporal prediction and network optimization by combining graph attention mechanisms, trend decomposition, and low-rank adaptation to support traffic and mobile network forecasting [108,109,110].
In virtual reality and other immersive settings, generative models can produce scalable amounts of content and support interactive user experiences. Analyses report that language models, diffusion models, and 3D generative methods are used to generate text, images, video, and 3D objects for avatar design and for building dynamic virtual worlds [111]. Taken together, these studies show that GenAI has wide social and economic effects across many areas of use.

10. Unified Classification Framework and Cross-Field Comparison Dimensions

GenAI and foundation models (FMs) have developed rapidly through their application in natural language processing (NLP) and computer vision (CV) and robotics and biomedicine, which has resulted in multiple separate systems that use different domain-specific classification systems. The existing requirement for a single unified view can be resolved through the creation of a cross-disciplinary classification system which combines architectural advancements and training methods and operational functions into standard measurement dimensions.
The framework permits users to compare model performance and system performance between different research domains [113,114,115]. Foundation model systems require four distinct taxonomic categories to represent their complete operational life cycle and their system performance capabilities. The first classification system for models separates them into two groups based on their current development status and their methods for adapting to new situations [114]. The base foundation models, which include GPT and LLaMA, function as general-purpose backbones because they contain large-scale architectures that have undergone pretraining on extensive and diverse datasets [114,116]. Instruction-tuned variants undergo two different training methods, supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), which help them achieve better performance in following specific task instructions and meeting human expectations [114]. Multimodal variants build upon the basic premise of multimodal information systems by creating/processing and generating several types of information concurrently (i.e., using text, images, sound at the same time) with shared latent representations across the different types of information or separate modality-specific adapters [117,118,119].
The second dimension refers to the type of system design (i.e., the actual physical design of each of these systems and/or their function within systems) [113,120]. In one type of system, FMs may be the main units of perception; therefore, they will be performing feature extraction, object identification and interpretation of sensory data within the scope or region of CV and robotics use cases [121,122]. In contrast, under another type of system configuration, they may, instead, function mostly as generative engines synthesizing all types of data such as text in an NLP use case and images or video in an AI-generated use case (AIGC) [121,123]. Nevertheless, in more complex examples of multimodal systems, FMs can be considered within the context of agentic frameworks where FMs could act as planners, coordinators, or decision-makers capable of interfacing with other tools or APIs (application programming interfaces) to complete more sophisticated/perhaps complex tasks [120,122]. Systems developed using unified architectures integrate both the perception and generation functions of FMs into a single monolithic or modular end-to-end system and allow for a closer interdependencies between sensory function and reasoning and action [115,121].
The third dimension focuses on operational modality and unifiability concerns [29,115]. Single-modality models are designed specifically to work with one particular type of information (e.g., they may be used only for analyzing clinical text or medical images) [124,125]. In contrast, cross-modal alignment models map disparate sources of data into a common latent space so that the models can jointly reason about data. One example of a cross-modal alignment model is CLIP, which aligns representations from images and words [119,123].
At the most advanced level, Generalist Multimodal AI (GMM) architecture emphasizes the concepts of modularity, extensibility, and adaptability, allowing them to include new modalities and tasks with minimal retraining [115,126], and they incorporate responsible-AI-by-design principles as an added fourth dimension. The governance of pretraining involves dataset curation and filtering strategies designed to reduce bias and improve safety [113]. Safety alignment includes post-training interventions aimed at mitigating harmful or undesirable outputs, such as red-teaming and safety-constrained reinforcement learning [114,119]. In addition, system-level controls include architectural safeguards, such as filtering, monitoring, and transparency mechanisms, that are built into FM-based deployments to ensure compliance with ethical and regulatory expectations [113] across all domains. Researchers can evaluate models across all domains based on a consistent set of evaluation dimensions. Benchmark performance and task generality will be assessed according to domain-specific task suites, such as GLUE in NLP and ImageNet in CV, to determine accuracy and reasoning ability [117,121,124].
The zero-shot and few-shot performance also reflects their generalization ability, as they have shown successful completion of tasks without task-dependent fine-tuning [114,122]. The number of parameters updated through base model adaptation for downstream tasks shows their ability to adapt and transfer to other domains. This has been shown through low-rank adaptation (LoRA) to complete downstream tasks using a low amount of domain-specific data [115,122,127]. In robotics and healthcare, the robustness and real-world grounding metrics evaluate their ability to resist distribution shifts and determinewhether their digital abstraction has a direct correlation to the physical and/or clinical reality they relate to, respectively [113,125,126]. Efficiency and scalability metrics evaluate the computational power, memory requirements, and inference time necessary to achieve real-time performance; these are very important when building edge devices [114,123,127]. The alignment and responsible AI metrics assess the degree to which the model adheres to human-centric principles, indicators of fairness, privacy, explainability, and the standards of safety for each domain, particularly in clinical situations [113,114,116,125,128,129]. The unified classification framework and comparison parameters can be mapped across four major domains, as shown in Table 8.
Table 8. Cross-domain taxonomy fragments, comparison axes, and representative evaluation metrics.
Comparison displays divergence/convergence between domains; NLP has been developing with strong emphasis on alignment, instruction tuning, linguistic generalization, with more mature benchmarks, and human preference evaluations as bases for developing the field. CV/multimodal research has had a greater emphasis on cross-modal alignment and unified representation learning, with integration of visual/text modalities as major areas of focus. In robotics, embodied grounding, real-time constraints, and sim-to-real transfer are unique for this area of research, illustrating physical consequences of model decisions. Biomedical/healthcare applications are particularly focused on safety alignment and regulatory compliance and metrics that are clinically meaningful—typically requiring more stringent validation requirements than others. Despite these differences, the four domains have been converging around scalable pretraining, modular adaptation methods/strategies, multimodal integration, and responsible AI governance; providing standardized vocabulary/analytical structure for comparing foundation models across/within disciplines; classifying models according to training paradigms; architectural roles; modality integration; architecture/design of responsible-AI; and evaluating using common performance/dimensions of adaptability/robustness/efficiency/alignment [29,115,122].

Unified Cross-Domain Evaluation Framework for Foundation Models

Continuous development of foundational models (e.g., natural language processing, computer vision, etc.) across various modalities provides insight into the current limitations of fragmented and singular benchmarking approaches. Because of the rapid expansion of foundational models into high-stakes applications, the need for a more unified and holistic evaluation system is increasingly evident. One way to address concerns regarding incomplete discussions surrounding evaluation strategies is through the creation of a modular framework for cross-domain evaluation, which will allow for standardized evaluations that still meet the needs of each domain. Our proposed framework contains six key components in order to establish an overarching evaluative framework for GenAI: task performance; safety/harm; bias/fairness; robustness; efficiency/cost; and factuality/explanation (see Table 9). This evaluation framework will draw from many recent frameworks (i.e., HEMM, ChEF, GAICo) in order to expand upon our current understanding of evaluating GenAI.
To date, there has been no way to assess foundational models based on multiple criteria, which limits comparability between various foundational models and creates uncertainty regarding deployment in application areas (e.g., healthcare and robotics) that carry significant risks and/or potential trade-offs. To fill the voids identified in previous frameworks, the proposed framework provides explicit axes of evaluation that represent capabilities, risks, and operational costs [130,131]. Standardized measurement and reporting practices allow meaningful comparison across domains and enable the evaluation of models across not only performance but also safety, fairness, and robustness [132,133]. Although task performance has traditionally served as the baseline dimension against which performance is evaluated through task-specific benchmarks and capability suites [130,131,133], task performance goes beyond simply measuring a model’s ability to perform on tasks.
Performance, in the case of natural language processing (NLP), includes the capabilities for reasoning, summarizing, and answering questions [130,133]. In multimodal systems, performance consists of evaluating the fine detail of image understanding and alignment [131,133]. Clinical performance in the practice of medicine relies on accuracy, completeness, reliability and soundness of reasoned clinical decisions [134,135]; however, it does not adequately reflect safety considerations for high-risk industries without combining performance with complementary measures related to risk. Evaluating the safety of a given model includes the objective of eliminating the potential for producing an unsafe or unreliable output, especially within high-risk industries [130,133]. In healthcare, clinical frameworks provide clinicians with tools to detect hallucinations, assess refusal behavior, and mitigate risk of making an unsafe or unreliable clinical decision [134]. With respect to the robotics platform, an ideal safety evaluation framework would include the incorporation of closed-loop failure analysis and simulation-based hazard testing methods, although there is limited availability of standardized approaches and guidelines at this time [136].
In addition to determining whether a model produced a reliable or an unreliable output, determining whether or not the outputs produced by a given model are biased or fair is essential in medicine and healthcare economic outcomes research (HEOR). Ethical audits, demographic parity analyses, and continual monitoring of fairness must be performed regularly to ensure that potential sources of bias in decision making do not affect healthcare or HEOR clinical outcomes [133,134,137]. Although the fields of natural language processing and medicine have seen progress in measuring fairness metrics, there is very little research on how to measure fairness with regards to robotic systems. Robustness is measured by stability to changes in distributions, adversarial prompting, or using multistep reasoning for stress tests on robotic systems [131,133]. The idea of multimodal robustness considers whether the robot can keep working correctly while being exposed to sensory noise or occlusion [131]. In clinical settings, methods of cross-examining truthfulness can be used to evaluate factual consistency across multiple situations [134,138]. As models increase in size, efficiency and cost have become important in terms of evaluating their use by measuring latency, memory usage, and the trade-offs of resources to performance [137,139,140]. These issues will be especially important in robotics that operate in real time or in screening patients at large clinics [137,139]. The determination of whether a model is factual is done using methods such as hallucination detection and grounding tests [133,141]. Methods that can be broadly applied, like “LLM juries”, which consist of combining the output from many models or expert humans, can be used to evaluate the factuality of complex tasks like clinical summaries [135,138]. Robotics and vision systems both provide interpretable planning traces and visualization methods for both explainability and factuality, resulting in an increase in transparency and trust for those systems in these two dimensions [131,133,136]. This modular framework contains several elements, including scenario banks, recipe systems, metric suites, human assessment processes, and structured reporting for operational purposes [133,134]. To investigate particular capabilities of a robot or vision system, scenario banks provide a curated set of datasets to use and useful examples to replicate [131,133]. Recipe systems standardize prompts and settings used for inference, which leads to the reproducibility of results in similar situations [133]. Metric suites include both automatic reference-based metric systems and structured-output validators in a common library encapsulated within one system of equations and the like [133,134,136]. Human-in-the-loop workflows and computer-aided majority voting for high-profile tasks allows for the evaluation of factuality and safety to be scaled [135,138]. In addition, multiaxis reporting profiles and domain-specific checklists will replace single score leaderboards, providing a basis for governance and deployment decisions [130,133].
Table 9. Cross-domain evaluation dimensions and representative measurement examples.
The development of evaluation frameworks has made progress, yet research gaps still exist between existing scientific studies. The literature lacks sufficient standardized methods to assess safety and bias elements in robotics which creates a research area that requires further investigation according to source two. Although cross-modal evaluation metrics have been developed, their effectiveness in diverse domain testing needs more comprehensive validation, according to sources six and twelve. The use of automated image quality assessment metrics like SSIM presents a challenge because these metrics fail to provide complete semantic understanding of visual content which doctors need for making critical decisions, according to sources two and seven. The unified cross-domain evaluation framework provides a complete evaluation system for foundation model assessment because it combines performance metrics with safety, fairness, robustness, efficiency, and factuality through domain-specific measurement tools. The framework creates a unified evaluation system which combines various test components to help researchers understand model behavior while providing complete assessment tools for their research work. The framework creates both theoretical and practical research foundations which will guide future studies on GenAI evaluation science.

11. Challenges and Open Issues

Despite their rapid progress, GenAI and foundation models face persistent challenges that constrain scalability, reliability, and societal adoption. The main concerns are the long-term computing costs of these systems, ethical and safety risks, risks to data privacy, and the limits of current evaluation methods. The practical relevance of mitigation strategies will improve when their summary demonstrates their specific methods. Evidence-based practices include retrieval-augmented generation to reduce hallucinations and bias auditing with benchmark datasets, parameter-efficient fine-tuning for domain control and energy-efficient training protocols, and human-in-the-loop validation in high-stakes applications. The forward-looking claims about general intelligence and bias elimination should be treated as dreams because they present unattainable goals. The process of separating established methods from untested paths will enhance both understanding and trustworthiness and practical usefulness.

11.1. Computational and Energy Requirements

Many foundation models are built by training at very large scales, with billions of parameters and datasets that are also very large. Scaling can improve performance, but it requires large amounts of compute and funding, which limits state-of-the-art development to a small number of organizations [22]. Large-scale training can also have a substantial environmental impact, producing carbon emissions on the order of what would be generated over the lifetimes of several cars, especially when the electricity supply relies on carbon-intensive sources [143,144]. Deploying the model at scale tends to increase the energy used for inference. These concerns have motivated research on efficiency methods, including quantization, pruning, sparsity, and mixture-of-experts architectures [145], but it remains unclear how to obtain large efficiency gains without reducing performance.
Addressing computational and energy challenges requires a combination of architectural and operational strategies. Hardware-aware training, sparse attention mechanisms, adaptive computation routing, and modular mixture-of-experts designs can significantly reduce redundant computation. Furthermore, quantization-aware training and post-training compression techniques allow lower-precision inference without substantial performance degradation. The integration of renewable energy sources in large-scale data centers and transparent reporting of energy consumption metrics should also become standard practice to ensure sustainable deployment of GenAI systems.

11.2. Ethical Concerns

Ethical risks can arise when models are trained on large, uncurated datasets that contain societal biases, which may lead to discriminatory outcomes in downstream applications [146]. Generative models can also produce hallucinations, where they generate fluent text that is false, which creates risks in areas such as healthcare and education [147]. The capacity to produce realistic synthetic media raises concerns about misinformation, deepfakes, and reduced trust in digital content [148]. Copyright and intellectual property disputes can make deployment more difficult because training may rely on copyrighted material without clear permission from rights holders.
Mitigation strategies for ethical concerns include rigorous dataset documentation, bias auditing using standardized fairness benchmarks, and continual post-deployment monitoring. Retrieval-augmented generation can reduce hallucination risks by grounding outputs in verified external knowledge sources. Additionally, structured red-teaming, adversarial testing, and alignment tuning through reinforcement learning from human feedback can enhance ethical robustness. Establishing clear data provenance policies and transparent model cards further strengthens accountability and responsible AI governance.

11.3. Safety and Robustness

GenAI systems can still be targeted through adversarial inputs and prompt manipulation that evade built-in safety controls [149]. Alignment methods that rely on human feedback can still allow misaligned behavior, especially in new or unclear settings [22]. Security risks include producing code with weak security and supporting social engineering attacks [150]. Performance varies across languages, cultures, and rare edge cases, which suggests a need for stronger alignment and clearer safety guarantees.
Improving safety and robustness requires multilayered safeguards. These include adversarial training pipelines, structured stress-testing under simulated edge cases, and closed-loop evaluation environments for high-risk applications such as robotics and healthcare. Neuro-symbolic reasoning integration and constraint-based decoding can enhance logical consistency and reduce unsafe outputs. Furthermore, incorporating uncertainty estimation and human-in-the-loop validation mechanisms provides additional safeguards in safety-critical domains.

11.4. Data Privacy

Web-scale pretraining can create privacy risks because a model may memorize and later reproduce sensitive or proprietary information [151]. These risks are higher in regulated settings and can increase through user interactions with deployed systems [8]. Methods such as differential privacy and federated learning can reduce some privacy risks, but they often lower model performance, so privacy-preserving GenAI remains an unresolved research problem.
Advances in privacy-preserving GenAI should focus on improving the trade-off between data utility and confidentiality. Differential privacy mechanisms can be optimized through adaptive noise calibration, while federated learning frameworks can incorporate secure aggregation and encrypted gradient exchange. Synthetic data generation, when properly validated, may reduce reliance on sensitive datasets. Clear governance policies, privacy audits, and compliance with regulatory standards such as GDPR and HIPAA are essential for responsible real-world deployment.

11.5. Evaluation Metrics

Common metrics such as BLEU, ROUGE, and FID do not reflect important aspects of model quality, such as factual accuracy, reasoning ability, safety, and bias [152]. This constraint is even stronger for multimodal models because the available benchmarks are scattered and often do not reflect real-world settings. Although human evaluation can provide useful insight, it is expensive and does not scale well. As benchmarks become saturated, reliability tends to decrease, which supports the need for broader, more resilient evaluation frameworks [153].
To improve evaluation practices, future research should adopt unified cross-domain evaluation frameworks that incorporate multidimensional metrics including reliability, safety, robustness, fairness, efficiency, and hallucination detection. Standardized benchmark suites aligned with real-world deployment contexts should complement traditional task-based metrics. Hybrid evaluation strategies that combine automated scoring, structured factuality probes, and expert human assessment can enhance validity. Transparent reporting of dataset characteristics, statistical variance, and experimental configurations will further strengthen reproducibility and cross-study comparability.

12. Future Directions

As GenAI and foundation models improve, future research should move beyond scaling for higher benchmark results and focus on efficiency, clear reporting of methods and limits, generalization across different data types, and broader access with appropriate safeguards. Large-scale models can perform well, but their lasting value will depend on how research addresses limits in sustainability, interpretability, and fair access. This section describes the main research directions that are guiding the development of the next generation of foundation models.

12.1. Efficient Training and Inference

A central future direction is to develop training and inference methods that reduce compute demands, lower energy use, and limit environmental impact. Many current foundation models depend mainly on brute-force scaling, and this approach is becoming less sustainable over time. Methods such as quantization, pruning, and knowledge distillation can reduce model size and inference time while keeping accuracy close to that of the original model. Quantization lowers numeric precision to reduce memory use and speed up inference [154]. Pruning removes redundant parameters or attention heads to cut computation [155]. Knowledge distillation trains a smaller student model to match the behavior of a larger teacher model, which supports deployment in resource-limited settings [156]. Parameter-efficient fine-tuning approaches, such as adapters and low-rank adaptation, can make foundation models easier to adapt and personalize for specific tasks [157]. Future work will likely combine these methods with adaptive model designs and hardware-aware efficiency constraints to support sustainable GenAI while maintaining model quality and speed.

12.2. Improved Interpretability and Explainability

Even when foundation models perform well, their internal reasoning is often unclear, which can reduce trust, hinder error analysis, and complicate use in safety-critical settings such as healthcare and law. Current interpretability methods, such as attention visualizations, probing classifiers, and concept attribution, can provide some insight, but they still do not fully account for how the model reaches its decisions [158]. Mechanistic interpretability studies the internal circuits of neural networks and is still an early-stage area of research [159]. Explainability matters for multimodal and agentic systems because their reasoning can move across different data types and unfold over many time steps. Developing interpretable model architectures and standard explanation frameworks is necessary to meet regulatory requirements, support human oversight, and enable the responsible deployment of AI systems.

12.3. Multimodal Foundation Models Beyond Text and Vision

Most current multimodal foundation models combine text and images, but later systems are likely to include audio, video, sensor inputs, spatial data, and data from embodied interaction. Recent studies examine joint language–audio–vision models for video understanding, speech-based interaction, and robotics [160]. Beyond perception alone, embodied AI and autonomous agents in physical settings need to combine haptic input, proprioception, and signals from the environment to support reliable action [161]. As multimodal systems expand, they face practical issues with aligning and synchronizing data across modalities, as well as with assessing performance, which calls for scalable fusion methods and reliable cross-modal reasoning for use in real-world settings.

12.4. Human–AI Collaboration and Agentic AI Systems

A major area for research is human–AI collaboration, especially the design of agentic AI systems that can plan over long time spans, reason through complex tasks, and take actions toward defined goals. Future foundation models are often described not as passive tools but as collaborative partners that can support problem solving, creative work, and decision making. Recent work on tool-augmented language models and autonomous agents suggests that these systems can interact with external systems, run code, and divide tasks into smaller steps [162]. Agentic behavior raises challenges for reliability, controllability, and alignment. Future research should focus on safe agent design, human-in-the-loop oversight, methods for representing uncertainty, and fail-safe controls so that greater autonomy supports human decision making rather than weakening it.

12.5. Democratization and Open-Source Foundation Models

Making foundation models more widely accessible is an important step toward AI research and deployment that includes diverse groups and supports fairness. The growing dominance of proprietary models raises questions about transparency, accountability, and unequal access to these systems. Open-source foundation models offer an alternative because they allow more researchers to take part in study, review and audit the models, adapt them to specific needs, and deploy them in practice [163]. They support reproducible research and allow low-resource communities to adapt models to local languages and specific needs. At the same time, open access can increase the risk of misuse and create security concerns. Future work should balance openness and responsibility by using clear governance frameworks, careful licensing choices, and coordinated work between researchers, industry partners, and public officials.

13. Conclusions

This review has systematically examined the evolution of GenAI and foundation models, with a focus on their underlying architectures, training strategies, and emerging application domains. Recent work on Transformer LLMs, diffusion generative models, and multimodal systems has shifted research toward general-purpose AI that can learn reusable representations and perform many tasks with limited supervision. Combining large-scale self-supervised pretraining with efficient fine-tuning has allowed foundation models to reach strong results in natural language processing, computer vision, robotics, and biomedical tasks.
Even with recent progress, several key challenges still remain. Training and running very large models often requires high levels of computing power and electricity, which raises concerns about environmental impact and fair access to these systems. Unresolved issues, including limited interpretability, vulnerability to bias and misinformation, risks to data privacy, and inconsistent evaluation methods, still constrain the safe and trustworthy use of GenAI in real-world contexts.
Despite its broad coverage and systematic structure, this survey is subject to certain limitations. In particular, empirical reproducibility within the reviewed literature is often constrained by the proprietary nature of many state-of-the-art foundation models and the restricted accessibility of their training datasets, alignment strategies, and evaluation pipelines. Consequently, several reported performance outcomes are derived from closed-source systems that cannot be independently examined or fully replicated. This reliance on partially undisclosed industrial platforms contributes to a reproducibility gap within the field and limits the extent to which empirical findings can be directly validated within the scope of this review.
Future work should move away from improvements based only on scaling and focus instead on model design that is efficient, interpretable, and responsible. Future research should focus on parameter-efficient model designs, stronger multimodal reasoning across more than text and images, alignment and safety methods with clear guarantees, and shared evaluation standards that match practical use settings. Alongside technical progress, open and collaborative research efforts are needed to support transparency, reproducibility, and responsible broad access to foundation models. Addressing these challenges is necessary to ensure that GenAI systems develop in ways that meet scientific standards and support public interests.

Author Contributions

Conceptualization, A.E., S.E., P.D., D.P., Q.Z. and S.S.; data curation, A.E. and S.E.; formal analysis, A.E., S.E., P.D., D.P., Q.Z. and S.S.; funding acquisition, A.E.; methodology, A.E., S.E., P.D., D.P., Q.Z. and S.S.; project administration, A.E., S.E., P.D., D.P., Q.Z. and S.S.; supervision, A.E., S.E., P.D., D.P., Q.Z. and S.S.; validation, A.E., S.E., P.D., D.P., Q.Z. and S.S.; visualization, A.E., S.E., P.D., D.P., Q.Z. and S.S.; writing—original draft, A.E. and S.E.; writing—review and editing, A.E., S.E., P.D., D.P., Q.Z. and S.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed during this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
A2CAdvantage Actor–Critic
AASAsset Administration Shell
AIArtificial Intelligence
APIApplication Programming Interface
AutoMLAutomated Machine Learning
BARTBidirectional and Auto-Regressive Transformers
BERTBidirectional Encoder Representations from Transformers
BiFPNBi-directional Feature Pyramid Network
BLEUBilingual Evaluation Understudy
BM25Best Matching 25
CBAMConvolutional Block Attention Module
CLIPContrastive Language–Image Pretraining
CNNConvolutional Neural Network
CoTChain of Thought
CSVComprehensive Survey
DDIMDenoising Diffusion Implicit Model
DDPMDenoising Diffusion Probabilistic Model
DLDeep Learning
DQNDeep Q-Network
ECGElectrocardiogram
EDEmergency Department
EHRElectronic Health Record
EMRElectronic Medical Record
ETExtra Trees
FIDFréchet Inception Distance
GANGenerative Adversarial Network
GATGraph Attention Network
GCNGraph Convolutional Network
GLMGeneralized Language Model
GNNGraph Neural Network
GPTGenerative Pretrained Transformer
GRUGated Recurrent Unit
IPFSInterPlanetary File System
IoTInternet of Things
KANKolmogorov–Arnold Network
LLMLarge Language Model
LoRALow-Rank Adaptation
LSTMLong Short-Term Memory
MAPMean Average Precision
MeSHMedical Subject Headings
MLMMasked Language Modeling
MoEMixture of Experts
MRRMean Reciprocal Rank
NFTNon-Fungible Token
NLPNatural Language Processing
NTPNext-Token Prediction
PEFTParameter-Efficient Fine-Tuning
PPOProximal Policy Optimization
QLoRAQuantized Low-Rank Adaptation
RAGRetrieval-Augmented Generation
RLReinforcement Learning
RLHFReinforcement Learning from Human Feedback
ROUGERecall-Oriented Understudy for Gisting Evaluation
RNNRecurrent Neural Network
SBERTSentence-BERT
SSLSelf-Supervised Learning
STLSeasonal-Trend Decomposition using Loess
ST-LLMSpatio-Temporal Large Language Model
SVMSupport Vector Machine
SVSurvey
SRSystematic Review
T2IText-to-Image
Time2VecTime-to-Vector Embedding
VAEVariational Autoencoder
ViTVision Transformer
VLAVision–Language–Action
VLMVision–Language Model
VQAVisual Question Answering
VRVirtual Reality
XAIExplainable Artificial Intelligence
XGBoostExtreme Gradient Boosting

References

  1. Larrañaga, P.; Atienza, D.; Diaz-Rozo, J.; Ogbechie, A.; Puerto-Santana, C.E.; Bielza, C. Industrial Applications of Machine Learning; CRC Press: Boca Raton, FL, USA, 2018. [Google Scholar]
  2. Myers, D.; Mohawesh, R.; Chellaboina, V.I.; Sathvik, A.L.; Venkatesh, P.; Ho, Y.H.; Henshaw, H.; Alhawawreh, M.; Berdik, D.; Jararweh, Y. Foundation and large language models: Fundamentals, challenges, opportunities, and social impacts. Clust. Comput. 2024, 27, 1–26. [Google Scholar] [CrossRef] [Scilit]
  3. Elhanashi, A.; Essahraui, S.; Dini, P.; Saponara, S. Early fire and smoke detection using deep learning: A comprehensive review of models, datasets, and challenges. Appl. Sci. 2025, 15, 10255. [Google Scholar] [CrossRef] [Scilit]
  4. Chen, J.; Geng, Y.; Chen, Z.; Pan, J.Z.; He, Y.; Zhang, W.; Chen, H. Zero-shot and few-shot learning with knowledge graphs: A comprehensive survey. Proc. IEEE 2023, 111, 653–685. [Google Scholar] [CrossRef] [Scilit]
  5. Liu, X.; Zhang, F.; Hou, Z.; Mian, L.; Wang, Z.; Zhang, J.; Tang, J. Self-supervised learning: Generative or contrastive. IEEE Trans. Knowl. Data Eng. 2021, 35, 857–876. [Google Scholar] [CrossRef]
  6. Yu, J.; Zhu, Z.; Wang, Y.; Zhang, W.; Hu, Y.; Tan, J. Cross-modal knowledge reasoning for knowledge-based visual question answering. Pattern Recognit. 2020, 108, 107563. [Google Scholar] [CrossRef] [Scilit]
  7. Santosh, K.C.; Gaur, L. Privacy, security, and ethical issues. In Artificial Intelligence and Machine Learning in Public Healthcare; Springer: Cham, Switzerland, 2022; pp. 65–74. [Google Scholar]
  8. Mühlhoff, R. Predictive privacy: Towards an applied ethics of data analytics. Ethics Inf. Technol. 2021, 23, 675–690. [Google Scholar] [CrossRef] [Scilit]
  9. Essahraui, S.; Elhanashi, A.; Zheng, Q.; Almutairi, N.; Saponara, S. Tiny deep learning models for real-time and efficient embedded driver state detection. J. Real-Time Image Process. 2026, 23, 39. [Google Scholar] [CrossRef] [Scilit]
  10. Ong, C.W.M.; Blackbourn, H.D.; Migliori, G.B. GPT-4, artificial intelligence and implications for publishing. Int. J. Tuberc. Lung Dis. 2023, 27, 425–426. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  11. Seth, I.; Lim, B.; Cevik, J.; Sofiadellis, F.; Ross, R.J.; Cuomo, R.; Rozen, W.M. Utilizing GPT-4 and generative artificial intelligence platforms for surgical education. Eur. J. Plast. Surg. 2024, 47, 19. [Google Scholar] [CrossRef] [Scilit]
  12. Golec, J.; Hachaj, T. Ten natural language processing tasks with generative artificial intelligence. Appl. Sci. 2025, 15, 9057. [Google Scholar] [CrossRef] [Scilit]
  13. Bousetouane, F. Generative AI for Vision: A Comprehensive Study of Frameworks and Applications. arXiv 2025, arXiv:2501.18033. [Google Scholar] [CrossRef] [Scilit]
  14. Zhang, K.; Yun, P.; Cen, J.; Cai, J.; Zhu, D.; Yuan, H.; Zhao, C.; Feng, T.; Wang, M.Y.; Chen, Q.; et al. Generative artificial intelligence in robotic manipulation: A survey. arXiv 2025, arXiv:2503.03464. [Google Scholar] [CrossRef] [Scilit]
  15. Rabbani, S.A.; El-Tanani, M.; Sharma, S.; Rabbani, S.S.; El-Tanani, Y.; Kumar, R.; Saini, M. Generative artificial intelligence in healthcare: Applications, implementation challenges, and future directions. BioMedInformatics 2025, 5, 37. [Google Scholar] [CrossRef] [Scilit]
  16. Li, T.; Long, Q.; Chai, H.; Zhang, S.; Jiang, F.; Liu, H.; Huang, W.; Jin, D.; Li, Y. Generative ai empowered network digital twins: Architecture, technologies, and applications. ACM Comput. Surv. 2025, 57, 157. [Google Scholar] [CrossRef] [Scilit]
  17. Chakraborty, S. Generative AI in modern education society. arXiv 2024, arXiv:2412.08666. [Google Scholar] [CrossRef] [Scilit]
  18. Sai, S.; Arunakar, K.; Chamola, V.; Hussain, A.; Bisht, P.; Kumar, S. Generative AI for finance: Applications, case studies and challenges. Expert Syst. 2025, 42, e70018. [Google Scholar] [CrossRef] [Scilit]
  19. Pradhan, D.; Arora, L.; Shetgaonkar, A.; Girija, S.S.; Kapoor, S.; Raj, A. Opportunities and applications of genai in smart cities: A user-centric survey. In 2025 IEEE International Conference on Omni-Layer Intelligent Systems (COINS); IEEE: Piscataway, NJ, USA, 2025; pp. 1–7. [Google Scholar]
  20. Maksoud, N.; AlJassmi, H.; Ali, L.; Masoud, A.R. Applications of large language models and generative AI in transportation: A systematic review and bibliometric analysis. Transp. Res. Interdiscip. Perspect. 2025, 34, 101699. [Google Scholar] [CrossRef] [Scilit]
  21. Rahimi, F.; Sadeghi-Niaraki, A.; Choi, S.-M. Generative AI Meets Virtual Reality: A Comprehensive Survey on Applications, Challenges, and Future Direction. IEEE Access 2025, 13, 94893–94909. [Google Scholar] [CrossRef] [Scilit]
  22. Bommasani, R.; Hudson, D.A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M.S.; Bohg, J.; Bosselut, A.; Brunskill, E.; et al. On the opportunities and risks of foundation models. arXiv 2021, arXiv:2108.07258. [Google Scholar] [CrossRef] [Scilit]
  23. Qian, S.; Zhou, Z.; Xue, D.; Wang, B.; Xu, C. From linguistic giants to sensory maestros: A survey on cross-modal reasoning with large language models. arXiv 2024, arXiv:2409.18996. [Google Scholar]
  24. Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In Advances in Neural Information Processing Systems 27; Curran Associates: Red Hook, NY, USA, 2014. [Google Scholar]
  25. Wei, R.; Garcia, C.; El-Sayed, A.; Peterson, V.; Mahmood, A. Variations in variational autoencoders-a comparative evaluation. IEEE Access 2020, 8, 153651–153670. [Google Scholar] [CrossRef] [Scilit]
  26. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems 30; Curran Associates: Red Hook, NY, USA, 2017. [Google Scholar]
  27. Ho, J.; Jain, A.; Abbeel, P. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33; Curran Associates: Red Hook, NY, USA, 2020; pp. 6840–6851. [Google Scholar]
  28. Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 19–24 June 2022; pp. 10684–10695. [Google Scholar]
  29. Li, C.; Gan, Z.; Yang, Z.; Yang, J.; Li, L.; Wang, L.; Gao, J. Multimodal foundation models: From specialists to general-purpose assistants. Found. Trends Comput. Graph. Vis. 2024, 16, 1–214. [Google Scholar] [CrossRef] [Scilit]
  30. Ibrahim, M. Fine-grained languagebased reliability detection in spanish new with fine-tuned llama-3 model. In Iberian Languages Evaluation Forum (IberLEF 2024); CEUR: Aachen, Germany, 2024. [Google Scholar]
  31. Adetayo, A.J.; Aborisade, M.O.; Sanni, B.A. Microsoft Copilot and Anthropic Claude AI in education and library service. Libr. Hi Tech News 2024. ahead of print. [Google Scholar] [CrossRef] [Scilit]
  32. Imran, M.; Almusharraf, N. Google Gemini as a next generation AI educational tool: A review of emerging educational technology. Smart Learn. Environ. 2024, 11, 22. [Google Scholar] [CrossRef] [Scilit]
  33. Waisberg, E.; Ong, J.; Masalkhi, M.; Lee, A.G. OpenAI’s Sora in ophthalmology: Revolutionary generative AI in eye health. Eye 2024, 38, 2502–2503. [Google Scholar] [CrossRef] [Scilit]
  34. McCoy, R.T.; Yao, S.; Friedman, D.; Hardy, M.D.; Griffiths, T.L. Embers of autoregression show how large language models are shaped by the problem they are trained to solve. Proc. Natl. Acad. Sci. USA 2024, 121, e2322420121. [Google Scholar] [CrossRef] [Scilit]
  35. Zhang, C.; Zhang, C.; Zhang, M.; Kweon, I.S. Text-to-image diffusion models in generative ai: A survey. arXiv 2023, arXiv:2303.07909. [Google Scholar]
  36. Kim, W. Multimodal foundation models: A taxonomy and reflections. Int. J. Web Grid Serv. 2024, 20, 505–531. [Google Scholar] [CrossRef] [Scilit]
  37. Mehmood, R.; Bashir, R.; Giri, K.J. Text conditioned generative adversarial networks generating images and videos: A critical review. SN Comput. Sci. 2024, 5, 935. [Google Scholar] [CrossRef] [Scilit]
  38. Dofitas, C., Jr.; Kim, Y.-W.; Byun, Y.-C. A Weighted Ensemble Approach Integrating Large Language Models for Enhanced Agricultural Knowledge Retrieval. In Proceedings of the International Conference on Multimedia Information Technology and Applications; Springer: Cham, Switzerland, 2025; pp. 16–27. [Google Scholar]
  39. Li, J.; Shen, Y.; Zhou, W.; Chen, H. SearchExpert: A GenAI-driven framework for reasoning-intensive multimedia information fusion through fine-tuning and reinforcement learning. Inf. Fusion 2025, 126, 103665. [Google Scholar] [CrossRef] [Scilit]
  40. Chen, Q.; Zhou, W.; Cheng, J.; Yang, J. An Enhanced Retrieval Scheme for a Large Language Model with a Joint Strategy of Probabilistic Relevance and Semantic Association in the Vertical Domain. Appl. Sci. 2024, 14, 11529. [Google Scholar] [CrossRef] [Scilit]
  41. Shinohara, E.; Kawazoe, Y. Efficient medical NER with limited data: Enhancing LLM performance through annotation guidelines. Int. J. Med. Inform. 2025, 208, 106230. [Google Scholar] [CrossRef] [Scilit]
  42. Qin, M.; Feng, L.; Lu, J.; Sun, Z.; Yu, Z.; Han, L. ZeroTuneBio NER: A three-stage framework for zero-shot and zero-tuning biomedical entity extraction using large language models and prompt engineering. Comput. Methods Programs Biomed. 2025, 272, 109070. [Google Scholar] [CrossRef] [Scilit]
  43. Hsu, E.; Roberts, K. Leveraging large language models for knowledge-free weak supervision in clinical natural language processing. Sci. Rep. 2025, 15, 8241. [Google Scholar] [CrossRef] [Scilit]
  44. El Janah, H.; Nachid-Idrissi, Y.; Sarrouti, M.; Najah, S. Exploring transformer models: Fine-tuning VS inference on relation extraction from biomedical texts. Comput. Struct. Biotechnol. J. 2025, 31, 157–168. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  45. Bermejo, V.J.; Gago, A.; Gálvez, R.H.; Harari, N. LLMs outperform outsourced human coders on complex textual analysis. Sci. Rep. 2025, 15, 40122. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  46. Farmer, E.E.; Brown, D.; Gore, M.A.; Tufan, H.A. Applying large language models to extract information from crop trait prioritization studies. Plants People Planet 2026, 8, 176–184. [Google Scholar] [CrossRef] [Scilit]
  47. Fang, M.Z.; Nakashima, M.; Singh, K.; Galvani, E.; Sun, X.; Sorathia, S.; Dorocak, K.; Kwon, D.; Nguyen, C.; Chen, D. Cardiac magnetic resonance imaging-large language model Meta AI: A finetuned large language model for identifying findings and associated attributes in cardiac magnetic resonance imaging reports. J. Cardiovasc. Magn. Reson. 2025, 27, 2. [Google Scholar] [CrossRef] [Scilit]
  48. Radaideh, M.I.; Kwon, O.H. Fairness and social bias quantification in Large Language Models for sentiment analysis. Knowl.-Based Syst. 2025, 319, 113569. [Google Scholar] [CrossRef] [Scilit]
  49. Akallouch, O.; Fardousse, K. In-Context Learning for Low-Resource Machine Translation: A Study on Tarifit with Large Language Models. Algorithms 2025, 18, 489. [Google Scholar] [CrossRef] [Scilit]
  50. Liu, X.; Lu, J.; Xu, A. Dynamic agenda-aware real-time meeting summarization with large language models. J. King Saud Univ. Comput. Inf. Sci. 2025, 37, 275. [Google Scholar] [CrossRef] [Scilit]
  51. El-Deeb, R.H.; Abdelmoez, W.; El-Bendary, N. Enhancing E-Recruitment Recommendations Through Text Summarization Techniques. Information 2025, 16, 333. [Google Scholar] [CrossRef] [Scilit]
  52. Ning, W.; Shi, S.; Yin, Y.; Guo, Z.; Xie, J. Intent-aware personalized summarization for educational texts with large language models. J. King Saud Univ. Comput. Inf. Sci. 2025, 37, 308. [Google Scholar] [CrossRef] [Scilit]
  53. Song, Y.; Qin, Y.; Huang, R.; Chen, Y.; Lin, C. Legal text summarization via judicial syllogism with large language models. J. King Saud Univ. Comput. Inf. Sci. 2025, 37, 111. [Google Scholar] [CrossRef] [Scilit]
  54. Hark, C. Using Graph-Based Maximum Independent Sets with Large Language Models for Extractive Text Summarization. Appl. Sci. 2025, 15, 6395. [Google Scholar] [CrossRef] [Scilit]
  55. Choi, J.; Hong, S.; Hong, S.; Park, J.; Jung, E.-S. Towards Generating Quality Test Questions and Answers using Quantized Low-Rank Adapters in LLMs. IEEE Access 2025, 13, 87793–87809. [Google Scholar] [CrossRef] [Scilit]
  56. Pham, Q.-H.; Le, H.-L.; Dang Nhat, M.; Tran, T.K.; Tran-Tien, M.; Dang, V.-H.; Vu, H.-T.; Nguyen, M.-T.; Phan, X.-H. Towards vietnamese question and answer generation: An empirical study. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 2024, 23, 1–28. [Google Scholar] [CrossRef] [Scilit]
  57. Jin, W.; Gao, Y.; Tao, T.; Wang, X.; Wang, N.; Wu, B.; Zhao, B. Veracity-Oriented Context-Aware Large Language Models-Based Prompting Optimization for Fake News Detection. Int. J. Intell. Syst. 2025, 2025, 5920142. [Google Scholar] [CrossRef] [Scilit]
  58. Kanakaris, N.; Ping, H.; Xiao, X.; Ahmed, N.K.; Luceri, L.; Ferrara, E.; Bogdan, P. Network-informed Prompt Engineering against Organized Astroturf Campaigns under Extreme Class Imbalance. Companion Proc. Acm Web Conf. 2025, 37, 33. [Google Scholar]
  59. Jin, W.; Su, D.; Tao, T.; Wang, X.; Wang, N.; Zhao, B. Courtroom-FND: A multi-role fake news detection method based on argument switching-based courtroom debate. J. King Saud Univ. Comput. Inf. Sci. 2025, 37, 33. [Google Scholar] [CrossRef] [Scilit]
  60. Pan, R.; García-Díaz, J.A.; Valencia-García, R. Comparing Fine-Tuning, Zero and Few-Shot Strategies with Large Language Models in Hate Speech Detection in English. CMES-Comput. Model. Eng. Sci. 2024, 140, 2849–2868. [Google Scholar] [CrossRef] [Scilit]
  61. Usman, M.; Ahmad, M.; Sidorov, G.; Gelbukh, I.; Tellez, R.Q. A Large Language Model-Based Approach for Multilingual Hate Speech Detection on Social Media. Computers 2025, 14, 279. [Google Scholar] [CrossRef] [Scilit]
  62. Rathore, B.S.; Chaurasia, S. Fine tuning large language models for hate speech detection in Hinglish and code mixed custom dataset through a socially responsible approach for safer digital platforms. Discov. Sustain. 2025, 6, 1409. [Google Scholar] [CrossRef] [Scilit]
  63. Punnaivanam, M.; Velvizhy, P. Contextual fine-tuning of language models with classifier-driven content moderation for text generation. Entropy 2024, 26, 1114. [Google Scholar] [CrossRef] [Scilit]
  64. Valois, P.H.V.; Souza, L.S.; Shimomoto, E.K.; Fukui, K. Frame representation hypothesis: Multi-token llm interpretability and concept-guided text generation. Trans. Assoc. Comput. Linguist. 2025, 13, 1436–1458. [Google Scholar] [CrossRef] [Scilit]
  65. Rashid, M.R.A.; Das, A.; Hasan, K.F.; Hasan, M.R.; Sultana, M.; Hasan, M.; Islam, R.U.; Tuhin, R.A.; Khan, M.S.H. BERT-KAN: Enhancing bilingual sentiment analysis in bangladeshi E-commerce through fine-tuned large language models. Nat. Lang. Process. J. 2025, 13, 100190. [Google Scholar] [CrossRef] [Scilit]
  66. Sun, Y.; Girón, O.; Ahmed, R. Decoding the Beige Book: LLM-Powered Sentiment Analysis for Real-Time Recession Forecasting. In Proceedings of the 6th ACM International Conference on AI in Finance; Association for Computing Machinery: New York, NY, USA, 2025; pp. 317–325. [Google Scholar]
  67. Chen, W.; Liu, W.; Zheng, J.; Zhang, X. Leveraging large language model as news sentiment predictor in stock markets: A knowledge-enhanced strategy. Discov. Comput. 2025, 28, 74. [Google Scholar] [CrossRef] [Scilit]
  68. Demirtas, E.E.; Lu, Y.; Delarive, L.; Ebrahimi, T. A generative AI approach to cost-effective advertisement based on synthetic images. In Applications of Digital Image Processing XLVIII; SPIE: Cergy-Pontoise, France, 2025; Volume 13605, pp. 350–361. [Google Scholar]
  69. Seo, H.; Jeong, Y.; Choi, Y.S. TAECE: T2I-Adapter with Enhanced Color Expression for Improving Conditional Text-to-Image Generation Capabilities. In 40th ACM/SIGAPP Symposium on Applied Computing; Association for Computing Machinery: New York, NY, USA, 2025; pp. 1180–1187. [Google Scholar]
  70. Davari, S.; Kim, D.; Tohidifar, A. ControlNet-based domain adaptation for synthetic construction images via graphical simulation and generative AI. Autom. Constr. 2025, 180, 106562. [Google Scholar] [CrossRef] [Scilit]
  71. Agarwal, S.; Wood, D.; Murray, B.A.K.; Wei, Y.; Busaidi, A.A.; Kafiabadi, S.; Guilhem, E.; Lynch, J.; Townend, M.; Mazumder, A.; et al. Impact of hospital-specific domain adaptation on BERT-based models to classify neuroradiology reports. Eur. Radiol. 2025, 35, 5299–5313. [Google Scholar] [CrossRef] [Scilit]
  72. Zhong, S.; Ruan, W.; Jin, M.; Li, H.; Wen, Q.; Liang, Y. Time-vlm: Exploring multimodal vision-language models for augmented time series forecasting. arXiv 2025, arXiv:2502.04395. [Google Scholar]
  73. Min, X.; Ye, Y.; Xiong, S.; Chen, X. Computer Vision Meets Generative Models in Agriculture: Technological Advances, Challenges and Opportunities. Appl. Sci. 2025, 15, 7663. [Google Scholar] [CrossRef] [Scilit]
  74. Zitkovich, B.; Yu, T.; Xu, S.; Xu, P.; Xiao, T.; Xia, F.; Wu, J.; Wohlhart, P.; Welker, S.; Wahid, A.; et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning; PMLR: Rockville, MD, USA, 2023; pp. 2165–2183. [Google Scholar]
  75. Shi, L.X.; Ichter, B.; Equi, M.; Ke, L.; Pertsch, K.; Vuong, Q.; Tanner, J.; Walling, A.; Wang, H.; Fusai, N.; et al. Hi robot: Open-ended instruction following with hierarchical vision-language-action models. arXiv 2025, arXiv:2502.19417. [Google Scholar]
  76. Zhao, W.; Ding, P.; Zhang, M.; Gong, Z.; Bai, S.; Zhao, H.; Wang, D. Vlas: Vision-language-action model with speech instructions for customized robot manipulation. arXiv 2025, arXiv:2502.13508. [Google Scholar]
  77. Zhu, J.; Sun, X.; Zhang, Q.; Liu, M. VLA-Grasp: A vision-language-action modeling with cross-modality fusion for task-oriented grasping. Complex Intell. Syst. 2025, 11, 272. [Google Scholar] [CrossRef] [Scilit]
  78. Yu, L.; Lezama, J.; Gundavarapu, N.B.; Versari, L.; Sohn, K.; Minnen, D.; Cheng, Y.; Birodkar, V.; Gupta, A.; Gu, X.; et al. Language Model Beats Diffusion: Tokenizer is Key to Visual Generation. arXiv 2023, arXiv:2310.05737. [Google Scholar]
  79. Zhen, H.; Qiu, X.; Chen, P.; Yang, J.; Yan, X.; Du, Y.; Hong, Y.; Gan, C. 3d-vla: A 3d vision-language-action generative world model. arXiv 2024, arXiv:2403.09631. [Google Scholar]
  80. Niu, Y.; Chen, S.; Chen, Y.; Li, F.; Xu, R.; Da, H. CoFiVLA: Synergistic Coarse-Fine Vision-Language Alignment for Image Aesthetic Assessment. In 33rd ACM International Conference on Multimedia; Association for Computing Machinery: New York, NY, USA, 2025; pp. 3458–3467. [Google Scholar]
  81. He, J.; Yu, C.-Y.; Zhang, W. Explainable and Efficient Hateful Meme Detection using Multimodal Vision-Language Fusion. In Proceedings of the 2025 International Conference on Artificial Intelligence and its Applications; Association for Computing Machinery: New York, NY, USA, 2025; pp. 1–9. [Google Scholar]
  82. Xu, Y.; Jia, H.; Wang, M.; Feng, J.; Xu, X.; Wang, H.; Chen, J.; Zheng, Z.; Yang, X.; Shen, Y.; et al. Enhancing clinical documentation with voice processing and large language models: A study on the LAOS system. npj Digit. Med. 2025, 8, 798. [Google Scholar] [CrossRef] [Scilit]
  83. Fraile Navarro, D.; Kocaballi, A.B.; Berkovsky, S. Understanding Clinician Perceptions of GenAI: A Mixed Methods Analysis of Clinical Documentation Tasks. J. Med. Syst. 2025, 49, 101. [Google Scholar] [CrossRef] [Scilit]
  84. Yang, X.; Xiao, Y.; Liu, D.; Zhang, Y.; Deng, H.; Huang, J.; Shi, H.; Liu, D.; Liang, M.; Jin, X.; et al. Enhancing doctor-patient communication using large language models for pathology report interpretation. BMC Med. Inform. Decis. Mak. 2025, 25, 36. [Google Scholar] [CrossRef] [Scilit]
  85. Weber, M.T.; Noll, R.; Marchl, A.; Facchinello, C.; Grünewaldt, A.; Hügel, C.; Musleh, K.; Wagner, T.O.F.; Storf, H.; Schaaf, J. MedBot vs RealDoc: Efficacy of large language modeling in physician-patient communication for rare diseases. J. Am. Med. Inform. Assoc. 2025, 32, 775–783. [Google Scholar] [CrossRef] [Scilit]
  86. Olayiwola, O.; Yang, R.; Stein, D.; Wang, L.; Foer, D. Management of patients with comorbid asthma and obesity: A large language model evaluation of clinical documentation. J. Allergy Clin. Immunol. Pract. 2025, 14, 138–150.e6. [Google Scholar] [CrossRef] [Scilit]
  87. Gaber, F.; Shaik, M.; Allega, F.; Bilecz, A.J.; Busch, F.; Goon, K.; Franke, V.; Akalin, A. Evaluating large language model workflows in clinical decision support for triage and referral and diagnosis. npj Digit. Med. 2025, 8, 263. [Google Scholar] [CrossRef] [Scilit]
  88. Ong, J.C.L.; Jin, L.; Elangovan, K.; San Lim, G.Y.; Lim, D.Y.Z.; Sng, G.G.R.; Ke, Y.H.; Tung, J.Y.M.; Zhong, R.J.; Koh, C.M.Y.; et al. Large language model as clinical decision support system augments medication safety in 16 clinical specialties. Cell Rep. Med. 2025, 6, 102323. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  89. Shawi, R.E.; Jamel, L. Leveraging ChatGPT and explainable AI for enhancing clinical decision support. Sci. Rep. 2025, 15, 38786. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  90. Tanaka, Y.; Nakata, T.; Aiga, K.; Etani, T.; Muramatsu, R.; Katagiri, S.; Kawai, H.; Higashino, F.; Enomoto, M.; Noda, M.; et al. Performance of generative pretrained transformer on the national medical licensing examination in Japan. PLoS Digit. Health 2024, 3, e0000433. [Google Scholar] [CrossRef] [Scilit]
  91. van Velzen, M.; van der Willigen, R.F.; De Beer, V.J.; de Graaf-Waar, H.I.; Janssen, E.R.C.; van Leeuwen, S.; van der Willigen, M.F.; Van Der Willigen, M.J.; Renardus, G.; El Maaroufi, R.; et al. Privacy-, linguistic-, and information-preserving synthesis of clinical documentation through generative agents. Front. Artif. Intell. 2025, 8, 1644084. [Google Scholar] [CrossRef] [Scilit]
  92. Vilanti, T.; Luiro, K.; Dahlqvist, I.; Piipponen, J.; Hemminki-Reijonen, U.; Tkalcan, S.; Ketamo, H.; Koivisto, J.-M. Contraception-related topics in chat dialogues between healthcare students and generative AI patients: A natural language processing analysis. BMC Med. Educ. 2025, 25, 1458. [Google Scholar] [CrossRef] [Scilit]
  93. Ashfaq, F.; Jhanjhi, N.Z.; Khan, N.A.; Javaid, D.; Masud, M.; Shorfuzzaman, M. Enhancing ECG Report Generation With Domain-Specific Tokenization for Improved Medical NLP Accuracy. IEEE Access 2025, 13, 85493–85506. [Google Scholar] [CrossRef] [Scilit]
  94. Bhadra, S.; Pundir, C.; Mukherjee, M.; Kar, A.; Banerjee, S.; Charoensup, R.; Duangyod, T.; Mukherjee, P.K. Reimagining Ethnopharmacology with Generative AI: Towards Inclusive, Ethical, and Data-Driven Traditional Medicine. Pharmacol. Res. 2025, 221, 108002. [Google Scholar] [CrossRef] [Scilit]
  95. Islayem, R.; Musamih, A.; Salah, K.; Jayaraman, R.; Yaqoob, I. Enhancing medical digital twins within metaverse using blockchain, NFTs and LLMs. Internet Things 2025, 32, 101648. [Google Scholar] [CrossRef] [Scilit]
  96. Shi, D.; Li, J.; Meyer, O.; Bauernhansl, T. Enhancing retrieval-augmented generation for interoperable industrial knowledge representation and inference toward cognitive digital twins. Comput. Ind. 2025, 171, 104330. [Google Scholar] [CrossRef] [Scilit]
  97. Xia, Y.; Xiao, Z.; Jazdi, N.; Weyrich, M. Generation of asset administration shell with large language model agents: Toward semantic interoperability in digital twins in the context of industry 4.0. IEEE Access 2024, 12, 84863–84877. [Google Scholar] [CrossRef] [Scilit]
  98. Sheng, X.; Yu, C.; Cui, X.; Zhou, Y. Large Language Model and Digital Twins Empowered Asynchronous Federated Learning for Secure Data Sharing in Intelligent Labeling. Mathematics 2024, 12, 3550. [Google Scholar] [CrossRef] [Scilit]
  99. Mikołajewska, E.; Mikołajewski, D.; Mikołajczyk, T.; Paczkowski, T. Generative AI in AI-based digital twins for fault diagnosis for predictive maintenance in Industry 4.0/5.0. Appl. Sci. 2025, 15, 3166. [Google Scholar] [CrossRef] [Scilit]
  100. Rojek, I.; Mikołajewski, D.; Piszcz, A.; Małolepsza, O.; Kozielski, M. Role of Generative AI in AI-Based Digital Twins in Industry 5.0 and Evolution to Industry 6.0. Appl. Sci. 2025, 15, 10102. [Google Scholar] [CrossRef] [Scilit]
  101. Ka, K.S.D.; Bosman, I.d.V.; Smit, D. Virtual reality in automotive digital twins: A large language model-assisted content analysis of an expert evaluation. In 27th International Academic Mindtrek Conference; Association for Computing Machinery: New York, NY, USA, 2024; pp. 68–80. [Google Scholar]
  102. Riello, P.; Quille, K.; Jaiswal, R.; Sansone, C. Reimagining Student Success Prediction: Applying LLMs in Educational AI with XAI. In 2024 Conference on Human Centred Artificial Intelligence-Education and Practice; Association for Computing Machinery: New York, NY, USA, 2024; pp. 34–40. [Google Scholar]
  103. Hacini, A.D.; Benabdelouahad, M.; Abassi, I.; Houhou, S.; Boulmerka, A.; Farhi, N. LLM-Assisted Financial Fraud Detection with Reinforcement Learning. Algorithms 2025, 18, 792. [Google Scholar] [CrossRef] [Scilit]
  104. Bakumenko, A.; Hlaváčková-Schindler, K.; Plant, C.; Hubig, N.C. Advancing Anomaly Detection: Non-Semantic Financial Data Encoding With Large Language Models. IEEE Access 2025, 13, 146757–146771. [Google Scholar] [CrossRef] [Scilit]
  105. Zhao, H.; Silverajan, B. Evaluating Cyber Security Dashboards for Smart Cities and Buildings: Enhancing User Modeling with LLMs. In 19th International Conference on Availability, Reliability and Security; Association for Computing Machinery: New York, NY, USA, 2024; pp. 1–10. [Google Scholar]
  106. Hasan, M.M.; ElTazi, N.; Moawad, R.; Eissa, A.H.B. TSB-Forecast: A Short-Term Load Forecasting Model in Smart Cities for Integrating Time Series Embeddings and Large Language Models. IEEE Access 2025, 13, 141694–141716. [Google Scholar] [CrossRef] [Scilit]
  107. He, X.; Li, X.; Kuai, X.; Qiu, Z.; Gu, R. Toward efficient cross-departmental relational data retrieval: A knowledge-constrained LLM-graph framework for smart cities. In 5th International Conference on Internet of Things and Smart City (IoTSC 2025); SPIE: Cergy-Pontoise, France, 2025; Volume 13739, pp. 321–328. [Google Scholar]
  108. Wang, H.; Li, Y.; Zhao, W.; Zhu, H.; Zhang, J.; Wu, X. GSF-LLM: Graph-Enhanced Spatio-Temporal Fusion-Based Large Language Model for Traffic Prediction. Sensors 2025, 25, 6698. [Google Scholar] [CrossRef] [Scilit]
  109. Yang, K.; Shi, H.; Wang, R.; Zhou, L.; Peng, J.; Liu, C. Trend Decomposition Enhanced Large Language Model for Non-Stationary Network Traffic Prediction. In 2025 2nd International Conference on Generative Artificial Intelligence and Information Security; Association for Computing Machinery: New York, NY, USA, 2025; pp. 12–19. [Google Scholar]
  110. Madi, R.; Elayoubi, S.-E.; Do Lago, R.C.; Estanislau, V.O. Large Language Models for Spatio-Temporal Mobile Traffic Predictions. In Workshop on Next-Gen Networks through LLMs, Action Models, and Multi-Agent Systems (NextGenAI), at the Fifth IEEE International Mediterranean Conference on Communications and Networking (IEEE MEDITCOM 2025); IEEE: Piscataway, NJ, USA, 2025. [Google Scholar]
  111. Chamola, V.; Bansal, G.; Das, T.K.; Hassija, V.; Reddy, N.S.S.; Wang, J.; Zeadally, S.; Hussain, A.; Yu, F.R.; Guizani, M.; et al. Beyond reality: The pivotal role of generative ai in the metaverse. arXiv 2023, arXiv:2308.06272. [Google Scholar] [CrossRef] [Scilit]
  112. Essahraui, S.; Lamaakal, I.; El Hamly, I.; Maleh, Y.; Ouahbi, I.; El Makkaoui, K.; Filali Bouami, M.; Pławiak, P.; Alfarraj, O.; Abd El-Latif, A.A. Real-time driver drowsiness detection using facial analysis and machine learning techniques. Sensors 2025, 25, 812. [Google Scholar] [CrossRef] [Scilit]
  113. Lu, Q.; Zhu, L.; Xu, X.; Liu, Y.; Xing, Z.; Whittle, J. A taxonomy of foundation model based systems through the lens of software architecture. In Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering—Software Engineering for AI, Lisbon, Portugal, 15–16 April 2024; pp. 1–6. [Google Scholar]
  114. Graham, O.; Balford, J. A comparative survey of large language models: Foundation, instruction-tuned, and multimodal variants. Preprints 2025, 2025061134. [Google Scholar]
  115. Munikoti, S.; Stewart, I.; Horawalavithana, S.; Kvinge, H.; Emerson, T.; Thompson, S.; Pazdernik, K. Generalist multimodal AI: A review of architectures, challenges and opportunities. Neurocomputing 2026, 676, 132933. [Google Scholar] [CrossRef] [Scilit]
  116. Ciubotaru, B.-I. Generative AI and large language models: A comprehensive scientific review. Preprints 2025, 2025040413. [Google Scholar]
  117. Huang, D.; Yan, C.; Li, Q.; Peng, X. From large language models to large multimodal models: A literature review. Appl. Sci. 2024, 14, 5068. [Google Scholar] [CrossRef] [Scilit]
  118. Ayodele, R.A.; Oseghale, I.; Osayi, E.; Ehisuoria, A.; Adebayo, N.K.; Oluwole, O.A. Multimodal foundation models for unified image, video and text understanding. Open Access Res. J. Sci. Technol. 2024, 12, 081–093. [Google Scholar] [CrossRef] [Scilit]
  119. Yijun, H.; Zhihao, S. An in-depth survey of multimodal foundation models and their challenges. Eng. Arch. 2025. [Google Scholar]
  120. Salimpour, S.; Fu, L.; Rachwał, K.; Bertrand, P.; O’Sullivan, K.; Jakob, R.; Keramat, F.; Militano, L.; Toffetti, G.; Edelman, H.; et al. Towards embodied agentic AI: Review and classification of LLM- and VLM-driven robot autonomy and interaction. arXiv 2025, arXiv:2508.05294. [Google Scholar] [CrossRef] [Scilit]
  121. Tang, S.; Wang, Y.; Chen, L.; Wang, Y.; Peng, S.; Xu, D.; Ouyang, W. Human-centric foundation models: Perception, generation and agentic modeling. arXiv 2025, arXiv:2502.08556. [Google Scholar] [CrossRef] [Scilit]
  122. Khan, M.T.; Waheed, A. Foundation model driven robotics: A comprehensive review. arXiv 2025, arXiv:2507.10087. [Google Scholar] [CrossRef] [Scilit]
  123. Awais, M.; Naseer, M.; Khan, S.; Anwer, R.M.; Cholakkal, H.; Shah, M.; Yang, M.-H.; Khan, F.S. Foundation models defining a new era in vision: A survey and outlook. IEEE Trans. Pattern Anal. Mach. Intell. 2025, 47, 2245–2264. [Google Scholar] [CrossRef] [Scilit]
  124. Khan, W.; Leem, S.; See, K.B.; Wong, J.K.; Zhang, S.; Fang, R. A comprehensive survey of foundation models in medicine. IEEE Rev. Biomed. Eng. 2025, 19, 283–304. [Google Scholar] [CrossRef] [Scilit]
  125. Buess, L.; Keicher, M.; Navab, N.; Maier, A.; Tayebi Arasteh, S. From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine. Biomed. Eng. Lett. 2025, 15, 845–863. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  126. Hu, Y.; Xie, Q.; Jain, V.; Francis, J.; Patrikar, J.; Keetha, N.; Kim, S.; Xie, Y.; Zhang, T.; Fang, H.-S.; et al. Toward general-purpose robots via foundation models: A survey and meta-analysis. arXiv 2023, arXiv:2312.08782. [Google Scholar] [CrossRef] [Scilit]
  127. Wang, R.; Chen, Z.S. Large-scale foundation models and generative AI for BigData neuroscience. Neurosci. Res. 2025, 215, 3–14. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  128. Wang, S.; Zhou, X.; Li, C.; Wang, S.; Li, Y.; Tan, T.; Zheng, H. Generative artificial intelligence in medical imaging: Foundations, progress, and clinical translation. Research 2025, 8, 1029. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  129. Panayides, A.S.; Chen, H.; Filipovic, N.D.; Geroski, T.; Hou, J.; Lekadir, K.; Marias, K.; Matsopoulos, G.K.; Papanastasiou, G.; Sarder, P.; et al. Position paper: Artificial intelligence in medical image analysis: Advances, clinical translation, and emerging frontiers. IEEE J. Biomed. Health Inform. 2025, 30, 1187–1202. [Google Scholar] [CrossRef] [Scilit]
  130. Balachandran, V.; Chen, J.; Joshi, N.; Nushi, B.; Palangi, H.; Salinas, E.; Vineet, V.; Woffinden-Luey, J.; Yousefi, S. Eureka: Evaluating and understanding large foundation models. arXiv 2024, arXiv:2409.10566. [Google Scholar] [CrossRef] [Scilit]
  131. Chafekar, T.; Goindani, A.; Liang, P.P.; Mathur, L.; Morency, L.-P.; Salakhutdinov, R.; Yu, H. HEMM: Holistic evaluation of multimodal foundation models. In Advances in Neural Information Processing Systems 37; Curran Associates: Red Hook, NY, USA, 2024; pp. 42899–42940. [Google Scholar]
  132. Daniels, B. Designing a robust, explainable, and bias-aware foundational autorater framework for cross-domain automatic evaluation of generative AI systems. Preprints 2025, 2025040413. [Google Scholar]
  133. Shi, Z.; Wang, Z.; Fan, H.; Yin, Z.; Sheng, L.; Qiao, Y.; Shao, J. CHEF: A comprehensive evaluation framework for standardized assessment of multimodal large language models. arXiv 2023, arXiv:2311.02692. [Google Scholar]
  134. Kanithi, P.K.; Christophe, C.; Pimentel, M.A.F.; Raha, T.; Saadi, N.; Javed, H.; Maslenkova, S.; Hayat, N.; Rajan, R.; Khan, S. MEDIC: Towards a comprehensive framework for evaluating LLMs in clinical applications. arXiv 2024, arXiv:2409.07314. [Google Scholar] [CrossRef] [Scilit]
  135. Tam, T.Y.C.; Sivarajkumar, S.; Kapoor, S.; Stolyar, A.V.; Polanska, K.; McCarthy, K.R.; Osterhoudt, H.; Wu, X.; Visweswaran, S.; Fu, S.; et al. A framework for human evaluation of large language models in healthcare derived from literature review. npj Digit. Med. 2024, 7, 258. [Google Scholar] [CrossRef] [Scilit]
  136. Gupta, N.; Koppisetti, P.; Lakkaraju, K.; Srivastava, B. GAICo: A deployed and extensible framework for evaluating diverse and multimodal generative AI outputs. arXiv 2025, arXiv:2508.16753. [Google Scholar] [CrossRef] [Scilit]
  137. Fleurence, R.L.; Dawoud, D.; Bian, J.; Higashi, M.K.; Wang, X.; Xu, H.; Chhatwal, J.; Ayer, T. ELEVATE-GenAI: Reporting guidelines for the use of large language models in health economics and outcomes research: An ISPOR Working Group on Generative AI report. Value Health 2025, 28, 1611–1625. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  138. Grolleau, F.; Alsentzer, E.; Keyes, T.; Chung, P.; Swaminathan, A.; Aali, A.; Hom, J.; Huynh, T.; Lew, T.; Liang, A.; et al. MedFactEval and MedAgentBrief: A framework and workflow for generating and evaluating factual clinical summaries. In Proceedings of the Pacific Symposium on Biocomputing 2026, Kohala Coast, HI, USA, 3–7 January 2026; pp. 388–399; World Scientific: Singapore, 2025. [Google Scholar]
  139. Abbasian, M.; Khatibi, E.; Azimi, I.; Oniani, D.; Shakeri Hossein Abad, Z.; Thieme, A.; Sriram, R.; Yang, Z.; Wang, Y.; Lin, B.; et al. Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI. npj Digit. Med. 2024, 7, 82. [Google Scholar] [CrossRef] [Scilit]
  140. Liang, J.; Zhou, S.; Wang, K. OmniBench-RAG: A multi-domain evaluation platform for retrieval-augmented generation tools. arXiv 2025, arXiv:2508.05650. [Google Scholar]
  141. Manes, I.; Ronn, N.; Cohen, D.; Ber, R.I.; Horowitz-Kugler, Z.; Stanovsky, G. K-QA: A real-world medical Q&A benchmark. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, Bangkok, Thailand, 15 August 2024; pp. 277–294. [Google Scholar]
  142. Bhattacharjee, A. Overview of model evaluation. In A Practical Guide to Generative AI Using Amazon Bedrock: Building, Deploying, and Securing Generative AI Applications; Springer: Cham, Switzerland, 2025; pp. 283–309. [Google Scholar]
  143. Strubell, E.; Ganesh, A.; McCallum, A. Energy and policy considerations for deep learning in NLP. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL); Association for Computational Linguistics: Stroudsburg, PA, USA, 2019. [Google Scholar]
  144. Essahraui, S.; Lamaakal, I.; El Makkaoui, K.; Ouahbi, I.; Bouami, M.F.; Maleh, Y. Kolmogorov–Arnold networks: Overview of architectures and use cases. In Proceedings of the 2025 International Conference on Circuit, Systems and Communication (ICCSC); IEEE: Piscataway, NJ, USA, 2025; pp. 1–6. [Google Scholar]
  145. Fedus, W.; Zoph, B.; Shazeer, N. Switch transformers: Scaling to trillion parameter models. arXiv 2021, arXiv:2101.03961. [Google Scholar]
  146. Bender, E.M.; Gebru, T.; McMillan-Major, A.; Shmitchell, S. On the dangers of stochastic parrots. In ACM Conference on Fairness, Accountability, and Transparency (FAccT); Association for Computing Machinery: New York, NY, USA, 2021. [Google Scholar]
  147. Ji, Z.; Lee, N.; Frieske, R.; Yu, T.; Su, D.; Xu, Y.; Ishii, E.; Bang, Y.J.; Madotto, A.; Fung, P. Survey of hallucination in natural language generation. ACM Comput. Surv. 2023, 55, 248. [Google Scholar] [CrossRef] [Scilit]
  148. Floridi, L.; Cowls, J.; Beltrametti, M.; Chatila, R.; Chazerand, P.; Dignum, V.; Luetge, C.; Madelin, R.; Pagallo, U.; Rossi, F.; et al. AI4People: An ethical framework for a good AI society. Minds Mach. 2018, 28, 689–707. [Google Scholar] [CrossRef] [Scilit]
  149. Zou, A.; Wang, Z.; Carlini, N.; Nasr, M.; Kolter, J.Z.; Fredrikson, M. Universal and transferable adversarial attacks on aligned language models. arXiv 2023, arXiv:2307.15043. [Google Scholar] [CrossRef] [Scilit]
  150. Pearce, H.; Ahmad, B.; Tan, B.; Dolan-Gavitt, B.; Karri, R. Asleep at the keyboard? Assessing the security of GitHub Copilot. Proc. IEEE Symp. Secur. Priv. 2022, 68, 96–105. [Google Scholar]
  151. Carlini, N.; Tramèr, F.; Wallace, E.; Jagielski, M.; Herbert-Voss, A.; Lee, K.; Roberts, A.; Brown, T.; Song, D.; Erlingsson, Ú.; et al. Extracting training data from large language models. In USENIX Security Symposium; USENIX Association: Berkeley, CA, USA, 2021. [Google Scholar]
  152. Liu, Y.; Iter, D.; Xu, Y.; Wang, S.; Xu, R.; Zhu, C. G-Eval: NLG evaluation using GPT-4. arXiv 2023, arXiv:2303.16634. [Google Scholar] [CrossRef] [Scilit]
  153. Raji, I.D.; Xu, P.; Honigsberg, C.; Ho, D. AI benchmarks and the illusion of progress. In ACM Conference on Fairness, Accountability, and Transparency (FAccT); Association for Computing Machinery: New York, NY, USA, 2022. [Google Scholar]
  154. Banner, R.; Nahshan, Y.; Soudry, D. Post training 4-bit quantization of convolutional networks for rapid-deployment. In Advances in Neural Information Processing Systems (NeurIPS); Curran Associates: Red Hook, NY, USA, 2019. [Google Scholar]
  155. Frankle, J.; Carbin, M. The lottery ticket hypothesis. In International Conference on Learning Representations (ICLR); ICLR: Appleton, WI, USA, 2019. [Google Scholar]
  156. Hinton, G.; Vinyals, O.; Dean, J. Distilling the knowledge in a neural network. arXiv 2015, arXiv:1503.02531. [Google Scholar] [CrossRef] [Scilit]
  157. Hu, E.J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Chen, W. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR); ICLR: Appleton, WI, USA, 2022. [Google Scholar]
  158. Belinkov, Y.; Glass, J. Analysis methods in neural language processing. Trans. Assoc. Comput. Linguist. 2019, 7, 49–72. [Google Scholar] [CrossRef] [Scilit]
  159. Olah, C.; Cammarata, N.; Schubert, L.; Goh, G.; Petrov, M.; Carter, S. Zoom in: An introduction to circuits. Distill 2020, 5, e00024.001. [Google Scholar] [CrossRef] [Scilit]
  160. Alayrac, J.-B.; Donahue, J.; Luc, P.; Miech, A.; Barr, I.; Hasson, Y.; Lenc, K.; Mensch, A.; Millican, K.; Reynolds, M.; et al. Flamingo: A visual language model for few-shot learning. In Advances in Neural Information Processing Systems (NeurIPS); Curran Associates: Red Hook, NY, USA, 2022. [Google Scholar]
  161. Driess, D.; Xia, F.; Sajjadi, M.S.M.; Lynch, C.; Chowdhery, A.; Ichter, B.; Wahid, A.; Tompson, J.; Vuong, Q.; Yu, T.; et al. PaLM-E: An embodied multimodal language model. In Proceedings of the International Conference on Machine Learning (ICML); PMLR: Rockville, MD, USA, 2023. [Google Scholar]
  162. Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.; Cao, Y. ReAct: Synergizing reasoning and acting in language models. In Advances in Neural Information Processing Systems (NeurIPS); Curran Associates: Red Hook, NY, USA, 2023. [Google Scholar]
  163. Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.A.; Lacroix, T.; Rozière, B.; Hambro, N.G.; Azhar, F.; Rodriguez, A.; et al. LLaMA: Open and efficient foundation language models. arXiv 2023, arXiv:2302.13971. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.