Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.3
2.5
Journals
Applied Sciences
Special Issues
Transformer Deep Learning Architectures: Advances and Applications
applsci-logo
Submit to Applied Sciences Review for Applied Sciences Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Transformer Deep Learning Architectures: Advances and Applications

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: closed (25 April 2025) | Viewed by 18159

Special Issue Editor

Dr. Ting Xiao

E-Mail Website
Guest Editor
Department of Information Science, Department of Computer Science and Engineering (Joint Appointment), University of North Texas, Denton, TX 76203, USA
Interests: big data/data science; machine/deep learning; software development; health informatics; sensor information extraction

Special Issue Information

Dear Colleagues,

This Special Issue spotlights the advancements in and applications of Transformer-based deep learning architectures. Transformers have significantly influenced artificial intelligence (AI), particularly natural language processing (NLP), with their innovative approach to handling sequential data. This Special Issue explores the core components of these architectures, including their self-attention mechanism and positional encoding, and discusses recent developments that enhance efficiency, interpretability, and scalability.

The Special Issue also delves into the broad spectrum of applications of Transformers, ranging from traditional tasks such as text summarization, machine translation, and sentiment analysis, to innovative utilizations in language generation and conversational AI, including chatbots and dialogue systems like ChatGPT. Beyond these conventional domains, the Special Issue also highlights breakthrough applications in emerging fields such as computer vision, bioinformatics, health informatics and climate modeling. It provides insight into how models such as BERT and GPT are changing paradigms across various sectors.

Moreover, this Special Issue tackles the existing challenges in utilizing Transformer models, giving readers a well-rounded view of this field. It outlines potential future directions, providing a roadmap for continued innovation. This comprehensive guide offers invaluable insights to researchers, students, and practitioners interested in the cutting edge of deep learning technology.

Dr. Ting Xiao
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • AI
  • NLP
  • transformer
  • self-attention
  • ChatGPT
  • BERT
  • GPT
  • deep learning

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (4 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

15 pages, 5620 KiB  
Article
Automated Vertebral Bone Quality Determination from T1-Weighted Lumbar Spine MRI Data Using a Hybrid Convolutional Neural Network–Transformer Neural Network
by Kristian Stojšić, Dina Miletić Rigo and Slaven Jurković
Appl. Sci. 2024, 14(22), 10343; https://doi.org/10.3390/app142210343 - 11 Nov 2024
Viewed by 1180
Abstract
Vertebral bone quality (VBQ) is a promising new method that can improve screening for osteoporosis. The drawback of the current method is that it requires manual determination of the regions of interest (ROIs) of vertebrae and cerebrospinal fluid (CSF) by a radiologist. In [...] Read more.
Vertebral bone quality (VBQ) is a promising new method that can improve screening for osteoporosis. The drawback of the current method is that it requires manual determination of the regions of interest (ROIs) of vertebrae and cerebrospinal fluid (CSF) by a radiologist. In this work, an automatic method for determining the VBQ is proposed, in which the ROIs are obtained using a trained neural network model. A large, publicly available dataset of sagittal lumbar spine MRI images with ground truth segmentations was used to train a BRAU-Net++ hybrid CNN–transformer neural network. The performance of the trained model was evaluated using the dice similarity coefficient (DSC), accuracy, precision, recall and intersection-over-union (IoU) metrics. The trained model performed similarly to state-of-the-art lumbar spine segmentation models, with an average DSC value of 0.914 ± 0.007 for the vertebrae and 0.902 for the spinal canal. Four different methods of VBQ determination with automatic segmentation are presented and compared with one-way ANOVA. These methods use different algorithms for CSF extraction from the segmentation of the spinal canal using T1- and T2-weighted image data and applying erosion to the vertebral ROI to avoid a sharp change in SI at the edge of the vertebral body. Full article
(This article belongs to the Special Issue Transformer Deep Learning Architectures: Advances and Applications)
►▼ Show Figures

Figure 1

36 pages, 40330 KiB  
Article
Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency
by Sakib Shahriar, Brady D. Lund, Nishith Reddy Mannuru, Muhammad Arbab Arshad, Kadhim Hayawi, Ravi Varma Kumar Bevara, Aashrith Mannuru and Laiba Batool
Appl. Sci. 2024, 14(17), 7782; https://doi.org/10.3390/app14177782 - 3 Sep 2024
Cited by 18 | Viewed by 10434
Abstract
As large language models (LLMs) continue to advance, evaluating their comprehensive capabilities becomes significant for their application in various fields. This research study comprehensively evaluates the language, vision, speech, and multimodal capabilities of GPT-4o. The study employs standardized exam questions, reasoning tasks, and [...] Read more.
As large language models (LLMs) continue to advance, evaluating their comprehensive capabilities becomes significant for their application in various fields. This research study comprehensively evaluates the language, vision, speech, and multimodal capabilities of GPT-4o. The study employs standardized exam questions, reasoning tasks, and translation assessments to assess the model’s language capability. Additionally, GPT-4o’s vision and speech capabilities are tested through image classification and object-recognition tasks, as well as accent classification. The multimodal evaluation assesses the model’s performance in integrating visual and linguistic data. Our findings reveal that GPT-4o demonstrates high accuracy and efficiency across multiple domains in language and reasoning capabilities, excelling in tasks that require few-shot learning. GPT-4o also provides notable improvements in multimodal tasks compared to its predecessors. However, the model shows variability and faces limitations in handling complex and ambiguous inputs, particularly in audio and vision capabilities. This paper highlights the need for more comprehensive benchmarks and robust evaluation frameworks, encompassing qualitative assessments involving human judgment, as well as error analysis. Future work should focus on expanding datasets, investigating prompt-based assessment, and enhancing few-shot learning techniques to test the model’s practical applicability and performance in real-world scenarios. Full article
(This article belongs to the Special Issue Transformer Deep Learning Architectures: Advances and Applications)
►▼ Show Figures

Figure 1

31 pages, 15142 KiB  
Article
Scaling Implicit Bias Analysis across Transformer-Based Language Models through Embedding Association Test and Prompt Engineering
by Ravi Varma Kumar Bevara, Nishith Reddy Mannuru, Sai Pranathi Karedla and Ting Xiao
Appl. Sci. 2024, 14(8), 3483; https://doi.org/10.3390/app14083483 - 20 Apr 2024
Cited by 3 | Viewed by 2420
Abstract
In the evolving field of machine learning, deploying fair and transparent models remains a formidable challenge. This study builds on earlier research, demonstrating that neural architectures exhibit inherent biases by analyzing a broad spectrum of transformer-based language models from base to x-large configurations. [...] Read more.
In the evolving field of machine learning, deploying fair and transparent models remains a formidable challenge. This study builds on earlier research, demonstrating that neural architectures exhibit inherent biases by analyzing a broad spectrum of transformer-based language models from base to x-large configurations. This article investigates movie reviews for genre-based bias, which leverages the Word Embedding Association Test (WEAT), revealing that scaling models up tends to mitigate bias, with larger models showing up to a 29% reduction in prejudice. Alternatively, this study also underscores the effectiveness of prompt-based learning, a facet of prompt engineering, as a practical approach to bias mitigation, as this technique reduces genre bias in reviews by more than 37% on average. This suggests that the refinement of development practices should include the strategic use of prompts in shaping model outputs, highlighting the crucial role of ethical AI integration to weave fairness seamlessly into the core functionality of transformer models. Despite the basic nature of the prompts employed in this research, this highlights the possibility of embracing structured prompt engineering to create AI systems that are ethical, equitable, and more responsible for their actions. Full article
(This article belongs to the Special Issue Transformer Deep Learning Architectures: Advances and Applications)
►▼ Show Figures

Figure 1

20 pages, 6446 KiB  
Article
ChatGPT Translation of Program Code for Image Sketch Abstraction
by Yulia Kumar, Zachary Gordon, Oluwatunmise Alabi, Jenny Li, Kathryn Leonard, Linda Ness and Patricia Morreale
Appl. Sci. 2024, 14(3), 992; https://doi.org/10.3390/app14030992 - 24 Jan 2024
Cited by 3 | Viewed by 2479
Abstract
In this comprehensive study, a novel MATLAB to Python (M-to-PY) conversion process is showcased, specifically tailored for an intricate image skeletonization project involving fifteen MATLAB files and a large dataset. The central innovation of this research is the adept use of ChatGPT-4 as [...] Read more.
In this comprehensive study, a novel MATLAB to Python (M-to-PY) conversion process is showcased, specifically tailored for an intricate image skeletonization project involving fifteen MATLAB files and a large dataset. The central innovation of this research is the adept use of ChatGPT-4 as an AI assistant, pivotal in crafting a prototype M-to-PY converter. This converter’s capabilities were thoroughly evaluated using a set of test cases generated by the Bard bot, ensuring a robust and effective tool. The culmination of this effort was the development of the Skeleton App, adept at image sketching and skeletonization. This live and publicly available app underscores the enormous potential of AI in enhancing the transition of scientific research from MATLAB to Python. The study highlights the blend of AI’s computational prowess and human ingenuity in computational research, making significant strides in AI-assisted scientific exploration and tool development. Full article
(This article belongs to the Special Issue Transformer Deep Learning Architectures: Advances and Applications)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-4
Back to TopTop