applsci-logo

Journal Browser

Journal Browser

Design and Application of High-Performance Computing Systems

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Electrical, Electronics and Communications Engineering".

Deadline for manuscript submissions: closed (20 September 2024) | Viewed by 1626

Special Issue Editors


E-Mail Website
Guest Editor
College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China
Interests: parallel programming; heterogeneous computing; opencl; compilers

E-Mail Website
Guest Editor
School of Computing, University of Leeds, Leeds LS2 9JT, UK
Interests: distributed systems; computing systems; compilers; machine learning; artificial intelligence; parallel programming; programming languages

Special Issue Information

Dear Colleagues,

High-performance computing (HPC) systems have revolutionized the way we process and analyze vast amounts of data, enabling breakthroughs in various scientific, engineering, and industrial domains. These systems, characterized by their exceptional processing power and parallel computing capabilities, have become indispensable tools for tackling complex computational problems that were once considered intractable.

The design and application of high-performance computing systems have witnessed significant advancements in recent years, leading to improved performance, energy efficiency, and scalability. This Special Issue aims to explore the latest developments and applications of high-performance computing systems, shedding light on their potential to address real-world challenges across diverse fields.

We encourage submissions that cover a wide range of topics, including but not limited to novel architectures, parallel programming, scalable algorithms, software frameworks/tools, and the applications of high-performance computing systems in solving complex problems.

We hope that this Special Issue will contribute to advancing state-of-the-art high-performance computing systems and inspire further research and development in this exciting field.

Dr. Jianbin Fang
Prof. Dr. Zheng Wang
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (1 paper)

Order results
Result details
Select all
Export citation of selected articles as:

Research

20 pages, 600 KiB  
Article
Optimizing Large Language Models on Multi-Core CPUs: A Case Study of the BERT Model
by Lanxin Zhao, Wanrong Gao and Jianbin Fang
Appl. Sci. 2024, 14(6), 2364; https://doi.org/10.3390/app14062364 - 11 Mar 2024
Viewed by 1306
Abstract
The BERT model is regarded as the cornerstone of various pre-trained large language models that have achieved promising results in recent years. This article investigates how to optimize the BERT model in terms of fine-tuning speed and prediction accuracy, aiming to accelerate [...] Read more.
The BERT model is regarded as the cornerstone of various pre-trained large language models that have achieved promising results in recent years. This article investigates how to optimize the BERT model in terms of fine-tuning speed and prediction accuracy, aiming to accelerate the execution of the BERT model on a multi-core processor and improve its prediction accuracy in typical downstream natural language processing tasks. Our contributions are two-fold. First, we port and parallelize the fine-tuning training of the BERT model on a multi-core shared-memory processor. We port the BERT model onto a multi-core processor platform to accelerate the fine-tuning training process of the model for downstream tasks. Second, we improve the prediction performance of typical downstream natural language processing tasks through fine-tuning the model parameters. We select five typical downstream natural language processing tasks (CoLA, SST-2, MRPC, RTE, and WNLI) and perform optimization on the multi-core platform, taking the hyperparameters of batch size, learning rate, and training epochs into account. Our experimental results show that, by increasing the number of CPUs and the number of threads, the model training time can be significantly reduced. We observe that the reduced time is primarily concentrated in the self-attention mechanism. Our further experimental results show that setting reasonable hyperparameters can improve the accuracy of the BERT model when applied to downstream tasks and that appropriately increasing the batch size under conditions of sufficient computing resources can significantly reduce training time. Full article
(This article belongs to the Special Issue Design and Application of High-Performance Computing Systems)
Show Figures

Figure 1

Back to TopTop