AI Test

A special issue of Electronics (ISSN 2079-9292). This special issue belongs to the section "Artificial Intelligence".

Deadline for manuscript submissions: closed (31 January 2024) | Viewed by 3474

Special Issue Editors


E-Mail Website
Guest Editor
School of Engineering, Computing and Mathematics, Oxford Brookes University, Oxford OX33 1HX, UK
Interests: software engineering; software development methodology; automated and intelligent software development tools
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Department of Information Science, University of North Texas, Denton, TX 76203, USA
Interests: machine learning; software engineering; legal intelligence; biomedical computation
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Grenoble INP*, University Grenoble Alpes, LCIS, 26000 Valence, France
Interests: security and trust in embedded and distributed, autonomous applications and systems; test and security of mobile applications; system-level online fault diagnosis (hardware components, middleware, software components)

Special Issue Information

Dear Colleagues,

With the rapid growth of AI applications, especially machine learning applications, software engineering is currently confronted with a grave challenge to test and ensure the quality of such computer applications. Many traditional software testing techniques, tools and methodologies cannot be simply applied to such applications or become less effective and efficient due to the fundamental differences between machine learning models and traditional coded programs.  On the other hand, AI techniques offer new approaches to solve software testing problems. The scope of this journal Special Issue covers both “testing for AI” (i.e., testing and quality assurance for AI applications) and “testing by AI” (i.e., testing and quality assurance of software applications by employing AI techniques).

It has been four years since the successful launch of IEEE International Conferences on Artificial Intelligence Testing (IEEE AITest) in April 2019. The AITest 2023 conference will celebrate its fifth edition in July 2023 in Athens, Greece. In this period of time, both “testing for AI” and “testing by AI” have been developed rapidly. More importantly, a new direction of research on “testing x AI”, which represents the interplay between “testing for AI” and “testing by AI”, starts to emerge, which was one of the key motivations of the IEEE AITest conferences. A rapidly increasing number of research works have been published on the subject. Yet, grave challenges remain in the practice as well as in the research community.

The journal Special Issue aims at setting a milestone in this rapid growing subject area with archive articles in the journal to reflect the current state of the art in the research and/or current practices, as well as a set of survey, review and visionary research papers that summarizes the results so far, analyses the challenges ahead and sets a roadmap for the future directions. The best papers of the AITest 2023 will be invited to submit a revised and extended paper with at least 30% new material than the conference paper. It will also combine with invited papers for authors who have made significant contributions to the subject area and open call for papers for new contributions. All submissions will be rigorously reviewed according to the standard of internationally top ranked journals fairly and scientifically according to the criteria of (a) the scientific and technological soundness, (b) the maturity of the research work, (c) the relevance to the theme of the Special Issue, (d) the timeliness of the work, (e) the significance of the contribution, and (f) the presentation quality.

Topics of interest include, but are not limited to:

  • Quality models, quality attributes and metrics for AI applications, such as robustness, fairness, reliability and performance.
  • Testing methods, techniques and tools for various aspects and activities of testing and quality assurance, especially for test case generation, test oracle, and test adequacy measurements.
  • Test automation environment and platforms for AI applications and by employing AI techniques.
  • Domain specific testing techniques and methods for various special domains of AI applications, such as AI applications in natural language processing such as ChatGPT, image recognitions, time series and Internet of Things data analysis, medical and healthcare, robotics, software code generation/debugging/design, etc.
  • Testing and quality assurance for various specific AI techniques, such as clusters and classifiers, regression machine learning models, deep neural networks, big models, recurrent neural networks, etc.

Prof. Dr. Hong Zhu
Prof. Dr. Junhua Ding
Prof. Dr. Aktouf Oum-El-Kheir
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • artificial intelligence
  • software testing
  • machine learning
  • software engineering

Published Papers (3 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

14 pages, 917 KiB  
Article
Improving Text Classification with Large Language Model-Based Data Augmentation
by Huanhuan Zhao, Haihua Chen, Thomas A. Ruggles, Yunhe Feng, Debjani Singh and Hong-Jun Yoon
Electronics 2024, 13(13), 2535; https://doi.org/10.3390/electronics13132535 - 28 Jun 2024
Viewed by 677
Abstract
Large Language Models (LLMs) such as ChatGPT possess advanced capabilities in understanding and generating text. These capabilities enable ChatGPT to create text based on specific instructions, which can serve as augmented data for text classification tasks. Previous studies have approached data augmentation (DA) [...] Read more.
Large Language Models (LLMs) such as ChatGPT possess advanced capabilities in understanding and generating text. These capabilities enable ChatGPT to create text based on specific instructions, which can serve as augmented data for text classification tasks. Previous studies have approached data augmentation (DA) by either rewriting the existing dataset with ChatGPT or generating entirely new data from scratch. However, it is unclear which method is better without comparing their effectiveness. This study investigates the application of both methods to two datasets: a general-topic dataset (Reuters news data) and a domain-specific dataset (Mitigation dataset). Our findings indicate that: 1. ChatGPT generated new data consistently enhanced model’s classification results for both datasets. 2. Generating new data generally outperforms rewriting existing data, though crafting the prompts carefully is crucial to extract the most valuable information from ChatGPT, particularly for domain-specific data. 3. The augmentation data size affects the effectiveness of DA; however, we observed a plateau after incorporating 10 samples. 4. Combining the rewritten sample with new generated sample can potentially further improve the model’s performance. Full article
(This article belongs to the Special Issue AI Test)
Show Figures

Figure 1

26 pages, 1502 KiB  
Article
Robustness Assessment of AI-Based 2D Object Detection Systems: A Method and Lessons Learned from Two Industrial Cases
by Anne-Laure Wozniak, Sergio Segura and Raúl Mazo
Electronics 2024, 13(7), 1368; https://doi.org/10.3390/electronics13071368 - 4 Apr 2024
Viewed by 805
Abstract
The reliability of AI-based object detection models has gained interest with their increasing use in safety-critical systems and the development of new regulations on artificial intelligence. To meet the need for robustness evaluation, several authors have proposed methods for testing these models. However, [...] Read more.
The reliability of AI-based object detection models has gained interest with their increasing use in safety-critical systems and the development of new regulations on artificial intelligence. To meet the need for robustness evaluation, several authors have proposed methods for testing these models. However, applying these methods in industrial settings can be difficult, and several challenges have been identified in practice in the design and execution of tests. There is, therefore, a need for clear guidelines for practitioners. In this paper, we propose a method and guidelines for assessing the robustness of AI-based 2D object detection systems, based on the Goal Question Metric approach. The method defines the overall robustness testing process and a set of recommended metrics to be used at each stage of the process. We developed and evaluated the method through action research cycles, based on two industrial cases and feedback from practitioners. Thus, the resulting method addresses issues encountered in practice. A qualitative evaluation of the method by practitioners was also conducted to provide insights that can guide future research on the subject. Full article
(This article belongs to the Special Issue AI Test)
Show Figures

Figure 1

20 pages, 4090 KiB  
Article
Uncertainty Quantification of Machine Learning Model Performance via Anomaly-Based Dataset Dissimilarity Measures
by Gabriele Incorvaia, Darryl Hond and Hamid Asgari
Electronics 2024, 13(5), 939; https://doi.org/10.3390/electronics13050939 - 29 Feb 2024
Viewed by 872
Abstract
The use of Machine Learning (ML) models as predictive tools has increased dramatically in recent years. However, data-driven systems (such as ML models) exhibit a degree of uncertainty in their predictions. In other words, they could produce unexpectedly erroneous predictions if the uncertainty [...] Read more.
The use of Machine Learning (ML) models as predictive tools has increased dramatically in recent years. However, data-driven systems (such as ML models) exhibit a degree of uncertainty in their predictions. In other words, they could produce unexpectedly erroneous predictions if the uncertainty stemming from the data, choice of model and model parameters is not taken into account. In this paper, we introduce a novel method for quantifying the uncertainty of the performance levels attained by ML classifiers. In particular, we investigate and characterize the uncertainty of model accuracy when classifying out-of-distribution data that are statistically dissimilar from the data employed during training. A main element of this novel Uncertainty Quantification (UQ) method is a measure of the dissimilarity between two datasets. We introduce an innovative family of data dissimilarity measures based on anomaly detection algorithms, namely the Anomaly-based Dataset Dissimilarity (ADD) measures. These dissimilarity measures process feature representations that are derived from the activation values of neural networks when supplied with dataset items. The proposed UQ method for classification performance employs these dissimilarity measures to estimate the classifier accuracy for unseen, out-of-distribution datasets, and to give an uncertainty band for those estimates. A numerical analysis of the efficacy of the UQ method is conducted using standard Artificial Neural Network (ANN) classifiers and public domain datasets. The results obtained generally demonstrate that the amplitude of the uncertainty band associated with the estimated accuracy values tends to increase as the data dissimilarity measure increases. Overall, this research contributes to the verification and run-time performance prediction of systems composed of ML-based elements. Full article
(This article belongs to the Special Issue AI Test)
Show Figures

Figure 1

Back to TopTop