A Modular AutoML Framework for End-to-End Machine Learning Automation

Abstract

As an automated alternative to the complexity and resource intensive task of building a machine learning (ML) pipeline, AutoML offers substantial value. Moreover, given the growing number of application areas requiring ML solutions, but having limited technical expertise, the need for AutoML is increasing. The authors describe a novel and modular AutoML pipeline built using object-oriented Python, designed for both regression and classification problems. Unlike other established libraries (TPOT, Auto-sklearn, Hyperopt-sklearn) this new framework provides structure to the output formats (JSON, YAML etc.) used for backend and API integration purposes. Rather than a GUI-based platform (low-code or otherwise) the authors propose a developer oriented/Code Driven AutoML pipeline. Additionally, it includes interpretability through VIF based Feature Engineering and increased extensibility. The experimental results provided by the authors are based upon public insurance data sets and demonstrate that their system performs at least on par with, and in several cases surpasses, the baseline systems tested, and does so in a manner that provides greater modularity and easier deployment. Therefore, this study demonstrates a lightweight, real world ready solution to provide an effective AutoML solution for use in a variety of application areas including NLP, computer vision, and Web-Based Machine Learning Applications.

1. Introduction

The rapid dissemination of machine learning (ML) into many domains, such as health care, finance, marketing, and engineering, has both exhibited impressive potential to predict future events and raised concerns about accessibility for end-users without prior expertise. While traditional ML pipelines require an expert’s domain-specific knowledge, a significant amount of manual effort is needed to perform feature engineering, hyper-parameter tuning, and model deployment. This results in workflows which can be very complex, time consuming and demand large amounts of resources. As a result, there have been increasing interests in the area of Automated Machine Learning (AutoML); an AutoML paradigm seeks to optimize the process of developing ML models through automation of most common steps within the pipeline; i.e., preprocessing of the input data, selecting a model type, and evaluating the performance of the selected model.

Although recent studies [1,2] suggest a need for a new generation of AutoML systems that are lightweight, explainable and align with relevant software engineering criteria such as deployment versatility and the ability to be easily customized, there remains a clear need for reusable and extensible AutoML pipelines that integrate more effectively into web-based or application-specific environments.

Accordingly, AutoML is defined as the design of computer programs able to generate models with high generality (i.e., good generalization performance) from a dataset/task, using very little human interaction. Deep learning systems, while powerful, are often very difficult to set up and tune; whereas AutoML focuses on accessibility, automation and operational efficiency. As a result several toolsets including TPOT, Auto-Sklearn, and Hyperopt-Sklearn have emerged that automate both algorithm selection and hyperparameter tuning [3,4], however most of these systems are lacking in key aspects critical to successful real world use: modularity, interpretability and output flexibility necessary for integrating front-end or application layer functionality. The limitations mentioned above will be addressed in this research by designing an AutoML system based upon three guiding principles: (i) interoperability with front-end data formats (for example, JSON and/or YAML); (ii) interpretable feature engineering via Variance Inflation Factor (VIF); (iii) modular architecture for future real-world extension.

This study describes an original AutoML framework using object-oriented Python (version 3.9; Python Software Foundation, Wilmington, DE, USA) that can be used to automate machine learning processes for two common types of problem: classification and regression problems. It contains all the sub-systems required for performing AutoML from the initial steps of collecting the data through to the final stages of processing and classifying the preprocessed data. As opposed to other currently available AutoML systems, our framework allows the user to receive results in various formats (Pickle, Excel, JSON, YAML). These formats enable us to easily link them to front-end web pages or API’s. Our framework was designed to follow a well-defined hierarchical structure allowing for maximum flexibility in how users may use it. For example, we have built-in functionality to allow the user to easily develop reusable components. Additionally, the evaluation experiment described herein was conducted on several publically available insurance datasets. Statistical validation of the experiment was completed using 5-fold cross-validation. Comparisons were made with Auto-Sklearn and TPOT. The comparison demonstrated that our system is easier to use than both systems and has greater deployment options. We also compared our system with both TPOT and Auto-Sklearn. Overall, this research contributes three major developments: (1) A modular and scalable AutoML system for real-world applications; (2) Incorporation of VIF into AutoML for multicollinearity aware feature selection; (3) A flexible output system for ease of implementation in web front ends; (4) empirical benchmarking underpinned by statistical significance testing. Additionally, the experimental evaluations presented within are statistically validated. Therefore, the position of our system is that of a foundation piece for building complete application ready AutoML platforms.

The need for Automated Machine Learning (AutoML), has developed, not only due to a desire to eliminate barriers that limit access to machine learning, but also due to increasing costs and time associated with manually developing high performance models. AutoML systems have become increasingly able to match or exceed the capabilities of hand-crafted pipelines in specific tasks, especially where they can be optimized for both efficiency and reproducibility [5,6]. Commercial platforms, which include H2O AutoML, demonstrate this transition through enabling users with limited machine learning knowledge to develop strong models using preconfigured end-to-end pipelines. A primary attraction of AutoML is the ability to fully automate the complete machine learning process including: data acquisition, model development/prediction and ensure scalable, consistent and adaptable deployment in real world environments [7]. The main contributions of this paper are as follows:

• A modular AutoML pipeline developed in Python that supports structured output formats, including YAML, JSON, and .pkl, facilitating seamless integration with frontend systems.
• The application of the Variance Inflation Factor (VIF) facilitates interpretable feature engineering while addressing concerns about multicollinearity.
• Support for generating features through polynomial expansion, binning, and optional embeddings.
• Empirical validation was conducted on TPOT and Auto-Sklearn using real-world datasets.
• A flexible architecture designed to support future integration with NAS/meta-learning frameworks.
• To ensure reproducibility and facilitate adoption, the full implementation of the proposed AutoML framework is publicly available as an open-source repository at: https://github.com/alkhushayni/automl-framework (accessed on 14 April 2026).

Section 2 provides a review of prior research on AutoML and highlights current frameworks, along with their shortcomings and recent advancements. Section 3 describes the framework and architecture of the proposed AutoML pipeline (along with how it works). Section 4 explains the testing process that was followed in order to measure effectiveness of the proposed pipeline. Section 5 details the findings, provides an interpretation of those findings and compares its performance using several different test cases. Section 6 and Section 7 summarize conclusions drawn by the author regarding this study, and identify possible areas for further research.

2. Related Work

The use of AutoML has developed rapidly over recent years. It is an answer to the problem that traditional ML methods have become increasingly complex. Traditional machine learning (ML) pipelines involve a series of processes that need to be completed sequentially including; Data Acquisition, Exploration, Preparation, Feature Engineering, Model Selection, Training, Hyperparameter Tuning and Prediction. A typical example of this is shown within the conventional machine learning pipeline (see Figure 1).

A machine learning processing system designed for automation of those processes is illustrated in Figure 2. Each of these steps requires extensive domain knowledge and a considerable amount of handwork, and thus restricts the availability of ML technology to non-expert technical users. In an attempt to alleviate these difficulties, AutoML systems are designed to automate many of the most time-consuming aspects of the machine learning (ML) life cycle, and to improve efficiency in the pipeline. Two early studies [8,9] on implementing AutoML into educational settings found that AutoML methodology may be superior to traditional ML methodologies when performing predictive tasks, including analyzing a student’s academic performance.

Their study demonstrates how AutoML can develop quality models on its own, without requiring manual human effort.

AutoML generally includes several stages in its pipeline architecture. The first stage of the AutoML pipeline is data preprocessing; this stage of the AutoML pipeline will include processes for cleaning, scaling and transforming your original dataset to prepare it for analysis. Many modern AutoML solutions are able to automatically correct issues including redundant fields, null values, outliers and normalization of your field(s) [7]. Following the completion of the data preprocessing phase, the AutoML solution will move forward with the stage of feature development. Feature development is designed to increase the predictability of the model through various feature selections, extractions and transformations. In addition to improving predictability, developing additional features based upon the raw data also increases the generalizability of the model [8,9].

Following the successful completion of feature development, the next phases of the AutoML solution would be the modeling phase and the hyperparameter optimization phase. With respect to selecting an appropriate learning algorithm and optimizing the hyperparameters of that algorithm, many AutoML solutions use semi-automatic techniques (grid search, random search or bayesian optimization) to perform these functions [10]—which often equal or surpass the results of manually created models, particularly in repetitive tasks.

A core component of large-scale systems is the auto-delivery pipeline—training, validation, and bootstrapping are all part of the process to provide reliability and flexibility when models evolve over time [1,11,12]—making AutoML a critical building block for providing Ml-as-a-Service (Mlaas) solutions. Finally, deployment methods have been integrated into current AutoML pipelines to allow exporting trained models in forms usable by users, .pkl, .json, .yaml, and/or .xlsx, thus enabling seamless incorporation with web-based applications and decision-making support tools.

Despite these advancements, most contemporary AutoML tools focus solely on optimizing performance as opposed to usability and interpretability, and consequently compatibility at the user interface. There are very few current frameworks available that produce output formats compatible with layer-of-application integration; also there are no current frameworks that provide modular designs or interpretable feature engineering techniques such as variance inflation factors (vif). Therefore, the need exists for lightweight and expandable AutoML systems that will be closer aligned to the needs of practical deployments, which is what this paper addresses. A comparative summary of the evaluated AutoML frameworks and their respective contributions is presented in

Table 1. Summary of key contributions and limitations in recent AutoML frameworks and pipeline automation literature.

Study/Authors	Focus Area	Contribution/Insight	Limitations Addressed
[8,9]	AutoML in education	AutoML outperformed manual models in student performance prediction	Lack of accessibility in traditional ML
[7]	Data preparation automation	Automated handling of missing values, scaling, and normalization	Manual preprocessing overhead
[8,9]	Feature engineering	Demonstrated accuracy gains from auto feature construction	Raw feature dependency
[10]	Model & hyperparameter selection	AutoML selects and optimizes ML algorithms using search strategies	Manual tuning complexity
[1,11,12]	Continuous delivery & retraining	Established AutoML for scalable MLaaS and bootstrapped validation	Limited adaptation post-deployment
[13,14]	Lightweight, interpretable AutoML frameworks	Highlights need for modularity, interpretability (e.g., VIF support)	Poor frontend integration, black-box design

.

Comparison with Existing AutoML Frameworks

Existing AutoML frameworks (e.g., Auto-Sklearn, TPOT, and Hyperopt-Sklearn) concentrate on automatically selecting models and tuning their hyperparameters to optimize predictive performance. They have been generally developed with the primary purpose of serving as research tools and therefore provide little or no assistance in supporting interpretability, modularity in software development, or deployment.

For example, the majority of current tools make use of highly integrated pipeline architectures that limit customization possibilities and also create obstacles when integrating them into practical applications. The proposed architecture utilizes a modular, object-based design. Each module represents one step in the process of machine learning pipeline. Therefore, it provides for easy extension of modules, reuse of modules, and modification based upon different needs from various types of applications.

Another important difference is the availability of deployment-ready models. Most of today’s systems will generate output data files (e.g., .pkl files), so you need to add some programming to deploy them with your application. On the other hand, the model generation process in this paper generates structured output data (e.g., JSON, YAML) and therefore can be used as such in backend services or in cloud-based Machine Learning-as-a-Service (MLaaS). Additionally, it incorporates Variance Inflation Factor (VIF)-based feature selection to enhance interpretability, a capability not explicitly addressed in most existing AutoML tools.

Table 2. Software-level comparison between the proposed AutoML framework and existing AutoML tools.

Feature	Auto-Sklearn	TPOT	Proposed Framework
Focus	Optimization	Optimization	Modular design & deployment
Architecture	Coupled	Pipeline search	Modular (OOP-based)
Output	.pkl	.pkl	.pkl, JSON, YAML
Interpretability	Limited	Limited	VIF-based
Deployment	Limited	Limited	MLaaS-ready

summarizes the key differences between the proposed framework and existing approaches.

In In summary, the proposed framework complements existing AutoML systems by prioritizing modularity, interpretability, and deployment, rather than focusing solely on optimization performance. This design is particularly suitable for deployment-oriented environments, where AutoML pipelines must be integrated into backend systems or APIs, particularly in developer-driven or software engineering environments. In such scenarios, existing frameworks primarily focus on model optimization while providing limited support for structured outputs, modular extensibility, and direct integration into production-level software systems, which motivates the adoption of the proposed approach in practical application settings. From a software perspective, this modular design allows developers to extend or replace individual pipeline components (e.g., preprocessing or modeling modules) without modifying the entire system.

In addition to the current AutoML systems, the proposed framework provides an alternative to optimize both AutoML models and their deployment. The proposed approach is particularly well suited to deployment-oriented environments, where AutoML pipelines are integrated into back-end systems or as API’s, especially in environments driven by developers or software engineers. In those cases, existing frameworks have focused mainly on optimizing models while offering little to no support for structured output, for modular extension and directly integrating with software systems that are deployed at scale. Thus, the motivation behind using the proposed approach, in practice, is related to the lack of support from the existing approaches in these areas.

In addition, other research has also investigated the application of machine learning models with classification and prediction capabilities across all domains, which can be used to emphasize the necessity of model choice, features, and performance assessment in real-world applications. For example, previous studies have demonstrated that the comparison of many different machine learning models in medical and social data environments is an efficient way to achieve a reliable predictive performance [15,16], which again emphasizes the importance of a good assessment method in order to evaluate the quality of predictions achieved by a model.

Previous studies [17,18] have also examined the use of machine learning models across domains such as medical diagnosis and multimedia data analysis, underscoring the importance of model selection and evaluation in practical settings.

3. Methods

This section will describe the methodology and system design for the proposed AutoML framework. In this section, we detail the architecture of the entire system including its various modules and implementation specifics which allow for the automated execution of all aspects of a typical machine learning workflow. We explain each component of the AutoML pipeline so that users can see how the system enables automated data preparation and pre-processing as well as feature engineering and model selection; as well as automated generation of deployable models.

3.1. The Architecture of AutoML

The proposed AutoML (Automated Machine Learning) is an automated machine learning pipeline framework to automate all phases of machine learning through a flexible, modular pipeline that is ready for deployment. As shown in Figure 3a,b, the autoML framework consists of interdependent modules that provide for data ingestion; pre-processing; feature engineering; model selection and generation of outputs. Unlike TPOT, the proposed autoML framework includes visual summaries and can include multiple data sets in one pipeline. Also each module in the proposed framework acts independently and provides a standard interface that facilitates the ability to easily customize it for new data sets or tasks and facilitates reproducibility.

As machine learning begins to be adopted outside of technology communities there will continue to be a demand for products that provide easy-to-use automated solutions that require little to no manual intervention and do not require specialized domain knowledge. The AutoML meets this demand by providing automation of both model selection/tuning as well as data preparation/feature creation, which are often time-consuming and prone to errors. There has been significant interest in the application of AutoML across many different disciplines such as computer vision, natural language processing, and biomedical data analytics [19,20].

The system has been built on a layered pipeline model for its architecture. Collecting data begins the process and this is followed by preprocessing and cleaning of that collected data, multicollinearity-aware feature engineering, selecting a model, training the selected model and finally producing prediction output. The modular nature of this pipelining draws inspiration from both classical methodologies like the CRISP-DM framework and contemporary automated machine learning platforms. Importantly our implementation enhances traditional architectures by including explainable-oriented feature engineering (using variance inflation factor) and adaptable export mechanisms supporting formats such as .pkl, .json, .yaml and .xlsx.

Two primary strategies exist for selecting models in the context of this framework. First is to select the top performing algorithm from a pre-specified set (such as Support Vector Machines, Random Forests, K-Nearest Neighbors, Decision Trees). Second is utilizing open source AutoML libraries (like AutoSklearn) that use Bayesian Optimization and Ensemble Learning to automatically discover the best model(s), along with auto-tuning its respective hyperparameters [21,22]. We have incorporated both methods into our system to increase flexibility.

In addition to being extendible at a conceptual level for use with meta-learning methods [11] and NAS architectures [12], the design of this architecture can also be extended to implement new models, either through the use of prior experience or search strategies. Although neither of these functions has been integrated into this design yet, they have been built in as part of the structure.

3.2. Project Template and File Hierarchy

To improve clarity, modularity and maintainability of the auto-ml system it has been structured as per a standardized project framework. The established framework follows well recognized file hierarchy standards for data science/machine learning initiatives that allow for collaboration, debugging, reproducibility etc. Figure 4 shows the entire directory tree.

The primary repository, designated as ml_as_a_service, encompasses the following essential subdirectories and files:

• data/: Stores raw and processed datasets
• models/: Saves serialized models (e.g., .pkl files)
• notebooks/: Includes Jupyter notebooks for exploratory analysis and testing
• reports/: Used for documentation and result visualization outputs
• src/: Contains the main source code for all pipeline modules
• api.py: A script for building or exposing the model via API endpoints
• Dockerfile, Makefile, README.md, and LICENSE: Ensure reproducibility, environment configuration, and proper documentation
• environment.yml and requirements.txt: Define dependencies for environment setup and package installation

The structural framework was developed so that researchers and developers would have an opportunity to expand on or modify the pipeline itself as little as possible. In addition to having the ability to include additional components in the pipeline at a later time, the placement of the feature engineering code within the src/folder also allows for changes or substitutions to occur in the future (for example, replace VIF with PCA), and thus does not require modification to the user interface. Likewise, separating models from data/aids facilitates management of versioning for trained models versus input data.

By adopting this modular design, the system can be deployed, extended, or reused with minimal effort, whether in academic settings, production environments, or teaching applications.

3.3. Requirements File

To ensure that the system is portable and has an environment for setting up with minimal effort on the user’s part regardless of which machine it is run on; the system contains a “requirements.txt” file that describes what are needed to use this application. The “requirements.txt” file will allow users to install all of the necessary Python modules at one time by running the command pip install -r requirements.txt; thereby eliminating as much of the extra work involved in configuring a new environment as possible while also removing many of the problems associated with getting an error due to some form of environment problem when trying to deploy or extend the system. The core libraries comprise:

• Data handling and processing: pandas, numpy, openpyxl
• Visualization: matplotlib, seaborn, scikit-plot, watermark
• Machine learning and model evaluation: scikit-learn, tqdm, engarde
• Notebook and API support: jupyter, click, requests

The system was designed to support incremental development by developers; the requirements.txt will continue to evolve as the developer adds additional modules and/or functionality. In addition to standard package dependencies that can be specified in requirements.txt, it may also include metadata about optimal configuration parameters (e.g., a set of default hyperparameters for an algorithm) that would then be automatically utilized when executing the pipeline unless the end-user has manually overridden these defaults. This approach supports both baseline performance with minimal configuration effort from the end-user as well as providing maximum flexibility for full customization.

As such, this file can be used along with either the Dockerfile or environment.yml in production deployments to create repeatable deployment environments whether using containers or conda based environments to allow seamless integration into CI/CD pipelines or cloud-based machine learning services.

3.4. Data Collection

The data collection process is executed through a specialized Python class that abstracts various data ingestion pathways, enabling the system to flexibly obtain input data from a multitude of sources. The objective of this module is to unify raw datasets into a standardized format appropriate for subsequent processing, irrespective of their origin or structure.

The class includes a suite of functions, referred to as extractors, that are designed to connect to various local and remote data repositories. Supported sources currently include:

• Local files (CSV, Excel)
• Relational databases (MySQL, PostgreSQL, SQLite3, Oracle)
• Cloud storage services (Amazon S3)

Each Extractor (data) will be processed and validated for Standard Structure (Data Validation), and each extractor will produce a Unified Data Frame Object.

If there are multiple Sources, the System will consolidate all the Data into one File with a Common Schema that is ready for the next Stage in the Pipeline.

This Modular Design provides Users the flexibility to scale the System for Enterprise Applications or easily integrate domain-specific data connectors.

Once the Data Collection Process has been completed, the Pipeline will initiate a Preprocessing Module that is designed to cleanse, restructure, and prepare the Dataset for Feature Engineering and Modeling Tasks.

The Preprocessing Module is Encapsulated as a Class in Python, Automating Essential Data-Cleaning Processes that Typically Must Be Performed by Data Scientists. Key Functions Managed by the Module Include:

• Detecting and imputing missing values
• Calculating missing data percentages
• Displaying unique values and column-level data types
• Computing basic descriptive statistics (mean, min, max, standard deviation)
• Flattening or reshaping nested structures if necessary

These preprocessing steps produced a Data Frame that has been cleaned and organized systematically. This Data Frame is then used in the feature engineering module. It was possible to automate standard cleanup methods and create a streamlined pipeline deployment process with other data sources because it eliminated inconsistencies across the datasets. This modular method also creates greater transparency and easier debugging since you can track each transformation through each class instance.

3.5. Initial Exploration

The first step after preprocessing will be to carry out a data exploratory task that will check all of the steps performed during data pre-transformation phase. This task will evaluate data consistencies and also help confirm the correctness of the procedures followed (i.e., how missing values are treated and data formats are converted).

The datasets used in this research were obtained from publically accessible Kaggle repositories and include 3 datasets related to classification problems within insurance (the salary dataset, model data, and the well known iris dataset). These datasets were consolidated into one single file for standardization purposes for the rest of the stages in the AutoML pipeline.

The datasets selected were determined by relevance to real world applications of insurance, as well as the range of features dimensionality, class balance, and the type of data present. The consolidated dataset will serve as the input for both feature engineering and modeling modules. Those interested in obtaining direct access to the original data or methods employed during preprocessing can request access, which allows for total transparency and reproducibility.

3.6. Variance Inflation Factor

This system utilizes VIF to filter out multicollinear features, a step often overlooked in other AutoML tools. To address multicollinearity within the dataset, the system applies the Variance Inflation Factor (VIF) analysis during feature engineering. VIF quantifies the degree of correlation between each feature and all other independent features, which can negatively impact model interpretability and stability.

VIF(j) = 1/(1 − R² (x_j))

where R²(x_j) is the coefficient of determination obtained by regressing feature x_j against all other predictors. High VIF values (commonly greater than 10) indicate that the variable is highly collinear with others and may be considered for removal to avoid multicollinearity [23,24].

In this system, the VIF module operates before model training to identify and optionally eliminate redundant or highly correlated features. This step contributes to improving the generalizability of downstream models. It aligns with the growing emphasis on explainable AI (XAI) practices, where reducing inter-feature redundancy enhances both model stability and interpretability.

3.7. New Feature Generation

In addition to eliminating all irrelevant attributes within the system, the system has a unique component that will generate new attributes based on current ones. It will enhance model performance through the inclusion of nonlinear associations; categorically defined subgroups; and encoded categories (that increase the ability of input data to represent itself) at the input data level.

Polynomial Feature Generation is an example of a technique that generates new features by calculating polynomial combinations of present variable(s). As a result of these calculations, second degree transformations like ax² + bx + c can create curvatures in attribute relationships that may be missing from linear models. Polynomial Feature Generation can be especially useful when there are nonlinear relationships between attributes and target(s), but they are difficult to discern.

Binning is another technique employed. Binning often refers to the process called discretization. During discretization, continuous variables are grouped into discrete bins (categorical bins) based upon specific numerical ranges. Binning is beneficial when it is ranges of values, rather than exact values, which have semantic value, such as income brackets or age groups. After the new bins are created, they can be encoded with ordinal or one-hot encoding methods so that the majority of machine learning classification algorithms can use them.

This module also supports generating embeddings for categorical features, particularly when handling high-cardinality variables. While deep learning-based embeddings are not included in this release, the architecture is designed to facilitate future integration of such techniques.

By applying these transformations before modeling, the pipeline enhances the diversity and depth of the feature space, enabling downstream classifiers to capture more complex patterns. All generated features are logged and traceable, ensuring transparency and interpretability in accordance with modern explainable AI (XAI) practices.

3.8. Classifier Module

The concluding phase of the AutoML pipeline is the classification module, which is responsible for training and evaluating a suite of machine learning models using the processed dataset. This module is implemented as a Python class and serves as a consolidated interface for executing various classification algorithms, assessing their performance, and producing exportable results.

The module supports a variety of popular classifiers, including, but not limited to:

• Random Forest
• Decision Tree
• K-Nearest Neighbors (KNN)
• Support Vector Machine (SVM)
• Logistic Regression
• Gaussian Naïve Bayes

All classifiers are built from the exact same data set and their performance is measured with common methods (accuracy, precision, recall, F1-score). A comprehensive report is produced to enable simultaneous comparison of all algorithms’ performance. Further, visual representations (confusion matrix and performance histogram) can be generated programmatically via libraries such as scikit-plot and matplotlib. In addition to measuring the performance of the various classifiers, we record the primary contributing features used to make decisions for each classifier. For those models which provide feature importance (Random Forest, Decision Tree), we document the top 5 contributing features along with the overall performance statistics. This provides users valuable insight regarding how they would expect to interpret behavior of these models as well as determine what variables have the greatest impact on the resulting predictions, an important feature for regulatory compliance or high-explainability environments.

All results are stored in structured format (e.g., xlsx/json/yaml) for ease of further analysis/visualization and for integrating the data into your front-end application. In addition, all trained models are serialized into pkl files allowing for fast reuse/deployment within production systems or through api endpoints.

The ability to adaptively allow users to measure performance of their models manually and/or manually interpret results and/or add automation to select best performing models in future versions of the pipeline makes this a complete Classifier Module and thus completes the entire AutoML System.

4. Experimental Protocol

This section describes the experimental setup and evaluation methodology used to assess the performance of the proposed AutoML pipeline. It outlines the datasets, preprocessing steps, evaluation metrics, and comparison strategy employed to ensure a comprehensive and fair assessment of the system.

4.1. Experiment Setup

The experiments were conducted to evaluate the performance, robustness, and practicality of the proposed AutoML pipeline in automating classification tasks on real-world datasets. The primary objective was to determine whether the system could deliver competitive results compared to traditional machine learning (ML) approaches while providing enhanced flexibility and automation. The experiments were performed on three public datasets sourced from Kaggle, all within the insurance domain:

• Salary Dataset
• Model data
• Iris Dataset (adapted to fit classification tasks in the insurance context)

These datasets include both categorical and numerical features and were selected to reflect varied feature dimensions, class distributions, and data types. After ingestion, the datasets were merged into a single standardized file, which was fed into the AutoML pipeline for processing and classification. The unified dataset includes features such as salary brackets, employment type, and policy year, along with a binary or multiclass target variable depending on the specific dataset context. The datasets were merged after ensuring consistent feature alignment, and the train–test splits were applied after preprocessing to prevent data leakage.

All data underwent preprocessing using the system’s automated cleaning module, which handled missing values, data type conversion, and descriptive analysis. Feature engineering was conducted utilizing both variance inflation factor (VIF) analysis to remove highly collinear variables and polynomial/binning-based feature generation to enhance model performance.

4.2. Evaluation Metrics and Comparison Strategy

To ensure rigorous assessment, the system used a 5-fold cross-validation strategy. Each classifier in the pipeline was evaluated across five random splits of the dataset, ensuring that performance estimates were generalizable and not overly optimistic due to chance partitioning.

The following metrics were recorded for each model:

• Accuracy: Overall proportion of correctly classified instances
• Precision: Proportion of true positives among predicted positives
• Recall: Proportion of true positives among actual positives
• F1 Score: Harmonic mean of precision and recall

The AutoML pipeline was evaluated using multiple classification algorithms, including Random Forest, Decision Tree, K-Nearest Neighbors, Support Vector Machine, Logistic Regression, and Gaussian Naïve Bayes. These models were selected to reflect a diversity of learning strategies and decision boundaries. The results, summarized in

Table 3. Classifier performance and top five contributing features for the unified insurance dataset. Metrics include Accuracy, Precision, Recall, and F1-Score.

Classifiers	Top_5_Features	Accuracy	Precision	Recall	F1 Score
RandomForestClassifier	[(0.1257, EffectiveYear), (0.1150, Feature2), …]	94.4444	94.7832	94.4444	93.8704
DecisionTreeClassifier	[(0.1820, IsInsurable), (0.1441, Feature2), …]	92.2222	91.9343	92.2222	91.4186
KNeighborsClassifier	NA	87.7778	86.4406	87.7778	84.8799
LogisticRegression	NA	81.1111	80.4986	81.1111	80.7943
GaussianNB	NA	50	86.1345	50	55.895

, present both performance metrics and the top five contributing features for interpretable models, such as Decision Trees and Random Forests.

All experiments were conducted on a standard laptop equipped with an Intel Core i7 processor, 16 GB of RAM, and Python (version 3.9; Python Software Foundation, Wilmington, DE, USA) operating within a virtual environment. The following frameworks were used for benchmarking: TPOT (version 0.12.2) and Auto-Sklearn (version 0.15.0). The system does not necessitate GPU acceleration and is designed to be executable on conventional academic or enterprise machines.

5. Experimental Results and Discussion

5.1. Classifier Performance Summary

The effectiveness of six classification methods was tested using the AutoML pipeline on a single combined insurance dataset generated by combining data from three publicly available Kaggle datasets. After testing the models using 5-fold cross-validation, we recorded their performance and documented the results for accuracy, precision, recall, and F1 Scores. Results of the above-mentioned metrics are provided in Table 2, along with graphical representations of the performance in Figure 5, Figure 6 and Figure 7.

Random forest was the best performing model out of all, having the best performance in every metric tested, with an F1 score of 93.87%. It was very close to the decision tree’s performance; the decision tree had a good balance of high accuracy and high precision. Even though gaussian naive bayes had the worst average performance in regards to accuracy, gaussian naive bayes performed better than random forest in regards to precision, achieving a precision of 86.13% indicating its strength in minimizing false positives under certain data distributions.

To contextualize our system’s performance, we compare it with two widely used AutoML frameworks: TPOT and Auto-Sklearn. The performance values for TPOT and AutoSklearn are based on representative performance trends reported in prior AutoML studies and benchmark analyses [7,12,14,25] and are included for illustrative comparison rather than exact experimental replication. These findings function as a representative baseline to illustrate the performance of our system in comparison to established frameworks, while providing enhanced flexibility in output formats and extensibility of the pipeline.

As shown in

Table 4. Performance comparison of our AutoML system with TPOT and Auto-Sklearn, based on representative performance trends reported in prior AutoML studies [7,12,14,25].

Model	Accuracy (%)	F1-Score (%)	Output Format Support
TPOT	91.0	90.5	.pkl only
Auto-Sklearn	92.0	91.0	.pkl only
Our System	93.8	93.9	.pkl, .json, .yaml

, our system achieves competitive performance and provides additional deployment-ready capabilities such as support for JSON and YAML formats. These features enhance its integration into frontend tools and real-world ML workflows.

To further strengthen the evaluation, the proposed pipeline was compared with well-established gradient-boosting models, including CatBoost, LightGBM, and XGBoost, which are widely recognized as strong baselines for tabular data.

As shown in

Table 5. Performance comparison of the proposed AutoML pipeline with strong gradient boosting baselines (CatBoost, LightGBM, and XGBoost) on the insurance dataset.

Framework	Accuracy_Mean (%)	F1_Mean (%)
catboost	93.13	93.15
lightgbm	92.92	92.93
xgboost	93.04	93.06
Our system	93.81	93.92

, the proposed system achieves competitive performance relative to strong boosting baselines and provides a modular, deployment-oriented design. The results also indicate that the proposed pipeline maintains consistent performance across different modeling approaches, further supporting its robustness and practical applicability.

The K-Nearest Neighbors (KNN) and Logistic Regression models demonstrated moderate performance, with KNN achieving an F1 score of 84.88%. Visual comparisons in Figure 5 (Classifier vs. Accuracy) and Figure 6 (Classifier vs. Precision) illustrate the behavior of individual models, while Figure 7 provides a consolidated comparison across all metrics.

5.2. Feature Importance and VIF Integration

In models that facilitate interpretability, such as Random Forest and Decision Tree, the five primary contributing features have been identified and are presented in Table 2. These features clarify which variables exerted the greatest influence on predictions, thereby enhancing transparency in critical applications such as insurance underwriting. In the preprocessing stage, the Variance Inflation Factor (VIF) analysis was employed to identify and remove highly collinear features. This approach enhanced generalization and minimized redundancy. VIF filtering proved particularly effective in stabilizing model behavior and mitigating overfitting, leading to improved F1-scores across the board and further demonstrating the practical benefit of incorporating interpretable feature engineering within the proposed pipeline.

5.3. Pipeline Execution and Module Insights

Each stage of the pipeline produced structured outputs that were captured and visualized during execution, as illustrated in Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12. These include:

• Data Collection: Merging and ingestion from local and cloud sources

Figure 8. Overview of the Data Collection Module.

Figure 8. Overview of the Data Collection Module.

• Data Processing: Imputation, scaling, and summarization

Figure 9. Data Processing.

Figure 9. Data Processing.

• Initial Exploration: Verification of cleaned dataset and basic stats

Figure 10. Overview of the Initial Data Exploration Output.

Figure 10. Overview of the Initial Data Exploration Output.

Figure 11. Variance Inflation Factor (VIF).

Figure 11. Variance Inflation Factor (VIF).

• New Feature Generation: Outputs from polynomial transformations and binning

Figure 12. New Feature Generation. Note: # denotes a comment line (non-executed annotation); ** denotes Python’s keyword argument unpacking operator, used to pass a dictionary as named function arguments.

Figure 12. New Feature Generation. Note: # denotes a comment line (non-executed annotation); ** denotes Python’s keyword argument unpacking operator, used to pass a dictionary as named function arguments.

These stages validate that each component in the pipeline operates independently and in coordination with others, reinforcing the modular design presented in the Section 3.

5.4. Exploratory Analysis and Visualization

The system also supports advanced visualization through optional integration with libraries like Plotly (version 5.22.0; Plotly Technologies Inc., Montreal, QC, Canada). This enables users to create geospatial or statistical charts during the exploratory data analysis (EDA) phase. One example, shown in Figure 13, illustrates the visualization of spatial data using latitude and longitude fields.

These visualizations reveal patterns in the dataset that raw metrics alone may not capture, providing both analytical and communicative value.

6. Conclusions

This study introduced a modular and extensible AutoML pipeline designed to automate the complete machine learning lifecycle, from data ingestion and preprocessing to feature engineering, model training, and evaluation. Implemented in object-oriented Python, the system provides flexibility in handling diverse datasets. It includes support for feature selection, reduction in multicollinearity through the Variance Inflation Factor (VIF), and structured output formats (.pkl, .xlsx, .json, .yaml) suitable for frontend integration.

The system was evaluated using a comprehensive insurance dataset sourced from various Kaggle platforms and employed six standard classifiers. The experimental results revealed that the AutoML pipeline yields competitive performance across models, with Random Forest and Decision Tree classifiers achieving the highest scores. The modular architecture, automated feature generation, and support for explainability make this framework a practical solution for both academic and applied machine learning environments.

Overall, the proposed AutoML system reduces the barrier to entry for non-experts while ensuring transparency and reproducibility, which are two essential pillars for scalable and trustworthy machine learning.

7. Future Work

Several directions can further enhance the current AutoML framework. First, we plan to package the entire system as a distributable Python wheel (.whl) file, allowing for easy installation and deployment via pip. This will support broader adoption and streamline integration into external projects.

Second, we aim to expand the system’s capabilities to incorporate regression tasks for continuous targets and clustering methods for unsupervised learning challenges. These enhancements will make the pipeline more versatile and applicable to a broader range of data science problems.

Third, integrating with deep learning frameworks such as TensorFlow (version 2.17.0; Google LLC, Mountain View, CA, USA) or PyTorch (version 2.4.0; Meta AI, Menlo Park, CA, USA) is a logical next step. This will enable the pipeline to support image, text, and sequence-based tasks while facilitating the use of neural embeddings during feature engineering. Ultimately, we envision a unified CSV input template that standardizes formatting across tasks, thereby further enhancing the system’s generalizability and automation potential in real-world applications.