Classification Machine Learning Models for Enhancing the Sustainability of Postal System Modules Within the Smart Transportation Concept

Abstract

Postal traffic and transport face challenges related to the rapid growth of parcel volumes, increasing demands for sustainability, and the need for integration into the smart transportation concept. This study explores the application of machine learning (ML) models for the classification of postal delivery times, with the aim of improving service efficiency and quality. As a case study, the Postal Center Zenica, one of the seven organizational units of the Public Enterprise “BH Pošta” in Bosnia and Herzegovina, was analyzed. The available dataset comprised 11,138 instances, which were cleaned and filtered, then expanded through two iterations of data augmentation using an autoencoder neural network. Five ML models, Random Forest, Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), k-Nearest Neighbors (kNN), and Multi-Layer Perceptron (MLP), were developed and compared, with hyperparameters optimized using the Bayesian method and evaluated through standard classification metrics. The results indicate that the data augmentation method significantly improves model performance, particularly in the classification of delayed shipments, with ensemble, especially Random Forest and XGBoost, emerging as the most robust solutions. Beyond contributions in the context of postal traffic and transport, the proposed methodological framework demonstrates interdisciplinary relevance, as it can also be applied in telecommunication traffic classes, where similar network dynamics require reliable predictive models.

1. Introduction

The postal system is a complex multimodular transport network involving both public and private operators, inseparably linked to the sophisticated roles and urban needs of businesses and citizens. The development of e-commerce has made it one of the key drivers of economic growth and technological progress at the local and regional level, while global trends simultaneously indicate continuous growth in the parcel transport module and a slowdown in the decline of the letter mail module. Today, sustainable development is oriented not only toward continuously updated economic objectives and achieved business outcomes, but also toward overcoming the limitations of operators’ capacity to adapt to new cultural and societal needs through innovative and digital approaches. Within the smart transportation concept (STC), postal systems are becoming part of a broader ecosystem of smart mobility, where the integration of new technologies, data exchange, and the application of machine learning (ML) enable resource optimization, more accurate predictions, and the provision of more efficient and environmentally friendly services [1].

The STC operates through a cyclical loop of research, planning, execution, control, monitoring, and reflection, based on the principles of sustainability, equity, and resilience to different categories of constraints in the business processes of multimodal transport systems. An important aspect of STC is urban mobility, which develops in an environment that incorporates realistic sociodemographic attributes, allowing heterogeneous and context-sensitive behavior of service users and postal operators in multimodal and unstable urban scenarios. The smart transportation framework encompasses multidimensional roles of digital technologies and data-driven approaches in environments often characterized by unstable states and dynamic actions, where ML models stand out as key methods, techniques, and algorithms (tools). According to [2], smart transportation is a concept based on the application of advanced sensors, information and communication technologies, and management strategies with the aim of improving the efficiency and safety of transport systems. Its core principles—sustainability, integration, safety, and adaptability—are directed toward mobility, environmental responsibility, and economic development.

ML plays a central role in this context, as it enables the analysis of large datasets to identify patterns, predict demand for product and service portfolios, and estimate delivery times of transport shipments, as well as optimize resources in line with the needs of business and urban users. Applications include the analysis of shipment flows, assessment of operator competitiveness, and planning of new logistics capacities. Nevertheless, the broader application of proven methods such as Random Forest and boosting algorithms is still lacking, which opens space for research aimed at developing solutions that contribute to the sustainability of the postal transport system and its integration into the smart transportation environment [3], making it capable of reflective adaptation to environmental disruptions and changes in the personal preferences of postal service users and transport operators.

Previous research by the authors has addressed the prediction of the number of express mail parcels using statistical (SARIMA, Prophet) and machine learning models (Support Vector Regression—SVR, ANN), enabling the identification of seasonal and daily patterns and more efficient resource planning [4]. In addition, models for classifying parcel arrival rates (Random Forest, XGBoost, MLP, SVM) were developed, with SVM optimized by the Bayesian method achieving the highest accuracy, confirming the importance of ML approaches for enhancing the sustainability of the postal system [5].

Although ML and data-driven approaches have been widely applied in logistics and multimodal transport, the smart transportation framework in the postal system remains insufficiently explored. This study aims to fill that gap through the development and comparison of multiple ML models for the classification of delivery time, considered a particularly sensitive categorical and temporal process variable. The case study focuses on Postal Center Zenica, one of the seven organizational units of the Public Enterprise “BH Pošta” in Bosnia and Herzegovina, which allows for validation of the proposed approach on real data. In addition, the research examines the potential transfer of the developed solutions to the field of telecommunication traffic, where similar network dynamics create opportunities for interdisciplinary application. Although it is possible to analyze and investigate different modules of the postal system, such as the parcel module, the letter mail module, the logistics and distribution module, the universal postal service module, the financial-postal module, the philatelic module, the electronic and hybrid services module, the courier-express module, the international shipment module, and even innovative drone-based solutions, the focus of this paper is on express mail. This module in Bosnia and Herzegovina is characterized by strong competitiveness and represents the most sensitive area with regard to the accuracy of delivery time prediction, which further justifies its selection as the central focus of the research.

The main contributions of this study can be summarized as follows:

• Comparative analysis of multiple ML models, Random Forest (RF), Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), k-Nearest Neighbors (kNN), and Multi-Layer Perceptron (MLP), through a systematic evaluation of different performance metrics.
• Development of an original methodology that combines the data augmentation method using an autoencoder with hyperparameter optimization through the Bayesian method, thereby improving classification accuracy and overcoming the limitations of a relatively small dataset.
• Application to real-world research data from the postal transport system in a geographic area where no similar studies have been conducted, giving the research a unique case-study character of significance.
• Practical applicability of the proposed solutions, reflected in the relative ease of integration into a real postal transport system, with direct potential to improve service quality and reliability.
• Interdisciplinary significance, as the methodological framework developed for the postal transport system can also be applied in other domains of smart transportation, including telecommunications and urban logistics.

The paper is structured into five sections. Following the introduction, the second section provides a review of relevant literature on the development of the postal transport system within the smart transportation concept, with particular emphasis on the application of machine learning. The third section describes in detail the research data, their preparation, and the applied modeling methodology. The fourth section presents the experimental results and their interpretation in the context of postal transport system sustainability. Finally, the fifth section offers conclusions and outlines directions for future research.

2. Review of Relevant Published Research

In a study conducted in Turkey [6], ML algorithms Random Forest, Gradient Boosting (GB), kNN, and Neural Networks (NN) were applied to predict the delivery time of postal shipments using the e-scooter module. The best results were achieved by GB (R² = 0.845), with the lowest error values, confirming its reliability in the assessment of the duration of certain activities of delivery of mail to recipients. A study based on Canada Post data for the Toronto area [7] examined the application of deep learning for delivery time prediction in the last-mile segment. The authors developed an end-to-end neural model within Internet of Things (IoT) and cloud-based smart city architecture, using Origin–Destination (OD) data and weather conditions as inputs. Results showed that convolutional neural networks significantly outperformed classical ML models and OD reference approaches, enabling more accurate delivery time predictions and enhancing last-mile logistics. For Deutsche Post in Germany, a courier-oriented routing algorithm and an ML model for predicting delivery time windows were developed [8]. By combining operations research, statistics, and ML, the model leverages the implicit knowledge of experienced couriers through historical delivery sequence data.

Medić et al. [3], in their systematic literature review on the application of ML in urban logistics systems, emphasized studies focused on delivery time estimation in the last-mile segment. They noted that models such as random forest, convolutional, and residual neural networks were applied to large postal datasets (e.g., Canada Post) to accurately predict delivery times from depots to end-users. In [9], the authors compared linear regression models and ensemble approaches (random forest, bagging, boosting) and demonstrated that algorithms such as LightGBM and CatBoost achieved superior results compared to classical methods. Dobrodolac et al. [10] analyzed potential applications of artificial intelligence in postal systems, highlighting that AI models enable route optimization and more precise delivery time estimation, including adjustments of delivery schedules according to customer expectations. Karakaya [11] compared various ML models and ensemble learning methods using a large Amazon delivery dataset for delivery time classification. Results showed that ensemble methods, particularly a combination of SVM, Naive Bayes (NB), and Linear Discriminant Analysis (LDA) with Extra Trees (ET) as a meta-model, achieved extremely high accuracy (99.89%).

In [12], the authors focused on service time prediction (STP) in last-mile delivery, which is a key component of total delivery time. They proposed the Meta-learning Service Time Prediction (MetaSTP) model, a neural network based on meta-learning and Transformer architecture, which uses location-based features to better represent complex delivery conditions and address class imbalance. Experiments on two real datasets showed that MetaSTP outperformed baseline models by at least 9.5% and 7.6%. Küp et al. [13] developed a framework for delay prediction using real-world data from an e-commerce logistics company. The delivery process was modeled in two variants—with 11 and 15 operational steps, where each step represented a phase of shipment handling (e.g., acceptance, transfer, transport, terminal operations). Separate models were trained for each phase to estimate delay probabilities in real time. Logistic Regression (LR), Random Forest, XGBoost, and Categorical Boosting (CatBoost) were compared, with boosting methods achieving the highest Area Under the Curve (AUC) values (up to 99.9%).

In [14], the authors addressed the prediction of on-time delivery rates at the courier level, which is essential for workforce productivity and customer satisfaction in the express industry. They proposed a deep spatio-temporal neural network, the Regional Courier Correlation Network (RCCNet), combining Node2vec for road network graph encoding, Graph Convolutional Networks (GCN) for modeling courier interdependencies, and Long Short-Term Memory (LSTM) networks for leveraging historical sequences. In [15], factors affecting food delivery time were analyzed, and different ML models were tested on a Kaggle dataset. Linear Regression, Decision Trees, Random Forest, and Extreme Gradient Boosting Regressor (XGBRegressor) were compared, with XGBRegressor achieving the best results. In [16], the focus was on Service Time Prediction (STP), which, combined with travel time, determines the number of deliveries that can be planned in a day. Traditional approaches rely on planners’ manual estimates, which are often slow and inaccurate. Instead, a data-driven method based on GPS data was proposed for automatically determining historical service times and applying them in a kNN regression model.

In [17], the authors introduced Transformer-based multi-task Package Delivery Time prediction (TransPDT) for accurately estimating delivery times in complex logistics scenarios where couriers simultaneously perform deliveries and pickups. The model employs a Transformer encoder to capture spatio-temporal dependencies from couriers’ historical routes and shipment sets, while a memory module with attention improves the modeling of pickups, which have stricter time constraints. In [18], the possibility of predicting Service Level Agreement (SLA) travel times for goods and document shipments of PT Pos Indonesia from Java to the islands of Kalimantan, Sulawesi, Maluku, and Papua was analyzed. Random Forest was used for modeling, achieving an average accuracy of 83.86% across four experiments.

Table 1. Comparative overview of selected studies relevant to the application of machine learning in postal and logistics systems within the smart transportation framework.

Reference	Context/Case Study	Methods/Models	Main Contribution	Relevance to Smart Transportation	Improvement in This Study
[6]	Delivery time prediction with e-scooters	RF, GB, kNN, NN	GB achieved best results (R2 = 0.845)	Highlights sustainable urban delivery (green transport, micro-mobility)	- Data augmentation (autoencoder-based) for small datasets - Improved recognition of minority class - Systematic comparison of multiple ML models - Regional case study in Bosnia and Herzegovina - Interdisciplinary methodological relevance transferable to telecommunication traffic - Simpler, faster to train, and implementable in standard environments without specialized hardware
[7]	Last-mile delivery time prediction	Deep learning (DL) within IoT and cloud	Outperformed classical ML and OD models	Integration of IoT, cloud, and AI in smart city context
[8]	Routing and delivery time windows	OR + statistics + ML	Leveraged couriers’ implicit knowledge from historical data	Courier-centered optimization in postal transport
[3]	Urban logistics systems	RF, ResNet	Identified ML models for accurate last-mile delivery time estimation	Provides systematic evidence of ML use in smart transportation
[11]	Delivery time classification	SVM, NB, LDA, ET (ensemble)	Achieved 99.89% accuracy	Large-scale application of ensemble ML in e-commerce logistics
[12]	Service time prediction	Meta-learning, Transformer	Outperformed baselines by 7–9.5%	Advanced DL architecture for dynamic urban conditions
[13]	Real-time delay prediction	LR, RF, XGBoost, CatBoost	Boosting achieved highest AUC (99.9%)	Real-time monitoring and prediction in supply chains
[14]	On-time delivery rate	RCCNet (Node2vec, GCN, LSTM)	Modeled courier interdependencies	Intelligent network modeling for courier efficiency
[17]	Mixed delivery and pickup services	Transformer-based multi-task model	Improved pickup-sensitive delivery predictions	Addresses complexity of urban courier services
[18]	SLA delivery time prediction	Random Forest	83.86% accuracy in four experiments	ML support for inter-island postal transport

provides a comparative overview of selected studies relevant to the application of machine learning in postal and logistics systems within the smart transportation framework. The reviewed papers demonstrate a wide range of approaches, from classical ML models such as Random Forest and Gradient Boosting to advanced deep learning architectures including convolutional and Transformer-based neural networks. While different contexts were analyzed ranging from e-scooter deliveries and last-mile parcel logistics to large-scale e-commerce platforms and national postal operators—the common objective was to improve delivery time estimation, route optimization, and service quality. These contributions highlight the growing importance of integrating ML and data-driven methods into smart transportation systems, with particular relevance for enhancing the efficiency and sustainability of postal transport.

3. Materials and Research Methods

The research methodology was based on a multi-stage process of data processing and modeling, as illustrated in Figure 1. The initial dataset of 11,138 instances first underwent a feature engineering process, in which new variables were derived, followed by a data cleaning phase that reduced the number of instances to 9691. Input variables were defined as numerical, while the output variable was categorical. In this stage, an initial evaluation of several ML models with default hyperparameters was conducted in order to obtain a preliminary insight into performance and directions for further modeling. After additional filtering of the data, resulting in a subset of 4166 instances, the models were tested again under the same conditions to compare results on a smaller but higher-quality sample. Subsequently, two iterations of augmentation were carried out: in Iteration I the dataset was expanded to 8332 instances, while in Iteration II the number of instances was increased to 16,664. In both cases, training, validation, and testing of five ML models with hyperparameter optimization were performed. Finally, an analysis and evaluation of model performance were conducted for both iterations.

3.1. Research Variables and Data

The postal transport data used in this study were provided by the Public Enterprise “BH Pošta,” Postal Center Zenica, based on a formal request. A portion of these structured research data is presented in

Table 2. Research Data.

Instance	Origin Post Office	Date and Time of Acceptance	Destination Post Office	Date and Time of Delivery Recording
1	71300	2 March 2022 08:48:16.347	78418	3 March 2022 09:02:41.000
2	71300	2 March 2022 09:01:11.347	77220	3 March 2022 14:39:20.300
3	71300	2 March 2022 09:16:51.577	77220	3 March 2022 14:39:20.300
4	71300	2 March 2022 09:51:18.597	72220	3 March 2022 08:33:27.537
⋮	⋮	⋮	⋮	⋮
11,138	74266	31 March 2022 13:57:48.097	78230	1 April 2022 14:30:01.000

. The spatial and temporal variables (1) Origin Post Office, (2) Date of Acceptance, (3) Destination Post Office, and (4) Date and Time of Delivery Recording are shown in the table columns, comprising a total of 11,138 instances.

The total number of unique origin post codes (Table 2) within the area organizationally covered by the Postal Center Zenica, where shipments were accepted, is 39. In the observed dataset, the number of destination post offices across Bosnia and Herzegovina is significantly higher, amounting to 365. The data were collected during the period from 2 March 2022 to 31 March 2022.

3.2. Derivation of New Variables

Analysis of the data presented in Table 2 shows that they are not fully suitable for processing with ML methods, which makes it necessary to derive new variables using feature engineering techniques that carry more informative value. The derivation of new variables from the original temporal and spatial variables is performed to identify patterns in the dynamics of postal shipment acceptance, improve the predictive power of the models, and enable easier practical interpretation of the results. Based on the postal addresses of the origin and destination post offices, geocoding was performed using the OpenStreetMap service to obtain the coordinates of these locations in the format of latitude and longitude (lat/long) pairs. The aerial distance between these two points was then calculated using the Haversine distance formula [19]:

$$d=2r·\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{s}\mathrm{i}\mathrm{n}\left(\sqrt{{\sin }^2\left({\displaystyle \frac{∆\phi }{2}}\right)+\cos \left({\phi }_1\right)\cos \left({\phi }_2\right){\sin }^2\left({\displaystyle \frac{∆\lambda }{2}}\right)}\right)$$

(1)

where φ₁ and φ₂ are the geographic latitudes (in radians); λ₁ and λ₂ are the geographic longitudes (in radians); Δφ = φ₂ − φ₁ and Δλ = λ₂ − λ₁; r is the radius of the Earth (≈6371 km); and d is the surface distance on the Earth (in km). Furthermore, based on the coordinates of the destination post office, reverse geocoding was applied to determine the geographical affiliation of this location to a broader region—Republika Srpska (RS), Federation of BiH (FBiH), or the Brčko District in Bosnia and Herzegovina.

From the temporal data of shipment acceptance, two additional variables were derived: day of acceptance in the week (Monday–Saturday) and hour of acceptance during the day (6:00–20:00). In addition, the difference between the acceptance date and time and the delivery date and time was calculated as the delivery time, expressed in hours. These derived variables enable more precise modeling and identification of patterns in postal flows. Thus, in this stage of the research, a total of five independent and dependent variables were derived, as presented in

Table 3. Overview of Derived Variables.

Variable Name	Label	Context	Role in Models
Distance between origin and destination post offices (km)	X1	Spatial	Independent
Territorial affiliation of destination (RS, FBiH, or Brčko District)	X2	Spatial	Independent
Day of acceptance in the week (Monday–Saturday)	X3	Temporal	Independent
Hour of acceptance during the day (6–20 h)	X4	Temporal	Independent
Delivery time required for the shipment (h)	Y	Temporal	Dependent

, two spatial and three temporal.

3.3. Data Cleaning

Out of the initial 11,138 instances, the geocoding algorithm used for determining the distance between the origin and destination post offices failed to assign correct values in 1447 cases. Instead of the actual distance expressed in kilometers, the values of variable X₁ in these instances were equal to zero. By removing such rows, the final dataset was reduced to 9691 instances.

3.4. Training, Validation, and Testing of ML Models with Default Hyperparameters

Before building the ML models, it is a common practice to categorize variables in order to reduce the influence of noise and outliers, making the models more stable and interpretable. This also facilitates the operation of algorithms such as decision trees, where high variability is grouped into broader categories. Variable X₁ was categorized according to the following rules:

• if 0 ≤ distance < 50, then X₁ = “small”;
• if 50 ≤ distance < 150, then X₁ = “medium”;
• if distance ≥ 150, then X₁ = “large.”

Variables X₂ and X₃ already contained natural categorical values:

• X₂: Republika Srpska, Federation of BiH, or Brčko District;
• X₃: 1 (Monday), 2 (Tuesday), …, 6 (Saturday).

The rules for categorizing variable X₄ were as follows:

• if 6 ≤ acceptance hour < 10, then X₄ = “morning”;
• if 10 ≤ acceptance hour < 14, then X₄ = “noon”;
• if 14 ≤ acceptance hour < 20, then X₄ = “afternoon”.

The values of the dependent variable Y were grouped into two classes:

• “express”—delivery time of up to 24 h, and
• “delay”—delivery time exceeding 24 h.

After data preparation, ML models were developed in the IBM SPSS Modeler 18.0 environment using the Auto Classifier node. The dataset with 9691 instances was split into three subsets: 80% for training, 10% for hold-out validation, and 10% for testing. This node automatically generates multiple classification models with default hyperparameters and ranks them according to the accuracy. The auto classifier node generated three default models with the highest accuracy, namely: SVM, C5.0 and Chi-squared Automatic Interaction Detection (CHAID). The SVM was trained with a Radial Basis Function (RBF) kernel (C = 10, gamma = 1.0, epsilon = 0.1), the C5.0 decision tree in simple mode optimized for accuracy without boosting or cross-validation, and the CHAID model with a maximum depth of five levels, significance thresholds of 0.05, and Bonferroni adjustment.

3.5. Data Filtering

The assumption was that focusing on a single dominant location would enable more precise modeling and potentially improve prediction results. The location with the postal code 72102 Zenica, which had the highest frequency of accepted shipments in the observed one-month period, was selected. After filtering, the dataset was reduced to 4166 instances with this postal code, while all others were removed.

3.6. Training, Validation, and Testing of ML Models with Default Hyperparameters and Filtered Data

In this step, ML models were created on the filtered dataset in the same manner as previously described, using the Auto Classifier node. The set of 4167 instances was divided in the same ratio as in the earlier processing, i.e., 80:10:10. For the filtered dataset, in addition to the previously described C5.0 decision tree, two further models were created. The C&R Tree was trained in expert mode with the Gini index, pruning enabled, and prior probabilities based on the training data (minimum 2% parent, 1% child records). The Logistic Regression was implemented as a multinomial main effects model in simple mode, with all predictors entered simultaneously (Enter method) and the constant term included.

3.7. Data Augmentation Using Autoencoder Neural Network—Iteration I

Artificial dataset expansion was carried out using an autoencoder neural network implemented in the MATLAB R2016a environment, with its architecture presented in Figure 2. An autoencoder is a type of feedforward neural network in which information flows strictly from the input layer toward the output, forming a directed acyclic structure. Its purpose is not classification but learning a compact internal representation of the data. The input is projected into a reduced-dimensional space through the hidden layer, where the network seeks to capture the most relevant features. From this latent representation, the model then reconstructs the original input as closely as possible. In this sense, the hidden layer functions as a feature extractor, enabling both dimensionality reduction and input reconstruction [20]. Since the autoencoder network in MATLAB works only with numerical values, each category of independent variables was represented by a unique number using integer coding.

The architecture of the autoencoder, shown in Figure 2, consists of an input layer with four nodes that serve as inputs for independent variables (X₁,…, X₄). The hidden layer has three neurons, and the output layer has four nodes in which the input vectors (X₁′…X₄′) are reconstructed. The transfer functions of the neurons of the hidden and output layer have the form of a logistic sigmoid function (logsig) [20]:

$$f\left(z\right)={\displaystyle \frac{1}{1+e^{-z}}}$$

(2)

where z is the input to the neuron. In addition to the independent variables, the dependent variable was numerically encoded with binary values 1 (“express”) and 2 (“delay”). However, it was not processed through the neural network; instead, its values were directly appended to the synthetic copies generated from the real inputs. In this way, the dataset was doubled, resulting in a total of 8332 instances. The autoencoder was trained with MATLAB 2016a using the trainAutoencoder function with default parameters: logistic sigmoid functions for both encoder and decoder, 1000 training epochs, scaled conjugate gradient optimization, L2 regularization (0.001), sparsity regularization (1) with target sparsity of 0.05, mean squared error with sparsity adjustment as loss function, and automatic data scaling.

3.8. Training, Validation, and Testing of Five ML Models with Hyperparameter Optimization—Iteration I

The ML models were developed in the R programming language and the Rstudio 4.4.3 integrated development environment, and their implementation can be summarized in the following algorithmic steps:

• Data preparation. The research dataset was imported from an Excel file and included independent variables converted into the numeric type (continuous or discrete numerical variables), while the dependent variable was transformed into the factor format (categorical variable).
• Stratified data partitioning. The dataset was split into training (80%), validation (10%), and test (10%) subsets while preserving the class distribution. This ensured an unbiased evaluation of performance.
• Task and resampling definition. A classification task was created on the training and validation sets, while the test set was reserved for the final evaluation. The validation set was processed through a custom resampling procedure with predefined indices.
• Model and hyperparameter definition. Five models were developed: Random Forest, SVM (RBF kernel), XGBoost, kNN, and MLP [21]. For each model, a hyperparameter space or search interval was predefined, as presented in

  Table 4. Search Intervals for the Optimal Values of ML Model Hyperparameters.

  Model	Hyperparameter	Search Interval
  Random Forest	num.trees—number of trees in the forest	200–1000
  	mtry.ratio—proportion of variables randomly selected at each split	0.2–1.0
  	min.node.size—minimum number of instances in a terminal node	1–20
  SVM (RBF)	cost—regularization parameter (C)	0.1–10
  	gamma—RBF kernel parameter (γ)	0.0001–1
  XGBoost	nrounds—number of boosting iterations	100–700
  	eta—learning rate	0.01–0.3
  	max_depth—maximum depth of each tree	2–8
  	subsample—proportion of data used for each tree	0.5–1.0
  	colsample_bytree—proportion of variables for each tree	0.5–1.0
  kNN	k—number of nearest neighbors	3–31
  MLP	size—number of neurons in the hidden layer	3–30
  	decay—weight decay regularization rate	0.00001–0.1
  	maxit—maximum number of training iterations	100–500

  .

5.

Optimization and training. Hyperparameters were optimized using Bayesian optimization with 30 evaluation cycles, where the target metric was overall classification accuracy. Each model was trained on the training set and validated on the validation subset, and then retrained with the best hyperparameters and tested on the independent test set.

6.

Performance evaluation. Model performance was assessed using the following metrics:

• Overall accuracy—the ratio of the total number of correctly classified cases to the total number of cases (instances), calculated according to the following formula [22]:

  $$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{l}\mathrm{y} \mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{e}\mathrm{d} \mathrm{c}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{s}}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{a}\mathrm{s}\mathrm{e}\mathrm{s}}}$$

  (3)
• Balanced accuracy. Accuracy measures the proportion of correctly classified instances but can be misleading in the case of imbalanced classes. Balanced accuracy is calculated as the average accuracy across classes [23]:

  $$\mathrm{B}\mathrm{a}\mathrm{l}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{d} \mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{1}{K}}\sum _{i=1}^K{\displaystyle \frac{T{P}_i}{T{P}_i+F{N}_i}}$$

  (4)

  where TP_i (True Positive) is the number of correctly recognized instances of class i, FN_i (False Negative) is the number of misclassified instances of that class, and K is the total number of classes.
• F-beta—represents the harmonic mean of precision and recall [24]:

  $${\mathrm{F}}_{\beta }=(1+{\beta }^2)·{\displaystyle \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}·\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\left({\beta }^2·\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\right)+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

  (5)

  where the parameter β (beta) determines the relative importance given to recall versus precision (or vice versa):
  β = 1: the classical F1 score (balanced precision and recall).
  β > 1: recall is emphasized more.
  β < 1: precision is emphasized more.
• Precision is defined as the ratio of correctly classified positive instances (TP) to the total number of instances predicted as positive by the model [22]:

  $$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{TP}{TP+FP}}$$

  (6)
• Recall represents the ratio of correctly classified positive instances to the total number of actual positive instances in the dataset [22]:

  $$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{TP}{TP+FN}}$$

  (7)
• Area Under the Curve (AUC). The AUC is defined as the integral of the True Positive Rate (TPR) (also called sensitivity) over the False Positive Rate (FPR) (corresponding to 1 − specificity) [25]:

  $$\mathrm{A}\mathrm{U}\mathrm{C}={\int }_0^1{\displaystyle \frac{TP}{P}}d{\displaystyle \frac{FP}{N}}$$

  (8)

  where P and N are the counts of positive and negative observations. In addition to the presented metrics, confusion matrices were generated, while the results of the optimal hyperparameters and predictions were saved in separate files. Within the implemented algorithm, the following R libraries were used: readxl, dplyr, mlr3, mlr3verse, mlr3learners, mlr3tuning, paradox, data.table, tidygeocoder, writexl, geosphere. To ensure the reproducibility of results, a fixed random seed value (set.seed(123)) was applied in the experiments.

3.9. Data Augmentation Using Autoencoder Neural Network—Iteration II

Analyzing the results of the ML models created on the data expanded through the first augmentation iteration, it was concluded that there was still room for improving their classification performance. Therefore, in this case, the dataset of 8332 instances of independent variables was used as input to the autoencoder network, while reconstructed copies were generated at the output, as explained in Section 3.7. By merging the input data with the generated synthetic data, the total dataset amounted to 16,664 instances (Figure 3).

3.10. Training, Validation, and Testing of Five ML Models with Hyperparameter Optimization—Iteration II

The second iteration of building Random Forest, SVM, XGBoost, kNN, and MLP models was based on the same algorithm described in Section 3.8. The models were trained, validated, and tested on a dataset enlarged fourfold, with a total size of 16,664 instances.

3.11. Analogy of Applying the Solution in Telecommunication Traffic

The methodological framework developed for classification modeling of the postal transport system can also be applied in telecommunication traffic, given the numerous similarities between these domains, which can be summarized as follows:

• Telecommunications originated within postal companies (telegraph and telephone as support for faster transmission of letter mail) [26];
• The basic unit of transfer or transport can be either a physical postal item or a digital data packet traveling through a network of nodes (post offices or routers/base stations) from source to destination;
• Both postal and telecommunication systems function as complex networks of nodes connected by routes or communication channels for data exchange [27,28];
• Both postal and telecommunication services require advanced methods of planning and flow monitoring to ensure efficiency and service quality [29,30];
• Continuous technological advancement affects not only the volume of services but also their nature, introducing new forms of service offerings and transforming the way traditional services are used and perceived;
• Both sectors are subject to regulation that ensures their role within the broader networked system [29].

The variables used in this study, such as physical (geographical) distance, territorial affiliation, acceptance time, and delivery time, have direct parallels in telecommunication traffic variables such as latency, jitter, throughput, and packet loss. The application of ML, which is already successfully used in telecommunications for traffic classification, congestion prediction, and quality of service/experience (QoS/QoE) optimization, can further improve reliability and efficiency in this context. A clear parallel can also be drawn between postal shipments and traffic classes in telecommunication networks. Express shipments correspond to streaming traffic, where the priority is fast and reliable delivery with minimal delay, similar to real-time video or audio streams. Conversely, delayed shipments can be associated with background traffic, where delivery is mandatory and content integrity must be preserved, but delivery time is neither strictly defined nor expected in real time [31]. This analogy demonstrates that the classification and prediction methodology applied in the postal transport system is directly transferable to telecommunication networks, confirming its interdisciplinary significance.

3.12. Software and Computational Environment

Machine learning models with hyperparameter optimization were implemented in R version 4.4.3 (28 February 2025, “Trophy Case”) on the platform x86_64-w64-mingw32/x64. The following R packages were used: readxl 1.4.5, dplyr 1.1.4, mlr3 1.0.0, mlr3verse 0.3.1, mlr3learners 0.12.0, mlr3tuning 1.4.0, paradox 1.0.1, data.table 1.17.0, tidygeocoder 1.0.6, writexl 1.5.4, and geosphere 1.5–20. Baseline models with default hyperparameters were created in IBM SPSS Modeler 18.0 (IBM Corp., Armonk, NY, USA). Data augmentation was carried out in MATLAB R2016a (MathWorks Inc., Natick, MA, USA). All computations were performed on a workstation running Windows 10 Pro (64-bit) with an Intel(R) Core(TM) i5-6300U CPU @ 2.40 GHz, 16 GB RAM, without GPU acceleration.

Compared with deep learning approaches such as ResNet, Transformer-based models, or RCCNet, which demand large datasets, GPU acceleration, and high computational resources, the proposed framework (Random Forest, XGBoost, SVM, kNN, MLP) is simpler, faster to train, and implementable in standard environments without specialized hardware. This ensures a favorable trade-off, achieving competitive accuracy with lower resource consumption, while ensemble methods further reduce sensitivity to error propagation compared to more complex deep learning architectures.

4. Results

4.1. ML Models with Default Hyperparameters Created on the Cleaned Dataset

The initial modeling results on the cleaned dataset of 9691 instances are presented in

Table 5. Three Best-Ranked ML Models by Accuracy.

Model	Test Set			Training Set
	Accuracy (%)	Precision	Recall	Accuracy (%)	Precision	Recall
SVM	73.810	0.722	0.721	74.14	0.729	0.728
C5.0	73.700	0.721	0.721	74.03	0.728	0.728
CHAID	73.390	0.718	0.717	73.16	0.719	0.717

. Using the automated modeling method, three models with the best performance in terms of overall classification accuracy on the test set were created and selected: SVM, C5.0, and CHAID. In addition to the accuracy metrics, Table 5 also shows the Precision and Recall macro average values for the training and test sets.

Based on the results presented in Table 5, it can be concluded that all three models achieved similar performance. The best result was obtained by the SVM model (73.810%), closely followed by C5.0 (73.700%), while CHAID achieved a slightly lower value (73.390%). The small differences in model accuracy shown in Table 5 indicate that the cleaned dataset has a limited informational capacity, and no algorithm achieves significantly higher results. The comparison shows that all three models achieved very similar precision and recall values on both the training and test sets, indicating stable performance without signs of overfitting. The confusion matrix (

Table 6. Confusion Matrix Obtained by Automated Modeling.

Prediction\Actual Class	Express	Delay	Total (Prediction)
express	4629	1268	5897
delay	1260	2534	3794
Total (actual class)	5889	3802	9691

) presents the aggregated results for the three best models on the entire dataset. In total, the models correctly classified 4629 express shipments and 2534 delayed shipments, while in 1260 cases an express shipment was misclassified as delay, and in 1268 cases a delayed shipment was classified as express, resulting in a total of 9691 instances.

According to Table 6, the models achieved an overall accuracy of approximately 73.91%, with 7163 out of 9691 shipments correctly classified. Class-wise analysis shows that the models are somewhat more reliable in recognizing express deliveries. One of the reasons for these results is a slight imbalance between the classes.

4.2. ML Models with Default Hyperparameters Created on the Filtered Data

The results of training and testing the models on the filtered dataset of 4166 instances are presented in

Table 7. Three Best-Ranked ML Models by Accuracy (for Filtered Data).

Model	Test Set			Training Set
	Accuracy (%)	Precision	Recall	Accuracy (%)	Precision	Recall
C5.0	79.460	0.748	0.653	79.33	0.769	0.677
C&R Tree	79.230	0.744	0.648	79.33	0.771	0.675
Logistic Regression	79.010	0.758	0.626	78.79	0.796	0.645

. It can be concluded that the C5.0 model achieved slightly better results (79.460%) compared to the C&R Tree (79.230%) and Logistic Regression (79.010%). Although the differences are minimal, it may be concluded that the C5.0 model was the most successful in classification in this experiment. C5.0 and C&R Tree delivered a balanced relation between precision and recall, whereas Logistic Regression emphasized higher precision at the expense of lower recall.

According to the confusion matrix (

Table 8. Confusion Matrix Obtained by Automated Modeling (for Filtered Data).

Prediction\Actual Class	Express	Delay	Total (Prediction)
express	2837	688	3525
delay	178	463	641
Total (actual class)	3015	1151	4166

), the model achieved an overall accuracy of approximately 79.2%, with 3300 out of 4166 shipments correctly classified. Class-wise analysis reveals that the model is significantly more reliable in recognizing express deliveries (2837 correctly classified cases) compared to delayed shipments, where only 463 cases were correctly identified.

By comparing the results in Table 5 and Table 7 an increase in accuracy for all presented models is evident, despite the reduction in dataset size. The increase in accuracy between the best-ranked models amounts to 5.65%, confirming the validity of the assumption that focusing on a single postal location is beneficial. Furthermore, the results suggest that all models achieved approximately the same accuracy, which indicates that the dataset has reached its informational limit. In other words, in this case, performance does not largely depend on the ML algorithm itself or on model hyperparameters.

4.3. ML Models Created with Hyperparameter Optimization—Iteration I

The results presented in Figure 4 and Figure 5 show that all models achieved consistent performance on both the training and test sets, with only minor differences, which indicates that there is no evident overfitting. Random Forest and XGBoost recorded the most stable outcomes, with accuracy of 0.90 on the training set and 0.91 on the test set, F-beta values of 0.93 and 0.94, and AUC of 0.94 and 0.93, respectively. These results confirm the robustness of ensemble approaches. MLP closely followed, with training accuracy of 0.89 and test accuracy of 0.91, accompanied by F-beta values of 0.93 and 0.94, which suggests that it generalizes well to unseen data. SVM (RBF) showed slightly lower values, with accuracy of 0.88 (train) and 0.89 (test), and stable F-beta of 0.92–0.93, while its AUC remained significantly lower (0.84–0.85), reflecting reduced class separability. The kNN model performed at a similar level to SVM, with accuracy of 0.88–0.89 and F-beta of 0.92, but without reaching the balanced performance of Random Forest, XGBoost, and MLP. Overall, the comparison of training and test results demonstrates that the models are well-calibrated, with ensemble models (Random Forest and XGBoost) and neural networks (MLP) providing the most reliable solutions, while SVM and kNN exhibit modest but consistent performance.

The analysis of the confusion matrices presented in Figure 6 reveals that all models achieved a high level of accuracy, though the distribution of errors varied across classes. Random Forest and XGBoost confirmed their robustness, with XGBoost showing the lowest number of misclassifications of Class 1, while Random Forest maintained balanced predictions between both classes. MLP also delivered stable results, with only a slightly increased number of errors for Class 2. SVM produced acceptable outcomes but remained less effective compared to ensemble methods and neural networks. The kNN model, on the other hand, exhibited the weakest performance, primarily due to the highest number of misclassified instances of Class 1, while its errors in Class 2 were comparable to those of the other models. Overall, ensemble models (Random Forest and XGBoost) and MLP stood out as the most reliable approaches, whereas SVM and kNN showed clear limitations in classification consistency.

4.4. ML Models Created with Hyperparameter Optimization—Iteration II

Figure 7 and Figure 8 illustrates that all evaluated models performed at a high level, though with certain differences between the metrics. Random Forest and XGBoost proved to be the most consistent, both reaching accuracy of 0.95 on training and test data, balanced accuracy in the range of 0.91–0.93, F-beta values of 0.96–0.97, and the highest AUC scores of 0.98–0.99. SVM and kNN followed closely, with slightly lower balanced accuracy (0.90–0.92), which points to a modest sensitivity to class imbalance, while their F-beta (0.96) and AUC (0.97–0.98) remained at a strong level. MLP showed the weakest results, recording accuracy of 0.89–0.90, balanced accuracy of 0.83–0.84, and AUC values of 0.93–0.94, making it less competitive under the given conditions. When comparing training and test outcomes, it can be observed that all models achieved nearly identical values across metrics. This indicates that there is no evident overfitting or underfitting, as the models generalized well to unseen data. The consistency of ensemble methods (Random Forest and XGBoost) further underlines their robustness, while neural networks (MLP) demonstrated slightly lower but still stable generalization performance.

In the new set of heat maps (Figure 9), all models maintained a high level of accuracy, with Random Forest and XGBoost again delivering the most balanced results and the fewest misclassifications across both classes, while MLP recorded a notably higher number of errors for Class 2 compared to the previous set. Compared to the matrices in Figure 6, SVM and kNN remained stable but continued to lag behind the ensemble approaches, confirming the consistent advantage of Random Forest and XGBoost in both iterations.

In summary, the comparison between Iteration I and Iteration II demonstrates that Random Forest, XGBoost, SVM, and kNN achieved clear performance gains, with accuracy rising to about 0.95, balanced accuracy improving by 7–8 percentage points, F1 scores exceeding 0.96, and AUC values approaching 0.99. These consistent improvements across multiple test metrics (accuracy, balanced accuracy, F1, precision, recall, and AUC) indicate that the observed gains are statistically significant, while MLP showed no notable change.

Table 9. Optimal Hyperparameter Values of ML Models Created in the Second Iteration.

Model	Hyperparameter	Optimal Value
Random Forest	num.trees	204
	mtry.ratio	0.207
	min.node.size	13
SVM (RBF)	cost	9.958
	gamma	0.997
XGBoost	nrounds	433
	eta	0.243
	max_depth	5
	subsample	0.658
	colsample_bytree	0.882
kNN	k	27
MLP (nnet)	size	20
	decay	0.037
	maxit	182

presents the optimal hyperparameter values for all ML models obtained through Bayesian optimization. The results indicate that ensemble methods, particularly Random Forest and XGBoost, required fine-tuning of several parameters to achieve robust and balanced performance. For kNN and MLP, the optimization process identified values that enhanced model stability and ensured proper convergence, while for SVM the relatively high cost and gamma confirmed its sensitivity to the structure of the dataset. Overall, the optimized hyperparameters calibrated each model to the specific data characteristics, allowing for a reliable and fair comparison of their predictive performance.

5. Conclusions

For the purpose of this study, postal systems were selected as the subject of analysis due to their highly complex organizational and functional structures within transport activities. By their target function, these are spatially distributed systems designed to provide products and services that meet a broad portfolio of urban needs, including urban mobility that develops in environments characterized by heterogeneous and context-sensitive behavior of service users and postal operators in multimodal and unstable urban scenarios.

This article presents an interdisciplinary comparative approach to improving the classification of postal delivery times using machine learning (ML) models. Several classification models—Random Forest, SVM, XGBoost, kNN, and MLP—were analyzed to address business and functional challenges in the context of applying the smart transportation concept. Their hyperparameters were optimized using the Bayesian method and evaluated through standard classification metrics for postal delivery time classification, with the aim of enhancing efficiency and service quality. The focus on applying ML models for delivery time classification is well-motivated and aligned with broader trends in digital transformation within logistics and service optimization.

The Smart Transportation Concept (STC) enables postal systems to become part of a wider ecosystem of smart mobility, where the integration of new technologies, data exchange, and ML applications supports resource optimization, more accurate predictions, and the provision of more efficient and environmentally sustainable services for both individual and business users. The study shows that this concept has not yet reached the necessary level of implementation within the cyclical loop of planning, execution, management, and reflection, in relation to the requirements of sustainability, equity, and resilience to various categories of constraints in the business processes, strategies, and policies of multimodal transport systems.

An important innovative contribution of this study is the development of an original methodological framework for implementing the smart transportation concept in the evolution of the postal multimodal system, with the goal of creating a functional, ecologically, and economically sustainable system capable of reflective adaptation to environmental disruptions and changes in the personal preferences of postal service users as well as the business demands placed on postal operators.

The particular relevance of this study, which addresses critical challenges in postal traffic and transport, is reinforced by the unpredictable and growing parcel volumes, sustainability pressures, and the need for integration into smart transportation systems. For this purpose, a real dataset with numerous instances was collected at the Postal Center Zenica, Bosnia and Herzegovina, and processed through procedural operations, including data augmentation, followed by analytical application of five ML models. All developed models are consistently described and comparatively analyzed, with results presented concisely through textual explanations, tables, graphics, and visual interpretations.

The final dataset was divided into training, testing, and evaluation groups in the ratio of 80:10:10. The presented results demonstrate the application of a relevant methodological procedure, with performance metrics confirming the value of data augmentation in improving performance, particularly for minority classes (delayed deliveries). The emphasis on ensemble models (Random Forest, XGBoost) as robust solutions further strengthens the practical implications. The claim of interdisciplinary applicability in telecommunications adds additional significance to this article. The final conclusion is that the paper contains all key elements necessary for adequate evaluation, including compliance with the scope of the journal, supported by an appropriate visual overview and systematization of the cited literature.

The most important limitation of this study is the size of the dataset collected within the observed geographic area. Data augmentation proved useful in overcoming this limitation and in improving model performance, but it should be noted that augmentation must be applied with caution and limited to a certain number of iterations.

Future research will be oriented toward realistic sociodemographic attributes of service users and postal operators, enabling heterogeneous and context-sensitive behavior in multimodal and unstable urban scenarios. Furthermore, replicability could be enhanced by considering techniques such as the Synthetic Minority Over-sampling Technique (SMOTE) or class weighting to address class imbalance issues.