Multi-Criteria Genetic Algorithm for Optimizing Distributed Computing Systems in Neural Network Synthesis

Abstract

Artificial neural networks (ANNs) are increasingly effective in addressing complex scientific and technological challenges. However, challenges persist in synthesizing neural network models and defining their structural parameters. This study investigates the use of parallel evolutionary algorithms on distributed computing systems (DCSs) to optimize energy consumption and computational time. New mathematical models for DCS performance and reliability are proposed, based on a mass service system framework, along with a multi-criteria optimization model designed for resource-intensive computational problems. This model employs a multi-criteria GA to generate a diverse set of Pareto-optimal solutions. Additionally, a decision-support system is developed, incorporating the multi-criteria GA, allowing for customization of the genetic algorithm (GA) and the construction of specialized ANNs for specific problem domains. The application of the decision-support system (DSS) demonstrated performance of 1220.745 TFLOPS and an availability factor of 99.03%. These findings highlight the potential of the proposed DCS framework to enhance computational efficiency in relevant applications.

1. Introduction

The widespread adoption of constantly connected devices over the past decade has led to a notable increase in energy consumption, creating challenges for the sustainability and efficiency of computational systems [1]. As global energy demands rise and electricity costs grow, the need to find effective solutions to optimize energy usage in communication and computation is becoming more evident [2,3]. Resource Block Configurations (RBCs) provide a practical approach to addressing these challenges by reducing both the training time for neural networks and the energy consumption of computing clusters [4,5].

Artificial Neural Networks (ANNs), as mathematical constructs inspired by the structure and function of biological neural networks, have transformed various domains by enabling advancements in prediction, pattern recognition, and control [6]. Originating from early studies on cognitive processes, ANNs have evolved into powerful tools supported by learning algorithms, making them indispensable in tackling complex real-world problems. However, the computational demands associated with training and deploying ANNs remain a barrier, particularly as models grow in complexity and scale [7].

Recent research has underscored the potential of machine learning (ML) algorithms, including ANNs, in diverse applications such as wireless networks, materials science, and industrial diagnostics [8,9]. For instance, ML has been successfully applied to optimize wireless communication systems [10], automate diagnostics in industrial equipment [11], and predict material properties with high accuracy [12]. Yet, these applications often demand substantial computational resources, underscoring the importance of optimizing the underlying computational frameworks.

Despite advancements in learning algorithms, ANNs still face fundamental challenges. For instance, the ability of ANNs to dynamically connect and integrate information is critical for developing an understanding of symbolic entities, such as objects [13]. Inadequacies in this area limit generalization capabilities and decision-making consistency. Addressing these limitations requires frameworks that combine segregation, representation, and composition of information [14].

Parallel and distributed computing systems (DCSs) have emerged as a transformative solution to manage the escalating computational demands of ANNs. By leveraging parallelism, distributed systems divide large-scale computational tasks into smaller, more manageable subtasks, which are then processed concurrently [15]. This approach not only improves computational efficiency but also enables scalable solutions for resource-intensive applications [16]. Furthermore, distributed evolutionary algorithms offer advantages over traditional sequential methods, particularly in optimizing ANN architectures [17].

This study presents a novel, integrated framework that combines RBCs with DCSs for energy-efficient and performance-optimized training of ANNs. Unlike prior works that apply traditional optimization methods in static or homogeneous computing environments [18,19,20], this approach introduces the use of parallel GAs specifically adapted to heterogeneous and dynamically reconfigurable DCS architectures. The framework uniquely incorporates the following:

• Custom mathematical models for evaluating both performance and availability metrics of DCS nodes, enabling precise matching between computational tasks and resource profiles.
• A multi-objective optimization scheme that leverages Pareto-front analysis to jointly optimize energy consumption, computational latency, and system reliability—parameters that are rarely optimized in combination in the related literature.
• A GA-based decision-support system, which does not merely optimize ANN hyperparameters but also orchestrates resource allocation strategies across distributed nodes, adapting in real time to system feedback and load conditions.

2. Related Works

To present the results of the following studies that use GA and multi-criteria optimization methods in different fields, Table 1 summarizes the key aspects of each paper. It includes references to the respective studies, the focus of each work, the applied models, results, and the limitations of the proposed approaches

Chan et al. [21] present a multi-criterion genetic optimization algorithm for addressing distribution network problems in supply chain management, integrating GA with analytic hierarchy processes (AHPs) [22]. GA generates potential solutions iteratively and evaluates their fitness based on predefined criteria. In the proposed approach, the AHP enhances GA by enabling decision-makers to assign weightings to multiple criteria using pairwise comparisons, facilitating more informed decision-making. Chromosome representation captures demand allocations among customers, warehouses, and manufacturers, with a mechanism to split and optimize orders exceeding supply capacities. Fitness values are calculated using the AHP, considering total cost, delivery time, and capacity utilization, providing a robust multi-criterion evaluation. Operators such as selection, crossover, mutation, and monitoring refine the solution pool iteratively, ensuring convergence towards optimal solutions. Numerical results demonstrate that the combined GA+AHP algorithm is reliable, offering decision-makers greater control and insight into distribution network optimization. This approach improves decision-making by balancing cost, efficiency, and service level priorities effectively.

Job scheduling is a key factor influencing the efficiency of complex, heterogeneous distributed systems, such as computational Grids. These environments demand multi-criteria scheduling algorithms to address their diverse operational requirements effectively. Gkoutioudi et al. [23] introduce a multi-criteria GA that prioritizes minimizing security risks and power consumption alongside job completion time. While GAs are well-suited for large-scale problems like job scheduling, their computational demands can be prohibitive for real-time applications. To overcome this limitation, we propose an Accelerated Genetic Algorithm (AGA), optimized for faster convergence, and an AGA with Overhead (AGAwO) [24], which incorporates the algorithm’s computational delay. A simulation-based evaluation compares the AGA and AGAwO with established heuristics, demonstrating their superior performance across multiple criteria. The proposed approach leverages an open queuing network model with geographically distributed clusters, utilizing diverse processor characteristics for better energy efficiency and security. The results confirm the effectiveness of the AGA and AGAwO in achieving balanced and efficient job scheduling in multi-factor computational Grids.

In optimizing electrical distribution systems, Pareto front analysis effectively handles multiple conflicting objectives. Mazza et al. [25] focus on reconfiguration to minimize network losses and energy not supplied, employing a GA-based solver to build and update the Pareto front. Key innovations include extended crossover and mutation operators, and ranking solutions using multi-criteria decision-making methods like AHP and TOPSIS [26]. These enhancements allow automatic decision-making support and identification of preferred solutions on the Pareto front. The approach is validated on reference test networks, revealing effective multi-objective optimization and insightful performance comparisons using geometrical metrics. This framework is extendable to microgrids and scalable networks, offering a comprehensive optimization strategy.

Jing et al. [27] develop a two-stage framework for distributed energy system design combining $\mathsf{ϵ}$-constraint optimization and multi-criteria evaluation. In the first stage, design and dispatch variables (e.g., solar collector area, SOFC stack size, battery capacity) are optimized via an $\mathsf{ϵ}$-constraint formulation—minimizing annualized cost while bounding CO₂ emissions and exergy efficiency at discrete $\mathsf{ϵ}$-levels—to generate a Pareto frontier. Candidate solutions are then ranked using LINMAP (Euclidean distance to an ideal point), TOPSIS (relative closeness to positive/negative ideals), and Shannon entropy-derived weights. In the second stage, a subset of Pareto solutions is evaluated through the Analytic Hierarchy Process (AHP) and the Gray Relational Analysis (GRA). The AHP employs Saaty’s 1–9 scale and enforces consistency ratios < 0.10 to derive criterion weights [28]. These weights feed into the GRA, where gray relational coefficients (distinguishing coefficient 0.5) produce a composite grade for each alternative. Applied to a solar-assisted SOFC system across residential and commercial building archetypes in three climate zones, the framework uses a MINLP model with temperature-dependent SOFC curves and hourly solar data, solved in GAMS/DICOPT (tolerance 10⁻⁶). Results show 12–18% cost savings and 20–25% emissions reduction versus baseline, demonstrating robust, multi-objective decision support.

Wen et al. [29] apply GA with binary-real encoding and tournament selection to optimize distributed energy systems (DESs) in shopping malls, office buildings, and hotels. Seven dispatch strategies—FTL, FEL, FHL-ns, FHL-ms, FHL-nst, FMLR, and FSLR—are evaluated using hourly load profiles and energy simulations. Optimal CHP capacities range from 623 kW to 1 782 kW, with electric cooling ratios (electric chiller capacity/total cooling load) between 0.5 and 0.9. Performance is measured by primary energy saving

$${\eta }_{\mathrm{PES}}=1-{\displaystyle \frac{E_{\mathrm{DES}}}{E_{\mathrm{ref}}}},$$

and CO₂ reduction

$$\Delta {\mathrm{CO}}_2={\displaystyle \frac{M_{\mathrm{ref}}-M_{\mathrm{DES}}}{M_{\mathrm{ref}}}},$$

reaching 29.8% and 49.5%, respectively. FMLR proves most effective in malls and hotels, while FHL-nst excels in office buildings by time-shifting heat pump operation (COP ≈ 3.5). Sensitivity analysis shows that higher coal-to-gas fuel mixes degrade both ${\eta }_{\mathrm{PES}}$ and $\Delta {\mathrm{CO}}_2$, underscoring the GA’s utility in complex, multi-objective DES design.

Table 1. Summary of MCDM approaches using GAs.

Reference	Focus	Applied Model	Results	Limitations
[21]	Multi-criterion optimization for distribution networks in supply chains	GA integrated with Analytic Hierarchy Process (AHP)	Enhanced decision-making with better balance between cost, efficiency, and service levels.	May require extensive computational resources for large-scale problems.
[23]	Job scheduling in computational Grids considering multiple criteria	AGA and AGA with Overhead (AGAwO)	Superior performance across multiple criteria for energy efficiency and security.	High computational demands; overhead considerations for real-time applications.
[25]	Electrical distribution system optimization using Pareto fronts	GA with AHP and TOPSIS for decision-making	Effective multi-objective optimization with scalable applications to microgrids and larger networks.	Results dependent on the selection of criteria weighting and operator tuning.
[27]	Planning of distributed energy systems with multi-criteria evaluation	$\epsilon$-constraint optimization with AHP and Gray Relation Analysis	Comprehensive framework integrating optimization and evaluation for diverse scenarios.	Complex model setup; reliance on accurate input data for optimization.
[29]	Optimization of distributed energy systems in commercial buildings	GA	Identified optimal strategies for energy and CO2 reductions, tailored by building types.	Context-specific findings may not generalize to non-commercial settings.
[30]	Logistics performance evaluation using MCDM techniques	GA for criteria weighting	More accurate logistics rankings and frequent evaluations with GA outperforming traditional methods.	Limited to the Logistics Performance Index framework; model effectiveness depends on input data quality.
[31]	Framework for Distributed Multi-Energy Systems (DMES)	GA with Maximum Rectangle Method	Enhanced DMES performance with tailored operation strategies like Following Electrical Load (FEL).	Performance influenced by load uncertainties and market conditions.
[32]	Hybrid energy system for electricity and desalination	GA and ANN	Improved efficiencies and reduced costs; path to sustainable energy and water production.	System performance depends on optimal parameter selection and operational stability.

Table 1. Summary of MCDM approaches using GAs.

Reference	Focus	Applied Model	Results	Limitations
[21]	Multi-criterion optimization for distribution networks in supply chains	GA integrated with Analytic Hierarchy Process (AHP)	Enhanced decision-making with better balance between cost, efficiency, and service levels.	May require extensive computational resources for large-scale problems.
[23]	Job scheduling in computational Grids considering multiple criteria	AGA and AGA with Overhead (AGAwO)	Superior performance across multiple criteria for energy efficiency and security.	High computational demands; overhead considerations for real-time applications.
[25]	Electrical distribution system optimization using Pareto fronts	GA with AHP and TOPSIS for decision-making	Effective multi-objective optimization with scalable applications to microgrids and larger networks.	Results dependent on the selection of criteria weighting and operator tuning.
[27]	Planning of distributed energy systems with multi-criteria evaluation	$\epsilon$-constraint optimization with AHP and Gray Relation Analysis	Comprehensive framework integrating optimization and evaluation for diverse scenarios.	Complex model setup; reliance on accurate input data for optimization.
[29]	Optimization of distributed energy systems in commercial buildings	GA	Identified optimal strategies for energy and CO2 reductions, tailored by building types.	Context-specific findings may not generalize to non-commercial settings.
[30]	Logistics performance evaluation using MCDM techniques	GA for criteria weighting	More accurate logistics rankings and frequent evaluations with GA outperforming traditional methods.	Limited to the Logistics Performance Index framework; model effectiveness depends on input data quality.
[31]	Framework for Distributed Multi-Energy Systems (DMES)	GA with Maximum Rectangle Method	Enhanced DMES performance with tailored operation strategies like Following Electrical Load (FEL).	Performance influenced by load uncertainties and market conditions.
[32]	Hybrid energy system for electricity and desalination	GA and ANN	Improved efficiencies and reduced costs; path to sustainable energy and water production.	System performance depends on optimal parameter selection and operational stability.

Gurler et al. [30] evaluate countries’ logistics performance using the Logistics Performance Index (LPI) and address its limitations by introducing a GA-based model for criteria weighting. Employing 11 Multi-Criteria Decision-Making (MCDM) techniques across 33 indicators ensures robust performance analysis. Missing data are imputed using methods like mean, median, linear regression, and KNN, with linear regression identified as the most effective. GA outperforms traditional weighting methods like CRITIC and Entropy, highlighting key criteria such as goods exports, road quality, and GDP per capita. These weights align more closely with actual LPI scores. The approach enables more frequent performance evaluations, supporting countries in improving logistics rankings.

The building sector, as a significant energy consumer, requires a transition to cleaner energy systems. A Distributed Multi-Energy System (DMES) integrates renewable energy and storage solutions. Wang et al. [31] propose a framework which includes a Maximum Rectangle Method and GAs for capacity optimization and operational strategy design. The evaluation combines the Analytic Hierarchy Process, Entropy Weight Method, and Set Pair Analysis. Performance analysis of the DMES in various scenarios, such as energy prices and load uncertainties, highlights its superiority over alternatives. Optimal operation strategies like Following Electrical Load (FEL) enhance DMES performance. Recommendations include increasing renewable energy integration and improving load forecasting. The findings offer valuable guidance for sustainable energy applications in buildings.

Hai et al. [32] explore an innovative hybrid energy system that integrates flame-assisted fuel cells and a multi-effect desalination process to produce electricity and potable water efficiently and affordably. Using advanced modeling through the Engineering Equation Solver (EES), the system’s performance is analyzed in terms of energy, exergy, economics, and environmental sustainability. Key findings reveal high energy (62%) and exergy (55%) efficiencies with a cost of USD 0.123/kWh and an environmental impact of 524 kg/MWh. Optimization via GAs and artificial ANNs improves exergy efficiency to 61.1% and reduces costs to USD 0.03/kWh. Parametric studies show that lower fuel utilization factors and higher current densities enhance performance but increase power costs. Exergy destruction is most significant in the mixer, desalination unit, and fuel cells. Sustainability improves with lower equivalence ratios and optimized compressor pressure and fuel cell temperatures. This approach offers a path to cost-effective, sustainable energy and water production.

3. Problem Statement

ANNs are currently recognized as powerful tools for addressing a wide range of complex scientific and technical challenges. However, the lack of a standardized approach for structuring and synthesizing the parameters of ANN models significantly impedes their practical application. Additionally, the computational demands of evolutionary algorithms often require substantial hardware resources, making their execution difficult on conventional personal computers.

To mitigate this issue, the use of parallel evolutionary algorithms or DCSs can significantly reduce computational time. Parallel execution allows for the simultaneous processing of calculations, a feature that sequential programming cannot offer. Distributed computing networks (CNs) provide the necessary hardware infrastructure for parallel processing, offering a promising solution to the computational challenges of ANN-based models.

Advancements in network information technology and the availability of high-performance personal computers have greatly expanded access to computational resources, enabling more efficient solutions to complex problems. The capability of leveraging portable networks further enhances the computational potential of these systems.

In tackling complex scientific and technical problems through computing, it is essential to develop formal methodologies that optimize network models, ultimately resulting in hardware–software systems capable of performing primary functions. The application of a Decision Support System (DSS) during the early stages of CN design or when modifying existing networks is vital. These systems are typically tailored to address intricate problems, driving technological advancements based on expert knowledge.

DCSs are characterized by the spatial separation of their components and subsystems, which may arise due to size, environmental constraints, or specific functions. Among the most commonly used DCSs are centralized CNs, and this study focuses on their functional capabilities in addressing complex challenges.

The choice of the GA as the core optimization method in this study is grounded in several methodological and practical considerations. First and foremost, GAs exhibit a well-established capacity for handling complex, multi-modal, and high-dimensional optimization tasks, which are characteristic of resource allocation and control in heterogeneous DCSs [33,34].

Second, the inherently population-based structure of GAs maps naturally onto DCSs: each subpopulation (or “island”) can be assigned to a different compute node, enabling concurrent evaluation and recombination with minimal synchronization overhead [35]. This island-model parallelism both accelerates convergence and enhances solution diversity in spaces such as those arising in ANN hyperparameter tuning and resource allocation. In contrast, PSO’s update uses a swarm-wide global best:

$$v_i^{t+1}=w{v}_i^t+c_1{r}_1(pbes{t}_i-x_i^t)+c_2{r}_2(gbest-x_i^t),$$

which explicitly depends on the global best position [36]. Maintaining this global information, each iteration across nodes incurs significant synchronization. DE’s standard mutation also often uses global best or random pairs, which, similarly, can need coordination. The Spark PSO study highlights this: synchronous PSO had large idle times from global sync, while an asynchronous variant removed this overhead [37]. By contrast, GA can run fully asynchronously; for example, one distributed GA algorithm is reported to effectively use all CPU cores with multilevel parallelism [38]. Moreover, GA’s operators (bit/string crossover and mutation) are simple and local, which suits resource-limited nodes. Taken together, GA’s natural parallelism and lack of global dependencies make it compatible with heterogeneous or bandwidth-limited networks.

Third, GAs are highly modular and extensible: new variation operators or hybridization schemes can be incorporated without restructuring the core framework, allowing seamless integration of domain-specific heuristics (for instance, specialized mutation operators that respect ANN architecture constraints) [39]. While DE and PSO excel at continuous weight tuning in moderate dimensions, they can struggle as dimensionality grows (risking velocity explosion or slow exploration) [40]. GA can simultaneously encode discrete choices (e.g., network layers, connections) and continuous weights, making it well-suited to joint topology-and-weight search [41].

Fourth, extensive empirical evidence supports the energy-efficiency of GA populations when executed in parallel on heterogeneous hardware; because evaluation cost is amortized across many simultaneous fitness computations [42], idle time is minimized and energy usage per solution improves relative to sequential metaheuristics. Finally, the interpretability of GA outcomes—through the analysis of evolving gene pools and Pareto-front approximations—provides actionable insights [43] into the trade-offs between energy consumption, runtime, and model accuracy, thereby enriching the decision-support capabilities of the proposed framework.

3.1. Multicriteria Optimization of ANN Structure

The problem of multicriteria unconditional optimization of the ANN structure can be formally expressed as follows [44,45]:

$$\left\{E(C,W,af)\to \underset{C,W,af}{min}CD(C,af)\to \underset{C,af}{min}\right\}$$

(1)

where $E(·)$ quantifies the overall root-mean-square error during ANN training, while $CD(·)$ measures the network’s computational workload. The variables $C\in {\mathbb{R}}^{N_n\times N_n}$, $W\in {\mathbb{R}}^{N_n\times N_n}$, and $\mathbf{af}\in {\mathbb{R}}^{N_n}$ represent the connectivity matrix, the weight matrix for each connection, and the activation-function vector for all neurons, respectively. The symbol $N_n$ indicates the total count of neurons in the network.

The total RMS error of ANN training is calculated by the following equation:

$$E(C,W,af)={\displaystyle \frac{1}{m}}\left(\sum _{j=1}^m\sqrt{{\displaystyle \frac{1}{n}}\sum _{k=1}^m{\left(OU{T}_j^k(C,W,af)-y_{kj}\right)}^2}\right)$$

(2)

where $OU{T}_j^k(C,W,af)$ is the actual output of the k-th neuron in the ANN with structure $(C,W,af)$ when the j-th image is presented as input, and $y_{kj}$ is the desired output for the k-th neuron. Here, n represents the number of neurons at the output of the network, and m is the size of the training sample.

To estimate the computational complexity of the ANN, the following expression is used:

$$CD(C,af,P)=\sum _{i=1}^{N_{link}\left(C\right)}{T}_{link}(C,P)N_{link}\left(C\right)+\sum _{i=1}^{N_n}{T}_{iact}(af,P)N_n$$

(3)

where $N_{link}(·)$ is the number of ANN links, $T_{link}(·)$ is the processing time for the i-th ANN link, $T_{iact}(·)$ is the time to compute the activation function on the i-th neuron, $N_n$ is the number of neurons in the ANN, and P is the performance of the computing system.

The processing time for a single connection within the ANN is given by the following equation:

$$T_i^{link}(C,P)=T\left(\sum _{j=1}^{n+1}{w}_{ij}\right)-T\left(\sum _{j=1}^n{w}_{ij}\right)$$

(4)

where $T\left(x\right)$ represents the computation time of x, and $w_{ij}$ is the weight of the i-th input of the j-th neuron.

The processing time remains constant across all links and depends on the software and hardware configuration of the ANN, leading to the following reformulation of Equation (3):

$$CD(C,af,P)=N_{link}\left(C\right)·T_{link}\left(P\right)+\sum _{i=1}^{N_n}{T}_{iact}(af,P)$$

(5)

where $T_{link}\left(P\right)$ represents the processing time required for a single ANN link.

The time $T_{link}\left(P\right)$ depends on the computational power of a specific system. To standardize the computational complexity of the ANN, dimensionless values independent of time are used:

$${\displaystyle \frac{CD(C,af,P)}{T_{link}\left(P\right)}}=N_{link}\left(C\right)·T_{link}\left(P\right)+\sum _{i=1}^{N_n}{T}_{iact}(af,P)·T_{link}\left(P\right)$$

(6)

Next, $CD(C,af)={\displaystyle \frac{CD(C,af,P)}{T_{link}\left(P\right)}}$ is defined, and a hardware-independent coefficient for the relative complexity of calculating the activation function on the i-th neuron is introduced as $K_i\left(af\right)={\displaystyle \frac{T_{iact}(af,P)}{T_{link}\left(P\right)}}$. As a result, the criterion for assessing the computational complexity of the ANN model reaches its final form:

$$CD(C,af)=N_{link}\left(C\right)+\sum _{i=1}^{N_n}{K}_i\left(af\right)$$

(7)

With the derived expressions for calculating the tuning error (2) and computational complexity (7), the problem of multicriteria unconditional optimization of the ANN structure can be formulated as follows:

$$\sum _{k=1}^n\sum _{j=1}^m\sqrt{{\left(OU{T}_j^k(C,W,af)-y_{kj}\right)}^2}\to \underset{C,W,af}{min}$$

(8)

$$N_{link}\left(C\right)+\sum _{i=1}^{N_n}{K}_i\left(af\right)\to min$$

(9)

3.2. Parallel GA for ANN Structure Synthesis

The application of adaptive search algorithms for calculations demands significant computational resources, particularly in terms of speed and memory capacity. This requirement can complicate the effective implementation of GAs on traditional von Neumann computers [46]. To address these challenges and accelerate problem-solving, the use of parallel genetic algorithms (PGAs) in conjunction with DCSs is highly recommended [47].

When parallelizing a GA, several factors influence the chosen strategy [48]. These include the method of evaluating suitability and implementing mutation, the choice between using a single population or multiple subpopulations, and the mechanism for exchanging individuals between subpopulations when applicable. Moreover, the choice of selection mechanism—global selection, which ranks and chooses individuals across the entire population, versus local selection, which confines competition to neighborhood or subpopulation subsets—critically affects selection pressure, convergence speed, maintenance of genetic diversity, and synchronization overhead in parallel implementations.

Based on how these factors are implemented, various techniques for the parallelization of GAs emerge. PGAs can be broadly classified into several categories [49]. One category involves PGAs with distributed suitability assessment, also known as the Master–Slave PGA model. Another includes multipopulation PGAs, which incorporate coarse-grained algorithms and the island model. PGAs that leverage massive parallelism, referred to as fine-grained algorithms, form another category, while PGAs with dynamic subpopulations introduce additional flexibility. Moreover, PGAs can be classified as either stationary or unsteady, depending on the stability of subpopulations over time. Hybrid approaches integrate static island topologies with periodic migration of elite individuals and localized fitness evaluations, thereby balancing exploration and exploitation and enhancing resilience to premature convergence. Knysh et al. [50] systematically survey these and other GA parallelization techniques for multi-criteria optimization, detailing models such as master–slave, island, and fine-grained frameworks. Moreover, Kazimipour et al. [51] classify evolutionary algorithms according to their initialization strategies—random seeding, structured composition, and general-purpose heuristics—providing a comprehensive taxonomy for tailoring population startup to problem characteristics.

Based on this analysis, the multi-population GA was selected as the method to address the complex computational problem. Consequently, training an ANN involves a multivariate optimization aimed at identifying the set of ANN weights that best minimize errors. A parallel GA is implemented to tackle this issue. As a result, the optimal ANN structure can be approached as a multi-parameter optimization problem, similar to tuning the parameters of the ANN. Recent years have demonstrated that GAs are one of the most effective methods for solving such problems.

However, the application of PGAs demands considerable computational power. Often, cost-effective hardware solutions for server equipment are employed. In this research, we propose the development of a DSS to facilitate the selection of an efficient CN structure. This DSS would be applicable in enterprises where efficient CN structure selection is needed, with a focus on creating optimal computing infrastructure based on the equipment available at the enterprise. The main goal of implementing this system is to optimize resource utilization and enhance the efficiency of the computing infrastructure. The solution is proposed to be integrated during the design phase of the enterprise, enabling the optimal use of available hardware.

4. Materials and Methods

To establish effective parallel computations for ANN models, it is crucial to decide on the hardware configuration of the computing systems. CNs have become a common and cost-effective solution for realizing distributed computing.

The authors of study [52] investigate approaches to heterogeneous computing at runtime, including techniques at the algorithm, programming, compiler, and application levels. They explore workload partitioning techniques that allow leveraging both central and graphics processors to enhance performance and energy efficiency.

Several software solutions are available for building DCSs, including TensorFlow [53], MXNet [54], GSPMD [55], and others. However, existing software systems often fall short of addressing the specific requirements of the tasks at hand. Furthermore, many distributed systems may encounter issues such as network problems [56] and overloading [57].

The application of computing systems to solve complex scientific and technical problems necessitates the development and implementation of formal optimization modeling methods that reflect the hardware structure of networks aligned with real-time challenges. Therefore, this paper establishes mathematical models to evaluate the stability and performance of DCSs within the context of practical applications involving infrared open competition. The operating DSS for constructing computational systems to tackle challenging problems is informed by the perspective systems derived from these mathematical models and algorithms.

4.1. Performance Model for Heterogeneous Client–Server Networks

Neuromorphic computing encompasses a wide range of information processing approaches and differs from traditional computing systems due to its neurobiological inspiration [58]. Presently, determining the appropriate type and amount of specialization is essential to ensure that a heterogeneous system remains flexible and programmable. For instance, the authors in study [59] established a framework for designing self-optimizing and self-programmable computing systems (SOSPCS), which enhance programmability and flexibility while exploiting the heterogeneity of computing resources.

Empirical studies conducted by the authors in [60] reveal that new relationships can be predicted between elements related to network topology and the properties of the components involved. Predicting these connections will enable the determination of network link formation processes based on currently observed connections. As parallel systems gained popularity, many researchers faced the challenge of optimizing them and reducing computational costs [61]. Figure 1 illustrates the overall architecture of the heterogeneous client–server computing network under consideration.

The computational problem is characterized by four key metrics [62]: $N{O}_{\mathrm{alg}}$, the average number of elementary machine operations performed per iteration of the primary solution algorithm; $N{O}_{\mathrm{ctrl}}$, the average operation count of the control subroutine per iteration; $V_{\mathrm{alg}}$, the mean volume of data (in bits) exchanged between client and server each iteration; and $T_{\lim }$, the maximum allowable end-to-end execution time for the problem.

The operation of the CN can be expressed as a collection of states varying with time. If we describe this process within the framework of mass service theory [63], we can analogize the functioning of the CN to a closed mass service system (MSS) with waiting and random distribution of requests across each server processor.

Assume that there are N types of clients, with an arbitrary number of each type $(m_1,m_2,\cdots ,m_N)$. Each client necessitates servicing at intervals determined randomly. Services are executed via n homogeneous processors.

Client request arrivals for each user class i ($i=1,\cdots ,\overline{N}$) follow a Dien–Wang distribution with rate parameter ${\lambda }_i$. Service times for each request are exponentially distributed with rate ${\mu }_i$ [64]. When one or more servers are idle, an arriving request is assigned uniformly at random to any idle server. If all servers are busy, incoming requests join a finite-capacity queue; once the queue reaches its limit, the system applies a random-replacement policy, selecting an existing queued request with equal probability for service admission or drop.

MSS is described by states of the form

$$a_{j_1,\cdots ,j_N,k,l},$$

where $j_i$ is the number of requests from client type i ($i=1,\cdots ,N$), k denotes the number of busy server processors, and l is the number of queued requests. The following special cases illustrate this notation:

• $a_{0,\cdots ,0,0,0}$: no requests in the system, all n servers idle.
• $a_{1,0,\cdots ,0,1,0}$: one request from type-1 clients is being served on one processor, with no queue.
• $a_{0,\cdots ,0,1,0}$: one request from type-N clients is in service, with no queue.

In the general state $a_{j_1,\cdots ,j_N,k,l}$, the total number of requests in the system is

$$M=\sum _{i=1}^N{j}_i,$$

and the relationship between M, k, and l is given by

$$k=min(M,n),l=M-k,$$

where n is the total number of server processors. Figure 2 shows a fragment of the MSS state-transition graph.

Applying the rules of composing a system of differential equations, it is possible to formulate the following system for the considered MSS [65]:

$$\begin{array}{c}{\displaystyle \frac{d{P}_{0,0,\cdots ,0}\left(t\right)}{dt}}=-\sum _{i=1}^N{m}_i{\lambda }_i{P}_{0,0,\cdots ,0}\left(t\right)+{\mu }_1{P}_{1,0,\cdots ,1,0}\left(t\right)+\hfill \\ \hfill +{\mu }_2{P}_{0,1,\cdots ,1,0}\left(t\right)+\cdots +{\mu }_N{P}_{0,0,\cdots ,1,0}\left(t\right)\\ \hfill \cdots \hfill \end{array}$$

(10)

$$\begin{array}{c}{\displaystyle \frac{d{P}_{j_1,j_2,\cdots ,j_N,k,l}\left(t\right)}{dt}}=-\sum _{i=1}^N\left[(m_i-j_i){\lambda }_i+d_i{\mu }_i\right]P_{j_1,j_2,\cdots ,j_N,k,l}\left(t\right)\hfill \\ +(m_1-j_1+1){\lambda }_1{\displaystyle \frac{n-k+1}{n}}{P}_{j_1-1,j_2,\cdots ,j_N,k-1,l}\left(t\right)\\ +(m_2-j_2+1){\lambda }_2{\displaystyle \frac{n-k+1}{n}}{P}_{j_1,j_2-1,\cdots ,j_N,k-1,l}\left(t\right)+\cdots \\ +(m_N-j_N+1){\lambda }_N{\displaystyle \frac{n-k+1}{n}}{P}_{j_1,j_2,\cdots ,j_N-1,k-1,l}\left(t\right)\\ +(m_1-j_1+1){\lambda }_1{\displaystyle \frac{k}{n}}{P}_{j_1-1,j_2,\cdots ,j_N,k,l-1}\left(t\right)\\ +(m_2-j_2+1){\lambda }_2{\displaystyle \frac{k}{n}}{P}_{j_1,j_2-1,\cdots ,j_N,k,l-1}\left(t\right)+\cdots \\ \hfill +(m_N-j_N+1){\lambda }_N{\displaystyle \frac{k}{n}}{P}_{j_1,j_2,\cdots ,j_N-1,k,l-1}\left(t\right)\\ \hfill \cdots \hfill \end{array}$$

(11)

$$\begin{array}{c}{\displaystyle \frac{d{P}_{m_1,m_2,\cdots ,m_N,h,M-h}\left(t\right)}{dt}}=-\sum _{i=1}^N{\mu }_i{P}_{m_1,m_2,\cdots ,m_N,h,M-h}\left(t\right)\hfill \\ +{\lambda }_1{P}_{m_1-1,m_2,\cdots ,m_N,h,M-h-1}\left(t\right)+\sum _{i=1}^N{\lambda }_2{P}_{m_1,m_2-1,\cdots ,m_N,h,M-h-1}\left(t\right)+\\ \hfill \cdots +{\lambda }_N{P}_{m_1,m_2,\cdots ,m_N-1,h,M-h-1}\left(t\right)\end{array}$$

(12)

where $d_i=\left\{\begin{array}{cc}{j}_i\hfill & \mathrm{if}{j}_i<k\hfill \\ k\hfill & \mathrm{if}{j}_i\ge k\hfill \end{array}\right.$.

Assuming ${\rho }_i={\displaystyle \frac{{\lambda }_i}{{\mu }_i}}\le 1$, the MSS operates in a stationary mode, which is characterized by a system of linear equations.

By focusing our analysis on the stationary mode condition (9), we aim to establish correlations among the parameters of the computing system, specifically, ${\omega }_i$, ${\omega }_{srv}$, and ${\nu }_i$, to satisfy the stationarity requirements of the MSS under investigation.

The flow parameters ${\lambda }_i$ and ${\mu }_i$ depend on the values $T_{0i}$ and ${\tau }_i$, where $T_{0i}$ represents the average time between requests for the i-th type client, indicating the average execution time for one iteration of the algorithm executed on i-th type clients, influenced predominantly by the system architecture and specific task parameter $N{O}_{alg}$ [66]:

$$T_{0i}={\displaystyle \frac{N{O}_{alg}}{{\omega }_i}}$$

where ${\tau }_i$ is the average service time of the client’s request of the i-th type, which depends on the hardware realization of the server, bandwidth of communication channels, as well as on the given parameters of the task $N{O}_{alg}$ and $Valg$:

$${\tau }_i={\tau }_{trn{s}_i}+{\tau }_{srv},{\tau }_{trn{s}_i}=Valg·{\nu }_i,{\tau }_{srv}=N{O}_{ctrl}·{\omega }_{srv},$$

$${\tau }_i=Valg·{\nu }_i+N{O}_{ctrl}·{\omega }_{srv}.$$

Request arrival intensity is defined as the average number of requests arriving per unit of time:

$${\lambda }_i={\displaystyle \frac{I}{I·T_{0i}-I{\tau }_i}}={\displaystyle \frac{1}{T_{0i}-{\tau }_i}},$$

where I is the average number of iterations on one branch of the algorithm for solving the problem.

The service intensity ${\mu }_i$ is characterized by the inverse of the average request servicing time:

$${\mu }_i={\displaystyle \frac{1}{{\tau }_i}}.$$

The system of linear equations describing the functioning of the MSS, taking into account $Q_{k,l}$—the probability that the system being in the state $a_{k,l}$, after servicing one of $k+l$ requests, the total number of requests waiting for service in queues will not change [67]—has the following form:

$$\begin{array}{c}-\sum _{i=1}^N{m}_i{\lambda }_i{P}_{0,0,\cdots ,0,0}+{\mu }_1{P}_{1,0,\cdots ,1,0}+{\mu }_2{P}_{0,1,\cdots ,1,0}+\hfill \\ \hfill \cdots +{\mu }_N{P}_{0,0,\cdots ,1,0}=0\end{array}$$

(13)

$$\begin{array}{c}-\sum _{i=1}^N\left((m_i-j_i){\lambda }_i+d_i{\mu }_i\right)P_{j_1,j_2,\cdots ,j_N,k,l}+\hfill \\ (m_1-j_1+1){\lambda }_1{\displaystyle \frac{n-k+1}{n}}{P}_{j_1-1,j_2,\cdots ,j_N,k-1,l}+\cdots \\ +(m_N-j_N+1){\lambda }_N{\displaystyle \frac{n-k+1}{n}}{P}_{j_1,j_2,\cdots ,j_N-1,k-1,l}+\\ +(m_1-j_1+1){\lambda }_1{\displaystyle \frac{k}{n}}{P}_{j_1-1,j_2,\cdots ,j_N,k,l-1}+\cdots \\ +\left[1-\left({\displaystyle \frac{k-1}{k}}\right)(l+1)\right]d_1{\mu }_1{P}_{j_1+1,j_2,\cdots ,j_N,k,l+1}+\cdots \\ +\left[1-\left({\displaystyle \frac{k-1}{n}}\right)(l+1)\right]d_N{\mu }_N{P}_{j_1,j_2,\cdots ,j_N+1,k,l+1}+\left({\displaystyle \frac{k}{k+1}}\right)l{d}_1{\mu }_1{P}_{j_1+1,j_2,\cdots ,j_N,k+1,l}+\cdots \\ \hfill +\left({\displaystyle \frac{k}{k+1}}\right)l{d}_N{\mu }_N{P}_{j_1,j_2,\cdots ,j_N+1,k+1,l}=0\end{array}$$

(14)

$$\begin{array}{c}-\sum _{i=1}^N{\mu }_i{P}_{m_1,m_2,\cdots ,m_N,h,M-h}+\hfill \\ +{\lambda }_1{P}_{m_1-1,m_2,\cdots ,m_N,h,M-h-1}+{\lambda }_2{P}_{m_1,m_2-1,\cdots ,m_N,h,M-h-1}+\cdots \\ \hfill +{\lambda }_N{P}_{m_1,m_2,\cdots ,m_N-1,h,M-h-1}=0\end{array}$$

(15)

To fulfill stationarity in the system, let us utilize the normalization condition:

$$P_{0,0,\dots ,0}^{0,0}+\sum _{k=1}^h\sum _{\mathcal{\ell }=0}^{M-k}\sum _{j_1=0}^{k+\mathcal{\ell }}\sum _{j_2=0}^{k+\mathcal{\ell }-j_1}···\sum _{j_{N-1}=0}^{k+\mathcal{\ell }-R}{P}_{j_1,j_2,\dots ,j_N-1,\left(k+\mathcal{\ell }-R\right)}^{k,\mathcal{\ell }}=1$$

(16)

where $k+l-j_1{j}_2=0$ and $j_1=0,M-k,l=0,h=1$

The linear equation system (14) is complete and possesses a unique solution because in the equation system (15), ${\sum }_{i=1}^N{j}_i=(k+l)$, which allows the indices k and l in the probability notation to be disregarded.

For the equations in general form as established by the previous reference, we need to employ substitutions to express each equation as one for an MSS composed of multiple request sources (clients) of identical types and serving devices n (i.e., server processors). The resultant equations from this derived system can be sequentially solved. Consequently, utilizing induction method transformations, we derive general forms for subsequent solutions.

Applying the normalization condition leads us to the following:

$$P_{0,0}+\sum _{k=1}^h\sum _{l=0}^{m-k}{P}_{k,l}=1=1$$

(17)

The resultant expression permits determining the probabilities $P_{k,l}$ associated with the MSS states (18):

$$P_{k,l}={\displaystyle \frac{\left(\genfrac{}{}{0pt}{}{k+l-1}{l}\right)\left(\begin{array}{c}n\\ k\end{array}\right)\frac{m!}{(m-k-l)!}{\displaystyle \frac{1}{n^{k+l}}}{\rho }^{\left(k+l\right)}}{{\displaystyle \sum _{i=0}^n}{\displaystyle \sum _{j=0}^{m-i}}\left(\genfrac{}{}{0pt}{}{i+j-1}{j}\right)\left(\begin{array}{c}n\\ j\end{array}\right)\frac{m!}{(m-i-j)!}{\displaystyle \frac{1}{n^{i+j}}}{\rho }^{\left(i+j\right)}}}$$

(18)

where

$$h=\left\{\begin{array}{cc}n\hfill & \mathrm{if}m>n,\hfill \\ m\hfill & \mathrm{if}m\le n.\hfill \end{array}\right.$$

The comprehensive expression for solving the system of equations represented in (14) while considering the constraints stipulated in conditions (16) and (18) is streamlined into (19)

$${\tilde{P}}_{ji}=\left({\displaystyle \genfrac{}{}{0pt}{}{k+l-1}{l}}\right){\displaystyle \frac{\left(n^k\right)m_i!(m_i-j_i)!}{n^{k+l}{\rho }_i^{j_i}}}{\tilde{P}}_{0i}$$

(19)

where

$$\sum {\tilde{P}}_{ji}{m}_i{j}_i=0\ne 1.$$

By implementing inverse substitution of (19) within (14) with the number of paths $\chi (k+l),j_i$ leading to state $a_{j_1,j_2,\cdots ,j_N,k,l}$, which in the general case corresponds to the polynomial coefficient equal to the equation in [47], let us derive the general equation for the system (20):

$$P_{k,\mathcal{\ell }}=\sum _{j_1=0}^{k+\mathcal{\ell }}\sum _{j_2=0}^{k+\mathcal{\ell }-j_1}···\sum _{j_N-1=0}^{k+\mathcal{\ell }-R}{P}_{j_1,j_2,\dots ,j_{N-1},\left(k+\mathcal{\ell }-R\right)}^{k,\mathcal{\ell }}$$

(20)

In evaluating the performance of the client–server network, it is essential to consider the total number of requests in service and those queued, irrespective of client type. The stationary probability distribution $P_{k,l}$ from (21) provides the probability of observing k busy servers and l queued requests. From this, the expected queue length

$$\overline{L}=\sum _{k=0}^n\sum _{l=0}^{L_{max}}l{P}_{k,l}$$

as defined in (22), quantifies the average backlog. Moreover, client-type performance degradation is modeled by coefficients ${\theta }_i$ in (23), which weight each type’s sensitivity to waiting and service delays. Together, these measures enable a comprehensive assessment of both system throughput and user-perceived performance under a steady-state condition:

$$P_{k,\mathcal{\ell }}=\sum _{j_1=0}^{k+\mathcal{\ell }}\sum _{j_2=0}^{k+\mathcal{\ell }-j_1}···\sum _{j_N-1=0}^{k+\mathcal{\ell }-R}{P}_{j_1,j_2,\dots ,j_{N-1},\left(k+\mathcal{\ell }-R\right)}^{k,\mathcal{\ell }}$$

(21)

$${\mathcal{\ell }}_{avg}=\sum _{k=1}^h\sum _{\mathcal{\ell }=1}^{M-k}{P}_{k,\mathcal{\ell }}·\mathcal{\ell }$$

(22)

$${\theta }_i=1+{\displaystyle \frac{{\mathcal{\ell }}_{cp}{\tau }_i}{T_{oi}+{\tau }_i}}$$

(23)

Consequently, the average productivity of the CN can be derived from (24):

$${\Pi }_{avg}==\sum _{i=1}^N{\omega }_i{\displaystyle \frac{m_i}{{\theta }_i}}$$

(24)

In practical computing systems, request flows deviate from Poisson and exponential distributions and are contingent on the architecture of the computing system, as well as the parameters of algorithms being executed. For client–server networks (CSNs) of this type, the intensity parameter for requests ${\lambda }_i$ is computed using Equation (25):

$${\lambda }_i={\displaystyle \frac{1}{\frac{N{O}_{alg}}{{\omega }_i\frac{V_{alg}}{{\nu }_i\frac{N{O}_{ctrl}}{{\omega }_{srv}}}}}}$$

(25)

where $i=1,2,3,\cdots ,N$.

The service intensity parameter ${\mu }_i$ is determined by expression (26):

$${\mu }_i={\displaystyle \frac{1}{\frac{V_{alg}}{{\nu }_i\frac{N{O}_{ctrl}}{{\omega }_{srv}}}}}$$

(26)

The computed average performance $P_{avg}$ allows for estimation of the average time for solving the problem $T_{avg}$ within the designed heterogeneous CN (27):

$$T_{avg}={\displaystyle \frac{{\mathrm{NO}}_{alg}}{{\Pi }_{avg}}}$$

(27)

This proposed approach facilitates the selection of a performance-efficient heterogeneous client–server network structured to address complex problems of a certain class while adhering to specified constraints on allowable solution time $T_{lim}$.

4.2. Reliability Assessment Model for Radial Type Distributed Client–Server Architecture CN

Analytical modeling is an effective tool for studying various characteristics of designed computing systems, enabling the identification of their most efficient structures. However, current analytical methods for assessing the reliability of DCSs, in relation to their configurations, remain underdeveloped [68].

CN with a radial architecture consists of an arbitrary number of client types, with each type containing an arbitrary number of clients, all interacting with a multiprocessor server. Let the network consist of N client types, each containing $m_i$ clients ($i=1,\dots ,N$). Each client connects to the hub via communication channels, while the server is made up of n homogeneous processors. Radial computing theory posits that individual components of a computing system are prone to failure and will eventually require repairs.

The failure rates of the various components in the system are clearly defined as follows:

• ${\lambda }_{\mathrm{icl}}$—failure rate of client nodes of type i, $i=1,\cdots ,N$;
• ${\lambda }_{\mathrm{srv}}$—the failure rate of the server processors;
• ${\lambda }_{\mathrm{hub}}$—the failure rate of the concentrator.

Recovery times for all elements exiting the operational state follow an exponential distribution with the following parameters:

• ${\mu }_{\mathrm{icl}}$— recovery intensity of client nodes of the i-th type ($i=1,N$);
• ${\mu }_{\mathrm{srv}}$— the intensity of server processor recovery;
• ${\mu }_{\mathrm{hub}}$—the recovery intensity of the hub.

When a new recovery request finds the service attendant free, it is accepted. Conversely, if the servicing device is busy, the incoming request must queue until it is serviced, following a random equal probability selection from the queue.

The closed MSS can occupy a range of states depending on which components are operational or under repair. Key configurations include the following:

• $a_{0,\cdots ,0}$: No servers, no hub, and no clients are available; all elements are down and undergoing restoration, halting computation.
• $a_{1,0,\cdots ,0}$: Exactly one server processor is functional while the remaining $n-1$ processors, the hub, and all clients are failed and being repaired; computation is suspended.
• $a_{0,1,0,\cdots ,0}$: Only the central hub is up, with every server processor and client node failed and in repair; no processing occurs.
• $a_{0,0,1,0,\cdots ,0}$ or $a_{0,0,0,\cdots ,1}$: A single client of type 1 (or type N) is operational, while its $m_i-1$ peers, the hub, and all servers are down and repairing; computational tasks remain paused.
• $a_{j_{\mathrm{srv}},0,j_{1,\mathrm{cl}},\cdots ,j_{N,\mathrm{cl}}}$: A subset of $j_{\mathrm{srv}}$ server processors and $j_{i,\mathrm{cl}}$ clients of each type i are working, with the hub and the other components in failed-repair mode; processing is not active.
• $a_{j_{\mathrm{srv}},1,j_{1,\mathrm{cl}},\cdots ,j_{N,\mathrm{cl}}}$: The active group includes $j_{\mathrm{srv}}$ servers, the hub, and $j_{i,\mathrm{cl}}$ clients of each class, while the rest are under repair; computation proceeds.
• $a_{n,1,m_1,\cdots ,m_N}$: The entire network—n server processors, the hub, and all $m_i$ clients of each type—is up and executing tasks.

Figure 2 depicts a fragment of this state-transition graph.

A fragment of the state graph is shown in Figure 3.

For the stationary mode, a system of linear Equations (28)–(30) is obtained:

$$\begin{array}{c}-\left(n{\mu }^{srv}+{\mu }^{hub}+\sum _{i=1}^N{m}_i{\mu }_i^{cl}\right)P_{0,0,0,\dots ,0}+{\lambda }^{srv}{P}_{1,0,0,\dots ,0}+\hfill \\ +{\lambda }^{hub}{P}_{0,1,0,\dots ,0}+{\lambda }_1^{cl}{P}_{0,0,1,\dots ,0}+\dots +{\lambda }_N^{cl}{P}_{0,0,0,\dots ,1}=0\\ \hfill \cdots \end{array}$$

(28)

$$\begin{array}{c}-[\left(n-j^{srv}\right){\mu }^{srv}+j^{srv}{\lambda }^{srv}+\left(1-j^{hub}\right){\mu }^{hub}+\hfill \\ +j^{hub}{\lambda }^{hub}+\sum _{i=1}^N\left((m_i-j_i^{cl}){\mu }_i^{cl}+j_i^{cl}{\lambda }_i^{cl}\right)]\times \\ P_{j^{srv},j^{hub},j_1^{cl},\dots ,j_N^{cl}}+(n-j^{srv}+1){\mu }^{srv}{P}_{j^{srv}-1,j^{hub},j_1^{cl},\dots ,j_N^{cl}}\\ +j^{hub}{\mu }^{hub}{P}_{j^{srv},0,j_1^{cl},\dots ,j_N^{cl}}+\\ +(m_1-j_1^{cl}+1){\mu }_1^{cl}{P}_{j^{srv},j^{hub},j_1^{cl}-1,\dots ,j_N^{cl}}+\\ \dots +(m_N-j_N^{cl}+1){\mu }_N^{cl}{P}_{j^{srv},j^{hub},j_1^{cl},\dots ,j_N^{cl}-1}+\\ +(j^{srv}+1){\lambda }^{srv}{P}_{j^{srv}+1,j^{hub},j_1^{cl},\dots ,j_N^{cl}}+\\ +(1-j^{hub}){\lambda }^{hub}{P}_{j^{srv},1,j_1^{cl},\dots ,j_N^{cl}}+\\ +(j_1^{cl}+1){\lambda }_1^{cl}{P}_{j^{srv},j^{hub},j_1^{cl}+1,\dots ,j_N^{cl}}+\\ \hfill \dots +(j_N^{cl}+1){\lambda }_N^{cl}{P}_{j^{srv},j^{hub},j_1^{cl},\dots ,j_N^{cl}+1}=0\cdots \end{array}$$

(29)

$$\begin{array}{c}-\left(n{\lambda }^{srv}+{\lambda }^{hub}+\sum _{i=1}^N{m}_i{\lambda }_i^{cl}\right)P_{n,1,m_1,\dots ,m_N}+\hfill \\ +{\mu }^{srv}{P}_{n-1,1,m_1,\dots ,m_N}+{\mu }^{hub}{P}_{n,0,m_1,\dots ,m_N}+\\ \hfill +{\mu }_1^{cl}{P}_{n,1,m_1-1,\dots ,m_N}+\dots +{\mu }_N^{cl}{P}_{n,1,m_1,\dots ,m_N-1}=0\end{array}$$

(30)

The system of Equations (28)–(30) has a single solution subject to the normalization condition (31):

$$\sum _{j^{srv}=0}^n\sum _{j^{hub}=0}^1\sum _{j_1^{cl}=0}^{m_1}\dots \sum _{j_N^{cl}=0}^{m_N}{P}_{j^{srv},j^{hub},j_1^{cl},\dots ,j_N^{cl}}=1$$

(31)

To solve the system of Equations (28)–(30) in general form, the following substitution (32) is made:

$$P_{j^{srv},j^{hub},j_1^{cl},\dots ,j_N^{cl}}=P_{j^{srv}}^{srv}·P_{j^{hub}}^{hub}·\prod _{i=1}^N{P}_{j_i^{cl}}^{c{l}_i}$$

(32)

The system of equations for the i-th type of CN elements is written, introducing the following notations:

$$\mathrm{When}k\ge 1,z_k=(n_i-k){\mu }_i{P}_{k-1}^i+{\lambda }_i{P}_{k+1}^i$$

The equation in the general case (31) is given by the following:

$$P_{j_i}^i={\displaystyle \frac{n_i!}{(n_i-j_i)!}}{\rho }_i^{j_i}{P}_0^i$$

(33)

Backward substitution of (32) into (31) is performed, taking into account the binomial and polynomial coefficients. Additionally, the general designations of the characteristics of the CN elements are substituted. The solution of the system of Equation (28) is then obtained in general form (32):

$$\begin{array}{c}\hfill P_{j^{srv},j^{hub},j_1^{cl},\dots ,j_N^{cl}}={\displaystyle \frac{\frac{n!}{(n-j^{srv})!}{\rho }_{srv}^{j^{srv}}·{\rho }_{hub}^{j^{hub}}·{\prod }_{i=1}^N\frac{m_i!}{(m_i-j_i^{cl})!}{\rho }_{c{l}_i}^{j_i^{cl}}}{{\sum }_{j^{srv}=0}^n{\sum }_{j^{hub}=0}^1{\sum }_{j_1^{cl}=0}^{m_1}\dots {\sum }_{j_N^{cl}=0}^{m_N}\frac{n!}{(n-j^{srv})!}{\rho }_{srv}^{j^{srv}}·{\rho }_{hub}^{j^{hub}}·{\prod }_{i=1}^N\frac{m_i!}{(m_i-j_i^{cl})!}{\rho }_{c{l}_i}^{j_i^{cl}}}}\end{array}$$

The main indicator of aircraft operation reliability in steady-state mode is the availability factor $K_r$. The availability factor is defined as the probability that the system will be in an operable state at any point in time, except for planned periods when the system is not intended to be used [69].

Taking into account the definitions of probability $P_{j_{\mathrm{srv}},j_{\mathrm{hub}},j_{1_{\mathrm{cl}}},\cdots ,j_{N_{\mathrm{cl}}}}$ of the system being in the state $a_{j_{\mathrm{srv}},j_{\mathrm{hub}},j_{1_{\mathrm{cl}}},\cdots ,j_{N_{\mathrm{cl}}}}$, the availability factor of the considered distributed client–server CN of radial type is defined as follows (33):

$$\begin{array}{c}{K}_r=\sum _{j^{srv}=1}^n\sum _{j_1^{cl}=1}^{m_1}\sum _{j_2^{cl}=0}^{m_2}\dots \sum _{j_N^{cl}=0}^{m_N}{P}_{j^{srv},1,j_1^{cl},\dots ,j_N^{cl}}+\hfill \\ \hfill \dots +\sum _{j^{srv}=1}^n\sum _{j_1^{cl}=0}^{m_1}\dots \sum _{j_{N-1}^{cl}=0}^{m_{N-1}}\sum _{j_N^{cl}=1}^{m_N}{P}_{j^{srv},1,j_1^{cl},\dots ,j_N^{cl}}\end{array}$$

(34)

4.3. Setting the Problem of Selecting an Effective Configuration of a Computer Network

Identifying the optimal configuration of a computer network is crucial for system design. The performance of the network is directly influenced by its configuration parameters. Therefore, determining the optimal configuration is essential to achieving optimal system performance, characterized by minimal response times and high throughput [70].

The process of optimization involves three criteria: maximizing performance, maximizing reliability, and minimizing costs. Task variables, such as power consumption and speed, may be subject to specific constraints.

In this formulation of the optimization problem, one primary criterion is selected from each of the three categories, creating a framework that focuses on a single target criterion while the remaining two act as constraints. The selection of the target criterion is dependent, as altering the remaining criteria does not contribute to the relevance of the optimization framework. The degree of inconsistency between criteria is subsequently analyzed.

First, consistency within each triplet of criteria is considered. For example, in solving a complex problem within the computer network, total operations per second initially increase as the number of idle computing nodes grows. However, this decreases as more nodes are introduced, leading to longer system uptime due to lower failure probabilities. Costs generally increase with system complexity. Consequently, a single dominant criterion is identified in each group, while the remaining criteria are converted into constraints. It is important to note that these criteria cannot be entirely disregarded. Experience shows that improving one dominant criterion may occasionally result in unacceptable values for the other criteria.

Second, it is noted that cost-related criteria do not directly reflect performance and reliability criteria. As the number of computational nodes in the network increases, both performance and reliability metrics improve. Thus, the reliability criterion can be transformed into constraints by combining it with performance and cost criteria, while adhering to specific reliability and performance limitations.

In summary, the problem of selecting an efficient configuration for a heterogeneous client–server network is formalized as a multi-criteria optimization problem. The problem centers on two primary criteria—performance and cost—while incorporating essential constraints to address additional requirements.

The optimization model can be formally expressed as follows:

$$\begin{array}{c}P(n,{\omega }_{\mathrm{srv}},m_1,{\omega }_1,v_1,\cdots ,m_N,{\omega }_N,v_N,N{O}_{\mathrm{alg}},N{O}_{\mathrm{ctrl}},V_{\mathrm{alg}})\to max,\hfill \\ \hfill C(n,{\omega }_{\mathrm{srv}},m_1,{\omega }_1,v_1,\cdots ,m_N,{\omega }_N,v_N)\to min,\end{array}$$

(35)

subject to the following conditions:

$$\begin{array}{c}{K}_r(n,{\lambda }_{\mathrm{srv}},{\mu }_{\mathrm{srv}},{\lambda }_{\mathrm{hub}},{\mu }_{\mathrm{hub}},m_1,{\lambda }_{1\mathrm{cl}},{\mu }_{1\mathrm{cl}},\cdots ,m_N,{\lambda }_{N\mathrm{cl}},{\mu }_{N\mathrm{cl}})\ge K_r^0,\hfill \\ n^{-}\le n\le n^{+},\\ \hfill m_i^{-}\le m_i\le m_i^{+},i=1,\cdots ,N\end{array}$$

(36)

where

• N: Total distinct client categories;
• $m_i$: Quantity of clients in category i ($i=1,\cdots ,N$);
• n: Number of identical server CPUs;
• ${\omega }_i$: Computational throughput of a type-i client (in FLOPS);
• ${\omega }_{\mathrm{srv}}$: Processing capability of each server CPU (in FLOPS);
• ${\nu }_i$: Data transfer rate between type-i clients and the server (bits/s);
• ${\lambda }_{i,\mathrm{cl}}$: Failure rate for client nodes of type i ($i=1,\cdots ,N$);
• ${\lambda }_{\mathrm{srv}}$: Failure rate of a server CPU;
• ${\lambda }_{\mathrm{hub}}$: Failure rate of the network hub;
• ${\mu }_{i,\mathrm{cl}}$: Repair rate of type-i client nodes ($i=1,\cdots ,N$);
• ${\mu }_{\mathrm{srv}}$: Repair rate of server CPUs;
• ${\mu }_{\mathrm{hub}}$: Repair rate of the network hub;
• P: Chosen performance metric;
• C: Cost metric, context-dependent;
• $K_r$: Achieved availability of the heterogeneous CN;
• $K_r^0$: Target (maximum allowable) availability level;
• $n^{+}$, $n^{-}$: Upper and lower bounds on the number of server CPUs;
• $m_i^{+}$, $m_i^{-}$: Upper and lower bounds on count of client nodes in category i ($i=1,\cdots ,N$).

Existing methods [71] face challenges in distinguishing circumstances where the optimal solution is feasible from those where it is not—a critical aspect for direct search algorithms. To address this limitation, the adaptive penalty method, an enhancement of the dynamic penalty method, has been proposed. Unlike its predecessor, the penalties in this approach depend not only on the iteration count but also on the frequency with which the best representative of the population resides in acceptable versus unacceptable regions.

This dependency is expressed as the following system of equations:

$$\lambda (t+1;\mathit{\beta })=\left\{\begin{array}{cc}{\beta }_1\lambda \left(t\right),\hfill & \mathrm{if}{b}_i\in F\mathrm{for}\mathrm{all}i\in \{t-k+1,\cdots ,t\},\hfill \\ {\beta }_2\lambda \left(t\right),\hfill & \mathrm{if}{b}_i\in S\setminus F\mathrm{for}\mathrm{all}i\in \{t-k+1,\cdots ,t\},\hfill \\ \lambda \left(t\right),\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(37)

where $b_i$ is the best individual of the i-th population, and ${\beta }_1,{\beta }_2>1$ with ${\beta }_1\ne {\beta }_2$ to avoid looping.

4.4. Methods for Solving Multi-Criteria Optimization Problems

A common strategy for addressing multi-criteria optimization problems involves generating a set of Pareto-optimal solutions and selecting the most attractive option from this set. The success of this approach critically depends on the designer’s ability to manage and interpret the Pareto set, particularly regarding its size and distribution [72]. Achieving Pareto-optimal solutions typically requires tuning various parameters. Classical methods, such as normal constraint, trade-off programming, and physical programming, have been developed to achieve this goal [73].

However, traditional techniques often exhibit inefficiencies when tackling complex problems. They may fail to ensure Pareto optimality or cover the entire Pareto set. To overcome these challenges, evolutionary approaches are employed, which navigate the complexities of multi-criteria optimization by providing a global perspective of the solution space. These methods avoid local minima and offer multiple alternative solutions in a single optimization cycle.

Despite these advantages, identifying satisfactory solutions that encompass the entire Pareto-optimal set is often impractical. The primary objectives of multi-criteria optimization are as follows:

• Reduce the gap between the discovered non-dominated front and the true Pareto boundary.
• Provide a diverse set of solutions, ensuring a wide range of options.
• Maximize the positive effects derived from the non-dominated front, highlighting unattainable values for each criterion within the results.

When employing evolutionary multi-criteria optimization algorithms, two significant challenges arise:

• Assigning fitness and selecting candidates to achieve a Pareto-optimal set.
• Population dispersion by introducing diversity-preserving operations to avoid early convergence and ensure an evenly spread non-dominated set.

Unlike single-criteria optimization, where the objective and fitness functions align, multi-criteria optimization requires consideration of both dimensions. Evolutionary strategies addressing criteria independently, methods based on classic convolution, and Pareto dominance principles have emerged as distinct approaches.

Instead of merging criteria into a single convolution function, some evolutionary algorithms alternate between criteria during the selection phase. For instance, Ref. [31] suggested dividing the interim population into equal segments based on various criteria. Similarly, Ref. [74] introduced a selection mechanism where individuals are compared using a specific or random criterion order. Ref. [75] proposed a new program to enhance Pareto optimality by targeting optimal reduction.

The concept of evaluating fitness based on Pareto dominance was pioneered by [76], who expanded this principle into new fitness assignment schemes rooted in Pareto dominance [77].

Several additional methodologies address multi-criteria optimization using evolutionary techniques. These include parameter variation, pre-determination, population reinitialization, convolution, elitism, and diversity-enhancing “niching” methods. Examples of such approaches include fitness equalization, limited crossover, and isolation.

In this study, one of the most renowned multi-criteria evolutionary algorithms, Fonseca and Fleming’s Multiobjective Genetic Algorithm (FFGA), is employed to address the multi-criteria optimization challenge of selecting an efficient ANN structure.

4.5. FFGA Method

The FFGA method [78] is a Pareto dominance-based procedure for ranking individuals, where the rank of each individual is determined by the number of individuals that dominate it.

Algorithm for assigning fitness in the FFGA method:

• Input: $P_t$ (population).
• Output: F (suitability values).

• For each $i\in P_t$, calculate its rank:

  $$r\left(i\right)=1+\left|\{j\mid j\in P_t\wedge j\succ i\}\right|,$$

  (38)

  where $|·|$ denotes the cardinality of the set.
• Sort the population according to rank $r\left(i\right)$. Each $i\in P_t$ is assigned a raw fitness $F^{\prime }\left(i\right)$ by interpolating from the best ($r\left(i\right)=1$) to the worst individual ($r\left(i\right)\le N$) using linear ranking [79].
• Calculate suitability values $F\left(i\right)$ by averaging the raw suitability values $F^{\prime }\left(i\right)$ among individuals $i\in P_t$ with identical rank $r\left(i\right)$ (suitability equalization in the target space).

An individual whose decision vector is non-dominated with respect to $m\left(P\right)$ has rank 1. The temporal population is stochastically filled based on the rank value. This basic concept has evolved further, incorporating adaptive fitness equalization and continuous introduction of random migrants.

5. Results

5.1. Automated DSS for Efficient Distributed CN Selection

The object-oriented approach was chosen for designing and developing software tools, enabling the creation of highly reliable systems that meet modern quality and user interface requirements [80]. The core of the program system was implemented in the C programming language [81], while the user interface was developed using C++ [82]. This approach enhances program portability across operating environments. Furthermore, numerous libraries of mathematical subroutines written in C significantly improve development speed and program reliability.

Embarcadero RAD Studio, a rapid application development (RAD) environment, was selected as the toolkit for software development. This integrated development environment (IDE) accelerates the process of visual design [83]. Software products were tested during and after development to ensure quality and reliability [84]. The DSS enhances the management of complex systems. One of their primary functions is generating possible decision options acceptable to the manager [85].

An automated DSS was developed to select an efficient structure for distributed computing CSNs considering performance, reliability, and cost. This system employs mathematical models to evaluate network performance and reliability, making it applicable to the design or modification of CN architectures for specific complex problems. The software is a Windows application, developed in C++ using the Embarcadero RAD Studio environment.

Figure 4 illustrates the generalized structural scheme of the DSS for selecting an effective configuration of a CN.

The “DCS Settings” window is shown in Figure 5a, where users specify both the hardware profile of the client–server network and the algorithmic parameters governing the problem to be addressed. For the server tier, one defines the permissible range of processor counts and the corresponding per-core performance limits (in FLOPS), as well as the failure and recovery intensities for both CPU units and the central hub. On the client side, users indicate the distinct node classes, the quantity of nodes in each class, their minimum and maximum processing capacities, the data-link bandwidths, and the associated failure and repair rates. Algorithmically, the user must provide the average computational workload per iteration—both for the primary solution routine and its control subroutine—and the mean volume of data exchanged in each client–server interaction. Finally, a global availability constraint may be imposed by specifying the minimum acceptable uptime for the entire distributed system.

The “Settings and Launch of the GA” window, as shown in Figure 5b, is where the Genetic Algorithm (GA) parameters are configured and the computational process is initiated. In this window, the optimization problem-solving process is visualized graphically, with the current non-dominated set of solutions displayed. Each point on the graph represents a particular computer network configuration that meets the availability constraint, showcasing specific performance and cost values. During program execution, users can press the “Pause” button to temporarily halt the process, enabling them to adjust all GA parameters except for the sampling step and the number of individuals in the population. In this window, users can configure several GA parameters, including the number of individuals in the population, the number of generations, the species selection method, the discretization step for all DCS elements, the types of crossover, and the mutation type.

Upon selecting the “Suitability Graphs” button, the window shown in Figure 6 appears. This window displays graphs of the fitness of the best and worst individuals, as well as the average fitness of the population.

5.2. Efficient Configuration Selection for Heterogeneous CNs

This research addressed the challenge of selecting an effective configuration for heterogeneous CNs designed to tackle complex scientific and technical problems. A comprehensive set of analytical models was developed using mass service theory to evaluate the performance of heterogeneous client–server CNs in solving particular classes of complex problems.

The mathematical performance evaluation model enables the selection of a performance-efficient heterogeneous client–server CN. This configuration can address complex problems under specific constraints regarding permissible solution times. Additionally, the reliability indices estimation model for radial-type heterogeneous client–server CNs helps identify a CN structure that meets essential reliability requirements of the designed system. These developed models serve as a mathematical framework for formulating and solving optimization problems, focusing on choosing an effective heterogeneous client–server CN configuration tailored to specific problem classes.

The program, developed based on the proposed DCS functioning model to evaluate its performance and utilizing the FFGA algorithm, underwent thorough testing. It was applied in practice to solve the problem of selecting an efficient DCS structure for the company LLC “RITS” in Krasnoyarsk.

Based on the cost and performance metrics of the obtained configurations for the distributed computing network (CN), the enterprise selected a configuration consisting of 15 client nodes with a performance of 3 TFLOPS and a data link with an average speed of 8 Gbit/s, 38 client nodes with a performance of 7.2 TFLOPS and an average data link speed of 8 Gbit/s, 10 client nodes with 9.2 TFLOPS performance and a data link with an average speed of 8 Gbit/s, 11 client nodes with a performance of 12 TFLOPS and a data link with an average speed of 9 Gbit/s, and a four-processor server node with a performance of 461 TFLOPS per processor.

Following a detailed analysis of the available computer market products, the CN’s hardware structure was finalized, incorporating 15 client nodes equipped with Intel Celeron G5905 processors (Intel Corporation, Santa Clara, CA, USA), 38 client nodes with Intel Pentium Dual Core G4400 processors (Intel Corporation, Santa Clara, CA, USA), 10 client nodes with Intel Core i3-9100F processors (Intel Corporation, Santa Clara, CA, USA), 11 client nodes with Intel Core i3-12100F processors (Intel Corporation, Santa Clara, CA, USA), and a server node based on four Intel Xeon E-2176M processors (Intel Corporation, Santa Clara, CA, USA). The problem-solving effort resulted in an approximation of the Pareto-optimal set comprising 22 points, along with its front. Figure 7 illustrates the approximation of the Pareto front.

Approbation of this software tool demonstrated that the developed program system effectively solves the tasks associated with designing and modifying the architecture of computer networks intended for addressing complex scientific and technical issues.

Table 2. Configuration of DCSs for the enterprise.

Node Type	Number of Nodes	Performance (TFLOPS)	Data Link Speed (Gbit/s)
Client Node (Celeron G5905 )	15	3	8
Client Node (Pentium Dual Core G4400)	38	7.2	8
Client Node (Intel Core i3-9100F)	10	9.2	8
Client Node (Intel Core i3-12100F)	11	12	9
Server Node (Xeon E-2176M)	1	461 (per processor)	N/A

summarizes the key performance metrics (TFLOPS), availability percentages, costs, and corresponding hardware configurations for the selected nodes.

The quantitative evaluation of the three configurations is presented in

Table 3. Numerical comparison of solution time ${\overline{T}}_{\mathrm{solve}}$, system availability $A_{\mathrm{sys}}$, and total hardware cost $C_{\mathrm{total}}$ across three CN architectures under a ${10}^5$-task workload.

Architecture	${\displaystyle {\overline{\mathit{T}}}_{\mathbf{solve}}}$ (s)	${\mathbf{A}}_{\mathbf{sys}}$ (%)	${\mathbf{C}}_{\mathbf{total}}$ (kUSD)
Homogeneous cluster	3.6	98.5	270
Heuristic heterogeneous configuration	2.9	98.9	255
FFGA-optimized configuration (this work)	$\mathbf{2.2}$	$\mathbf{99.2}$	$\mathbf{250}$

. Here, the mean solution time ${\overline{T}}_{\mathrm{solve}}$ reflects the average duration required to complete each task in a workload of ${10}^5$ jobs, measured in seconds. System availability $A_{\mathrm{sys}}$ denotes the percentage of uptime achieved under realistic failure and repair rates, and total hardware cost $C_{\mathrm{total}}$ represents the procurement expense in thousands of U.S. dollars (kUSD). This comparison enables a clear assessment of trade-offs between computational performance, reliability, and investment requirements.

6. Discussion

This research focused on addressing the challenge of selecting an effective configuration for heterogeneous CNs designed to solve complex scientific and technical problems. A comprehensive set of analytical models based on mass service theory was developed to evaluate the performance of heterogeneous CSNs. These models serve as a mathematical framework for formulating and solving optimization problems aimed at choosing an effective HCN configuration tailored to specific problem classes. One of the key findings is that the proposed models and algorithms, including the FFGA method for optimization, successfully address the task of selecting the appropriate structure for HCNs. This is validated through testing conducted with LLC “RITS” in Krasnoyarsk. As a result of choosing the optimal network configuration, significant improvements were made both in terms of performance and the cost associated with building computational capabilities.

6.1. Limitations

However, several limitations were identified during this research that must be addressed in future work. First, the current mass-service (M/M/n) model—while tractable for analysis and optimization—fails to capture complex client–server dependencies and dynamic interactions observed in real-world distributed computing environments [86]. Extending the framework to a G/G/n setting would account for general arrival and service distributions, but this generalization renders closed-form analysis more challenging. To maintain rigorous performance guarantees under both M/M/n and G/G/n assumptions, exponential tail bounds on queue lengths and waiting times can be derived via Chernoff inequalities [87,88]. Specifically, if $Q\left(t\right)$ denotes the queue length at time t, then for any $\theta >0$,

$$Pr\left(Q\left(t\right)>x\right)\le exp\left(-\theta x+\Lambda \left(\theta \right)t\right),$$

where $\Lambda \left(\theta \right)$ is the cumulant-generating function of the net arrival process; optimizing $\theta$ yields a tight upper bound. In the Halfin–Whitt (QED) regime [89], diffusion-limit approximations further provide asymptotic lower bounds such as $\mathbb{E}\left[W\right]\gtrsim {(1-\rho )}^{-1}/\sqrt{n}$, characterizing performance under heavy traffic.

For availability analysis, minimal cut-set enumeration in a Continuous-Time Markov Chain model yields complementary upper and lower bounds. If ${\mathcal{C}}_1,\cdots ,{\mathcal{C}}_m$ are minimal cut-sets with component availabilities $A_i$, then

$$1-A\le \sum _{k=1}^m\prod _{i\in {\mathcal{C}}_k}(1-A_i),$$

while inclusion–exclusion provides corresponding lower bounds. For non-exponential or multi-state components, matrix-analytic methods in the Laplace domain invert to explicit time-dependent availability bounds.

Stability and convergence under extreme loads or variability can be established using Foster–Lyapunov criteria [90] and fluid-limit arguments. By constructing a Lyapunov function $V\left(x\right)$, for example $V\left(x\right)=x^2$ where x is the queue length, and demonstrating a negative drift

$$\mathbb{E}\left[V\right(Q(t+1))-V(Q\left(t\right))\mid Q(t)=x]\le -ϵ\mathrm{for}x>K,$$

one ensures positive Harris recurrence and geometric ergodicity. Fluid-limit techniques then show that suitably scaled processes converge to deterministic trajectories, guaranteeing the existence and uniqueness of long-run performance and availability averages.

Sensitivity analysis of these bounds with respect to load $\rho$, squared coefficients of variation $c_a^2,c_s^2$, and failure or repair rates allows identification of critical thresholds beyond which performance degrades sharply or availability targets cannot be met. These closed-form bounds and convergence proofs will both validate empirical and simulation results and provide practitioners with explicit guarantees for system behavior across a wide spectrum of operational conditions.

To empirically validate the proposed analytical models of performance and energy consumption, a dedicated verification section will be introduced using a controlled hardware testbed. This testbed will consist of a small cluster of heterogeneous servers—each equipped with Intel CPUs supporting RAPL power-measurement interfaces and NVIDIA GPUs exposing power metrics via NVML—so that both peak and real-time power draw can be recorded with millisecond precision. Prior to workload execution, each node’s peak FLOPS capacity and base power consumption will be profiled through microbenchmarks, and network latency and bandwidth will be calibrated using tools such as iPerf3.

Representative deep-learning workloads—such as training a ResNet-50 on ImageNet and a convolutional autoencoder on CIFAR-10—will be deployed across the cluster. During each run, CPU/GPU utilization, core clock frequencies, and instantaneous power consumption will be logged alongside throughput (GFLOPS) and end-to-end training time. From these measurements, the per-component energy-per-FLOP coefficients (${\alpha }_{\mathrm{CPU}},{\alpha }_{\mathrm{GPU}}$) will be estimated via linear regression of power versus achieved FLOPS under varying utilization levels. Network service-time distributions will be characterized by fitting empirical inter-arrival and service intervals to known distributions (e.g., Weibull or log-normal) based on timestamped packet-exchange logs.

Model parameters—service rates ${\mu }_i$, energy coefficients, and network delay statistics—will then be calibrated by minimizing the mean absolute percentage error (MAPE) between predicted and observed metrics:

$$\mathrm{MAPE}={\displaystyle \frac{100\%}{N}}\sum _{k=1}^N\left|\frac{y_k^{\mathrm{pred}}-y_k^{\mathrm{obs}}}{y_k^{\mathrm{obs}}}\right|.$$

Finally, Kubernetes will be used to orchestrate containerized replicas of each node type, effectively scaling the testbed to emulate larger network sizes. By comparing model forecasts against these scaled experiments, we will identify and quantify systematic deviations—such as thermal throttling effects or network-fan-in bottlenecks—and incorporate correction factors into the decision-support system. This rigorous calibration and validation procedure will anchor the theoretical framework in real-world observations, ensuring that the system’s recommendations remain accurate and robust across both laboratory and production scales.

Another limitation is the need for additional factors. The current model does not consider factors such as external instability, changing user needs, or technological advancements. These factors can influence the long-term efficiency and sustainability of the chosen configuration, making it important for future models to incorporate such variables for a more comprehensive analysis [91].

Additionally, while the FFGA Algorithm demonstrated good performance in solving optimization tasks, it has its own limitations. Specifically, it does not always guarantee the identification of a global optimum [92], especially given the complexity involved in searching through multi-criteria solution spaces. To address this, future work may need to explore more advanced optimization approaches, such as multi-grid algorithms or deep learning techniques, which could enhance prediction accuracy for optimal configurations.

Finally, there are scalability challenges that need attention. The tests conducted for LLC “RITS” used a relatively limited network model, and in order to scale the system for larger networks or more complex problem sets [93], more advanced infrastructure will be required. Furthermore, the algorithms must be improved to handle big data and multitasking efficiently, ensuring that the system can scale effectively to meet the needs of larger, more dynamic environments.

6.2. Decision Criteria for Applying the Proposed Approach

The decision to apply the proposed approach to large, heterogeneous datasets must be based on an assessment of several interrelated factors. First, the structure and composition of the data—including tabular records, semi-structured formats (JSON, XML), and unstructured content (text, images, time series)—directly influences preprocessing requirements, memory allocation, and network bandwidth utilization [94]. Second, the characteristics of available computational and networking resources—namely, the heterogeneity of compute nodes [95] (CPU versus GPU capabilities, clock speeds, and memory capacities) and the throughput and latency of inter-node communication—determine the feasibility of workload balancing and the frequency of gradient synchronizations without incurring excessive delays [96]. Third, performance and reliability requirements—such as stringent training-time constraints in real-time monitoring scenarios and the necessity for fault tolerance in case of node or network failures—define the trade-off between latency minimization and system resilience [97]. Fourth, economic and organizational constraints—including budgetary limits on resource provisioning and energy consumption, as well as project timelines and future scalability needs—must be weighed against anticipated improvements in training efficiency and model accuracy [98]. Finally, a multi-objective optimization framework employing Pareto-based analysis and user-defined weightings for cost, performance, and availability provides the methodological basis for navigating these trade-offs. Integration of these considerations enables identification of an optimal distributed computing configuration for artificial neural network modeling on any given large, heterogeneous dataset.

The relationship between cost, performance, and availability in the selection of a DCS can be characterized as a tripartite trade-off, in which improvement along one dimension frequently entails compromise in one or both of the others. Cost is typically quantified in terms of capital and operational expenditures—covering hardware acquisition, energy consumption, and maintenance overhead—while performance is assessed by metrics such as throughput (e.g., examples per second), latency of individual tasks, and overall time-to-solution. Availability, by contrast, reflects the system’s ability to sustain service in the face of component failures and transient network disruptions, often expressed as mean-time-between-failures (MTBFs) or expected uptime percentage [99].

From an optimization standpoint, these three objectives conflict: lowering cost by employing fewer or less powerful nodes generally degrades performance and may reduce redundancy mechanisms crucial for high availability; conversely, over-provisioning for peak performance or fault tolerance increases both capital outlay and ongoing energy costs. To navigate these interdependencies, a multi-objective formulation is adopted whereby each candidate DCS configuration is evaluated along the axes of total cost, benchmarked performance, and modeled availability. Pareto-optimal fronts are then constructed to identify configurations for which no single metric can be improved without degrading another [100].

Within this framework, user-defined weightings enable explicit prioritization—e.g., “minimize cost subject to a minimum 99.9% uptime” or “maximize throughput within a fixed budget”. Sensitivity analysis on these weights reveals the regions of diminishing returns; for instance, doubling node count may yield only marginal latency gains beyond a certain scale, while the incremental cost of additional redundancy may plateau in terms of availability improvement once required reliability thresholds are surpassed.

In practice, the chosen configuration emerges from the intersection of organizational constraints and application demands: real-time or safety-critical systems will typically favor availability and latency over cost, whereas exploratory research workflows may accept lower availability in exchange for reduced expenditure. By explicitly parameterizing and quantifying each dimension, the proposed toolchain empowers decision-makers to select DCS architectures that best align with the economic, performance, and reliability requirements of their specific ANN-modeling tasks [101].

6.3. Future Work

Directions for further work include several key areas of improvement and expansion. First, there is a need for the improvement of mathematical models. Future research should focus on enhancing the existing mathematical models to incorporate new factors, such as workload changes [102], adaptation to technological shifts [103], and the prediction of evolving needs in distributed computing systems. The current use of the M/M/n model’s assumption [104] of exponential inter-arrival and service times ($c_a^2=c_s^2=1$) simplifies analysis via the Erlang-C formula but fails to capture observed burstiness ($c_a^2>1$) and heavy-tailed service times (e.g., Pareto or Weibull) in production traces [105]. Adopting a G/G/n framework allows specification of the squared coefficients of variation $c_a^2$ and $c_s^2$, enabling use of Kingman’s approximation for mean waiting time

$$\mathbb{E}\left[W\right]\approx {\displaystyle \frac{{\rho }^{\sqrt{2(n+1)}-1}}{n(1-\rho )}}·{\displaystyle \frac{c_a^2+c_s^2}{2}},$$

and diffusion-limit results in the Halfin–Whitt (QED) regime [89] for large n. Such generalizations can be implemented via matrix-analytic methods or fluid approximations, yielding accurate bounds on queue lengths and server utilizations under non-Markovian traffic [106].

Transitioning to G/G/n requires estimating $c_a^2$ and $c_s^2$ from empirical workload logs (e.g., MapReduce job arrivals) and integrating these into discrete-event simulations or asymptotic formulas. Although this increases computational overhead, it provides tighter performance envelopes and more reliable energy-availability trade-offs. In particular, accounting for long-tail service variances can prevent underestimation of queueing delays and overcommitment of resources.

Rather than revising the entire framework, future work will extend the existing M/M/n analysis by (1) fitting general arrival and service distributions to collected DCS workload data, (2) applying Kingman’s and Halfin–Whitt approximations to derive new performance bounds, and (3) validating these against simulation results. This incremental approach preserves analytical tractability while aligning the model with realistic, non-exponential behaviors in distributed neural-network training workloads.

The existing reliability assessment assumes identical nodes and independent failure events, which understates the impact of cascading outages in multi-tier or mesh networks. In practical deployments, individual servers exhibit distinct failure rates and repair times that depend on hardware characteristics, workload, and network position, and a fault in one node can elevate the failure risk of its neighbors [107]. To address this, the model can be extended using multi-state Continuous-Time Markov Chains [108] in which each node transitions among operational, degraded, and failed states with empirically derived rates ${\lambda }_i$ and ${\mu }_i$, and inter-node dependencies are captured by state-dependent modifiers that increase a neighbor’s failure rate by a conditional probability $p_{ij}$. Complementing this, a fault-propagation graph represents the system as a directed network $G=(V,E)$, where edges weighted by $p_{ij}$ quantify the likelihood that failure of i triggers failure of j, enabling system availability to be computed via minimal cut-set enumeration or Monte Carlo simulation in large meshes [109]. For hierarchical, multi-tier architectures, tier-specific redundancy factors $r_k$ and inter-tier dependency coefficients ${\delta }_{k,k+1}$ can be incorporated into a layered reliability block diagram [110] to derive overall uptime $A={\prod }_k{A}_k^{{\delta }_{k,k+1}}$. Finally, Dynamic Bayesian Networks can model the temporal evolution of faults and repairs, accommodating arbitrary sojourn time distributions and allowing real-time inference of unavailability and cascade likelihood [111]. Integrating these topology-aware, stateful reliability methods into the decision-support system will produce more accurate availability estimates, identify critical nodes via centrality metrics, and guide the placement of redundancy where it most effectively mitigates propagation risk in heterogeneous distributed computing deployments.

The integration of evolutionary game theory (EGT) or distributed-control theory into the resource-allocation module can endow the system with dynamic adaptability, provable convergence, and robustness against perturbations. In an EGT framework, computational nodes are modeled as players in a game whose strategies correspond to resource-allocation policies. Payoff functions can be defined in terms of local utility metrics—such as throughput, energy efficiency, or queueing delay—and evolutionary dynamics (e.g., replicator equations)

$${\dot{x}}_i=x_i\left(u_i\left(\mathbf{x}\right)-\overline{u}\left(\mathbf{x}\right)\right),$$

drive the population state $\mathbf{x}$ toward an evolutionarily stable strategy (ESS) [112]. Under mild assumptions on payoff continuity and concavity, standard Lyapunov arguments guarantee global asymptotic stability of the ESS, ensuring that resource shares converge without oscillation or deadlock.

Alternatively, distributed-control theory offers a complementary approach where each node executes a local control law based on neighbor information, leveraging consensus protocols and passivity-based design. For example, define $r_i\left(t\right)$ as the resource allocation at node i. A distributed gradient-descent law

$${\dot{r}}_i=-{\alpha }_i{\nabla }_{r_i}{J}_i\left(r_i\right)+\sum _{j\in {\mathcal{N}}_i}{\beta }_{ij}\left(r_j-r_i\right),$$

combines a local cost gradient ${\nabla }_{r_i}{J}_i$ with consensus coupling over the communication graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$. Under standard connectivity and step-size conditions (${\alpha }_i,{\beta }_{ij}>0$), this protocol converges to the global minimizer of the aggregate cost $J_{\mathrm{tot}}={\sum }_i{J}_i$ with exponential rate [113]. Robustness to link failures and time-delays can be established via input-to-state stability (ISS) or small-gain theorems, granting formal guarantees in volatile network conditions.

By unifying EGT’s payoff-driven adaptation with distributed control’s consensus and stability theory, the proposed decision-support system can automatically adjust resource allocations in response to workload fluctuations, topology changes, or node failures, while mathematically ensuring convergence to optimal or near-optimal operating points and robustness margins against parameter uncertainty.

To improve the applicability of the theoretical models presented, it would be beneficial to validate them through empirical testing using a laboratory system comprising computers with different hardware configurations. A verification section will be introduced using a controlled hardware testbed. This testbed will consist of a small cluster of heterogeneous servers—each equipped with Intel CPUs supporting RAPL power-measurement interfaces and NVIDIA GPUs exposing power metrics via NVML—so that both peak and real-time power draw can be recorded with millisecond precision. Prior to workload execution, each node’s peak FLOPS capacity and base power consumption will be profiled through microbenchmarks, and network latency and bandwidth will be calibrated using tools such as iPerf3.

7. Conclusions

To solve complex scientific and technical issues in heterogeneous CSNs with DCSs, effective mathematical models were developed to evaluate the operation and reliability of such networks. The success of this approach validates the proposed methodology.

Based on these mathematical models, modern software tools were constructed to facilitate the selection of an effective client–server type CN structure, specifically for solving ANN modeling problems in complex systems across various applications. These tools enhance system efficacy and support complex distributed computing processes to tackle these challenges.

The developed DSS was tested in the context of selecting an efficient DCS structure for LLC “RITS” in Krasnoyarsk city. Using the available enterprise data, the system determined the following results:

• The total hardware cost of the FFGA-optimized configuration is 250 kUSD.
• The performance is 1220.745 TFLOPS;
• The availability factor is 99.03%.

In future work, the automated DSS will be integrated into both governmental and commercial sectors, boosting the efficiency of distributed problem-solving for structural and parametric synthesis of ANN models in complex systems, leveraging GRID technology. Future research should focus on refining the current models, broadening the system’s capabilities, and applying it across various industrial and scientific domains. This will ultimately enhance the efficiency of solving complex problems within distributed computing networks.