Multi-Objective Joint Optimization for Microservice Deployment and Request Routing

Abstract

Microservice deployment and request routing can help improve server efficiency and the performance of large-scale mobile edge computing (MEC). However, the joint optimization of microservice deployment and request routing is extremely challenging, as dynamic request routing easily results in asymmetric network structures and imbalanced microservice workloads. This article proposes multi-objective joint optimization for microservice deployment and request routing based on structural symmetry. Firstly, the structural symmetry of microservice deployment and request routing is defined, including spatial symmetry and temporal symmetry. A constrained nonlinear multi-objective optimization model was constructed to jointly optimize microservice deployment and request routing, where the structural symmetric metrics take into account the flow-aware routing distance, workload balancing, and request response delay. Secondly, an improved artificial plant community algorithm is designed to search for the optimal route to achieve structural symmetry, including the environment preparation and dependency installation, service packaging and image orchestration, arrangement configuration and dependency management, deployment execution and status monitoring. Thirdly, a benchmark experiment is designed to compare with baseline algorithms. Experimental results show that the proposed algorithm can effectively optimize structural symmetry and reduce the flow-aware routing distance, workload imbalance, and request response delay, while the computational overhead is small enough to be easily deployed on resource-constrained edge computing devices.

1. Introduction

Microservice deployment (MD) and request routing (RR) are crucial in improving service quality, reducing maintenance costs, enhancing system reliability and security [1,2]. The application of microservice deployment and request routing can more efficiently process and analyze the massive data from mobile edge computing (MEC) devices, providing strong support for intelligent data processing, remote monitoring, and fault prediction [3,4]. However, microservice deployment and request routing also faced many challenges. Firstly, the microservice environment may be highly dynamic [5], such as having different types of service requests, request routing fluctuations, the diversity of user needs, unpredictable traffic spikes (e.g., flash crowds), etc. Microservice deployment and request routing must be able to adapt to these changes and maintain stable operation. Secondly, many industrial applications, such as automated control and monitoring systems, are extremely sensitive to delay [6]. Microservice deployment and request routing must provide rapid responses to ensure real-time processing of tasks. Thirdly, edge computing devices and edge servers typically have limited computing power and storage capacity.

Given the complexity of the problem, a common approach is to break it down into two sub-problems and study them separately, i.e., the microservice deployment sub-problem and the request routing sub-problem. Both sub-problems are considered NP-hard by existing research, though the models used by different scholars vary greatly, such as a mixed-integer nonlinear program (MINLP) [1], or an integer nonlinear program (INLP) [3]. The authors of [7] considered the problem of microservice latency optimization as an NP-hard mathematical optimization problem and provided a fine-tuned Large Language Model (LLM) to solve it. The authors of [8] insisted that the cost optimization of microservice deployment is a statistical NP-hard problem and employed a revised machine learning (ML) algorithm to make decisions regarding task offloading. The authors of [9] reviewed a total of 239 related works and selected 38 of them to identify three types of deployment approaches and seven different communication patterns, where eight deployment challenges and six communication challenges were identified. The communication patterns of microservices may impact the sub-problem of request routing.

Considering that traditional algorithms struggle to solve such NP-hard problems, various artificial intelligence (AI) algorithms have been introduced to help us solve them. The authors of [3] proposed a fine-grained reinforcement learning (RL)-based algorithm, and the authors of [5] put forward a deep reinforcement learning (DRL) algorithm based on reward shaping. The authors of [10] treated the microservice placement problem as fuzzy graph clustering followed by heuristic packing, and the authors of [11] introduced a graph deep learning (DL) model to meet various quality of service (QoS) requirements. Among them, various heuristic algorithms, with their simple computation and strong universality, have also been used to solve microservice deployment problems. The authors of [12] introduced a non-dominated sorting genetic algorithm (NSGA-III) to seek the optimal deployment strategy of microservice instances in various resource centers, and the authors of [13] presented a particle swarm optimization (PSO) algorithm to solve the multi-objective optimization problem of container-based microservice scheduling. In addition, there are ant colony optimization (ACO) [14], gray wolf optimizer (GWO) [15], greedy algorithm [15], and whale optimization algorithm (WOA) [16], which have also been proven to be effective.

The above research did not effectively integrate microservice deployment and request routing issues as a whole and consider the structural symmetry between the two. In order to complete the fast solution on the resource-limited edge computing platform, we improved the artificial plant community (APC) algorithm in [17]. Our contributions are as follows.

Firstly, the structural symmetry of microservice deployment is defined, including spatial symmetry and temporal symmetry. A constrained nonlinear multi-objective optimization (MOO) model was constructed to jointly optimize microservice deployment and request routing, systematically considering the structural symmetric metrics, i.e., the flow-aware routing distance, workload balancing, and request response delay. Various deployment factors are considered, which are very complex and interact with each other, forming extremely challenging constraints.

Secondly, considering the NP-hard nature of this problem, an improved artificial plant community algorithm is designed to search for the optimal route to achieve structural symmetry. It includes the environment preparation and dependency installation, service packaging and image orchestration, arrangement configuration and dependency management, deployment execution and status monitoring. The global and local search capabilities of the proposed algorithm are enhanced, and the computing process is simplified for deployment on edge computing devices.

Thirdly, a benchmark experiment is designed to compare with baseline algorithms. Experimental results show that the proposed algorithm can effectively reduce the flow-aware routing distance, workload balancing, and request response delay, while the computational overhead is small enough to be easily deployed on resource-constrained edge computing devices.

The article is structured as follows: Section 2 reviews related work. Section 3 introduces the multi-objective optimization model, and Section 4 describes an APC-based solution mechanism. Section 5 illustrates a benchmark test set and a set of benchmark experiments. Section 6 concludes the article and points out the next research work.

2. Related Work

With the continuous advancement of the microservices, the number of mobile edge computing devices has risen sharply, leading to an astonishing rate of data growth [2]. Microservice deployment and request routing, by decentralizing computing power to the network’s edge, not only alleviates the pressure on the cloud computing center but also significantly reduces the cost of data transmission and data response time [3]. In microservice deployment and request routing, there are also significant challenges.

The first major challenge in microservice deployment and request routing is dynamic service requirements. In the microservice environment, service requirements are constantly changing, and some edge computing services have higher requirements for real-time performance. The authors of [12] considered the computation and storage resource utilization rate, load balancing rate, and actual microservice usage rate in resource service centers. The authors of [13] took into account the network transmission overhead, ineffective load balancing, and low service reliability. The authors of [18] aimed at minimizing both system communication overhead and microservice deployment cost, and the authors of [19] minimized the execution cost and communication costs among the microservices. The authors of [20] improved the secure and efficient placement of microservices in a larger attack environment. The authors of [21] developed a multi-channel directed graph convolutional network model to capture the spatial dynamics of edge device distribution and jointly considered the heterogeneity of the edge devices and the links between them.

The second major challenge is that many microservice applications are latency sensitive. The authors of [3] introduced delay-aware optimization of fine-grained microservice deployment and routing in the edge via reinforcement learning. The authors of [5] proposed a reinforcement learning algorithm based on reward shaping to minimize user waiting delay and edge network resource consumption. The authors of [15] aimed at minimizing users’ latency and maximizing edge providers’ profits. The authors of [22] formulated the microservice deployment problem in satellite edge computing networks as a partially observable Markov decision process (POMDP) to minimize the workload imbalance and the service latency. Due to the potential hardware breakdowns, each edge server’s failures may affect its lifetime, i.e., the time length to work continuously without interruptions, and in turn lead to the microservice collapse or the request latency. The authors of [23] considered dynamic task generations and the asynchronization of various decision variables with different microservices and designed an online optimization algorithm integrating randomized rounding, Lagrangian method, and convex optimization, to iteratively solve the microservice deployment problem over different timescales.

The third major challenge is that the edge computing devices deployed with microservices often have limited computing power and energy. Due to the limited CPU and storage resources in edge computing devices, the coarse-grained service deployment on edge devices easily leads to performance bottlenecks. The authors of [5] illustrated a heuristic algorithm to scale microservice instances horizontally in dynamic user request states with less edge network resource consumption. The authors of [24] designed a microservice deep-learning edge detection framework for easy deployment and distributed processing in edge computing environments. The authors of [25] present a microservice architecture to provide multi-user cloud virtual reality and an efficient deployment scheme in edge computing networks. The authors of [26] defined the service placement problem in microservice systems as a fractional polynomial problem to achieve rapid development and continuous delivery in a highly distributed edge computing environment. The authors of [27] exploited a hybrid particle swarm optimization method to solve the microservice-based SaaS deployment problem and reduce the increase in energy consumption.

To address the above challenges, AI algorithms represented by deep learning and heuristic learning are widely used. Deep learning is based on multi-layer neural networks, which automatically extract features from training data and pursue data-driven accurate prediction, such as the DL [3,24], RL [3,20], DRL [5,21,22,28], ML [8], graph deep learning [11], distributed multi-agent reinforcement learning (DMARL) [22], graph neural networks [29], deep Q learning [30], etc. It requires a large amount of data support, and the objective function is usually differentiable. It is suitable for complex data pattern recognition and fields that require high-precision prediction. However, heuristic learning relies on heuristic functions to evaluate states, focusing on quickly finding feasible solutions through empirical strategies without strict guarantees on solution quality, such as genetic algorithm (GA) [4,12], PSO [13,27], ACO [14], GWO [15], greedy algorithm [15,26], WOA [16], APC algorithm [17,31], adaptive crested porcupine optimizer (ACPO) algorithm [18], sand cat swarm optimization [32], etc. It is suitable for NP-hard problems or time-sensitive problems with large solution spaces.

Therefore, existing research in recent years continues to pay attention to the optimization challenges in microservice deployment and request routing. However, there are still research gaps. Firstly, most research work has not effectively integrated microservice deployment and request routing, making it challenging to analyze the correlation between the dynamic request routing, asymmetric network structures, and imbalanced microservice workloads. Second, given that both sub-problems are NP-hard, their joint optimization is also NP-hard. It is very challenging to design efficient algorithms to solve such NP-hard problems on the edge computing platform. Thirdly, there is a lack of suitable benchmark datasets for baseline algorithm comparison or for deep learning algorithm training. The research motivation of this manuscript is to attempt to fill these research gaps.

3. Materials and Methods

This paper first uses an example to achieve microservice deployment and request routing in an edge computing manner. The microservice deployment and request routing are integrated in a constrained nonlinear multi-objective optimization model to achieve structural symmetry and joint optimization. It defines the flow-aware routing distance, workload balancing, and request response delay, enabling the microservice system to dynamically adjust its behavior according to real-time use requests.

3.1. Symbol Definition

The example for microservice deployment and request routing is shown in Figure 1, which includes the remote cloud to remotely access computing resources in the cloud, sensor nodes for data collection, the edge computing devices for data access, and the edge servers for data processing. Container technology, such as Docker, packages applications and all their dependencies into lightweight, portable containers to ensure consistent service performance across different environments, providing an ideal runtime environment for microservices. Figure 1 employs a decentralized architecture for microservice deployment and request routing on the edge servers. The sensor nodes and the edge computing devices communicate through wireless channels, while the edge server is connected to the remote cloud via wired links. Asymmetric structure can easily lead to workload imbalance among server clusters with different geographical locations and network structures, resulting in excessive local data traffic and network overload.

In the microservice system, there is a set of microservices, $M=\{1,2,\dots ,m,\dots ,m_{max}\}$, and a set of edge servers, $S=\{1,2,\dots ,s,\dots ,s_{max}\}$, where $C_s^{comp}$ and $C_s^{stor}$ represent the computing capacity and storage capacity of edge server $s$, respectively. There are microservice deployments that operate in discrete time slots $T=\{1,2,\dots ,\dots ,t,\dots ,t_{max}\}$, where $T$ represents a finite time horizon. Compared to the limited storage and computing resources of edge servers, it is assumed that the remote cloud has sufficient computing and storage capacity and can complete all microservice requests in time. For request routing, $R_m^{comp}$ (in CPU cycles) represents the computing resource required to process microservice $m$, and $R_m^{stor}$ (in bits) is the storage capacity required to place microservice $m$. This ratio, ${\gamma }_{s,m}^{comp}(t)$, is a number between 0 and 1 that represents how computing resources in edge server $s$ are shared among different microservices.

Definition 1.

Structural symmetry refers to maintaining the local similarity and periodicity of the network structure during microservice deployment and request routing, including spatial and temporal symmetry. Spatial symmetry refers to maintaining the visual and topological symmetry of the network structure during microservice deployment and request routing, including the local similarity of nodes and request routing in microservice deployment. Temporal symmetry refers to the periodicity of similar businesses and functions before and after microservice deployment and request routing.

Unlike traditional metrics such as network centrality, the proposed definition attempts to measure the structural symmetry of microservice deployment and request routing from both spatial and temporal dimensions.

The example in Figure 1 describes the process of microservice deployment and request routing, based on which a constrained nonlinear multi-objective optimization model is built in the following section. For clarity, the main symbols used in our multi-objective optimization model are shown in

Table 1. Main symbols.

Notation	Meaning
$S$	Set of edge servers
$s$	An edge server
$s_{max}$	Number of edge servers
$c$	Core network
$C_s^{comp}$	The computing capacity of edge server $s$
$C_s^{stor}$	The storage capacity of edge server $s$
$M$	Set of microservices
$m_{max}$	The number of microservices
$m$	A microservice
$t$	A time slot
$T$	Set of discrete time slots
$t_{max}$	The finite time horizon
$R_m^{comp}$	Computing resource for processing microservice $m$
$R_{s,m}^{\sum comp}(t)$	The overall computation workload required when microservice $m$ is operated on edge server $s$ at time slot $t$
$R_m^{stor}$	Storage capacity required to place microservice $m$
$R_{s,m}^{\sum stor}(t)$	The overall storage capacity required when microservice $m$ is operated on edge server $s$ at time slot $t$
${\gamma }_{s,m}\left(t\right)$	A microservice deployment indicator. We set ${\gamma }_{s,m}\left(t\right)=1$ (or 0) if microservice $m$ is placed on the edge server $s$ (or not) at time slot $t$.
${\gamma }_{s,m}^{\ast }\left(t\right)$	The optimal microservice deployment indicator
$x_M\left(t\right)$	All microservice deployment decisions
$x_M^{\ast }\left(t\right)$	The optimal microservice deployment decision at time slot $t$
${\gamma }_{s,m}^{comp}\left(t\right)$	Fraction of computing capacity for microservice $m$ allocated by edge server $s$ at time slot $t$, ${\gamma }_{s,m}^{comp}\left(t\right)\in \left[0,1\right].$
$x_R\left(t\right)$	All resource allocation strategies
$m_{s,m}^{requ}(t)$	The number of requests for microservice $m$ on edge server $s$ at time slot $t$
$m_m^{\sum requ}(t)$	Total number of requests for microservice $m$ at time slot $t$
$L\left(t\right)$	All request routing strategies
$L_{s,m}\left(t\right)$	Workload ratio where microservice $m$ is operated on edge server $s$ at time slot $t$
$\beta$	The probability that microservice $m$ is outsourced to the remote cloud at time slot $t$
$∆L_{s,m}\left(t\right)$	The minimum processing unit of the request routing scheme
$d_{s,s^{\prime }}$	The flow-aware routing distance between two edge servers $s$ and $s$’
$w_{{ss}^{\prime }}$	The actual request flows or traffic volumes between two edge servers $s$ and $s$’
$d_{s,m}^{comp}\left(t\right)$	The computational delay when microservice $m$ is handled on edge server $s$
$r_{edge}^{tran}(t)$	The transmission rate between edge servers
$d_{s,m}^{tran}(t)$	The transmission delay when microservice $m$ is handled on edge server $s$
$r_{c,m}^{tran}\left(t\right)$	The transmission rate of core network when microservice $m$ is transmitted at time slot $t$
$d_{c,m}^{proc}\left(t\right)$	The processing delay of microservice $m$ outsourced to a remote cloud at time slot $t$
$d_m^{avg}\left(t\right)$	The average response delay of microservice $m$ on all edge servers
$d_{sens}^{tran}(t)$	The transmission delay between sensing devices and edge servers
$u_s\left(t\right)$	The state of edge server $s$ at time slot $t$
$v_s\left(t\right)$	The action space of edge server $s$ at time slot $t$
$F\left(t\right)$	The reward function at time slot $t$
$Q(u,v)$	The expected total sum of future rewards for $T$ time steps
$\alpha$	The discount factor to diminish the influence of future rewards on present decision-making
$E$	The expectation with respect to the time-varying system environments

.

3.2. A Constrained Nonlinear Multi-Objective Optimization Model

3.2.1. The Flow-Aware Routing Distance

According to Definition 1, the primary objective is to evaluate spatial symmetry, which means maintaining the image symmetry and topological symmetry of the network structure during microservice deployment, including the local similarity of nodes and request routing in microservice deployment. Here we consider a flow-aware routing distance metric weighting by actual request flows or traffic volumes.

A binary variable ${\gamma }_{s,m}\left(t\right)$ is defined to indicate whether microservice $m$ is deployed on the edge device or edge server $s$ at time slot $t$. If microservice $m$ is deployed on edge server $s$, it is set as ${\gamma }_{s,m}\left(t\right)=1$; otherwise, ${\gamma }_{s,m}\left(t\right)=0$. A decision variable is defined in Equation (1) for microservice deployment.

$$x_M\left(t\right)=\left\{{\gamma }_{s,m}\left(t\right)\right|s\in S,m\in M\}$$

(1)

As the task must be implemented, microservice $m$ not deployed on the edge server will be processed by core network $c$. There is

$${\sum }_{s=1}^{S\cup c}{\gamma }_{s,m}\left(t\right)=1,\forall m,\forall t.$$

(2)

It is assumed that there are two edge servers $s$ and $s^{\prime }$ with coordinates $s(x_s,y_s)$ and $s^{\prime }(x_{s^{\prime }},y_{s^{\prime }})$, and the actual request flows or traffic volumes between them are described as a weight, $w_{{ss}^{\prime }}$; so the flow-aware routing distance between them can be calculated as a nonlinear constraint in Equation (3).

$$d_{{ss}^{\prime }}=w_{{ss}^{\prime }}\sqrt{{(x_s-x_{s^{\prime }})}^2+{(y_s-y_{s^{\prime }})}^2}$$

(3)

The first objective of the joint optimization of microservice deployment and request routing is to minimize the flow-aware routing distance, which reflects the structural symmetry of microservice deployment, such as unpredictable traffic spikes or flash crowds. According to Equation (3), the minimization of the total flow-aware routing distance can be calculated as follows:

$${obj}_1=\mathrm{m}\mathrm{i}\mathrm{n}\{{\sum }_{s,s^{\prime }}{d}_{s{s}^{\prime }}|s,s^{\prime }\in S\}.$$

(4)

3.2.2. The Workload Balancing

According to Definition 1, temporal symmetry is also an important feature of structural symmetry, indicating the similarity of businesses and functions before and after maintaining route changes during microservice deployment.

The total flow-aware routing distance in Equation (4) describes the structural symmetry of microservice deployment from a static routing perspective, while the workload balancing in the second objective determines the structural symmetry from a dynamic routing perspective. In dynamic routing, all microservice scheduling strategies are defined as

$$x_R\left(t\right)=\left\{{\gamma }_{s,m}^{comp}\left(t\right)L_{s,m}\left(t\right)\right|s\in S,m\in M\},$$

(5)

where $L_{s,m}(t)$ represents the proportion of edge computing power $C_s^{comp}$ allocated to microservice $m$, so the higher the value, the more computing resources are allocated.

The fraction of computing capacity for microservice $m$ is allocated by the edge device or edge server $s$ at time slot $t$.

$${\gamma }_{s,m}^{comp}\left(t\right)\in \left[0,1\right],\hspace{1em} \forall m,\forall t$$

(6)

Workload ratio can be calculated as $L_{s,m}\left(t\right)$, where microservice $m$ is operated on edge server $s$ at time slot $t$.

$$L_{s,m}\left(t\right)\in \left[0,1\right],\forall m,\forall t$$

(7)

At time slot $t$, the total number of requests for microservice $m$ in the microservice can be obtained as

$$m_m^{\sum requ}\left(t\right)={\sum }_{s=1}^S{m}_{s,m}^{requ}\left(t\right),\forall m,\forall t$$

(8)

Based on the microservice layout and scheduling strategy in Equations (6)–(8), the total computing capacity $R_{s,m}^{\sum comp}\left(t\right)$ and total storage capacity $R_{s,m}^{\sum stor}\left(t\right)$ running on the edge server $s$ can be expressed as follows:

$$R_{s,m}^{\sum comp}\left(t\right)={\gamma }_{s,m}^{comp}\left(t\right)L_{s,m}(t)R_m^{comp}{m}_m^{\sum requ}\left(t\right)$$

(9)

$$R_{s,m}^{\sum stor}\left(t\right)={\gamma }_{s,m}^{comp}\left(t\right)L_{s,m}(t)R_m^{stor}{m}_{s,m}^{requ}\left(t\right)$$

(10)

We note that if $R_{s,m}^{\sum comp}\left(t\right)\ge C_s^{comp}{\gamma }_{s,m}^{comp}\left(t\right)$, the actual allocated resources are higher than the theoretical required resources, then the computational workload required for microservice $m$ comes from the edge server $s$; otherwise, the surplus workload $(R_{s,m}^{\sum comp}\left(t\right)-C_s^{comp}{\gamma }_{s,m}^{comp}\left(t\right))$ corresponds to the nearby edge servers. Hence, the proposed system can cope with sudden and unpredictable spikes, such as traffic spikes.

The second objective of the joint optimization of microservice deployment and request routing is to minimize the workload balancing, which reflects the difference between the maximum workload and the minimum workload. According to Equations (9) and (10), the minimization of the workload balancing can be calculated as follows:

$${obj}_2=\left\{\min \{{\displaystyle \frac{\underset{s,m}{\max }{R}_{s,m}^{\sum comp}\left(t\right)-\underset{s,m}{\min }{R}_{s,m}^{\sum comp}\left(t\right)}{\underset{s,m}{\max }{R}_{s,m}^{\sum comp}\left(t\right)}}\},\mathrm{m}\mathrm{i}\mathrm{n}\{{\displaystyle \frac{\underset{s,m}{\max }{R}_{s,m}^{\sum stor}\left(t\right)-\underset{s,m}{\min }{R}_{s,m}^{\sum stor}\left(t\right)}{\underset{s,m}{\max }{R}_{s,m}^{\sum stor}\left(t\right)}}\}\right\}.$$

(11)

The workload balancing objective adopts the max–min normalization method between servers and microservices, which is very sensitive to extremely heavy and light workloads, and only requires two parameters with low computational overhead. On lightweight computing platforms, it may be faster and more energy-efficient than variance- or entropy-based balancing methods.

3.2.3. The Request Response Delay

According to Definition 1, temporal symmetry also includes maintaining similar functionality before and after routing changes and exhibiting periodicity in time during microservice deployment.

The first objective in Equation (4) and the second objective in Equation (11) describe the structural symmetry of microservice deployment from a global perspective, while the request response delay in the third objective measures the structural symmetry from a local perspective. Due to the different locations of edge servers in the network, the request response delay will also vary, even if the total flow-aware routing distance and workload balancing are similar. The microservice response delay mainly consists of two parts: the computational delay and transmission delay.

The computational delay refers to the time required to complete a certain amount of workload on an edge device, which is related to the computing power of the device. The computational delay for processing microservice $m$ on the edge server $s$ can be represented as the ratio of the required workload to the allocated computing capacity. At the same time, the computational delay is also affected by the proportion of computing capacity ${\gamma }_{s,m}^{comp}\left(t\right)$ allocated to the microservice $m$, because even if microservice $m$ is allocated to a certain edge server, if the computing capacity allocated to it is very small, then the computational delay will also be very high. Regarding the computational delay, since the computing capacity of the cloud is relatively large, only the computational delay of data on the edge server is considered. The computational delay $d_{s,m}^{comp}\left(t\right)$ for microservice $m$ processed on edge server $s$ can be obtained as

$$d_{s,m}^{comp}\left(t\right)={\gamma }_{s,m}\left(t\right){\displaystyle \frac{R_{s,m}^{\sum comp}\left(t\right)}{{\gamma }_{s,m}^{comp}\left(t\right)C_s^{comp}}}.$$

(12)

The transmission delay refers to the time required for transmission between two devices, which is related to the location of the device in the network. It contains the transmission delay of data between the edge server and the cloud, the transmission delay of data being transported to the cloud, and the transmission delay or wireless delay between sensing devices and edge servers. It is considered that there are multiple adjacent edge servers with a transmission rate of $r_{edge}^{tran}(t)$, and the transmission delay between sensing devices and edge servers is $d_{sens}^{tran}(t)$; the transmission delay $d_{s,m}^{tran}\left(t\right)$ of the edge server can be obtained as follows:

$$d_{s,m}^{tran}\left(t\right)={\displaystyle \frac{{\gamma }_{s,m}\left(t\right)}{r_{edge}^{tran}\left(t\right)}}\mathit{max}\left\{\left(R_{s,m}^{\sum comp}\left(t\right)-C_s^{comp}{\gamma }_{s,m}^{comp}\left(t\right)\right),0\right\}+d_{sens}^{tran}(t).$$

(13)

Equation (13) includes two parts, where the first part is the transmission delay of data between the edge server and the cloud or the transmission delay of data being transported to the cloud, and the second part is the transmission delay between sensing devices and edge servers. When $R_{s,m}^{\sum comp}\left(t\right)\ge C_s^{comp}{\gamma }_{s,m}^{comp}\left(t\right)$, it indicates that microservice $m$ is running on multiple edge servers. Therefore, the transmission delay between edge servers is the net data volume divided by the transmission rate.

The transmission rate of the core network for microservice $m$ at time slot $t$ is defined as $r_m^{tran}(t)$, so the cloud processing delay $d_{c,m}^{proc}\left(t\right)$ can be expressed as:

$$d_{c,m}^{proc}\left(t\right)={\displaystyle \frac{1}{r_{c,m}^{tran}\left(t\right)}}(1-{\sum }_{s=1}^S{\gamma }_{s,m}\left(t\right))(1-{\sum }_{s=1}^S{L}_{s,m}\left(t\right)){\sum }_{s=1}^S{R}_{s,m}^{\sum comp}\left(t\right).$$

(14)

The processing delay for outsourcing to the core network is determined by the ratio of the total data volume to be transmitted and the transmission rate. In Equation (14), the two parts, $1-{\sum }_{s=1}^S{\gamma }_{s,m}\left(t\right)$ and $1-{\sum }_{s=1}^S{L}_{s,m}\left(t\right)$, constrain microservice $m$ to be processed, that is, microservice $m$ at time slot $t$ is either processed in the edge server $s$ or outsourced to the remote cloud.

Considering the computational delay $d_{s,m}^{comp}\left(t\right)$ in Equation (12), the transmission delay $d_{s,m}^{tran}\left(t\right)$ in Equation (13), and the cloud processing delay $d_{c,m}^{proc}\left(t\right)$ in Equation (14), the average response delay for microservice $m$ can be obtained as follows:

$$d_m^{avg}\left(t\right)={\sum }_{s=1}^S\left(d_{s,m}^{comp}\left(t\right)+d_{s,m}^{tran}\left(t\right)\right)+d_{c,m}^{proc}\left(t\right).$$

(15)

In the text, $d_m^{avg}\left(t\right)$ refers to the average value of the total response delay for microservice $m$, which includes three parts of delays. As the processing at the remote cloud usually involves longer transmission delays, it is important to reduce the outsourcing of microservices to the remote cloud. Therefore, an outsourcing penalty term $\beta$ is introduced, which is directly proportional to the outsourcing volume and the penalty coefficient $R_m^{comp}$ for the outsourced microservice. This penalty term reflects the impact of the outsourced microservice on the total response delay and encourages the algorithm to process as many microservice as possible on the edge server to reduce delay.

Hence, the request response delay we propose takes into account the delay of both the request and response processes, rather than just minimizing the delay. According to the average response delay in Equation (15), the third objective is to minimize the request response delay, so the corresponding optimization objective can be expressed as follows:

$${obj}_3=min\left\{d_m^{avg}\left(t\right)+\beta \left(1-\sum _{s=1}^S{\gamma }_{s,m}\left(t\right)\right)\left(1-\sum _{s=1}^S{L}_{s,m}\left(t\right)\right)\sum _{s=1}^S{R}_{s,m}^{\sum comp}\left(t\right)R_m^{comp}\right\}$$

(16)

From the perspective of ease of calculation, outsourcing penalty term $\beta$ can be quickly determined by user delay requirements and network size. The higher the user’s delay requirements and the larger the network size, the higher the $\beta$ value, and more effort is needed to reduce this objective.

Therefore, the solution is to search for the optimal microservice deployment strategies, which are defined as

$$x_M^{\ast }\left(t\right)=\left\{\right({\gamma }_{s,m}^{\ast }\left(t\right)\left|\right({\gamma }_{s,m}\left(t\right)=0 or 1,s\in S,m\in M\}.$$

(17)

Each edge server $s$ has two choices for each service $m$: to provide (${\gamma }_{s,m}\left(t\right)=1$) or not to provide (${\gamma }_{s,m}\left(t\right)=0$). Therefore, for a single edge server and all $M$ microservices, there are $2^M$ possible placement combinations. However, since there are $S$ edge servers, and each edge server can independently decide its microservice deployment, the strategy space for the entire system is the product of each edge servers, $2^{M\ast S}$, that is, the strategy space for microservice deployment is $2^M$ for each edge server $S$.

Then, the nonlinear multi-objective optimization model to jointly optimize microservice deployment and request routing can be obtained by the first objective (the flow-aware routing distance) in Equation (4), the second objective (the workload balancing) in Equation (11), and the third objective (the request response delay) in Equation (16).

$${obj}_{\sum }=\{{obj}_1,{obj}_2,{obj}_3\}$$

(18)

3.2.4. The Constraints

Section 3.2.1, Section 3.2.2 and Section 3.2.3 describe how to minimize the flow-aware routing distance, the workload balancing, and the request response delay of microservice deployment and request routing without considering the constraints. However, in microservice environments, various factors are very complex and interact with each other, forming extremely challenging constraints.

Constraint 1.

Since the computing capacity on the edge device or edge server is limited, the fraction of computing capacity for microservice $m$ allocated by edge server $s$ at time slot $t$ cannot exceedthe maximum value of 1, that is,

$$\sum _{m=1}^M{\gamma }_{s,m}^{comp}\left(t\right)\le 1,\forall s,\forall t$$

(19)

Constraint 2.

Due to the limited computing capacity on the edge device or edge server, the size of the computing resources allocated to microservice $m$ cannot exceed its maximum, that is,

$$\sum _{m=1}^M{\gamma }_{s,m}\left(t\right)R_m^{comp}\le C_s^{comp},\forall s,\forall t.$$

(20)

Constraint 3.

Similarly, due to the limited storage capacity on the edge device or edge server, the size of the storage resources allocated to microservice $m$ cannot exceed its maximum, that is,

$$\sum _{m=1}^M{\gamma }_{s,m}\left(t\right)R_m^{stor}\le C_s^{stor},\forall s,\forall t$$

(21)

Constraint 4.

As the task must be performed, so microservice $m$ not deployed on edge servers will be processed by core network.

$$\sum _{s=1}^{S\cup c}{L}_{s,m}\left(t\right)=1,\forall m$$

(22)

Constraint 5.

The $L_{s,m}(t)$ indicates the workload ratio of microservice $m$ running on the edge server $s$ at time slot $t$, and the number of requests for microservice $m$ on edge server $s$ at time slot $t$ is $m_{s,m}^{requ}(t)$.

$$L\left(t\right)=\left\{L_{s,m}\right(t\left)\right|s\in S,m\in M\}.$$

(23)

Constraint 6.

The state space represents the set of all states, including the number of requests for each microservice $m_{s,m}^{requ}\left(t\right)$, the current optimal resource allocation strategy ${\gamma }_{s,m}^{\ast }\left(t\right)$, the computing resources $C_s^{comp}$ of edge server $s$, and the storage resources $C_s^{stor}$ of edge server $s$.

$$u_s\left(t\right)=\left\{m_{s,m}^{requ}\left(t\right),C_s^{comp},C_s^{stor},{\gamma }_{s,m}^{*}\left(t\right)\right\}>\left\{0\right\}$$

(24)

Constraint 7.

The request routing $L_{s,m}\left(t\right)$ indicate the workload ratio of microservice $m$ running on the edge server $s$ at time slot $t$. $L_{s,m}\left(t\right)$ is a continuous value, so it needs to be discretized. Here, $L_{s,m}\left(t\right)$ is evenly divided and each part is $∆L_{s,m}\left(t\right)$, thus the load scheduling decision $∆L_{s,m}\left(t\right)$ can be expressed as follows:

$$∆L_{s,m}\left(t\right)\in \left\{∆L_{s,m}\left(t\right),\dots ,m∆L_{s,m}\left(t\right),\dots ,1\right\}$$

(25)

Constraint 8.

The action space represents the collection of all actions, mainly including microservice deployment strategies and request routing strategies. Therefore, the action space $v_s\left(t\right)$ of the edge server $s$ at time slot t is expressed as follows:

$$v_s\left(t\right)=\left\{m\in M,s\in S,{\gamma }_{s,m}\left(t\right)\in \left\{0,1\right\},L_{s,m}\left(t\right)\in \left\{ ∆L_{s,m}\left(t\right),\dots ,m∆L_{s,m}\left(t\right),\dots ,1\right\}\right\}$$

(26)

Constraint 9.

The reward function is used to evaluate the immediate reward from flow-aware routing distance for taking a certain action in a given situation. Therefore, we can define the reward function as

$$F\left(t\right)=-\sum _{m=1}^M{d}_m^{avg}\left(t\right)<0$$

(27)

Constraint 10.

The reward probability function is used to describe the reward value of the new state reached after taking action $v_s\left(t\right)$ from the given state $u_s\left(t\right)$, such as unpredictable traffic spikes or flash crowds.

$$Q(u,v)=E(\sum _{t=1}^T{\alpha }^{t}F\left(t+1\right)\left|u_s\left(t\right)=u_s\left(t+1\right),v_s\left(t\right)\right)>0$$

(28)

where $E[ ]$ is the expectation with respect to the time-varying system environment, that is, the weighted average of all possible future states and rewards. The time step starts from 1 to $T$. $F\left(t+1\right)$ represents the immediate reward obtained when $u_s\left(t\right)$ transitioning to a new state $u_s\left(t+1\right)$ after taking an action $v_s\left(t\right)$ in the current state.

Constraint 11.

The microservice deployment ${\gamma }_{s,m}\left(t\right)$ indicate whether microservice $m$ is deployed on the edge server $s$ at time slot $t$, assuming there are $S$ edge servers and $M$ microservices.

$${\gamma }_{s,m}\left(t\right)\in \left\{0,1\right\}$$

(29)

4. APC-Based Solution Mechanism

Considering the NP-hard nature of this problem, an improved artificial plant community algorithm is designed to search for the optimal route to achieve structural symmetry. The proposed solution mechanism is improved from the APC approach through enhancing its search ability and reducing its computational overhead.

4.1. An Improved APC Approach

The improved APC approach for joint optimization of microservice deployment and request routing employs simpler calculation steps and fewer parameters to achieve lightweight computation. The primary operations of the improved APC approach include seeding, growing, and fruiting. Hence, an artificial plant individual can be presented in three forms, i.e., a seed, an individual, and a fruit. Through three main operations, each individual form can have a certain probability of transitioning to another form or dying until the APC population gathers near the optimal solution.

The APC-based solution mechanism comprises four main steps, i.e., environment preparation and dependency installation, service packaging and image orchestration, arrangement configuration and dependency management, deployment execution and status monitoring.

Step 1: Environment preparation and dependency installation. Before deployment, it is necessary to ensure that the target environment has installed the container, such as the local development machine or server.

Step 2: Service packaging and image orchestration. Each microservice needs to package files through tools, write and define image orchestration for each service, and build context.

It is the first step of the improved APC approach to initialize the whole solution system, such as the main parameters of the problem to be solved and the APC approach.

The main parameters of the problem to be solved include a set of microservices, $M=\{1,2,\dots ,m,\dots ,m_{max}\}$, a set of edge servers, $S=\{1,2,\dots ,s,\dots ,s_{max}\}$, the position of edge servers $s$ with coordinates $s(x_s,y_s)$, the computing capacity ($C_s^{comp}$) and storage capacity ($C_s^{stor}$) of edge server $s$, a discrete time slot set $T=\{1,2,\dots ,\dots ,t,\dots ,t_{max}\}$, $R_m^{comp}$ (in CPU cycles) for the computing resource required to process microservice $m$, $R_m^{stor}$ (in bits) for the storage capacity required to place microservice $m$, the reward function of $F\left(t\right)$ at time slot $t$, the discount factor of $\alpha$ to diminish the influence of future rewards on present decision-making, and the probability of $\beta$ that microservice $m$ is outsourced to the remote cloud at time slot $t$.

The initialization parameters for an APC approach include the population size $P$, the seeding probability $p_{seed}$, the growing probability $p_{grow}$, and the fruiting probability $p_{fruit}$. An artificial plant individual is encoded as $x_M\left(t\right)=\left\{{\gamma }_{s,m}\left(t\right)|s\in S,m\in M\right\}$ in Equation (1). The fitness function $fit$ of APC can select the nonlinear multi-objective optimization model to jointly optimize microservice deployment and request routing in Equation (18).

$$fit={obj}_{\sum }$$

(30)

The constraints include Equations (19)–(29).

Step 3: Arrangement configuration and dependency management. Use containers to define dependencies, networks, and resource allocation between services. This step will list all services (such as databases, gateways, business services) and associate them with images, port mappings, environment variables, and dependencies (such as depending on to ensure service startup order). It will then create custom networks to enable communication between services through service names, and mount persistent data volumes, such as database storage directories.

Step 3-1: Seeding. In the first iteration $k=0$, a group of random APC seeds is produced with a population size of $P$, which means that there is no fruit and all seeds $\left\{x_{seed}^{rand}\left(k\right)\right\}$ are randomly produced. The population size of seeds is $P$.

$${\left\{x_{seed}\right(k=0\left)\right\}}_P={\left\{x_{seed}^{rand}\left(k\right)\right\}}_P=rand{\left\{{\gamma }_{s,m}\left(t\right)\right|s\in S,m\in M\}}_P.$$

(31)

In subsequent iterations $k>0$, the seeding probability $p_{seed}\in [0,1]$ determines the global search ability of the APC, which helps prevent the risk of the algorithm falling into local optima. The seed population $\left\{x_{seed}\right(k\left)\right\}$ includes two parts: the random seeds $\left\{x_{seed}^{rand}\left(k\right)\right\}$ for global search with a probability $p_{seed}$, and the fruits ${\{x}_{seed}^{fruit}\left(k-1\right)\}$ in the previous iteration with a probability $(1-p_{seed})$.

$${\left\{x_{seed}\right(k>0\left)\right\}}_P={\{x_{seed}^{rand}\left(k\right)p_{seed},x_{seed}^{fruit}\left(k-1\right)(1-p_{seed})\}}_P.$$

(32)

Step 3-2: Growing. In the growing step, it is to provide the fittest comparison and selection for the seed population through the fitness function $fit$ in Equation (30). Only the optimal seeds $\left\{x_{seed}\right(k\left)\right\}$ will be selected to grow with a growing probability $p_{grow}\in [0,1]$. Seeds with lower fitness will die, so the population size of the growing individuals $\left\{x_{grow}\right(k\left)\right\}$ will decrease to $P{p}_{grow}$. The growing step improves the convergence ability of the APC approach to provide any convergence guarantees or bounds on solution quality for the NP-hard multi-objective optimization problem.

$${\left\{x_{grow}\right(k\left)\right\}}_{P{p}_{grow}}=\{x_{seed}(k)|\underset{P{p}_{\mathit{grow}}}{\max }fit\left(x_{seed}\left(k\right)\right){\}}_{P{p}_{grow}}.$$

(33)

Step 3-3: Fruiting. The fruiting step provides the growing individuals $\left\{x_{grow}\right(k\left)\right\}$ with more opprtunities to produce more solutions $\left\{x_{fruit}\right(k\left)\right\}$, so as to strengthen the local search ability of the APC approach. The fruit population $\left\{x_{fruit}\right(k\left)\right\}$ includes two parts: the parthenogenesis of the parent $\left\{x_{grow}\right(k\left)\right\}$ for optimal solution preservation with a probability of $p_{seed}$, and the crossing fruits $\{x_{fruit}^{cross}(k)\}$ of the parent for local search with a population size of $P{p}_{grow}$.

$${\left\{x_{fruit}\right(k\left)\right\}}_{2P{p}_{grow}}=\left\{\{x_{grow}(k){\}}_{P{p}_{grow}},\{x_{fruit}^{cross}(k){\}}_{P{p}_{grow}}\right\}$$

(34)

Therefore, the total population size of the fruits has doubled, where better individuals may have more fruiting opportunities.

The crossing fruit is determined by a fruiting probability $p_{fruit}\in [0,1]$, which indicates how much original information a parent can retain. Assuming there are two parents, $x_{grow}^a\left(k\right)$ and $x_{grow}^b\left(k\right)$, so two possible fruits can be produced as follows:

$$\{\begin{array}{c}{x}_{fruit}^{cross}(k)=\{{x_{grow}^a(k)}_{p_{fruit}}//{x_{grow}^b(k)}_{{1-p}_{fruit}}\}\\ x_{fruit}^{{cross}^{\prime }}(k)=\{{x_{grow}^b(k)}_{p_{fruit}}//{x_{grow}^a(k)}_{{1-p}_{fruit}}\}\end{array}.$$

(35)

where ‘//’ denotes the concatenation of two strings.

Step 3-4: End Justification. The end justification may establish a percentage error threshold $e_{th}$ according to the fitness function $fit$ in Equation (30), the maximum number of iterations $k_{max}$, or the maximum solving period $T_{max}$. If any of the end conditions are met, the APC approach will terminate the iterative calculation and output the optimal solution $x_M^{\ast }\left(t\right)$. Otherwise, the fruits will be returned to Step 2 for the next iterative calculation until the end justification is fit.

Step 4: Deployment execution and status monitoring. These are to deploy microservices and request routing according to the optimal solution in step 3, start all services in the background, check service status, and view logs to troubleshoot issues.

4.2. The Flowchart

The flowchart of the APC-based solution mechanism is shown in Figure 2, where there are four main steps and a big loop to solve the joint optimization of microservice deployment and request routing. After environment preparation and dependency installation, service packaging and image orchestration, arrangement configuration and dependency management, deployment execution and status monitoring, the application will be constructed as a set of loosely coupled microservices. Figure 2 indicates only a large computational loop, where the APC will return and continue the next round of iterative calculations. In each loop, any APC individual can be calculated based on the population size, $P$, and each solution is encoded based on Equation (1) for joint optimization of microservice deployment and request routing. Since the loop in Figure 2 depends on the number of iterations, $k_{max}$, the computational complexity of the improved APC approach is $O\left(m_{max}\times k_{max}\times P\right)$, where $m_{max}$ is the number of microservices, $k_{max}$ is the maximum number of iterations, and $P$ is the population size.

4.3. The Pseudo-Code

For the challenges in joint optimization of microservice deployment and request routing, the pseudo-code is listed in Algorithm 1, which is based on the improved APC approach in Section 4.1. The proposed pseudo-code decreases computational complexity and makes it easier to be deployed on an edge server for fast calculation.

In Algorithm 1, the inputs include a set of microservices, $M=\{1,2,\dots ,m,\dots ,m_{max}\}$, a set of edge servers, $S=\{1,2,\dots ,s,\dots ,s_{max}\}$, the position of edge servers $s$ with coordinates $s(x_s,y_s)$, the actual request flows or traffic volumes between edge servers ($w_{{ss}^{\prime }}$), the computing capacity ($C_s^{comp}$) and storage capacity ($C_s^{stor}$) of edge server $s$, a discrete time slot set $T=\{1,2,\dots ,\dots ,t,\dots ,t_{max}\}$, $R_m^{comp}$ (in CPU cycles) for the computing resource required to process microservice $m$, $R_m^{stor}$ (in bits) for the storage capacity required to place microservice $m$, the reward function of $F\left(t\right)$ at time slot $t$, the discount factor of $\alpha$ to diminish the influence of future rewards on present decision-making, and the probability of $\beta$ that microservice $m$ is outsourced to the remote cloud.

The constraints meet all the requirements in Section 3.2.4, including Constraint 1 in Equation (19), Constraint 2 in Equation (20), Constraint 3 in Equation (21), Constraint 4 in Equation (22), Constraint 5 in Equation (23), Constraint 6 in Equation (24), Constraint 7 in Equation (25), Constraint 8 in Equation (26), Constraint 9 in Equation (27), Constraint 10 in Equation (28), and Constraint 11 in Equation (29).

The initialization parameters for an APC approach include the population size $P$, the seeding probability $p_{seed}$, the growing probability $p_{grow}$, and the fruiting probability $p_{fruit}$. The fitness function $fit$ of APC can select the nonlinear multi-objective optimization model in Equation (30).

In the optimal solution $x_M^{\ast }\left(t\right)=\left\{\right({\gamma }_{s,m}\left(t\right)\left|\right({\gamma }_{s,m}\left(t\right)=0 or 1,s\in S,m\in M\}$, ${\gamma }_{s,m}\left(t\right)$ indicates each edge server $s$ has two choices for each service $m$, where ${\gamma }_{s,m}\left(t\right)=1$ signifies that microservice $m$ is deployed on edge server $s$, and ${\gamma }_{s,m}\left(t\right)=0$ indicates microservice $m$ is not deployed on edge server $s$. Algorithm 1 will search for $x_M^{\ast }\left(t\right)=\left\{\right({\gamma }_{s,m}\left(t\right)\left|\right({\gamma }_{s,m}\left(t\right)=0 or 1,s\in S,m\in M\}$ to optimize structural symmetry and reduce the flow-aware routing distance, workload balancing, and request response delay.

Algorithm 1. APC-based microservice deployment and request routing algorithm

Input: $M=\{1,2,\dots ,m,\dots ,m_{max}\}$, $S=\{1,2,\dots ,s,\dots ,s_{max}\}$, $\left\{s\right(x_s,y_s\left)\right\}$, {$w_{{ss}^{\prime }}$}, $\left\{C_s^{comp}\right\}$, $\left\{C_s^{stor}\right\}$, $T=\{1,2,\dots ,\dots ,t,\dots ,t_{max}\}$, $\left\{R_m^{comp}\right\}$, $\left\{R_m^{stor}\right\}$, $F\left(t\right)$, $\alpha$, and $\beta$.

Constraints: Equations (19)–(29)

Set: $P$, $p_{seed}$, $p_{grow}$, $p_{fruit}$, and $fit$.

Output: The optimal solution $x_M^{\ast }\left(t\right)=\left\{{\gamma }_{s,m}^{\ast }\left(t\right)\right|s\in S,m\in M\}.$

1: Environment preparation and dependency installation

2: Service packaging and image orchestration

3: $d_{{ss}^{\prime }}=w_{ss\prime }\sqrt{{(x_s-x_{s^{\prime }})}^2+{(y_s-y_{s^{\prime }})}^2}$

4: ${\left\{x_{seed}\right(k=0\left)\right\}}_P={\left\{x_{seed}^{rand}\left(k\right)\right\}}_P=rand{\left\{{\gamma }_{s,m}\left(t\right)\right|s\in S,m\in M\}}_P$

5: Arrangement configuration and dependency management

6: for ${k=1:k}_{max}$

7:    $m_m^{\sum requ}\left(t\right)={\sum }_{s=1}^S{m}_{s,m}^{requ}\left(t\right),\forall m,\forall t$

8:    $R_{s,m}^{\sum comp}\left(t\right)={\gamma }_{s,m}^{comp}\left(t\right)L_{s,m}(t)R_m^{comp}{m}_m^{\sum requ}\left(t\right)$

9:    $R_{s,m}^{\sum stor}\left(t\right)={\gamma }_{s,m}^{comp}\left(t\right)L_{s,m}(t)R_m^{stor}{m}_{s,m}^{requ}\left(t\right)$

10:   $d_m^{avg}\left(t\right)={\sum }_{s=1}^S\left(d_{s,m}^{comp}\left(t\right)+d_{s,m}^{tran}\left(t\right)\right)+d_{c,m}^{proc}\left(t\right)$

11:   ${\left\{x_{seed}\right(k>0\left)\right\}}_P={\{x_{seed}^{rand}\left(k\right)p_{seed},x_{seed}^{fruit}(k-1)(1-p_{seed})\}}_P$

12:   ${obj}_1=\mathrm{m}\mathrm{i}\mathrm{n}\{{\displaystyle \frac{1}{{\sum }_{s,m}{\gamma }_{s,m}\left(t\right)}}{\sum }_{s,s\prime }{d}_{{ss}^{\prime }}\}$

13:   ${obj}_2=\left\{\min \{\underset{s,m}{\max } R_{s,m}^{\sum comp}\left(t\right)-\underset{s,m}{\min } R_{s,m}^{\sum comp}\left(t\right)\},\mathrm{m}\mathrm{i}\mathrm{n} \{\underset{s,m}{\max } R_{s,m}^{\sum stor}\left(t\right)-\underset{s,m}{\min } R_{s,m}^{\sum stor}\left(t\right)\}\right\}$

14:   ${obj}_3=min\left\{d_m^{avg}\left(t\right)+\beta \left(1-{\sum }_{s=1}^S{\gamma }_{s,m}\left(t\right)\right)\left(1-{\sum }_{s=1}^S{L}_{s,m}\left(t\right)\right){\sum }_{s=1}^S{R}_{s,m}^{\sum comp}\left(t\right)R_m^{comp}\right\}$

15:   $fit={obj}_{\sum }=\{{obj}_1,{obj}_2,{obj}_3\}$

16:   ${\{x_{grow}(k)\}}_{P{p}_{grow}}=\{x_{seed}(k)| \underset{P{p}_{\mathit{grow}}}{\max } fit(x_{seed}(k)){\}}_{P{p}_{grow}}$

17:   $x_{fruit}^{cross}(k)=\{{x_{grow}^a(k)}_{p_{fruit}}//{x_{grow}^b(k)}_{{1-p}_{fruit}}\}$

18:   ${\left\{x_{fruit}\right(k\left)\right\}}_{2P{p}_{grow}}=\left\{\{x_{grow}(k){\}}_{P{p}_{grow}},\{x_{fruit}^{cross}(k){\}}_{P{p}_{grow}}\right\}$

19:   $x^{*}\left(k\right)=\left\{{\gamma }_{s,m}\left(t\right)\right|\mathrm{m}\mathrm{a}\mathrm{x} \left\{fit\right(x\left(k\right)\left)\right\}\}$

20:   if $|fit{(x}^{\ast }\left(k\right)-fit{(x}^{\ast }\left(k-1\right)|\ge e_{th}$ then return to line 7

21: end for

22: Output the optimal solution $x_M^{\ast }\left(t\right)$

23: Deployment execution and status monitoring

In Algorithm 1, the end justification in line 20 is established by a percentage error threshold $e_{th}$ through the fitness function $fit$ in Equation (30), which can be replaced by other ending conditions, i.e., the maximum number of iterations $k_{max}$ or the maximum solving period $T_{max}$. The computational complexity of Algorithm 1 can be determined by the number of microservices ($m_{max}$), the population size ($P$), and the maximum number of iterations ($k_{max}$), i.e., $O\left(m_{max}\times k_{max}\times P\right)$.

5. Benchmark Test

This section develops a benchmark dataset and conducts a set of benchmark experiments to validate and compare our proposed method and some baseline algorithms.

5.1. Benchmark Dataset

The benchmark map with 100 edge devices is shown in Figure 3, where all edge devices are randomly generated as edge servers. In the benchmark map, the entire testing area was divided into a 1 km × 1 km square, and a large number of microservice requests were randomly distributed. Our task is to jointly optimize microservice deployment and request routing, systematically considering the structural symmetric metrics, i.e., the flow-aware routing distance, workload balancing, and request response delay.

According to Figure 3, the benchmark dataset and configuration for jointly optimizing microservice deployment and request routing are shown in

Table 2. Benchmark Dataset and Configuration.

Dataset	Configuration
$m_{max}$	100
$s_{max}$	100
$s(x_s,y_s)$	Rand [1, 100]
$w_{{ss}^{\prime }}$	1
$C_s^{comp}$	Rand [0.0%, 100.0%]
$C_s^{stor}$	Rand [0.0%, 100.0%]
$t_{max}$	1000 ms
$R_m^{comp}$	Rand [0.0%, 100.0%]
$R_m^{stor}$	Rand [0.0%, 100.0%]
$F\left(t\right)$	Rand [10.0%, 20.0%]
$\alpha$	Rand [5.0%, 10.0%]
$\beta$	Rand [0.0%, 10.0%]

. In the benchmark test, all tested algorithms should search for the optimal microservice placement location and device connection based on Table 2. Microservices should be deployed near access nodes, which helps to reduce network delay and improve data transmission efficiency. Request nodes should connect to the nearest edge server through access routes, ensuring that all nodes can quickly access microservice resources.

The benchmark experiments were run on an AMD A6-9225 RADEON R4, a 2.60 GHz CPU, 4.00 GB of RAM, a 64-bit Windows 10 operating system, and MATLAB R2021b simulation software.

5.2. Experimental Results

The solutions of the APC-based microservice deployment and request routing algorithm on 100 edge devices are depicted in Figure 4a–l, and they are conducted by Algorithm 1. In the APC algorithm, the parameters were set as follows: the seeding probability $p_{seed}=0.2$, the growing probability $p_{grow}=0.4$, the fruiting probability $p_{fruit}=0.7$, and the population size of APC is set to $P=200$. When microservice requirements undergo dynamic changes, the proposed APC algorithm can quickly divide large and complex system functions into a series of small, independent microservices based on dynamic requirements, which can be deployed, run, and expanded independently to meet the constantly changing microservice requirements. At the same time, the proposed APC approach can efficiently search for the optimal route to edge server $s$ and achieve structural symmetry, i.e., the flow-aware routing distance, workload balancing, and request response delay.

In Figure 4a–l, when microservice requests change, although the edge device position remains fixed, there is a significant change in routing, resulting in a change in edge server $s$ and structural symmetry. If microservice deployment cannot respond to such changes in a timely manner, it will result in longer flow-aware routing distances, load imbalance, and longer request response delays due to constrained capacity. As shown in Figure 4g, due to changes in service request nodes, the proposed APC algorithm calculates the shortest flow-aware routing distance as 8605.69 m, edge server $s=57$, which is the minimum value shown in Figure 4a–l. In the scenario shown in Figure 4e, the shortest flow-aware routing distance obtained by the APC algorithm is 8870.70 m, edge server $s=31$, which is the maximum value in Figure 4a–l when the service request node changes. Hence, the subtle changes in service request nodes can easily lead to significant changes in the structural symmetry of microservice deployment and request routing, thereby affecting indicators related to structural symmetry, including the flow-aware routing distance, workload balancing, and request response delay.

Unlike traditional metrics such as network centrality, the proposed definition attempts to measure the structural symmetry of microservice deployment and request routing from both spatial and temporal dimensions. It is worth noting that although shorter flow-aware routing distances can help reduce workload imbalance and request response delay in many cases, they are not entirely consistent because workload imbalance and request response delay are also affected by the local location of the requesting node and the symmetry of the request routing; that is to say, flow-aware routing distance, workload balancing, and request response delay can help us comprehensively examine the structural symmetry of microservice deployment and request routing from a perspective of spatial symmetry and temporal symmetry.

In order to explore the relationship and differences among the three objectives, statistical comparisons of microservice deployment and request routing are summarized in

Table 3. Statistical comparisons of microservice deployment and request routing.

	Flow-Aware Routing Distance (m)			Workload Balancing (%)			Request Response Delay (ms)
	Max	Avg	Min	Max	Avg	Min	Max	Avg	Min
$s=9$	8964.32	8718.55	8631.92	35.315%	18.583%	8.933%	21.93	9.38	1.71
$s=18$	9182.41	9030.58	8784.46	30.942%	17.026%	7.848%	19.79	7.79	2.34
$s=20$	9124.53	8926.38	8806.33	19.213%	9.830%	5.733%	12.45	3.20	1.57
$s=28$	8900.95	8729.51	8653.28	21.825%	11.477%	6.003%	16.06	6.36	3.77
$s=31$	9131.07	8945.09	8870.70	36.265%	19.671%	9.593%	23.77	10.15	1.45
$s=43$	9025.06	8863.74	8735.56	18.140%	9.175%	5.721%	15.29	6.11	4.14
$s=57$	8955.87	8800.28	8605.69	27.964%	11.583%	6.289%	14.46	4.11	1.90
$s=59$	8985.12	8876.18	8624.22	34.888%	17.407%	7.963%	20.58	8.44	1.55
$s=69$	9115.63	8969.28	8703.62	24.770%	10.946%	6.808%	18.32	6.75	4.39
$s=83$	9141.80	8926.60	8792.37	22.588%	10.516%	6.047%	15.12	4.29	2.51
$s=91$	9165.95	8939.39	8751.36	30.613%	13.871%	7.600%	17.64	5.39	2.11
$s=95$	8936.24	8762.39	8668.27	19.854%	10.356%	5.945%	15.85	4.84	3.19

, where the first line indicates the flow-aware routing distance, workload balancing, and request response delay, and the first column indicates the number of edge servers. After 20 experiments with dynamic microservice requests, the maximum, average, and minimum values of microservice deployment and request routing were calculated.

Table 3 shows the dynamic change process of microservice requests, as well as the proposed APC algorithm responding to microservice requests while maintaining the structural symmetry of microservice deployment and request routing. For the minimum total distance of 8605.69 m ($s=57$), there is a 3.079% improvement compared to the maximum distance of 8870.70 m ($s=31$); for the minimum workload balancing value of 5.721% ($s=43$), there is a 67.686% improvement compared to the maximum value of 9.593% ($s=31$); for the minimum request response delay of 1.45 ms ($s=31$), there is a 202.718% improvement compared to the maximum value of 4.39 ms ($s=69$). Therefore, the proposed APC algorithm can quickly deploy microservices on edge servers, achieving nearby allocation of computing resources, where microservices on edge servers can quickly respond to business requests, reduce end-to-end delay, and help with load balancing.

However, different request nodes have different routing structures and improvements to the three objectives due to their local locations. As indicated in Table 3, when edge server $s=$57, the APC algorithm obtains the minimum total distance of 8605.69 m; when edge server $s=$ 43, the APC algorithm obtains the minimum value of average workload balancing of 9.175%; when edge server $s=$ 20, the APC algorithm obtains the minimum value of average request response delay of 3.20 ms. Therefore, placing edge servers at the center of the network often leads to better performance, such as $s=20$, $s=43$, and $s=57$; conversely, servers placed at the edge of the network tend to achieve worse performance, such as $s=9$, $s=31$, and $s=69$.

To test the convergence performance of the improved APC algorithm, 100 scalability tests were conducted, as shown in Figure 5a,b. The parameters were set as follows: the seeding probability $p_{seed}=0.2$, the growing probability $p_{grow}=0.4$, the fruiting probability $p_{fruit}=0.7$, and the population size of APC $P=200$.

Figure 5a shows the fitness curve as the number of iterations increases from 200 to 2000, where the fitness values were normalized to dimensionless values in the [0, 1] interval for observation. It can be observed that the improved APC algorithm can provide a convergence guarantee or bound on solution quality for the proposed NP-hard multi-objective optimization problem. In Figure 5a, a larger number of iterations may lead to a better solution, but it also requires longer solving time and greater computational overhead. When the number of iterations is 1000, the optimal solution is already very stable, so this value is the trade-off between computational speed and global optimality.

Figure 5b displays the calculation time curve as the number of edge devices increases from 100 to 1000, where the parameters of the APC algorithm remain unchanged. It can be seen that the solving time of the APC algorithm increases linearly with the size of the problem. In Figure 5b, a larger number of edge devices may require longer solving time and greater computational overhead, but the solving performance of the APC algorithm did not show a sharp deterioration. It demonstrates that the proposed APC algorithm has good scalability.

5.3. Comparative Experiments

The comparative experiments selected some baseline algorithms, i.e., GA [4,12], DRL [5,21,22,28], PSO [13,27], ACO [14], and GWO [15]. The algorithm parameters in comparative experiments were set according to

Table 4. Algorithm parameters in comparative experiments.

Algorithm	Population Size	Other Parameters
APC	$P=200$ of the APC individuals	The seeding probability $p_{seed}=0.2$, the growing probability $p_{grow}=0.4$, the fruiting probability $p_{fruit}=0.7$.
GA [4,12]	$P=200$ of the chromosomes	The chromosome length Lind = 20, the crossover probability $px$ = 0.7, and the mutation probability pm = 0.01.
DRL [5,21,22,28]	$P=200$ of the neurons in each hidden layer	A state embedding network comprised a shared stack and multiple heads. This shared stack contained two hidden layers, and each head block contained 100 ReLU units followed by |v| + 1 units with zero-centered tanh activation.
PSO [13,27]	$P=200$ of the particles	The location limitation loc = 0.5, the speed limitation sp = [−0.5, 0.5], the self-learning factor c1 = 1.5, and the social learning factor c2 = 1.5.
ACO [14]	$P = 200$ the of ants	The importance of heuristic factors h = 5.0, the pheromone volatilization factor p = 0.1, and the pheromone importance phi = 1.0.
GWO [15]	$P=200$ of the gray wolves	Problem dimension dim = 2, and initial positions of the wolf leader (alpha), wolf deputy (beta), and wolf advisor (delta) pos = rand(dim) × 10−5.

.

Figure 6a–c display a comparison of the flow-aware routing distance, workload balancing, and request response delay under the improved APC approach, GA [4,12], DRL [5,21,22,28], PSO [13,27], ACO [14], and GWO [15].

Firstly, for a comparison of the flow-aware routing distance with 100 edge devices, there is an optimal route that results in the shortest flow-aware routing distance. Shorter flow-aware routing distances can help reduce the deployment cost of microservices and improve structural symmetry, but the results obtained by different algorithms show slight variations. Figure 6a demonstrates that the improved APC approach consistently outperforms the other approaches, with a maximum improvement by up to 3.217%. Although the DRL [5,21,22,28] algorithm is also good, it requires longer solving time.

Secondly, a lower workload balancing rate can measure the structural symmetry and service resilience of microservice deployment from another perspective, although it may not necessarily be consistent with shorter flow-aware routing distances. But it also indicates that multi-objective evaluation of microservice deployment and request routing is complex and necessary. In Figure 6b, the APC approach is superior to other algorithms with a maximum improvement of 15.759% benefiting from the enhanced global and local search abilities.

Thirdly, shorter request response delay means that microservice deployment can respond to user requests faster, which is related to shorter flow-aware routing distances but not entirely consistent. The deployment of microservices located at the edge of the network can easily result in longer response delays, and wireless delay may also affect the optimization results. In Figure 6c, the proposed APC approach can be effectively applied to minimize the request response delay with a maximum improvement of 15.655% over other algorithms because of its better search ability.

5.4. Discussions

This paper deeply explores the joint optimization of microservice deployment and request routing based on an improved APC algorithm.

Firstly, structural symmetry has a significant impact on the efficiency of microservice deployment and request routing, but it cannot be evaluated through a single metric. Our work first develops a multi-objective optimization model to jointly optimize microservice deployment and request routing, and our work systematically considers the spatial symmetry and temporal symmetry, i.e., the flow-aware routing distance, workload balancing, and request response delay. Figure 4a–l and Table 3 analyze the common background of microservice and the key role of edge computing, pointing out the challenges faced by dynamic microservice deployment, request routing, and resource allocation, such as limited resources, unstable environment, and variable demands.

Secondly, in response to these challenges, this paper detailed the APC-based microservice deployment and request routing algorithm, including the improvement of seeding, growing, and fruiting processes. Compared with the classic baseline algorithm, the calculation process of the APC algorithm is greatly simplified, using only three probability parameters to cover global search ability, local optimization ability, and fast convergence ability. It is more suitable for this joint optimization problem, especially under edge resource constraints. The experimental results in Figure 5a,b show that the microservice configuration strategy based on Algorithm 1 and Figure 2 provides an effective optimization method for request routing and resource allocation. Figure 6a–c show that the proposed algorithm has a strong potential and application value in the face of highly dynamic edge computing environments.

Thirdly, the proposed benchmark experiment in Figure 3 provides a good description of the dynamic process and synchronous optimization of microservice deployment. In Figure 4a–l and Figure 5a,b, the performance of the proposed algorithm under different microservice requests, dynamic topologies, and edge server computing capabilities was verified, showing that the APC algorithm can effectively improve microservice deployment and request routing. Our proposal is based on a time-varying network dynamic model, so it can support real-time adaptation, but the optimization calculation requires updating the input.

Lastly, in order to simplify the analysis, this article ignores some factors to provide a comprehensive deployment framework. Due to the limitations of the experimental conditions, our experiments are based on the simulation platform and are not deployed on the real edge computing platform. The convergence and scalability of this algorithm on real MEC platforms with the increasing number of microservices and edge servers, as well as the representativeness and generalizability of the proposed benchmark tests, will be the next step of research. The impact of a real server cluster on joint optimization of microservice deployment and request routing is extremely complex, requiring further research.

6. Conclusions

In large-scale mobile edge computing, microservice deployment and efficient routing are of great significance for improving system performance, optimizing resource allocation, and ensuring real-time requirements. Different from traditional technology, our solution constructed a constrained nonlinear multi-objective optimization model to jointly optimize microservice deployment and request routing, systematically considering the spatial symmetry and temporal symmetry metrics, i.e., the flow-aware routing distance, workload balancing, and request response delay. Then, a lightweight APC algorithm was developed to solve it, and a benchmark experiment was designed to compare with baseline algorithms. The experimental results revealed that the improved APC algorithm is efficient in solving the joint optimization problem of microservice deployment and request routing and is superior to some typical baseline algorithms, with an up to 3.217% improvement in the flow-aware routing distance, an up to 15.759% improvement in the workload balancing, and an up to 15.655% improvement in the request response delay. However, the joint optimization of microservice deployment and request routing is extremely challenging, where the flow-aware routing distance, workload balancing, and request response delay can jointly evaluate the structural symmetry, but they are not completely consistent.

Due to the limitations of the experiment, this work is simulated rather than deployed on a real server cluster. Therefore, the actual running time, memory, wireless delay, communication costs of edge hardware, variance- or entropy-based balancing, Pareto analysis or sensitivity study to tune β, and performance improvement under dynamic workload peaks or server failures may be overlooked. The proposed structural symmetry definition and evaluation metrics for microservice deployment may encounter inconsistencies or conflicts in real-world operations. In the real world, with the increasing number of microservices and edge servers, the scalability and convergence of the proposed algorithm need further testing. In future work, we plan to deploy the proposal on a real server cluster and consider more practical factors to address the aforementioned shortcomings and deficiencies.