Opportunities for Adapting Data Write Latency in Geo-Distributed Replicas of Multicloud Systems

Abstract

This paper proposes an AI-based approach to adapting the data write latency in multicloud systems (MCSs) that supports data consistency across geo-distributed replicas of cloud service providers (CSPs). The proposed approach allows for dynamically forming adaptation scenarios based on the proposed model of multi-criteria optimization of data write latency. The generated adaptation scenarios are aimed at maintaining the required data write latency under changes in the intensity of the incoming request flow and network transmission time between replicas in CSPs. To generate adaptation scenarios, the features of the algorithmic Latord method of data consistency, are used. To determine the threshold values and predict the external parameters affecting the data write latency, we propose using learning AI models. An artificial neural network is used to form rules for changing the parameters of the Latord method when the external operating conditions of MCSs change. The features of the Latord method that influence data write latency are demonstrated by the results of simulation experiments on three MCSs with different configurations. To confirm the effectiveness of the developed approach, an adaptation scenario was considered that allows reducing the data write latency by 13% when changing the standard deviation of network transmission time between DCs of MCS.

1. Introduction

Interest in multicloud systems (MCSs), which make it possible to create and manage copies of data—replicas—in independent geo-distributed databases or storage systems, has been steadily growing [1]. A distinctive feature of MCSs for cloud service customers is the ability to use resources from multiple CSPs, which allows clients to remain independent of the terms of cooperation with any single CSP [2]. In addition, the advantages of MCSs include flexible management of cloud service costs across multiple CSPs, whose data centers (DCs) can be located as close as possible to end users.

When organizing the functioning of MCSs, all issues related to coordinating and managing interactions between services and data hosted in different CSPs fall under the responsibility of cloud service customers [3]. The trade-off between data consistency in MCS replicas and application response latency, incurred during data writes and reads, represents a fundamental principle of any distributed system [4]. At the same time, the development of methods to ensure the relevance and consistency of data within the limits acceptable from the point of view of business logic in each replica of any CSP in a given MCS remains an unresolved scientific and practical problem.

Traditionally, in the literature, the advantages of a new method of data consistency or replication in cloud systems are confirmed by the result of a numerical comparison of write/read delays arising when using the proposed method and a reference method. In most cases, real experiments are conducted on the resources of one CSP, with explanations of the difficulties that may occur when using the obtained results for MCSs. This way of presenting data consistency methods does not allow for estimating the latency that may arise when implementing these methods in other cloud system configurations, especially in MCSs. This significantly reduces the likelihood of the practical use of the presented developments. In addition, it limits the possibility of a priori assessment or control of data write and read latency in the MCS required for switching to another data consistency method.

A reasonable assessment of switching to a new and faster data consistency method in MCSs requires long-term and expensive test implementation and validation, and the process of its implementation may lead to changes in the overall architecture or in the interaction between existing services. Thus, managing data write latency in replicas by switching to new data consistency methods may be difficult to implement in practice. A more realistic option, in this case, is the possibility of initially using such a data consistency method that allows managing data write latency in changing operating conditions for the MCS.

Our main contribution is an approach to adapting the data write latency in geo-distributed MCSs based on the use of AI models and internal parameters of the method Latord of data consistency, in replicas of different CSPs.

Unlike non-algorithmic data consistency methods in cloud systems, the interval parameters of the Latord method determine the data write latency in replicas. The proposed approach to adapting the data write latency in MCSs combines the interval parameters of data reconciliation in replicas, the significance vector of DCs in the MCS, the properties of global communication channels, and incoming request flows into a multi-criteria model.

The results of simulation experiments in three MCSs with different configurations demonstrate a significant effect of the parameters of data reconciliation on individual components of the data write latency, even with a 50-fold increase in the intensity of the incoming write request flow. Using the results of simulation experiments, we propose an example of the formation of adaptation scenarios, the use of which reduces the data write latency in replicas by approximately 13% when changing the properties of global communication channels between DCs of the MCS.

Features of the data consistency the Latord method which define the distinct components of data write latency, the multi-criteria model of the data write optimization, and the approach to the formation of adaptation scenarios on AI models, are presented in Section 3. Section 4 describes the results of simulated experiments. A scenario of the data write latency adaptation based on the experimental results is discussed in Section 5. Section 6 provides our conclusions and outlines the focus of our future work.

2. Related Work

In single-provider cloud systems, the problem of reducing data replication latency can be solved by choosing the optimal number and location of replicas to work with the selected volume of data [5,6]. In geo-distributed systems, when a client requests a read operation, long tail latency may occur, to reduce which a predictive replica selection strategy was developed in [7]. In [8], the authors proved that a machine learning (ML) method can be effective in automatic threshold-based mechanisms of data placement and replication activation.

To mitigate the well-known conflict between the cost and latency of read requests, the geo-distributed cloud storage GeoCol was proposed in [9]. GeoCol integrates AI-based methods for request partitioning and storage planning. GeoCol uses an ML technique to predict request latency and a reinforcement learning (RL) model to ensure that each sub-request is sent in parallel to the optimally located DC of the CSP.

In [10], based on a comparison of various principles for classifying data replication strategies, a conclusion was made about the advisability of using not only the response time to requests but also an assessment of the economic effect as the goal of replication in single-provider cloud systems.

In [11], the authors propose making decisions about the need to create or delete replicas in single-provider cloud systems based on the assessment of the response time to read requests and the costs that affect the profitability of the cloud provider. The paper’s approach to calculating the response time to read requests allocates the maintenance of an up-to-date list of replicas in the cloud system to the internal mechanisms of a database, which would be difficult for MCSs.

A dynamic replica placement strategy in MCSs based on predefined service level agreement (SLA) thresholds for the response time to read requests, cost, and data popularity is proposed in [12]. This approach can be very effective in achieving tenant performance objectives, given their economic constraints.

Data processing involves distributed execution and partitioning, even within a single-provider cloud system. SLA-defined limit request response time, as a parameter of the quality of service of one provider, can be used as an inter-region delay only within one CSP. If the query execution plan contains data from replicas in other CSPs, it is necessary to determine the data transfer time between different CSPs, which is not regulated by SLA.

The average backbone network latency between Azure datacenters [13] and the AWS regions’ latency matrix [14] demonstrates that there can be a tenfold difference in the average inter-region latency, even for single providers. The average network latency of packet transmission between replicas of different CPSs can differ significantly from the average inter-region latency in one CSP, which must be taken into account when calculating request response time in MCSs.

The strategy of data placement in DCs of various CSPs was analyzed in [15]. The authors demonstrated that placing data as close as possible to the location of data users reduces the response time to read requests by more than 20%, but in this case, it is necessary to replicate and check data consistency across all CSPs.

Various groups of methods were used to ensure data consistency in geo-distributed replicas belonging to different CSPs. The largest group comprised replication protocols that are variations of the Paxos protocol [16,17,18,19], differing in the way that a leader replica is chosen.

It is also possible to identify a group of replication protocols in which data consistency is achieved using timestamps and various methods for synchronizing physical and logical clocks [20,21,22]. However, the data reconciliation methods of this group are effective when applied to the resources of a single CSP, but they lose their advantages when implemented in the MCSs.

In [23], the VIP-Grab data replication protocol based on an algorithmic method of data consistency in MCSs was proposed. However, principles of the formation of a complete ordered sequence of numbers to data write requests in this protocol can be the cause of long response time to data write requests.

The above methods do not contain parameters whose change could affect the latency of data write and read requests in geo-distributed replicas of different CSPs of the MCS. At the same time, the interval method of ordering data write requests proposed in [24] contains such parameters.

AI models are used to address various problems that arise in cloud services: from adapting infrastructure to traffic times [25] to forecasting workload demand and dynamically managing multiple resources [26,27]. In [28], the authors developed a deep learning algorithm that predicts Round-Trip delay Time (RTT) and the availability of cloud services. Real-time live data from Amazon Web Services, Alibaba Cloud, and Tencent Cloud serve as historical data for the real-time analysis and prediction of RTT between the DCs of CSPs. The article discusses how the research results can help cloud service customers choose CSPs to create more efficient MCSs without reducing the data write latency in them.

As far as we know, AI models have not been applied to adapt data write delay based on the data consistency method in MCSs. Predictions of external factors for MCSs together with the use of our method to coordinate data write requests will allow the data write delay to be adapted to them in advance. We note that our work is novel to the best of our knowledge in that it demonstrates the feasibility of adapting data write latency in MCSs based on a query consistency method and AI models.

3. Materials and Methods

3.1. MCS Configuration

Our goal is to investigate the possibility of managing data write latency in geo-distributed replicas of different CSPs when there are changes in the external operating parameters of the MCS. The interaction between hardware and software resources (servers, virtual machines, databases, and services), as well as the methods of data replication within each CSP, is carried out by means of the CSP itself and can differ significantly, in principle, from each other. Meanwhile, interactions between CSPs are performed via identical instances of the middleware, located in a single DC of each CSP, referred to as a broker Br_x, x = 1… BrMax, BrMax—the number of brokers in an MCS. Thus, we consider an MCS as a network of geo-distributed brokers containing data replicas. Each broker serves as the geographical center of a virtual cluster of end users whose requests are directed to it. Figure 1, Figure 2 and Figure 3 show geographical maps of broker locations as red dots and their corresponding virtual user clusters as red ovals for the three MCSs whose parameters are used in this study.

• MCS A contains brokers in Brazil, South Africa, China, and Malaysia;
• MCS B contains brokers in India, Canada, Israel, and the United Kingdom;
• MCS C contains brokers in China, Japan, Malaysia, and South Korea.

The green lines in these figures depict the conditional connections between brokers, illustrating the distances between them.

3.2. General Principles of Interval-Based Coordination of the Data Write Requests with the Latord Method in Geo-Distributed Replicas

Each broker, upon receiving a data request from a user, appends a key packet PR(Br_x(to), Ngl(to)) [29] in the form of a tuple of elements (

Table 1. Elements of tuple PR(Br_x(to), Ngl(to)).

Element of Tuple	Description
Brx(from)	the source broker that received the data packet from the user
NWtemp(from)	the sequence number of the current packet among those received by the source broker Brx(from) within the availability window Wx
Ntemp(from)	the sequence number of the current packet among those received by the source broker Brx(from) within the residual window ∆Ix
Ngl(from)	the packet number in the common order sequence at the source broker Brx(from) at the time of key packet formation
I(from)	the number of the adjustment interval during which a packet from a user arrived at the source broker Brx(from)
NWtemp(to)	the sequence number of the current packet among those received by the current broker Brx(to) within the availability window Wx
Ntemp(to)	the sequence number of the current packet among those received by broker Brx(to) within the residual window ∆Ix
Ngl(to)	the current packet number in the common order sequence at broker Brx(to)
I(to)	the number of the adjustment interval during which the key packet was received by broker Brx(to)

) and forwards it to all other brokers via the TCP protocol.

At that moment, a broker Br_x(from) can be considered an active replication node. Upon receiving a key packet from another broker, a broker Br_x(to) is considered a passive node. Thus, each broker functions as both a leader and a follower in MCS replication at different points in time.

It is assumed that all passive nodes receive the packet sent by the active node only once during the specified time interval. In this case, the order in which key packets are received by each broker may not correspond to the order in which these packets were sent by the active node. As a result, each broker eventually receives copies of all requests handled by the MCS, but according to its own observed chronology, i.e., the order of receipt. The main task of the Latord method is to assign, in each broker, specific numbers to all packets (i.e., user requests and key packets), in accordance with the common order sequence within the established acceptable interval. These idempotently consistent numbers determine the order of data writes across all replicas, thereby defining the degree of data consistency in the MCS.

For packet identification by monotonically increasing natural numbers that form the common order sequence, the incoming stream at each broker is treated as a sequence of closed intervals I(to) with a fixed duration. Sorting packets within a closed interval, i.e., sorting a finite subset of packets from an otherwise infinite incoming packet stream, eliminates the problem of sorting elements of an infinite set [30]. Sorting the packet numbers within each interval creates, in each of them, a complete partially ordered set, in which, for any pair of numbers a and b, the condition a < b holds. The Latord method determines the minimum number Nfree(I_x(to)) of the first packet received by the broker in the current adjustment interval I_x(to) as one greater than the maximum packet number in the sequence formed in the previous interval I_x(to − 1). Such a linear alignment of numbers between adjacent intervals makes it possible to treat the tuple of ordered packet numbers from the incoming stream as a total order sequence.

The correct number of the current packet in a common order sequence is determined immediately upon its arrival at any broker. This means that, from all packets previously received by a given broker, a common order sequence has already been formed, in which the correct position for the current packet must be determined. The rules for determining the correct position, i.e., the number in a common order sequence, of the current packet are implemented in our Corr algorithm. In this case, inserting the current packet into the correct position within the sequence of numbers results in the adjustment of the numbers previously assigned to earlier packets, whereas adding the current packet with a new maximum number in the sequence does not.

Accordingly, the use of the Latord method for interval-based coordination of data write requests at each broker, for any packet received within the current interval, can be divided into two consecutive steps:

• Formation of the tuple Mlate of prior time intervals, whose packets may be adjusted, and the tuple Mcor of packet numbers previously received by this broker from Mlate;
• Sorting the packet numbers from Mcor, with the consideration of the parameters of the currently received packet, using the Corr algorithm.

3.2.1. Formation of the Mlate Time Interval Tuple and the Mcor Packet Tuple

In the MCS, global network channels are used for packet transmission, both between users and brokers and among brokers themselves, with uncontrolled fluctuations in signal transmission speed. This implies that all time intervals describing packet transmission should be treated as random variables.

For each broker Br_x, the interval I_x, in which packet numbers are sorted, is represented as a tuple consisting of the availability window W_x and the residual window ∆I_x. The duration of the sorting interval equals the sum of the corresponding windows [24]:

$$I_x=W_x+\Delta I_x.$$

(1)

The duration of the availability window W_x depends on the area of the notional user cluster of the current broker Br_x and can be estimated numerically as the average time required for a request to reach the corresponding nearest broker [24]. This implies that for a notional cluster of active MCS users distributed over a larger geographical area, the availability window must be larger than for a cluster with a more compact distribution of active users. The duration of the residual window ∆I_x should be no less than the average time of packet transmission between the current broker and the most distant broker within the given MCS [24].

The sorting interval I_x differs across brokers; therefore, a single value, referred to as the adjustment interval I, is selected for all brokers to synchronize the packet coordination process in the MCS [24]:

$$I=\underset{x=1\dots BrMax}{\max }{I}_x.$$

(2)

Thus, the time axis is represented as a sequence of non-overlapping adjustment intervals I, consecutively numbered with natural numbers. Meanwhile, the adjustment interval I at each broker preserves its structure, remaining a tuple consisting of the availability window W_x, defined for each broker, and the residual window ∆I_x.

Using a global network to transmit packets between brokers may result in a key packet being sent from interval I(from) and arriving at another broker after the end of the same interval I(from). In other words, a packet may “arrive late” relative to the end of the interval from which it was sent and will be received during the following adjustment intervals I(to). We will consider it a delay when a single key packet contains I(to) > I(from). The maximum allowable packet delay, MaxLate, is determined by the following relationship [24]:

$$MaxLate=K_{Late}\cdot I,$$

(3)

where K_Late is a positive integer. For example, if a packet is sent by a broker from window W_x of interval I(to), and MaxLate equals 2⸱I, then the latest, i.e., the final, window in which this packet can be received by another broker will be W_x of interval I(to + 2).

The value of MaxLate is used in the Latord method to determine the order of packet transmission without relying on timestamps. Knowing the window type and the interval number of the current packet’s arrival, for example, ∆I_x from I(to), and assuming that the packet is maximally delayed, we consider the source of this packet to be the corresponding window ∆I_x of interval I(to−K_Late). If this assumption is correct, the number of the current packet should be smaller than the number of all previously received packets that were sent by other brokers after the current packet was sent. In other words, the number of the current packet should be smaller than the number of previously received packets that were sent from the tuple of windows Mlate, ranging from ∆I_x of interval I(to−K_Late) to ∆I_x of interval I(to). Such a tuple of windows, Mlate, will contain 2·K_Late + 1 windows, while the tuple Mcor will include all packets that have already arrived at the current broker from these 2·K_Late + 1 windows. Representing the arrays Mlate and Mcor as tuples emphasizes the need to arrange both the windows and the packets within them in ascending order of their previously assigned sequence numbers. The hypothesis regarding the transmission order of the current packet among the packets in the tuple Mcor is verified according to the rules of the Corr sorting algorithm.

Let us consider an example of the formation of the tuples Mlate and Mcor in the MCS consisting of three brokers, for which the maximum allowable packet delay MaxLate is 2·I. Let the current packet, marked in red in Figure 4, have arrived at broker Br₂ during availability window ∆I₂ of adjustment interval I(57).

For such a packet, the Mlate window tuple contains 5 elements:

$$Mlate=〈\Delta I_2(55),W_2(56),\Delta I_2(56),W_2(57),\Delta I_2(57)〉,$$

(4)

while the Mcor packet tuple contains 6 packets previously received by Br₂, marked in green in Figure 4. The Mcor tuple contains both packets sent by other brokers from the windows of the Mlate tuple and packets received from users during the same windows.

3.2.2. The Corr Algorithm for Determining the Correct Packet Number in the Common Order Sequence of Numbers

The sorting conditions according to the Corr algorithm are checked between the currently received packet and each packet in the Mcor tuple by iterating through all elements of Mcor sequentially, starting from the first.

In the Corr algorithm, the conditions for the pairwise agreement of packet numbers correspond not to the actual chronology of packet arrivals at the brokers, but rather to the chronology of the windows in the Mlate tuple. This implies that all packets sent from earlier windows must be assigned smaller numbers in the sequence than those sent from later windows. For requests received from users in a key packet, the broker that received the request will be considered the source broker of the packet; that is, Br_x(from) and Br_x(to) in the key packet will be equal.

When comparing packets sent from the same window of a given adjustment interval but originating from different brokers, the packet from the broker with a higher priority is assigned the smaller sequence number. A broker priority corresponds to the duration of its availability window W_x: a broker with a longer W_x is assigned a higher service priority than a broker with a shorter W_x [24]. This allows packets sent by users who are theoretically farther from their broker in larger clusters to be “moved ahead” of packets sent by users who are closer to the broker in smaller clusters.

When comparing packets sent by the same broker from a single window W_x or ∆I_x, the packet sent earlier—that is, the packet with the smaller number N^Wtemp(from) or Ntemp(from), respectively—receives the smaller sequence number.

3.3. Data Write Latency in the MCS Implementing the Latord Method

The maximum allowable interval, MaxLate, for packet delivery between brokers effectively defines the period during which the number of the current packet can be adjusted according to the parameters of subsequent packets. After 2·K_Late subsequent windows have passed, the current packet will no longer be included in the Mcor tuples created for sorting new packets. This indicates that after 2·K_Late subsequent windows have elapsed, the packet number Ngl(to) in the total order sequence can be considered finally adjusted and used to establish the order of writing the corresponding user data to all replicas. We will consider LatencyWr(Ngl(to)) as the delay in writing data to the replica—the time interval between the moment it becomes impossible to adjust the Ngl(to) number of the packet and the moment the corresponding request from the user reaches the active broker Br_x(from):

$$LatencyWr\left(Ngl\left(to\right)\right)=\Delta r\left(Ngl\left(to\right)\right)+MaxLate=\Delta r\left(Ngl\left(to\right)\right)+K_{Late}\cdot I,$$

(5)

where ∆r(Ngl(to)) is the time interval from the moment a user request arrives until the end of the precise incoming window—W_x or ∆I_x. Thus, the latency LatencyWr(Ngl(to)) of writing data when using the Latord method is determined by the maximum allowable packet delay MaxLate, which depends on the adjustment interval I, as well as the proportion of the window W_x to the window ∆I_x in each broker. The component ∆r(Ngl(to)) contributes to the random nature of data write latency. Moreover, the procedure for writing data from a packet with the number Ngl(to) into each replica must begin immediately after the end of the interval defined by expression (5), provided that the packet with the number Ngl(to)−1 has already been written onto the current replica. In each replica, the difference between the moment of receiving a packet with the number Ngl(to) and the start of the procedure for writing data from it, even if the packet Ngl(to) − 1 has already been written earlier, is the SLA-limited available time of the corresponding resource. Available time, as a parameter of the quality of service of the database’s request response time, is included in the SLA of each CSP, which makes up the MCS and requires the presence of mechanisms for automatic verification of this productivity metric [31,32]. Confirmation of the quality of service guaranteed in the SLA is a well-known non-trivial task for MCSs [33,34] and does not depend on the method of data reconciliation in the replicas.

3.3.1. Parameters of the Latord Method for Assigning Data Write Latency in the MCS

We will consider the latency Latency_x(Ngl(to)) of forming the correct number Ngl(to) of the packet at broker Br_x as the interval between the moment $t_x^{cor}\left(Ngl(to)\right)$ of the last adjustment of the packet number Ngl(to) and the moment $t_x^{in}\left(Ngl(to)\right)$ this packet arrives at Br_x:

$$Latenc{y}_x\left(Ngl(to)\right)=t_x^{cor}\left(Ngl(to)\right)-t_x^{in}\left(Ngl(to)\right).$$

(6)

The number of sorts for each packet in any broker equals the number of packets that arrive at that broker after the current packet within the current window and in the subsequent 2·K_Late windows. Considering that not every sorting operation may lead to an adjustment of a previously assigned packet number, the moment $t_x^{cor}\left(Ngl(to)\right)$ of the last adjustment of the number Ngl(to) is a random variable. The moment $t_x^{in}$ when a key packet arrives at the current broker Br_x, as well as the moment $t_x^{in}$ when a request arrives from a user, is a random variable and depends on the geographical location of the brokers. Consequently, in expression (6), both terms are determined by random events, and Latency_x(Ngl(to)) should also be considered a random variable, whose parameters are defined by the arrangement of brokers in the MCS.

Let us consider the principle of forming the latency LatencyWr(Ngl(to)) of writing data and the latency Latency_x(Ngl(to)) of forming the correct number Ngl(to) of the packet on the MCS, as illustrated in the example in Figure 4. Suppose a user request arrives at broker Br₁ at moment $t_1^{in}$ during the availability window W₁ of adjustment interval I(14) (Figure 5).

According to the chosen MaxLate from expression (5), writing a packet to all replicas can begin at t^wr(Ngl(to)), i.e., through $2\cdot I$ after the completion of W₁ from I(14). Therefore, the data write latency for the current packet to all replicas will be equal to

$$LatencyWr\left(Ngl(to)\right)=t^{wr}\left(Ngl(to)\right)-t_1^{in}=\Delta r\left(Ngl(to)\right)+MaxLate=\Delta r\left(Ngl(to)\right)+2\cdot I.$$

(7)

Let us assume that the key packet corresponding to the user’s input request arrived at the other brokers at moments $t_2^{in}$ and $t_3^{in}$. Suppose that the number of sorting procedures for the packet number Ngl(to) in the brokers was different, so that the adjustments were performed independently and completed at different moments $t_x^{cor}\left(Ngl\left(to\right)\right)$, x = 1 … 3. Then, the numerical values of the latencies Latency_x(Ngl(to)) in generating the correct number Ngl(to) of the packet in the brokers differ and equal, accordingly, as follows:

$$\begin{array}{l}Latenc{y}_1\left(Ngl\left(to\right)\right)=t_1^{cor}\left(Ngl\left(to\right)\right)-t_1^{in},\\ Latenc{y}_2\left(Ngl\left(to\right)\right)=t_2^{cor}\left(Ngl\left(to\right)\right)-t_2^{in},\\ Latenc{y}_3\left(Ngl\left(to\right)\right)=t_3^{cor}\left(Ngl\left(to\right)\right)-t_3^{in}.\end{array}$$

(8)

Generally, the data write latency LatencyWr(Ngl(to)) can be expressed using the varying values of the latency Latency_x(Ngl(to)) for generating the packet number Ngl(to) at different brokers:

$$LatencyWr\left(Ngl\left(to\right)\right)=\Delta t_{from-to}^{tr}\left(Ngl\left(to\right)\right)+Latenc{y}_x\left(Ngl\left(to\right)\right)+{\tau }_x,x=1\dots BrMax,$$

(9)

where $\Delta t_{from-to}^{tr}\left(Ngl\left(to\right)\right)=t_{to}^{in}\left(Ngl\left(to\right)\right)-t_{from}^{in}\left(Ngl\left(to\right)\right)$ is the packet Ngl(to) transmission duration from the active broker Br(from) to the current broker Br(to), and τ_x is the interval between the data write moment t^wr(Ngl(to)) and the moment $t_x^{cor}\left(Ngl\left(to\right)\right)$ of the last adjustments of the packet Ngl(to). It is assumed that for a user request at broker Br₁ (Figure 5),

$$\Delta t_{from-to}^{tr}\left(Ngl\left(to\right)\right)=\Delta t_{1-1}^{tr}\left(Ngl\left(to\right)\right)=0.$$

(10)

The interval τ_x is an excessive ballast random component of the data write latency LatencyWr(Ngl(to)), since not all brokers perform the adjustment of the packet number N_gl(to) throughout the entire interval from $t_x^{in}$ to t^wr(N_gl(to)). Therefore, changes in the data recording delay, expressed by Formula (9), can be achieved by changing the duration of packet transmission between brokers $\Delta t_{from-to}^{tr}\left(Ngl\left(to\right)\right)$ and Latency_x(Ngl(to)) of the formation of the correct packet number Ngl( to) in each broker.

3.3.2. Parameters That Determine the Latency of Forming the Correct Packet Number in Brokers

The number of sorts for the packet number Ngl(to) at each broker is determined by the number of incoming packets during the fixed interval MaxLate and, therefore, depends on the intensity of the incoming user request stream UtoBr_x at each broker. In turn, the number of sorts for the packet number in the interval from $t_x^{in}$ to t^wr(Ngl(to)) affects the latency Latency_x(Ngl(to)) when generating the Ngl(to) number of this packet. The closer these moments are to each other, the smaller Latency_x(Ngl(to)) becomes. The position of t^wr(Ngl(to)) on the time axis depends on the a priori chosen MaxLate value; so, the latency Latency_x(Ngl(to)) is determined by the duration of the adjustment interval I.

The initial selection of the adjustment interval I, as well as subsequent modifications of its value during the operation of the MCS, leads to the problem of revising the relationship between the W_x and ∆I_x windows in each broker, i.e., the problem of secondary selection of these windows within I for all brokers.

It is also possible that an inverse case, when revising the window durations, will lead to a change in the adjustment interval I, thereby affecting both the latency Latency_x(Ngl(to)) and the data write latency.

It should be noted that difficulties in synchronizing clocks between the devices of all active MCS users and the brokers of the corresponding clusters may turn the selection of W_x based on real measurements into an empirical problem requiring ad-hoc solutions. Meanwhile, the packet transmission time between brokers, as a random variable, has a minimum limit determined by the geographical locations of the corresponding DCs of CSPs and can serve as a starting point for the secondary selection of windows. Moreover, packet transport between brokers is strongly affected by fluctuations in Internet traffic and global communication channel parameters.

Thus, the latency of generating packet numbers in each broker and write data latency are influenced by the set of internal parameters of the Latord method:

$$U\left(t\right)=\left\{W_x\left(t\right),\Delta I_x\left(t\right),I_x\left(t\right),MaxLate\right\},x=1\dots BrMax,$$

(11)

and by external parameters—the intensity of the incoming user request flow UtoBr and the duration of packet transmission $\Delta t_{from-to}^{tr}$ between brokers of the MCS.

3.4. Objective Function for Optimizing Data Write Latency in the MCS

This study assumes that information about the forecasts of fluctuations in external operating parameters of the MCS and their influence on the latency of generating packet numbers will make it possible to create a mechanism for adapting the data write latency to such fluctuations.

We define the optimal data write latency in an MCS as the set Lat(t), in which the minimum latency may be achieved not across all brokers, but only within the most significant ones. The numerical significance values Signif_x of a broker can be determined based on the number of active users or the business value of the corresponding user cluster and can be represented as a vector $\left[Signi{f}_x\right]$. Hence, the data write latency is the set of elements:

$$Lat(t)=\left\{LatencyW{r}_x\left(UtoB{r}_x(t),\Delta t_{from-to}^{tr}\left(t\right),t\right) \left| \left[Signi{f}_x\right]\right.\right\},x=1\dots BrMax .$$

(12)

Expression (11) defines a set of internal parameters whose changes at time t can lead to an optimal value of data write delay in each broker.

Expressions (1) and (2) serve as constraints on the adjustment interval, similar to the probability P(K_x) of receiving packets by a passive broker after the end of the MaxLate period:

$$\mathrm{P}\left(K_x\right)\le U{p}_x , x=1\dots BrMax,$$

(13)

where Up_x denotes the maximum allowable probability of the occurrence of “late” packets in each broker Br_x. The initial selection of the residual window ∆I_x(t₀) for each broker must satisfy the following condition:

$$\Delta I_x(t_0)\le \max mean\left(\Delta t_{x-to}^{tr}\right),x=1\dots BrMax .$$

(14)

Accordingly, the objective function for the multi-objective optimization of data write latency in the MCS can be defined as:

$$\begin{array}{l}\min Lat\left(t\right),\\ \left\{\begin{array}{l}U\left(t\right)=\left\{W_x\left(t\right),\Delta I_x\left(t\right),I_x\left(t\right),MaxLate\right\}\\ \Delta I_x(t_0)\le \max mean\left(\Delta t_{x-to}^{tr}\right)\\ I(t_0)=\underset{x=1\dots BrMax}{\max }{I}_x(t_0).\\ I_x\left(t\right)=W_x\left(t\right)+\Delta I_x\left(t\right)\\ \mathrm{P}\left(K_x\right)\le U{p}_x\\ x=1\dots BrMax\end{array}\right.\end{array}$$

(15)

Any MCS can be considered a dynamic system subject to unstable external influences, including flows of incoming user requests and random disturbances on global communication channels. Fluctuations in the external parameters of UtoBr and $\Delta t_{from-to}^{tr}$ may be caused by reasons independent of the MCS operation. The level of these fluctuations may be related to the day of the week, season, fashion, and special events in one or more user clusters.

3.5. Formation of Adaptation Scenarios of Data Write Latency in the MCS

To respond promptly to incoming request flow fluctuations, reliable forecasts of such changes are required. For this purpose, AI technologies employing machine learning for real-time data processing can be utilized. AI models should provide dynamic predictions of changes in the incoming request flow for each broker separately, based on which decisions regarding the feasibility and magnitude of adjustments in U(t) will be made. At the same time, to make a decision on how to change the elements of the set U(t) to correct changes in UtoBr, additional relevant data are required on:

• What magnitude of change in the incoming request flow rate should be considered significant for triggering adjustments;
• Which changes and in which broker, given the current configuration of variations in incoming request flow intensity, should be used to initiate adjustments? For example, consider a situation where, in the broker with the highest significance, a significant increase in input flow intensity is predicted, but it is twice as small as a predicted significant increase for the same time in a broker with lower significance. In such a case, it is not necessary to make an additional decision regarding which of the two significant predicted values of U(t) should be applied;
• What duration of predicted significant changes in the intensity of the incoming request flow should be considered sufficient to trigger adjustments;
• What magnitude of changes in user activity should prompt a revision of the significance vector $\left[Signi{f}_x\right]$?

Such data constitute a set of initial constraints or minimum gradients Y(t), which, when exceeded, trigger decision-making procedures regarding changes in U(t). The elements of Y(t) may also vary depending on current user activity trends, requiring decisions regarding their determination based on AI models employing machine learning (ML) techniques.

A similar situation occurs with the second external component, the random nature of which affects the data write latency—$\Delta t_{from-to}^{tr}$. In the conducted simulation experiments, it was assumed that the packet transmission time between brokers can be characterized by a single probability distribution law and a single standard deviation (SD) value. In this case, the SD of $\Delta t_{from-to}^{tr}$ was adjusted to the same level simultaneously across all brokers. Under real conditions, changes in the mean value and SD of $\Delta t_{from-to}^{tr}$ will occur between one or several pairs of brokers.

Changes in the parameters of the random variable $\Delta t_{from-to}^{tr}$ can be predicted if their causes are associated with variations in user activity within clusters with limited communication line capacities, rather than with emergency events. To make decisions regarding adjustments to the values of U(t), which compensate for changes in packet transmission durations between brokers, additional up-to-date data are required on:

• What magnitude of change in the average and SD of $\Delta t_{from-to}^{tr}$ should be considered significant for triggering adjustments;
• What duration of predicted significant changes in the average and SD of $\Delta t_{from-to}^{tr}$ should be considered sufficient to trigger adjustments.

These data also form part of the set of gradients Y(t).

Thus, to determine the magnitude and specific elements of U(t) from Equation (11), which should be adjusted, it is necessary to employ an AI model for the real-time prediction of changes in the incoming request traffic and parameters of global communication channels. The output of this AI model will serve as the input for another tool, the AI classification model, which is responsible for categorizing current gradients Y(t). In this model, simulation methods or neural networks trained on historical data from a specific MCS can be employed to determine the minimum gradients within the set Y(t).

To solve the optimization problem described by expression (15), it is not enough to use simple algorithms based on “if–then” instructions [35]. The set of decisions regarding changes in U(t) will be called a scenario for adapting the data write latency. Adaptation scenarios are customized for each MCS, depending on the specific conditions of its operation. To form effective scenarios for adapting the data recording delay, we can use a neural network model, the input data for which are the output data of the previously described models. The interaction diagram among these three AI models is shown in Figure 6.

4. Experimental Setup and Results

The application of the reasonably selected parameters of the Latord method of data consistency, namely, I, W_x, and ∆I_x, will allow for adapting the data write latency LatencyWr(Ngl(to)) to changes in the external parameters UtoBr and $\Delta t_{from-to}^{tr}$. In accordance with expressions (15), (12), and (9), the formation of scenarios for adapting the data write latency in MCSs is based on the influence of the elements of the set U(t) on the latency Latency_x(Ngl(to)) at different values of the external parameters UtoBr and $\Delta t_{from-to}^{tr}$. To confirm this hypothesis, two groups of simulation experiments were carried out.

The sections above show that the latency Latency_x(Ngl(to)) of forming the correct packet number as a separate term in determining the data write latency LatencyWr(Ngl(to)) according to Formula (9) depends both on the adjustment interval I and on the ratio of the windows W_x and ∆I_x within it, as given in expression (1). The study of the effect of decreasing I on the latency Latency_x(Ngl(to)) of the packet number formation can be carried out in two ways:

• By decreasing the residual window ∆I_x with a fixed availability window W_x. This type of change in the adjustment interval I was used in the experiments of the “var∆I” group;
• By decreasing the availability window W_x with a fixed residual window ∆I_x. This type of change in the adjustment interval I was used in the experiments of the “varW” group.

4.1. MCS Configurations for Experiments

To study the influence of the duration of packet transmission between brokers on the delay Latency_x(Ngl(to)), independently of other terms that determine the data write latency LatencyWr(Ngl(to)) according to expression (9), simulation experiments were conducted in three MCSs: A, B, and C.

Each MCS consisted of four brokers (BrMax = 4), whose geographical locations are shown in Figure 1, Figure 2 and Figure 3 and remain unchanged throughout this study. Based on the defined user clusters, with boundaries marked by red ovals in Figure 1, Figure 2 and Figure 3, the availability window values were selected for each MCS (

Table 2. Availability windows for the MCS brokers A, B, and C.

MCS	Availability Window Wx (ms)
	W1	W2	W3	W4
A	169.87	69.77	21.75	10.94
B	35.09	27.34	11.09	10.73
C	20.00	14.00	12.00	10.00

).

Studies [36,37,38] show that the characteristics of the duration of packet transmission on the Internet depend not only on the distance between the final destinations of the packets but also on the region, country, continent, and Internet user density. The duration of packet transmission over global communication channels in any region of the globe can be described as a random variable that takes on all possible discrete values in a certain range with equal probability. So, in simulation experiments, the packet transmission time $\Delta t_{from-to}^{tr}$ between all pairs of brokers in all MCSs was considered a random variable with a uniform probability distribution. The selection of numerical values of the average $\Delta t_{from-to}^{tr}$ between brokers in MCSs A, B, and C (

Table 3. Mean of packet transmission time between brokers in MCS A, B, and C.

MCS	Brokers	Average of Packet Transmission Between Brokers (ms)
		Br1	Br2	Br3	Br4
A	Br1	4.83	339.73	319.73	334.88
	Br2	339.45	4.39	242.35	279.81
	Br3	318.30	241.30	3.43	43.73
	Br4	337.13	279.08	42.69	3.96
B	Br1	8.50	221.36	167.63	138.79
	Br2	226.44	5.11	159.44	92.95
	Br3	175.47	155.75	4.61	71.49
	Br4	139.76	91.11	73.95	4.30
C	Br1	3.43	49.51	43.73	39.94
	Br2	51.01	4.82	84.57	29.60
	Br3	44.75	85.85	3.96	80.65
	Br4	41.78	27.51	79.98	5.59

) was carried out by taking into account the average latency between Azure DCs on the backbone network [13] and the AWS region latency matrix [14].

The average values of $\Delta t_{from-to}^{tr}$ used in all simulation experiments were kept constant.

The long-tailed Pareto distribution, as a special case of the power-law distribution, is widely used in economics, finance, computer science, and the social sciences to describe time intervals between random events [39,40,41]. So, in simulation experiments, the inter-arrival times of user requests were modeled as independent random variable streams, following a Pareto distribution, with equal intensity across all clusters within each MCS. In this case, the average values of the incoming request flow intensity UtoBr arriving at each broker were 20.00, 50.00, 100.00, and 500.00 requests per second.

For each MCS, simulation experiments of the “var∆I” and “varW” groups were conducted, as described below.

All experiments were implemented using MATLAB R2023b, a platform for computation, data analysis, and visualization.

4.2. Group of Experiments “var∆I”

In the experiments of the “var∆I” group, the effect of changing the adjustment interval I on the latency of forming the correct packet number in each broker of each MCS was investigated. In addition, different intensities of UtoBr were used to study the adjustment interval I as the independent term determining the data write latency according to expression (9). In this case, the changes in I were performed by changing the term ∆I_x with a fixed W_x. This situation may arise when the parameters of packet transmission between brokers and from users to brokers, i.e., $\Delta t_{from-to}^{tr}$ and W_x, are defined and considered stable, and the task is to select the optimal I, at which the data write latency will have the required value.

4.2.1. Setup of Experiments “var∆I“ in the MCSs

A distinctive feature of the experiments of the “var∆I“ group is that the values of the availability windows W_x for brokers of each MCS, given in Table 2, did not change. At the same time, the values of the adjustment interval I_x, x = 1 … 17, changed into a wide range of values, the boundary values of which are given in

Table 4. Range of changes of the adjustment interval I_x (ms) for the experiments “var∆I” in MCS A, B, and C.

MCS	Range of Changes of Ix (ms)
A	[169.900 … 489.900]
B	[113.250 … 265.250]
C	[53.125 … 108.525]

.

The changes in the duration of the packet transmission between brokers in the simulation experiments were achieved by using different values of its SD with constant average values (Table 3). For each MCS, two SD variants were used: variant SD1 with basic values and variant SD2 with values that were at least 10 times higher than the values of variant SD1 (

Table 5. SD values of the duration $\Delta t_{from-to}^{tr}$ of packet transmission between brokers in the MCSs.

MCS	Brokers	Standard Deviation SD1				Standard Deviation SD2
		Br1	Br2	Br3	Br4	Br1	Br2	Br3	Br4
A	Br1	0	3.9229	3.6919	3.8669	0	49.0358	46.1491	48.3358
	Br2	3.9196	0	2.7984	3.2310	48.9954	0	34.9802	40.3871
	Br3	3.6754	2.7863	0	0.5050	45.9426	34.8287	0	6.3119
	Br4	3.8928	3.2225	0.4929	0	48.6605	40.2817	6.1618	0
B	Br1	0	2.5560	1.9356	1.6026	0	31.9506	24.1953	20.0326
	Br2	2.6147	0	1.8411	1.0733	32.6838	0	23.0132	13.4162
	Br3	2.0262	1.7984	0	0.8255	25.3269	22.4806	0	10.3187
	Br4	1.6138	1.0520	0.8539	0	20.1726	13.1506	10.6738	0
C	Br1	0	0.5717	0.5050	0.4612	0	7.1462	6.3119	5.7648
	Br2	0.5890	0	0.9765	0.3418	7.3627	0	12.2066	4.2724
	Br3	0.5167	0.9913	0	0.9313	6.4591	12.3914	0	11.6408
	Br4	0.4824	0.3177	0.9235	0	6.0304	3.9707	11.5441	0

).

In each experiment, the incoming flow to each broker consisted of 10,000 user requests for data write.

4.2.2. Results of Experiments “var∆I”

The mean of the packet number formation latencies with SD2 in each broker of MCS A, B, and C form the upper surface in Figure 7, Figure 8 and Figure 9, whereas with SD1, they form the lower surface.

Figure 7, Figure 8 and Figure 9 show that within each MCS, the mean of Latency_x(Ngl(to)) can vary significantly with changes in the adjustment interval I, due to variations in the residual window ∆I_x, even when the intensity of the incoming request flow UtoBr remains constant. Moreover, the magnitude of these changes is not uniform, even within a single MCS. For example, with SD1 and UtoBr = 20 req/s:

• In MCS A, an increase of 188% in the adjustment interval I resulted in an increase in Latency_x(Ngl(to)) of 42% in Br₁, 136% in Br₂, 100% in Br₃, and 88% in Br₄;
• In MCS B, an increase of 134% in I led to an increase in Latency_x(Ngl(to)) of 14% in Br₁, 50% in Br₂, 65% in Br₃, and 84% in Br₄;
• In MCS C, an increase of 104% in I caused an increase in Latency_x(Ngl(to)) of 14% in Br₁, 31% in Br₂, 42% in Br₃, and 40% in Br₄.

The observed differences in the absolute values of Latency_x(Ngl(to)) among brokers within each MCS are explained by the mutual influence of the random duration of the packet transmission between brokers. Thus, the obtained results indicate that the varying interval I relative to the residual window ∆I_x is a significant factor in developing scenarios for adapting the data write latency.

The general conclusion from the results in Figure 7, Figure 8 and Figure 9 is that there exists a positive correlation between the SD of $\Delta t_{from-to}^{tr}$ and the mean of the latency of the formation of the correct packet number. This implies that, for example, an increase in the SD of $\Delta t_{from-to}^{tr}$ at a fixed average of $\Delta t_{from-to}^{tr}$ leads to an increase in Latency_x(Ngl(to)). Therefore, when generating adaptation scenarios for data write latency, it becomes necessary to revise W_x, ∆I_x, and I. It should be noted that when changing the average of $\Delta t_{from-to}^{tr}$, new experiments must be conducted to estimate the average of latency of the formation of the correct packet numbers and the selection of the optimal interval I with the subsequent selection of the windows W_x and ∆I_x in each broker.

Finally, the comparison of the results in Figure 7, Figure 8 and Figure 9 shows that for each intensity UtoBr of the incoming request flow, there exists a distinct rate and trajectory of influence of ∆I_x within the adjustment interval I on the Latency_x(Ngl(to)). This emphasizes the importance of forecasting changes in the parameters of the incoming request flow in order to design individualized scenarios for adapting data write latency in the given MCS.

4.3. Group of Experiments “varW”

In the experiments of the “varW” group, the dependence of the latency of forming the correct packet numbers on changes in the width of the availability window W_x within the adjustment interval I was investigated at different intensities UtoBr.

4.3.1. Setup of Experiments “varW“ in the MCSs

In the experiments of the “varW” group, the values of W_x for the brokers of each MCS were organized into predefined sets. In this case, for each adjustment interval I_x, x = 1 … 7 from the range, the boundary values (Table 4) and sets of W_x, named {W₀ − 50%, W₀ − 25%, W₀, W₀ + 25%, W₀ + 50%} (

Table 6. Sets of availability windows W in MCS A, B, and C for experiments “varW”.

MCS	Name of Set	Availability Windows Wx (ms)
		W1	W2	W3	W4
A	W0 − 50%	84.9350	34.8850	10.8750	5.4700
	W0 − 25%	127.4025	52.3275	16.3125	8.2050
	W0	169.8700	69.7700	21.7500	10.9400
	W0 + 25%	212.3375	87.2125	27.1875	13.6750
	W0 + 50%	254.8050	104.6550	32.6250	16.4100
B	W0 − 50%	17.5450	13.6700	5.5450	5.3650
	W0 − 25%	26.3175	20.5050	8.3175	8.0475
	W0	35.0900	27.3400	11.0900	10.7300
	W0 + 25%	43.8625	34.1750	13.8625	13.4125
	W0 + 50%	52.6350	41.0100	16.6350	16.0950
C	W0 − 50%	10.0000	7.0000	6.0000	5.0000
	W0 − 25%	15.0000	10.5000	9.0000	7.5000
	W0	20.0000	14.0000	12.0000	10.0000
	W0 + 25%	25.0000	17.5000	15.0000	12.5000
	W0 + 50%	30.0000	21.0000	18.0000	15.0000

), were used.

In the experiment of the “varW” group, the average of $\Delta t_{from-to}^{tr}$ between brokers, as presented in Table 3, were used, while the SD of $\Delta t_{from-to}^{tr}$ was taken from the SD1 set from Table 5.

In each experiment, the input stream to each broker consisted of 10,000 user requests for data writing.

4.3.2. Results of Experiments “varW”

The results of the experiments of the “varW” group are presented in Figure 10, Figure 11 and Figure 12 in the form of four surfaces. The upper surface in all cases was obtained at UtoBr = 500 req/s, and all surfaces below were obtained at {100, 50, 20} req/s, respectively.

The average of the latency of generating packet numbers Latency_x(Ngl(to)) for the base set W₀, as shown in Figure 10, Figure 11 and Figure 12, is equal to the results of the experiments in Figure 7, Figure 8 and Figure 9 for SD1 across all MCSs. The results in Figure 10, Figure 11 and Figure 12 demonstrate that when wider availability windows are used for the same adjustment interval I, i.e., the W₀ + 50% sets, the latency Latency_x(Ngl(to)) in all MCSs is lower than with narrower windows, i.e., the W₀ − 50% sets. However, the magnitude of reduction in latency Latency_x(Ngl(to)) at availability window widening is not uniform for different durations of the adjustment interval I at the same request intensity UtoBr. For example, with intensity UtoBr = 500 req/s, expanding the availability window from W₀−50% to W₀+50% resulted in a decrease in Latency_x(Ngl(to)):

• In Br₁ by 5% at I = 489.900 ms and by 6% at I = 169.900 ms; in Br₄ by 2.5% at I = 489.900 ms and by 11% at I = 169.900 ms in MCS A;
• In Br₃ by 16% at I = 265.250 ms and by 9% at I = 113.250 ms; in Br₄ by 6% at I = 265.250 ms and by 5% at I = 113.250 ms in MCS B;
• In Br₁ by 16% at I = 108.525 ms and by 8% at I = 53.125 ms; in Br₄ by 10% at I = 108.525 ms and by 4% at I = 53.125 ms in MCS C.

Thus, modifying the width of the availability window W_x without altering the adjustment interval I can significantly influence Latency_x(Ngl(to)) and can therefore be employed in forming individual scenarios for adapting the data write latency in the MCSs.

The analysis of latency Latency_x(Ngl(to)) changes between the surfaces in Figure 10, Figure 11 and Figure 12 for all MCSs shows that the increase in Latency_x(Ngl(to)) with increasing incoming request intensity UtoBr is not uniform. For example, in MCS C (Figure 12), an increase in UtoBr from 100 req/s to 500 req/s (i.e., fivefold) led to a maximal increase in Latency_x(Ngl(to)) of ≈16%. In contrast, in MCS A (Figure 10), an increase in UtoBr from 20 req/s to 50 req/s (i.e., 2.5-fold) produced a maximal increase in Latency_x(Ngl(to)) of 29%. The obtained results not only emphasize the individual nature of the influence of the parameters of the incoming request flow on the brokers but also demonstrate the necessity of accounting for the rate of change in UtoBr when creating scenarios for adapting the data write latency.

5. Discussion

The analysis of the results presented in the section above confirms the hypothesis about the presence of a significant influence of the internal parameters of the Latord method of data consistency on one distinct component of the data write latency—the latency Latency_x(Ngl(to)). In addition, the results demonstrated a non-uniform influence on the Latency_x(Ngl(to)) not only of the intensity UtoBr but also of the duration of packet transmission between brokers, which is another component of LatencyWr(Ngl(to)) from expression (9). Thus, the experimental results demonstrate that the choice of the values of the internal parameters of the Latord method that provide the required level of data write latency in replicas is a multi-criteria optimization problem.

In the conducted simulation experiments, it was assumed that the user request flows to different brokers follow the same probability distribution parameters within a single MCS. Under real conditions, each broker receives a non-stationary incoming request flow, and the data write latency needs to be adapted to the fluctuations in the incoming request flow. By analogy, fluctuations in user activity in real cases may not occur simultaneously at all brokers, but only at one or several of them. This demonstrates the need to develop individual scenarios for adapting the data write latency for each MCS based on forecasts of changes in external parameters and data on the influence of elements of the set U(t) on the addenda of expression (9) separately.

Let us consider the principles of forming an adaptation scenario for the data write latency in MCS A based on the results of the experiments shown in Figure 7. Let the MCS operate with the following conditions: I = 289.90 ms, the SD of $\Delta t_{from-to}^{tr}$ corresponds to the SD2 set from Table 5, and the intensity of the incoming request flow to each broker is 100 req/s. The average of the data write latency in each broker with such operating parameters in MCS A, calculated using Formula (5), is given in

Table 7. Average of the data write latency in MCS A for I = 289.90 ms, SD2 of $\Delta t_{from-to}^{tr}$, and UtoBr = 100 req/s.

Average of the Data Write Latency (ms)
Br1	Br2	Br3	Br4
664.6392	682.7719	713.6057	724.7734

.

The average values of the duration of the formation of the correct packet number in each broker of MCS A for this configuration are presented in

Table 8. Average of the latency of the packet number formation in MCS A brokers for different SD values of packet transmission durations between brokers.

Adjustment Interval I (ms)	Standard Deviation (ms)	Average of the Latency of the Formation of Packets Number (ms)
		Br1	Br2	Br3	Br4
289.90	SD2	144.1962	227.7739	259.4436	261.4491
	SD1	107.4606	189.0890	225.3060	227.1781
209.90	SD1	96.5219	167.9406	203.1084	205.3815
229.90		98.5903	173.5104	209.5391	212.5863
249.90		101.8672	179.1206	215.1092	216.7648
269.90		104.5551	182.0716	217.8441	219.7127

and are shown as white points on the upper surfaces in Figure 7.

Suppose that the implementation of new technological solutions for dispersion compensation on one segment of the global communication line used for packet transmission between brokers in MCS A resulted in a reduction in the SD of $\Delta t_{from-to}^{tr}$, such that it now corresponds to the SD1 set from Table 5. All other characteristics of the external variables remained unchanged.

According to the results of the conducted experiments, a decrease in SD will lead to a decrease in the average latency of the formation of the packet number (Table 8) in each broker, resulting in values indicated by the red points on the lower surfaces in Figure 7. Consequently, this will require a secondary selection of the parameters of the Latord method, namely, I, W_x, and ∆I_x. The selection of a new adjustment interval I from the neighboring candidate values, marked by black points in Figure 7, should be based not only on the mean of the latencies of the packet number formation but also on the corresponding estimates of the ballast interval τ_x in each broker. Choosing the interval I, for which the mean value of τ_x is close to zero, may increase the likelihood of packet loss during transmission between brokers.

Table 9. Average of the data write latency in MCS A for different adjustment intervals.

Adjustment Interval I (ms)	Average of the Data Write Latency (ms)				Number of “Late” Packets out of 10,000
	Br1	Br2	Br3	Br4
209.90	502.7898	487.6762	515.2307	523.9986	6696
229.90	539.7138	535.8083	564.8082	574.1685	1838
249.90	579.9570	584.8174	615.1671	624.2518	0
269.90	621.2052	632.7332	664.1750	673.3707	0
289.90	664.6258	682.4046	716.1017	724.1544	0

presents the data write latency calculated using Formula (5) based on the conditions of the “var∆I” experiment group in MCS A for SD1 and for the adjustment intervals, among which a new value of the interval I may be selected.

The analysis of the results presented in Table 9 shows that the mean data write latency in each broker decreases as I decreases. However, at I < 229.90 ms, the number of packets received by passive brokers from the active one after MaxLate is more than 3.7% of the total number of sent packets, which means that the new value of I should be equal to 249.90 ms or 269.90 ms.

It should be noted that a decrease in the data write latency in brokers due to decreasing I, considered in this adaptation scenario, is possible if, before changing the external operational parameters of the MCS, the chosen interval I value was the one that allow to its decrease, rather than the minimum acceptable interval I. This means that the decision regarding the Latord method’s parameters that affect the data write latency must be made by taking into account the forecasts of their changes.

As can be seen from Table 9, at I = 249.90 ms, the average of the data write latency for all brokers is not less than that at I = 269.90 ms. To achieve a more acceptable data write latency, the possibilities of changing W_x within these two values of I can be assessed in the next step of the adaptation scenario. The experimental results shown in Figure 10 suggest that adjusting the ratio between windows W_x and ∆I_x within these intervals I is expected to further reduce the data write latency in MCS A. The average values of the data write latency for such adjustment intervals I under different availability window sets are presented in

Table 10. Average of the data write latency in MCS A for different adjustment intervals I and availability window sets W.

Adjustment Interval I (ms)	Name of Set	Average of the Data Write Latency (ms)
		Br1	Br2	Br3	Br4
249.90	W0 − 50%	578.1322	604.7697	627.1246	627.8015
	W0 − 25%	571.9400	594.0697	621.5324	625.6943
	W0	579.9570	584.8174	615.1671	624.2518
	W0 + 25%	601.1039	577.7559	611.6084	620.2109
	W0 + 50%	637.6934	573.0511	606.0604	617.2175
269.90	W0 − 50%	627.0558	655.2516	675.6664	679.8482
	W0 − 25%	618.0937	643.8742	670.4398	677.5652
	W0	621.2052	632.7332	664.1750	673.3707
	W0 + 25%	635.8850	626.6528	660.3239	671.7383
	W0 + 50%	666.5338	620.0096	656.6366	669.9980

.

The analysis of the data in Table 10 shows that using the set window W₀−25% instead of the set W₀ for the adjustment interval I = 249.90 ms reduces the mean of the data write latency only in broker Br₁, which is highlighted in bold in Table 10, while in brokers Br₂–Br₄, it leads to an increase. A similar situation occurs when using the W₀ + 25% instead of W₀: the mean of the data write latency decreases in brokers Br₂–Br_4, which is highlighted in bold in Table 10, but increases in broker Br₁. In this case, the selection of the adjustment interval I and the corresponding set of windows can be based on the vector $\left[Signi{f}_x\right]$ of the brokers’ significance. If reducing the average of the data write latency is more important for users from the cluster with broker Br₁ than for other users, then the W₀ − 25% set of the availability windows at I = 249.90 ms should be selected. If any of the Br₂–Br₄ brokers is more important, then the W₀ + 25% set at I = 249.90 ms would be more optimal.

The considered adaptation scenario will reduce the data write latency when changing the SD of $\Delta t_{from-to}^{tr}$ in MCS A by ≈13% in all brokers when using the W₀ − 25% set of availability windows or by ≈9% in broker Br₁ and by 14–15% in brokers Br₂–Br₄ when using the set W₀ + 25%.

The considered adaptation scenario also emphasizes the individual nature of changes in the data write latency when the external conditions of the MCS operation fluctuate.

It should be noted, however, that in the simulation experiments, it was assumed that the packet transmission times between brokers follow the same probability distribution with a common SD value for all brokers. Moreover, the SD was varied simultaneously and to the same level in all brokers at once. Under real conditions, changes in the average and SD values may occur for one or more broker pairs and by different amounts, which should be taken into account in the predictive AI model.

In conclusion, the Latord method of data consistency demonstrates robust capabilities for generating the adaptation scenarios of the data write latency in geo-distributed MCSs, even when the intensity of incoming request flow changes by 50 times (Figure 7, Figure 8 and Figure 9). This property is particularly valuable for load scaling, making the proposed approach a promising solution for adaptive latency management in MCSs.

6. Conclusions and Future Work

We present an approach to creating scenarios for adapting the data write latency in geo-distributed MCSs using learning AI models to solve a multi-criteria optimization problem. The ability to control data write latency is achieved by selecting optimal parameters of the Latord method of data consistency in the presence of data on changes in incoming packet flows and in the parameters of packet transmission between brokers. Simulation experiments in three MCSs of different configurations demonstrate the need to use individual scenarios for adapting the data write latency; for its formation, a scheme of interaction between three different learning AI models is proposed.

Future work will focus on studying the impact of non-uniform external influences on the data write latency in MCSs using the algorithmic data consistency method. Non-uniform changes in the incoming user requests more closely reflect the real operating conditions of the MCSs. This will allow us to determine the structure and parameters of the AI model for determining the minimum gradients from the set Y(t) and, therefore, create more effective scenarios for adapting the data write latency in replicas to changing external conditions.