Context-Aware Online Model Splitting and Device Association for Semi-Decentralized Federated Learning in Internet of Things

Abstract

As a distributed approach to Artificial Intelligence (AI) model construction over wireless networks, federated learning (FL) based on multi-device collaborative training can protect data privacy, as well as increase the computing load of local model updates. In contrast, split learning (SL) with proper model splitting can adapt to the computation and transmission capabilities among devices. In this paper, while taking advantage of FL and SL, we concentrate on a semi-decentralized hybrid federated split learning (SD-HFSL) framework, in which we surpass the limitations of a single central server and allow the shared split models to be aggregated among multiple edge servers. To verify the importance of latency optimization for training efficiency, we analyze the convergence performance of SD-HFSL while jointly considering the limited computation and communication resources. Then, aiming at maximizing the long-term training efficiency, we propose an online optimization problem that includes local model splitting and device association. Considering that the training latency is unknown to the system a priori, a context-aware online training algorithm with sublinear regret is proposed based on the framework of contextual multi-armed bandit (CMAB), where the edge servers can observe the context information of device sites for latency estimation, followed by the iterative optimization based on the evaluated information in different contexts. Experiments on several neural network models show that the proposed algorithm reduces training latency and improves test accuracy compared with the selected benchmarks.

1. Introduction

Internet of Things (IoT) technology has been extensively applied across a variety of domains, including industry and transportation, where demand for intelligent applications continues to grow [1,2]. These intelligent applications are often implemented based on Artificial Intelligence (AI) technologies with a large amount of training data collected from widely distributed IoT devices [3]. Meanwhile, considering the increasing complexity of neural network models, it is more difficult for resource-constrained devices to efficiently build training models to support rapidly iterating AI applications. In particular, although cloud computing [4] or edge computing [5] has the ability to provide extra computing resources for large-scale data training, the third-party computing model has difficulty with meeting the privacy requirements among the involved devices [6]. To solve this issue, taking advantage of the distributed devices and their local computing power, the federated learning (FL) [7] algorithm is an approach that can be used for cooperatively training AI models by using the local data of several devices while meeting privacy requirements. In contrast to conventional centralized training, which exchanges local data with a central server to train on massive volumes of raw data, FL can effectively safeguard data privacy without local data interaction. Recent works have designed a variety of FL frameworks to accommodate a large number of devices by using techniques to reduce training overhead [8,9]. Moreover, to facilitate FL over dynamic wireless networks, strategies of resource allocation were fully addressed, considering communication link unreliability [2,10,11,12] and training efficiency [13,14]. Moreover, some interesting reviews of FL can be found in [15,16,17,18].

Nevertheless, with the increasing complexity of AI models, low-performance devices under the short-plate effect can significantly increase training latency and weaken the training efficiency of FL in real-world scenarios. Most importantly, the local computing tasks of traditional FL will be completely dependent on the device itself, which can not fully mobilize the existing computing resources. Considering the problem that a single device is insufficient to support complex training tasks, a deep neural network (DNN) can be divided into several sections via split learning (SL) [19], which is a cooperative training method that can send procedures of forward propagation and gradient calculations to other computing nodes for processing, such as devices and edge servers. Compared with FL, SL can lower a single device’s processing load and energy usage by performing model splitting and distributing the intermediate outcomes, such as model features and gradients across other computing units. To reduce the computational load of model training and inference processes, the SL framework is implemented in open-source applications, such as indoor positioning, cooperative identification, and rapid reasoning [20]. Moreover, in academia, SL is deployed over wireless networks, maximizing learning performance with resource adaptation [21,22,23,24].

However, the amount of data possessed by a single device in SL is limited, and the trained model is insufficient in generalization ability or learning ability, especially in the heterogeneous data environment. To solve this issue, jointly considering the advantages of FL and SL, recent works have incorporated the low computational load on SL devices and collaborative training of FL [25,26,27,28,29,30,31,32,33]. In [25], a novel learning framework, namely, hybrid federated split learning (HSFL), was proposed to obtain a trade-off between training time and energy consumption with the joint optimization problem, including splitting decisions. Moreover, the time consumption of the stragglers in HSFL was considered in [26], where the local models were trained among all devices along the ring through a pre-defined direction. We can see that HSFL, in fact, retains the characteristics of FL, integrated with existing learning mechanisms. For instance, in [27], a special split from the perspective of clustering was proposed in HFSL, which was similar to clustered FL, but optimized the model updating and cluster splitting schemes in the training stage to accelerate model convergence. Moreover, the authors in [28] adapted HFSL to intrusion detection systems, considering the reliability of split information in the sharing process. To strike a balance between latency and performance, optimization was proposed in [29] by jointly optimizing client selection, model splitting, and bandwidth allocation policies. Recent work [30] analyzed and optimized the HFSL under multi-tier systems, improving the flexibility of model splitting. An asynchronous framework that enabled personalized model splitting and aperiodic model aggregation was proposed in [31] to minimize the long-term average training latency. Moreover, HFSL can also introduce differential privacy and other security mechanisms under the premise of ensuring learning efficiency [32,33]. These studies indicate that HFSL can combine the low local computational load of SL with the collaborative training ability of FL.

With regard to HFSL, there is a practical issue that has not been fully considered in previous works. In particular, the current HFSL framework is trained under the presumption that a central server has complete control over learning, which is obviously difficult to achieve due to the fact that the communication and computing power of the central server are limited. In particular, when the hardware of the central server fails, the local model of the device will lose the object of model uploading, resulting in extremely severe stagnation of the training process. To solve this issue, semi-decentralized edge learning frameworks were recently proposed in [34,35,36], which can greatly improve the robustness of the training process by performing model aggregation among various edge servers without a central server. Specifically, to reduce the consumption of spectrum resources, an adaptive control technique combining edge aggregation with device-to-device (D2D) technologies was proposed in [34]. Moreover, under the multiple edge server scenario, a tradeoff between training effectiveness and energy consumption was considered in [35]. Additionally, convergence performance of the global model in the case of decentralized architecture was analyzed in [36]. HFSL can be integrated with a semi-central architecture by reorganizing the local model update process into the interaction process between devices and edge servers. However, this combination is not simple to achieve and will create new challenges. On the one hand, adding the model splitting mechanism under a decentralized architecture needs to be redesigned according to the characteristics of each device in order to improve the training efficiency with limited wireless resources. On the other hand, it is difficult to describe the convergence performance of HFSL in a scenario of multiple edge servers accurately, and there is a lack of correspondence between training efficiency and splitting strategy. Therefore, we need to design an efficient training mechanism that includes model splitting and device association, where a semi-decentralized hybrid federated split learning (SD-HFSL) framework is designed.

In this paper, the considered SD-HFSL framework can use SL to reduce the computational load of FL devices with intra- and inter-aggregations among multiple edge servers. Aiming at maximizing the long-term training efficiency, an online problem of model splitting decisions is formulated. Specifically, considering that the training latency is uncertain in different states of devices, i.e., the communication environment varies across time, and the local computing power changes accordingly. Learning the training latency for each device site precisely with a cold start (i.e., no prior knowledge available) is the first step toward efficient model splitting. Fortunately, to solve this issue, multi-armed bandit (MAB) algorithms have been studied to address the tradeoff between exploration and exploitation in sequential decision-making with uncertain information. The classic MAB algorithm, e.g., upper confidence bound (UCB), is concerned with learning the single optimal action among a set of candidate actions with unknown rewards [37]. Using the known information adequately, contextual bandits extend the basic MAB to contextual multi-armed bandit (CMAB) [38,39], where the edge servers can observe the context information (i.e., channel state, allocated bandwidth, transmit power, interference) of device sites for the training latency estimation, especially local computing power estimation. Since the problem of contextual bandits is already much more common and difficult than the basic MAB, this paper tackles solutions based on the framework of CMAB. Different from the previous works [38,39], this paper can merge local computing power evaluation with the training process by considering the training, exploration, and exploitation phases jointly. Meanwhile, we investigate multiple servers with incomplete information and compare them with the existing works of HFSL. Finally, the considered SD-HFSL framework based on the proposed scheme can improve learning efficiency, even with limited resources and unknown information. As summarized in

Table 1. Comparison between related learning frameworks and the proposed SD-HFSL.

Work	Architecture	Multiple Edge Servers	Model Splitting	Device Association	Online/Context Learning	Regret Analysis
HSFL [25]	Centralized hybrid FL-SL	No	Yes	Limited	No	No
RingSFL [26]	Ring-based split FL	No	Yes	No	No	No
Hierarchical split FL [30]	Multi-tier hierarchy	Yes	Yes	Limited	No	No
Semi-asynchronous FSL [31]	Asynchronous edge learning	No	Yes	Client selection	No	No
Semi-decentralized FL [34,35,36]	Edge-to-edge aggregation	Yes	No	Yes	No	No
Proposed SD-HFSL	Semi-decentralized HFSL	Yes	Three-part split	Jointly optimized	Context-aware CMAB	Yes

, the main technical distinction of this work is not merely combining FL and SL, but jointly considering three coupled aspects that are usually treated separately, including semi-decentralized aggregation among multiple edge servers, three-part split training without label uploading, and context-aware online optimization of both split points and device–server association under unknown latency.

As such, the main contributions of this paper are as follows.

• We propose a novel SD-HFSL framework over wireless networks, where multiple edge servers are deployed, and each of them can coordinate a device cluster with splitting-based local model updating. Moreover, the edge models can be periodically updated based on intra-cluster model aggregation and inter-cluster model aggregation. According to the analysis results of convergence performance, aiming at maximizing the long-term training efficiency, an online optimization problem of model splitting and device association is formulated.
• An online decision-making algorithm based on the framework of CMAB is proposed, which allows the edge servers to observe the context information of device sites for training latency estimation. Meanwhile, combined with the evaluated information, devices can update the model splitting and association decisions according to the estimated context-dependent latency through exploration and exploitation in sequential decision-making under uncertainty. Meanwhile, we prove that the proposed algorithm can provide a provable performance, achieving sublinear regret compared to an oracle algorithm that knows the expected training latency.
• Our experiments adapt several model structures, including AlexNet, VGG16, and ResNet18, and present several comparison algorithms based on the existing works. The simulation results show that the proposed algorithm achieves lower training latency and higher test accuracy in the considered settings, especially when prior latency information is unavailable.

The organization of this paper is structured as follows. The system model with convergence analysis is presented in Section 2. The long-term optimization problem is introduced in Section 3. The details of the proposed online algorithm for training efficiency improvement are introduced in Section 4. Finally, we present extensive experimental results in Section 5, and the conclusions are drawn in Section 6.

2. System Model

2.1. SD-HFSL Framework

The proposed SD-HFSL framework is shown in Figure 1, which consists of a set of edge servers $\mathcal{S}=\{1,2,\dots ,S\}$ and a set of devices $\mathcal{K}=\{1,2,\dots ,K\}$.

Each device $k\in \mathcal{K}$ has its local dataset ${\{x_{k,i},y_{k,i}\}}_{i=1}^{D_k}$ with $D_k$ training samples, where $x_{k,i}$ and $y_{k,i}$ are the i-th training data and the corresponding training label, respectively. Given the training round set $\mathcal{T}=\{1,2,\dots ,T\}$, by performing T rounds of multi-devices collaborative learning, our goal is to find the optimal global model ${\overline{\mathit{w}}}^{\ast }\left(T\right)$ from the trained edge model set $\overline{\mathbf{W}}\left(T\right)=\{{\overline{\mathit{w}}}_1\left(T\right),{\overline{\mathit{w}}}_2\left(T\right),\dots ,{\overline{\mathit{w}}}_S\left(T\right)\}$ that can minimize the global training loss, i.e.,

$$\underset{\overline{\mathit{w}}\left(T\right)\in \overline{\mathbf{W}}\left(T\right)}{min}F\left(\overline{\mathbf{w}}\left(T\right)\right)={\displaystyle \frac{{\sum }_{k\in \mathcal{K}}\left|{\mathcal{D}}_k\right|f_k\left(\overline{\mathit{w}}\left(T\right)\right)}{{\sum }_{k\in \mathcal{K}}\left|{\mathcal{D}}_k\right|}},$$

(1)

where $f_k\left(\overline{\mathit{w}}\left(T\right)\right)$ is the local loss of device k with edge model $\overline{\mathit{w}}\left(T\right)$ and can be evaluated at the end of each training round.

The main notation used in the system model is summarized in

Table 2. Summary of key notation.

Notation	Definition
$\mathcal{K},\mathcal{S}$	Sets of IoT devices and edge servers
$D_k$	Number of training samples at device k
$I_k^{\mathrm{a},\mathrm{b}},I_k^{\mathrm{b},\mathrm{c}}$	Two split points separating parts a, b, and c
$x_{k,s}\left(t\right)$	Association decision between device k and edge server s
$z_k\left(t\right)$	Successful participation indicator under the latency budget
$a_{k,s}\left(t\right)$	Effective participation, i.e., $a_{k,s}\left(t\right)=x_{k,s}\left(t\right)z_k\left(t\right)$
${\tau }_k\left(t\right),\tau \left(t\right)$	Device latency and synchronous round latency
$c_k\left(t\right),{\tilde{c}}_s\left(t\right)$	Context states of device k and edge server s
${\psi }_k,{\tilde{\psi }}_s$	Estimated computing capability under the observed context

.

Specially, compared with the training process in traditional semi-decentralized FL [40], the characteristic of the considered SD-HFSL framework is that we redesign the device and edge server model interaction process, assuming that the trained local model on the device can be split to offload part of the computation to the associated edge server. For instance, we design a three-part splitting strategy without label sharing [26], where the DNN is beyond the execution capacity of the device and can be split into three parts, namely part a, part b, and part c. It is divided into three parts because we consider that in the case of model splitting, the labels of devices do not leave the local area in the gradient descent calculation, so as to improve the privacy protection capabilities. We note that keeping labels locally does not eliminate all privacy risks because intermediate features and back-propagated gradients may still contain information about local samples. Therefore, the proposed framework focuses on reducing label exposure and communication/computation latency, while stronger protection can be incorporated by adding feature perturbation, differential privacy noise, or secure aggregation to the exchanged activations and gradients.

Then, in each training round t, the key steps of the SD-HFSL framework include local model updates, intra-cluster model aggregation, and inter-cluster model aggregation.

(1) Local Model Updates: Considering the case of model splitting, the initial parameter $\mathit{w}\left(t\right)$ of the device k in training round t can be written as ${\mathit{w}}_k\left(t\right)=\{{\mathit{w}}_k^{\mathrm{a}}\left(t\right),{\mathit{w}}_k^{\mathrm{b}}\left(t\right),{\mathit{w}}_k^{\mathrm{c}}\left(t\right)\}$, where only the model parameters of part a and part c are downloaded to devices. Since the model in splitting is usually large in scale and occupies more computing resources, we adopt parallel SGD with fast aggregation, in which the local model ${\mathit{w}}_k\left(t\right)$ obtained by device k is updated as

$${\mathit{w}}_k\left(t\right)={\mathit{w}}_k(t-1)-\eta {\mathit{g}}_k({\zeta }_k\left(t\right),{\mathit{w}}_k(t-1)),$$

(2)

where $\mathit{g}({\zeta }_k\left(t\right),{\mathit{w}}_k(t-1))$ is the gradient computed on the batch of randomly-sampled local training dataset ${\zeta }_k\left(t\right)$ with model parameter ${\mathit{w}}_k(t-1)$, and $\eta$ is the learning rate. Moreover, the parameter update for devices with splitting is performed based on the vanilla SL without label sharing. Specifically, in the forward propagation of SL, ${\mathit{w}}_k^{\mathrm{a}}\left(t\right)$ is executed locally and the output feature is uploaded to the associated edge servers for the calculation of ${\mathit{w}}_k^{\mathrm{b}}\left(t\right)$. Then, the output of the edge server is downloaded by the device for the execution of ${\mathit{w}}_k^{\mathrm{c}}\left(t\right)$. Similarly, in the backward propagation, the gradients of part c, part b and part a are calculated in order, denoted as ${\mathit{g}}_k^{\mathrm{c}}\left({\mathit{w}}_k\left(t\right)\right)$, ${\mathit{g}}_k^{\mathrm{b}}\left({\mathit{w}}_k\left(t\right)\right)$, and ${\mathit{g}}_k^{\mathrm{a}}\left({\mathit{w}}_k\left(t\right)\right)$, respectively.

(2) Intra-cluster Model Aggregation: When the local models are updated, each device can upload its complete or partial model parameters to the associated edge server, performing intra-cluster model aggregation to obtain the edge mode ${\tilde{\mathit{w}}}_s\left(t\right)$, i.e.,

$${\tilde{\mathit{w}}}_s\left(t\right)={\displaystyle \frac{{\sum }_{k\in {\mathcal{V}}_s\left(t\right)}{D}_k{\mathit{w}}_k\left(t\right)}{{\sum }_{k\in {\mathcal{V}}_s\left(t\right)}{D}_k}},$$

(3)

where the set of devices associated with the s-th edge server and successfully participating in training round t is denoted as ${\mathcal{V}}_s\left(t\right)$. Then, the updated edge models can be broadcast to the associated devices for the following local model updates in the current training round.

(3) Inter-cluster Model Aggregation: When the edge model is updated, each edge server can upload its model parameters to the associated edge servers, performing inter-cluster model aggregation, i.e.,

$$\begin{array}{cc}\hfill & {\tilde{\mathit{w}}}_s(t+1)=\hfill \\ & {\displaystyle \frac{{\sum }_{k\in {\mathcal{V}}_s\left(t\right)}{D}_k{\mathit{w}}_k\left(t\right)+{\sum }_{s^{\prime }\in {\mathcal{C}}_s\left(t\right)}{\sum }_{k^{\prime }\in {\mathcal{V}}_{s^{\prime }}\left(t\right)}{D}_{k^{\prime }}{\mathit{w}}_{k^{\prime }}\left(t\right)}{{\sum }_{k\in {\mathcal{V}}_s\left(t\right)}{D}_k+{\sum }_{s^{\prime }\in {\mathcal{C}}_s\left(t\right)}{\sum }_{k^{\prime }\in {\mathcal{V}}_{s^{\prime }}\left(t\right)}{D}_{k^{\prime }}}},\hfill \end{array}$$

(4)

where ${\mathcal{C}}_s\left(t\right)$ is the set of edge servers that are placed in the converging area of edge server s with successful model interaction in training round t.

Finally, at the end of each training round, the updated edge models of edge servers are transmitted to the associated devices to perform the next round of training. As the complexity of the model increases, it is usually required to perform many training rounds to obtain the desired learning performance.

2.2. Latency Model

The latency of the SD-HFSL framework consists of forward propagation and back propagation. Meanwhile, the single-round training process involves multiple stages of model transmission and computation.

(1) Transmission Latency: The number of divisible layers contained in a model is denoted as $I^{\max }$. Then, let $1\le I_k^{\mathrm{a},\mathrm{b}}\left(t\right)\le I^{\max }-2$ and $I_k^{\mathrm{a},\mathrm{b}}\left(t\right)<I_k^{\mathrm{b},\mathrm{c}}\left(t\right)\le I^{\max }-1$ be the two breakpoints of the breakpoints for device k in training round t, corresponding to the model, divided into three parts. In addition, if the local model of device k is not split, we can derive that $I_k^{\mathrm{a},\mathrm{b}}\left(t\right)=I_k^{\mathrm{b},\mathrm{c}}\left(t\right)=0$. To avoid ambiguity between association and successful participation, we use $x_{k,s}\left(t\right)$ to denote the device–server association before training latency is evaluated. After ${\tau }_k\left(t\right)$ is obtained, $z_k\left(t\right)$ denotes whether device k satisfies the latency budget, and $a_{k,s}\left(t\right)=x_{k,s}\left(t\right)z_k\left(t\right)$ denotes effective participation in aggregation and utility evaluation.

In the forward propagation of device k, denote ${\mathcal{T}}_k^{\mathrm{U}}\left(t\right)=\{{\tau }_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right),{\tau }_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)\}$ as the uploading latency set, where ${\tau }_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right)$ and ${\tau }_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)$ are the latencies of uploading the output features of part a to part b, and part b to part c, respectively. Mathematically, we can evaluate ${\tau }_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right)$ and ${\tau }_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)$ with feature size $M_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right)$ and $M_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)$ as

$$\begin{array}{cc}\hfill {\tau }_k^{\mathrm{U},\mathrm{a},\mathrm{b}}(t)& =\mathbb{I}(I_k^{\mathrm{a},\mathrm{b}}\left(t\right))\sum _{s\in \mathcal{S}}{\displaystyle \frac{x_{k,s}\left(t\right)M_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right)}{W_{k,s}^{a,b}\left(t\right){log}_2\left(1+\frac{p_k\left(t\right)H_{k,s}\left(t\right)}{n_k^{\mathrm{intra-cluster}}\left(t\right)+n_k^{\mathrm{inter-cluster}}\left(t\right)+{\sigma }^2}\right)}}\hfill \\ & +(1-\mathbb{I}(I_k^{\mathrm{a},\mathrm{b}}(t))){\tau }_k^{\mathrm{U},0}(t)\hfill \end{array}$$

(5)

and

$${\tau }_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)=\mathbb{I}\left(I_k^{\mathrm{b},\mathrm{c}}\left(t\right)\right)\sum _{s\in \mathcal{S}}{\displaystyle \frac{x_{k,s}\left(t\right)M_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)}{{\tilde{W}}_{s,k}^{b,c}\left(t\right){log}_2\left(1+\frac{{\tilde{p}}_{s,k}\left(t\right){\tilde{H}}_{s,k}\left(t\right)}{{\tilde{n}}_s^{\mathrm{intra-cluster}}\left(t\right)+{\tilde{n}}_s^{\mathrm{inter-cluster}}\left(t\right)+{\sigma }^2}\right)}},$$

(6)

respectively, where $\mathbb{I}(·)$ is the indicator function that determines whether there is model splitting, ${\tau }_k^{\mathrm{U},0}\left(t\right)$ is the latency for forward propagation without considering model splitting, $W_{k,s}^{a,b}\left(t\right)$ is the bandwidth allocated to the device k, $p_k\left(t\right)$ refers to the transmit power of device k, $H_{k,s}\left(t\right)$ is the channel gain between the device k and its associated edge server, $n_k^{\mathrm{intra-cluster}}\left(t\right)$ is the intra-cluster interference caused by the devices associated with the same edge server, $n_k^{\mathrm{inter-cluster}}\left(t\right)$ is the inter-cluster interference caused by the other elements in the system with the same frequency, and ${\sigma }^2$ is the additive white Gaussian noise (AWGN). Similarly, ${\tilde{W}}_{s,k}^{b,c}\left(t\right)$ is the bandwidth allocated to the edge server to transmit model parameters to device k, ${\tilde{p}}_{s,k}\left(t\right)$ refers to the transmit power of edge server s, ${\tilde{H}}_{s,k}\left(t\right)$ is the channel gain between the device k and its associated edge server s, and ${\tilde{n}}_s^{\mathrm{intra-cluster}}\left(t\right)$ and ${\tilde{n}}_s^{\mathrm{inter-cluster}}\left(t\right)$ are the intra- and inter-cluster interferences for edge server s. Similarly, the transmission latency in back propagation is denoted as ${\mathcal{T}}_k^{\mathrm{D}}\left(t\right)=\{{\tau }_k^{\mathrm{D},\mathrm{c},\mathrm{b}}\left(t\right),{\tau }_k^{\mathrm{D},\mathrm{b},\mathrm{a}}\left(t\right)\}$. Assuming the channel and noise environment of the system in the process of feature uploading and gradient downloading are unchanged in the same training round, let ${\overline{M}}_k^{\mathrm{D},\mathrm{c},\mathrm{b}}\left(t\right)$ and ${\overline{M}}_k^{\mathrm{D},\mathrm{b},\mathrm{a}}\left(t\right)$ be the size of the gradients determined by the split points for device k in training round t, then we can evaluate ${\tau }_k^{\mathrm{D},\mathrm{c},\mathrm{b}}\left(t\right)$ and ${\tau }_k^{\mathrm{D},\mathrm{b},\mathrm{a}}\left(t\right)$ as

$$\begin{array}{cc}\hfill {\tau }_k^{\mathrm{D},\mathrm{c},\mathrm{b}}\left(t\right)& =\mathbb{I}\left(I_k^{\mathrm{b},\mathrm{c}}\left(t\right)\right){\displaystyle \frac{{\overline{M}}_k^{\mathrm{D},\mathrm{c},\mathrm{b}}\left(t\right)}{M_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)}}{\tau }_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)\hfill \end{array}$$

(7)

and

$$\begin{array}{c}\hfill {\tau }_k^{\mathrm{D},\mathrm{b},\mathrm{a}}\left(t\right)=\mathbb{I}\left(I_k^{\mathrm{a},\mathrm{b}}\left(t\right)\right){\displaystyle \frac{{\overline{M}}_k^{\mathrm{D},\mathrm{b},\mathrm{a}}\left(t\right)}{M_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right)}}{\tau }_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right),\end{array}$$

(8)

respectively.

(2) Computing Latency: Denote $C^{F,l}$ and $C^{B,l}$ as the number of floating point operations (FLOPs) required by the l-th layer in the forward and backward propagation. Moreover, the local computing power in cycle/s of device k and edge server s are denoted as $f_k\left(t\right)$ and ${\tilde{f}}_s\left(t\right)$, respectively. Then, the computing latency neural networks in forward propagation for part a, part b, and part c can be evaluated as

$$\begin{array}{cc}\hfill {\tau }_k^{\mathrm{C},\mathrm{a}}(t)& =\mathbb{I}(I_k^{\mathrm{a},\mathrm{b}}\left(t\right)){\displaystyle \sum _{l=1}^{I_k^{\mathrm{a},\mathrm{b}}\left(t\right)}}{\displaystyle \frac{C^{F,l}}{f_k\left(t\right)}}+(1-\mathbb{I}(I_k^{\mathrm{a},\mathrm{b}}\left(t\right))){\tau }_k^{\mathrm{C},0}\left(t\right),\hfill \end{array}$$

(9)

$${\tau }_k^{\mathrm{C},\mathrm{b}}\left(t\right)=\mathbb{I}\left(I_k^{\mathrm{b},\mathrm{c}}\left(t\right)\right)\sum _{s\in \mathcal{S}}{x}_{k,s}\left(t\right){\displaystyle \sum _{l=I_k^{\mathrm{a},\mathrm{b}}\left(t\right)+1}^{I_k^{\mathrm{b},\mathrm{c}}\left(t\right)}}{\displaystyle \frac{C^{F,l}}{{\tilde{f}}_s\left(t\right)}},$$

(10)

and

$${\tau }_k^{\mathrm{C},\mathrm{c}}\left(t\right)=\mathbb{I}\left(I_k^{\mathrm{b},\mathrm{c}}\left(t\right)\right){\displaystyle \sum _{l=I_k^{\mathrm{b},\mathrm{c}}\left(t\right)+1}^{I^{\max }}}{\displaystyle \frac{C^{F,l}}{f_k\left(t\right)}},$$

(11)

respectively, where ${\tau }_k^{\mathrm{C},0}\left(t\right)$ can be the computing latency for forward propagation without considering model splitting in device k. Similarly, let ${\tau }_k^{\mathrm{B},0}\left(t\right)$ be the computing latency for back propagation without considering model splitting in device k. Then, the computing latency in back propagation can be evaluated as

$$\begin{array}{c}\hfill {\tau }_k^{\mathrm{G},\mathrm{c}}(t)=\mathbb{I}(I_k^{\mathrm{b},\mathrm{c}}(t)){\displaystyle \sum _{l=I_k^{\mathrm{b},\mathrm{c}}(t)+1}^{I^{\max }}}{\displaystyle \frac{C^{B,l}}{f_k\left(t\right)}}+(1-\mathbb{I}(I_k^{\mathrm{a},\mathrm{b}}(t))){\tau }_k^{\mathrm{B},0}(t),\end{array}$$

(12)

$${\tau }_k^{\mathrm{G},\mathrm{b}}\left(t\right)=\mathbb{I}(I_k^{\mathrm{b},\mathrm{c}}\left(t\right))\sum _{s\in \mathcal{S}}{x}_{k,s}\left(t\right){\displaystyle \sum _{l=I_k^{\mathrm{a},\mathrm{b}}\left(t\right)+1}^{I_k^{\mathrm{b},\mathrm{c}}\left(t\right)}}{\displaystyle \frac{C^{B,l}}{{\tilde{f}}_s\left(t\right)}},$$

(13)

and

$${\tau }_k^{\mathrm{G},\mathrm{a}}\left(t\right)=\mathbb{I}(I_k^{\mathrm{a},\mathrm{b}}\left(t\right)){\displaystyle \sum _{l=1}^{I_k^{\mathrm{a},\mathrm{b}}\left(t\right)}}{\displaystyle \frac{C^{B,l}}{f_k\left(t\right)}},$$

(14)

respectively.

In addition, similar to traditional FL, the SD-HFSL framework can also be designed for synchronous aggregation. The dominant latency terms are the split forward/backward computation and feature/gradient transmission. The aggregation, inter-edge exchange, model broadcast, and edge queuing delays are therefore omitted in the main expression for analytical tractability. If these components are non-negligible in a practical deployment, they can be added as an extra term ${\tau }^{\mathrm{agg}}\left(t\right)+{\tau }^{\mathrm{inter}}\left(t\right)+{\tau }^{\mathrm{bc}}\left(t\right)+{\tau }^{\mathrm{q}}\left(t\right)$ without changing the proposed online decision structure, where ${\tau }^{\mathrm{agg}}\left(t\right)$ denotes the intra-cluster aggregation latency, ${\tau }^{\mathrm{inter}}\left(t\right)$ denotes the inter-edge model exchange latency, ${\tau }^{\mathrm{bc}}\left(t\right)$ denotes the model broadcast latency from edge servers to devices, and ${\tau }^{\mathrm{q}}\left(t\right)$ denotes the edge-server queuing latency in training round t.

Thus, without considering the latency of model aggregation, inter-cluster model interaction, and model broadcast, we can evaluate the actual training latency of the system in round t as

$$\begin{array}{cc}\hfill \tau \left(t\right)& =\underset{k\in \mathcal{K}}{max}{\tau }_k\left(t\right),\hfill \end{array}$$

(15)

where

$$\begin{array}{cc}\hfill {\tau }_k\left(t\right)& ={\tau }_k^{\mathrm{C},\mathrm{a}}\left(t\right)+{\tau }_k^{\mathrm{U},\mathrm{a},\mathrm{b}}\left(t\right)+{\tau }_k^{\mathrm{C},\mathrm{b}}\left(t\right)\hfill \\ & +{\tau }_k^{\mathrm{U},\mathrm{b},\mathrm{c}}\left(t\right)+{\tau }_k^{\mathrm{C},\mathrm{c}}\left(t\right)+{\tau }_k^{\mathrm{G},\mathrm{c}}\left(t\right)\hfill \\ & +{\tau }_k^{\mathrm{D},\mathrm{c},\mathrm{b}}\left(t\right)+{\tau }_k^{\mathrm{G},\mathrm{b}}\left(t\right)+{\tau }_k^{\mathrm{D},\mathrm{b},\mathrm{a}}\left(t\right)+{\tau }_k^{\mathrm{G},\mathrm{a}}\left(t\right).\hfill \end{array}$$

(16)

Moreover, since the value of 0–1 variable $a_{k,s}\left(t\right)$ is related to the successful participation in training, we can set the maximum training latency ${\tau }^{\max }\left(t\right)$ to avoid the existence of equipment with extremely poor latency performance, i.e.,

$$z_k\left(t\right)=\left\{\begin{array}{c}1,{\tau }_k\left(t\right)\le {\tau }^{\max }\left(t\right),\hfill \\ 0,{\tau }_k\left(t\right)>{\tau }^{\max }\left(t\right),\hfill \end{array}\right.$$

(17)

and the actual device–server participation variable is $a_{k,s}\left(t\right)=x_{k,s}\left(t\right)z_k\left(t\right)$. This separation removes the circular dependence between association selection and latency-feasible participation.

We can observe that the model splitting strategy and the device association strategy can directly affect the defined training latency, which is affected by multi-dimensional factors such as bandwidth, transmit power, local computing power, and interference. Meanwhile, enhancing learning performance constitutes one of the primary objectives of SD-HFSL. Nevertheless, the influence of optimizing training latency on learning performance warrants additional investigation.

2.3. Convergence Analysis

We conducted the convergence analysis of SD-HFSL to quantify the impact of training latency and considered the following three standard assumptions on the loss functions, which are consistent with the FL literature [27,30]. These assumptions are adopted to obtain a tractable upper bound for the semi-decentralized split aggregation process in resource-constrained IoT networks. They are standard in FL convergence analysis and allow us to explicitly connect device participation, inter-edge aggregation reliability, and the loss bound.

Assumption 1.

Each gradient ${\mathit{g}}_k\left(\mathit{w}\right)$ of the local loss $f_k\left(\mathit{w}\right)$ is Lipschitz continuous with a positive constant L, i.e., $\parallel {\mathit{g}}_k\left(\mathit{w}\right)-{\mathit{g}}_k\left({\mathit{w}}^{\prime }\right)\parallel \le L\parallel \mathit{w}-{\mathit{w}}^{\prime }\parallel$. $\forall \mathit{w},{\mathit{w}}^{\prime }$.

Assumption 2.

Each local loss $f_k\left(\mathit{w}\right)$ is strongly convex with a positive constant μ, i.e., $f_k\left(\mathit{w}\right)\ge f_k\left({\mathit{w}}^{\prime }\right)+{(\mathit{w}-{\mathit{w}}^{\prime })}^T{\mathit{g}}_k\left({\mathit{w}}^{\prime }\right)+{\displaystyle \frac{\mu }{2}}{\parallel \mathit{w}-{\mathit{w}}^{\prime }\parallel }^2$, $\forall \mathit{w},{\mathit{w}}^{\prime }$.

Assumption 3.

Each local gradient ${\mathit{g}}_k\left(\mathit{w}\right)$ satisfies ${\xi }_1^2\le {\parallel {\mathit{g}}_k\left(\mathit{w}\right)\parallel }^2\le {\xi }_2^2$ with ${\xi }_1,{\xi }_2>0$, $\forall \mathit{w}$.

Theorem 1.

Denote the global optimal model as ${\mathit{w}}^{\ast }$, then the upper bound of the training loss on edge server s can be expressed as

$$\begin{array}{cc}\hfill & \mathbb{E}(\widetilde{F_s}\left({\tilde{\mathit{w}}}_s(t+1)\right)-{\tilde{F}}_s\left({\mathit{w}}^{\ast }\right))\hfill \\ & \le \left(1-{\left(1-{\displaystyle \frac{\mu }{L}}\right)}^{t+1}\right){\displaystyle \frac{4L{\xi }_2^2\left({\sum }_{k\in \mathcal{K}}{D}_k-N{U}^{\min }\right)}{\mu {\sum }_{k\in \mathcal{K}}{D}_k}}\hfill \\ & +{\left(1-{\displaystyle \frac{\mu }{L}}\right)}^{t+1}\mathbb{E}\left(F_s\left({\overline{\mathit{w}}}_s\left(1\right)\right)-F_s\left({\mathit{w}}^{\ast }\right)\right),\hfill \end{array}$$

(18)

where ${\tilde{\mathit{w}}}_s\left(t\right)={\displaystyle \frac{{\sum }_{k\in {\mathcal{V}}_s\left(t\right)}{D}_k{\mathit{w}}_k\left(t\right)}{{\sum }_{k\in {\mathcal{V}}_s\left(t\right)}{D}_k}}$. Here, $u_k^{\mathrm{D}}\left(t\right)$ denotes the probability that device k successfully participates in the current round under the latency budget, $u_{s,s^{\prime }}^{\mathrm{E}}\left(t\right)$ denotes the reliability of inter-edge model exchange from edge server $s^{\prime }$ to s, and $NU\left(t\right)$ denotes the amount of successfully aggregated data, including both intra-cluster and inter-cluster contributions.

$$\begin{array}{cc}\hfill & N{U}^{\min }=\underset{t\in \mathcal{T}}{min}NU\left(t\right)\hfill \\ & =\underset{t\in \mathcal{T}}{min}\sum _{k\in {\mathcal{V}}_s\left(t\right)}{u}_k^{\mathrm{D}}\left(t\right)D_k+\sum _{s^{\prime }\in {\mathcal{C}}_s\left(t\right)}\sum _{k^{\prime }\in {\mathcal{V}}_{s^{\prime }}\left(t\right)}{u}_{k^{\prime }}^{\mathrm{D}}\left(t\right)u_{s,s^{\prime }}^{\mathrm{E}}\left(t\right)D_{k^{\prime }}.\hfill \end{array}$$

(19)

According to Theorem 1, increasing the device participation probability or increasing the reliability of communication between edge servers in $N{U}^{\min }$ can reduce the upper bound of training loss (see Appendix A). Moreover, it is worth noting that the factors affecting the learning performance among devices are not only the amount of locally owned data, but also the convergence parameters, such as L and $\mu$. Specially, we define the local gradient of device k as ${\mathit{g}}_k\left(t\right)$ rather than ${\mathit{g}}_k({\zeta }_k^j\left(t\right),{\mathit{w}}_k^{(j-1)}\left(t\right))$, which can represent the average gradient or the gradient based on local complete data. This simplification has little effect on the results of the analysis, since this paper focus on the cost of edge servers rather than the local updates among the devices. In particular, the data held by the devices that do not have a low local loss based on the current edge model should be more important for improving learning performance. Finally, in light of the analytical findings of the convergence performance, we may rebuild the objective function using the acquired upper limit. It should be emphasized that the above theorem provides theoretical insight under simplified convex assumptions. The DNNs used in the experiments, such as AlexNet, VGG16, and ResNet18, are non-convex. Therefore, the bound should not be interpreted as a complete convergence guarantee for these neural networks. Instead, it explains why reducing latency and increasing latency-feasible participation are useful design principles. The experimental results then provide empirical evidence that the same principles improve SD-HFSL performance in practical non-convex training.

3. Problem Formulation

The considered problem for SD-HFSL is a sequential decision-making problem, and the goal is to maximize the training efficiency. Specifically, to jointly optimize the learning performance and training latency, the training efficiency can be evaluated as the decrease in training losses per unit of time. Hence, for each device k, the training efficiency can be defined as

$$u_k\left(t\right)=\sum _{s\in \mathcal{S}}{\displaystyle \frac{a_{k,s}\left(t\right)\left({\tilde{F}}_s\left({\tilde{\mathit{w}}}_s(t-1)\right)-{\tilde{F}}_s\left({\tilde{\mathit{w}}}_s\left(t\right)\right)\right)}{{\tau }_k\left(t\right)}},$$

(20)

where ${\tilde{F}}_s\left({\tilde{\mathit{w}}}_s(t-1)\right)-{\tilde{F}}_s\left({\tilde{\mathit{w}}}_s\left(t\right)\right)$ indicates the reduction of training loss under the associated edge model.

Then, the system utility to represent the training efficiency can be expressed as

$$U\left(t\right)=\sum _{k\in \mathcal{K}}{u}_k\left(t\right).$$

(21)

Different from common optimization problems, this paper considers the unknown randomness in local computing power changes, resulting in training latency is uncertain in different states of devices, which can be evaluated based on the context information $\widehat{C}\left(t\right)$, including channel state, allocated bandwidth, transmit power, interference, etc. Then, the formulation of the online maximization problem with model splitting strategy $\mathit{I}$ and device association strategy $\mathit{X}$ can be expressed as

$$\begin{array}{}\mathrm{(22a)}& \underset{\{\mathit{I},\mathit{X}\}}{max}{\displaystyle \sum _{t=1}^T}U(\mathit{I}\left(t\right),\mathit{X}\left(t\right),\widehat{C}\left(t\right))\mathrm{(22b)}& \mathrm{s}.\mathrm{t}.1\le I_k^{\mathrm{a},\mathrm{b}}\left(t\right)\le I^{\max }-2,\forall k\in \mathcal{K},t\in \mathcal{T},\mathrm{(22c)}& I_k^{\mathrm{a},\mathrm{b}}\left(t\right)<I_k^{\mathrm{b},\mathrm{c}}\left(t\right)\le I^{\max }-1,\forall k\in \mathcal{K},t\in \mathcal{T},\mathrm{(22d)}& x_{k,s}\left(t\right)\in \{0,1\},\forall k\in \mathcal{K},\forall s\in \mathcal{S},t\in \mathcal{T},\mathrm{(22e)}& \sum _{s\in \mathcal{S}}{x}_{k,s}\left(t\right)\le 1,\forall k\in \mathcal{K},t\in \mathcal{T},\mathrm{(22f)}& \sum _{k\in \mathcal{K}}{x}_{k,s}\left(t\right)\le A_s^{\max },\forall s\in \mathcal{S},t\in \mathcal{T},\end{array}$$

where the model splitting strategy is coupled with the device association strategy, the maximum number of devices associated with edge server s is defined as $A_s^{\max }$, and each device can be associated with, at most, one edge server. The latency-feasible variable $z_k\left(t\right)$ and the effective participation variable $a_{k,s}\left(t\right)=x_{k,s}\left(t\right)z_k\left(t\right)$ are then evaluated after the candidate association and split decision are determined.

There are two challenges to be addressed to solve this online maximization problem. On the one hand, precise training latency estimation is necessary for the model splitting decision to improve the expected utility when implemented. Since the system evaluates long-term performance and is based on cold starts, it cannot utilize historical information and can only continuously estimate the training latency of the device from multiple rounds. On the other hand, the relationship between training loss and optimization variables in the utility function can not accurately describe the effect of the splitting strategy on learning performance.

To solve these issues, we examine the convergence of the introduced framework under the previous analysis results in [10]. Specially, with limited latency budget ${\tau }^{\max }$, allowing more devices to participate in training can accelerate the convergence rate of training loss, i.e.,

$$\begin{array}{cc}\hfill & {\tilde{F}}_s\left({\tilde{\mathit{w}}}_s(t-1)\right)-{\tilde{F}}_s\left({\tilde{\mathit{w}}}_s\left(t\right)\right)\propto \hfill \\ & \sum _{k\in {\mathcal{V}}_s\left(t\right)}{a}_{k,s}\left(t\right)D_k{\varpi }^2+\sum _{s^{\prime }\in {\mathcal{C}}_s\left(t\right)}\sum _{k^{\prime }\in {\mathcal{V}}_{s^{\prime }}\left(t\right)}{a}_{k^{\prime },s^{\prime }}\left(t\right)D_{k^{\prime }}{\varpi }^2,\hfill \end{array}$$

(23)

where ${\varpi }^2\ge {\parallel {\mathit{g}}_k\left(\mathit{w}\right)\parallel }^2$ is the upper bound of the local gradient. Equation (23) uses the effective participation variable $a_{k,s}\left(t\right)$ and therefore counts the devices that successfully contribute to the model update, while the residual term in Theorem 1 is reduced when this successful data mass increases.

Then, the right side of the proportional Formula (23) is brought into the utility function (21) and is transformed into a problem that can be solved with a deterministic solution.

4. Joint Model Splitting and Device Association

We assume that only splitting and association strategies related to transmission latency can be further optimized, while other parameters, such as bandwidth allocation, channel gain, transmit power, and co-frequency interference are predetermined as the context information of the current training round. Specifically, we formulate the proposed long-term model splitting problem as a CMAB with online decisions. The strategy is “online” because the training latency among devices and edge servers is unknown in advance, and is evaluated based on the context associated with the system. Moreover, given the estimated latency information, we propose a joint splitting and association algorithm to further improve system utility.

4.1. Online Latency Estimation Based on CMAB

In CMAB, assuming that the existing control center can observe the context of the system at the beginning of each training round before performing model updates. Let $c_k\left(t\right)\in \mathit{c}\left(t\right)$ be the context of device k observed in training round t, where $\mathit{c}\left(t\right)$ is the context space and can be limited by quantitative coding. Similarly, we denote the context of edge server s as ${\tilde{c}}_s\left(t\right)\in \mathit{c}\left(t\right)$. Then, the context of the system is collected in

$$\begin{array}{cc}\hfill \mathit{c}\left(t\right)=& \{c_1\left(t\right),c_2\left(t\right),\dots ,c_K\left(t\right),{\tilde{c}}_1\left(t\right),{\tilde{c}}_2\left(t\right),\dots ,{\tilde{c}}_S\left(t\right)\}.\hfill \end{array}$$

(24)

The local computing power $\{f_k\left(t\right),{\tilde{f}}_s\left(t\right)\}$ of device k and edge server s is a random variable parameterized by the context $\{c_k\left(t\right),{\tilde{c}}_s\left(t\right)\}$. Hence, we can rewrite the computing power vector in a context-aware form, i.e.,

$$\begin{array}{cc}\hfill \mathit{f}\left(t\right)=& \{f_1\left(c_1\left(t\right)\right),f_2\left(c_2\left(t\right)\right),\dots ,f_K\left(c_K\left(t\right)\right),\hfill \\ & {\tilde{f}}_1\left({\tilde{c}}_1\left(t\right)\right),{\tilde{f}}_2\left({\tilde{c}}_2\left(t\right)\right),\dots ,{\tilde{f}}_S\left({\tilde{c}}_S\left(t\right)\right)\}.\hfill \end{array}$$

(25)

Moreover, let ${\psi }_k\left(c_k\left(t\right)\right)\triangleq \mathbb{E}[f_k\left(t\right)\mid c_k\left(t\right)]$ and ${\tilde{\psi }}_s\left({\tilde{c}}_s\left(t\right)\right)\triangleq \mathbb{E}[{\tilde{f}}_s\left(t\right)\mid {\tilde{c}}_s\left(t\right)]$ be the expected values of the unknown local computing power of device k and edge server s, respectively. Then, the vector of the expected local computing power is given by

$$\begin{array}{cc}\hfill \mathit{\psi }\left(t\right)=& \{{\psi }_1\left(c_1\left(t\right)\right),{\psi }_2\left(c_2\left(t\right)\right),\dots ,{\psi }_K\left(c_K\left(t\right)\right),\hfill \\ & {\tilde{\psi }}_1\left({\tilde{c}}_1\left(t\right)\right),{\tilde{\psi }}_2\left({\tilde{c}}_2\left(t\right)\right),\dots ,{\tilde{\psi }}_S\left({\tilde{c}}_S\left(t\right)\right)\}.\hfill \end{array}$$

(26)

The core of the CMAB approach is the use of contextual information $\mathit{c}\left(t\right)$ and computing power $\mathit{f}\left(t\right)$ to simulate the actual training latency. In practice, the control center does not have a priori knowledge of the training latency affected by the local computing power. Hence, the joint splitting and association strategy is optimized based on the estimation results, which can be replaced by $\mathit{\psi }\left(t\right)$ in each training round. More specifically, the context is the quantized vector $\mathit{c}\left(t\right)$, the action is the joint decision $\left\{\mathit{I}\right(t),\mathit{X}(t\left)\right\}$, and the reward is the system utility $U\left(t\right)$ defined in (21). The oracle policy is the policy that knows the expected computing capability $\mathit{\psi }\left(t\right)$ for every context and selects the best feasible joint decision in each round.

The context-aware online optimization algorithm designed in the framework of CMAB is executed simultaneously with the model training process. In each time slot or training round t, sequential decision-making is performed at a hypothetical control center as follows: (1) The control center observes the context set $\mathit{c}\left(t\right)$ of all devices and edge servers. (2) The control center determines the model splitting decision $\mathit{I}\left(t\right)$ and the device association strategy $\mathit{X}\left(t\right)$ based on the observed latest computing power information $\mathit{\psi }\left(t\right)$ in the current training round. (3) The optimized strategy is applied for model training, where if $I_k^{\mathrm{a},\mathrm{b}}\left(t\right)=I_k^{\mathrm{b},\mathrm{c}}\left(t\right)=0$, and device k can update its local model locally with complete model transfer to the associated edge servers; otherwise, three-part training is performed. (4) At the end of each training round, the actual local computing power among devices and edge servers is observed, which is then used to update the computing power estimation $\mathit{\psi }\left(t\right)$ with the observed context $\mathit{c}\left(t\right)$. Especially, if the latency budget cannot be satisfied, the local computation time concerned is upper-bounded to meet ${\tau }_k\left(t\right)={\tau }^{\max }\left(t\right)$.

To make the context-aware demand estimation tractable, the context space scope needs to be designed to be limited. To be specific, the system context $\mathit{c}\left(t\right)$ can be divided into small hypercubes with context partition. Meanwhile, the state of each device or edge server in different training rounds will be different, especially considering the complex communication environment and interference that is difficult to predict in advance. Therefore, the computing power estimation process needs to be performed separately on each device and edge server with fine-grained induction. Then, a key issue is estimating the local computing power pattern for context hypercubes at each device or each edge server. The proposed online optimization algorithm runs with a cold start, where local computing power can only be estimated based on the context of the hypercubes observed in each training round with an accumulation of historical information. In particular, to estimate the number of times that device k or edge server s contributes local computing power up to training round t, the control center has two types of counters $O_k\left(c_k\left(t\right)\right)$ and ${\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)$, with different contexts. The control center also keeps two experiences, ${\mathcal{E}}_k\left(c_k\left(t\right)\right)$ and ${\tilde{\mathcal{E}}}_s\left({\tilde{c}}_s\left(t\right)\right)$, from hypercube in training round t to store the context-power pair $\left\{\mathit{c}\right(t),\mathit{f}(t\left)\right\}$. Then, given the experiences, the local computing power estimated for device k and edge server s are obtained by estimator $\mathit{\theta }$, i.e.,

$${\psi }_k\left(c_k\left(t\right)\right)=\mathit{\theta }({\mathcal{E}}_k\left(c_k\left(t\right)\right),O_k\left(c_k\left(t\right)\right)),$$

(27)

and

$${\tilde{\psi }}_s\left({\tilde{c}}_s\left(t\right)\right)=\mathit{\theta }({\tilde{\mathcal{E}}}_s\left({\tilde{c}}_s\left(t\right)\right),{\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)),$$

(28)

respectively. In particular, in this paper, we consider an unbiased estimation $\mathit{\theta }(·)$ using maximum likelihood estimation, i.e.,

$${\psi }_k\left(c_k\left(t\right)\right)={\displaystyle \frac{{\sum }_{c\in {\mathcal{E}}_k\left(c_k\left(t\right)\right)}{f}_k\left(c\right)}{O_k\left(c_k\left(t\right)\right)}},$$

(29)

and

$${\tilde{\psi }}_s\left({\tilde{c}}_s\left(t\right)\right)={\displaystyle \frac{{\sum }_{c\in {\tilde{\mathcal{E}}}_s\left({\tilde{c}}_s\left(t\right)\right)}{\tilde{f}}_s\left(c\right)}{{\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)}}.$$

(30)

Algorithm 1 illustrates the pseudo code of the online optimization algorithm.

Specifically, in each training round t, the algorithm is in either an exploration phase or an exploitation phase. To determine the phase, the most important criterion is whether the local computing power corresponding to the current context information is fully explored. Hence, the sets of under-explored devices and edge servers are denoted as ${\mathcal{K}}^{\mathrm{ue}}\left(t\right)=\{k\in \mathcal{K},O_k\left(c_k\left(t\right)\right)\le O_k\left(t\right)\}$ and ${\mathcal{S}}^{\mathrm{ue}}\left(t\right)=\{s\in \mathcal{S},{\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)\le {\tilde{O}}_s\left(t\right)\}$, respectively, where $\mathit{O}\left(t\right)=\{O_k\left(t\right),{\tilde{O}}_s\left(t\right)\}$ is a threshold set to determine whether the historical data of devices and edge servers can enter the exploitation stage. In particular, since the training latency affected by the local computing power is the final expression in the proposed problem, we can evaluate the training latency based on the joint splitting and association strategy indirectly. In addition, since the combination of device $k\in \mathcal{K}$ and edge server $s\in \mathcal{S}$ can be viewed as an arm, according to the sets of under-explored devices and edge servers, the procedures of exploration and exploitation are as follows.

(1) Exploration If the under-explored set is non-empty, device $k\in {\mathcal{V}}_s\left(t\right)$ and its associated edge server s in the under-explored set have two cases. In case 1, for edge server s, the number of devices associated with the current training round satisfies ${\sum }_{k\in {\mathcal{K}}^{\mathrm{ue}}}{a}_{k,s}\left(t\right)=A_s^{\max },\exists s\in \mathcal{S}$, where the remaining devices $k\in {\mathcal{K}}^{\mathrm{ue}}\left(t\right)/{\mathcal{V}}_s\left(t\right)$ in the collection can be removed from the association policy in the current arm. In case 2, when the under-explored devices and edge servers satisfy ${\sum }_{k\in {\mathcal{K}}^{\mathrm{ue}}}{a}_{k,s}\left(t\right)<A_s^{\max },\forall s\in \mathcal{S}$. In this case, the devices can greedily associate existing edge servers in order.Meanwhile, since we have defined the latency budget ${\tau }^{\max }$, the latency that exceeds the budget during the exploration process is always denoted as ${\tau }_k={\tau }^{\max }$.

Algorithm 1 Context-Aware Online Optimization Algorithm

1:  Input: $\mathcal{K}$, $\mathcal{S}$, T, context partition, exploration thresholds, and latency budget ${\tau }^{max}$.   2:  Output: online splitting and association decisions ${\{\mathit{I}\left(t\right),\mathit{X}\left(t\right)\}}_{t=1}^T$.   3:  Initialization. initialize $O_k\left(c_k\left(t\right)\right)=0$, ${\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)=0$, $\forall k,s,t$, and choose an estimator $\mathit{\theta }$ based on maximum likelihood estimation.   4:   for $t=1,2,\dots ,T$ do   5:     Observe the context $\mathit{c}\left(t\right)$ and counter $\mathit{O}\left(t\right)$   6:     Evaluate under-explored devices and edge servers   7:     if ${\mathcal{K}}^{\mathrm{ue}}\left(\mathit{c}\left(t\right)\right)\cup {\mathcal{S}}^{\mathrm{ue}}\left(\mathit{c}\left(t\right)\right)\ne \varnothing$ then ▷ Exploration   8:     else   9:     Solving the problem in (22) ▷ Exploitation 10:     end if 11:     for each devices $k\in {\mathcal{K}}^{\mathrm{ue}}\left(t\right)$ do 12:    Update counter $O_k\left(c_k\left(t\right)\right)=O_k\left(c_k\left(t\right)\right)+1$ 13:    Update estimations ${\psi }_k\left(c_k\left(t\right)\right)={\displaystyle \frac{{\sum }_{c\in {\mathcal{E}}_k\left(c_k\left(t\right)\right)}{f}_k\left(c\right)}{O_k\left(c_k\left(t\right)\right)}}$ 14:    Update experiences ${\mathcal{E}}_k\left(c_k\left(t\right)\right)={\mathcal{E}}_k\left(c_k\left(t\right)\right)\cup \{c_k\left(t\right),{\psi }_k\left(c_k\left(t\right)\right)\}$ 15:     end for 16:     for each edge server $s\in {\mathcal{S}}^{\mathrm{ue}}\left(t\right)$ do 17:    Update counter ${\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)={\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)+1$ 18:    Update estimations ${\tilde{\psi }}_s\left({\tilde{c}}_s\left(t\right)\right)={\displaystyle \frac{{\sum }_{c\in {\tilde{\mathcal{E}}}_s\left({\tilde{c}}_s\left(t\right)\right)}{\tilde{f}}_s\left(c\right)}{{\tilde{O}}_s\left({\tilde{c}}_s\left(t\right)\right)}}$ 19:    Update experiences ${\tilde{\mathcal{E}}}_s\left({\tilde{c}}_s\left(t\right)\right)={\tilde{\mathcal{E}}}_s\left({\tilde{c}}_s\left(t\right)\right)\cup \{{\tilde{c}}_s\left(t\right),{\tilde{\psi }}_s\left({\tilde{c}}_s\left(t\right)\right)\}$ 20:     end for 21:    Perform joint model splitting and device association strategy 22:   end for 23:   Return $\left\{\mathit{I}\right(t),\mathit{X}(t\left)\right\}$.

(2) Exploitation: If the explored set is non-empty, we can optimize the joint problem of model splitting and device association strategy based on the current estimated value $\mathit{\psi }\left(t\right)$. Finally, given the feedback latency information, the counter, estimations, and experiences are also updated. Then, the model splitting problem can be decoupled from the device association problem, i.e.,

$$\begin{array}{}\mathrm{(31a)}& \hspace{1em}\underset{\{\mathit{I}(t\left)\right\}}{max}U({\mathit{X}}^{\ast }\left(t\right),\widehat{C}\left(t\right))\mathrm{(31b)}& \mathrm{s}.\mathrm{t}. \left(22\mathrm{b}\right), \left(22\mathrm{c}\right),\end{array}$$

and

$$\begin{array}{}\mathrm{(32a)}& \hspace{1em}\underset{\{\mathit{X}(t\left)\right\}}{max}U({\mathit{I}}^{\ast }\left(t\right),\widehat{C}\left(t\right))\mathrm{(32b)}& \mathrm{s}.\mathrm{t}. \left(22\mathrm{d}\right)–\left(22\mathrm{f}\right),\end{array}$$

respectively. Hence, given a fixed device association strategy, the optimal model splitting can be evaluated based on the exhaustive search. Since the model splitting strategy between the device and the edge server is carried out independently, we can infer that the complexity for the problem (31) is $\mathcal{O}\left({\left(K{I}^{\max }\right)}^2\right)$. Meanwhile, given the fixed model splitting strategy, the evaluated training latency among devices and edge servers can be evaluated. Then, a utility matching matrix $\mathbf{H}$, including the latency from the devices to the edge servers, is designed. Since each edge server can associate multiple devices, the original association is a many-to-one assignment. To use the KM algorithm, edge server s is expanded into $A_s^{max}$ virtual server slots, each of which has the same utility weight as device k. The resulting matrix has K device nodes and ${\sum }_{s\in \mathcal{S}}{A}_s^{max}$ virtual slots. A one-to-one matching over the virtual slots is therefore equivalent to the original capacity-constrained association because each device is matched to, at most, one slot, and no edge server receives more than $A_s^{max}$ devices because it owns only $A_s^{max}$ slots. If the two sides have different cardinalities, dummy nodes with zero utility are added to obtain a square matrix.

For a device–slot pair corresponding to edge server s, the matching weight is defined as the estimated utility, e.g., $1/{\tau }_k$ or the latency-normalized utility in (21), after the split decision is fixed.

The model splitting strategy and device association strategy are performed iteratively until the objective function is convergent, where the convergence criterion for a given threshold $ϵ$ is defined as $U({\mathit{X}}^{\ast }\left(t\right),\widehat{C}\left(t\right))-U({\mathit{I}}^{\ast }\left(t\right),\widehat{C}\left(t\right))\le \epsilon$ and the strategies in each iteration j are denoted as ${\mathit{I}}^{\left(j\right)}\left(t\right)$ and ${\mathit{X}}^{\left(j\right)}\left(t\right)$, respectively.

Moreover, the performance of online decision-making is measured by utility loss, termed regret, compared to the utility achieved by the oracle policy. Let $U^{★}\left(\mathit{c}\left(t\right)\right)$ be the expected utility achieved by the oracle policy under context $\mathit{c}\left(t\right)$, and let $U\left(t\right)$ be the utility achieved by the proposed online policy. The cumulative regret is

$$\begin{array}{c}\hfill R\left(T\right)={\displaystyle \sum _{t=1}^T}\left(U^{★}\left(\mathit{c}\left(t\right)\right)-U\left(t\right)\right).\end{array}$$

(33)

Under bounded utility and a finite context partition, the regret consists of exploration regret and exploitation regret. For $\left|\mathcal{C}\right|$ context hypercubes and an exploration threshold growing as $O(t^{\rho }logt)$, $0<\rho <1$, the number of forced exploration rounds is sublinear in T. The exploitation regret is also sublinear because the sample-mean estimates of the context-dependent computing capabilities converge to their expectations. Following the standard finite-partition CMAB concentration argument in [38], which bounds forced exploration and estimation error separately, the expected regret is upper bounded by

$$\begin{array}{c}\hfill \mathbb{E}\left[R\left(T\right)\right]\le C_1(K+S)\left|\mathcal{C}\right|T^{\rho }logT+C_2{T}^{1-\rho /2},\end{array}$$

(34)

where $C_1$ and $C_2$ are constants related to the bounded utility range and estimation error. Thus, $\mathbb{E}\left[R\right(T\left)\right]/T\to 0$ as $T\to \infty$, which indicates sublinear regret. Larger K, S, context dimension, or finer context partition increases the constants and slows convergence, which is also observed in the simulations.

4.2. Complexity Optimization for Online Learning

It is worth noting that the complexity of the KM algorithm in (32) is $\mathcal{O}\left(\max {\{K,S\}}^3\right)$, thus, it is more complicated to use the KM algorithm in each exploitation stage. To solve this issue, as shown in Algorithm 2, we can use a simple greedy algorithm with complexity $\mathcal{O}\left(K\right)$ to solve device association in the first few exploitation stages. Then, a joint optimization algorithm is adopted when the estimates are stabilized. In particular, we set a threshold $\kappa$, and if $t\le \kappa$, we allow each device to be independently associated with the edge server based on its optimal training latency until the maximal association constraint is satisfied. Moreover, even if the model splitting strategy can be selected among the limited optimization results, the complexity of the iterative optimization strategy can not be ignored. To solve this issue, considering the model splitting process, according to the transmission and computing processes, the model splitting strategy for exploration can be divided into two cases. (1) The device completes the model training locally without model splitting. (2) Only the first and last layer neural networks are trained locally at the device, and the remaining layers are trained at the edge server. Finally, according to the estimated local computing power of the current round, a better model splitting strategy for exploration is chosen from the above two cases to achieve lower complexity.

Algorithm 2 Joint Model Splitting and Device Association Algorithm

1:   Input: $\mathcal{K}$, $\mathcal{S}$, t, $\widehat{C}\left(t\right)$, ${\psi }_k\left(c_k\left(t\right)\right)$, ${\tilde{\psi }}_s\left({\tilde{c}}_s\left(t\right)\right)$, $A_s^{max}$, $\kappa$, and $\epsilon$.   2:   Output: splitting strategy ${\mathit{I}}^{\ast }\left(t\right)$ and association strategy ${\mathit{X}}^{\ast }\left(t\right)$.   3:   Initialization. $j=0$, $\mathit{I}\left(t\right)$, $\mathit{X}\left(t\right)$.   4:   Repeat.   5:     if $t\ge \kappa$   6:       for $k=1,2,\dots ,K$   7:         Ergodic search $I_k^{\mathrm{a},\mathrm{b}}\left(t\right)$ and $I_k^{\mathrm{b},\mathrm{c}}\left(t\right)$ with fixed association strategy ${\mathit{X}}^{\ast }\left(t\right)$ and obtain the temporary splitting strategy ${\mathit{I}}^{\ast }\left(t\right)={\mathit{I}}^{\left(j\right)}\left(t\right)$   8:       else   9:         Perform model splitting from two fixed strategies 10:       end for 11:   Update the current latency matching matrix $\mathbf{H}$ 12:   if $t\ge \kappa$ 13:       Obtain the association strategy ${\mathit{X}}^{\ast }\left(t\right)={\mathit{X}}^{\left(j\right)}\left(t\right)$ based on KM algorithm with fixed model splitting strategy ${\mathit{I}}^{\ast }\left(t\right)$ 14:     else 15:       Obtain the association strategy ${\mathit{X}}^{\ast }\left(t\right)={\mathit{X}}^{\left(j\right)}\left(t\right)$ based on greedy algorithm with fixed model splitting strategy ${\mathit{I}}^{\ast }\left(t\right)$ 16:     end if 17:     $j=j+1$ 18:   Until $U({\mathit{X}}^{\ast }\left(t\right),\widehat{C}\left(t\right))-U({\mathit{I}}^{\ast }\left(t\right),\widehat{C}\left(t\right))\le \epsilon$. 19:   Return. ${\mathit{I}}^{\ast }\left(t\right)$, ${\mathit{X}}^{\ast }\left(t\right)$.

5. Numerical Results

In this section, we present the details of the simulation results and evaluate the performance of the proposed SD-HFSL based on the framework of CMAB [41].

5.1. Experiment Settings

For the simulation results, we assume that there are $S=5$ edge servers and $K=25$ devices (if not specified) randomly located in a square area with the size of $500\mathrm{m}\times 500\mathrm{m}$. Moreover, each edge server can associate at most with five devices, i.e., $A_s^{\max }=5,\forall s\in \mathcal{S}$. In addition, the context space has two dimensions, namely “task workload of edge servers” and “wireless communication environment”. For instance, the context “task workload of edge servers” indicates the case where the devices and edge servers take on other computing tasks, and only a part of local computing power can be used for training. The “wireless communication environment” is the communication rate between devices and edge servers. For reproducibility, the wireless links use distance-dependent path loss with small-scale fading, the transmit powers of devices and edge servers are set to 23 dBm and 30 dBm, respectively, the total bandwidth is 20 MHz and equally divided among active links, and the noise power spectral density is $-174$ dBm/Hz. All reported curves are averaged over 20 independent trials with random locations, computing powers, and non-IID data partitions. Across these repeated trials, the result variation is small. The relative standard deviation of the latency and utility curves is within about $2.5\%$ of the mean, and the 95% confidence interval of the final test accuracy is within about $\pm 0.4$ percentage points. Since each figure contains multiple baseline curves, these narrow intervals are visually difficult to distinguish and would reduce readability. This behavior is different from reinforcement learning training curves, where policy exploration often causes much wider fluctuations. Then, to describe the unknown local computing power under different contexts, we can assume that the local computing power of device k and edge server s is uniformly distributed in the interval $[f_k^{\min }\left(\mathit{c}\left(t\right)\right),f_k^{\max }\left(\mathit{c}\left(t\right)\right)]$ $[{\tilde{f}}_s^{\min }\left(\mathit{c}\left(t\right)\right),{\tilde{f}}_s^{\max }\left(\mathit{c}\left(t\right)\right)]$, respectively, where the actual computing power of the device s is on the order of ${10}^9$ cycles/second, and the actual computing power of the edge server s is usually on the order of ${10}^{11}$ cycles/second, and the upper and lower bounds of computing power vary with the context information. In particular, to reduce the complexity of simulation, we consider four contextual situations that occur at equal intervals over 1000 training rounds, and for each device and edge server pair, the elements in the threshold set $\mathit{O}\left(t\right)$ are set as 50. Moreover, for the learning task, we consider non-i.i.d. local training data among devices and conduct the experiments using the CIFAR-10 datasets. By default, we train the learning model based on AlexNet, which consists of five convolutional layers, three fully connected layers, and finally the Softmax classification layer, where the batch size is set to 64, and the learning rate is set to 0.02. The number of model splitting points is $I^{\max }=8$. Moreover, to simulate the imbalance of the number distribution, we consider that each device has only two types of training samples. The experimental data setup and the main parameters are summarized in

Table 3. Experimental data setup.

Parameter	Value
K	25
S	5
$A_s^{max}$	5
${\tau }^{\max }$	5 s
Training samples per device	500
Types of labels per device	2
$f_k^{\min }\left(\mathit{c}\left(t\right)\right),f_k^{\max }\left(\mathit{c}\left(t\right)\right)$	$1\times {10}^9,5\times {10}^9$
${\tilde{f}}_s^{\min }\left(\mathit{c}\left(t\right)\right),{\tilde{f}}_s^{\max }\left(\mathit{c}\left(t\right)\right)$	$1\times {10}^{11},5\times {10}^{11}$
$\kappa$	50
T	1000

. These values are selected to emulate resource-constrained IoT devices assisted by stronger edge servers. The latency budget and exploration threshold are set to make the system experience both successful and unsuccessful participation events, which is necessary to evaluate online adaptation under incomplete information.

Finally, the proposed algorithm is compared with the following five benchmarks: (1) Oracle algorithm. The latency values between devices and edge servers are known in advance, and the optimal latency is evaluated when all devices can be successfully associated. (2) Greedy algorithm. Based on the existing experience, devices are associated with the edge server with the lowest training latency and set $\kappa =T$; that is, the KM algorithm is not used to optimize the association strategy. (3) UCB algorithm. The expected utility of each local computing power is evaluated over time by enumerating all splitting and association decisions without considering the context information. Moreover, when the association constraint is not satisfied, devices randomly select the edge server. (4) Random. Each device is randomly associated with the edge server. When the edge server association constraint is not satisfied, the device with large training latency is removed. (5) Noncooperative algorithm. Regardless of model splitting, devices perform model training locally. The proposed method also has limitations. It requires enough repeated observations in each context hypercube to obtain reliable estimates, and the matching step may become costly when the number of devices and virtual edge-server slots is very large. In addition, the present simulations focus on latency and learning efficiency rather than attacks on exchanged features or gradients.

5.2. Results and Discussions

In

Table 4. Estimated single-round training latency under different model splitting schemes.

${\mathit{I}}^1$, ${\mathit{I}}^2$	2	3	4	5	6	7
1	3.451	3.407	3.275	3.179	2.559	2.338
2	-	6.930	6.815	6.747	6.160	5.897
3	-	-	6.913	6.885	6.284	5.995
4	-	-	-	6.946	6.292	6.078
5	-	-	-	-	6.398	6.133
6	-	-	-	-	-	6.739

, to intuitively reflect the impact of the model splitting strategy on training latency, we randomly select a device k in training round t, and define its splitting strategy $I_k^{\mathrm{a},\mathrm{b}}\left(t\right)$ and $I_k^{\mathrm{b},\mathrm{c}}\left(t\right)$ as $I^1$ and $I^2$, respectively.

From Table 4, we can observe that when there is an obvious difference between the device’s local computing power and the edge server, the device tends to send as many computing tasks as possible to the edge server to reduce the training latency. Specifically, considering the parameter distribution of AlexNet, the training latency can increase significantly when the splitting strategy $I^1\ge 2$ due to the large computational amount of the second convolution layer. Moreover, because of the maximum training budget ${\tau }^{\max }$, device k is unable to successfully participate in training in this case. Hence, the splitting strategy has a direct impact on the training latency and the number of devices participating in the training, and needs to be fully optimized.

Figure 2 shows the cumulative performance of the proposed algorithm and the baselines. In particular, in Figure 2a, we compare the cumulative training latency ${\sum }_{t=1}^T\tau \left(t\right)$ of different algorithms. The results show that, except for the oracle and greedy algorithms, the proposed algorithm can obtain the lowest training latency. In addition, it is interesting that the proposed algorithm has no obvious advantage over the greedy algorithm and the UCB algorithm in the initial training rounds. This is because, to reduce the complexity, the proposed algorithm also completes device association in a greedy way under the exploration phase and has the same execution process as the comparison algorithms. Specifically, when the KM algorithm is taken into account, the advantages of the proposed algorithm in different contexts are constantly amplified. In Figure 2b, we compare the cumulative number of participating devices ${\sum }_{t=1}^T{\sum }_{k\in \mathcal{K}}{\sum }_{s\in \mathcal{S}}{a}_{k,s}\left(t\right)$. We can observe that our proposed algorithm is also the best, except for the oracle algorithm, due to the fact that we can realize the maximum total matching by the KM algorithm in the utilization phase. Other comparison algorithms cannot deal with the maximum number of association constraints, so more devices cannot participate in the training when the association strategy conflicts. Moreover, in Figure 2c, we evaluate the cumulative system utility ${\sum }_{t=1}^T{\displaystyle \frac{{\sum }_{k\in \mathcal{K}}{\sum }_{s\in \mathcal{S}}{a}_{k,s}\left(t\right)}{\tau \left(t\right)}}$, considering the training latency and the number of the participating devices. The proposed algorithm obtains a higher cumulative utility than the non-oracle baselines. In addition, in order to further reflect the gap between the proposed algorithm and the optimal strategy, we define the cumulative inability; that is, we calculate the difference between the proposed algorithm and the oracle algorithm in the training latency, the number of participating devices, and the system utility. In Figure 2d–f, it can be seen that after full exploration, the training latency and the system utility of the proposed algorithm have the slowest growing trend, while the number of devices participating in training is almost consistent with the optimal case, which can demonstrate the effectiveness of the proposed context-based estimation strategy.

Moreover, to further explain the reasons for the advantages of our proposed algorithm, we especially compare the proposed algorithm with the greedy and UCB algorithms. Specifically, in Figure 3a, we changed the proportions of devices to edge servers while evaluating the average number of associated devices in training, which can be denoted as ${\displaystyle \frac{{\sum }_{t=1}^T{\sum }_{k\in \mathcal{K}}{\sum }_{s\in \mathcal{S}}{a}_{k,s}\left(t\right)}{T}}$. We can observe that when the number of devices is small, i.e., $K=25$, and the number of edge servers is large, i.e., $S=10$, more devices can be associated with the ideal edge server, and there is little difference between the proposed algorithm and the greedy algorithm. On the contrary, when the number of terminals is large, i.e., $K=30$, there is an upper bound on the maximum number of associations due to the limit of the maximum number of devices associated in a single edge server, and there is no significant change as the number of devices increases. In addition, we further compare the average training latency ${\displaystyle \frac{{\sum }_{t=1}^T\tau \left(t\right)}{T}}$ of the proposed algorithm and the UCB algorithm under different numbers of context types. In particular, we assume that the larger the context index, the longer the training latency of a single training round. Moreover, different contextual situations occur in order per each 250 training rounds. Then, the results show that when the context type is small, the performance of the proposed algorithm is close to that of the UCB algorithm, and the advantages of the proposed algorithm are more significant with the increase of the context types. This is because our proposed algorithm considers the context when estimating training latency, and dynamically updates the model splitting and association strategies under different context backgrounds to improve performance.

Finally, based on the above simulation results, we also trained both VGG16 and ResNet18 networks in the process of verifying learning performance. Specifically, VGG16 consists of 13 convolutional layers and three fully connected layers, which are separated by the maximization pool, and the activation units of all hidden layers use ReLU functions. Moreover, the network structure of ResNet18 consists of 17 convolutional layers and a final fully connected layer, where each basic block consists of two convolution layers and a skip connection. In Figure 4, we evaluate the learning performance of different neural network structures under different numbers of training rounds. The final test accuracy increases when more devices participate in training.

Moreover, in Figure 5, except for the oracle algorithm, we can observe that the proposed algorithm based on different DNNs can obtain higher test accuracy in the same time frames, especially in the case of VGG16.

Moreover, with the increase in neural network complexity, test accuracy and convergence speed can increase simultaneously. Interestingly, although the computational amount of these neural networks is very different, the training latency in the simulation results does not seem to have a large gap. This is because the local computing capacity of the device is weak, considering that the training latency of the edge server cannot completely offset the local computing latency when the model is split.

6. Conclusions

In this paper, we investigated the long-term training efficiency of the SD-HFSL framework. A joint optimization problem of model splitting and device association was formulated based on the convergence performance. Meanwhile, a context-aware online training algorithm was proposed based on the framework of CMAB. We defined the training of the split training process and evaluated the learning performance based on the number of participating devices. The devices and edge servers can observe the context information of device sites for the latency estimation, followed by the iterative optimization based on the evaluated updated information in different contexts. Our experiments used different baselines and model structures to show that the proposed algorithm achieves lower training latency and higher test accuracy in the considered settings, especially when prior latency information is unavailable. Future work will consider privacy protection against feature and gradient leakage and will further evaluate the proposed online algorithm in real-world edge testbeds. Scalable low-complexity matching methods are also needed for larger IoT and 6G networks with fast-varying contexts [42].