Federated Semi-Supervised Learning with Uniform Random and Lattice-Based Client Sampling

Abstract

Federated semi-supervised learning (Fed-SSL) has emerged as a powerful framework that leverages both labeled and unlabeled data distributed across clients. To reduce communication overhead, real-world deployments often adopt partial client participation, where only a subset of clients is selected in each round. However, under non-i.i.d. data distributions, the choice of client sampling strategy becomes critical, as it significantly affects training stability and final model performance. To address this challenge, we propose a novel federated averaging semi-supervised learning algorithm, called FedAvg-SSL, that considers two sampling approaches, uniform random sampling (standard Monte Carlo) and a structured lattice-based sampling, inspired by quasi-Monte Carlo (QMC) techniques, which ensures more balanced client participation through structured deterministic selection. On the client side, each selected participant alternates between updating the global model and refining the pseudo-label model using local data. We provide a rigorous convergence analysis, showing that FedAvg-SSL achieves a sublinear convergence rate with linear speedup. Extensive experiments not only validate our theoretical findings but also demonstrate the advantages of lattice-based sampling in federated learning, offering insights into the interplay among algorithm performance, client participation rates, local update steps, and sampling strategies.

1. Introduction

1.1. Background

With the exponential growth of decentralized data generated by devices such as smartphones, federated learning (FL) [1,2] provides a framework for training high-quality shared global models while preserving data privacy. However, compared with the decentralized optimization [3,4], FL gives rise to significant challenges, particularly in terms of communication efficiency and client availability. To address these challenges, various FL algorithms have been developed [2], with federated averaging (FedAvg) [1] being one of the most widely used. FedAvg works by randomly selecting a subset of clients, performing local stochastic gradient descent (SGD) [5] on the selected clients, and aggregating the resulting models at the server [6]. Numerous studies have analyzed the convergence of FedAvg [7] and have proposed its variations [8,9,10], particularly in scenarios with partial client participation.

Existing federated learning approaches [11,12] commonly assume a supervised learning setup, where local private data is fully labeled. However, data labeling is often expensive and time-consuming, while unlabeled data is readily available in abundance. This drives us to effectively leverage these massive, distributed pools of unlabeled data to enhance federated learning. A straightforward approach might be to apply semi-supervised learning (SSL) techniques [13,14,15] to handle unlabeled data, while using traditional federated learning algorithms to aggregate the learned weights.

1.2. Related Works

Semi-Supervised Learning. Numerous studies have explored centralized semi-supervised learning. One prominent approach is pseudo-labeling [16], which generates one-hot labels from highly confident predictions on unlabeled data and uses these labels as training targets with a standard cross-entropy loss. Another major category of SSL methods involves consistency regularization. These methods, such as the ladder network [17], the $\Pi$ model [18], the mean teacher [19], virtual adversarial training (VAT) [20,21], and MixMatch [22], assume that the class semantics remain invariant under transformations of the input instances. To leverage this assumption, they enforce the model’s predictions to be consistent across different perturbations of the input data. Notably, Du et al. [23] further advanced this direction by proposing label propagation for imbalanced multi-label classification, and Ref. [24] developed specialized augmentation strategies for semi-supervised medical image segmentation.

Federated Semi-Supervised Learning. Federated semi-supervised learning (Fed-SSL) [25] extends traditional federated learning to scenarios characterized by limited labeled data and abundant distributed unlabeled data. Notable methods in this area include FedSem [26], which trains a global model on labeled data and incorporates unlabeled data using pseudo-labeling. Jeong et al. [27] introduced an inter-client consistency loss to align predictions across clients, enhancing the utilization of unlabeled data. Recently, Wang et al. [28] proposed a personalized Fed-SSL framework that leverages adaptive variance reduction and normalized aggregation to address client heterogeneity and improve convergence. Similarly, Liang et al. [29] developed RSCFed, which constructs multiple sub-consensus models by randomly sampling clients and then aggregates these sub-consensus models into the global model for improved reliability and performance. Ref. [30] proposed a two-stage sampling method that uses the predicted distribution changes of samples after different data augmentations.

Partial Client Participation. In federated learning (FL), clients may randomly join or leave the system, leading to a stochastic and time-varying set of active participants across communication rounds. For FL with non-i.i.d. datasets and partial client participation [31], several advancements have been made. Yang et al. [7] established a linear speedup in the convergence of FedAvg under partial client participation. Wang et al. [11] provided a unified convergence analysis for FL with arbitrary client participation, and Wang et al. [32] proposed FedAU, which adaptively weights client updates using online estimates of optimal weights without requiring prior knowledge of participation statistics. Ref. [33] considered a variance reduction algorithm applied at the server that eliminates error due to partial client participation.

Despite these efforts, most convergence analyses focus solely on traditional FL. In the context of federated semi-supervised learning, especially with non-i.i.d. datasets and partial client participation, the convergence behavior remains underexplored. A fundamental question arises: Can an algorithm in Fed-SSL still achieve the same linear speedup for convergence under non-i.i.d. data distributions and varying degrees of client participation? Addressing this challenge is crucial for extending the theoretical foundations of Fed-SSL and ensuring its robustness in practical applications.

Quasi-Monte Carlo Methods in Subsampling. The challenge of partial client participation in FL shares similarities with high-dimensional numerical integration. Quasi-Monte Carlo (QMC) methods [34], rooted in number-theoretic techniques [35], address this by constructing low-discrepancy sequences that achieve superior uniformity in sample distribution—akin to the uniform design in experimental design theory. Zhang et al. [36] and Zhou et al. [37] proposed an efficient model-free subsampling method based on uniform design, which generates representative subdata from the original dataset. This inspires us to similarly employ quasi-Monte Carlo approaches to address the partial client participation.

1.3. Contributions

In this paper, we tackle the federated semi-supervised learning problem with partial client participation, which introduces significant complexity compared to traditional federated optimization problems. Specifically, Fed-SSL involves two sets of optimization variables: the global model and the pseudo-labels, along with the added challenge of constraints. As a result, proving the convergence of algorithms for this problem is notably more difficult.

The main contributions of this paper are as follows:

• We propose an efficient federated learning algorithm, FedAvg-SSL, which incorporates partial client participation and alternates between updating the global model and refining pseudo-labels on local clients. The algorithm supports both uniform random sampling and a more structured lattice-based sampling strategy at the server side. While uniform sampling ensures simplicity and unbiased selection, the lattice-based approach offers more balanced client participation, which improves model stability under non-i.i.d. data.
• We establish a rigorous convergence analysis for FedAvg-SSL, demonstrating a sublinear convergence rate and linear speedup under partial client participation.
• Experimental results validate the performance of the proposed algorithm, showing its consistency with theoretical findings and illustrating the relationship between algorithm performance, the number of participating clients, and the number of local steps.

1.4. Organization

The rest of the paper is organized as follows. Section 2 introduces the federated SSL optimization problem. In Section 3, we present an efficient federated SSL algorithm that incorporates partial client participation via both uniform random sampling and a more structured lattice-based sampling strategy at the server side. In addition, the proposed algorithm alternates between updating the global model and refining pseudo-labels on local clients.

Section 4 provides a convergence rate analysis of the proposed algorithm. In Section 5, we discuss the practical implications of our results. Finally, we conclude in Section 6, with all proofs deferred to the appendix.

2. Problem Formulation

Consider a FL setting with a server and K distributed clients. We assume that both labeled data $\mathcal{L}$ and unlabeled data $\mathcal{U}$ are distributed across these K clients. Specifically, for each client k, the local dataset ${\mathcal{D}}_k$ consists of both labeled and unlabeled data, i.e.,

$${\mathcal{D}}_k={\mathcal{L}}_k\cup {\mathcal{U}}_k,$$

where ${\mathcal{L}}_k=\left\{(x_{k,i},y_{k,i}),i=1,\dots ,N_k\right\}$ is the local labeled dataset and ${\mathcal{U}}_k=\{u_{k,i},i=1,\dots ,M_k\}$ is the local unlabeled dataset. Here, $x_{k,i}$ is the i-th labeled sample for the k-th client with the corresponding label $y_{k,i}\in {\mathbb{R}}^C$, which is a one-hot vector representing the true class label, and C is the number of classes. Similarly, $u_{k,i}$ is the i-th unlabeled sample for the k-th client. The datasets ${\mathcal{D}}_k$, for $k\in [K]\triangleq \{1,\dots ,K\}$, are assumed to be non-overlapping. Additionally, $N_k$ and $M_k$ denote the number of labeled and unlabeled samples, respectively, for client k.

Let $\widehat{\mathbf{y}}=\{{\widehat{\mathbf{y}}}_1,\dots ,{\widehat{\mathbf{y}}}_K\}$ denote the collection of pseudo-labels across all clients, where ${\widehat{\mathbf{y}}}_k=\{{\widehat{y}}_{k,1},\dots ,{\widehat{y}}_{k,M_k}\}$ represents the set of pseudo-labels for the k-th client. Each pseudo-label ${\widehat{y}}_{k,i}\in {\mathbb{R}}^C$ belongs to the feasible set ${\mathcal{Y}}_k=\left\{{\widehat{\mathbf{y}}}_k\right|e^{\top }{\widehat{y}}_{k,i}=1,{\widehat{y}}_{k,i}\ge 0,i\in [M_k]\}$, ensuring proper probability distributions. Following [38], we formulate the Fed-SSL optimization problem as follows:

$$\begin{array}{cc}\hfill \underset{\begin{array}{c}\theta ,\widehat{\mathbf{y}}\end{array}}{min}& F(\theta ,\widehat{\mathbf{y}})\triangleq {\displaystyle \frac{1}{K}}\sum _{k=1}^K\left[\underset{\triangleq F_k(\theta ,{\widehat{\mathbf{y}}}_k)}{\underbrace{{\mathit{\ell }}_k(\theta ,{\widehat{\mathbf{y}}}_k)+{\alpha }_1{r}_1\left({\widehat{\mathbf{y}}}_k\right)}}\right]\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\mathrm{s}.\mathrm{t}.\hfill & {\widehat{\mathbf{y}}}_k\in {\mathcal{Y}}_k,k\in [K].\hfill \end{array}$$

(1b)

The loss function for each client consists of two components: a supervised loss on labeled data and a pseudo-labeled loss on unlabeled data, defined as follows: ${\mathit{\ell }}_k(\theta ,{\widehat{\mathbf{y}}}_k)=L_{\mathrm{CE}}(\theta ;{\mathcal{L}}_k)+{\alpha }_0{L}_{\mathrm{CE}}(\theta ;{\mathcal{U}}_k,{\widehat{\mathbf{y}}}_k)$ where the cross-entropy loss is given by the following:

$$\begin{array}{c}\hfill L_{\mathrm{CE}}(\theta ;\mathcal{L})=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N〈y_i,log\left(f_{\theta }\left(x_i\right)\right)〉.\end{array}$$

To prevent overconfident pseudo-labels, we introduce a regularization term $r_1\left({\widehat{\mathbf{y}}}_k\right)$:

$$\begin{array}{c}\hfill r_1\left({\widehat{\mathbf{y}}}_k\right)={\displaystyle \frac{1}{M_k}}\sum _{i=1}^{M_k}\mathrm{KL}({\widehat{y}}_{k,i},\mathbf{u}),\end{array}$$

(2)

where $\mathbf{u}=[{\displaystyle \frac{1}{C}},\dots ,{\displaystyle \frac{1}{C}}]\in {\mathbb{R}}^C$ represents a uniform distribution, and $\mathrm{KL}(·,·)$ denotes the Kullback–Leibler divergence.

The optimization problem in (1) presents two main challenges: (a) it involves two groups of variables ($\theta$ and $\widehat{\mathbf{y}}$), and (b) the pseudo-labels must satisfy probability simplex constraints. These characteristics naturally suggest an alternating optimization approach for updating these variables.

3. FedAvg-SSL Algorithm

In this section, we propose the FedAvg semi-supervised learning (FedAvg-SSL) algorithm as a solution approach to the optimization problem formulated in (1).

The FedAvg-SSL framework operates under the standard federated learning paradigm, where a central server coordinates training across K distributed clients. In each communication round t, the server selects a subset of clients ${\mathcal{S}}^t\subseteq \mathcal{K}=\{1,2,\dots ,K\}$ of fixed size M to participate in model updates. When $M=K$, all clients are involved in every round. In this section, we explore two distinct strategies for determining the active client subset ${\mathcal{S}}^t$ at each round t.

3.1. Random Sampling Method

The random sampling method in federated learning selects clients for each round independently and uniformly from the entire client. Let $\mathcal{K}=\{1,2,\dots ,K\}$ be the set of all clients. For each iteration $t\in \{0,1,\dots ,T-1\}$, we select a subset of clients ${\mathcal{S}}^t$ via uniform random sampling without replacement from set $\mathcal{K}$ with size M; then, the probability of any client being selected in a round is

$$\begin{array}{c}\hfill P(i\in {\mathcal{S}}^t)={\displaystyle \frac{M}{K}},\forall i=1,\dots ,K.\end{array}$$

Although each client has the same probability of being selected in each round, the random sampling method can lead to uneven client participation where some clients might be selected multiple times while others might never be selected, especially when the number of communication rounds is limited.

3.2. Lattice-Based Sampling Method

To improve participation uniformity, we propose a lattice-based sampling method that leverages a uniform design to construct a deterministic selection matrix. This method aims to ensure that all clients are sampled more evenly across communication rounds.

Let T be the number of total rounds and M be the number of clients selected per round. Typically, the selected subset size M is a divisor of the total client number K. Let $q=K/M$, then the set of all clients $\mathcal{K}=\{1,2,\dots ,K\}$ could be divided into M groups with q clients in each group: $\{1,2,\dots ,q\},\{q+1,q+2,\dots ,2q\},\dots ,\left\{\right(M-1)q+1,(M-1)q+2,\dots ,K\}$. Our selection strategy requires choosing exactly one client from each of M predefined groups, resulting in M selected clients per round. It is noticed that each group contains q distinct clients, which naturally evokes the concept of q-level uniform designs in experimental design theory. The uniform designs always have a good space-filling property on the experimental domain. We treat the M groups, each containing q distinct clients, as the value ranges for M factors in the experimental design framework. To ensure that all clients are sampled more evenly across T rounds, we perform sampling according to the rows of the uniform design.

Let $U=U_T\left(q^M\right)$ denote a uniform design comprising T runs for M factors, each having q levels, such that we

(1)

Let $U(t,k)\in \{1,2,\dots ,q\}$ denote the element in the t-th row and k-th column of the matrix U for all $t=0,\dots ,T-1,k=1,\dots ,K$;

(2)

Each level appears equally often in every column.

To obtain a uniform design $U=U_T\left(q^M\right)$, the first method is finding designs straightforwardly from the library of uniform designs given by [39], which does not require any calculations. If there is no such design in the library, two other methods can be used to construct such a design. Let $k\left(T\right)=\varphi \left(T\right)/2+1$, where $\varphi (·)$ is the Euler function. If $M<k(T+1)$, the leave-one-out good lattice point method combined with pseudo-level transformations [40] can be used to construct a nearly uniform design. For the other cases, we can use the R package UniDOE (proposed by [41]) on R version 3.4.1 to search for a nearly uniform design. Actually, the UniDOE package can be used to search for a nearly uniform design for arbitrary $T,M$. However, the UniDOE package is a little slower than the leave-one-out good lattice point method in general; then, we recommend to use the former only when $M\ge k(T+1)$.

Based on the uniform design U, the sampling design D is obtained by a deterministic offset

$$\begin{array}{c}\hfill D(t,k)=U(t,k)+(k-1)q,\forall t=0,\dots ,T-1,\forall k=1,\dots ,K.\end{array}$$

(3)

Then, all elements of the sampling design D are the actual client IDs. At each communication round t, one row of the matrix D is selected (either sequentially or randomly without replacement), and the resulting client indices form the active set ${\mathcal{S}}^t$.

This lattice-based scheme systematically spreads participation over time, significantly reducing the risk of client over-sampling or under-sampling (see Figure 1). As a result, it is particularly advantageous in federated semi-supervised learning scenarios, where balanced exposure to heterogeneous client data is crucial for effective model generalization.

3.3. The Proposed Federated SSL Algorithm

In this subsection, we present the proposed Fed-SHVR algorithm for solving problem (1), and its detailed pseudo code is presented in Algorithms 1 and 2. Since the considered problem (1) involves two blocks of variables $\theta$ and $\widehat{\mathbf{y}}$, we propose to train a global model for (1) in a novel way that combines an alternatively updating strategy with the local SGD algorithm.

Specifically, for each communication round, our algorithm has two parts: one is the client update and the other is the server update.

Client Side. After the server broadcasts the current global model ${\theta }^t$ to all active clients in ${\mathcal{S}}^t$, each active client k first updates its pseudo-labels by solving the following optimization problem:

$$\begin{array}{cc}\hfill {\widehat{\mathbf{y}}}_k^{t+1}={argmin}_{\begin{array}{c}{\widehat{\mathbf{y}}}_k\end{array}}& {\alpha }_0{L}_{\mathrm{CE}}({\theta }^t;u_k,{\widehat{\mathbf{y}}}_k)+{\alpha }_1{r}_1\left({\widehat{\mathbf{y}}}_k\right)\hfill \end{array}$$

(4a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& {\widehat{\mathbf{y}}}_k\in {\mathcal{Y}}_k.\hfill \end{array}$$

(4b)

Following [38], this optimization problem admits a closed-form solution:  

$$\begin{array}{c}\hfill {[{\widehat{y}}_{k,i}^{t+1}]}_j={\displaystyle \frac{{[f_{{\theta }^t}\left(u_{k,i}\right)]}_j^{\frac{{\alpha }_0}{{\alpha }_1}}}{{\sum }_{j=1}^C{[f_{{\theta }^t}\left(u_{k,i}\right)]}_j^{\frac{{\alpha }_0}{{\alpha }_1}}}},j=1,\dots ,C,\end{array}$$

(5)

where ${[{\widehat{y}}_{k,i}]}_j$ denotes the j-th entry of ${\widehat{y}}_{k,i}$. After updating the pseudo-labels, each client performs local model updates:

• Samples mini-batches of labeled data ${\xi }_k$ and unlabeled data ${\zeta }_k$ uniformly at random from ${\mathcal{L}}_k$ and ${\mathcal{U}}_k$, respectively.
• Computes the stochastic gradient of the local loss function

  $$\begin{array}{c}\hfill g_k({\theta }_k,{\widehat{\mathbf{y}}}_k):={\nabla }_{\theta }{F}_k({\theta }_k,{\widehat{\mathbf{y}}}_k;{\xi }_k,{\zeta }_k).\end{array}$$

  (6)
• Updates its local model copy ${\theta }_k$ using SGD.

Server Side. Each active client transmits its updated local model ${\theta }_{k,Q}$ back to the server, which aggregates these updates to compute the new global model ${\theta }^{t+1}$ via (7).

Algorithm 1 FedAvg SSL                                                       [Server]

1: Input: initial model parameters ${\theta }^0$. 2: for communication round $t=0$ to $T-1$ do 3:     Sample clients ${\mathcal{S}}^t$ uniformly randomly so that $|{\mathcal{S}}^t|=M$, or according to the sampling design D obtained by (3). 4:     for each client $k\in {\mathcal{S}}^t$ in parallel do 5:       Send ${\theta }^t$ to client k. 6:       Receive ${\theta }_k^{t+1}$ from client k via Algorithm 2. 7:     end for 8:     Update global model $$\begin{array}{c}\hfill {\theta }^{t+1}=(1-{\eta }_{\theta }){\theta }^t+{\displaystyle \frac{{\eta }_{\theta }}{M}}\sum _{k\in {\mathcal{S}}^t}{\theta }_k^{t+1}.\end{array}$$ (7) 9: end for

Algorithm 2 FedAvg SSL                       [Client]

1: Receive ${\theta }^t$ from the server. 2: Update pseudo-label ${\widehat{\mathbf{y}}}_k^{t+1}$ from (5). 3: Initialize ${\theta }_{k,0}^t={\theta }^t$. 4: for$q=0,\dots ,Q-1$do 5:     Select data ${\xi }_{k,q}^t$ and ${\zeta }_{k,q}^t$ uniformly at random from ${\mathcal{L}}_k$ and ${\mathcal{U}}_k$. 6:     Update local client model $$\begin{array}{c}\hfill {\theta }_{k,q+1}^t={\theta }_{k,q}^t-{\eta }_g{g}_k({\theta }_{k,q}^t,{\widehat{\mathbf{y}}}_k^{t+1}).\end{array}$$ (8) 7: end for 8: Set ${\theta }_k^{t+1}={\theta }_{k,Q}^t$. 9: Send ${\theta }_k^{t+1}$ back to the server.

4. Convergence Analysis with Partial Client Participation

In this section, we establish the convergence properties of FedAvg-SSL when only a subset of clients updates their variables per outer loop.

4.1. Assumptions

We first make some standard assumptions.

Assumption 1.

The regularization term $r_1\left(\widehat{\mathbf{y}}\right)$ is a continuously differentiable function. Furthermore, $r_1\left(\widehat{\mathbf{y}}\right)$ is μ-strongly convex, where $\mu >0$. Specifically, for any ${\widehat{\mathbf{y}}}_1,{\widehat{\mathbf{y}}}_2\in {\mathcal{Y}}_k$, the following inequality holds:

$$\begin{array}{c}\hfill r_1\left({\widehat{\mathbf{y}}}_1\right)-r_1\left({\widehat{\mathbf{y}}}_2\right)-\langle \nabla r_1\left({\widehat{\mathbf{y}}}_2\right),{\widehat{\mathbf{y}}}_1-{\widehat{\mathbf{y}}}_2\rangle \ge {\displaystyle \frac{\mu }{2}}{\parallel {\widehat{\mathbf{y}}}_1-{\widehat{\mathbf{y}}}_2\parallel }^2.\end{array}$$

(9)

The regularization term $r_1\left({\widehat{\mathbf{y}}}_k\right)$ defined in (2) is strongly convex over the probabilistic simplex ${\mathcal{Y}}_k$ (see [42], Definition 2 and Example 2). However, it is important to note that the objective function $F_k(\theta ,{\widehat{\mathbf{y}}}_k)$ in (1a) is not jointly convex with respect to $(\theta ,\widehat{\mathbf{y}})$.

Assumption 2.

The local cost $F_k(\theta ,{\widehat{\mathbf{y}}}_k)$ is L-smooth (possibly non-convex) with respect to $(\theta ,{\widehat{\mathbf{y}}}_k)$ for $k\in [K]$, i.e.,

$$\begin{array}{c}\hfill \parallel {\nabla }_{\theta }{F}_k(\theta ,{\widehat{\mathbf{y}}}_k)-{\nabla }_{\theta }{F}_k({\theta }^{\prime },{\widehat{\mathbf{y}}}_k^{\prime })\parallel \le L\sqrt{\parallel \theta -{\theta }^{\prime }{\parallel }^2+{\parallel {\widehat{\mathbf{y}}}_k-{\widehat{\mathbf{y}}}_k^{\prime }\parallel }^2},\end{array}$$

(10)

for all $\theta ,{\theta }^{\prime }$ and ${\widehat{\mathbf{y}}}_k,{\widehat{\mathbf{y}}}_k^{\prime }\in {\mathcal{Y}}_k$.

Assumption 3.

Given ${\widehat{\mathbf{y}}}_k$, for any $k\in [K]$, the stochastic gradient is unbiased for any θ. Specifically, the following conditions hold:

$$\begin{array}{cc}\hfill & \mathbb{E}[g_k({\theta }_k,{\widehat{\mathbf{y}}}_k)]={\nabla }_{\theta }{F}_k({\theta }_k,{\widehat{\mathbf{y}}}_k),\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill & \mathbb{E}[\parallel g_k({\theta }_k,{\widehat{\mathbf{y}}}_k)-{\nabla }_{\theta }{F}_k({\theta }_k,{\widehat{\mathbf{y}}}_k){\parallel }^2]\le {\sigma }_{\theta }^2,\hfill \end{array}$$

(12)

where ${\sigma }_{\theta }$ represents the noise variance, and $\mathbb{E}$ denotes the expectation with respect to all random variables $\{{\xi }_k,{\zeta }_k\}$.

Assumption 4

(Bounded Gradient Dissimilarity). There exists a constant ${\sigma }_G>0$ such that for any θ and $\widehat{\mathbf{y}}$, the following inequality holds:

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{K}}\sum _{k=1}^K{\parallel {\nabla }_{\theta }{F}_k(\theta ,{\widehat{\mathbf{y}}}_k)-{\nabla }_{\theta }F(\theta ,\widehat{\mathbf{y}})\parallel }^2\le {\sigma }_G^2.\end{array}$$

(13)

Assumptions 2 and 3 are standard in first-order stochastic algorithms, while Assumption 4 constrains the data heterogeneity by bounding the gradient dissimilarity between different clients, which is commonly used in [7,11,28].

4.2. Convergence Analysis of FedAvg-SSL

From (7) and (8), the update for the global model across consecutive outer loops is given by

$$\begin{array}{cc}\hfill {\theta }^{t+1}-{\theta }^t& ={\displaystyle \frac{{\eta }_{\theta }}{M}}\sum _{k\in {\mathcal{S}}^t}({\theta }_k^{t+1}-{\theta }^t)=-{\displaystyle \frac{\eta }{M}}\sum _{k,q}{\mathbb{I}}_{k\in {\mathcal{S}}^t}{g}_k({\theta }_{k,q}^t,{\widehat{\mathbf{y}}}_k^{t+1}),\hfill \end{array}$$

(14)

where $\eta ={\eta }_{\theta }{\gamma }_{\theta }$ and ${\mathbb{I}}_{k\in {\mathcal{S}}^t}$ denote the characteristic function of the set ${\mathcal{S}}^t$, representing the subset of clients participating in the current round.

A critical challenge arises because the right-hand side of the above equation does not provide an unbiased estimation of the full gradient. Specifically, we have

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{M}}\sum _{k,q}\mathbb{E}[{\mathbb{I}}_{k\in {\mathcal{S}}^t}{g}_k({\theta }_{k,q}^t,{\widehat{\mathbf{y}}}_k^{t+1})]\ne \nabla F(\theta ,{\mathbf{y}}^{t+1}),\end{array}$$

(15)

where $\nabla F(\theta ,{\mathbf{y}}^{t+1})$ is the gradient of the full objective function.

This discrepancy stems from the stochastic nature of partial client participation, as only a subset of clients ${\mathcal{S}}^t$ is actively involved in each communication round. Consequently, the aggregated gradient ${\displaystyle \frac{1}{M}}{\sum }_{k,q}{\mathbb{I}}_{k\in {\mathcal{S}}^t}{g}_k$ introduces bias, which complicates the convergence analysis. Specifically, controlling this bias becomes essential to establish rigorous convergence guarantees for FedAvg-SSL under partial client participation.

Before we show the convergence, let us first define the following optimality gap:

$$\begin{array}{c}\hfill g_t\triangleq \mathbb{E}[\parallel {\nabla }_{\theta }F({\theta }^t,{\widehat{\mathbf{y}}}^{t+1}){\parallel }^2]+{\displaystyle \frac{1}{K}}\sum _{k=1}^K\mathbb{E}[\parallel {\widehat{\mathbf{y}}}_k^{t+1}-{\widehat{\mathbf{y}}}_k^t{\parallel }^2].\end{array}$$

(16)

which measures how far an iteration point is away from the stationary point. From the definition, we know $g_t\ge 0$. If $g_t=0$, the iterate $\{{\theta }^t,{\widehat{\mathbf{y}}}^t\}$ will be a stationary point of problem (1). According to (16), we have

$$\begin{array}{c}\hfill {\nabla }_{\theta }F({\theta }^t,{\widehat{\mathbf{y}}}^t)=0,{\widehat{\mathbf{y}}}_k^{t+1}-{\widehat{\mathbf{y}}}_k^t=0.\end{array}$$

(17)

Since ${\widehat{y}}_k^{t+1}$ is the optimal solution of $F_k({\theta }^t,\widehat{\mathbf{y}})$, from the first order optimality condition, we get

$$\begin{array}{c}\hfill \langle {\nabla }_{{\widehat{\mathbf{y}}}_k}{F}_k({\theta }^t,{\widehat{\mathbf{y}}}_k^{t+1}),{\widehat{\mathbf{y}}}_k-{\widehat{\mathbf{y}}}_k^{t+1}\rangle \ge 0,\mathrm{for}\forall {\widehat{\mathbf{y}}}_k\in {\mathcal{Y}}_k.\end{array}$$

(18)

Combing (17) and (18), for $\forall \theta ,{\widehat{\mathbf{y}}}_k\in {\mathcal{Y}}_k$, we have

$$\begin{array}{c}\hfill \langle {\nabla }_{\theta }F({\theta }^t,{\widehat{\mathbf{y}}}^t),\theta -{\theta }^t\rangle +\sum _{k=1}^K\langle {\nabla }_{{\widehat{\mathbf{y}}}_k}{F}_k({\theta }^t,{\widehat{\mathbf{y}}}_k^t),{\widehat{\mathbf{y}}}_k-{\widehat{\mathbf{y}}}_k^t\rangle \ge 0.\end{array}$$

The above inequality implies that $\{{\theta }^t,{\widehat{\mathbf{y}}}^t\}$ is a stationary point to problem (1).

Theorem 1.

If ${\eta }_g^2\le {\displaystyle \frac{1}{64Q(Q-1)L^2}}$ and $\eta :={\eta }_g{\eta }_{\theta }\le {\displaystyle \frac{M(K-1)}{(3+12Q)QL(K-M)}}$, we have that the sequence $\{{\theta }^t,{\widehat{\mathbf{y}}}^t\}$ generated by Algorithm 1 satisfies

$$\begin{array}{cc}\hfill {\displaystyle \frac{1}{T}}\sum _{t=0}^{T-1}{g}_t& \le {\displaystyle \frac{4\mathbb{E}[F({\theta }^0,{\widehat{\mathbf{y}}}^0)]}{\eta QT}}+{\eta }_g^2{B}_1+4\eta {\eta }_g^2{B}_2+4\eta B_3,\hfill \end{array}$$

(19)

where

$$\begin{array}{cc}{B}_1\hfill & =L^2\left(Q(Q-1)\right){\sigma }_G^2+2L^2(Q-1){\sigma }_{\theta }^2,\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill B_2& ={\displaystyle \frac{24Q^3{L}^3(K-M)}{M(K-1)}}{\sigma }_G^2+{\displaystyle \frac{6Q(Q-1)L^3(K-M)}{M(K-1)}}{\sigma }_{\theta }^2,\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill B_3& ={\displaystyle \frac{3LQ(K-M)}{2M(K-1)}}{\sigma }_G^2+{\displaystyle \frac{L}{2M}}{\sigma }_{\theta }^2.\hfill \end{array}$$

(22)

Proof. 

See Appendix A. □

Remark 1.

Theorem 1 provides an explicit upper bound on the optimality gap, which offers useful insights into how several key factors influence the convergence behavior of the proposed FedAvg-SSL algorithm:

• Number of participating clients (M): When all clients participate in each round, i.e., $M=K$, the term $B_2$ in the bound becomes zero. As a result, the convergence rate improves, demonstrating the benefit of increased client participation.
• Number of local steps (Q): As Q increases, the accumulated local update noise can affect convergence. Theorem 1 suggests that in such cases, the global and local learning rates η and ${\eta }_g$ should be chosen smaller to maintain stability and ensure convergence.

These observations provide practical guidance for selecting different parameters in FedAvg-SSL.

With appropriate step sizes, we can derive the following corollary.

Corollary 1.

Let ${\eta }_g={\displaystyle \frac{1}{\sqrt{T}QL}}$ and $\eta ={\displaystyle \frac{\sqrt{M}}{\sqrt{TQ}L}}$; the convergence rate of the FedAvg-SSL algorithm under partial client participation is given by

$$\begin{array}{cc}\hfill \underset{t=0,\dots ,T-1}{min}{g}_t& \le \mathcal{O}\left({\displaystyle \frac{1}{\sqrt{MTQ}}}+{\displaystyle \frac{1}{T}}\right).\hfill \end{array}$$

(23)

Remark 2.

Corollary 1 demonstrates that the required number of outer loop iterations T to achieve an ϵ-solution decreases linearly as M increases. This phenomenon is commonly referred to as linear speedup with respect to M. To the best of our knowledge, this is the first work to establish the linear speedup of a federated semi-supervised learning algorithm.

5. Simulation Results

We conducted extensive experiments on the MNIST dataset to evaluate the effectiveness of our proposed federated semi-supervised learning approach. The experiments were implemented using PyTorch version 1.8 on a CUDA-enabled GPU, utilizing the MNIST dataset which contains 60,000 training images and 10,000 test images of handwritten digits.

To simulate a realistic federated learning scenario, we configured our system with 100 clients $(K=100)$ and implemented a partial participation strategy where only 10% of clients (10 clients) participate in each federation round. The dataset was divided into labeled and unlabeled portions, with 30% of the data being labeled and the remaining 70% unlabeled. We investigated two distinct data distribution scenarios:

• IID Setting: The training data is randomly and uniformly distributed across clients, ensuring each client receives approximately equal amounts of both labeled and unlabeled samples. This setting serves as our baseline distribution.
• Non-IID Setting: Labeled data is randomly shuffled and split into twice as many shards as there are clients, with each client randomly assigned two unique shards. This ensures that each client receives a distinct, non-overlapping subset of the labeled data, with each shard containing a random mix of classes. The unlabeled data is distributed using a Dirichlet distribution ($\alpha =0.01$) to further enhance the non-IID nature of the data across clients.

For our model architecture, we employed a simple multi-layer perceptron (MLP) with one hidden layer containing 256 units and ReLU activations. The training process utilized a mini-batch size of 32 for both labeled and unlabeled data, with learning rates set to ${\eta }_g={\eta }_{\theta }=0.01$. Each participating client performed 10 local epochs during their training phase.

Impact of hyperparameters ${\alpha }_0$ and ${\alpha }_1$. The hyperparameter ${\alpha }_0$ controls the model’s reliance on the unlabeled data during training. To prevent the model from prematurely overfitting to noisy pseudo-labels, we adopt a gradually increasing schedule for ${\alpha }_0$, inspired by the work in [16]. Specifically, we define

$${\alpha }_0=\left\{\begin{array}{cc}exp\left(-5{\left(1-{\displaystyle \frac{t}{T_1}}\right)}^2\right),\hfill & \mathrm{if}t<T_1\hfill \\ \\ 1,\hfill & \mathrm{if}t\ge T_1.\hfill \end{array}\right.$$

The second hyperparameter, ${\alpha }_1$, serves as a regularization weight in the pseudo-label refinement process. It controls the entropy of the pseudo-labels: smaller values encourage confident (i.e., lower-entropy) predictions, while larger values encourage more conservative updates by penalizing overconfidence. Thus, tuning ${\alpha }_1$ balances the trade-off between exploration and exploitation of pseudo-labels.

To investigate the impact of these hyperparameters, we conduct a sensitivity analysis and summarize the results in the table below. The test accuracy is evaluated under different combinations of $T_1$ and ${\alpha }_1$ in the non-IID setting. As shown in

Table 1. Test accuracy of the proposed algorithm by using different parameters ${\alpha }_1$ and ${\alpha }_2$ on the non-IID case.

	${\mathit{T}}_1=10$	${\mathit{T}}_1=50$
${\alpha }_1=0$	96.91%	97.21%
${\alpha }_1=0.5$	97.5%	96.89%
${\alpha }_1=1$	96.71%	96.76%

, the highest accuracy of 97.5% occurs when $T_1=10$ and ${\alpha }_1=0.5$, outperforming other configurations. Based on these results, we adopt this parameter configuration for all subsequent experiments. Figure 2 further illustrates the effects of these parameters on training dynamics.

Impact of non-IID datasets. As illustrated in Figure 3a, we present comprehensive experimental results comparing the performance of federated semi-supervised learning under both IID and non-IID data distributions. The test accuracy curves in Figure 3a demonstrates that the non-IID data distribution significantly impacts the model’s convergence and final performance. Under the IID setting with SSL, the model achieves the highest accuracy of approximately 83%, while the same approach under non-IID conditions reaches only 78%, indicating a clear performance degradation due to data heterogeneity.

A key observation from the loss curves in Figure 3b is that the non-IID data distribution notably slows down the convergence rate. The IID+SSL configuration shows the fastest convergence and reaches the lowest loss value of approximately 1.15, while its non-IID counterpart maintains a consistently higher loss of around 1.4. This pattern aligns with theoretical expectations that higher degrees of data heterogeneity lead to slower convergence and higher final loss values.

Furthermore, the impact of non-IID data is evident in both SSL and supervised-only approaches, though with varying degrees. The performance gap between IID and non-IID settings is more pronounced in the SSL approach (5% difference) compared to the supervised-only setting (1% difference). This observation suggests that while SSL techniques can effectively leverage unlabeled data, their benefits are somewhat diminished under non-IID conditions, highlighting the fundamental challenges that data heterogeneity poses in federated learning scenarios.

Impact of label ratios. To investigate how the proportion of labeled data affects model performance, we conducted experiments with varying label ratios. Figure 4 presents the test accuracy and loss curves for different label ratios ranging from 0.1 to 0.9.

Figure 4a demonstrates a clear correlation between label ratio and model performance. With a high label ratio of 0.9, the model achieves the best performance, reaching approximately an 85% accuracy after 100 epochs. The convergence is also notably faster, showing steep improvement in the early stages (20–40 epochs). In contrast, with only 10% labeled data (ratio 0.1), the model struggles significantly, achieving only about 62% accuracy and showing much slower convergence throughout the training process. The loss curves in Figure 4b further reinforce these findings. Models trained with higher label ratios (0.7 and 0.9) demonstrate more efficient optimization, reaching lower loss values (around 0.9–1.0) and showing steeper descent curves.

Impact of Local Training Steps. To validate Theorem 1 and investigate the effect of local training steps on Fed-SSL’s performance under non-IID settings, we conducted experiments with varying numbers of local steps (5, 10, 20, and 30). Figure 5 presents the experimental results.

The experimental results strongly align with our theoretical analysis in Theorem 1 regarding the convergence of Fed-SSL. As shown in Figure 5a, increasing the number of local steps significantly improves both convergence speed and final model performance. With 30 local steps, the model achieves the highest accuracy of approximately 85% and demonstrates the fastest convergence rate, particularly during the early stages (20–40 communication rounds). In contrast, with only five local steps, the model shows slower convergence and reaches a lower final accuracy of about 73%.

The loss curves in Figure 5b further validate our theoretical findings. Models trained with more local steps (20 and 30) exhibit more efficient optimization trajectories, achieving lower final loss values (around 0.9–1.0) and showing steeper descent curves. This observation is consistent with Theorem 1, which suggests that increasing local training steps can enhance convergence stability and final performance, provided the learning rate is properly chosen.

Impact of Client Number. As shown in Figure 6, we compare the test accuracy and loss curves among three different participation rates: full worker participation ($K=100$) and partial worker participation ($M=30$ and $M=10$) under the same hyperparameter settings. The results demonstrate that full worker participation consistently outperforms partial participation scenarios, achieving approximately 83% accuracy compared to 82.5% and 82% for 10% and 30% participation rates, respectively. This performance gap can be attributed to the additional randomness introduced by partial worker participation, resulting in slower convergence rates as evidenced by the more gradual slopes of both accuracy and loss curves.

This phenomenon is particularly pronounced in our non-IID setting, where system heterogeneity plays a crucial role. Full worker participation effectively neutralizes this heterogeneity in each communication round by incorporating updates from all clients. In contrast, partial participation struggles to maintain this balance, as the selected subset of workers may not adequately represent the overall data distribution. For instance, with only 10 or 30 active workers, the available training data might cover only a subset of the MNIST dataset’s 10 digit classes, leading to biased model updates. This effect is clearly visible in both the accuracy and loss trajectories, where partial participation schemes exhibit consistently inferior performance throughout the training process.

Impact of Sampling Strategies. As shown in Figure 7, the lattice-based sampling consistently outperforms random sampling in both test accuracy and test loss. The reason may be that the lattice-based sampling method ensures that all clients contribute equally across communication rounds, effectively mitigating the bias introduced by data heterogeneity. In contrast, random sampling may result in some clients being over- or under-represented, leading to less representative model updates (see Figure 1).

6. Conclusions

In this paper, we propose an efficient FedAvg-SSL algorithm that alternately updates model parameters and pseudo-labels in non-IID federated settings for general non-convex optimization. To address the challenge of partial client participation, we introduce and compare two sampling strategies—uniform random sampling and lattice-based sampling—at the server side. The lattice-based method is shown to enhance participation balance and reduce selection variance across communication rounds. We provide a rigorous theoretical analysis demonstrating that FedAvg-SSL achieves a sublinear convergence rate with linear speedup under partial participation. Extensive experiments on the MNIST dataset support our theoretical claims and offer practical insights into how sampling strategy, participation rate, and local update steps jointly influence learning performance. These results provide valuable guidance for the design of efficient and robust federated semi-supervised learning systems. In future work, we plan to explore fairness-aware QMC-based client selection strategies, building on insights from [43], to ensure more equitable participation and performance across clients.