Federated and Differentially Private Learning for Root Cause Classification Across Heterogeneous Semiconductor Fabs

Abstract

Federated learning (FL) can train a shared model without pooling client records, but it does not by itself provide differential privacy (DP). We compare centralized training, local-only training, federated averaging (FedAvg), and client-level DP-FedAvg on a fully released synthetic benchmark of free-text root-cause-analysis (RCA) records from six simulated semiconductor fabs. The benchmark uses a fixed public vocabulary, 240 records per client, a 480-record in-distribution test set, a separately authored 480-record template-disjoint test set, and five training seeds. All methods receive 24 passes over their available training data. The reference classifier is an auditable multinomial softmax regression with 512 parameters. The DP mechanism assumes fixed-roster replace-one client adjacency, clips only private client updates, adds Gaussian noise calibrated to the sensitivity of the weighted aggregate, and composes twelve releases with Rényi DP. In-distribution accuracy was 53.94% [53.60, 54.27] (mean, 95% confidence interval) for local-only training, 74.71% [73.05, 76.37] for FedAvg, and 91.88% [90.62, 93.13] for centralized training. Template-disjoint accuracy was 19–24% for all three. DP-FedAvg reached 65.92% [61.40, 70.44] at $\epsilon \approx 204$ and fell to chance level at $\epsilon \approx 3.6$ ($\delta ={10}^{-5}$). These results characterize a controlled benchmark and do not establish applicability to real fabs.

1. Introduction

Root cause analysis (RCA) records in semiconductor manufacturing are short free-text narratives that describe an excursion, the investigation, and the assigned cause. They can contain proprietary process, equipment, and yield information, which makes pooling them across sites or companies difficult. Federated learning (FL) addresses part of this problem: federated averaging (FedAvg) trains a shared model by aggregating locally computed model updates, so raw records stay at each site [1,2]. Model updates can still leak information about the underlying data, so FL is commonly combined with differential privacy (DP) [3,4,5] and secure aggregation [6].

This study asks a narrow question. Under controlled, fully documented conditions, how do centralized, local-only, FedAvg, and client-level DP-FedAvg training compare on a multi-site RCA text classification task? How sensitive are they to label skew across sites and to a change in the wording templates of the records? All records are synthetic, and no claim is made about performance on real fab data.

1.1. Related Work

1.1.1. Federated Optimization Under Heterogeneous Data

FedAvg [1] is an established aggregation method, not a contribution of this work. Its accuracy can deteriorate when clients hold different label or feature distributions. FedProx [7] and SCAFFOLD [8] are among the methods proposed to reduce the resulting client drift. Sampling client class proportions from a symmetric Dirichlet distribution is a common way to create controlled label skew [9]. We use plain FedAvg so that the effect of the controlled partitions can be observed without additional optimizer components.

1.1.2. Client-Level Differential Privacy and Secure Aggregation

Example-level DP, as in differentially private stochastic gradient descent (DP-SGD) [10], bounds the influence of a single training record. Client-level DP bounds the influence of an entire client’s dataset, which is the relevant unit when a client is a fab [4,5]. A valid client-level guarantee requires four things: a defined adjacency relation, a bound on each client’s contribution (usually by clipping its update), noise calibrated to the sensitivity of the released aggregate, and composition across all releases. Rényi DP (RDP) gives a convenient composition rule for repeated Gaussian releases [11]. Secure aggregation [6] is complementary. It can prevent the server from seeing individual updates, but it does not provide a DP guarantee for the released model.

1.1.3. Federated Learning in Semiconductor Manufacturing

Zhang et al. evaluated decentralized FL for wafer-defect-pattern recognition with image data [12]. Patel et al. combined FL with explainability for semiconductor fault detection using structured fault data [13]. These studies address visual or structured inputs. The present task instead classifies free-text RCA narratives into eight root-cause categories.

1.2. Contribution and Scope

The contribution is a controlled, reproducible application benchmark, not a new FL or DP algorithm. Specifically, we provide the following:

• A released synthetic six-client RCA text benchmark with documented generator, seeds, Dirichlet partitions at three skew levels, and a separately authored template-disjoint test set.
• A matched-budget comparison of centralized, local-only, FedAvg, and DP-FedAvg training with five-seed uncertainty.
• An explicit client-level DP derivation (adjacency, sensitivity, noise calibration, and RDP composition) with the corresponding implementation, and a privacy–utility sweep over eight noise multipliers.
• A reproducibility package in which every reported number is regenerated from released raw per-run outputs by a single script, with automated consistency checks.

2. Materials and Methods

2.1. Synthetic Data Generation

Each record is a short narrative built from a symptom frame for one of eight root-cause classes (equipment drift, contamination, recipe deviation, consumable degradation, calibration error, material defect, human procedural error, environmental fluctuation). The frame is combined with an optional opener, an optional closer, and a lot/wafer tag. Frame slots are filled with process parameters, consumables, process steps, and tool identifiers. Six simulated fabs differ in process-step emphasis and in tool-identifier ranges. Each fab acts as one FL client, so the terms fab and client are used interchangeably below. The in-distribution (ID) frame bank contains 41 symptom frames, 6 openers, and 4 closers.

For each client and each Dirichlet concentration $\alpha \in \{100,1,0.1\}$, an eight-class probability vector was drawn from $\mathrm{Dir}\left(\alpha {\mathbf{1}}_8\right)$, and 240 labels were sampled from it [9]. Each client’s pool was split into 204 training and 36 validation records. An incorrect-class label-noise rule (probability 0.06, uniformly over the other seven classes) was applied to training and validation records only. Partitions use seeds 42100 ($\alpha =100$), 42001 ($\alpha =1$), and 42010 ($\alpha =0.1$). The main experiments use $\alpha =1$, for which the largest class held 38–72 of a client’s 204 training records.

2.2. Evaluation Sets and Vocabulary

Two class-balanced test sets of 480 records (60 per class) were generated with seed 42017 and contain no label noise. The ID test set uses the training frame bank. The template-disjoint test set uses a separately authored bank of 40 symptom frames, 4 openers, and 3 closers, none of which appears in the ID bank. It shares only slot values (parameters, consumables, steps, and tool identifiers) with the training data. This set probes transfer to unseen phrasing within the same synthetic domain. It is not a model of deployment-level distribution shift. Automated checks confirm zero record-identifier overlap, zero exact-text overlap across all generated sets, and zero frame overlap between the two banks.

Features are computed from a fixed public vocabulary of 63 tokens. The tokens are the eight class names, the process-step names, and generic domain terms, taken from a public corpus file that was released with the previous version of the package. The vocabulary builder reads only this file. No training, validation, or test file is read during vocabulary construction. The template-disjoint bank was written after the vocabulary had been fixed. Test records contain on average 4.1 (ID) and 2.1 (template-disjoint) in-vocabulary tokens.

2.3. Classifier and Training Budget

The reference classifier is multinomial softmax regression. Its input is the $L_2$-normalized vector of public-vocabulary token counts plus an intercept, giving 64 input features, 8 classes, and 512 parameters. We chose this model because it is small enough to audit completely, trains deterministically in seconds, and makes the DP noise-to-signal ratio easy to analyze (Section 4). Results should not be extrapolated to pretrained language models.

All methods use mini-batch stochastic gradient descent (batch size 32, learning rate 0.5, initialization $\mathcal{N}(0,{0.01}^2)$) and cross-entropy loss on the observed labels. These hyperparameters were fixed before any experiment and were not tuned on validation or test data. Validation accuracy is recorded for diagnostics only. It is not used for early stopping, hyperparameter selection, or checkpoint selection.

• Centralized: The 1224 training records of all six clients ($\alpha =1$) are pooled and trained for 24 epochs.
• Local-only: Each client trains a separate model on its own 204 training records for 24 epochs. We report the mean accuracy over the six client models.
• FedAvg: $T=12$ communication rounds with 2 local epochs per round and full participation.

Each method therefore makes 24 passes over its available training records. This matches data exposure, not wall-clock time, communication, or optimization effort.

2.4. FedAvg and Client-Level DP-FedAvg

Let $D=(D_1,\dots ,D_K)$ be a fixed roster of $K=6$ clients with public, fixed aggregation weights $w_k=n_k/{\sum }_j{n}_j$. Here $n_k=204$ for every client, so $w_k=1/6$. In round t, client k starts from the global parameters ${\theta }_{t-1}$, runs local training, and returns the update $u_k={\theta }_k-{\theta }_{t-1}\in {\mathbb{R}}^d$ with $d=512$. Ordinary FedAvg sets ${\theta }_t={\theta }_{t-1}+{\sum }_k{w}_k{u}_k$ and does not clip client updates.

For DP-FedAvg, two federated datasets are adjacent if the roster and weights are unchanged and all records of exactly one client may be replaced. Each update is clipped to $L_2$ norm at most C:

$${\overline{u}}_k=u_k\min \left(1,{\displaystyle \frac{C}{{\parallel u_k\parallel }_2}}\right).$$

(1)

Replacing one client changes the weighted aggregate $q\left(D\right)={\sum }_k{w}_k{\overline{u}}_k$ by at most $w_k{\parallel {\overline{u}}_k-{\overline{u}}_k^{\prime }\parallel }_2\le 2C{w}_k$, so its $L_2$ sensitivity is

$${\Delta }_2\left(q\right)\le 2C\underset{k}{\max }{w}_k.$$

(2)

The server releases

$$\tilde{q}\left(D\right)=q\left(D\right)+Z,Z_j\stackrel{\mathrm{iid}}{\sim }\mathcal{N}\left(0,{\left(\sigma {\Delta }_2\right)}^2\right),j=1,\dots ,d,$$

(3)

and sets ${\theta }_t={\theta }_{t-1}+\tilde{q}\left(D\right)$. The noise standard deviation is not divided by the model dimension. With $C=1$ and $w_k=1/6$, ${\Delta }_2=1/3$. A single release is $(\lambda ,\lambda /\left(2{\sigma }^2\right))$-RDP for every order $\lambda >1$ [11]. All clients participate in every round, so no subsampling amplification is used. Composing T releases and converting to $(\epsilon ,\delta )$-DP gives

$$\epsilon =\underset{\lambda \in \{2,\dots ,256\}}{\min }\left[{\displaystyle \frac{T\lambda }{2{\sigma }^2}}+{\displaystyle \frac{\log (1/\delta )}{\lambda -1}}\right],T=12,\delta ={10}^{-5},$$

(4)

where T is the number of noisy aggregate releases (communication rounds). We evaluate $\sigma \in \{0.25,0.5,0.75,1,1.5,2,3,5\}$. The guarantee covers the sequence of released aggregates, and therefore the final model, under the stated adjacency. It does not cover data-dependent weights, add/remove adjacency, or the individual updates observed by the server.

2.5. Statistical Analysis and Communication Accounting

Every configuration is run with seeds 0–4. The seed controls parameter initialization, mini-batch order, and DP noise. The generated data are identical across seeds, so the intervals reflect training randomness only and not data-sampling variability. We report the mean and a two-sided 95% Student-t interval, $\overline{x}\pm t_{0.975,4}s/\sqrt{5}$, where s is the sample standard deviation. No hypothesis test was prespecified, and no significance or equivalence claims are made.

Communication is counted as transmission of the full dense float32 parameter vector in both directions, for every client and round. The comparator is a one-time upload of the UTF-8 bytes of the raw text field of the training records. Both counts exclude metadata, protocol overhead, compression, quantization, and sparsification.

2.6. Reproducibility Package

The released package contains the generator, both frame banks, the public vocabulary corpus, all generated splits and partition metadata, the experiment code, the 65 per-run result files (13 configurations × 5 seeds), the aggregation script that computes every table entry, the figure script, automated checks, and a SHA-256 manifest. One command regenerates all data, results, tables, and figures on a CPU in under one minute. The checks recompute every $\epsilon$, verify ${\Delta }_2$ and the noise scale in every DP run, confirm that ordinary FedAvg records no clipping bound, and verify the split sizes and overlap properties described above.

3. Results

All accuracies are the percentage of correctly classified test records, reported as the mean over five seeds with a 95% confidence interval in brackets. Both test sets are class-balanced, so chance accuracy is 12.5%. Section 3.1 compares the non-private training regimes, Section 3.2 reports the privacy–utility trade-off, Section 3.3 examines label skew, and Section 3.4 reports communication cost.

3.1. Centralized, Local-Only, and Federated Training

Table 1 reports accuracy on both test sets for $\alpha =1$. FedAvg exceeded local-only training on the ID test set by about 21 percentage points and remained about 17 points below centralized training. On the template-disjoint test set, all three methods fell to 19–24%, only modestly above the 12.5% chance level. The ID ranking was not clearly preserved there: the FedAvg and centralized intervals overlap almost completely.

Table 1. Accuracy for $\alpha =1$ (mean over five seeds; brackets give 95% confidence intervals). Local-only values are the mean over the six client models.

Method	In-Distribution	Template-Disjoint
Local-only	53.94% [53.60, 54.27]	19.43% [17.55, 21.31]
FedAvg	74.71% [73.05, 76.37]	23.46% [22.24, 24.68]
Centralized	91.88% [90.62, 93.13]	23.33% [22.62, 24.04]

3.2. Privacy–Utility Trade-Off

Table 2 and Figure 1 report DP-FedAvg at eight noise multipliers. ID accuracy decreased steadily as $\epsilon$ decreased. It went from 65.92% [61.40, 70.44] at $\epsilon =203.51$ ($\sigma =0.25$), about 9 points below non-private FedAvg, to 32.17% [26.63, 37.70] at $\epsilon =23.51$ and 12.88% [9.95, 15.80] at $\epsilon =3.56$, which is indistinguishable from chance. The decline is gradual across the tested range, and the data do not indicate a sharp threshold. Template-disjoint accuracy was already low without DP and declined toward chance as noise increased. The fraction of client updates reaching the clipping bound in the final round rose from about two-thirds at $\sigma =0.25$ to all updates for $\sigma \ge 2$. This is consistent with noisier global models producing larger local corrections.

Table 2. DP-FedAvg at $\delta ={10}^{-5}$, $C=1$, $T=12$, $\alpha =1$ (mean over five seeds; brackets give 95% confidence intervals). “Clipped” is the mean fraction of client updates with ${\parallel u_k\parallel }_2>C$ in the final round.

$\mathit{\sigma }$	$\mathit{\epsilon }$	Clipped	In-Distribution	Template-Disjoint
0.25	203.51	0.67	65.92% [61.40, 70.44]	22.25% [19.87, 24.63]
0.50	59.51	0.73	54.29% [50.30, 58.28]	19.92% [13.80, 26.03]
0.75	32.85	0.83	42.33% [38.66, 46.01]	18.33% [13.52, 23.15]
1.00	23.51	0.87	32.17% [26.63, 37.70]	16.75% [11.47, 22.03]
1.50	13.76	0.97	22.96% [17.78, 28.14]	15.83% [11.28, 20.39]
2.00	9.84	1.00	19.04% [13.13, 24.95]	13.50% [8.51, 18.49]
3.00	6.21	1.00	15.04% [9.70, 20.38]	12.71% [6.72, 18.70]
5.00	3.56	1.00	12.88% [9.95, 15.80]	11.58% [5.36, 17.80]

Figure 1. Privacy–utility trade-off of DP-FedAvg. Points are means over five seeds; error bars are 95% confidence intervals. The dashed line shows non-private FedAvg (ID), and the dotted line shows chance. Generated from the same aggregate file as Table 2.

3.3. Effect of Label Skew

Table 3 and Figure 2 report non-private FedAvg under the three partitions. ID accuracy decreased as label skew increased. Under severe skew ($\alpha =0.1$), accuracy fell below the local-only mean obtained at $\alpha =1$. Template-disjoint accuracy followed the same ordering but stayed within a narrow 20–26% range. At $\alpha =0.1$ the template-disjoint accuracy was identical across all five seeds. In non-private runs the seeds change only initialization and mini-batch order, and this convex model converges to nearly the same solution regardless, so the zero-width interval shows insensitivity to training randomness, not certainty about population performance.

Table 3. Non-private FedAvg under Dirichlet label skew (mean over five seeds; brackets give 95% confidence intervals).

Partition	In-Distribution	Template-Disjoint
$\alpha =100$ (near-IID)	82.71% [82.19, 83.23]	26.04% [25.24, 26.84]
$\alpha =1$ (moderate)	74.71% [73.05, 76.37]	23.46% [22.24, 24.68]
$\alpha =0.1$ (severe)	44.33% [43.87, 44.80]	20.00% [20.00, 20.00]

Figure 2. Non-private FedAvg accuracy under three levels of Dirichlet label skew. Bars are means over five seeds; error bars are 95% confidence intervals.

3.4. Communication

Each transfer of the 512-parameter float32 model is 2048 bytes. Six clients, 12 rounds, and both directions give 294,912 bytes (0.281 MiB). The raw text field of the 1224 training records is 186,525 bytes (0.178 MiB). For this very small model, FedAvg therefore transmits more bytes than a one-time upload of the raw text. The ratio would be much larger for a transformer-scale model. These figures are payload counts, not end-to-end network measurements.

4. Discussion

Federated versus isolated and pooled training. On this benchmark, collaboration helped. With moderate skew, FedAvg improved substantially over models trained on a single client’s 204 records, but it did not reach the pooled-data model. The gap to centralized training widened as skew increased. At $\alpha =0.1$, FedAvg fell below the local-only mean, consistent with the known sensitivity of FedAvg to non-IID clients [7,8,9]. Drift-correcting optimizers were not evaluated.

Template dependence. The template-disjoint results are the most important caveat of this benchmark. When the symptom frames, openers, and closers were replaced by independently written text with the same meaning, every method dropped to 19–24%. The public vocabulary is part of the explanation: template-disjoint records contain about half as many in-vocabulary tokens as ID records, so the bag-of-words features carry less signal. The main point, however, is that ID accuracy on template-generated text greatly overstates how well a classifier trained on those templates generalizes. Any synthetic RCA benchmark should report a template-disjoint or otherwise independently authored evaluation alongside ID accuracy.

Why client-level DP is costly here. The utility loss follows directly from the noise scale. With $C=1$ and six equally weighted clients, the aggregate has ${\Delta }_2=1/3$. Each of the $d=512$ coordinates therefore receives noise with standard deviation $\sigma /3$, and the expected noise norm is about $(\sigma /3)\sqrt{d}\approx 7.5\sigma$. The clipped aggregate itself has norm at most 1. Even at $\sigma =0.25$, the noise norm per round exceeds the signal norm, and learning survives only because noise averages out across rounds. Utility could be improved by more clients (which reduces ${\max }_k{w}_k$ and hence ${\Delta }_2$), valid sampling amplification, fewer or more informative releases, adaptive clipping, or lower-dimensional private representations. None of these were evaluated here. Because d enters the noise norm directly, larger models would face a harder trade-off under the same mechanism, unless the number of clients grows accordingly.

Limitations. The data are synthetic and generated from a limited number of frames, so no result here establishes applicability to real fab records. Credible external validation would require appropriately governed, site-held-out records from several independent fabs, harmonized label taxonomies, a vocabulary or tokenizer fixed without access to held-out sites, and prospective evaluation on independently authored records. The classifier is deliberately simple. The public lexicon was specified with knowledge of the ID frame bank, although not of the template-disjoint bank. The privacy analysis assumes a fixed public roster and fixed public weights. It does not cover add/remove adjacency or data-dependent weights. The central-DP mechanism protects the released aggregates, but the server still observes individual updates unless secure aggregation [6] is also used. Five seeds give only coarse uncertainty estimates and capture training randomness, not data-sampling variability.

5. Conclusions

On a fully released synthetic six-fab RCA text benchmark, non-private FedAvg improved over isolated local training but remained below centralized training, and its advantage shrank under stronger label skew. All methods generalized poorly to independently phrased records, which shows that in-distribution accuracy on template-generated text overstates generalization. Correctly calibrated client-level DP-FedAvg retained moderate utility only at very large $\epsilon$ and approached chance at single-digit $\epsilon$. The study provides a controlled, reproducible benchmark, not a new FL algorithm or evidence of production readiness. Claims about real cross-fab deployment require independently sourced, appropriately governed data.