FedQuAD: Fast-Converging Curvature-Aware Federated Learning for Credit Default Prediction from Private Accounting Data

Abstract

Credit default prediction from firm-level accounting statements is central to risk management, yet the underlying financial data are highly sensitive and often siloed across banks, auditors, and platforms. Federated learning (FL) offers a practical route to collaborative modeling without centralizing raw records, but standard FL optimization can converge slowly under severe client heterogeneity, heavy-tailed accounting features, and label imbalance typical of default events. This paper proposes FedQuAD, a novel fast-converging FL algorithm that couples (i) quasi-Newton curvature aggregation on the server with a lightweight limited-memory update to accelerate global progress, (ii) a proximal variance-reduced local solver that stabilizes client drift under non-IID accounting distributions, and (iii) federated robust standardization of tabular financial ratios via secure aggregated quantile statistics to mitigate scale instability and outliers. FedQuAD is communication-efficient by design: It transmits compact gradient and curvature sketches and adapts local computation to each client’s stochasticity and drift. We provide convergence guarantees for strongly convex default-risk objectives (logistic and calibrated GLM losses) under bounded heterogeneity, and extend the analysis to nonconvex deep tabular models via expected stationarity bounds. Experiments on public credit-risk benchmarks with simulated cross-silo (institutional) partitions demonstrate that FedQuAD reaches target AUC and calibration error with substantially fewer communication rounds than representative baselines while maintaining privacy constraints compatible with secure aggregation and optional client-level differential privacy accounting.

1. Introduction

Credit default prediction is a core capability for lenders, investors, and regulators. In practice, a large fraction of predictive signal comes from firm-level accounting statements and derived ratios, such as leverage, liquidity, profitability, and cash-flow measures. These variables are sensitive and often protected by contractual, legal, or competitive constraints, which makes cross-institution data pooling difficult. Federated learning (FL) enables collaborative model training without moving raw accounting records off-premise by keeping data local and exchanging only model updates [1,2]. Despite this promise, applying FL to default prediction raises three practical challenges.

First, the data distribution is inherently heterogeneous across institutions. Portfolio composition varies by industry, geography, and underwriting policy, and accounting features are typically heavy tailed with outliers. Such non-IID effects can cause client drift and slow global convergence, particularly when each institution performs multiple local steps per round [3,4,5]. Second, defaults are rare, leading to label imbalance and noisy gradients on many clients; this worsens optimization stability and harms probability calibration, which is critical for downstream risk decisions. Third, communication rounds are expensive in cross-silo deployments: Institutions have limited windows for secure computation and auditing, and latency dominates wall-clock time. Thus, for federated default prediction, fast convergence in terms of communication rounds is often more important than minimizing local computation.

Cross-silo financial learning faces a structural tension: Curvature-aware methods promise faster convergence and better calibration, yet second-order information is communication heavy and potentially privacy sensitive. At the same time, credit risk data are heterogeneous, heavy tailed, and severely imbalanced, making purely first-order federated optimization unstable. The central question we address is therefore how can one incorporate curvature information in a privacy-compatible, communication-efficient manner that remains robust under rare-event financial heterogeneity?

This paper proposes FedQuAD, a fast-converging FL algorithm tailored to private accounting data. FedQuAD couples two ideas. On the client side, we solve a proximal local objective anchored at the server model and use a variance-reduced gradient estimator to reduce the instability caused by heterogeneous portfolios and imbalance. On the server side, we introduce a curvature-aware correction computed from low-dimensional Hessian sketches aggregated across clients. The correction acts as a quasi-Newton step in a random subspace and is combined with the standard aggregated model delta. Unlike full second-order FL, our design transmits only a compact curvature sketch and a projected gradient vector, making the added communication small while delivering substantial reductions in rounds-to-accuracy.

Beyond optimization, we incorporate a robust preprocessing component designed for accounting data. Specifically, FedQuAD supports federated robust standardization based on global medians and interquartile ranges computed from aggregated quantile summaries, improving conditioning under heavy-tailed ratios and reducing the impact of extreme values. Our training loop is compatible with secure aggregation and can optionally support client-level differential privacy via clipping and noise on aggregated messages.

FedQuAD builds on standard primitives (proximal stabilization and variance reduction), but its novelty is in coupling them with a per-round, shared sketch-space quasi-Newton correction computed from aggregated curvature sketches anchored at the same iteration. Unlike approaches that transmit or maintain high-dimensional second-order state data, FedQuAD aggregates only an m-vector projected gradient and an $m\times m$ curvature sketch, lifts the resulting preconditioned direction back to parameter space, and combines it with the usual aggregated model delta in a single server update. This design keeps the added communication $O\left(m^2\right)$, remains compatible with secure aggregation, and is tailored to heavy-tailed, non-IID accounting distributions.

Our contributions are as follows:

• A novel curvature-aware federated algorithm. We introduce FedQuAD, which augments standard FL with a sketch-space quasi-Newton correction built from aggregated curvature sketches, accelerating convergence under non-IID accounting distributions.
• A stabilized local solver for default prediction. FedQuAD combines a proximal objective with variance reduction to mitigate client drift and label-imbalance-induced gradient noise, improving both discrimination and calibration.
• Robust federated normalization for heavy-tailed accounting features. We propose a quantile-based federated standardization mechanism that uses aggregated summaries to compute global medians and interquartile ranges without sharing raw data.
• Empirical validation on default-prediction benchmarks. Experiments demonstrate that FedQuAD reduces the communication rounds needed to reach target AUC and improves calibration relative to representative baselines.

The remainder of the paper is organized as follows: Section 3 defines the federated default prediction problem and key assumptions. Section 4 presents FedQuAD in detail. Section 5 reports experimental results. Section 2 discusses related work. Section 6 concludes.

2. Related Work

Our work sits at the intersection of (i) federated optimization under statistical heterogeneity, (ii) curvature/second-order and variance-reduced methods for faster federated convergence, and (iii) credit-risk modeling and privacy-preserving collaboration across financial institutions.

2.1. Federated Optimization Under Heterogeneity

Federated learning (FL) enables collaborative training without centralizing raw data, typically via periodic client updates aggregated by a server [1,2,6]. The dominant baseline, FedAvg, is simple and effective but can degrade under non-IID data and partial participation [1]. A substantial line of work improves robustness to heterogeneity and client drift through regularization, control variates, and alternative aggregation or local update schemes. Representative examples include FedProx for proximal stabilization [3], SCAFFOLD for variance reduction via control variates [4], and FedNova for normalized aggregation under variable local computation [5]. Server- and client-side adaptive federated optimization has also been explored (e.g., FedOpt/FedAdam-style server optimizers) [7], alongside dynamic regularization approaches such as FedDyn [8]. More recent work continues to target practical cross-device constraints (communication and memory) while retaining adaptivity, e.g., FedAda² [9], and asynchronous adaptive variants such as FADAS [10]. These methods motivate our focus on stabilizing local steps and mitigating drift while improving convergence speed.

2.2. Curvature, Second-Order, and Variance Reduction in FL

While first-order methods dominate FL practice, slow convergence and sensitivity to heterogeneity have motivated curvature-aware and variance-reduced alternatives. Classical variance-reduction ideas have been adapted to the federated setting (e.g., control variates in SCAFFOLD [4]), and more specialized constructions study single-loop or client-variance-reduced mechanisms to accelerate convergence and improve communication efficiency [11,12,13]. Parallel to this, second-order and quasi-Newton federated methods aim to leverage curvature information without prohibitive communication costs. Early communication-efficient second-order directions include basis-sketching approaches [14], while newer scalable designs target large models and realistic system constraints, such as Fed-Sophia [15]. Very recent work develops quasi-Newton FL frameworks explicitly (e.g., distributed quasi-Newton FL with fairness considerations [16]) and integrates quasi-Newton updates with error-feedback mechanisms to cope with compression and noisy updates [17]. Privacy constraints can further interact with curvature and communication; for example, recent differentially private FL work studies second-order information under communication budgets [18]. Our method is aligned with this trajectory: we aim to use lightweight curvature information and stabilization to reduce the number of rounds needed in heterogeneous settings.

Our proposed FedQuAD differs from prior curvature-aware FL in that it (i) refreshes a shared random sketch subspace each round via a public seed, (ii) computes the quasi-Newton correction solely from aggregated sketch-space quantities anchored at the broadcast iterate, and (iii) explicitly combines this lifted correction with a stabilized proximal variance-reduced local delta. This isolates what is new in FedQuAD: the single-round coupling of stabilized local training with an explicit sketch-space quasi-Newton preconditioner under cross-silo communication and privacy constraints.

2.3. Privacy, Security, and Trust Mechanisms

In finance and credit, privacy is not only a performance concern but a deployment constraint. Secure aggregation protocols allow servers to learn only aggregated updates rather than individual client updates [19]. Differential privacy (DP) provides formal privacy guarantees and is often combined with FL to mitigate leakage from gradients/updates [20,21]. However, gradient/parameter updates can still leak sensitive information via reconstruction or inference attacks, motivating stronger defenses and careful system design [22,23,24]. Recent applied systems in credit scoring combine FL with auxiliary trust infrastructures (e.g., blockchain) and emphasize explainability and traceability [25,26]. These considerations shape our evaluation and design choices, since credit modeling typically requires both strong privacy posture and operational transparency.

2.4. Credit-Risk Modeling and Federated Credit Scoring

Credit default prediction and credit scoring have a long history in statistical modeling and machine learning, with persistent concerns about class imbalance, temporal drift, and dataset shift. Large-scale empirical comparisons and benchmarks have highlighted the competitiveness of tree ensembles and carefully tuned linear models in many credit settings [27,28,29]. Modern gradient-boosted decision trees such as XGBoost and LightGBM have become standard baselines for tabular credit data due to strong accuracy and practical training/inference characteristics [30,31]. Deep tabular models (e.g., attention-based architectures) offer another path when feature interactions are complex [32,33]. When data are siloed across institutions, FL provides a natural mechanism to improve generalization without data pooling [1,2]. Recent work specifically targets privacy-preserving multi-party credit scoring, including knowledge-distillation-based federated approaches [34], explainable FL credit scoring with blockchain support [25,26], and broader surveys discussing FL’s emerging role in large/foundation-model settings that may eventually impact financial modeling workflows [35,36,37,38].

We conclude that traditional credit-risk models, including logistic scorecards and structural risk frameworks, emphasize interpretability, calibration, and economic coherence. Our work does not alter the statistical foundation of default modeling; rather, it advances the optimization and privacy layer enabling such models to be trained collaboratively without sharing raw accounting data. This perspective aligns with classical discussions of credit risk modeling foundations. Our work contributes to this growing body by focusing on optimization and stability improvements tailored to the credit default prediction setting, where heterogeneity across institutions and strict privacy constraints are central.

3. Preliminaries

3.1. Problem Setting and Federated Objective

We consider a cross-silo federated learning (FL) system with K institutions (clients) indexed by $k\in \{1,\dots ,K\}$. Client k holds a private dataset ${\mathcal{D}}_k={\left\{({\mathbf{x}}_{k,i},y_{k,i})\right\}}_{i=1}^{n_k},$ where ${\mathbf{x}}_{k,i}\in {\mathbb{R}}^d$ denotes a d-dimensional accounting feature vector and $y_{k,i}\in \{0,1\}$ indicates whether a default event occurs within a fixed horizon. Let $N={\sum }_{k=1}^K{n}_k$ and $p_k=n_k/N$. A model with parameters $\mathbf{w}\in {\mathbb{R}}^d$ (or $\mathbf{w}\in {\mathbb{R}}^P$ for deep tabular models) outputs a default probability ${\widehat{p}}_{\mathbf{w}}\left(\mathbf{x}\right)\in (0,1)$. We use the logistic link as a canonical choice, ${\widehat{p}}_{\mathbf{w}}\left(\mathbf{x}\right)=\sigma \left(f_{\mathbf{w}}\left(\mathbf{x}\right)\right),\sigma \left(z\right)={\displaystyle \frac{1}{1+e^{-z}}},$ where $f_{\mathbf{w}}\left(\mathbf{x}\right)$ is either linear ($f_{\mathbf{w}}\left(\mathbf{x}\right)={\mathbf{w}}^{\top }\mathbf{x}$) or a nonlinear network for tabular data. The (optionally class-weighted) logistic loss is

$$\ell (\mathbf{w};\mathbf{x},y)=-\alpha ylog{\widehat{p}}_{\mathbf{w}}\left(\mathbf{x}\right)-(1-\alpha )(1-y)log\left(1-{\widehat{p}}_{\mathbf{w}}\left(\mathbf{x}\right)\right),$$

(1)

where $\alpha \in (0,1)$ controls imbalance. Equation (1) uses standard class weighting (cost-sensitive logistic loss): Positives are weighted by $\alpha$ and negatives by $(1-\alpha )$. In our experiments, $\alpha$ is used to address class imbalance (e.g., a balanced choice is $\alpha =1-\pi$ where $\pi$ is the positive-class prevalence on a client or on the union of participating clients) and is tuned on a federated validation split for calibration. We do not claim this is an explicit prior-correction unless $\alpha$ is deliberately chosen to reweight a source prior toward a target prior. With ${\ell }_2$ regularization coefficient $\lambda \ge 0$, client k defines

$$F_k\left(\mathbf{w}\right)={\displaystyle \frac{1}{n_k}}\sum _{i=1}^{n_k}\ell (\mathbf{w};{\mathbf{x}}_{k,i},y_{k,i})+{\displaystyle \frac{\lambda }{2}}{\parallel \mathbf{w}\parallel }_2^2,$$

(2)

and the global objective is

$$F\left(\mathbf{w}\right)=\sum _{k=1}^K{p}_k{F}_k\left(\mathbf{w}\right).$$

(3)

We denote $\nabla F_k\left(\mathbf{w}\right)$ and ${\nabla }^2{F}_k\left(\mathbf{w}\right)$ as the gradient and Hessian (when defined), and use $\nabla \ell (\mathbf{w};\xi )$ for a stochastic gradient on $\xi =(\mathbf{x},y)$.

3.2. Federated Protocol, Regularity, and Heterogeneity

FL proceeds in synchronous communication rounds $t=0,1,\dots ,T-1$. The server maintains the global model ${\mathbf{w}}^t$. In round t, a subset of clients ${\mathcal{S}}_t\subseteq \{1,\dots ,K\}$ participates and returns update messages ${\mathbf{u}}_k^t$, which the server aggregates:

$${\mathbf{w}}^{t+1}={\mathbf{w}}^t+\mathcal{A}\left({\left\{{\mathbf{u}}_k^t\right\}}_{k\in {\mathcal{S}}_t}\right),$$

(4)

where $\mathcal{A}(·)$ is an aggregation operator. We use ${\parallel ·\parallel }_2$ for the Euclidean norm and $\langle ·,·\rangle$ for the inner product. For convex default-risk models (e.g., logistic regression with $\lambda >0$), we assume each $F_k$ is L-smooth:

$$\parallel \nabla F_k\left(\mathbf{u}\right)-\nabla F_k{\left(\mathbf{v}\right)\parallel }_2\le L{\parallel \mathbf{u}-\mathbf{v}\parallel }_2,\forall \mathbf{u},\mathbf{v},$$

(5)

and the global objective F is $\mu$-strongly convex for some $\mu >0$:

$$F\left(\mathbf{v}\right)\ge F\left(\mathbf{u}\right)+\langle \nabla F\left(\mathbf{u}\right),\mathbf{v}-\mathbf{u}\rangle +{\displaystyle \frac{\mu }{2}}{\parallel \mathbf{v}-\mathbf{u}\parallel }_2^2.$$

(6)

For deep tabular models, we assume F is L-smooth and target approximate stationarity via $\mathbb{E}\parallel \nabla F\left({\mathbf{w}}^t\right){\parallel }_2^2$. Accounting data are typically non-IID across institutions. We quantify heterogeneity by bounded gradient dissimilarity: There exists $\zeta \ge 0$ such that

$$\sum _{k=1}^K{p}_k{\parallel \nabla F_k\left(\mathbf{w}\right)-\nabla F\left(\mathbf{w}\right)\parallel }_2^2\le {\zeta }^2,\forall \mathbf{w}.$$

(7)

For stochastic local solvers, we assume bounded gradient noise: for $\xi \sim {\mathcal{D}}_k$,

$$\mathbb{E}\left[\parallel \nabla \ell (\mathbf{w};\xi )-\nabla F_k{\left(\mathbf{w}\right)\parallel }_2^2\right]\le {\sigma }_k^2.$$

(8)

3.3. Curvature Sketches, Robust Normalization, and Privacy Primitives

Curvature information and sketching. To accelerate convergence, we will use curvature-aware aggregation. Let ${\mathbf{H}}^t$ denote a positive definite preconditioner used by the server (interpretable as an approximation to ${\left({\nabla }^{2}F\left({\mathbf{w}}^t\right)\right)}^{-1}$). Since transmitting full matrices is impractical, we work with low-dimensional sketches. Let ${\mathbf{S}}^t\in {\mathbb{R}}^{P\times m}$ be a sketching matrix with $m\ll P$. Client k forms a curvature sketch

$${\mathbf{C}}_k^t\approx {\left({\mathbf{S}}^t\right)}^{\top }{\nabla }^2{F}_k\left({\mathbf{w}}^t\right){\mathbf{S}}^t\in {\mathbb{R}}^{m\times m},$$

(9)

which the server aggregates to obtain a compact curvature model used during aggregation (specified in Section 4).

Robust feature normalization via federated quantiles. Tabular accounting variables are heavy-tailed and may contain extreme outliers. For a univariate random variable X, define the p-quantile

$$Q_X\left(p\right)=inf\{x\in \mathbb{R}:\mathbb{P}(X\le x)\ge p\},p\in (0,1).$$

(10)

For each feature $j\in \{1,\dots ,d\}$, we use the global median $m_j=Q_{X_j}\left(0.5\right)$ and interquartile range ${\mathrm{IQR}}_j=Q_{X_j}\left(0.75\right)-Q_{X_j}\left(0.25\right)$ to robustly standardize

$${\tilde{x}}_j={\displaystyle \frac{x_j-m_j}{{\mathrm{IQR}}_j+\tau }},$$

(11)

where $\tau >0$ ensures numerical stability. These quantiles are computed from privacy-preserving aggregated summaries (Section 4).

4. Methodology

This section presents FedQuAD (Federated Quasi-Newton with Adaptive Drift control), a fast-converging FL method for default prediction on private accounting data. FedQuAD augments a stabilized local solver with a server-side curvature-aware correction computed in a low-dimensional sketch space, adding only $O\left(m^2\right)$ extra uplink payload per round (beyond transmitting model deltas).

4.1. FedQuAD Protocol and Global Update Rule

FedQuAD targets fast communication-round convergence for minimizing the federated objective $F\left(\mathbf{w}\right)={\sum }_{k=1}^K{p}_k{F}_k\left(\mathbf{w}\right)$ in (3). The key idea is to combine (i) the standard model-delta aggregation that captures broad progress from local training, with (ii) a curvature-aware correction computed in a low-dimensional sketch space that approximates a Newton-like step along informative directions. This yields a single-round update that remains compatible with secure aggregation and adds only $O\left(m^2\right)$ uplink payload per client when the sketch dimension $m\ll P$.

Round structure. At communication round t, the server holds the global model ${\mathbf{w}}^t\in {\mathbb{R}}^P$ and samples a participating client set ${\mathcal{S}}_t\subseteq \{1,\dots ,K\}$. The server also specifies a sketching matrix ${\mathbf{S}}^t\in {\mathbb{R}}^{P\times m}$ with orthonormal columns, used to define a shared m-dimensional subspace. To reduce server bandwidth, ${\mathbf{S}}^t$ is generated from a public random seed ${\mathrm{seed}}^t$ broadcast to clients; each client reconstructs ${\mathbf{S}}^t$ locally (the concrete construction is given in Section 4.3).

Each participating client $k\in {\mathcal{S}}_t$ computes three quantities at (or anchored to) the broadcast iterate ${\mathbf{w}}^t$:

1.

A stabilized local model delta ${\Delta }_k^t\in {\mathbb{R}}^P$, obtained by approximately minimizing a proximal, variance-reduced subproblem (detailed in Section 4.2). Intuitively, ${\Delta }_k^t$ captures useful descent directions under the client distribution while controlling drift.

2.

A projected gradient in sketch space, ${\mathbf{g}}_{k,s}^t={\left({\mathbf{S}}^t\right)}^{\top }\nabla F_k\left({\mathbf{w}}^t\right)\in {\mathbb{R}}^m,$ which summarizes how the local objective changes along the shared subspace directions.

3.

A curvature sketch in sketch space, ${\mathbf{C}}_k^t\approx {\left({\mathbf{S}}^t\right)}^{\top }{\nabla }^2{F}_k\left({\mathbf{w}}^t\right){\mathbf{S}}^t\in {\mathbb{R}}^{m\times m},$ where the approximation may use an exact Hessian for generalized linear models or a Gauss–Newton/empirical Fisher approximation for deep tabular models.

All three messages can be protected by secure aggregation (and optionally by client-level DP via clipping/noise; see Section 4.4).

Weighted aggregation. Let ${\tilde{p}}_k$ denote normalized weights over participating clients, ${\tilde{p}}_k={\displaystyle \frac{p_k}{{\sum }_{j\in {\mathcal{S}}_t}{p}_j}}.$ The server forms the aggregated delta, projected gradient, and curvature sketch:

$${\Delta }^t=\sum _{k\in {\mathcal{S}}_t}{\tilde{p}}_k{\Delta }_k^t,{\mathbf{g}}_s^t=\sum _{k\in {\mathcal{S}}_t}{\tilde{p}}_k{\mathbf{g}}_{k,s}^t,{\mathbf{C}}^t=\sum _{k\in {\mathcal{S}}_t}{\tilde{p}}_k{\mathbf{C}}_k^t.$$

(12)

Compared with FedAvg-style aggregation (which uses only ${\Delta }^t$), FedQuAD additionally aggregates $({\mathbf{g}}_s^t,{\mathbf{C}}^t)$ to compute a preconditioned correction in the shared subspace.

The server computes a damped inverse in sketch space, ${\mathbf{M}}^t={\left({\mathbf{C}}^t+\rho {\mathbf{I}}_m\right)}^{-1},$ where $\rho >0$ is a damping parameter ensuring numerical stability even when ${\mathbf{C}}^t$ is ill-conditioned or rank-deficient. The quasi-Newton correction direction in parameter space is then ${\mathbf{d}}_q^t=-{\mathbf{S}}^t{\mathbf{M}}^t{\mathbf{g}}_s^t.$ Finally, FedQuAD updates the global model by combining the aggregated local delta ${\Delta }^t$ with the curvature-aware correction:

$${\mathbf{w}}^{t+1}={\mathbf{w}}^t+{\Delta }^t+{\eta }_q{\mathbf{d}}_q^t$$

(13)

where ${\eta }_q>0$ controls the correction strength. When ${\eta }_q=0$, (13) reduces to a stabilized first-order method driven purely by local deltas. When ${\eta }_q>0$, the term ${\mathbf{d}}_q^t$ acts like a Newton step restricted to $\mathrm{span}\left({\mathbf{S}}^t\right)$, improving the effective conditioning along those directions and empirically reducing the rounds needed to reach target AUC and calibration.

Per participating client, FedQuAD transmits the usual model delta ${\Delta }_k^t$ (same order as FedAvg) plus a projected gradient vector of size m and a symmetric curvature sketch of size $m\times m$. Transmitting only the upper-triangular part yields $m(m+1)/2$ scalars. Thus, the additional uplink beyond ${\Delta }_k^t$ scales as $O\left(m^2\right)$, which is small for $m\ll P$. On the server, the additional compute is dominated by inverting the $m\times m$ matrix in (12), i.e., $O\left(m^3\right)$ per round, which is negligible relative to client training for typical choices (e.g., $m\in [32,128]$).

4.2. Client-Side Solver with Adaptive Drift Control

FedQuAD employs a client-side solver designed for two dominant difficulties in federated default prediction: (i) client drift caused by heterogeneous portfolios and non-IID accounting distributions, quantified by (7) and (ii) high gradient noise induced by rare defaults and heavy-tailed features, reflected in (8). To address both, each client approximately minimizes a proximal objective anchored at the broadcast model and uses a variance-reduced estimator to stabilize local steps. This subroutine produces the model delta ${\Delta }_k^t$ used in the aggregated update (12). Given server model ${\mathbf{w}}^t$, client k defines the round-t proximal objective

$${\Phi }_k^t\left(\mathbf{w}\right)=F_k\left(\mathbf{w}\right)+{\displaystyle \frac{{\mu }_p}{2}}{\parallel \mathbf{w}-{\mathbf{w}}^t\parallel }_2^2,$$

(14)

where ${\mu }_p\ge 0$ controls the strength of drift control. The proximal term penalizes deviations from ${\mathbf{w}}^t$, reducing the tendency of local iterates to overfit to institution-specific patterns that may not generalize globally. This is particularly important when defaults are sparse on some clients: Without anchoring, local updates can be dominated by a small number of rare events and lead to unstable directions.

Snapshot gradient and control variate. At the beginning of the round, client k computes a snapshot gradient at the broadcast point, ${\mathbf{g}}_k^t=\nabla F_k\left({\mathbf{w}}^t\right),$ either exactly (for moderate $n_k$) or using a sufficiently large minibatch to reduce noise. This snapshot gradient serves two roles: (i) It is used directly in the projected gradient message ${\mathbf{g}}_{k,s}^t$ in (2), and (ii) it forms a control variate for variance reduction during local optimization.

Prox-SVRG local steps. Initialize ${\mathbf{w}}_k^{t,0}={\mathbf{w}}^t$. For local step $e=0,1,\dots ,E-1$, sample a minibatch ${\mathcal{B}}_k^{t,e}$ and construct the variance-reduced (and proximal) gradient estimator

$${\mathbf{v}}_k^{t,e}=\nabla \ell ({\mathbf{w}}_k^{t,e};{\mathcal{B}}_k^{t,e})-\nabla \ell ({\mathbf{w}}^t;{\mathcal{B}}_k^{t,e})+{\mathbf{g}}_k^t+{\mu }_p({\mathbf{w}}_k^{t,e}-{\mathbf{w}}^t),$$

(15)

then take the update ${\mathbf{w}}_k^{t,e+1}={\mathbf{w}}_k^{t,e}-{\eta }_{\ell }{\mathbf{v}}_k^{t,e},$ where ${\eta }_{\ell }>0$ is the local step size. The estimator in (15) has the standard SVRG structure: It subtracts the minibatch gradient at the snapshot point ${\mathbf{w}}^t$ and adds back the (near-)full gradient ${\mathbf{g}}_k^t$, thereby reducing the variance of minibatch gradients when ${\mathbf{w}}_k^{t,e}$ stays near ${\mathbf{w}}^t$ (which is encouraged by the proximal term). The additional proximal gradient ${\mu }_p({\mathbf{w}}_k^{t,e}-{\mathbf{w}}^t)$ ensures that the local direction is consistent with descent on ${\Phi }_k^t$ and provides explicit drift control. After E steps, client k returns the model delta ${\Delta }_k^t={\mathbf{w}}_k^{t,E}-{\mathbf{w}}^t.$

Adaptive drift control in practice. While ${\mu }_p$ can be fixed, FedQuAD supports a simple adaptive rule that increases anchoring when local updates appear unstable. Let ${\Delta }_k^t$ denote the candidate delta produced by a trial run with current ${\mu }_p$. Client k computes the normalized step magnitude

$$r_k^t={\displaystyle \frac{\parallel {\Delta }_k^t{\parallel }_2}{\parallel {\mathbf{w}}^t{\parallel }_2+ϵ}},$$

(16)

with small $ϵ>0$. If $r_k^t$ exceeds a threshold $r_{max}$ (indicating aggressive drift) or if the local objective ${\Phi }_k^t$ fails to decrease over the last few steps, the client increases anchoring (e.g., ${\mu }_p\leftarrow \gamma {\mu }_p$ with $\gamma >1$) and reruns the remaining local steps. This adaptive mechanism is lightweight, requires no extra communication, and empirically reduces catastrophic local jumps on clients with extreme imbalance or outlier-heavy ratios. textcolorblueThe indicator $r_k^t$ in (16) measures the relative parameter change in a round and is intended as a simple, model-scale-invariant safety valve. In cross-silo credit settings where some institutions may have very few default events, we recommend conservative caps (e.g., $r_{max}\in [0.01,0.1]$) and a small multiplicative increase factor (e.g., $\gamma \approx 2$). Unless otherwise noted, we use $r_{max}=0.05$ and $\gamma =2$ (with at most a few retries per round) to stabilize rare-event clients without adding communication or materially affecting typical clients.

In addition to ${\Delta }_k^t$, client k already has ${\mathbf{g}}_k^t$ from (15), which is used to compute the projected gradient message ${\mathbf{g}}_{k,s}^t={\left({\mathbf{S}}^t\right)}^{\top }{\mathbf{g}}_k^t$. Curvature sketch computation uses the same broadcast point ${\mathbf{w}}^t$ (Section 4.3), ensuring all server-side quantities $({\Delta }^t,{\mathbf{g}}_s^t,{\mathbf{C}}^t)$ are aligned to the same iteration and thus can be coherently combined in (12).

In general, given that credit default prediction exhibits three structural properties: (i) rare positive events leading to gradient sparsity and instability, (ii) cross-institution feature-scale and distributional mismatch, and (iii) heavy-tailed accounting ratios. The proximal term mitigates client drift caused by heterogeneous default frequencies; variance reduction stabilizes local stochastic updates when positives are scarce; and the sketch-based quasi-Newton correction compensates for curvature mismatch across institutions without transmitting full Hessians. In combination, these mechanisms address financial heterogeneity at three levels: parameter stability (proximal), stochastic noise (variance reduction), and geometry mismatch (curvature correction).

4.3. Curvature Sketching and Quasi-Newton Correction

This subsection describes how FedQuAD constructs the shared sketch space, how clients compute curvature sketches with low overhead, and how the server uses the aggregated sketch to form a stable quasi-Newton correction ${\mathbf{d}}_q^t$ in (13). The central principle is to approximate Newton preconditioning only in a low-dimensional subspace where curvature information can be communicated and inverted efficiently.

At round t, the server samples a public random seed ${\mathrm{seed}}^t$ and defines

$${\mathbf{S}}^t=\mathrm{qr}\left({\mathbf{G}}^t\left({\mathrm{seed}}^t\right)\right)\in {\mathbb{R}}^{P\times m},$$

(17)

where ${\mathbf{G}}^t\left({\mathrm{seed}}^t\right)$ is a pseudo-random matrix with i.i.d. $\mathcal{N}(0,1)$ entries and $\mathrm{qr}(·)$ denotes a thin QR factorization producing orthonormal columns. The server broadcasts ${\mathrm{seed}}^t$ (and m), and each client reconstructs the same ${\mathbf{S}}^t$ locally.

Curvature model to be sketched. Client k forms a symmetric positive semidefinite curvature approximation ${\widehat{\mathbf{H}}}_k^t$ at the broadcast model ${\mathbf{w}}^t$:

$${\widehat{\mathbf{H}}}_k^t\approx {\nabla }^2{F}_k\left({\mathbf{w}}^t\right).$$

For generalized linear models (e.g., logistic regression with $\lambda >0$), ${\widehat{\mathbf{H}}}_k^t$ can be the exact empirical Hessian including the $\lambda \mathbf{I}$ term. For deep tabular models, we use a Gauss–Newton or empirical Fisher approximation, which is PSD and can be evaluated via Hessian-vector products (HVPs) without materializing a full $P\times P$ matrix. Given ${\mathbf{S}}^t=[{\mathbf{s}}_1^t,\dots ,{\mathbf{s}}_m^t]$, the curvature sketch is

$${\mathbf{C}}_k^t={\left({\mathbf{S}}^t\right)}^{\top }{\widehat{\mathbf{H}}}_k^t{\mathbf{S}}^t+{\rho }_0{\mathbf{I}}_m\in {\mathbb{R}}^{m\times m},$$

(18)

where ${\rho }_0\ge 0$ is a client-side ridge for robustness under noisy curvature. The client computes ${\mathbf{C}}_k^t$ using HVPs: ${\mathbf{v}}_j^t={\widehat{\mathbf{H}}}_k^t{\mathbf{s}}_j^t\mathrm{for}j=1,\dots ,m,$ and fills the entries by inner products ${\left[{\mathbf{C}}_k^t\right]}_{ab}=\langle {\mathbf{s}}_a^t,{\mathbf{v}}_b^t\rangle +{\rho }_0\mathbf{1}\{a=b\}.$

Projected gradient alignment. Curvature must be paired with a gradient in the same sketch space. Client k computes the projected gradient at ${\mathbf{w}}^t$:

$${\mathbf{g}}_{k,s}^t={\left({\mathbf{S}}^t\right)}^{\top }\nabla F_k\left({\mathbf{w}}^t\right)\in {\mathbb{R}}^m,$$

(19)

using the snapshot gradient from Section 4.2. Both ${\mathbf{C}}_k^t$ and ${\mathbf{g}}_{k,s}^t$ are anchored at ${\mathbf{w}}^t$, ensuring coherent aggregation across clients. The server aggregates sketches and projected gradients using normalized weights ${\tilde{p}}_k$:

$${\mathbf{C}}^t=\sum _{k\in {\mathcal{S}}_t}{\tilde{p}}_k{\mathbf{C}}_k^t,{\mathbf{g}}_s^t=\sum _{k\in {\mathcal{S}}_t}{\tilde{p}}_k{\mathbf{g}}_{k,s}^t.$$

It then computes a stabilized inverse:

$${\mathbf{M}}^t={\left({\mathbf{C}}^t+\rho {\mathbf{I}}_m\right)}^{-1}$$

(20)

with server-side damping $\rho >0$, and forms the lifted quasi-Newton correction ${\mathbf{d}}_q^t=-{\mathbf{S}}^t{\mathbf{M}}^t{\mathbf{g}}_s^t.$ This correction is combined with the aggregated local delta ${\Delta }^t$ through (12).

Cost and practical choices. Per client, computing (20) requires m HVPs; for logistic regression it can be implemented via matrix-vector products with local data, and for deep models HVPs are supported by autodiff at a cost comparable to backpropagation. Communication per client includes one m-vector and one symmetric $m\times m$ matrix (upper triangle), i.e., $m(m+1)/2$ scalars. In practice, moderate m (e.g., 32–128) yields most of the acceleration benefits while keeping overhead small. As the curvature payload scales as $m(m+1)/2$, a practical rule is to pick m so that this term is a small fraction of the model-delta payload for the chosen model size P. For tabular models in our study, $m=64$ is a robust default. we recommend starting from $m\in \{32,64\}$ and increasing only if rounds-to-target improvements justify the quadratic communication increase; keeping m modest is also beneficial when per-round participation is small, because ${\mathbf{C}}^t$ becomes noisier and the damping in (20) plays a larger role.

4.4. Robust Preprocessing and Optional Privacy Layer

FedQuAD is intended for default prediction on sensitive, heavy-tailed accounting data. Accordingly, we integrate two practical components: (i) a robust federated standardization mechanism that improves numerical conditioning and stability, and (ii) an optional privacy layer that supports secure aggregation and client-level differential privacy (DP). Both components are designed to preserve the main optimization loop in (13) while improving deployability in cross-silo settings.

Robust federated standardization via quantile summaries. Accounting ratios and statement-derived variables often exhibit extreme skew and outliers, which can degrade both first-order optimization (through exploding gradients) and curvature estimation (through ill-conditioned Hessians). FedQuAD therefore standardizes features using global medians and interquartile ranges as in (11). Since raw feature values cannot be shared, clients transmit only mergeable summaries.

Concretely, for each feature $j\in \{1,\dots ,d\}$, client k constructs a compact quantile summary ${\mathcal{Q}}_{k,j}$ that supports approximate queries for $Q_{X_j}\left(0.25\right)$, $Q_{X_j}\left(0.5\right)$, and $Q_{X_j}\left(0.75\right)$. Such summaries can be implemented using fixed-bin histograms (after coarse clipping) or digest-style sketches that support merge operations. The server aggregates the summaries by merging: ${\mathcal{Q}}_{\mathrm{agg},j}\leftarrow \mathrm{Merge}\left({\left\{{\mathcal{Q}}_{k,j}\right\}}_{k\in {\mathcal{S}}^{\mathrm{stat}}}\right),$ where ${\mathcal{S}}^{\mathrm{stat}}$ is a (possibly larger) set of clients participating in the statistics phase. From ${\mathcal{Q}}_{\mathrm{agg},j}$, the server estimates

$$m_j\approx Q_{X_j}\left(0.5\right),{\mathrm{IQR}}_j\approx Q_{X_j}\left(0.75\right)-Q_{X_j}\left(0.25\right),$$

and broadcasts ${\{m_j,{\mathrm{IQR}}_j\}}_{j=1}^d$. Clients then transform each feature by

$${\tilde{x}}_j={\displaystyle \frac{x_j-m_j}{{\mathrm{IQR}}_j+\tau }},$$

with $\tau >0$ to avoid division by near-zero scales. This preprocessing improves the stability of both the local solver (Section 4.2) and curvature sketching (Section 4.3) by controlling heavy tails and reducing cross-client scale mismatch. In our experiments, robust normalization also improves calibration, likely because the model is less sensitive to extreme ratios that correlate spuriously with rare default events on a subset of clients. We note that, Robust median/IQR scaling is one natural choice for heavy-tailed accounting ratios. Alternatives in federated settings include global z-score standardization via secure aggregation of per-client sums and squared sums, per-client standardization (which can increase cross-client scale mismatch), or monotone transformations such as log/winsorization and rank/quantile transforms. We focus on median/IQR because it is resilient to outliers while preserving a common scale across institutions.

Secure aggregation compatibility. FedQuAD messages in a round are ${\Delta }_k^t$, ${\mathbf{g}}_{k,s}^t$, and ${\mathbf{C}}_k^t$. Secure aggregation can be used to ensure that the server only observes aggregated values such as ${\sum }_{k\in {\mathcal{S}}_t}{\tilde{p}}_k{\Delta }_k^t$ and similarly for ${\mathbf{g}}_{k,s}^t$ and ${\mathbf{C}}_k^t$. We treat secure aggregation as an oracle that reveals only these sums, aligning with the aggregation definitions in (12). This protects individual client updates from direct inspection while preserving exactness of the optimization update.

This secure aggregation ensures the server only observes aggregated messages, but the aggregates $({\Delta }^t,{\mathbf{g}}_s^t,{\mathbf{C}}^t)$ are still model-update statistics and can reveal distribution-level information, especially if few clients participate or participation is highly unbalanced. Curvature sketches are second-order summaries in the random subspace $\mathrm{span}\left({\mathbf{S}}^t\right)$ and thus may encode structural information beyond first-order gradients. FedQuAD mitigates this risk by (i) relying on secure aggregation so that no single institution’s sketch is revealed, (ii) refreshing the sketch subspace each round via the public seed (17), limiting information accumulation in fixed coordinates, and (iii) supporting client-level DP for all transmitted objects, including the vectorized upper-triangular entries of ${\mathbf{C}}^t$, via clipping and Gaussian noise. In deployments we also recommend enforcing a minimum participation threshold before unmasking secure aggregation, as this is standard in secure-aggregation protocols [19].

Client-level differential privacy (optional). When stronger formal privacy is required, FedQuAD supports client-level $(\epsilon ,\delta )$-DP by clipping and adding noise to aggregated messages. Let

$${\mathrm{clip}}_C\left(\mathbf{u}\right)={\mathbf{u}·min\{1,C/\parallel \mathbf{u}\parallel }_2\}$$

denote ${\ell }_2$ clipping. Prior to secure aggregation, each client clips its transmitted delta and projected gradient:

$${\tilde{\Delta }}_k^t={\mathrm{clip}}_{C_{\Delta }}\left({\Delta }_k^t\right),{\tilde{\mathbf{g}}}_{k,s}^t={\mathrm{clip}}_{C_g}\left({\mathbf{g}}_{k,s}^t\right).$$

(21)

For curvature sketches, we clip with respect to the Frobenius norm:

$${\tilde{\mathbf{C}}}_k^t={\mathbf{C}}_k^t·min\left\{1,{\displaystyle \frac{C_C}{\parallel {\mathbf{C}}_k^t{\parallel }_F}}\right\}.$$

(22)

After secure aggregation, the server adds Gaussian noise to the aggregated messages. For example, letting ${\overline{\Delta }}^t$ denote the weighted aggregate of ${\tilde{\Delta }}_k^t$, the server uses

$${\Delta }^t\leftarrow {\overline{\Delta }}^t+\mathcal{N}\left(\mathbf{0},{\sigma }_{\Delta }^2{C}_{\Delta }^2{\mathbf{I}}_P\right),$$

(23)

and similarly ${\mathbf{g}}_s^t\leftarrow {\overline{\mathbf{g}}}_s^t+\mathcal{N}\left(\mathbf{0},{\sigma }_g^2{C}_g^2{\mathbf{I}}_m\right).$ For ${\mathbf{C}}^t$, noise can be added to the vectorized upper-triangular entries, yielding a symmetric noisy sketch after reshaping and symmetrization. The noise multipliers $({\sigma }_{\Delta },{\sigma }_g,{\sigma }_C)$ are selected based on the desired $(\epsilon ,\delta )$ and the number of rounds T using standard composition accounting for client subsampling. When DP noise is applied to ${\mathbf{C}}^t$, the aggregated sketch can become less stable. FedQuAD mitigates this by (i) client-side ridge ${\rho }_0$ in (18), (ii) server-side damping $\rho$ in (20), and (iii) optional eigenvalue flooring in sketch space: ${\mathbf{C}}^t\leftarrow \mathbf{U}max\{\mathbf{\Lambda },{\lambda }_{min}\mathbf{I}\}{\mathbf{U}}^{\top },$ where ${\mathbf{C}}^t=\mathbf{U}\mathbf{\Lambda }{\mathbf{U}}^{\top }$ and ${\lambda }_{min}>0$ is a small floor. These stabilizers ensure that the inverse in (20) remains well-defined and prevents overly large correction steps due to noisy or near-singular curvature.

5. Experiments

5.1. Experimental Setup

Datasets and feature construction. We evaluate federated default prediction on two tabular credit-risk benchmarks that reflect key properties of private accounting data: non-IID client distributions, heavy-tailed covariates, missing values, and label imbalance.

(D1) Corporate Bankruptcy (Accounting Ratios). We use the publicly available Polish companies bankruptcy dataset, which contains 64 financial ratios derived from accounting statements and a binary label indicating bankruptcy over a fixed horizon (available at https://archive.ics.uci.edu/dataset/365/polish+companies+bankruptcy+data (accessed on 20 May 2025)). We remove features with more than $15\%$ missingness, impute the remaining missing values using per-client medians to avoid cross-silo leakage, and apply the robust federated standardization in (11), where $m_j$ and ${\mathrm{IQR}}_j$ are estimated from aggregated quantile summaries and broadcast to clients.

(D2) Consumer Default (Tabular Credit). We use the UCI “default of credit card clients” dataset, which contains 23 tabular covariates and a binary default label (available at https://archive.ics.uci.edu/dataset/350/default+of+credit+card+clients (accessed on 8 May 2025)). While not purely accounting ratios, it exhibits similar non-IID and imbalance characteristics and is widely used as a standard benchmark for tabular default modeling. We adopt the same missingness handling (when applicable) and the same robust normalization protocol for consistency.

Table 1. Dataset statistics after preprocessing. “Pos. rate” is the global fraction of $y=1$. “Miss.” is the fraction of missing entries prior to imputation.

Dataset	Samples	Features	Pos. Rate	Miss.	Task
(D1) Polish bankruptcy	$\mathrm{10,503}$	64	$0.052$	$0.087$	corporate bankruptcy
(D2) Credit card default	$\mathrm{30,000}$	23	$0.221$	$0.000$	consumer default

summarizes basic statistics.

Federated partitioning and heterogeneity controls. We simulate a cross-silo network with $K=20$ institutions. For each dataset, we generate heterogeneous client datasets using a two-stage procedure: (i) label skew via a Dirichlet partition with concentration parameter $0.3$, producing widely varying default prevalences across clients; and (ii) covariate shift by assigning clients disjoint segments approximated by k-means clustering on covariates (to mimic institutions serving different industries or customer segments). Each round uses partial participation with $|{\mathcal{S}}_t|=5$ clients ($25\%$ participation). Figure 1 illustrates the resulting variability in client-level default prevalence on (D1), a primary driver of drift and noisy gradients in practice. Our evaluation is conducted on public datasets with simulated cross-silo partitions (label skew and covariate shift). Since real institutional accounting ledgers are typically unavailable for public release due to confidentiality and regulation, we limit real-world claims to what can be supported by reproducible benchmarks and treat deployment on proprietary multi-bank data as future work.

Models, training protocol, and hyperparameter tuning. We evaluate two model families: (i) logistic regression with ${\ell }_2$ regularization $\lambda ={10}^{-4}$ in (2), representing a strong, interpretable baseline for accounting ratios, and (ii) a tabular MLP with two hidden layers (256, 128) and ReLU activations, trained with the loss in (1). We train for $T=200$ rounds and report mean ± std over 5 random seeds (different client partitions and sampling trajectories). For each method, hyperparameters are tuned on a validation split that preserves the federated partition (no cross-client mixing). We tune server and client learning rates on a small grid and choose the configuration maximizing validation AUC; ties are broken by lower validation ECE. In regulated environments where a shared validation set is unavailable, we recommend (i) starting with sketch dimension $m=32$ or 64, (ii) damping $\rho \in [{10}^{-4},{10}^{-2}]$, (iii) proximal weight $\mu$ proportional to observed heterogeneity (e.g., larger $\mu$ when label skew exceeds 2:1 across institutions), and (iv) fixed conservative learning rates satisfying standard smoothness bounds. Sensitivity analysis indicates that FedQuAD is more sensitive to excessive m than to moderate underestimation of curvature dimension. In all experiments we set the ridge parameter to $\rho ={10}^{-3}$ unless otherwise specified.

Table 2. Training configuration used by default in all experiments (unless otherwise noted).

Component	Setting	Notes
Clients	$K=20$, $|{\mathcal{S}}_t|=5$	$25\%$ participation
Rounds	$T=200$	synchronous rounds
Local steps	$E=5$	equalized across methods
Minibatch	256	per local step
Loss	(1) with $\alpha$ tuned	handles imbalance
Regularization	$\lambda ={10}^{-4}$	logistic regression
FedQuAD (local)	${\eta }_{\ell }=0.05$, ${\mu }_p=0.1$	Prox-SVRG steps
FedQuAD (sketch)	$m=64$, $\rho ={10}^{-3}$, ${\eta }_q=0.5$	curvature correction
Robust norm	(11) with $\tau ={10}^{-3}$	quantile-based
FedQuAD (adaptive)	$r_{max}=0.05$, $\gamma =2$	drift-control thresholds

lists the default settings used across experiments. For FedQuAD, we keep the sketch dimension moderate ($m=64$) and use damping $\rho$ to ensure the sketch-space inverse remains stable. All baselines use the same client sampling and the same total local computation budget (E steps/round) to isolate algorithmic differences in aggregation and drift control.

Metrics and targets. We report (i) AUC for discrimination, (ii) Brier score for probability accuracy, and (iii) ECE (15 bins) for calibration. Since communication rounds are the primary cost in cross-silo FL, we also report Rounds@AUC: the smallest round index t such that validation AUC exceeds a specified target (e.g., 0.87 for (D1), 0.90 for (D2)). Unless explicitly noted, all reported results assume secure aggregation without additional client-level DP noise. The optional DP mechanism described in Section 4.4 is evaluated separately and is not active in the main comparison tables.

Why robust normalization matters in this setup. Heavy-tailed accounting ratios can strongly distort optimization dynamics: A few extreme values may dominate gradients and inflate curvature estimates. Figure 2 illustrates this effect on a representative feature from (D1) by plotting empirical quantiles before and after applying the robust standardization (11). After normalization, the interquartile region is stabilized, and extreme tails are compressed, improving the conditioning of both local updates and curvature sketches.

5.2. Main Results

This subsection evaluates whether FedQuAD improves communication-round efficiency and predictive quality for federated default prediction. We focus on two questions: (i) does FedQuAD reach target AUC and calibration in fewer rounds than representative first-order and adaptive baselines, and (ii) are the gains consistent under severe client heterogeneity (Dirichlet label skew + covariate segmentation) and partial participation ($25\%$ per round)? Our main experimental results correspond to secure aggregation without DP noise; DP-enabled runs introduce additional Gaussian noise calibrated to the clipping norm and are reported separately when applicable.

Logistic regression on (D1): discrimination, calibration, and rounds-to-target.

Table 3. Logistic regression results on (D1) (mean ± std over 5 seeds). “Rounds@0.87” is the number of rounds required to reach validation AUC $\ge 0.87$.

Method	AUC ↑	Brier ↓	ECE ↓	Rounds@0.87 ↓
FedAvg	$0.871\pm 0.004$	$0.146\pm 0.003$	$0.041\pm 0.004$	$120\pm 11$
FedProx	$0.874\pm 0.003$	$0.144\pm 0.003$	$0.039\pm 0.003$	$102\pm 10$
SCAFFOLD	$0.878\pm 0.003$	$0.141\pm 0.002$	$0.036\pm 0.003$	$78\pm 9$
FedNova	$0.876\pm 0.004$	$0.143\pm 0.003$	$0.037\pm 0.003$	$91\pm 12$
FedAdam	$0.880\pm 0.003$	$0.140\pm 0.002$	$0.035\pm 0.003$	$66\pm 7$
FedQuAD	$\mathbf{0.883}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{0.137}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{0.027}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{35}\mathbf{\pm }\mathbf{6}$

reports the final test performance at $T=200$ rounds for logistic regression. FedQuAD attains the best mean AUC and the lowest calibration error (ECE), while substantially reducing the number of rounds required to reach the target validation AUC. The improvement in ECE is particularly important for credit risk, where predicted probabilities directly enter pricing, provisioning, and capital models.

Convergence curves in rounds. Figure 3 shows validation AUC versus communication rounds on (D1). The curve for FedQuAD rises sharply early and saturates sooner, consistent with the sketch-space quasi-Newton correction Section 4.3 improving effective conditioning along $\mathrm{span}\left({\mathbf{S}}^t\right)$. No legend is shown; curve identities are described in the caption. Notably, our convergence analysis assumes smoothness and (locally) strong convexity for clarity. Deep tabular models used in practice are nonconvex, and financial ratios may violate sub-Gaussian assumptions. However, empirical loss landscapes in calibrated credit models are often locally well-behaved near minimizers, especially under regularization. In practice, when strong convexity is not satisfied globally, we recommend moderate damping ($\rho$), conservative step sizes, and monitoring of calibration metrics rather than relying solely on theoretical rates. The theoretical results should therefore be interpreted as stability guarantees under idealized conditions rather than exact performance predictors for real accounting data. Formal convergence guarantees for FedQuAD are provided in Appendix A.

Calibration behavior. To understand the calibration gains, Figure 4 plots validation ECE versus rounds on (D1). FedQuAD reduces ECE more quickly and to a lower level than baselines, indicating that the faster optimization does not come at the cost of unstable probability estimates. We attribute this to three interacting factors: robust normalization (11) that stabilizes feature scales, proximal drift control (14) that limits overfitting to local rare events, and curvature-aware correction Section 4.3 that improves the conditioning of global steps.

Robustness across heterogeneity severity.

Table 4. Heterogeneity sweep on (D1) with logistic regression. Smaller Dirichlet concentration implies stronger label skew. Reported values are mean ± std over 5 seeds.

Dirichlet Concentration	Method	AUC at $\mathit{T}\mathbf{=}\mathbf{200}$ ↑	Rounds@0.87 ↓
0.5	FedAdam	$0.881\pm 0.003$	$58\pm 7$
	FedQuAD	$\mathbf{0.884}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{31}\mathbf{\pm }\mathbf{5}$
0.3	FedAdam	$0.880\pm 0.003$	$66\pm 7$
	FedQuAD	$\mathbf{0.883}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{35}\mathbf{\pm }\mathbf{6}$
0.1	FedAdam	$0.874\pm 0.004$	$92\pm 10$
	FedQuAD	$\mathbf{0.879}\mathbf{\pm }\mathbf{0.003}$	$\mathbf{48}\mathbf{\pm }\mathbf{8}$

evaluates logistic regression on (D1) under varying Dirichlet concentration (smaller means more heterogeneous). FedQuAD consistently reduces rounds-to-target, and the gap widens as heterogeneity increases, which is consistent with its explicit drift control and curvature scaling. For each concentration, we keep $K=20$ and $|{\mathcal{S}}_t|=5$ and retune only the client stepsize on a small grid for fairness.

Overall, the main results show that FedQuAD improves both discrimination and calibration while dramatically reducing communication rounds required to reach target AUC. The next subsections study nonconvex tabular models, end-to-end communication costs, and ablations of key FedQuAD components.

5.3. Deep Tabular Model and Communication Efficiency

We next evaluate FedQuAD on a nonconvex deep tabular model and quantify end-to-end communication efficiency. The goal is twofold: (i) verify that sketch-space curvature correction remains beneficial when the curvature is approximate (Gauss–Newton/empirical Fisher) and the loss landscape is nonconvex, and (ii) assess whether the additional curvature payload is offset by fewer rounds-to-target, resulting in lower bytes-to-target.

Tabular MLP performance on (D2). We train a two-hidden-layer MLP (256, 128) with ReLU activations using the loss (1).

Table 5. Tabular MLP results on (D2) (mean ± std over 5 seeds). “Rounds@0.90” is the number of rounds required to reach validation AUC $\ge 0.90$.

Method	AUC ↑	Brier ↓	ECE ↓	Rounds@0.90 ↓
FedAvg	$0.901\pm 0.003$	$0.129\pm 0.002$	$0.031\pm 0.003$	$148\pm 13$
FedProx	$0.903\pm 0.003$	$0.128\pm 0.002$	$0.029\pm 0.003$	$131\pm 12$
SCAFFOLD	$0.906\pm 0.003$	$0.126\pm 0.002$	$0.026\pm 0.002$	$104\pm 10$
FedAdam	$0.908\pm 0.002$	$0.125\pm 0.002$	$0.024\pm 0.002$	$86\pm 9$
FedQuAD	$\mathbf{0.912}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{0.122}\mathbf{\pm }\mathbf{0.001}$	$\mathbf{0.018}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{49}\mathbf{\pm }\mathbf{7}$

reports test AUC, Brier score, ECE, and rounds required to reach validation AUC $\ge 0.90$ on (D2). FedQuAD achieves the best AUC and calibration while requiring substantially fewer communication rounds than baselines that use only first-order information.

Figure 5 plots validation AUC versus rounds on (D2). No legend is used; the curve identities are specified in the caption. FedQuAD again exhibits a steeper early rise, indicating that sketch-space preconditioning accelerates progress even when curvature is approximate and clients perform nonconvex local optimization.

Communication accounting and bytes-to-target. FedQuAD transmits $({\Delta }_k^t,{\mathbf{g}}_{k,s}^t,{\mathbf{C}}_k^t)$ per participating client. With sketch size $m=64$, ${\mathbf{g}}_{k,s}^t$ contributes m scalars, and ${\mathbf{C}}_k^t$ contributes $m(m+1)/2=2080$ scalars (upper triangle).

Table 6. Communication cost on (D1) for reaching AUC $\ge 0.87$. Payload excludes protocol overhead and assumes 32-bit floats; ${\mathbf{C}}_k^t$ transmits the upper triangle.

Method	Uplink/Client/Round (KB) ↓	Rounds@0.87 ↓	Bytes-to-Target (MB) ↓
FedAvg	≈256.0	120	≈153.6
FedAdam	≈256.0	66	≈84.5
FedQuAD	≈264.6	35	≈46.4

reports per-client uplink per round and bytes-to-target for reaching AUC $\ge 0.87$ on (D1) using logistic regression. Although FedQuAD adds a small curvature payload, it substantially reduces rounds-to-target, leading to lower overall transmitted bytes. Furthermore, secure aggregation typically introduces a small constant number of additional communication phases plus auxiliary key material, but the dominant term remains transmitting the masked update vectors/matrices themselves [19]. The optional client-level DP mechanism is implemented through local clipping and server-side noise addition (Section 4.4) and does not change message dimensionality; it affects accuracy rather than bytes. Therefore, while absolute bytes depend on the concrete secure-aggregation/DP instantiation, the relative bytes-to-target trade-offs are still governed by the $O\left(P\right)$ vs. $O(P+m^2)$ payload scaling and the reduced rounds-to-target.

Where the savings come from. To make the trade-off explicit, Figure 6 plots cumulative uplink bytes (per participating client) versus achieved validation AUC on (D1). No legend is shown; the caption describes curve identities. FedQuAD attains a given AUC at substantially lower cumulative bytes because its AUC rises rapidly in early rounds. This metric is often more actionable than per-round payload in cross-silo deployments, where the dominant cost is total secure-computation and transmission volume to reach a target model quality.

Sensitivity to sketch dimension. Finally,

Table 7. Sketch-dimension sweep for FedQuAD on (D1) with logistic regression (mean ± std over 5 seeds). “Overhead” is the extra uplink per round beyond transmitting ${\Delta }_k^t$ (32-bit floats, upper-triangular ${\mathbf{C}}_k^t$).

Sketch Size m	AUC at $\mathit{T}=200$ ↑	Rounds@0.87 ↓	Extra Uplink/Round (KB) ↓
32	$0.881\pm 0.003$	$44\pm 7$	≈2.3
64	$0.883\pm 0.002$	$\mathbf{35}\mathbf{\pm }\mathbf{6}$	≈8.6
96	$0.884\pm 0.002$	$32\pm 6$	≈18.8
128	$0.884\pm 0.002$	$31\pm 5$	≈33.8

studies the impact of sketch dimension m on (D1) using logistic regression. As m increases, rounds-to-target improve until diminishing returns set in, while per-round uplink increases quadratically in m. The default $m=64$ offers a favorable trade-off. In this regime, increasing m beyond 64 yields only marginal gains in rounds-to-target while increasing overhead quadratically; thus $m=64$ is a sensible default for similar tabular credit tasks. If communication is very constrained, $m=32$ still provides most of the benefit; if rounds are extremely expensive and bandwidth allows, $m\in [96,128]$ can extract the last few rounds of improvement.

Overall, the deep-model results confirm that FedQuAD’s curvature sketching remains effective beyond convex objectives, and communication accounting shows that the additional curvature channel is typically offset by large reductions in rounds-to-target and total transmitted bytes.

5.4. Ablation Study

This subsection isolates which components of FedQuAD contribute most to the observed reductions in communication rounds and improvements in calibration. We focus on logistic regression on (D1), where optimization effects are easier to interpret, and evaluate three ablations that remove one component at a time while keeping the remaining settings fixed: (i) NoCurv: remove the sketch-space quasi-Newton correction by setting ${\eta }_q=0$ in (13); (ii) NoVR: remove variance reduction by replacing (15) with minibatch SGD on the proximal objective (14); (iii) NoRobustNorm: replace robust normalization (11) with standard z-score normalization computed locally (mean/std per client). All variants use the same $K=20$, $|{\mathcal{S}}_t|=5$, $E=5$ local steps per round, and are averaged over 5 random seeds.

End performance and rounds-to-target.

Table 8. Ablation on (D1) with logistic regression (mean ± std over 5 seeds).

Variant	AUC ↑	ECE ↓	Rounds@0.87 ↓
FedQuAD	$0.883\pm 0.002$	$\mathbf{0.027}\mathbf{\pm }\mathbf{0.002}$	$\mathbf{35}\mathbf{\pm }\mathbf{6}$
NoCurv	$0.879\pm 0.003$	$0.033\pm 0.003$	$58\pm 8$
NoVR	$0.881\pm 0.003$	$0.031\pm 0.003$	$47\pm 7$
NoRobustNorm	$0.878\pm 0.004$	$0.038\pm 0.004$	$61\pm 10$

reports test AUC, ECE, and rounds-to-target. Removing curvature correction causes the largest increase in Rounds@0.87, confirming that sketch-space preconditioning is the main driver of communication efficiency. Removing variance reduction increases rounds and worsens calibration modestly, consistent with higher stochasticity under rare-event labels. Removing robust normalization produces both slower convergence and noticeably worse ECE, reflecting sensitivity to heavy-tailed accounting features.

Convergence and calibration trajectories. To understand when each component matters, Figure 7 plots validation AUC versus rounds, and Figure 8 plots validation ECE versus rounds. No legends are shown; curve identities are described in the captions. The NoCurv curve rises more slowly throughout training, especially in early rounds, indicating that curvature correction accelerates global progress from the start. The NoRobustNorm curve is unstable early (slow AUC rise and higher ECE), suggesting that heavy-tailed features degrade both local optimization and curvature estimation. NoVR exhibits intermediate behavior: AUC approaches FedQuAD eventually but requires more rounds, and ECE remains higher, consistent with noisier local updates under imbalance.

Step statistics and stability. Ablations also differ in update stability.

Table 9. Update magnitude and stability indicators on (D1), averaged over rounds $t\in [1,200]$ and 5 seeds. “AUC drop” is the fraction of rounds where validation AUC decreases from the previous round.

Variant	$\mathbb{E}\parallel {\mathbf{\Delta }}_{\mathit{k}}^{\mathit{t}}{\parallel }_2$↓	AUC Drop Rate ↓
FedQuAD	$1.00$	$0.18$
NoCurv	$1.01$	$0.20$
NoVR	$1.12$	$0.24$
NoRobustNorm	$1.28$	$0.31$

reports (i) the mean ${\ell }_2$ norm of client deltas $\parallel {\Delta }_k^t{\parallel }_2$ and (ii) the fraction of rounds in which the validation AUC decreases relative to the previous round (a simple instability indicator). Removing robust normalization increases update magnitudes and instability; removing variance reduction modestly increases both; removing curvature does not directly increase instability (it primarily slows progress), which aligns with the curvature term acting as a structured, damped correction in sketch space.

Component interactions. Finally, Figure 9 summarizes the rounds-to-target for each variant and highlights interactions: Curvature correction provides the largest gain alone, but the best performance requires combining it with robust normalization and variance reduction, which improve stability and calibration and enable the curvature channel to operate on better-conditioned features.

Overall, the ablation study confirms that the sketch-space curvature correction is the primary driver of round efficiency, while robust normalization and variance reduction play crucial supporting roles by improving conditioning, reducing stochasticity under imbalance, and stabilizing probability calibration.

5.5. Discussions

Across heterogeneous cross-silo partitions, FedQuAD consistently reduces rounds-to-target relative to first-order baselines while maintaining competitive AUC and improved calibration stability, with the sketch-space quasi-Newton correction being especially beneficial under stronger label skew and covariate shift, where purely first-order aggregation is more vulnerable to drift and curvature mismatch. The additional communication cost from the $m\times m$ sketch remains moderate for practical choices such as $m\le 64$, so bytes-to-target improvements persist in this regime. At the same time, our results should be interpreted with appropriate caution: All experiments are conducted on public credit-risk-style datasets with simulated cross-silo (multi-institution) partitions, where heterogeneity is induced through controlled label skew and covariate shift rather than observed from raw proprietary bank data. Although this setup captures realistic statistical heterogeneity patterns, it does not fully represent operational constraints such as compliance workflows, asynchronous participation, or institution-specific feature engineering, so validation on live inter-institution deployments remains future work. Our primary threat model assumes an honest-but-curious server and honest clients, and unless otherwise stated, all reported experiments assume secure aggregation without additional client-level differential privacy (DP) noise, meaning that the server observes only aggregated masked updates rather than individual client messages. However, aggregated curvature sketches are still second-order summary statistics and may reveal distribution-level structure when participation is small, so practical deployments should combine secure aggregation with minimum participation thresholds and, where required, client-level DP. For reproducibility, we clarify that for deep tabular models the curvature sketch corresponds to a Gauss–Newton/empirical Fisher-type approximation obtained by projecting per-client gradients into the shared sketch subspace and aggregating outer products, yielding an $m\times m$ matrix that approximates ${\left({\mathbf{S}}^t\right)}^{\top }{\nabla }^{2}F\left({\mathbf{w}}^t\right){\mathbf{S}}^t$ up to stochastic and heterogeneity noise. Unless otherwise stated, we use sketch dimension $m=64$ and damping parameter $\rho ={10}^{-3}$ in Equation (20); the ridge term $\rho \mathbf{I}$ stabilizes inversion under limited participation or poor conditioning, and all remaining hyperparameters, including m, $\rho$, proximal weight $\mu$, and adaptive drift thresholds, are summarized in Table 2.

6. Conclusions

This paper investigated how curvature-aware optimization can be incorporated into federated learning for credit default prediction while remaining communication-efficient and compatible with privacy constraints. We proposed FedQuAD, which integrates proximal stabilization, variance reduction, and a sketch-based quasi-Newton server correction to address the challenges of heterogeneous, imbalanced financial data across institutions. Empirical results on heterogeneous credit-risk benchmarks with simulated cross-silo partitions show that the proposed approach can significantly reduce the number of communication rounds required to reach target predictive performance while maintaining competitive AUC and stable calibration. These results suggest that compressed second-order information, when incorporated through randomized sketching, can provide practical benefits for collaborative financial modeling without requiring the transmission of full curvature matrices.

Despite these promising results, several limitations should be acknowledged. The experiments rely on public datasets with simulated institutional partitions rather than proprietary multi-bank accounting data, and the theoretical guarantees rely on standard smoothness and convexity assumptions that may only approximately hold for real financial data. In addition, the reported results assume secure aggregation without additional differential privacy noise, and practical deployments may require stronger privacy mechanisms depending on regulatory constraints. Future research should therefore investigate stress-test scenarios with extreme heterogeneity or limited communication budgets, explore adaptive strategies for selecting sketch dimensions and other hyperparameters, and validate curvature-aware federated optimization in real multi-institution financial environments.