FedDPA: Dynamic Prototypical Alignment for Federated Learning with Non-IID Data
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe abstract should be modified to illustrate the method of FedDPA step by step. And the mechanism of FedDPA is not analysis in mathematics formal style to compare with the state-of-art methods of references in 4.5. Theoretical analysis for FedDPA.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThis paper proposes the FedDPA method to solve the data heterogeneity problem in federated learning. The method mainly includes several components: 1) Adaptive regularization mechanism, which dynamically adjusts the regularization strength according to the heterogeneity of client data; 2) Hierarchical aggregation strategy to improve scalability and communication efficiency; 3) Contrastive alignment mechanism, which optimizes prototype representation through intra-class alignment and inter-class separation. Experiments show that FedDPA outperforms existing methods in terms of accuracy.
Overall, there are certain problems in the experimental results and theoretical analysis of this paper, which need to be greatly revised. There are four mainly weakness:
W1: Lack of SOTA related works. This paper focus on the data heterogeneity problem, but the authors do not adequately study SOTA related works, such as heuristic search-based methods [R1] and sharpness-aware minimization-based methods [R2]. The author should study more related works. Please note that I am only giving suggestions and the authors can supplement the relevant literature based on their own knowledge.
W2: Poor evaluation.
(1) The author only conducted experiments on the Cifar-10 and Cifar-100 datasets, but lacked experiments on more diverse datasets, such as TinyImageNet, FEMNIST, etc.
(2) The author only used simple CNN model. The author should conduct experiments using more models with different structures, such as ResNet and VGG.
(3) Lacks ablation study to evaluate the effectiveness of each part of the proposed method.
(4) Lacks evaluations of the impact of different configurations, such as different values of regularization parameters $\alpha$ and heterogeneity influencing factor $\beta$.
(5) In line 49-50, the manuscript claims that FedDPA can provide robust and scalable learning under dynamic conditions, but does not use experiments to prove this.
(6) In the conclusion of Section 7, line 444-445, it is mentioned that this method improves the convergence speed, but the experimental results given in the manuscript do not include a comparison of the convergence speed of different methods under heterogeneous data scenarios, which cannot support this conclusion.
W3: Lacks theoretical convergence analysis. In Section 4.5, the author claims to have conducted a meticulous and comprehensive theoretical analysis that systematically explores its convergence characteristics and computational complexity. But in fact, the analysis of convergence is limited to intuitive understanding and analysis of experimental phenomena, and no rigorous mathematical proof is given.
W4: Expression issues. In line 204, in the description of the Hierarchical aggregation component, the author only explains $Heterogeneity_k$ as the distance between vectors, but does not specify which distance it is, which is ambiguous. The algorithm diagram in the article does not give input and output either. In Figure 1, Figure 6, and Figure 7, the text is too small and difficult to read.
[R1] FedCross: Towards accurate federated learning via multi-model cross-aggregation. ICDE 2024
[R2] Generalized Federated Learning via Sharpness Aware Minimization. ICML2022
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for Authors
- The literature review briefly mentions The Federated Learning (FL) has emerged as a powerful framework for decentralized model training across distributed devices, preserving data privacy by keeping datasets localized.
- It would be highly relevant to look at more related articles to improve the paper.
- Introduction and the literature review can be improved after collecting more journals.
- No clear explanation and differences among Fingers 2-5.
- Figures 6 and 7 were not explained clearly, and I have a problem following them.
- Conclusion is more about future work than the actual conclusion.
I do not have any concern.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Comments and Suggestions for AuthorsThe authors address the majority of my concerns.
