Analysis of Function Approximation and Stability of General DNNs in Directed Acyclic Graphs Using Un-Rectifying Analysis
Round 1
Reviewer 1 Report
The manuscript ‘Analysis of function approximation and stability of general DNNs in directed acyclic graphs using un-rectifying analysis’ is dedicated an analysis to activation functions, non-linear transformations, and DNN architectures in order to use the un-rectifying method to analyze DNNs via directed acyclic graphs (DAGs).
The authors prove that a DNN DAG divides the input space via partition refinement using either a composition of activation functions along a path or a fusion operation combining inputs from more than one path in the graph. This makes it possible to approximate a target function in a coarse-to-fine manner by applying a local approximating function to each partition of the input space. The authors also consider that under mild assumptions related to point-wise CPWL activation functions and nonlinear transformations, the stability of a DNN against local input perturbations can be maintained using sparse/compressible weight coefficients associated with incident arcs to a node.
This article seems to be complete study which makes a significant contribution in the field o deep feedforward neural networks. The authors have proven that ta general DNN “divides" the input space, “conquers" the target function by applying a simple approximating function over each partition region, and “sparsifies" weight coefficients to ensure robustness against input perturbations.
Comments and Suggestions for Authors:
• Generally the conclusions consistent with the evidence and arguments presented.
• Unfortunately, the authors do not consider recent studies on this topic. For theoretical framework and bibliography additional current references should be included to new research 2022-2023.
• The research topic seems to be very relevant, it seems promising to analyze using a composition of activation functions or a fusion operation combining inputs from more than one path to a node. The authors could include a discussion of these issues in their study.
• Using an axiomatic approach, autors established a systematic approach to representing deep feedforward neural networks (DNNs) as directed acyclic graphs (DAGs). However, to make the work more convincing, рlease describe in more detail how specific datasets were selected for analysis and why these particular datases were chosen
• Also, the conclusions given in the article would be more convincing if the authors described their practical application or plans for practical application.
• The process of discussing the results can be extended by applying the results and extrapolating them to other similar studies.
• Please describe in detail how your study fits for aims and scope of Electronics
Comments for author File: 
Comments.pdf
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 2 Report
The work addresses the important need to better understand how neural network complete a learning task, and interpret the learning process. In particular, it is relevant to identify if different portions of the neural network are assigned to specific sub-tasks of the original learning problem.
The research described in this work is relevant for the interpretability and explainability of complex deep learning models.
Additional comments:
The main question addressed by this manuscript is to understand how different parts of a dense feed-forward neural networks architecture interact with each other to perform a specific predictive task. Such an understanding could potentially help researchers in identifying sub-components dedicated to specific sub-tasks. This partitioning of the DNN architecture into sub-components may help researchers in interpreting and/or explaining how the DNN architecture works and how it generates an inferential answer to a specific queried predictive task.
The original topic is relevant to the deep learning community, and it addresses the specific gap of interpretability and explainability of DNN models. Interpretability and explainability of DNN predictions are still an active field of research, and much still needs to be done towards understanding the intrinsic and complex mechanisms of deep learning models.
Many contributions in the deep learning literature describe how to construct an effective neural network that attains a desired predictive performance by solving an inverse problem. However, these methods do not address the need of explaining why the identified architectures work better than others. This is the specific gap addressed by this work.
The paper should ‘ease’ the reader into the topic. To this effect, adding a ‘Mathematical background section’ would help.
Moreover, there is an evident disconnection between the theoretical work presented in this manuscript and the relevance to solving specific problems.
The authors of the manuscript should make an effort in describing:
a. how the deep learning community will benefit from their contribution, and how this contribution gears towards advancing the technological field of deep learning applied to scientific and engineering problems.
b. What is the benefit of having an explainable and interpretable model compared to another model that (although very accurate and highly performance in making reliable predictions) is not explainable/interpretable at all?
The examples presented in the numerical section seem fragmented, and the authors should improve the explanation of how this fragmented collection of numerical results contributed to validating the theoretical discussion of the paper.
The English must be significantly improved. Many sentences are redundant, with several words repeated multiple times in the same sentence.
The abstract, introduction, and conclusion dive very quickly into too many technical details, and make the reading very difficult to follow.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
The reviewers appreciates the effort of the authors in revising the manuscript.
