A Bayesian Multi-Armed Bandit Algorithm for Dynamic End-to-End Routing in SDN-Based Networks with Piecewise-Stationary Rewards

Round 1

Reviewer 1 Report

- The paper proposes a variant of the well known Multi-Armed Bandit (MAB) algorithm, called React-UCB, to apply to a SDN architecture in order to control the network routing with the aim at discovering the path with the minimum delay. The authors claim that the proposed algorithm is able to find the optimum routing path between a couple of ingress-egress points, possibly reacting to sudden variations of the network congestion.

- The content of the paper is quite interesting, particularly as far as the description of the variation of the MAB algorithm is concerned. However, in my opinion, the limitation of the paper is that the authors do not prove the effectiveness of the proposed MAB algorithm and its applicability in real network scenarios . 

- The considered network model is quite synthetic and does not reflect the characteristic of a real network topology. Moreover, the reactiveness of the method is not proved; the question is: how much time is needed so that the method determine a path rerouting in presence of an emerging network congestion? In other words, which is the dynamic of the network congestion that the method is able to control?

- In my opinion, the considered network scenario is too simple and is not able to point out possible conflicts among the choice of routing in case of multiple paths that have to be associated to traffic generated by a set of source-destination hosts.

- The authors should consider a real complex network topology loaded by a traffic modelling a real network beahviour. The results shown in the paper do not allow to evaluate the congestion degree of the various network elements and, consequently, the advantages achieved by the proposed method. Moreover, a broader comparison with classical schemes is needed.

Author Response

Please see attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

- Introduction needs to be further expanded. 

- what are the practical implications of the difference between M-UCB and the proposed algorithm?

- the similarity of KL-UCB presents a lack of novelty in your algorithm? Please elaborate more. 

- How about a distributed solution? Please clarify better the nature of your algorithm.

- More experiments regarding the delay and the routing part are essential to show the network effect. How about other metrics that are influenced by the delay, such as the throughput?

The language is easy to follow. Some minor revision is required.

Author Response

Please see attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors focus their study on the field of software defined networking and they introduced the design of a piecewise stationary Bayesian multi armed bandit approach for online optimum end to end dynamic routing of the data flows considering the programmable characteristics of the networking systems. The manuscript is overall well written and easy to follow and the authors have well thought out their main contributions. The provided theoretical analysis is concrete, complete, and correct and the authors have provided all the intermediate steps in order to enable the average reader to easily follow it. Furthermore, the authors have provided a very detailed numerical evaluation in order to demonstrate the pure operation and the performance of the proposed framework. The authors are highly encouraged to consider the following suggestions provided by the reviewer in order to improve the scientific depth of their manuscript, as well as they need to address the following minor comments in order to improve the quality of presentation of their manuscript. Initially, in Section 1, the authors need to provide some pointers to the average reader who is not expert in the field to basic machine learning approaches that have been applied in networking systems, such as Huang, Xin-Lin, Xiaomin Ma, and Fei Hu. "Machine learning and intelligent communications." Mobile Networks and Applications 23 (2018): 68-70, in order for the reader to be able to follow the rest of the provided analysis. In section three, the authors need to include a table summarizing the main notation that has been used in the paper and provide the units of the corresponding metrics wherever this is appropriate as currently this is missing from the manuscript. In Section 3, the authors need to include an additional subsection providing the theoretical analysis of the computational complexity of the proposed algorithm and discuss also its implementation cost. In Section 4, the authors need to provide some comparative results to other approaches that have been introduced in the literature in order to quantify the drawbacks and benefits of the proposed framework. Currently, the provided comparative evaluation mailing compares the proposed framework to benchmarking approaches and cannot demonstrate its drawbacks and benefits. Finally, the overall manuscript needs to be checked for typos, syntax, and grammar errors in order to improve the quality of its presentation.

Finally, the overall manuscript needs to be checked for typos, syntax, and grammar errors in order to improve the quality of its presentation.

Author Response

Please see attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

My comments have been answered.

Reviewer 3 Report

The authors have addressed in detail the reviewers' comments. This reviewer has no further concerns regarding this paper.