# Supervised Learning of Neural Networks for Active Queue Management in the Internet

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

**Our contribution.**The aim of the work is to propose an algorithm for Active Queue Management based on supervised learning paradigm. We use a previously trained Convolutional Network to manage the queue. The ANN is trained based on the data obtained in simulations. We observe the impact on the behavior of the AQM mechanism based on the $P{I}^{\alpha}$ controller. In experiments we change the intensity and degree of self-similarity of network sources and observe behavior of the controller. The samples contain the sequence of incoming packets and the probability of packet dropping. The model trained this way is used as a new AQM mechanism. This paper presents its influence on the Internet transmission.

## 2. Related Works

## 3. Theoretical Background

- $H\in (0;0.5)$: negative correlation—the LRD does not occur (the SRD occurs).
- $H=0.5$: no correlation.
- $H\in (0.5;1)$: positive correlation—the LRD occurs.

## 4. Data Preparation and Neural Networks Training Process

- Learning features:
- (a)
- The last n items from the queue’s occupancy history (CNN History).
- (b)
- History of packet rejections in n last queue stateswhere $n\in [20;100;200;300;400;500;1000]$.

- Classes:
- (a)
- 11 labels that mapped the probability of packet rejection to the current transmission conditions, according to the principle shown in Table 2.

## 5. Evaluation of the Neural Network-Based AQM

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Index, C.V.N. Global Mobile Data Traffic Forecast Update, 2017–2022; White Paper; Cisco Press: Indianapolis, IN, USA, 2019. [Google Scholar]
- Lakshman, T.V.; Madhow, U. The performance of TCP/IP for networks with high bandwidth-delay products and random loss. IEEE/ACM Trans. Netw.
**1997**, 5, 336–350. [Google Scholar] [CrossRef] - Hema, R.M.; Murugesan, G.; Jude, M.J.A.; Diniesh, V.C.; Sree Arthi, D.; Malini, S. Active queue versus passive queue—An experimental analysis on multi-hop wireless networks. In Proceedings of the International Conference on Computer Communication and Informatics (ICCCI), Coimbatore, India, 5–7 January 2017; pp. 1–5. [Google Scholar] [CrossRef]
- Braden, B.; Clark, D.; Crowcroft, J.; Davie, B.; Deering, S.; Estrin, D.; Floyd, S.; Jacobson, V.; Minshall, G.; Partridge, C.; et al. Recommendations on Queue Management and Congestion Avoidance in the Internet. RFC
**1998**, 2309, 1–17. [Google Scholar] - Wu, H.; Feng, Z.; Guo, C.; Zhang, Y. ICTCP: Incast Congestion Control for TCP in Data-Center Networks. IEEE/ACM Trans. Netw.
**2013**, 21, 345–358. [Google Scholar] [CrossRef] - Floyd, S.; Jacobson, V. Random Early Detection gateways for congestion avoidance. IEEE/ACM Trans. Netw.
**1993**, 1, 397–413. [Google Scholar] [CrossRef] - Xue, L. Simulation of Network Congestion Control Based on RED Technology. In Proceedings of the 2013 International Conference on Computational and Information Sciences, Shiyan, Hubai, China, 21–23 June 2013; pp. 1497–1500. [Google Scholar]
- Bhatnagar, S.; Patro, R. A proof of convergence of the B-RED and P-RED algorithms for random early detection. IEEE Commun. Lett.
**2009**, 13, 809–811. [Google Scholar] [CrossRef] - Ho, H.J.; Lin, W.M. AURED—Autonomous Random Early Detection for TCP Congestion Control. In Proceedings of the 3rd International Conference on Systems and Networks Communications Malta, Sliema, Malta, 26–31 October 2008. [Google Scholar]
- Domańska, J.; Augustyn, D.; Domański, A. The choice of optimal 3-rd order polynomial packet dropping function for NLRED in the presence of self-similar traffic. Bull. Pol. Acad. Sci. Tech. Sci.
**2012**, 60, 779–786. [Google Scholar] [CrossRef] - Kachhad, K.; Lathigara, A. ModRED: Modified RED an Efficient Congestion Control Algorithm for Wireless Network. Int. Res. J. Eng. Technol. (IRJET)
**2018**, 5, 1879–1884. [Google Scholar] - Hamdi, M.M.; Rashid, S.A.; Ismail, M.; Altahrawi, M.A.; Mansor, M.F.; AbuFoul, M.K. Performance Evaluation of Active Queue Management Algorithms in Large Network. In Proceedings of the 2018 IEEE 4th International Symposium on Telecommunication Technologies (ISTT), Selangor, Malaysia, 26–28 November 2018; pp. 1–6. [Google Scholar] [CrossRef]
- Liu, Z.; Sun, J.; Hu, S.; Hu, X. An Adaptive AQM Algorithm Based on a Novel Information Compression Model. IEEE Access
**2018**, 6, 31180–31190. [Google Scholar] [CrossRef] - Tan, L.; Zhang, W.; Peng, G.; Chen, G. Stability of TCP/RED systems in AQM routers. IEEE Trans. Autom. Control
**2006**, 51, 1393–1398. [Google Scholar] [CrossRef] - Domańska, J.; Domański, A.; Czachórski, T.; Klamka, J.; Marek, D.; Szyguła, J. The Influence of the Traffic Self-similarity on the Choice of the Non-integer Order PI
^{α}Controller Parameters. In Communications in Computer and Information Science; Springer International Publishing: Berlin/Heidelberg, Germany, 2018; Volume 935, pp. 76–83. [Google Scholar] [CrossRef] - Melchor-Aquilar, D.; Castillo-Tores, V. Stability Analysis of Proportional-Integral AQM Controllers Supporting TCP Flows. Comput. Sist.
**2007**, 10, 401–414. [Google Scholar] [CrossRef] - Ustebay, D.; Ozbay, H. Switching Resilient PI Controllers for Active Queue Management of TCP Flows. In Proceedings of the 2007 IEEE International Conference on Networking, Sensing and Control, London, UK, 15–17 April 2007; pp. 574–578. [Google Scholar] [CrossRef]
- Melchor-Aquilar, D.; Niculescu, S. Computing non-fragile PI controllers for delay models of TCP/AQM networks. Int. J. Control.
**2009**, 82, 2249–2259. [Google Scholar] [CrossRef] - Grazia, C.A.; Patriciello, N.; Klapez, M.; Casoni, M. Which AQM fits IoT better? In Proceedings of the IEEE 3rd International Forum on Research and Technologies for Society and Industry (RTSI), Modena, Italy, 11–13 September 2017; pp. 1–6. [Google Scholar] [CrossRef]
- Krajewski, W.; Viaro, U. On robust fractional order PI controller for TCP packet flow. In Proceedings of the BOS Conference: Systems and Operational Research, Warsaw, Poland, 24–26 September 2014. [Google Scholar]
- Marek, D.; Domański, A.; Domańska, J.; Czachórski, T.; Klamka, J.; Szyguła, J. Combined diffusion approximation–simulation model of AQM’s transient behavior. Comput. Commun.
**2021**, 166, 40–48. [Google Scholar] [CrossRef] - Chollet, F. Deep Learning with Python; Manning: Berkeley, CA, USA, 2017. [Google Scholar]
- Chen, J.; Chen, W.; Huang, C.; Huang, S.; Chen, A. Financial Time-Series Data Analysis Using Deep Convolutional Neural Networks. In Proceedings of the 7th International Conference on Cloud Computing and Big Data (CCBD), Macau, China, 16–18 November 2016; pp. 87–92. [Google Scholar] [CrossRef]
- Jia, W.; Liu, Y.; Liu, Y.; Wang, J. Detection Mechanism Against DDoS Attacks based on Convolutional Neural Network in SINET. In Proceedings of the IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing China, 12–14 June 2020; Volume 1, pp. 1144–1148. [Google Scholar] [CrossRef]
- Filus, K.; Domański, A.; Domańska, J.; Marek, D.; Szyguła, J. Long-Range Dependent Traffic Classification with Convolutional Neural Networks Based on Hurst Exponent Analysis. Entropy
**2020**, 22, 1159. [Google Scholar] [CrossRef] [PubMed] - Szyguła, J.; Domański, A.; Domańska, J.; Czachórski, T.; Marek, D.; Klamka, J. AQM Mechanism with Neuron Tuning Parameters. In Intelligent Information and Database Systems; Springer International Publishing: Berlin/Heidelberg, Germany, 2020; pp. 299–311. [Google Scholar]
- Sun, J.; Zukerman, M. An Adaptive Neuron AQM for a Stable Internet. In NETWORKING. Ad Hoc and Sensor Networks, Wireless Networks, Next Generation Internet; Springer: Berlin/Heidelberg, Germany, 2007; Volume 4479, pp. 844–854. [Google Scholar] [CrossRef] [Green Version]
- Busoniu, L.; Babuska, R.; De Schutter, B. A Comprehensive Survey of Multiagent Reinforcement Learning. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev.
**2008**, 38, 156–172. [Google Scholar] [CrossRef] [Green Version] - Hamadneh, N.; Obiedat, M.; Qawasmeh, A.; Bsoul, M. HRED, An Active Queue Management Approach For The NS2 Simulator. Recent Patents Comput. Sci.
**2018**, 12. [Google Scholar] [CrossRef] - Krajewski, W.; Viaro, U. Fractional order PI controllers for TCP packet flow ensuring given modulus margins. Control Cybern.
**2014**, 43, 493–505. [Google Scholar] - Domanski, A.; Domanska, J.; Czachórski, T.; Klamka, J. The use of a non-integer order PI controller with an active queue management mechanism. Int. J. Appl. Math. Comput. Sci.
**2016**, 26, 777–789. [Google Scholar] [CrossRef] [Green Version] - Su, Y.; Huang, L.; Feng, C. QRED: A Q-Learning-based active queue management scheme. J. Internet Technol.
**2018**, 19, 1169–1178. [Google Scholar] [CrossRef] - Bisoy, S.K.; Pandey, P.K.; Pati, B. Design of an active queue management technique based on neural networks for congestion control. In Proceedings of the IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS), Bhubaneswar, Odisha, India, 17–20 December 2017; pp. 1–6. [Google Scholar]
- Wang, H.; Chen, J.; Liao, C.; Tian, Z. An Artificial Intelligence Approach to Price Design for Improving AQM Performance. In Proceedings of the 2011 IEEE Global Telecommunications Conference—GLOBECOM 2011, Houston, TX, USA, 5–9 December 2011; pp. 1–5. [Google Scholar]
- Xiao, P.; Tian, Y. Design of a Robust Active Queue Management Algorithm Based on Adaptive Neuron Pid. In Proceedings of the 2006 International Conference on Machine Learning and Cybernetics, Dalian, China, 13–16 August 2006; pp. 308–313. [Google Scholar]
- Meng, Z.; Qiao, J.; Zhang, L. Design and Implementation: Adaptive Active Queue Management Algorithm Based on Neural Network. In Proceedings of the 10th International Conference on Computational Intelligence and Security, Kunming, Yunnan, China, 15–16 November 2014; pp. 104–108. [Google Scholar] [CrossRef]
- Hu, M.; Mukaidani, H. Nonlinear Model Predictive Congestion Control Based on LSTM for Active Queue Management in TCP Network. In Proceedings of the 12th Asian Control Conference (ASCC), Kitakyushu-shi, Japan, 9–12 June 2019; pp. 710–715. [Google Scholar]
- Gomez, C.A.; Wang, X.; Shami, A. Intelligent Active Queue Management Using Explicit Congestion Notification. In Proceedings of the IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA, 9–13 December 2019; pp. 1–6. [Google Scholar] [CrossRef] [Green Version]
- Bisoy, S.K.; Pattnaik, P.K. An AQM Controller Based on Feed-Forward Neural Networks for Stable Internet. Arab. J. Sci. Eng.
**2018**, 3993–4004. [Google Scholar] [CrossRef] - Zhou, C.; Di, D.; Chen, Q.; Guo, J. An Adaptive AQM Algorithm Based on Neuron Reinforcement Learning. In Proceedings of the IEEE International Conference on Control and Automation, Christchurch, New Zealand, 9–11 December 2009; pp. 1342–1346. [Google Scholar]
- Jin, W.; Gu, R.; Ji, Y.; Dong, T.; Yin, J.; Liu, Z. Dynamic traffic aware active queue management using deep reinforcement learning. Electron. Lett.
**2019**, 55. [Google Scholar] [CrossRef] - Domańska, J.; Domański, A.; Czachórski, T. Estimating the Intensity of Long-Range Dependence in Real and Synthetic Traffic Traces. In Computer Networks; Springer International Publishing: Berlin/Heidelberg, Germany, 2015; Volume 522, pp. 11–22. [Google Scholar] [CrossRef]
- Willinger, W.; Leland, W.; Taqqu, M. On the self-similar nature of traffic. IEEE/ACM Trans. Netw.
**1994**. [Google Scholar] [CrossRef] [Green Version] - Paxson, V.; Floyd, S. Wide area traffic: The failure of Poisson modeling. IEEE/ACM Trans. Netw.
**1995**, 3, 226–244. [Google Scholar] [CrossRef] [Green Version] - Feldmann, A.; Gilbert, A.C.; Willinger, W.; Kurtz, T.G. The Changing Nature of Network Traffic: Scaling Phenomena. SIGCOMM Comput. Commun. Rev.
**1998**, 28, 5–29. [Google Scholar] [CrossRef] - Li, Z.; Xing, W.; Khamaiseh, S.; Xu, D. Detecting saturation attacks based on self-similarity of OpenFlow traffic. IEEE Trans. Netw. Serv. Manag.
**2019**, 17, 607–621. [Google Scholar] [CrossRef] - Willinger, W.; Taqqu, M.S.; Wilson, D.V. Lessons from “On the Self-Similar Nature of Ethernet Traffic”. SIGCOMM Comput. Commun. Rev.
**2019**, 49, 56–62. [Google Scholar] [CrossRef] - Yin, C.; Zhu, Y.; Fei, J.; He, X. A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access
**2017**, 5, 21954–21961. [Google Scholar] [CrossRef] - Kravchik, M.; Shabtai, A. Efficient cyber attack detection in industrial control systems using lightweight neural networks and pca. IEEE Trans. Dependable Secur. Comput.
**2021**. [Google Scholar] [CrossRef] - Jiang, F.; Fu, Y.; Gupta, B.B.; Liang, Y.; Rho, S.; Lou, F.; Meng, F.; Tian, Z. Deep learning based multi-channel intelligent attack detection for data security. IEEE Trans. Sustain. Comput.
**2018**, 5, 204–212. [Google Scholar] [CrossRef] - Ring, M.; Schlör, D.; Landes, D.; Hotho, A. Flow-based network traffic generation using generative adversarial networks. Comput. Secur.
**2019**, 82, 156–172. [Google Scholar] [CrossRef] [Green Version] - D’Angelo, G.; Palmieri, F. Network traffic classification using deep convolutional recurrent autoencoder neural networks for spatial–temporal features extraction. J. Netw. Comput. Appl.
**2021**, 173, 102890. [Google Scholar] [CrossRef] - Meidan, Y.; Bohadana, M.; Mathov, Y.; Mirsky, Y.; Shabtai, A.; Breitenbacher, D.; Elovici, Y. N-baiot—network-based detection of iot botnet attacks using deep autoencoders. IEEE Pervasive Comput.
**2018**, 17, 12–22. [Google Scholar] [CrossRef] [Green Version] - Li, J.; Liu, C.; Gong, Y. Layer trajectory LSTM. arXiv
**2018**. arXiv:1808.09522. [Google Scholar] - Fu, R.; Zhang, Z.; Li, L. Using LSTM and GRU neural network methods for traffic flow prediction. In Proceedings of the 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), Wuhan, China, 11–13 November 2016; IEEE: Red Hook, NY, USA, 2016; pp. 324–328. [Google Scholar] [CrossRef]
- Zhou, B.; Lapedriza, A.; Khosla, A.; Oliva, A.; Torralba, A. Places: A 10 Million Image Database for Scene Recognition. IEEE Trans. Pattern Anal. Mach. Intell.
**2018**, 40, 1452–1464. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Optimisation Algorithm—Adaptive Moment Estimation(Adam). Available online: https://towardsdatascience.com/optimisation-algorithm-adaptive-moment-estimation-adam-92144d75e232 (accessed on 12 June 2021).
- Domańska, J.; Domański, A. AQM in Linux based routers—Comparing with analytical and simulation results. In Proceedings of the 5th International Conference: Internet in the Information Society, Theoretical and Applied Informatics, Heraklion, Greece, 22–24 July 2008; Volume 20, pp. 277–292. [Google Scholar]

**Figure 1.**The conceptual structure of a Convolutional Neural Network based classifier used to model an Active Queue Management mechanism.

**Figure 3.**Distribution of queue length obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}1$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.4$, $H=0.5$ (

**left**), $H=0.9$ (

**right**).

**Figure 4.**Distribution of queue length obtained for CNN model with the last layer activation function Softmax, trained using data regarding $P{I}^{\alpha}1$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.4$, $H=0.5$ (

**left**), $H=0.9$ (

**right**).

**Figure 5.**Distribution of queue length obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}2$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.5$, $H=0.5$ (

**left**), $H=0.9$ (

**right**).

**Figure 6.**Distribution of queue length obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}3$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.6$, $H=0.5$ (

**left**), $H=0.9$ (

**right**).

**Figure 7.**Distribution of queue length obtained for CNN controller trained using data regarding three $P{I}^{\alpha}$ controllers, $H=0.5$ (

**left**), $H=0.9$ (

**right**).

**Figure 8.**Queue occupancy obtained for CNN controller trained using data regarding three $P{I}^{\alpha}$ controllers, with CNN History $=500$, $H=0.5$ (

**left**), $H=0.9$ (

**right**).

${\mathit{K}}_{\mathit{P}}$ | ${\mathit{K}}_{\mathit{I}}$ | $\mathit{\alpha}$ | |
---|---|---|---|

$P{I}^{\alpha}1$ | 0.0001 | 0.0004 | −0.4 |

$P{I}^{\alpha}2$ | 0.0001 | 0.0004 | −0.5 |

$P{I}^{\alpha}3$ | 0.0001 | 0.0004 | −0.6 |

Decision Class | Probability Interval [%] |
---|---|

1 | [0;5) |

2 | [5;15) |

3 | [15;25) |

4 | [25;35) |

5 | [35;45) |

6 | [45;55) |

7 | [55;65) |

8 | [65;75) |

9 | [75;85) |

10 | [85;95) |

11 | [95;100] |

**Table 3.**The accuracy measurements for testing the CNN model trained on data regarding three $P{I}^{\alpha}1$, $P{I}^{\alpha}2$ and $P{I}^{\alpha}3$ controllers, n last items in queue occupancy history taken into consideration (we used Softmax function as an activation function of the last layer).

Softmax | ||||||||
---|---|---|---|---|---|---|---|---|

n History Length | ||||||||

20 | 100 | 200 | 300 | 400 | 500 | 1000 | ||

CNN by behavior ${\mathit{PI}}^{\mathit{\alpha}}\mathbf{1}$ | 5 epochs | 52.27 | 54.48 | 54.50 | 56.67 | 58.40 | 58.81 | 51.59 |

6 epochs | 52.36 | 54.88 | 54.68 | 56.70 | 58.42 | 58.84 | 51.72 | |

10 epochs | 52.40 | 55.50 | 54.83 | 56.74 | 58.47 | 58.90 | 51.82 | |

CNN by behavior ${\mathit{PI}}^{\mathit{\alpha}}\mathbf{2}$ | 5 epochs | 48.37 | 47.23 | 40.29 | 41.99 | 43.68 | 44.06 | 43.94 |

6 epochs | 48.45 | 47.61 | 40.32 | 42.06 | 43.74 | 44.06 | 44.00 | |

10 epochs | 48.54 | 48.42 | 40.31 | 42.30 | 43.78 | 44.24 | 44.04 | |

CNN by behavior ${\mathit{PI}}^{\mathit{\alpha}}\mathbf{3}$ | 5 epochs | 47.07 | 41.83 | 33.80 | 32.30 | 32.92 | 33.79 | 38.45 |

6 epochs | 47.18 | 42.09 | 33.81 | 32.31 | 32.93 | 33.82 | 38.44 | |

10 epochs | 47.41 | 42.62 | 34.18 | 32.32 | 32.92 | 33.83 | 38.50 |

**Table 4.**The accuracy measurements for test data for a neural network model trained with data representing the behavior of $P{I}^{\alpha}1$, $P{I}^{\alpha}2$ and $P{I}^{\alpha}3$ controllers, n last items in queue occupancy history taken into consideration (we used Sigmoid function as an activation function of the last layer).

Sigmoid | ||||||||
---|---|---|---|---|---|---|---|---|

n Last Items in Queue Occupancy History | ||||||||

20 | 100 | 200 | 300 | 400 | 500 | 1000 | ||

CNN by behavior ${\mathit{PI}}^{\mathit{\alpha}}\mathbf{1}$ | 5 epochs | 55.98 | 73.95 | 80.76 | 83.67 | 85.22 | 86.15 | 88.46 |

6 epochs | 56.04 | 74.06 | 80.88 | 83.80 | 85.39 | 86.31 | 88.74 | |

10 epochs | 56.16 | 74.38 | 81.19 | 84.13 | 85.69 | 86.64 | 89.46 | |

CNN by behavior ${\mathit{PI}}^{\mathit{\alpha}}\mathbf{2}$ | 5 epochs | 50.34 | 70.70 | 75.94 | 76.92 | 77.03 | 77.20 | 83.71 |

6 epochs | 50.38 | 70.83 | 76.08 | 77.06 | 77.18 | 77.32 | 84.04 | |

10 epochs | 50.53 | 71.13 | 76.40 | 77.36 | 77.57 | 77.79 | 84.81 | |

CNN by behavior ${\mathit{PI}}^{\mathit{\alpha}}\mathbf{3}$ | 5 epochs | 48.77 | 67.16 | 67.54 | 65.65 | 64.39 | 64.04 | 77.46 |

6 epochs | 48.82 | 67.29 | 67.70 | 65.83 | 64.57 | 64.24 | 77.76 | |

10 epochs | 49.06 | 67.60 | 68.03 | 66.23 | 64.99 | 64.68 | 78.68 |

**Table 5.**The accuracy measurements for test data for a neural network model trained with data representing the behavior of three controllers simultaneously, given n recent queue occupancy history elements.

Sigmoid | ||||||||
---|---|---|---|---|---|---|---|---|

n Last Items in Queue Occupancy History | ||||||||

20 | 100 | 200 | 300 | 400 | 500 | 1000 | ||

CNN by behavior 3 ${\mathit{PI}}^{\mathit{\alpha}}$ | 5 epochs | 49.52 | 67.03 | 68.55 | 68.48 | 68.40 | 68.42 | 70.98 |

6 epochs | 49.58 | 67.12 | 67.70 | 68.65 | 68.60 | 68.61 | 71.30 | |

10 epochs | 49.74 | 67.34 | 68.99 | 69.03 | 69.15 | 69.21 | 72.10 |

**Table 6.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}1$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.4$ and $H=0.5$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}1$ | 249,878 | 168.98 |

CNN History = 20 | 251,198 | 172.97 |

CNN History = 100 | 248,936 | 176.45 |

CNN History = 200 | 250,063 | 175.69 |

CNN History = 300 | 249,510 | 166.87 |

CNN History = 400 | 250,104 | 166.23 |

CNN History = 500 | 250,800 | 174.31 |

CNN History = 1000 | 249,561 | 173.33 |

**Table 7.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}1$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.4$ and $H=0.9$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}1$ | 263,387 | 139.16 |

CNN History = 20 | 261,678 | 145.66 |

CNN History = 100 | 262,304 | 145.81 |

CNN History = 200 | 262,518 | 147.22 |

CNN History = 300 | 262,935 | 139.28 |

CNN History = 400 | 263,872 | 140.89 |

CNN History = 500 | 263,440 | 142.22 |

CNN History = 1000 | 263,654 | 143.14 |

**Table 8.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Softmax, trained using data regarding $P{I}^{\alpha}1$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.4$ and $H=0.5$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}1$ | 249,878 | 168.98 |

CNN History = 100 | 250,038 | 182.21 |

CNN History = 300 | 250,455 | 164.06 |

CNN History = 500 | 250,017 | 174.73 |

**Table 9.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Softmax, trained using data regarding $P{I}^{\alpha}1$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.4$ and $H=0.9$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}1$ | 263,387 | 139.16 |

CNN History = 100 | 261,271 | 148.89 |

CNN History = 300 | 263,609 | 134.64 |

CNN History = 500 | 264,569 | 145.64 |

**Table 10.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}2$ controller parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.5$ and $H=0.5$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}2$ | 250,314 | 134.72 |

CNN History = 20 | 249,610 | 135.27 |

CNN History = 100 | 250,657 | 140.37 |

CNN History = 200 | 249,633 | 137.95 |

CNN History = 300 | 249,752 | 142.29 |

CNN History = 400 | 248,852 | 134.86 |

CNN History = 500 | 249,960 | 138.85 |

CNN History = 1000 | 249,744 | 129.60 |

**Table 11.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}2$ controller parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.5$ and $H=0.9$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}2$ | 264,819 | 109.37 |

CNN History = 20 | 263,014 | 117.57 |

CNN History = 100 | 264,135 | 112.98 |

CNN History = 200 | 264,217 | 115.69 |

CNN History = 300 | 264,668 | 116.24 |

CNN History = 400 | 265,538 | 110.05 |

CNN History = 500 | 264,533 | 112.47 |

CNN History = 1000 | 265,839 | 105.25 |

**Table 12.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}3$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.6$ and $H=0.5$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}3$ | 250,840 | 117.53 |

CNN History = 20 | 251,892 | 139.26 |

CNN History = 100 | 250,362 | 136.35 |

CNN History = 200 | 248,878 | 117.67 |

CNN History = 300 | 250,533 | 116.40 |

CNN History = 400 | 250,011 | 118.85 |

CNN History = 500 | 250,166 | 118.67 |

CNN History = 1000 | 249,801 | 118.20 |

**Table 13.**Detailed results of queue occupancy obtained for CNN model with the last layer activation function Sigmoid, trained using data regarding $P{I}^{\alpha}3$ controller and parameters: ${K}_{P}=0.0001$, ${K}_{I}=0.0004$, $\alpha =-0.6$ and $H=0.9$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

$P{I}^{\alpha}3$ | 265,707 | 95.13 |

CNN History = 20 | 265,737 | 121.19 |

CNN History = 100 | 263,952 | 110.79 |

CNN History = 200 | 265,383 | 97.28 |

CNN History = 300 | 266,295 | 94.09 |

CNN History = 400 | 266,366 | 95.79 |

CNN History = 500 | 265,592 | 97.31 |

CNN History = 1000 | 266,184 | 96.90 |

**Table 14.**Detailed results of queue occupancy results obtained for CNN model trained using data regarding three $P{I}^{\alpha}$ controllers and $H=0.5$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

CNN 3 History = 20 | 249,727 | 128.75 |

CNN 3 History = 100 | 249,988 | 174.64 |

CNN 3 History = 200 | 250,583 | 164.87 |

CNN 3 History = 300 | 249,907 | 169.98 |

CNN 3 History = 400 | 249,593 | 173.41 |

CNN 3 History = 500 | 250,157 | 138.08 |

CNN 3 History = 1000 | 249,334 | 170.46 |

**Table 15.**Detailed results of queue occupancy results obtained for CNN model trained using data regarding three $P{I}^{\alpha}$ controllers and $H=0.9$.

AQM | Packet Dropped | Average Queue Length |
---|---|---|

CNN 3 History = 20 | 262,841 | 120.94 |

CNN 3 History = 100 | 263,859 | 152.15 |

CNN 3 History = 200 | 262,298 | 137.21 |

CNN 3 History = 300 | 263,205 | 131.49 |

CNN 3 History = 400 | 263,818 | 129.81 |

CNN 3 History = 500 | 263,872 | 127.37 |

CNN 3 History = 1000 | 263,554 | 138.32 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Szyguła, J.; Domański, A.; Domańska, J.; Marek, D.; Filus, K.; Mendla, S.
Supervised Learning of Neural Networks for Active Queue Management in the Internet. *Sensors* **2021**, *21*, 4979.
https://doi.org/10.3390/s21154979

**AMA Style**

Szyguła J, Domański A, Domańska J, Marek D, Filus K, Mendla S.
Supervised Learning of Neural Networks for Active Queue Management in the Internet. *Sensors*. 2021; 21(15):4979.
https://doi.org/10.3390/s21154979

**Chicago/Turabian Style**

Szyguła, Jakub, Adam Domański, Joanna Domańska, Dariusz Marek, Katarzyna Filus, and Szymon Mendla.
2021. "Supervised Learning of Neural Networks for Active Queue Management in the Internet" *Sensors* 21, no. 15: 4979.
https://doi.org/10.3390/s21154979