# Multi-Controller Deployment in SDN-Enabled 6G Space–Air–Ground Integrated Network

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- Firstly, based on the hierarchical SDN-enabled 6G SAGIN model, the controller hierarchical network design scheme is adopted, and the delay model of the network and the load model of the controller are constructed. The loss value, as an important basis for network division, is defined by combining the delay model and the load model.
- Then, in order to obtain the global optimal solution and reduce the time complexity of the algorithm, a multi-controller deployment strategy based on simulated annealing is proposed, and a concrete deployment scheme is given according to different delay and load requirements.
- Finally, considering the dynamic nature of the network topology, a switch migration strategy is proposed, in order to achieve load balance among the controllers, and some switches within the jurisdiction of the high-load controller are selected to be transferred to the low-load controller, improving the utilization of network resources. Due to the need to implement this strategy at certain intervals to avoid load imbalance between the controllers, resulting in a sharp decline in network performance, the time complexity of the switch migration algorithm cannot be too high.

#### 1.1. In SDN-Enabled SAGIN Architecture Design

#### 1.2. In Terms of Multi-Controller Deployment Strategies

## 2. Problem Formulation and Solution

#### 2.1. Problem Formulation

#### 2.1.1. Problem Description

#### 2.1.2. Hierarchical Multi-Controller Deployment Modeling

#### Network Delay Model

#### Controller Load Model

#### Loss Function

- Condition 1: Communication link status of switch and controller: ${e}_{ij}\in \left\{0,1\right\}$.
- Condition 2: The number of controllers is greater than 2.
- Condition 3: The controller to the jurisdiction of the switch node delay is less than the maximum delay: ${L}_{ij}\left(t\right)<{L}_{max}$.
- Condition 4: Controller resource capacity greater than 0: ${Q}_{{c}_{i}}>0$.
- The solution space, i.e., the latitude and longitude of the controller is limited in China (latitude: 3.86–53.55°; longitude: 73.66–135.05°).

#### 2.2. Problem Solution

#### Multi-Controller Deployment Algorithm Based on Simulated Annealing

- First, a new solution space is randomly generated, which is the controller’s new latitude and longitude, and is converted to Cartesian coordinates according to Formula (2).
- Second, the propagation delay from the satellite node to the controller is solved according to Formula (3).
- In the third step, the satellite node is assigned to the target controller with the goal of minimum time delay, and then the load of the controller is calculated; if a controller load is greater than $\rho \in (0,1)$ times the maximum load, the part of the controller controlled by the current controller is transferred to the controller with the least load, and the relation matrix R between the satellite node and the controller is obtained.

Algorithm 1 Multi-controller deployment algorithm based on simulated annealing. |

Input: Controller(Ground station) longitude and latitude matrix C, Satellite longitude and latitude matrix S, height matrix ${S}_{H}$, Traffic of switch nodes $SF$, and the maximum controller capacity $CF$. |

Output: Number of controllers and longitude and latitude matrix ${C}_{best}$, relation matrix of satellite corresponding controller R, propagation delay L and controller load B. |

1: Initialize The relation matrix of the satellite corresponding controller is R, initial temperature t, descending speed $step$ and final temperature e. |

2: $pos\_xyz\leftarrow smiller\_to\_xyz\left(S\right)$ |

3: while $t>e$ do |

4: for $iter\phantom{\rule{4pt}{0ex}}in\phantom{\rule{4pt}{0ex}}range\left(iteration\right)$ do |

5: $C\_temp\leftarrow product\_new\_point\left(C\right)$ |

6: $controller\_xyz\leftarrow miller\_to\_xyz(S,{S}_{H})$ |

7: $L\leftarrow calculate\_delay(controller\_xyz,pos\_xyz)$ |

8: $R,CF\leftarrow relation\_satellite\_controller(L,SF)$ |

9: $loss\_temp\leftarrow calculate\_loss(R,L,SF)$ |

10: $probability\leftarrow np.random.rand\left(\right)$ |

11: $pro\_new\leftarrow calculate\_pro(loss\_temp,loss,t)$ |

12: if $loss\_temp<loss$ or |

13: $pro\_new>probability$ then |

14: $loss\leftarrow loss\_temp$ |

15: $C\leftarrow C\_temp$ |

16: end if |

17: end for |

18: $t\leftarrow stepast\phantom{\rule{4pt}{0ex}}t$ |

19: $save(R\_best,C\_best,L,B)$ |

20: end while |

#### 2.3. Switch Node Migration Strategy for Controller Load Balancing

Algorithm 2 Switch node migration strategy for controller load balancing. |

Input: The longitude and latitude matrix C of the controller(ground station) is obtained by Algorithm 1, The satellite longitude and latitude matrix S and height matrix ${S}_{H}$ of 100 time slices, $SF$ packet-IN flow of switch nodes and the maximum capacity $CF$ of the controller increasing linearly with time slices. |

Output: The time slice T, the average propagation delay L, the relation matrix R of the satellite corresponding to the controller, and $CF$ the load of the controller. |

1: $controller\_xyz\leftarrow miller\_to\_xyz\left(C\right)$ |

2: for $i\phantom{\rule{4pt}{0ex}}in\phantom{\rule{4pt}{0ex}}range(1:timeslice+1)\phantom{\rule{4pt}{0ex}}$ do |

3: $S=read\phantom{\rule{4pt}{0ex}}switch\phantom{\rule{4pt}{0ex}}information\left(i\right)$ |

4: $position\_xyz\leftarrow miller\_to\_xyz\left(S\right)$ |

5: $L\leftarrow calculate\_delay(controller\_xyz,\phantom{\rule{4pt}{0ex}}pos\_xyz)$ |

6: $Linearly\phantom{\rule{4pt}{0ex}}increases\phantom{\rule{4pt}{0ex}}switch\phantom{\rule{4pt}{0ex}}traffic$ |

7: $\mathbf{R},\phantom{\rule{4pt}{0ex}}\mathbf{CF}\phantom{\rule{4pt}{0ex}}\leftarrow \mathbf{UpdateRelationMatrix}(\mathbf{L},\mathbf{SF},\mathbf{R})$ |

8: $save(R,T,\phantom{\rule{4pt}{0ex}}CF,L)$ |

9: end for |

#### 2.4. Algorithm Complexity Analysis

## 3. Results

#### 3.1. Experimental Scene Description

#### 3.2. Result Analysis

#### 3.2.1. In the Case of Different Number of Controllers, the Load of Each Controller and the Maximum Propagation Delay from the Switch Node to the Controller Are Calculated

#### 3.2.2. The Controller Load Varies with the Packet-In Request Packet of the Switch

#### 3.2.3. Compare the Load Variances of Different Algorithms with Time

#### 3.2.4. Compared with Different Algorithms, the Running Time of the Migration Algorithm Changes with the Increase in Switch Nodes

## 4. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

SAGIN | Space–Air–Ground Integrated Network |

LEO | Geosynchronous Earth Orbit |

MEO | Medium Earth Orbit |

LEO | Low Earth Orbit |

VLEO | Very Low Earth Orbit |

HAP | High-Altitude Platform |

LAP | Low-Altitude Platform |

UAV | Unmanned Aerial Vehicle |

SDN | Software Defined Network |

NFV | Network Function Virtualization |

n-k-means | n times k-means |

OCLDS | Optimizing Controller Load Dynamic Strategy |

SAA | Simulated Annealing Algorithm |

STK | Satellite Tool Kit |

## References

- Latva-aho, M.; Leppänen, K.; Clazzer, F.; Munari, A. Key Drivers and Research Challenges for 6G Ubiquitous Wireless Intelligence; Electronic Library: Koln, Germany, 2020. [Google Scholar]
- Zhang, Z.; Ning, H.; Shi, F.; Farha, F.; Xu, Y.; Xu, J.; Zhang, F.; Choo, K.K.R. Artificial intelligence in cyber security: Research advances, challenges, and opportunities. Artif. Intell. Rev.
**2021**, 55, 1029–1053. [Google Scholar] [CrossRef] - Liu, J.; Shi, Y.; Fadlullah, Z.M.; Kato, N. Space-Air-Ground Integrated Network: A Survey. IEEE Commun. Surv. Tutor.
**2018**, 20, 2714–2741. [Google Scholar] [CrossRef] - Ray, P.P. A review on 6G for space-air-ground integrated network: Key enablers, open challenges, and future direction. J. King Saud-Univ.-Comput. Inf. Sci. 2021; in press. [Google Scholar] [CrossRef]
- Liu, L.; Chen, C.; Pei, Q.; Maharjan, S.; Zhang, Y. Vehicular edge computing and networking: A survey. Mob. Netw. Appl.
**2021**, 26, 1145–1168. [Google Scholar] [CrossRef] - Cong, W.; Chen, C.; Qingqi, P.; Zhiyuan, J.; Shugong, X. An Information Centric In-network Caching Scheme for 5G-Enabled Internet of Connected Vehicles. IEEE Trans. Mob. Comput. 2021; in press. [Google Scholar] [CrossRef]
- Wan, S.; Ding, S.; Chen, C. Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles. Pattern Recognit.
**2022**, 121, 108146. [Google Scholar] [CrossRef] - Ma, J.; Li, T.; Cui, J.; Ying, Z.; Cheng, J. Attribute-based secure announcement sharing among vehicles using blockchain. IEEE Internet Things J.
**2021**, 8, 10873–10883. [Google Scholar] [CrossRef] - Kato, N.; Fadlullah, Z.M.; Tang, F.; Mao, B.; Tani, S.; Okamura, A.; Liu, J. Optimizing space-air-ground integrated networks by artificial intelligence. IEEE Wirel. Commun.
**2019**, 26, 140–147. [Google Scholar] [CrossRef] [Green Version] - Wei, D.; Ning, H.; Shi, F.; Wan, Y.; Xu, J.; Yang, S.; Zhu, L. Dataflow management in the internet of things: Sensing, control, and security. Tsinghua Sci. Technol.
**2021**, 26, 918–930. [Google Scholar] [CrossRef] - Cheng, N.; Quan, W.; Shi, W.; Wu, H.; Ye, Q.; Zhou, H.; Zhuang, W.; Shen, X.; Bai, B. A comprehensive simulation platform for space-air-ground integrated network. IEEE Wirel. Commun.
**2020**, 27, 178–185. [Google Scholar] [CrossRef] - Gao, H.; Liu, C.; Yin, Y.; Xu, Y.; Li, Y. A hybrid approach to trust node assessment and management for vanets cooperative data communication: Historical interaction perspective. IEEE Trans. Intell. Transp. Syst. 2021; in press. [Google Scholar] [CrossRef]
- Fu, S.; Ma, L.; Atiquzzaman, M.; Lee, Y.J. Architecture and performance of SIGMA: A seamless mobility architecture for data networks. In Proceedings of the IEEE International Conference on Communications, ICC 2005, Seoul, Korea, 16–20 May 2005; Volume 5, pp. 3249–3253. [Google Scholar]
- Zhao, Y.; Zhai, W.; Zhao, J.; Zhang, T.; Sun, S.; Niyato, D.; Lam, K.Y. A comprehensive survey of 6g wireless communications. arXiv
**2020**, arXiv:2101.03889. [Google Scholar] - Shi, Y.; Cao, Y.; Liu, J.; Kato, N. A cross-domain SDN architecture for multi-layered space-terrestrial integrated networks. IEEE Netw.
**2019**, 33, 29–35. [Google Scholar] [CrossRef] - Ye, J.; Dang, S.; Shihada, B.; Alouini, M.S. Space-air-ground integrated networks: Outage performance analysis. IEEE Trans. Wirel. Commun.
**2020**, 19, 7897–7912. [Google Scholar] [CrossRef] - Mozaffari, M.; Lin, X.; Hayes, S. Towards 6G with Connected Sky: UAVs and Beyond. arXiv
**2021**, arXiv:2103.01143. [Google Scholar] [CrossRef] - Qiu, T.; Zhang, S.; Si, W.; Cao, Q.; Atiquzzaman, M. A 3-D Topology Evolution Scheme with Self-Adaption for Industrial Internet of Things. IEEE Internet Things J.
**2020**, 8, 9473–9483. [Google Scholar] [CrossRef] - Guo, Y.; Li, Q.; Li, Y.; Zhang, N.; Wang, S. Service Coordination in the Space-Air-Ground Integrated Network. IEEE Netw.
**2021**, 35, 168–173. [Google Scholar] [CrossRef] - Qiu, T.; Wang, X.; Chen, C.; Atiquzzaman, M.; Liu, L. TMED: A spider-Web-like transmission mechanism for emergency data in vehicular ad hoc networks. IEEE Trans. Veh. Technol.
**2018**, 67, 8682–8694. [Google Scholar] [CrossRef] - Zhang, N.; Zhang, S.; Yang, P.; Alhussein, O.; Zhuang, W.; Shen, X.S. Software defined space-air-ground integrated vehicular networks: Challenges and solutions. IEEE Commun. Mag.
**2017**, 55, 101–109. [Google Scholar] [CrossRef] [Green Version] - Shirmarz, A.; Ghaffari, A. Taxonomy of controller placement problem (CPP) optimization in Software Defined Network (SDN): A survey. J. Ambient. Intell. Humaniz. Comput.
**2021**, 12, 10473–10498. [Google Scholar] [CrossRef] - Tian, X.; Shen, H.; Chen, X.; Li, C. Analysis of SDN Multi-controller Placement. In 2020 International Conference on Data Processing Techniques and Applications for Cyber-Physical Systems; Springer: Singapore, 2021; pp. 677–684. [Google Scholar]
- Chen, C.; Jiange, J.; Rufei, F.; Lanlan, C.; Cong, L.; Shaohua, W. An Intelligent Caching Strategy Considering Time-space Characteristics in Vehicular NamedData Networks. IEEE Trans. Intell. Transp. Syst. 2021; in press. [Google Scholar] [CrossRef]
- Wu, H.; Chen, J.; Zhou, C.; Shi, W.; Cheng, N.; Xu, W.; Zhuang, W.; Shen, X.S. Resource management in space-air-ground integrated vehicular networks: SDN control and AI algorithm design. IEEE Wirel. Commun.
**2020**, 27, 52–60. [Google Scholar] [CrossRef] - Wei, D.; Wei, N.; Yang, L.; Kong, Z. SDN-based multi-controller optimization deployment strategy for satellite network. In Proceedings of the 2020 IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS), Shenyang, China, 28–30 July 2020; pp. 467–473. [Google Scholar]
- Kumari, A.; Sairam, A.S. A survey of controller placement problem in software defined networks. arXiv
**2019**, arXiv:1905.04649. [Google Scholar] - Yan, B.; Liu, Q.; Shen, J.; Liang, D.; Zhao, B.; Ouyang, L. A survey of low-latency transmission strategies in software defined networking. Comput. Sci. Rev.
**2021**, 40, 100386. [Google Scholar] [CrossRef] - Semong, T.; Maupong, T.; Anokye, S.; Kehulakae, K.; Dimakatso, S.; Boipelo, G.; Sarefo, S. Intelligent load balancing techniques in software defined networks: A survey. Electronics
**2020**, 9, 1091. [Google Scholar] [CrossRef] - Mostafaei, H.; Menth, M.; Obaidat, M.S. A learning automaton-based controller placement algorithm for software-defined networks. In Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates, 9–13 December 2018; pp. 1–6. [Google Scholar]
- Qu, H.; Xu, X.; Zhao, J.; Yue, P. An SDN-Based Space-Air-Ground Integrated Network Architecture and Controller Deployment Strategy. In Proceedings of the 2020 IEEE 3rd International Conference on Computer and Communication Engineering Technology (CCET), Beijing, China, 14–16 August 2020; pp. 138–142. [Google Scholar]
- Qi, Y.; Wang, D.; Yao, W.; Li, H.; Cao, Y. Towards multi-controller placement for SDN based on density peaks clustering. In Proceedings of the ICC 2019-2019 IEEE International Conference on Communications (ICC), Shanghai, China, 20–24 May 2019; pp. 1–6. [Google Scholar]
- Tao, P.; Ying, C.; Sun, Z.; Tan, S.; Wang, P.; Sun, Z. The controller placement of software-defined networks based on minimum delay and load balancing. In Proceedings of the 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), Athens, Greece, 12–15 August 2018; pp. 310–313.
- Cheng, J.; Yuan, G.; Zhou, M.; Gao, S.; Liu, C.; Jiang, C. A Fluid Mechanics-Based Model to Estimate VINET Capacity in an Urban Scene. IEEE Trans. Intell. Transp. Syst. 2021; in press. [Google Scholar] [CrossRef]
- Chen, C.; Liu, L.; Wan, S.; Hui, X.; Pei, Q. Data Dissemination for Industry 4.0 Applications in Internet of Vehicles Based on Short-term Traffic Prediction. ACM Trans. Internet Technol. (TOIT)
**2021**, 22, 1–18. [Google Scholar] [CrossRef]

Parameters | Value | Parameters | Value |
---|---|---|---|

Orbital altitude | 780 km | Satellite planes | 6 |

Orbital type | Polar orbit | Satellites per plane | 11 |

Inter-satellite link bandwidth | 10 Mbps | inclination | 86.4° |

Angle difference between adjacent planes | 31.6° | Phase factor of the adjacent plane | 16.36° |

Time Slice | Switch No. | Src Controller No. | Des Controller No. |
---|---|---|---|

5 | S1 | C2 | C7 |

S5 | C3 | C1 | |

6 | S8 | C2 | C6 |

7 | S14 | C2 | C7 |

8 | S15 | C2 | C1 |

9 | S16 | C2 | C6 |

10 | S19 | C2 | C1 |

11 | S7 | C2 | C1 |

12 | S11 | C2 | C5 |

17 | S4 | C2 | C6 |

19 | S1 | C7 | C1 |

20 | S26 | C2 | C5 |

S12 | C7 | C6 |

Constellation Name | Altitude (km) | Orbital Planes | Satellites per Plane | Total Number of Satellites | |
---|---|---|---|---|---|

Walker 1 | 580 | 4 | 43 | 172 | |

Walker 2 | 550 | 11 | 22 | 242 | |

Walker 3 | 580 | 4 | 43 | 172 | 414 |

550 | 11 | 22 | 242 | ||

Hybrid orbit constellation | 570 | 36 | 20 | 720 | 786 |

780 | 6 | 11 | 66 | ||

Walker 4 | 550 | 72 | 22 | 1584 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Liao, Z.; Chen, C.; Ju, Y.; He, C.; Jiang, J.; Pei, Q.
Multi-Controller Deployment in SDN-Enabled 6G Space–Air–Ground Integrated Network. *Remote Sens.* **2022**, *14*, 1076.
https://doi.org/10.3390/rs14051076

**AMA Style**

Liao Z, Chen C, Ju Y, He C, Jiang J, Pei Q.
Multi-Controller Deployment in SDN-Enabled 6G Space–Air–Ground Integrated Network. *Remote Sensing*. 2022; 14(5):1076.
https://doi.org/10.3390/rs14051076

**Chicago/Turabian Style**

Liao, Zhan, Chen Chen, Ying Ju, Ci He, Jiange Jiang, and Qingqi Pei.
2022. "Multi-Controller Deployment in SDN-Enabled 6G Space–Air–Ground Integrated Network" *Remote Sensing* 14, no. 5: 1076.
https://doi.org/10.3390/rs14051076