Deep Reinforcement Learning for Network Dismantling: A K-Core Based Approach
Abstract
:1. Introduction
- We introduce the evaluation metric of accumulated 2-core size during the network dismantling process. We transform the problem into a learning problem and propose an end-to-end SmartCore model. This model allows training an agent across simple synthetic graphs, which can then be directly applied to real-world datasets.
- We conduct comprehensive experiments to validate our SmartCore model. Our results show that the SmartCore model outperforms state-of-the-art baseline methods in terms of both accuracy and speed.
2. Related Works
3. Preliminaries
3.1. K-Core
3.2. Accumulated 2-Core Size
3.3. Graph Neural Network
3.4. Reinforcement Learning
4. SmartCore Model
4.1. Architecture of SmartCore
4.2. Training Algorithms
Algorithm 1: Training Procedure of SmartCore |
1: Initialize experience replay memory with size 2: Initialize state–action pair value function with random weights 3: Initialize target function with weights 4: For episode = 1 to Do 5: Generate a graph from the BA model 6: Obtain the 2-core from 7: Initialize the state to an empty sequence 8: For = 1 to Do 9: With probability , select a random action on 10: Otherwise select 11: Execute action (remove node from ) and observe reward 12: Add to partial solution 13: If Then 14: Store transition in 15: Sample random mini–batch of transitions from 16: 17: Perform a stochastic gradient descent over (3) with respect to the parameters 18: Every steps reset 19: End If 20: Update the 2-core size from 21: End For 22: End For |
4.3. Computational Complexity Analysis
5. Experiments
5.1. Baselines
5.2. Datasets
5.3. Training Details
5.4. Results
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Albert, R.; Barabási, A.L. Statistical mechanics of complex networks. Rev. Mod. Phys. 2002, 74, 47. [Google Scholar] [CrossRef]
- Yang, Y.; Nishikawa, T.; Motter, A.E. Small vulnerable sets determine large network cascades in power grids. Science 2017, 358, eaan3184. [Google Scholar] [CrossRef]
- Fan, C.; Xiao, K.; Xiu, B.; Lv, G. A fuzzy clustering algorithm to detect criminals without prior information. In Proceedings of the 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014), IEEE, Beijing, China, 17–20 August 2014. [Google Scholar]
- Pastor-Satorras, R.; Vespignani, A. Epidemic spreading in scale-free networks. Phys. Rev. Lett. 2001, 86, 3200. [Google Scholar] [CrossRef] [PubMed]
- Frank, H.; Frisch, I. Analysis and design of survivable networks. IEEE Trans. Commun. Technol. 1970, 18, 501–519. [Google Scholar] [CrossRef]
- Bauer, D.; Boesch, F.; Suffel, C.; Tindell, R. Connectivity extremal problems and the design of reliable probabilistic networks. Theory Appl. Graphs 1981, 89–98. [Google Scholar]
- Harary, F. Conditional connectivity. Networks 1983, 13, 347–357. [Google Scholar] [CrossRef]
- Wu, J.; Barahona, M.; Tan, Y.J.; Deng, H.Z. Spectral measure of structural robustness in complex networks. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2011, 41, 1244–1252. [Google Scholar] [CrossRef]
- Dorogovtsev, S.N.; Goltsev, A.V.; Mendes, J.F. K-core organization of complex networks. Phys. Rev. Lett. 2006, 96, 040601. [Google Scholar] [CrossRef]
- Cohen, R.; Havlin, S. Complex Networks: Structure, Robustness and Function; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
- Zdeborová, L.; Zhang, P.; Zhou, H.J. Fast and simple decycling and dismantling of networks. Sci. Rep. 2016, 6, 37954. [Google Scholar] [CrossRef]
- Fomin, F.V.; Gaspers, S.; Pyatkin, A.V.; Razgon, I. On the Minimum Feedback Vertex Set Problem: Exact and Enumeration Algorithms. Algorithmica 2008, 52, 293–307. [Google Scholar] [CrossRef]
- Mugisha, S.; Zhou, H. Identifying optimal targets of network attack by belief propagation. Phys. Rev. E 2016, 94, 012305. [Google Scholar] [CrossRef]
- Addis, B.; Di Summa, M.; Grosso, A. Identifying critical nodes in undirected graphs: Complexity results and polynomial algorithms for the case of bounded treewidth. Discret. Appl. Math. 2013, 161, 2349–2360. [Google Scholar] [CrossRef]
- Arulselvan, A.; Commander, C.W.; Elefteriadou, L.; Pardalos, P.M. Detecting critical nodes in sparse graphs. Comput. Oper. Res. 2009, 36, 2193–2200. [Google Scholar] [CrossRef]
- Li, H.; Shang, Q.; Deng, Y. A generalized gravity model for influential spreaders identification in complex networks. Chaos Solitons Fractals 2021, 143, 110456. [Google Scholar] [CrossRef]
- Walteros, J.L.; Veremyev, A.; Pardalos, P.M.; Pasiliao, E.L. Detecting critical node structures on graphs: A mathematical programming approach. Networks 2019, 73, 48–88. [Google Scholar] [CrossRef]
- Deng, Y.; Wu, J.; Tan, Y.J. Optimal attack strategy of complex networks based on tabu search. Phys. A Stat. Mech. Its Appl. 2016, 442, 74–81. [Google Scholar] [CrossRef]
- Lozano, M.; Garcia-Martinez, C.; Rodriguez, F.J.; Trujillo, H.M. Optimizing network attacks by artificial bee colony. Inf. Sci. 2017, 377, 30–50. [Google Scholar] [CrossRef]
- Fan, C.; Zeng, L.; Sun, Y.; Liu, Y.Y. Finding key players in complex networks through deep reinforcement learning. Nat. Mach. Intell. 2020, 2, 317–324. [Google Scholar] [CrossRef]
- Zeng, L.; Fan, C.; Chen, C. Leveraging Minimum Nodes for Optimum Key Player Identification in Complex Networks: A Deep Reinforcement Learning Strategy with Structured Reward Shaping. Mathematics 2023, 11, 3690. [Google Scholar] [CrossRef]
- Fan, C.; Zeng, L.; Feng, Y.; Cheng, G.; Huang, J.; Liu, Z. A novel learning-based approach for efficient dismantling of networks. Int. J. Mach. Learn. Cybern. 2020, 11, 2101–2111. [Google Scholar] [CrossRef]
- Zhou, H.-J. Cycle-tree guided attack of random K-core: Spin glass model and efficient message-passing algorithm. Sci. China Phys. Mech. Astron. 2022, 65, 230511. [Google Scholar] [CrossRef]
- Zhou, J.; Zhou, H.J. Hierarchical Cycle-Tree Packing Model for Optimal K-Core Attack. J. Stat. Phys. 2023, 190, 200. [Google Scholar] [CrossRef]
- Schneider, C.M.; Moreira, A.A.; Andrade, J.S., Jr.; Havlin, S.; Herrmann, H.J. Mitigation of malicious attacks on networks. Proc. Natl. Acad. Sci. USA 2011, 108, 3838–3841. [Google Scholar] [CrossRef] [PubMed]
- Zhou, J.; Cui, G.; Hu, S.; Zhang, Z.; Yang, C.; Liu, Z.; Wang, L.; Li, C.; Sun, M. Graph neural networks: A review of methods and applications. AI Open 2020, 1, 57–81. [Google Scholar] [CrossRef]
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-level control through deep reinforcement learning. Nature 2015, 518, 529–533. [Google Scholar] [CrossRef] [PubMed]
- Hamilton, W.; Ying, Z.; Leskovec, J. Inductive Representation Learning on Large Graphs. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017. [Google Scholar]
- Hasselt, H. Double Q-learning. In Advances in Neural Information Processing Systems; Neural Information Processing Systems Foundation, Inc. (NeurIPS): La Jolla, CA, USA, 2010; Volume 23. [Google Scholar]
- Hooshmand, F.; Mirarabrazi, F.; MirHassani, S.A. Efficient Benders Decomposition for Distance–Based Critical Node Detection Problem. Omega 2020, 93, 102037. [Google Scholar] [CrossRef]
- Carmi, S.; Havlin, S.; Kirkpatrick, S.; Shavitt, Y.; Shir, E. A Model of Internet Topology Using K –Shell Decomposition. Proc. Natl. Acad. Sci. USA 2007, 104, 11150–11154. [Google Scholar] [CrossRef] [PubMed]
- Bavelas, A. Communication Patterns in Task-Oriented Groups. J. Acoust. Soc. Am. 1950, 22, 725–730. [Google Scholar] [CrossRef]
- Wandelt, S.; Sun, X.; Feng, D.; Zanin, M.; Havlin, S. A Comparative Analysis of Approaches to Network–Dismantling. Sci. Rep. 2018, 8, 13513. [Google Scholar] [CrossRef]
- Morone, F.; Makse, H.A. Influence maximization in complex networks through optimal percolation. Nature 2015, 524, 65–68. [Google Scholar] [CrossRef]
- Braunstein, A.; Dall’Asta, L.; Semerjian, G.; Zdeborová, L. Network Dismantling. Proc. Natl. Acad. Sci. USA 2016, 113, 12368–12373. [Google Scholar] [CrossRef] [PubMed]
- Ren, X.; Gleinig, N.; Helbing, D.; Antulov–Fantulin, N. Generalized network dismantling. Proc. Natl. Acad. Sci. USA 2018, 116, 6554–6559. [Google Scholar] [CrossRef] [PubMed]
- Leskovec, J.; Krevl, A. SNAP Datasets: Stanford Large Network Dataset Collection; SNAP: Santa Monica, CA, USA, 2014. [Google Scholar]
Data/Index | Node Number | Edge Number | Diameter |
---|---|---|---|
HI–II–14 | 4165 | 13,087 | 11 |
Digg | 29,652 | 84,781 | 12 |
Enron | 33,696 | 180,811 | 11 |
Epinion | 75,879 | 508,837 | 14 |
Name | Value | Description |
---|---|---|
Learning rate | 1 × 10−4 | The learning rate used by Adam optimizer |
Embedding dimension | 64 | Dimensions of node embedding vector |
Layer iterations | 5 | Number of GraphSAGE layers |
Q-learning steps | 3 | Number of Q-learning steps |
Batch size | 64 | Number of mini-batch training samples |
Method/Data Size | 30–50 | 50–100 | 100–200 | 200–300 | 300–400 | 400–500 |
---|---|---|---|---|---|---|
HDA | 59.80 ± 3.36 | 59.36 ± 3.00 | 57.62 ± 1.99 | 56.97 ± 1.49 | 56.57 ± 1.36 | 56.50 ± 1.10 |
HBA | 79.67 ± 9.69 | 84.37 ± 7.71 | 88.97 ± 5.55 | 91.18 ± 4.30 | 93.08 ± 3.74 | 92.97 ± 3.45 |
HCA | 62.12 ± 4.26 | 61.34 ± 2.93 | 59.65 ± 2.01 | 58.99 ± 1.66 | 58.76 ± 1.42 | 58.53 ± 1.36 |
HPRA | 60.12 ± 3.24 | 59.35 ± 3.06 | 58.04 ± 1.96 | 57.22 ± 1.53 | 56.85 ± 1.33 | 56.89 ± 1.19 |
SmartCore | 65.21 ± 3.88 | 62.26 ± 3.06 | 59.45 ± 1.82 | 57.81 ± 1.46 | 57.30 ± 1.30 | 57.31 ± 1.16 |
Method/Dataset | HI–II–14 | Digg | Enron | Epinions |
---|---|---|---|---|
HDA | 0.0482 | 0.3017 | 0.3250 | 0.0507 |
CI | 0.0581 | 0.2547 | 0.3263 | 0.0840 |
MinSum | 0.0615 | 0.2207 | 0.3344 | 0.1028 |
CoreHD | 0.0454 | 0.2241 | 0.3278 | 0.0464 |
BPD | 0.0541 | 0.2207 | 0.3259 | 0.0641 |
GND | 0.0730 | 0.2392 | 0.3361 | 0.0905 |
SmartCore | 0.0459 | 0.0660 | 0.0924 | 0.0504 |
Method/Dataset | HI–II–14 | Digg | Enron | Epinions |
---|---|---|---|---|
HDA | 0.78 | 117.23 | 139.30 | 311.70 |
CI | 1.96 | 113.96 | 135.42 | 835.78 |
MinSum | 2.03 | 113.32 | 134.82 | 876.25 |
CoreHD | 2.03 | 112.73 | 136.72 | 893.14 |
BPD | 2.02 | 114.24 | 136.08 | 895.85 |
GND | 2.02 | 115.22 | 136.67 | 864.12 |
SmartCore | 1.15 | 60.34 | 73.30 | 335.88 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Pu, T.; Zeng, L.; Chen, C. Deep Reinforcement Learning for Network Dismantling: A K-Core Based Approach. Mathematics 2024, 12, 1215. https://doi.org/10.3390/math12081215
Pu T, Zeng L, Chen C. Deep Reinforcement Learning for Network Dismantling: A K-Core Based Approach. Mathematics. 2024; 12(8):1215. https://doi.org/10.3390/math12081215
Chicago/Turabian StylePu, Tianle, Li Zeng, and Chao Chen. 2024. "Deep Reinforcement Learning for Network Dismantling: A K-Core Based Approach" Mathematics 12, no. 8: 1215. https://doi.org/10.3390/math12081215
APA StylePu, T., Zeng, L., & Chen, C. (2024). Deep Reinforcement Learning for Network Dismantling: A K-Core Based Approach. Mathematics, 12(8), 1215. https://doi.org/10.3390/math12081215