Geometry-V-Sub: An Efficient Graph Attention Network Struct Based Model for Node Classification
Abstract
:1. Introduction
2. Materials and Methods
2.1. Symbols
2.2. Reduce Parameters by Modifying GAT
- After the feature transformation, the activation function is added to ensure that the parameters of the feature transformation are relatively independent compared with the later processing steps, and more node features can be learned.
- The attention vector is widened by two or more times to enhance the expression of opposite edge information.
- By scaling the edge vectors spliced by nodes, the attention vectors on the edges can learn more hidden features of edges.
2.3. Geometric Vector Subtraction (Geometric-V-Sub) for Edge Feature Enhancement
3. Results
3.1. Dataset and Baseline
3.2. Experimental Parameters
3.2.1. Transductive Learning
3.2.2. Inductive Learning
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
- The access time of the above data sets is 10 December 2021.
- Datasets can also be downloaded using dataset classes in the deep graph library.
Conflicts of Interest
Sample Availability
References
- Getoor, L. Link-based classification. In Advanced Methods for Knowledge Discovery from Complex Data; Springer: London, UK, 2005; pp. 189–207. [Google Scholar]
- Sen, P.; Namata, G.; Bilgic, M.; Getoor, L.; Galligher, B.; Eliassi-Rad, T. Collective classification in network data. AI Mag. 2008, 29, 93. [Google Scholar] [CrossRef] [Green Version]
- Bhattacharya, I.; Getoor, L. Collective entity resolution in relational data. ACM Trans. Knowl. Discov. Data (TKDD) 2007, 1, 5-es. [Google Scholar] [CrossRef] [Green Version]
- Namata, G.; London, B.; Getoor, L.; Huang, B.; Edu, U.M.D. Query-driven active surveying for collective classification. In Proceedings of the 10th International Workshop on Mining and Learning with Graphs, Edinburgh, Scotland, UK, 26 June–1 July 2012; Volume 8, p. 1. [Google Scholar]
- Hamilton, W.; Ying, Z.; Leskovec, J. Inductive representation learning on large graphs. In Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017; Volume 30. [Google Scholar]
- Fakhraei, S.; Foulds, J.; Shashanka, M.; Getoor, L. Collective spammer detection in evolving multi-relational social networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, 10–13 August 2015; pp. 1769–1778. [Google Scholar]
- Bruna, J.; Zaremba, W.; Szlam, A.; LeCun, Y. Spectral networks and locally connected networks on graphs. arXiv 2013, arXiv:1312.6203. [Google Scholar]
- Defferrard, M.; Bresson, X.; Vandergheynst, P. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016; pp. 3844–3852. [Google Scholar]
- Kipf, T.N.; Welling, M. Semi-supervised classification with graph convolutional networks. arXiv 2016, arXiv:1609.02907. [Google Scholar]
- Gilmer, J.; Schoenholz, S.S.; Riley, P.F.; Vinyals, O.; Dahl, G.E. Neural message passing for quantum chemistry. In Proceedings of the International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; pp. 1263–1272. [Google Scholar]
- Atwood, J.; Towsley, D. Diffusion-convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems 29 (NIPS 2016), Barcelona, Spain, 5–10 December 2016; Volume 29. [Google Scholar]
- Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Lio, P.; Bengio, Y. Graph attention networks. arXiv 2017, arXiv:1710.10903. [Google Scholar]
- Zhang, J.; Shi, X.; Xie, J.; Ma, H.; King, I.; Yeung, D.Y. Gaan: Gated attention networks for learning on large and spatiotemporal graphs. arXiv 2018, arXiv:1803.07294. [Google Scholar]
- Tong, Z.; Liang, Y.; Sun, C.; Rosenblum, D.S.; Lim, A. Directed graph convolutional network. arXiv 2020, arXiv:2004.13970. [Google Scholar]
- Li, R.; Wang, S.; Zhu, F.; Huang, J. Adaptive graph convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LO, USA, 2–7 February 2018; Volume 32. [Google Scholar]
- Zhuang, C.; Ma, Q. Dual graph convolutional networks for graph-based semi-supervised classification. In Proceedings of the 2018 World Wide Web Conference, Lyon, France, 23–27 April 2018; pp. 499–508. [Google Scholar]
- Micheli, A. Neural Network for Graphs: A Contextual Constructive Approach. IEEE Trans. Neural Netw. 2009, 20, 498–511. [Google Scholar] [CrossRef] [PubMed]
- Bacciu, D.; Errica, F.; Micheli, A. Contextual graph markov model: A deep and generative approach to graph processing. In Proceedings of the International Conference on Machine Learning, Stockholm, Sweden, 10–15 July 2018; pp. 294–303. [Google Scholar]
- Klicpera, J.; Weißenberger, S.; Günnemann, S. Diffusion improves graph learning. arXiv 2019, arXiv:1911.05485. [Google Scholar]
- Wang, M.; Zheng, D.; Ye, Z.; Gan, Q.; Li, M.; Song, X.; Zhou, J.; Ma, C.; Yu, L.; Gai, Y. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv 2019, arXiv:1909.01315. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017; Volume 30. [Google Scholar]
- Bangyal, W.H.; Hameed, A.; Alosaimi, W.; Alyami, H. A new initialization approach in particle swarm optimization for global optimization problems. Comput. Intell. Neurosci. 2021, 2021, 6628889. [Google Scholar] [CrossRef] [PubMed]
- Pervaiz, S.; Ul-Qayyum, Z.; Bangyal, W.H.; Gao, L.; Ahmad, J. A systematic literature review on particle swarm optimization techniques for medical diseases detection. Comput. Math. Methods Med. 2021, 2021, 5990999. [Google Scholar] [CrossRef] [PubMed]
- Haider Bangyal, W.; Hameed, A.; Ahmad, J.; Nisar, K.; Haque, M.R.; Ibrahim, A.; Asri, A.; Rodrigues, J.J.; Khan, M.A.; B Rawat, D.; et al. New modified controlled bat algorithm for numerical optimization problem. Comput. Mater. Contin. 2022, 70, 2241–2259. [Google Scholar] [CrossRef]
- Coelho, Y.; Nguyen, B.; Santos, F.; Krishnan, S.; Bastos-Filho, T. A lightweight model for human activity recognition based on two-level classifier and compact CNN model. In Proceedings of the XXVII Brazilian Congress on Biomedical Engineering: Proceedings of CBEB 2020, Vitória, Brazil, 26–30 October 2020; Springer: Cham, Switzerland, 2022; pp. 1895–1901. [Google Scholar]
- Tao, Z.; Wu, C.; Liang, Y.; He, L. LW-GCN: A Lightweight FPGA-based Graph Convolutional Network Accelerator. arXiv 2021, arXiv:2111.03184. [Google Scholar]
- Sahbi, H. Lightweight Connectivity In Graph Convolutional Networks For Skeleton-Based Recognition. In Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA, 19–22 September 2021; pp. 2329–2333. [Google Scholar]
- Li, X.; Lv, Z.; Liu, B.; Wu, L.; Wang, Z. Improved feature learning: A maximum-average-out deep neural network for the game go. Math. Probl. Eng. 2020, 2020, 1397948. [Google Scholar] [CrossRef] [Green Version]
- Monti, F.; Boscaini, D.; Masci, J.; Rodola, E.; Svoboda, J.; Bronstein, M.M. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 22–25 July 2017; pp. 5115–5124. [Google Scholar]
Dataset | Features | Nodes | Edges | Classes | Graphs |
---|---|---|---|---|---|
Cora | 1433 | 2708 | 10,556 | 7 | 1 |
Citeseer | 3703 | 3327 | 9228 | 6 | 1 |
Pubmed | 500 | 19,717 | 88,651 | 3 | 1 |
PPI | 50 | 2372 | 818,716 | 121 | 24 |
Hidden Layer Dimension | Cora (%) | Citeseer (%) | Pubmed (%) | |
---|---|---|---|---|
GAT | 8 | 82 | 74 | 81.2 |
16 | 81.4 | 74.2 | 81.2 | |
32 | 82.4 | 72.6 | 80.8 | |
64 | 82.4 | 74 | 81.2 | |
GAT-A-FX | 8 | 82 | 75 | 80.8 |
16 | 81.8 | 72.8 | 80.8 | |
32 | 82.4 | 72.6 | 80.8 | |
64 | 82.2 | 72.8 | 80.8 | |
Geometric-V-Sub | 8 | 75 | 65 | 80.8 |
16 | 78.8 | 67.4 | 81.8 | |
32 | 80.4 | 67 | 81.2 | |
64 | 79.6 | 68 | 81.8 |
Hidden Layer Dimension | Cora | Citeseer | Pubmed | |
---|---|---|---|---|
GAT | 8 | 92,302 | 237,516 | 32,326 |
16 | 184,590 | 475,020 | 64,646 | |
32 | 369,692 | 950,552 | 129,804 | |
64 | 738,318 | 1,900,044 | 258,566 | |
GAT-A-FX | 8 | 92,444 | 237,656 | 32,460 |
16 | 184,860 | 475,288 | 64,908 | |
32 | 369,692 | 950,552 | 129,804 | |
64 | 739,356 | 1,901,080 | 259,596 | |
Geometric-V-Sub | 8 | 12,245 | 30,300 | 4421 |
16 | 24,445 | 60,628 | 9029 | |
32 | 49,613 | 122,052 | 19,013 | |
64 | 103,021 | 247,972 | 42,053 |
Hidden Layer Dimension | F1-Score | Total Parameters | |
---|---|---|---|
GAT | 32 | 91.8 | 110,578 |
64 | 97.2 | 351,986 | |
128 | 98.4 | 1,228,018 | |
256 | 98.6 | 4,552,946 | |
GAT-A-FX | 32 | 85.3 | 111,844 |
64 | 93.7 | 354,276 | |
128 | 95.9 | 1,232,356 | |
256 | 96.5 | 4,561,380 | |
Geometric-V-Sub | 32 | 83.9 | 97,405 |
64 | 91.9 | 130,877 | |
128 | 96.7 | 228,541 | |
256 | 98.4 | 546,749 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lyu, Z.; Aziguli, W.; Zhang, D. Geometry-V-Sub: An Efficient Graph Attention Network Struct Based Model for Node Classification. Appl. Sci. 2022, 12, 7246. https://doi.org/10.3390/app12147246
Lyu Z, Aziguli W, Zhang D. Geometry-V-Sub: An Efficient Graph Attention Network Struct Based Model for Node Classification. Applied Sciences. 2022; 12(14):7246. https://doi.org/10.3390/app12147246
Chicago/Turabian StyleLyu, Zhengyu, Wulamu Aziguli, and Dezheng Zhang. 2022. "Geometry-V-Sub: An Efficient Graph Attention Network Struct Based Model for Node Classification" Applied Sciences 12, no. 14: 7246. https://doi.org/10.3390/app12147246
APA StyleLyu, Z., Aziguli, W., & Zhang, D. (2022). Geometry-V-Sub: An Efficient Graph Attention Network Struct Based Model for Node Classification. Applied Sciences, 12(14), 7246. https://doi.org/10.3390/app12147246