Recent Research Progress of Graph Neural Networks in Computer Vision

Abstract

Graph neural networks (GNNs) have demonstrated significant potential in the field of computer vision in recent years, particularly in handling non-Euclidean data and capturing complex spatial and semantic relationships. This paper provides a comprehensive review of the latest research on GNNs in computer vision, with a focus on their applications in image processing, video analysis, and multimodal data fusion. First, we briefly introduce common GNN models, such as graph convolutional networks (GCN) and graph attention networks (GAT), and analyze their advantages in image and video data processing. Subsequently, this paper delves into the applications of GNNs in tasks such as object detection, image segmentation, and video action recognition, particularly in capturing inter-region dependencies and spatiotemporal dynamics. Finally, the paper discusses the applications of GNNs in multimodal data fusion tasks such as image–text matching and cross-modal retrieval, and highlights the main challenges faced by GNNs in computer vision, including computational complexity, dynamic graph modeling, heterogeneous graph processing, and interpretability issues. This paper provides a comprehensive understanding of the applications of GNNs in computer vision for both academia and industry and envisions future research directions.

1. Introduction

In recent years, deep learning has made significant progress in the field of computer vision (CV), particularly with convolutional neural networks (CNNs), which have become the core technology of modern visual systems, excelling in tasks such as image classification, object detection, and semantic segmentation [1]. CNNs, by simulating the function of biological visual neurons, effectively capture the spatial structural information of images. Through convolutional layers, they extract local features, reduce computational complexity with pooling layers, and make image-level decisions through fully connected layers, giving CNNs a significant advantage when processing two-dimensional image data.

However, despite CNNs’ excellence in handling structured grid data (e.g., images), their limitations become evident when dealing with complex topological visual data, particularly non-Euclidean data (such as relationships between different objects in an image or between video frames) [2]. CNNs assume that the input data have a regular grid structure, making it difficult for traditional CNN methods to effectively model the relationships and dependencies in complex graph-structured data.

To address this issue, graph neural networks (GNNs) have emerged as a deep learning approach that effectively handles irregular graph-structured data. GNNs model the relationships between nodes and edges, allowing them to capture deep information in image and video data processing, especially in multimodal inputs and complex scenarios, showing enormous potential [3]. Unlike CNNs, GNNs can process non-Euclidean data, adapting to object relationships in images and temporal dependencies in videos [4]. As a result, GNNs are playing an increasingly important role in tasks such as object recognition, video analysis, and temporal action localization.

What makes GNNs unique is their message passing mechanism, which allows information to propagate between nodes in the graph, capturing the dependencies between nodes. This enables GNNs to capture not only local spatial information but also the global dependencies between different nodes (e.g., objects, people, video frames, etc.) when processing visual data. Particularly in multimodal tasks, GNNs can better integrate data from different sources, providing more precise predictions and analyses. In video analysis, the graph convolution module can effectively capture temporal dependencies, significantly improving the performance of temporal action localization tasks [5]. In scene understanding tasks, GNNs, through efficient reasoning mechanisms, can optimize the fusion of data from different modalities, enhancing analytical efficiency [6].

Despite the progress GNNs have made in various fields, several challenges remain. Improving computational efficiency and addressing bottlenecks in large-scale data processing continue to be key issues. Additionally, the interpretability of GNN models and their adaptability across domains are also core challenges [7]. Future research will focus on optimizing the computational capabilities of GNNs, enhancing their adaptability to complex tasks, and further addressing the interpretability issues in their practical applications.

The structure of this paper is organized as follows: Section 2 reviews the basic concepts and development of GNNs; Section 3 discusses the applications of GNNs in computer vision, particularly in image processing; Section 4 analyzes the challenges faced by GNNs and explores future research directions; Section 5 concludes the paper, summarizing the findings and proposing directions for future work.

2. Development and Basic Concepts of Graph Neural Networks

2.1. Concept and Development of GNNs

Graph neural networks (GNNs) [8] were first proposed by Marco Gori et al. in 2005. The core idea is to iteratively aggregate the information from neighboring nodes and perform message passing to improve classification accuracy [9]. GNNs have shown excellent performance in image data processing, especially in capturing relationships between different regions. However, GNNs still face challenges, such as over-smoothing, over-compression, difficulty in handling heterogeneity, and difficulty in capturing long-range dependencies [10,11,12,13].

In 2009, F. Scarselli et al. introduced a graph-based data processing method that improved the accuracy of graph data processing [14]. Early GNNs, based on recurrent neural networks (RNNs), aggregated node features to generate vector representations, but their performance in complex image data processing was limited. To address this limitation, Bruna et al. proposed applying convolutional neural networks (CNNs) to graph data, introducing the graph convolutional network (GCN), which effectively reduced computational complexity [15]. However, GCN still had limitations when processing large-scale image data and was typically suitable only for shallow applications.

To overcome this limitation, William L et al. introduced the GraphSAGE model [16], which aggregates node features layer by layer using sampling and adjacency information, addressing computational and memory bottlenecks. However, GraphSAGE does not account for the differences in node importance, which limits its performance on data with large weight disparities. To address this, Petar Veličković et al. introduced the graph attention network (GAT) [17], which assigns different weights to neighboring nodes through an attention mechanism, achieving better results. Zhang et al. further proposed the gate attention network (GAAN) [18] by incorporating multi-head attention mechanisms, yielding significant results.

2.2. Evolution and Progress of GNN Models

To efficiently process complex graph-structured data, several classic GNN models have emerged, demonstrating strong capabilities in computer vision, including graph convolutional networks (GCN) [19], graph sample and aggregation (GraphSAGE) [20], graph attention networks (GAT) [21], and relational graph convolutional networks (R-GCN) [22]. These models significantly improve the efficiency and flexibility of feature learning through different aggregation and update strategies, especially in tasks such as image classification, object detection, and scene understanding. This section briefly discusses the working principles of these classic models, focusing on how they optimize the message passing and node feature update processes to capture the relationships between nodes in images. The meanings of the symbols used are provided in

Table 1. Symbols and their meanings.

Symbol	Meaning
v	Current node
u	Neighboring node
N(v)	Neighbor set of node v
$H_v^{\left(l\right)}$	Feature representation of node v at layer l
$H_u^{\left(l\right)}$	Feature representation of neighboring node u at layer l
W(l)	Learnable weight matrix at layer l
σ	Activation function (e.g., ReLU)
$m_v^{\left(l\right)}$	Message received by node v at layer l
$H_v^{(l+1)}$	Feature representation of node v at layer l + 1

.

2.2.1. Graph Convolutional Networks (GCN)

GCN updates node features by aggregating information from neighboring nodes, capturing structural relationships in images, and improving task performance. Its core mechanism involves message passing and feature updating. The message function and update function of GCN are given by Equations (1) and (2), respectively:

$$m_v^{\left(l\right)}={\displaystyle \sum _{u\in N\left(v\right)}}{H}_u^{\left(l\right)}$$

(1)

$$H_v^{(l+1)}=\sigma \left(\tilde{A}{H}^{\left(l\right)}{W}^{\left(l\right)}\right)$$

(2)

where $\tilde{A}=D^{-1/2}(A+I)D^{-1/2}$ represents the normalized adjacency matrix, I is the identity matrix, and D is the degree matrix.

GCN aggregates neighboring information progressively through multiple layers but it suffers from high computational complexity and may encounter the problem of over-smoothing, which leads to the loss of node features.

2.2.2. Graph Sampling and Aggregation (GraphSAGE)

GraphSAGE learns node representations by neighbor sampling and feature aggregation, making it suitable for large-scale datasets. The key steps include the following:

(a)

Sampling—random or stratified sampling is used to reduce computational complexity.

(b)

Feature Aggregation—the features of neighboring nodes are aggregated, with common methods including sum, average, and max pooling.

(c)

Message Function—the message passing process in GraphSAGE is given by Equation (3):

$$m_v^{\left(l\right)}=Aggregate\left(\right\{H_u^{\left(l\right)}:u\in N(v)\left\}\right)$$

(3)

(d)

Update Function—the function for updating node features is given by Equation (4):

$$H_v^{(l+1)}=\sigma (W^{\left(l\right)}\cdot Concat(H_v^{\left(l\right)},m_v^{\left(l\right)}\left)\right)$$

(4)

Here, Concat represents the concatenation of the current node features and the aggregated message. This update method helps to progressively refine the node representation in each layer.

GraphSAGE effectively handles large-scale graph data through these steps, but the treatment of neighbor node importance is relatively simple, which may fail to capture complex dependency relationships.

2.2.3. Graph Attention Networks (GAT)

GAT combines graph neural networks and self-attention mechanisms, enabling it to adaptively assign weights to neighboring nodes, enhancing the flexibility of information aggregation. Its core components are as follows:

(a) Attention Mechanism—by calculating the attention weights between each node and its neighboring nodes, GAT can dynamically adjust the flow of information. The attention coefficient is computed as shown in Equation (5):

$$a_{vu}={\displaystyle \frac{\mathit{exp}(\mathit{LeakyReLU}(a^T\left[W{H}_v\right|\left|W{H}_u\right]\left)\right)}{{\sum }_{k\in N\left(v\right)}\mathit{exp}(\mathit{LeakyReLU}(a^T\left[W{H}_v\right|\left|W{H}_k\right]\left)\right)}}$$

(5)

Here, W is a learnable weight matrix used for linearly transforming node features. a is a learnable vector used to compute the attention coefficient. The operation [.||.] represents concatenation.

(b) Message Function—using the calculated attention coefficients avu, GAT aggregates information from the neighboring nodes of node v. The message function is given by Equation (6):

$$m_v^{\left(l\right)}={\displaystyle \sum _{u\in N\left(v\right)}}{a}_{vu}W{H}_u$$

(6)

(c) Update Function—the update function of GAT is given by Equation (7):

$$H_v^{(l+1)}=\sigma (W^{\left(l\right)}{H}_v^{\left(l\right)}+m_v^{\left(l\right)})$$

(7)

GAT performs well in image tasks but has higher computational complexity, which can impact training efficiency, especially on large-scale data.

2.2.4. Relational Graph Convolutional Networks (R-GCN)

R-GCN is used to process graph data with multiple types of relationships. Its operation is as follows:

(a) Message Function—the message function of GCN is represented as shown in Equation (8):

$$m_v^{\left(l\right)}={\displaystyle \sum _{r\in R}}{\displaystyle \sum _{u\in N_r\left(v\right)}}{H}_u^{\left(l\right)}{W}_r^{\left(l\right)}$$

(8)

Here, R is the set of relationships, Nr(v) is the set of neighboring nodes of r connected by relationship v, and ${\mathrm{W}}_{\mathrm{r}}^{\left(\mathrm{l}\right)}$ is the learnable weight matrix associated with relationship r.

(b) Update Function—the update function of R-GCN is given by Equation (9):

$$H_v^{(l+1)}=\sigma \left({\displaystyle \sum _{r\in R}}{\displaystyle \sum _{u\in N_r\left(v\right)}}{W}_r{h}_u+b\right)$$

(9)

where $\mathsf{\sigma }$ is the activation function (e.g., ReLU) and $\mathrm{b}$ is the bias term. This mechanism enables R-GCN to handle graph data with multiple types of relationships, excelling in tasks like image segmentation and object recognition.

R-GCN is effective at processing graph data with multiple relationship types and performs well in tasks such as image segmentation and object recognition. However, it has high computational complexity and requires more computational resources.

2.2.5. Model Comparison

To evaluate the performance of different GNN architectures in computer vision tasks, as shown in Table 2, this paper provides a detailed analysis of four mainstream models from aspects such as computational efficiency and relationship modeling capabilities, offering key reference points for selecting the appropriate model.

2.3. Spectral and Spatial Graph Neural Networks

As the importance of graph-structured data in computer vision continues to increase, spectral and spatial graph neural networks (GNNs) have gradually become research hotspots. Spectral GNNs define convolution operations through graph spectral theory, effectively handling the non-Euclidean characteristics of image data and excelling at capturing image structures and spatial relationships. Spatial GNNs, on the other hand, handle the relationships between nodes through a message passing mechanism, making them suitable for pixel-level relationships and inter-object associations. These two approaches complement each other and drive the application of GNNs in computer vision. This section will briefly introduce the key advancements of these two types of GNNs in computer vision.

2.3.1. Spectral Graph Neural Networks

With the widespread application of convolutional neural networks (CNNs) in image processing and text analysis, researchers began to extend these methods to graph-structured data. Bruna et al. [15] introduced spectral convolutional networks, which perform spectral decomposition using the graph’s Laplacian matrix and define convolution operations in the spectral domain to better capture structural features in images [23]. However, this method has high computational complexity, especially when dealing with large-scale data, making efficiency a significant challenge. To address this issue, Defferrard et al. [24] proposed ChebyNet, which improves computational efficiency by using polynomial-form convolution kernels. Kipf and Welling [25] simplified this approach by introducing graph convolutional networks (GCN), which optimize computation through a first-order approximation of the convolution kernel, demonstrating significant advantages in tasks such as image classification and object detection.

Table 2. Model comparison.

Model	Features	Computational Complexity	Inference Time and Memory Consumption	Advantages	Limitations
GCN [24]	Approximate method based on spectral graph convolution, capturing local graph structure.	$O\left(\left|E\right|\right)$	The inference time is fast, influenced by the number of edges E and the structure of the graph, with relatively low memory consumption.	Computationally efficient, suitable for large-scale graphs, easy to implement.	Only applicable to fixed-structure graphs, cannot handle dynamic graphs, and ignores global relationships during inference.
GraphSAGE [16]	Generates node embeddings by sampling local neighborhood information.	$O\left(\left|E\right|K\right)$	The inference time is proportional to the number of edges E and the depth of neighborhood sampling K, with moderate memory consumption.	Supports inductive learning, capable of handling large-scale and dynamically changing graph data.	The aggregation function has high complexity and computational overhead.
GAT [17]	Introduces self-attention mechanism, capable of handling irregular graphs, suitable for dynamic structures.	$O\left(\left|V\right|F+\left|E\right|\right)$	The inference time is influenced by the number of nodes V, edges E, and feature dimensions F, with higher computational complexity and lower memory consumption.	Introduces self-attention mechanism, adapts to irregular graphs, supports handling dynamic structures.	Attention computation is limited, unable to scale to large graphs, and inference speed is slow.
R-GCN [22]	Designed specifically for handling multi-relational graph data, applied to knowledge graphs.	$O\left(\left|E\right|F\right)$	The inference time is higher, dependent on the number of edges E and feature dimensions F, with moderate memory consumption.	Capable of handling complex multi-relational graph data, suitable for knowledge graph tasks.	Requires a large number of parameters during training, with slower inference and higher model complexity.

Table 2. Model comparison.

Model	Features	Computational Complexity	Inference Time and Memory Consumption	Advantages	Limitations
GCN [24]	Approximate method based on spectral graph convolution, capturing local graph structure.	$O\left(\left|E\right|\right)$	The inference time is fast, influenced by the number of edges E and the structure of the graph, with relatively low memory consumption.	Computationally efficient, suitable for large-scale graphs, easy to implement.	Only applicable to fixed-structure graphs, cannot handle dynamic graphs, and ignores global relationships during inference.
GraphSAGE [16]	Generates node embeddings by sampling local neighborhood information.	$O\left(\left|E\right|K\right)$	The inference time is proportional to the number of edges E and the depth of neighborhood sampling K, with moderate memory consumption.	Supports inductive learning, capable of handling large-scale and dynamically changing graph data.	The aggregation function has high complexity and computational overhead.
GAT [17]	Introduces self-attention mechanism, capable of handling irregular graphs, suitable for dynamic structures.	$O\left(\left|V\right|F+\left|E\right|\right)$	The inference time is influenced by the number of nodes V, edges E, and feature dimensions F, with higher computational complexity and lower memory consumption.	Introduces self-attention mechanism, adapts to irregular graphs, supports handling dynamic structures.	Attention computation is limited, unable to scale to large graphs, and inference speed is slow.
R-GCN [22]	Designed specifically for handling multi-relational graph data, applied to knowledge graphs.	$O\left(\left|E\right|F\right)$	The inference time is higher, dependent on the number of edges E and feature dimensions F, with moderate memory consumption.	Capable of handling complex multi-relational graph data, suitable for knowledge graph tasks.	Requires a large number of parameters during training, with slower inference and higher model complexity.

2.3.2. Spatial Graph Neural Networks

Meanwhile, spatial graph neural networks have gained widespread attention in computer vision. Li et al. [26] introduced the gated graph neural network (Gated GNN), replacing the traditional RNN node update mechanism with gated recurrent units (GRU), which eliminates the limitations of compressed mappings and improves the efficiency of GNN in processing complex image data, especially in image and video analysis. Subsequently, several variants emerged, such as PATCHY-SAN [26], which mimics CNN convolution by node ordering and adjacent point selection; MoNet, which defines pseudo-coordinates for adjacent points and unifies multiple GNN models, showing advantages in scene understanding and object detection; graph attention network (GAT), which uses attention mechanisms to adaptively assign weights to neighboring nodes; and GraphSAGE, which extends to inductive learning via neighbor sampling to accelerate large-scale image data learning. The message passing network [27] unifies spatial GNNs into a message-passing framework, enhancing the efficiency of information exchange between nodes; Xu et al. [28] demonstrated that the expressive power of GNNs is equivalent to the Weisfeiler–Lehman graph isomorphism test, showing its strong capabilities in image classification and object detection.

After the advancements in spectral and spatial GNNs, researchers introduced various variants to address complex graph-structured data and dynamic application scenarios. For example, graph diffusion equations (Graph DEs) [29] improve image structure understanding via diffusion equations; graph transformation networks (GTN) [30] enhance the ability to process heterogeneous graphs; temporal graph networks (TGN) [31] are suitable for dynamic graph data; graph multi-layer perceptron (Graph-MLP) [32] enhances expressive power; and graph transformer (Graphormer) [33] leverages self-attention mechanisms to enhance long-range dependency modeling. Additionally, GPSE (graph positional encoding) [34] improves GNN performance in handling complex structural data by incorporating positional information into the graph. Through these variants, GNN has not only enriched its model architecture but also greatly expanded its application potential across various computer vision fields. Overall, the development chart of graph neural networks (see Figure 1) illustrates the evolution from basic models to various variants, reflecting the ongoing innovation and exploration by researchers in handling image data.

3. Applications of Graph Neural Networks in Computer Vision

In recent years, convolutional neural networks (CNNs) have made significant breakthroughs in the field of computer vision (CV). However, they still have limitations in capturing complex visual relationships, such as inter-region associations and temporal dependencies between video frames. For example, images can be viewed as spatial graphs, where each node represents an area of interest, and the edges represent the relationships between these regions, capturing inter-region dependencies. Graph neural networks (GNNs) are thus naturally suited to extract patterns from such graphs, aiding in the completion of computer vision tasks.

This section will explore the applications of GNNs in image processing, video analysis, and cross-media tasks [35], and discuss how they are adapted in representative algorithms to significantly improve task performance.

3.1. Image Processing

3.1.1. Object Detection

Object detection is one of the core tasks in computer vision and has garnered widespread attention in both academia and industry in recent years [36]. This task typically involves object detection and instance segmentation, with the goal of identifying the category and location of objects in an image. Deep learning methods, such as Faster-RCNN [37] and YOLO [38], have made significant progress in this field. Despite the success of these methods in terms of performance, most approaches overlook the interrelationships between categories, which can lead to performance degradation, especially under class imbalance or long-tail data distributions [39]. To address this issue, graph neural networks (GNNs) have been introduced to object detection tasks to model the dependencies between categories and enhance model performance. SGRN [40] (as shown in Figure 2) is a typical example, where the introduction of GNN improves detection accuracy and strengthens the modeling of inter-category relationships.

The challenge in object detection tasks lies not only in recognizing individual objects but also in reasoning about the semantic dependencies, co-occurrence relationships, and relative positions between objects. GNN-based methods address these issues through local and global relationship reasoning. The equation is shown in Equation (10):

$$G=\prod \left(I,O\right),\widehat{X}=GNN\left(G,A\right)$$

(10)

GNN effectively captures dependencies between objects by modeling the edges in the graph. For example, Reasoning-RCNN proposes an adaptive global reasoning network that enhances reasoning ability by combining a knowledge graph for global propagation of visual information. SGRN [40] introduces a mechanism for adaptive discovery of semantic and spatial relationships, without relying on manually constructed knowledge bases. RelationNet [41] incorporates an adaptive attention module to explicitly learn the relationships between objects. RelationNet++ [42] introduces a self-attention-based decoder module, enabling the fusion of different object/part representations within a single detection framework. Li et al. [43] introduce heterogeneous graphs to jointly model the relationships between objects and scenes. Recently, GraphFPN [44] proposed a graphical feature pyramid network that facilitates within-scale and cross-scale feature interactions through superpixel hierarchies and spatial and channel attention mechanisms.

As shown in

Table 3. Object detection model comparison.

Method	mAP Accuracy	Inference Speed (FPS)	Memory Consumption
Faster R-CNN [37]	36.2%	5 fps	200 MB
YOLO [38]	57.9%	40+ fps	250 MB
Mask R-CNN [39]	37.1%	10 fps	300 MB
SGRN [40]	38.3%	15 fps	320 MB
RelationNet++ [42]	45.2%	12 fps	350 MB
GraphFPN [44]	46.2%	13 fps	370 MB

, a performance comparison of several representative object detection methods on the COCO [45] dataset is presented, covering key evaluation metrics such as mAP accuracy, inference speed (FPS), and memory consumption.

Based on the comparison data in Table 3, we can observe the differences in accuracy and inference speed among different object detection methods. YOLO excels in inference speed, making it suitable for real-time detection applications, but has relatively lower accuracy. In contrast, Faster R-CNN and Mask R-CNN have an advantage in accuracy, particularly for tasks that require high detection precision, although their inference speeds are slower. RelationNet++ introduces an attention-based decoder module and performs well in terms of mAP, capable of integrating representations of different objects/parts, making it suitable for tasks that require precise relationship modeling, though with slower inference speed. SGRN stands out in enhancing inter-class relationship modeling, making it suitable for tasks that require complex relationship reasoning, though its inference speed is relatively low. GraphFPN improves performance through cross-scale feature interaction, performing well in complex scenarios, but with higher memory consumption and moderate inference speed.

Additionally, domain-adaptive object detection (DAOD) has attracted considerable attention for cross-domain data processing. A natural method for modeling cross-domain relationships is to construct a bipartite graph $G=\left\{V_s,V_t,E\right\}$, where the vertices are divided into two disjoint sets, and the edges connect vertices from different sets. Chen et al. [46] constructed a relational graph based on cross-domain consistency, modeling the dependencies between objects in the pixel and semantic spaces through bipartite graph convolution networks and graph attention mechanisms. SIGMA [47] formulates DAOD as a graph matching problem, modeling category distributions through cross-image graphs. SRR-FSD [48] introduces a semantic relationship reasoning module that integrates the relationships between base and novel classes, improving the detection of novel objects.

Due to the advantages of GNNs in modeling long-range dependencies, GNN-based detection methods can effectively leverage relationships between objects, across scales, and across domains [49]. However, as additional learning modules are introduced, the optimization process becomes more complex, and compatibility issues between Euclidean and non-Euclidean data still need to be addressed. Future research can explore better feature mapping methods, combine Transformer or pure GNN encoders to enhance node feature representation, or perform reasoning directly in the original feature space, avoiding reliance on feature mappings, thus better preserving the intrinsic structure of the image.

3.1.2. Image Segmentation

Inspired by the groundbreaking advancements in deep learning technologies, significant progress has been made in the field of image classification, with models like ResNet [50] and DenseNet [51] demonstrating outstanding performance in image classification tasks. However, models based on convolutional neural networks (CNNs) have limitations in modeling complex relationships between samples. To address this issue, graph neural networks (GNNs) have been introduced to image classification tasks, enhancing the classification performance of fine-grained image instances through semi-supervised learning [52].

Image segmentation tasks involve partitioning an image into semantic regions and performing pixel-level labeling. Despite advances in CNN architectures, they are unable to reason about distant regions with arbitrary shapes, which affects overall scene understanding. GNNs provide a unified framework capable of modeling both object appearance and image context, improving semantic segmentation performance.

$$G=\prod \left(I\right),A=E\left(R_a,R_c\right),\widehat{I}=P\left(\mathit{GNN}\left(G,A\right)\right)$$

(11)

Here, $E\left(\cdot \right)$ is a function that simultaneously encodes $R_a$ and $R_c$, $P\left(\cdot \right)$ is a pixel-level predictor, and $\widehat{I}$ represents the predicted result.

Zhang et al. [53] proposed a dual-GCN framework to separately model the spatial relationships between pixels and the dependencies between channel dimensions. Chen et al. [54] designed a global reasoning unit to perform relational reasoning, enhancing global feature aggregation. To avoid constructing a fully connected graph, DGMN [55] employs dynamic neighborhood sampling to predict node dependencies and information propagation affinity. Yu et al. [56] used dynamic sampling of representative nodes for relational modeling. Li et al. [57] improved the Laplacian formula to enable graph reasoning in the original feature space. Hu et al. [58] introduced a category dynamic graph convolution module for dynamic aggregation of pixels with the same category.

For comparison and analysis,

Table 4. Performance comparison of GNN-based semantic segmentation models.

Model	mIoU (%)	Parameter Count (M)	Key Innovation
Dual GCN [51]	78.5	28.1	Spatial-Channel Dual Graph Modeling
GRN [52]	76.2	45.3	Global Relationship Reasoning Unit
CDGC [56]	80.1	29.5	Category Dynamic Graph Convolution

presents the performance comparison of several GNN-based semantic segmentation methods on the PASCAL VOC 2012 dataset [59]. From the table, it can be seen that Dual GCN achieves a good balance between mIoU accuracy and parameter count through spatial-channel dual graph modeling; GRN introduces a global relationship reasoning unit, enhancing global feature aggregation capabilities, but with a larger parameter count; and CDGC improves mIoU accuracy through a category dynamic graph convolution module, particularly suitable for dynamic feature aggregation, though it comes with relatively higher computational overhead.

For single-shot image segmentation, Zhang et al. [60] introduced the pyramid graph attention module, which combines unlabeled pixels with semantic and contextual information. Xie et al. [61] proposed scale-aware GNNs, using cross-scale relationship reasoning and a node cooperation mechanism to perceive different resolutions of the same object. Zhang et al. [62] proposed affinity attention GNNs, which convert the image into a weighted graph and propagate semantic information to unlabeled pixels. Wu et al. [63] designed a bidirectional graph reasoning network for diffuse segmentation.

Table 5. Few-shot segmentation model comparison.

Model	mIoU (%)	Key Innovation
PGAM [57]	56.3	Pyramid Graph Attention
Scale-Aware GNN [58]	61.7	Cross-Scale Node Collaboration

presents the mIoU comparison of two single-shot segmentation models on the PASCAL-5i 1-shot dataset [64]. PGAM enhances the representation capability of images through the pyramid graph attention mechanism, although its mIoU accuracy is relatively lower; scale-aware GNN, on the other hand, achieves higher mIoU accuracy through cross-scale node collaboration, making it suitable for handling cross-scale relationships.

Many studies explore local and global context information using pyramid pooling, dilated convolutions, or self-attention mechanisms. For example, the non-local network [65] achieves this through a self-attention mechanism, although it comes with high computational cost. In contrast, GNN-based methods offer clear advantages in relationship modeling and training efficiency.

3.2. Video Analysis

3.2.1. Video Action Recognition

Human action recognition in videos is one of the core tasks in video processing and understanding, aimed at recognizing human actions in RGB/depth videos or skeletal data.

Regardless of the data format, modeling the spatiotemporal context of humans, objects, and joints is crucial for behavior recognition. Typically, a graph G is composed of objects, people, video frames, or skeletons, and GNNs predict actions by modeling the relationships between these elements.

$$G=\prod \left(\widehat{V},\widehat{E}\right),F=GNN\left(G\right)$$

(12)

Here, $\widehat{V}$ represents nodes that can represent objects, people, video frames, or skeletons; $\widehat{E}$ represents edges that capture spatial relationships between objects within a single frame or temporal relationships across multiple frames; and F represents the aggregated information for final prediction.

Early action recognition methods, such as hand-crafted improved dense trajectory (iDT) [66], two-stream ConvNets [67], C3D [68], and I3D [69] focused on using spatiotemporal appearance features. To better model long-term temporal information, researchers attempted to use recurrent neural networks (RNNs) to model videos as ordered frame sequences [70]. However, these methods overlooked the spatial and temporal relationships between object instances, which are especially important in actions involving interactions between objects and people, such as “opening a book”.

In recent years, researchers have proposed various deep learning models [54] that utilize graph neural networks (GNNs) to build spatiotemporal representations of videos [71]. These models treat objects in the video as nodes in a graph and learn spatiotemporal relationships to optimize video analysis tasks. Wang et al. [72] proposed a graph-based reasoning model that captures long temporal context, reasoning about human–object and object–object relationships. Figure 3 illustrates a video action recognition model based on CNN, where a 3D convolutional neural network (I3D) processes video frames to obtain feature maps $I\in R^{T\times H\times W\times d}$, where T represents the temporal dimension, $H\times W$ represents the spatial dimensions, and d represents the channel number. A region proposal network (RPN) [37] then extracts object bounding boxes, and RoIAlign [39] extracts features for each candidate object. The N output candidate objects correspond to N nodes in the graph, with connections based on appearance similarity and spatiotemporal relationships, and human behavior is predicted through graph convolutional networks (GCN). Ou et al. [73] proposed an object-level graph centered on the participants and applied GCNs to capture object context, improving spatiotemporal graphs. Zhang et al. [74] proposed a multi-scale temporal graph reasoning model that utilizes multiple GAT heads and temporal adjacency matrices to capture both short-term and long-term dependencies.

For comparison and summary,

Table 6. Comparison of GNN-based video action recognition models.

Model	Top-1 Acc (%)	Parameter Count (M)	Key Innovation
Region Graphs [72]	76.3	23.5	Spatio-Temporal Region Graph Reasoning
Object-Relation [73]	78.1	32.7	Participant-Centric Object-Level Graph
Temporal Reasoning [74]	77.5	28.4	Multi-Scale Temporal Graph Reasoning

presents the performance comparison of several GNN-based video action recognition models on the Kinetics-400 dataset [69]. Region graphs has a lower parameter count but relatively lower Top-1 accuracy; object-relation improves performance by introducing object-level graphs but at the cost of increased parameter count; temporal reasoning performs well in handling temporal information, achieving higher accuracy, making it suitable for long temporal tasks, though it comes with relatively higher computational overhead.

In summary, the application of graph neural networks (GNNs) in video action recognition provides a novel approach to addressing the limitations of traditional methods in capturing long-term dependencies and spatial relationships. GNNs can effectively model the relationships between objects and the spatiotemporal context information.

3.2.2. Temporal Action Localization

The task of temporal action localization aims to predict the boundaries and categories of action instances in a video. Most existing methods [75] adopt a two-stage pipeline; first, a set of candidate regions is generated, followed by classification and regression of the temporal boundaries for each candidate. However, these methods fail to effectively utilize the semantic relationships between candidates. To address this issue, graph neural networks (GNNs) have been used to model the interactions between candidates in a video, thereby enhancing the recognition ability of each candidate. P-GCN [76] proposes a method that uses graph convolutional networks (GCN) to model the relationships between candidates. The method first constructs a candidate action graph, treating each candidate action as a node, with the relationships between candidates represented by edges. P-GCN performs reasoning via GCN, transmitting information between candidates and updating node representations, ultimately optimizing the temporal boundaries and classification scores based on the dependencies between candidates, thus improving the accuracy of temporal action localization. In

Table 7. Comparison of core temporal action localization models.

Model	mAP@0.5 (%)	mAP@0.75 (%)	Key Innovation
CDC [75]	45.3	26.1	Deconvolutional Temporal Localization
P-GCN [76]	48.3	27.5	Candidate Graph Reasoning

, the performance comparison of the above temporal action localization models on the ActivityNet-1.3 dataset [77] is presented. It can be seen that CDC has a more conservative performance in temporal action localization, with an mAP@0.5 of 45.3%, but its computation is relatively light, making it suitable for applications with high inference speed requirements; P-GCN improves performance through candidate graph reasoning, achieving an mAP@0.5 of 48.3%, but compared to CDC, its computational complexity and inference time are higher. Both models have their advantages; CDC is suitable for real-time applications, while P-GCN is more suitable for tasks that require higher accuracy.

3.3. Other Related Work: Cross-Media

Graph-structured data are widely present in various modalities such as images, videos, and text, playing an important role in tasks such as visual description, visual question answering, and cross-media retrieval. Effectively utilizing graph neural networks (GNNs) can significantly enhance the performance of cross-media tasks.

3.3.1. Visual Description

The task of visual description aims to automatically generate natural language descriptions of images or videos, helping visually impaired individuals understand visual content, and has gained significant attention due to its challenge in visual understanding.

Many existing methods [78], inspired by machine translation, attempt to “translate” image information into natural language. These methods typically use convolutional neural networks (CNNs) or region-based CNNs (R-CNNs) to encode the image, followed by a recurrent neural network (RNN) decoder to generate the description. However, the interrelationships and interactions between objects in an image are a natural foundation for descriptions, and how to effectively utilize these visual relationships remains an underexplored issue.

To address this, Yao et al. [79] proposed a graph convolutional network–long short-term memory (GCN-LSTM) framework to enhance image description by exploring visual relationships. This framework starts by modeling the interactions between objects and regions, enriching region-level feature representations, and passing them to the sentence decoder. Figure 4 shows how this method learns semantic and spatial relationships through graph convolution layers, ultimately generating natural language descriptions.

Subsequently, Yang et al. [80] proposed the scene graph auto-encoder (SGAE) framework for image description tasks. This framework consists of two main steps:

(a)

Extract the scene graph from the image and encode it using graph convolutional networks (GCN) to re-represent the image content, ultimately generating a natural language description;

(b)

Incorporate the scene graph into the description generation model, combining image and language information to generate the description.

As shown in

Table 8. Comparison of visual description models’ performance.

Model	BLEU-4	METEOR	CIDEr
Global–Local [78]	36.5	27.7	108.2
GCN-LSTM [79]	38.2	28.3	112.5
SGAE [80]	39.1	28.9	114.8

, the performance comparison of each model on standard evaluation metrics such as BLEU-4, METEOR, and CIDEr is presented based on their performance on the COCO dataset. It can be seen that SGAE performs the best across BLEU-4, METEOR, and CIDEr, particularly with a CIDEr score of 114.8, making it suitable for tasks that require high-precision description generation; GCN-LSTM follows closely behind, with slightly lower metrics but still offering high performance and lower computational complexity; global–local shows relatively weaker performance, especially on the CIDEr metric, making it more suitable for applications with higher speed requirements.

3.3.2. Visual Question Answering (VQA)

The task of visual question answering (VQA) aims to enable a system to automatically answer natural language questions about visual information. Due to the need to understand and reason about the relationship between visual and textual modalities, this task presents significant challenges. In recent years, deep learning has driven advancements in VQA technology. Early methods [81] mainly focused on jointly representing the visual and language modalities, combined with encoder–decoder architectures and attention mechanisms, achieving significant success.

However, these methods did not fully consider the graph-structured information in VQA tasks. To address this, Zhang et al. [82] proposed a new method based on knowledge graphs, utilizing scene graphs to encode information and provide structured reasoning. As shown in Figure 5, experiments demonstrate that when scene graphs are combined with graph neural networks (GNNs), VQA performance significantly improves, particularly in tasks involving object counting, existence, attributes, and multi-object relationships [83].

Another study [84] proposed the relation-aware graph attention network (ReGAT) for visual question answering, aiming to model the relationships between multiple types of objects through a question-adaptive attention mechanism. This method first uses Faster R-CNN to generate candidate object regions and embeds the question into the model using a question encoder. The convolutional features of the object regions and the bounding box features are then fed into a relation encoder, learning relation-aware region-level features relevant to the question. Finally, these relation-aware visual features, along with the question embedding, are input into a multimodal fusion module to generate a joint representation, which is then processed by an answer prediction module to generate the final answer. As shown in

Table 9. Comparison of visual question answering models’ performance.

Model	Accuracy (%)	Key Innovation
Cycle-Consistency [81]	68.5	Cycle Consistency Constraint for Enhanced Cross-Modal Alignment
ReGAT [84]	70.2	Question-Adaptive Graph Attention Mechanism

, the accuracy comparison of each model is presented based on their performance on the VQA v2.0 dataset [85]. From Table 9, it can be seen that ReGAT performs the best in terms of accuracy, achieving 70.2%, by effectively improving cross-modal alignment performance through the introduction of a question-adaptive graph attention mechanism; cycle-consistency has an accuracy of 68.5%, enhancing cross-modal alignment through cycle consistency constraints, but its performance is slightly lower than that of ReGAT. The innovations of both models focus on different aspects; ReGAT is more suitable for tasks that require higher accuracy, while cycle-consistency is ideal for scenarios that need efficient alignment.

3.3.3. Cross-Media Retrieval

Image–text retrieval is an important direction in cross-modal research, aiming to retrieve relevant content from multimodal databases of images and texts. The main challenge of this task lies in understanding and measuring the semantic similarity between images and texts. Traditional methods [86] typically describe images and texts through global or local features. However, these methods neglect the object relationships within images and texts, resulting in poor retrieval performance.

To address this issue, as shown in Figure 6, Yu et al. [87] proposed a dual-channel neural network model based on graph convolutional networks. This model extracts structured features from both the visual and text modalities and couples these features through a graph neural network for matching in a shared semantic space. This method effectively captures the deep relationships between images and texts, improving the accuracy of cross-modal retrieval.

Additionally, Wang et al. [88] proposed a method to retrieve objects and their relationships from both images and text, forming visual and textual scene graphs. They designed a scene graph matching (SGM) model, which uses two customized graph encoders to encode the visual and textual scene graphs into feature maps. In each graph, it learns the features of objects and relationships at both the object and relational levels. This approach allows SGM to more effectively match corresponding features between different modalities on two levels.

For comparison and summary,

Table 10. Comparison of image–text retrieval models’ performance.

Model	Image→Text R@1	Text→Image R@1	Key Innovation
VSE++ [87]	52.9	39.6	Hard Negative Sample Mining for Optimized Embedding Space
Dual-Channel GNN [89]	58.3	42.7	Graph Convolutional Network Coupling Multimodal Features
SGM [90]	62.1	46.9	Scene Graph Hierarchical Matching

presents the performance comparison of three image–text retrieval models on the Flickr30K dataset [89]. From Table 10, it can be seen that SGM performs the best on both metrics, making it suitable for high-precision cross-modal retrieval tasks. Its innovative scene graph hierarchical matching method effectively improves retrieval accuracy; dual-channel GNN introduces graph convolutional networks to couple multimodal features, achieving good results, though slightly behind SGM in performance; VSE++ shows relatively weaker performance, particularly in the Text→Image R@1 metric, making it more suitable for applications where computational efficiency is prioritized.

3.3.4. Applications in Different Fields

Neural networks are a computational model that mimics the structure and function of biological neural networks, connecting a large number of artificial neurons (nodes) to process complex data and recognize patterns. In the field of image classification, Alhatemi et al. [91] designed a multiple pre-trained deep learning by combining ensemble learning with multiple classifiers for classification of stroke. Zheng et al. [92] proposed a broad sparse fine-grained image classification model based on dictionary selection strategy for image classification. Chen et al. [93] proposed a novel dual-scale complementary spatial-spectral joint classification model to mitigate the issues of detail loss and insufficient utilization of spatial information. Chattopadhyay [94] used the machine learning-based approach mimicking to diagnose a case with the help of senior doctors. Zhao et al. [95] proposed a fuzzy broad neuroevolutionary network via multiobjective evolutionary algorithms. Li et al. [96] proposed an approach using pulse signals containing emotional cues and deep learning to automatically detect the severity of stress in college students. Aher [97] proposed a political deer hunting optimization algorithm-based deep Q-network for detecting heart disease. Long et al. [98] proposed a novel principal space approximation ensemble discriminant edge least-squares regression for hyperspectral image classification. Long et al. [99] proposed a dual-model collaborative label adaptive correction BLS based on the collaboration of KRPBLS and SKBLS to significantly improve the robustness of BLS under label noise environment. In the field of fault diagnosis, Yao et al. [100] designed a parallel multiscale convolutional transfer neural network for cross-domain fault diagnosis by integrating 1-D feature maps into 2-D feature maps. Li et al. [101] proposed a hybrid framework for early depression detection that integrates multiple deep learning techniques and ensemble learning. In the field of traffic, Guo et al. [102] proposed an automation paradigm integrating controlling intent into the information processing loop through the spoken instruction-aware flight trajectory prediction framework. Deng et al. [103] proposed an autonomous path planning for unmanned aerial vehicles in dynamic environment. Lin et al. [104] proposed an automatic speech recognition method for improving air traffic safety. Zhu et al. [105] proposed an effective and robust genetic algorithm with hybrid multi-strategy and mechanism for airport gate allocation. Huang et al. [106] proposed a novel cylindrical coordinate particle swarm optimization planner with gene targeting for obtaining feasible and accurate flight routes for multiple UAVs. In another field, Li et al. [107] proposed an anti-poisoning attack decentralized privacy enhanced federated learning model for data sharing. Deng et al. [108] proposed a quantum differential evolutionary algorithm for high-dimensional problems. Huang et al. [109] proposed a multiple level competitive swarm optimizer for large-scale optimization problem. In addition, some other methods are proposed for different applications [110,111,112,113].

3.4. Frontier Issues of Graph Neural Networks in Computer Vision

This section introduces frontier issues of graph neural networks (GNNs) in computer vision, including advanced GNNs for computer vision and the broader applications of GNNs in the field.

3.4.1. Advanced Graph Neural Networks for Computer Vision

In computer vision, graph neural networks (GNNs) typically represent visual information as graph structures. Traditional methods often treat pixels, object bounding boxes, or image frames as nodes, constructing homogeneous graphs to model the relationships between them. In recent years, some novel modeling approaches have been applied to GNNs:

(a)

Person Feature Blocks: Yan et al. [114], Yang et al. [115], and Yan et al. [116] constructed spatial and temporal graphs for person re-identification (Re-ID) tasks. By horizontally dividing person feature maps into several small blocks, these blocks are treated as nodes, and graph convolutional networks (GCN) are used to model the relationships between body parts across frames.

(b)

Irregular Clustering Regions: Liu et al. [117] proposed a bipartite graph GNN for breast X-ray quality detection. Using k-nearest neighbors (kNN) forward mapping, the image feature map is divided into irregular regions, and the features of these regions are integrated as nodes. Nodes from images across different views represent geometric constraints and appearance similarities through the model.

(c)

Neural Architecture Search (NAS) Units: Lin et al. [118] introduced a graph-based neural architecture search algorithm (NAS), where operation units are represented as nodes, and GCNs are used to model the relationships between units, thus improving the efficiency and performance of network structure search.

3.4.2. Broader Applications of Graph Neural Networks in Computer Vision

(a) Point Cloud Analysis: The goal of point cloud analysis is to identify a set of points in a coordinate system, where points are represented by their coordinates and other features. Early studies, such as PointNet++ [119] and VoxelNet [120], attempted to convert point clouds into regular grids (such as images and voxels) to utilize convolutional neural networks (CNNs). Recent research [121] has adopted graph representations to preserve the irregularity of point clouds, with GCN aggregating local information in a manner similar to CNNs in image processing. Chen et al. [122] proposed a hierarchical graph network structure for 3D object detection, Lin et al. [123] introduced learnable GCN kernels and used max pooling to process the receptive field of k-nearest neighbor nodes, Xu et al. [124] proposed coverage-aware mesh queries and mesh context aggregation to accelerate 3D scene segmentation, and Shi and Rajkumar [125] designed Point-GNN for detecting multiple objects in a single sample.

In

Table 11. Comparison of point cloud 3D object detection performance.

Model	mAP@0.5	Speed (FPS)	Key Innovation
VoxelNet [120]	53.1	4.2	Voxelization + 3D CNN
PointNet++ [119]	58.3	12.4	Hierarchical Point Set Feature Learning
Point-GNN [125]	64.7	8.6	End-to-End Graph Neural Network Detection
Grid-GCN [124]	63.9	15.3	Efficient Grid Context Aggregation

, the performance comparison of several point cloud 3D object detection models on the ScanNet v2 dataset [126] is presented. It can be seen that Point-GNN performs the best on the mAP@0.5 metric, achieving 64.7%, and effectively improves 3D object detection accuracy through an end-to-end graph neural network detection method; Grid-GCN improves performance through efficient grid context aggregation, and although its inference speed is faster, its mAP is slightly lower than that of Point-GNN; PointNet++ employs hierarchical point set feature learning, performing well in terms of accuracy, making it suitable for more complex scenarios, but with slower inference speed; and VoxelNet processes point cloud data through voxelization and 3D CNN methods, achieving lower accuracy, but with less computational overhead, making it suitable for applications with higher speed requirements.

(b) Low-Resource Learning: Low-resource learning aims to learn from minimal data or prior knowledge. Wang et al. [127] and Kampffmeyer et al. [128] used knowledge graphs to guide zero-shot learning. Garcia and Bruna [129], Liu et al. [130], and Kim et al. [131] designed similarity metrics, modeling probabilistic learning problems as label propagation or edge-labeling problems, applied to facial recognition tasks. Wang et al. [132] inferred the link probability of nodes in face subgraphs using GCNs. Yang et al. [133] proposed a similarity-graph-based candidate-detection-segmentation framework for face clustering. Zhang et al. [134] proposed a global–local GCN framework for label cleaning in face recognition.

(c) Other Scenarios: Wei et al. [135] proposed the View-GCN model for recognizing 3D shapes through projected 2D images. Wald et al. [136] extended scene graphs to 3D indoor scenes. Ulutan et al. [137] utilized GCNs to reason about interactions between people and objects. Cucurull et al. [90] predicted fashion compatibility between two items by modeling edge prediction problems. Sun et al. [138] constructed social behavior graphs and used GNNs to propagate interaction information for trajectory prediction. Zhang et al. [139] established visual–language relationship graphs to mitigate hallucination problems in video description tasks.

Graph neural networks (GNNs) have demonstrated enormous potential in the field of computer vision but still face challenges related to interpretability. Due to the high connectivity and complexity of nodes and edges, the interpretability issue of GNNs is particularly prominent in decision-making tasks, such as medical diagnosis. Therefore, improving the interpretability and robustness of GNNs in visual tasks has become a key issue that needs to be addressed.

4. Challenges and Future Directions

Although graph neural networks (GNNs) have made significant progress in the field of computer vision, they still face numerous challenges that need to be addressed. Firstly, issues related to computational complexity and scalability limit the application of GNNs to large-scale image and video data. The traditional message passing mechanism leads to exponential growth in computational costs as the number of nodes V and edges E in the graph increases, particularly when handling large-scale video data, where the computational cost is often prohibitive. Although low-rank approximation methods, such as first-order approximated convolution kernels, significantly reduce the computational complexity of GCNs [140], existing algorithms still face significant memory bottlenecks when processing large-scale 3D point clouds or long temporal video data. Recent research by Zhou et al. [141] shows that through neural architecture search, redundant computations can be reduced by more than 80%, providing a new approach for developing real-time GNN systems.

When processing cross-modal spatiotemporal data, temporal graph networks (TGN) [31] achieve short-term temporal dependency modeling through a memory mechanism. However, in long temporal tasks (such as U-Action video analysis), the limitations of the memory mechanism lead to a decrease in action recognition accuracy [142]. Notably, Graphormer [33] introduced spatiotemporal positional encoding, achieving a 5.2% performance improvement on the OGB molecular dataset, demonstrating the potential of the Transformer architecture in dynamic graph modeling. However, balancing computational overhead with model accuracy remains an unresolved challenge, requiring joint optimization through graph sparsification and adaptive sampling [143].

Another key issue is the fusion of heterogeneous graphs and multimodal data. Currently, most GNN models assume that nodes and edges in a graph belong to the same modality, but in cross-modal tasks (such as visual question answering and image–text retrieval), the heterogeneity of graph structures and the integration of multimodal information have not been effectively addressed. Research on cross-modal graph structures [144] suggests that the design of heterogeneous graph GNNs and cross-modal GNNs will be a crucial direction for future research, as these methods can better handle data from different modalities, enhancing GNNs’ capability in cross-modal understanding and multimodal fusion.

The lack of interpretability significantly hinders the deployment of GNNs in high-risk domains such as medical diagnosis. Although GNNExplainer [145] provides local explanations for model behavior, its global interpretability is limited by the topological heterogeneity of complex graph structures. To address this challenge, Zhang et al. [146] innovatively combined medical knowledge graphs with GNNs, using symbolic clinical rule constraints to guide the message-passing process. This approach achieved a traceability accuracy of 92.3% in breast cancer pathology image classification tasks, providing an important practical basis for the collaborative paradigm of connectionism and symbolism. In terms of model credibility, uncertainty management in open environments has become a key bottleneck. Recent research [147] suggests that building trustworthy GNNs requires multi-dimensional collaborative optimization to improve robustness, defend against topological perturbation attacks, perform probabilistic calibration to quantify predictive uncertainty, and ensure privacy protection through differential privacy to secure sensitive medical data. The system framework developed by Wang et al. [148] partially validated this, with the GOODAT system controlling the misjudgment rate in autonomous driving scenarios to below 6.3%. However, the computational latency (averaging 83 ms/frame) still fails to meet real-time requirements.

Solving these challenges requires interdisciplinary collaborative innovation. On one hand, hardware-aware algorithm designs (such as GPU-accelerated graph partitioning [149]) can break computational bottlenecks. On the other hand, cognitive science-inspired explanation frameworks (such as visual concept extraction [150]) can enhance model transparency. Of particular interest is the fusion architecture of large language models with GNNs (such as GraphLLM [151]), which is opening new paradigms for multimodal reasoning. This could reshape the approach to solving computer vision tasks. With advancements in algorithms, hardware, and cross-domain integration, GNNs are expected to show broader application potential in the field of computer vision, driving further technological innovation and industry development.

5. Conclusions

This paper reviews the latest research progress of graph neural networks (GNNs) in the field of computer vision and provides an in-depth analysis of their broad application potential in image processing, video analysis, and multimodal data fusion. GNNs, with their advantages in handling non-Euclidean data structures, can effectively capture complex spatial relationships and semantic dependencies in images, particularly excelling in tasks such as object recognition, object detection, image segmentation, and video action recognition. By introducing the message passing mechanism, GNNs can efficiently transfer information between nodes in the graph, thus enabling a more comprehensive understanding and analysis of visual data.

However, despite the significant progress GNNs have made in the field of computer vision, they still face several challenges. Firstly, computational complexity and scalability issues remain constraints on their application in large-scale data processing, especially when dealing with high-dimensional image data and sequential video data, where these issues are particularly prominent. Secondly, the interpretability of GNN models has not yet been fully addressed, particularly in high-risk fields such as medical diagnostics, where the lack of transparent model behavior explanations limits their practical application. Furthermore, the effective fusion of heterogeneous graphs and multimodal data is also a major challenge in current research, and the effective integration of data from different modalities still requires further theoretical innovation and technical breakthroughs.

Future research should focus on improving the computational efficiency of GNN models, especially when processing large-scale datasets, and explore more efficient graph neural network architectures. To address the computational complexity issue, researchers could focus on optimizing the message passing mechanism, exploring low-complexity graph convolution methods, and incorporating sparsification techniques to reduce redundant computations. At the same time, enhancing the interpretability and robustness of GNN models remains a core challenge. In high-risk fields such as medical diagnostics and autonomous driving, interpretability issues limit the practical deployment of GNNs. Therefore, future research needs to enhance model transparency, develop interpretability frameworks, and ensure model stability and reliability in various environments. With the advent of new algorithms, advances in hardware acceleration, and cross-domain integration, GNNs are expected to play a greater role in practical applications in computer vision and other fields, driving continuous technological innovation and industry development.