Scene graphs can enhance the understanding capability of intelligent ships in navigation scenes. However, the complex entity relationships and the presence of significant noise in contextual information within navigation scenes pose challenges for na...
This paper advances contextual image understanding within perspective-aware Ai (PAi), an emerging paradigm in human–computer interaction that enables users to perceive and interact through each other’s perspectives. While PAi relies on mu...
Images captured by vision sensors provide visual evidence for scene understanding, including object appearances, pixel-level regions, and spatial relations among entities. Panoptic Scene Graph Generation (PSG) constructs structured scene representati...
The generation of images from scene graphs is an important area in computer vision, where structured object relationships are used to create detailed visual representations. While recent methods, such as generative adversarial networks (GANs), transf...
To semantically understand remote sensing images, it is not only necessary to detect the objects in them but also to recognize the semantic relationships between the instances. Scene graph generation aims to represent the image as a semantic structur...
Scene graph generation is the basis of various computer vision applications, including image retrieval, visual question answering, and image captioning. Previous studies have relied on visual features or incorporated auxiliary information to predict...
An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-Balanced Re-Weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating...
Panoptic scene graph generation (PSG) aims to simultaneously segment both foreground objects and background regions while predicting object relations for fine-grained scene modeling. Despite significant progress in panoptic scene understanding, curre...
Visually impaired individuals face significant challenges in autonomous mobility and situational awareness. Most existing assistive technologies address isolated tasks, such as object recognition or text reading, while failing to capture broader envi...
Conceptual representations of images involving descriptions of entities and their relations are often represented using scene graphs. Such scene graphs can express relational concepts by using sets of triplets
3D scene graphs connect spatial perception with high-level language reasoning, but unverified false-positive groundings can impair downstream subtask generation. We propose a feed-forward, spatially constrained framework that grounds task-relevant ob...
Surgical scene understanding is a key barrier for situation-aware robotic surgeries and the associated surgical training. With the presence of domain shifts and the inclusion of new instruments and tissues, learning domain generalization (DG) plays a...
Video scene graph generation (ViDSGG), the creation of video scene graphs that helps in deeper and better visual scene understanding, is a challenging task. Segment-based and sliding-window based methods have been proposed to perform this task. Howev...
Over the last several years, in parallel with the general global advancement in mobile technology and a rise in social media network content consumption, multimedia content production and reproduction has increased exponentially. Therefore, enabled b...
Group-activity scene graph (GASG) generation is a challenging task in computer vision, aiming to anticipate and describe relationships between subjects and objects in video sequences. Traditional video scene graph generation (VidSGG) methods focus on...
Street-view imagery supports fine-grained urban monitoring, but most street-scene change detection methods are pixel-centric or object-centric and cannot explicitly capture the evolution of inter-entity spatial relations needed for interpretable task...
A deep understanding of our visual world is more than an isolated perception on a series of objects, and the relationships between them also contain rich semantic information. Especially for those satellite remote sensing images, the span is so large...
Text-based image retrieval is one of the most common approaches for searching images acquired from vision sensors such as cameras. However, this method suffers from limitations in retrieval accuracy, particularly when the query contains limited infor...
We introduce a novel Maximum Entropy (MaxEnt) framework that can generate 3D scenes by incorporating objects’ relevancy, hierarchical and contextual constraints in a unified model. This model is formulated by a Gibbs distribution, under the MaxEnt fr...
Scientific studies have demonstrated how certain insect species can be used as bioindicators and reverse environmental degradation through their behavior and organization. Studying these species involves capturing and extracting hundreds of insects f...
Geometric similarity plays an important role in geographic information retrieval, map matching, and data updating. Many approaches have been developed to calculate the similarity between simple features. However, complex group objects are common in m...
Image scene graph is a semantic structural representation which can not only show what objects are in the image, but also infer the relationships and interactions among them. Despite the recent success in object detection using deep neural networks,...
Generalized stereo matching faces the radiation difference and small ground feature difference brought by different satellites and different time phases, while the texture-less and disparity discontinuity phenomenon seriously affects the corresponden...
Scene graph generation (SGG) provides a structured representation for visual understanding. However, most existing methods are designed to optimize global triplet recall rather than retrieve relation instances specified by a user query. In query-driv...
With the increasing demand for accurate multimodal data analysis in complex scenarios, existing models often struggle to effectively capture and fuse information across diverse modalities, especially when data include varying scales and levels of det...
As autonomous vehicles increasingly populate roads, robust testing is essential to ensure their safety and reliability. Due to the limitation that traditional testing methodologies (real-world and simulation testing) are difficult to cover a wide ran...
Machine learning (ML) technology is rapidly evolving, and the quality of ML systems is becoming an increasingly focal point of attention. Since the ML system is shaped by the dataset it learns from, its quality largely depends on the quality of the d...
In autonomous driving, retrieving a specific traffic scene in huge datasets is a significant challenge. Traditional scene retrieval methods struggle to cope with the semantic complexity and heterogeneity of traffic scenes and are unable to meet the v...
In scene parsing, the model is required to be able to process complex multi-modal data such as images and contexts in real scenes, and discover their implicit connections from objects existing in the scene. As a storage method that contains entity in...
As one of the fundamental tasks in remote sensing (RS) image understanding, multi-label remote sensing image scene classification (MLRSSC) is attracting increasing research interest. Human beings can easily perform MLRSSC by examining the visual elem...
Task-oriented scene data in big data and cloud environments of a smart city that must be time-critically processed are dynamic and associated with increasing complexities and heterogeneities. Existing hybrid tree-based external indexing methods are i...
The objective of cross-scene hyperspectral image (HSI) classification is to develop models capable of adapting to the “domain gap” that exists between different scenes, enabling accurate object classification in previously unseen scenes....
With the rapid development of remote sensing image data, the efficient retrieval of target images of interest has become an important issue in various applications including computer vision and remote sensing. This research addressed the low-accuracy...
Graph-based path planning commonly operates on an available topological representation, but robotic exploration may require searching for feasible paths while the environment graph is progressively revealed. This paper proposes MSG-RRT (Multi-Strateg...
The design of 3D scenes is of great significance, and one of the crucial areas is interior scene design. This study not only pertains to the living environment of individuals but also has applications in the design and development of virtual environm...
A rapidly expanding multimedia environment in recent years has led to an explosive increase in demand for multimodality that can communicate with humans in various ways. Even though the convergence of vision and language intelligence has shed light o...
Natural-language-driven robot navigation in complex indoor environments requires the joint capability of high-fidelity scene representation, structured semantic reasoning, and executable path planning. To address this challenge, this paper proposes H...
In deep neural network model training and prediction, due to the limitation of GPU memory and computing resources, massive image data must be cropped into limited-sized samples. Moreover, in order to improve the generalization ability of the model, t...
The recording of metaverse experiences supports various use cases in collaboration, VR training, and more. Such Metaverse Recordings can be created as multimedia and time series data during the 3D rendering process of the audio–video stream for...
With the development of artificial intelligence and deep learning technologies, image captioning has become an important research direction at the intersection of computer vision and natural language processing. The purpose of image captioning is to...
Spatial scenes, as fundamental units of geospatial cognition, encompass rich objects and spatial relationships, and their generation techniques hold significant application value in disaster simulation and emergency drills, delayed spatial reconstruc...
Hyperspectral images (HSIs) are inevitably degraded by mixed noise, which hampers downstream interpretation. Recent graph-signal-processing denoisers encode the spatial–spectral structure of an HSI through a product graph over superpixel bodies...
People’s interactions with each other form the social relations in society. Understanding human social relations in the public space is of great importance for supporting the public administrations. Recognizing social relations through visual data ca...
Semantic image synthesis (SIS) in remote sensing aims to generate high-fidelity satellite imagery from land use/land cover (LULC) labels, supporting applications such as map updating, data augmentation, and environmental monitoring. However, the exis...
This paper considers the interactive effects between the ego vehicle and other vehicles in a dynamic driving environment and proposes an autonomous vehicle lane-changing behavior decision-making and trajectory planning method based on graph convoluti...
LLM-CapGen is a lightweight framework based on large language models that generates video captions by combining diverse data sources at low computational cost. It captures spatial and temporal information from video frames to identify actions and mov...
A single receptive field limits the expression of multilevel receptive field features in point cloud registration, leading to the pseudo-matching of objects with similar geometric structures in low-overlap scenes, which causes a significant degradati...
Most graph-based networks utilize superpixel generation methods as a preprocessing step, considering superpixels as graph nodes. In the case of hyperspectral images having high variability in spectral features, considering an image region as a graph...
Referring remote sensing image segmentation (RRSIS) aims to segment a target instance from remote sensing imagery according to a natural-language expression. It provides a flexible way to retrieve and localize specific objects in remote sensing scene...
Geographic features in maps consist of a mixture of points, polylines, and polygons, generally including POIs, roads, buildings, and other geographic features. Due to the differing dimensionality of these various types of geographic data, traditional...
of 3