Next Article in Journal
Two-Dimensional Simulation of Barley Growth and Yield Using a Model Integrated with Remote-Controlled Aerial Imagery
Next Article in Special Issue
Data Fusion Using a Multi-Sensor Sparse-Based Clustering Algorithm
Previous Article in Journal
Exploring Appropriate Preprocessing Techniques for Hyperspectral Soil Organic Matter Content Estimation in Black Soil Area
Previous Article in Special Issue
A Multi-Sensor Fusion Framework Based on Coupled Residual Convolutional Neural Networks
Article

A Hybrid Attention-Aware Fusion Network (HAFNet) for Building Extraction from High-Resolution Imagery and LiDAR Data

by 1,2, 1,2,*, 1,2, 1,2, 3, 4 and 1,2
1
Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology, Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources, School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China
2
Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing 210023, China
3
School of Geography, Geomatics and Planning, Jiangsu Normal University, Xuzhou 221116, China
4
School of Earth Sciences and Engineering, Hohai University, Nanjing 211100, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2020, 12(22), 3764; https://doi.org/10.3390/rs12223764
Received: 24 September 2020 / Revised: 12 November 2020 / Accepted: 13 November 2020 / Published: 16 November 2020
(This article belongs to the Special Issue Advanced Multisensor Image Analysis Techniques for Land-Cover Mapping)
Automated extraction of buildings from earth observation (EO) data has long been a fundamental but challenging research topic. Combining data from different modalities (e.g., high-resolution imagery (HRI) and light detection and ranging (LiDAR) data) has shown great potential in building extraction. Recent studies have examined the role that deep learning (DL) could play in both multimodal data fusion and urban object extraction. However, DL-based multimodal fusion networks may encounter the following limitations: (1) the individual modal and cross-modal features, which we consider both useful and important for final prediction, cannot be sufficiently learned and utilized and (2) the multimodal features are fused by a simple summation or concatenation, which appears ambiguous in selecting cross-modal complementary information. In this paper, we address these two limitations by proposing a hybrid attention-aware fusion network (HAFNet) for building extraction. It consists of RGB-specific, digital surface model (DSM)-specific, and cross-modal streams to sufficiently learn and utilize both individual modal and cross-modal features. Furthermore, an attention-aware multimodal fusion block (Att-MFBlock) was introduced to overcome the fusion problem by adaptively selecting and combining complementary features from each modality. Extensive experiments conducted on two publicly available datasets demonstrated the effectiveness of the proposed HAFNet for building extraction. View Full-Text
Keywords: building extraction; high-resolution imagery (HRI); light detection and ranging (LiDAR); multimodal data fusion; deep learning; attention mechanism building extraction; high-resolution imagery (HRI); light detection and ranging (LiDAR); multimodal data fusion; deep learning; attention mechanism
Show Figures

Graphical abstract

MDPI and ACS Style

Zhang, P.; Du, P.; Lin, C.; Wang, X.; Li, E.; Xue, Z.; Bai, X. A Hybrid Attention-Aware Fusion Network (HAFNet) for Building Extraction from High-Resolution Imagery and LiDAR Data. Remote Sens. 2020, 12, 3764. https://doi.org/10.3390/rs12223764

AMA Style

Zhang P, Du P, Lin C, Wang X, Li E, Xue Z, Bai X. A Hybrid Attention-Aware Fusion Network (HAFNet) for Building Extraction from High-Resolution Imagery and LiDAR Data. Remote Sensing. 2020; 12(22):3764. https://doi.org/10.3390/rs12223764

Chicago/Turabian Style

Zhang, Peng, Peijun Du, Cong Lin, Xin Wang, Erzhu Li, Zhaohui Xue, and Xuyu Bai. 2020. "A Hybrid Attention-Aware Fusion Network (HAFNet) for Building Extraction from High-Resolution Imagery and LiDAR Data" Remote Sensing 12, no. 22: 3764. https://doi.org/10.3390/rs12223764

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop