Urban Informal Settlement Classification via Cross-Scale Hierarchical Perception Fusion Network Using Remote Sensing and Street View Images

Urban informal settlements (UISs), characterized by self-organized housing, a high population density, inadequate infrastructure, and insecure land tenure, constitute a critical, yet underexplored, aspect of contemporary urbanization. They necessitate scholarly scrutiny to tackle pressing challenges pertaining to equity, sustainability, and urban governance. The automated, accurate, and rapid extraction of UISs is of paramount importance for sustainable urban development. Despite its significance, this process encounters substantial obstacles. Firstly, from a remote sensing standpoint, informal settlements are typically characterized by a low elevation and a high density, giving rise to intricate spatial relationships. Secondly, the remote sensing observational features of these areas are often indistinct due to variations in shooting angles and imaging environments. Prior studies in remote sensing and geospatial data analysis have often overlooked the cross-modal interactions of features, as well as the progressive information encoded in the intrinsic hierarchies of each modality. We introduced a spatial network to solve this problem by combining panoramic and coarse-to-fine asymptotic perspectives, using remote sensing images and urban street view images to support a hierarchical analysis through fusion. Specifically, we utilized a multi-linear pooling technique and then established coarse-to-fine-grained and panoramic viewpoint details within an integrated structure, known as the panoramic fusion network (PanFusion-Net). Comprehensive testing was performed on a self-constructed WuhanUIS dataset as well as two open-source datasets, ChinaUIS and $S^{2}UV$. The experimental results confirmed that the performance of the introduced PanFusion-Net exceeded all comparative models across all of the above datasets.