Multi-Task Directional Field Learning for Geometry-Aware Building Extraction and Simplified Vector Reconstruction in High-Resolution Remote Sensing

This paper addresses the problem that high pixel-level segmentation accuracy does not necessarily lead to geometrically compact building boundaries in vectorized outputs. A multi-task directional field learning framework is proposed based on U-Net with a ResNet-50 encoder. The framework introduces directional field supervision and a mask-field alignment loss to jointly optimize building region prediction and local boundary orientation consistency. In addition, a mild topological simplification procedure with a fixed small tolerance is applied to reduce residual staircase-like artifacts during vectorization. Experiments on the WHU building dataset at 0.2 m and 0.3 m spatial resolutions show that the proposed framework produces compact vector representations while maintaining high overlap relative to the raster reference annotations. In the 0.2 m setting, directional field learning improves Boundary IoU compared with the Baseline U-Net, whereas the complete pipeline slightly reduces Mask IoU and F1-score due to the additional simplification step. In the 0.3 m setting, the complete method does not consistently outperform several baselines in conventional pixel-level metrics, but it shows a favorable trade-off between polygon compactness and vector overlap under raster-reference evaluation. These results indicate that the proposed method is more suitable for geometry-aware vector reconstruction and vector simplification than for maximizing general semantic segmentation accuracy. In particular, the average number of polygon vertices is substantially reduced while Vector IoU remains approximately 90–92%. To further address the limitation of evaluating only on the WHU dataset, an additional in-domain validation experiment was conducted on the JAX dataset, which contains more complex building appearances and scene variations. The results show that the proposed Directional Field + Mild DP pipeline consistently reduces polygon complexity on the JAX dataset while maintaining competitive vector overlap. The central objective of the proposed framework is not only to improve mask-level building extraction, but also to enhance boundary-oriented vector reconstruction by learning local boundary-direction consistency and reducing raster-induced polygonal redundancy.