Journal of Guangxi Normal University(Natural Science Edition) ›› 2025, Vol. 43 ›› Issue (2): 121-132.doi: 10.16088/j.issn.1001-6600.2024032002

• Intelligence Information Processing • Previous Articles     Next Articles

Hierarchical-scale Interaction-based U-Net for Remote Sensing Image Building Extraction

YU Kuai1,2, SONG Baogui1,2, SHAO Pan1,2*, YU Ao1,2   

  1. 1. Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering (China Three Gorges University), Yichang Hubei 443002, China;
    2. College of Computer and Information, China Three Gorges University, Yichang Hubei 443002, China
  • Received:2024-03-20 Revised:2024-06-20 Online:2025-03-05 Published:2025-04-02

PDF (PC)

291

Abstract: Aiming at the problem that U-Net and its improved network have insufficient feature characterisation ability due to ignoring the interactions between multi-level features in jump links, a building extraction method based on hierarchical scale interactions is proposed for U-Net remote sensing images. Firstly, a hierarchical scale interaction module is designed in the jump link of U-Net network to achieve the interaction enhancement of multilevel features and improve the characterisation ability of features. Then a multi-scale feature extraction module is proposed by improving the null-space pyramid pooling module and applying it to the highest level features to enhance the ability of the network to extract multi-scale features. Finally, self-calibrating convolution is introduced into the decoding process to promote better fusion of shallow and deep features. The method of this paper is compared with six remote sensing image building extraction methods on two publicly available building extraction datasets, WHU and Inria. The experimental results show that the IoU of the proposed method is 91.26% and 79.23%, respectively, which are better than the comparison methods.

Key words: remote sensing imagery, building extraction, U-Net, hierarchical-scale interaction, multi-scale, attention mechanism

CLC Number:  TP751
[1] 杨俊, 任嘉义, 于文博. 局地气候分区视角下城市气候与人居环境研究进展[J]. 生态学报, 2024, 44(11): 4489-4506. DOI: 10.20103/j.stxb.202310222297.
[2] 慈慧, 郭朋辉, 秦勇, 等. 基于多时相遥感影像的滨海湿地监测方法研究[J]. 广西师范大学学报(自然科学版), 2013, 31(3): 144-151. DOI: 10.16088/j.issn.1001-6600.2013.03.025.
[3] LIASIS G, STAVROU S. Building extraction in satellite images using active contours and colour features[J]. International Journal of Remote Sensing, 2016, 37(5): 1127-1153. DOI: 10.1080/01431161.2016.1148283.
[4] 施仲添, 沈正伟, 杨四海. 基于前景感知的遥感影像建筑物提取方法[J]. 测绘通报, 2023(2): 134-138. DOI: 10.13474/j.cnki.11-2246.2023.0053.
[5] 王明常, 朱春宇, 陈学业, 等. 基于FPN Res-Unet的高分辨率遥感影像建筑物变化检测[J]. 吉林大学学报(地球科学版), 2021, 51(1): 296-306. DOI: 10.13278/j.cnki.jjuese.20190321.
[6] 王俊, 秦其明, 叶昕, 等. 高分辨率光学遥感图像建筑物提取研究进展[J]. 遥感技术与应用, 2016, 31(4): 653-662. DOI: 10.11873/j.issn.1004-0323.2016.4.0653.
[7] LI Q Y, MOU L C, SUN Y, et al. A review of building extraction from remote sensing imagery: geometrical structures and semantic attributes[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 4702315. DOI: 10.1109/TGRS.2024.3369723.
[8] FANG F, XU R, LI S W, et al. Semisupervised building instance extraction from high-resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5619212. DOI: 10.1109/TGRS.2023.3309918.
[9] CHEN J, HE P E, ZHU J R, et al. Memory-contrastive unsupervised domain adaptation for building extraction of high-resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5605615. DOI: 10.1109/TGRS.2023.3262576.
[10] 宋冠武, 陈知明, 李建军. 基于ResNet-50的级联注意力遥感图像分类[J]. 广西师范大学学报(自然科学版), 2023, 41(6): 80-91. DOI: 10.16088/j.issn.1001-6600.2023031702.
[11] JIN X Y, DAVIS C H. Automated building extraction from high-resolution satellite imagery in urban areas using structural, contextual, and spectral information[J]. EURASIP Journal on Applied Signal Processing, 2005, 2005: 2196-2206. DOI: 10.1155/ASP.2005.2196.
[12] 张浩, 赵云胜, 陈冠宇, 等. 基于支持向量机的遥感图像建筑物识别与分类方法研究[J]. 地质科技情报, 2016, 35(6): 194-199.
[13] 马为駽. 基于形态学的高分辨率遥感影像建筑物自动提取[D]. 武汉: 武汉大学, 2022. DOI: 10.27379/d.cnki. gwhdu.2019.001850.
[14] 顾炼, 许诗起, 竺乐庆. 基于FlowS-Unet的遥感图像建筑物变化检测[J]. 自动化学报, 2020, 46(6): 1291-1300. DOI: 10.16383/j.aas.c180122.
[15] 唐晴, 徐胜华, 高贤君, 等. 融合矢量数据的高分遥感影像建筑物轮廓优化方法[J]. 测绘科学, 2023, 48(12): 143-152. DOI: 10.16251/j.cnki.1009-2307.2023.12.015.
[16] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2015: 3431-3440. DOI: 10.1109/CVPR.2015.7298965.
[17] RONNEBERGER O, FISCHER P, BROX T. U-Net:convolutional networks for biomedical image segmentation[C] // Medical Image Computing and Computer-Assisted Intervention: MICCAI 2015. Cham: Springer International Publishing AG Switzerland, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
[18] BADRINARAYANAN V, KENDALL A, CIPOLLA R.SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(12): 2481-2495. DOI: 10.1109/TPAMI.2016.2644615.
[19] CHEN L C, PAPANDREOU G, KOKKINOS I, et al.DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. DOI: 10.1109/TPAMI.2017.2699184.
[20] RAN S H, GAO X J, YANG Y W, et al. Building multi-feature fusion refined network for building extraction from high-resolution remote sensing images[J]. Remote Sensing, 2021, 13(14): 2794. DOI: 10.3390/rs13142794.
[21] 王一琛, 刘慧, 王海涛, 等. 面向遥感图像的建筑物轻量化语义分割方法[J]. 计算机工程与设计, 2022, 43(9): 2646-2653. DOI: 10.16208/j.issn1000-7024.2022.09.032.
[22] 林禹, 赵泉华, 沈昭宇, 等. 改进SegNet与迁移学习的遥感建筑物分割方法[J]. 测绘科学, 2022, 47(6): 78-89. DOI: 10.16251/j.cnki.1009-2307.2022.06.011.
[23] 王振庆, 周艺, 王世新, 等. IEU-Net高分辨率遥感影像房屋建筑物提取[J]. 遥感学报, 2021, 25(11): 2245-2254. DOI: 10.11834/jrs.20210042.
[24] KANG J, FERNANDEZ-BELTRAN R, SUN X, et al. Deep learning-based building footprint extraction with missing annotations[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 3002805. DOI: 10.1109/LGRS.2021.3072589.
[25] DIXIT M, CHAURASIA K, MISHRA K V. Dilated-ResUnet: a novel deep learning architecture for building extraction from medium resolution multi-spectral satellite imagery[J]. Expert Systems with Applications, 2021, 184: 115530. DOI: 10.1016/j.eswa.2021.115530.
[26] QIU W Y, GU L J, GAO F, et al. Building extraction from very high-resolution remote sensing images using refine-unet[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 6002905. DOI: 10.1109/LGRS.2023.3243609.
[27] CAI J H, CHEN Y M. MHA-Net: multipath hybrid attention network for building footprint extraction from high-resolution remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021,14: 5807-5817. DOI: 10.1109/JSTARS.2021.3084805.
[28] WANG Z Q, ZHOU Y, WANG F T, et al. Fine-grained building extraction with multispectral remote sensing imagery using the deep model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 4706013. DOI: 10.1109/TGRS.2023.3327370.
[29] GUO H N, SU X, WU C, et al. Decoupling semantic and edge representations for building footprint extraction from remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5613116. DOI: 10.1109/TGRS.2023.3287298.
[30] LIU J J, HOU Q B, CHENG M M, et al. Improving convolutional networks with self-calibrated convolutions[C] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2020: 10093-10102. DOI: 10.1109/CVPR42600.2020.01011.
[31] LI X T, YOU A S, ZHU Z, et al. Semantic flow for fast and accurate scene parsing[C] // Computer Vision-ECCV 2020: LNCS Volume 12346. Cham: Springer Nature Switzerland AG, 2020: 775-793. DOI: 10.1007/978-3-030-58452-8_45.
[32] ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2017: 6230-6239. DOI: 10.1109/CVPR.2017. 660.
[33] 朱节中, 陈永, 柯福阳, 等. 基于Siam-UNet++的高分辨率遥感影像建筑物变化检测[J]. 计算机应用研究, 2021, 38(11): 3460-3465. DOI: 10.19734/j.issn.1001-3695.2021.01.0070.
[34] JI S P, WEI S Q, LU M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574-586. DOI: 10.1109/TGRS.2018.2858817.
[35] MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark[C] // 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Piscataway, NJ: IEEE, 2017: 3226-3229. DOI: 10.1109/IGARSS.2017.8127684.
[36] PASZKE A, CHAURASIA A, KIM S, et al. ENet: a deep neural network architecture for realtime semantic segmentation[EB/OL]. (2016-06-07)[2024-03-20]. https://arxiv.org/abs/1606.02147. DOI: 10.48550/arXiv.1606.02147.
[37] ZHANG H, ZHENG X C, ZHENG N S, et al. A multiscale and multipath network with boundary enhancement for building footprint extraction from remotely sensed imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022,15: 8856-8869. DOI: 10.1109/JSTARS.2022.3214485.
[38] CHEN J, JIANG Y X, LUO L B, et al. ASF-Net: adaptive screening feature network for building footprint extraction from remote-sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 4706413. DOI: 10.1109/TGRS.2022.3165204.
[1] GUO Xiangyu, SHI Tianyi, CHEN Yannan, NAN Xinyuan, CAI Xin. Research on Foreign Object Detection in Railway Overhead Contact System Based on YOLO-CDBW Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(2): 56-69.
[2] SU Chunhai, XIA Haiying. Facial Expression Recognition Based on Noise-Resistant Dual Constraint Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(2): 70-82.
[3] LIU Yuna, MA Shuangbao. Fabric Defect Detection Based on Improved Lightweight YOLOv8n [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(2): 83-94.
[4] DAI Linhua, LI Yuansong, SHI Rui, HE Zhongliang, LI Lei. HSED-YOLO: A Lightweight Model for Detecting Surface Defects in Strip Steel [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(2): 95-106.
[5] LU Jiahui, CHEN Qingfeng, WANG Wenguang, YU Qian, HE Naixu, HAN Zongzhao. Multi-scale Attention Learning for Abdomen Multi-organ Image Segmentation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 138-148.
[6] DU Shuaiwen, JIN Ting. A Deep Hybrid Recommendation Algorithm Based on User Behavior Characteristics [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(5): 91-100.
[7] TIAN Sheng, HU Xiao. Vehicle Trajectory Prediction Based on Transformer Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 47-58.
[8] XIAO Yuting, LÜ Xiaoqi, GU Yu, LIU Chuanqiang. Classification of Diabetic Retinopathy Based on Split Residual Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 91-101.
[9] XI Lingfei, Yilihamu Yaermaimaiti, LIU Yajie. Surface Defect Detection Method for Aluminum Profile Based on Improved YOLOv5 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 111-119.
[10] SONG Guanwu, CHEN Zhiming, LI Jianjun. Remote Sensing Image Classification with Cascade Attention Based on ResNet-50 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 80-91.
[11] WU Zhengqing, CAO Hui, LIU Baokai. Chinese Fake Review Detection Based on Attention Convolutional Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 26-36.
[12] HUANG Yeqi, WANG Mingwei, YAN Rui, LEI Tao. Surface Quality Detection of Diamond Wire Based on Improved YOLOv5 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(4): 123-134.
[13] DENG Xizhen, JIANG Ming, CEN Mingcan, LUO Yuling. Ransomware Classification Based on Entropy Image Static Analysis Technology [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(3): 91-104.
[14] WANG Li’e, WANG Yihui, LI Xianxian. A Multi-source Data Fusion and Privacy Protection Method of POI Recommendation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(1): 87-101.
[15] WANG Yuhang, ZHANG Canlong, LI Zhixin, WANG Zhiwen. Image Captioning According to User’s Intention and Style [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(4): 91-103.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Guangxi Normal University(Natural Science Edition), All Rights Reserved.
Tel: 0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
Powered by Beijing Magtech Co. Ltd