Abstract: Knowledge distillation can improve the generalization ability of neural networks and solve the problem of insufficient labeled data when classifying remote sensing image scenes. And the high similarity between classes existing in remote sensing images can lead to the loss of intermediate knowledge features. To address this problem, a feature extraction method (SDCASA) based on the self-distillation cascaded attention mechanism is proposed. Firstly, a teacher and student network with shared weights is constructed; then the cascaded attention module is used to refine the features extracted by the deep teacher network while retaining the intermediate edge information filtered by the shallow neural network. Secondly, the refined features are used to guide the student network to learn. Finally, a linear classifier is trained downstream to complete feature classification. The classification accuracies of 85.17%, 90.10%, 91.13% and 85.50%, 92.13%, 91.17% are achieved on three publicly available datasets AID, MLRSNet, and EuroSAT using 20% and 50% of the samples trained, respectively. This method can effectively improve the classification accuracy of remote sensing image scenes and outperforms the mainstream self-supervised image classification methods SimSiam, SwAV, MoCov2, Deepcluster, and has good application value.

Key words: self-distillation, attention mechanism, remote sensing images, self-supervised learning, image classification