Abstract: Due to the characteristics of small inter-class differences and large intra-class distinctions in fine-grained image, many studies have utilized Vision Transformer to mine critical region features to improve the accuracy of fine-grained image classification. However, there still exists two major problems: firstly, background regions are also considered when the network mines critical classification cues, bringing additional noise interference to the model. Secondly, there is a lack of spatial connection between the local embeddings feature of the input images, and the model lacks the ability of object structure cognition, leading to inaccurate extracted category features. To address these problem, this paper proposes two modules: adaptive spatial mutual attention module and feature pair integrated discrimination module, which first learns the mutual attention weights of different embedding layers to select better discriminative regions through mutual attention spatial adaptive module, and adaptively learns the neighbor relationship of different regions through graph convolutional network. Then the feature pair integration discrimination module is utilized to treat the cue interactions between image pairs and reduce the confusion between fine-grained images. The final prediction results are derived under the token feature enhancement strategy. The proposed method achieves an accuracy of 92.5%, 93.3% and 91.8% on three benchmark datasets, namely, CUB-200-2011, Stanford Dogs and NABirds, which are better than many other existing state-of-the-art methods.

Key words: fine-grained image classification, adaptive spatial mutual attention, feature pair integration discrimination, graph convolutional network, token feature enhancement