Abstract: Multimodal sentiment analysis is an important task in the field of natural language processing, and modality fusion is its core problem. Previous research has not distinguished the primary and secondary status of each modality in sentiment analysis, treating each modality equally and not properly recognizing the quality and performance gaps between different modalities. Existing research shows that textual modalities tend to dominate sentiment analysis, but non-textual modalities contain key feature information that is essential for identifying correct sentiment. Therefore, this paper proposes a modality fusion strategy that focuses on text modality. Through a codec network with an attention mechanism to distinguish the shared and private semantics between different modalities, the two semantic enhancements of non-text modalities relative to text modalities are used to complement text features, achieve a joint robust representation of multiple modalities, and ultimately achieve sentiment prediction. Experiments on the CMU-MOSI and CMU-MOSEI video sentiment analysis datasets show that the accuracy of this method reaches 87.3% and 86.2% respectively, outperforming many existing state-of-the-art methods.

Key words: sentiment analysis, modal fusion, attentional mechanisms, common semantics, private semantics, augmented complementation