Multimodal Sentiment Analysis for Interactive Fusion of Dual Perspectives Under Cross-modalInconsistent Perception

In social media,people’s comments usually describe a certain sentiment region in the corresponding image,and there is correspondence information between image and text.Most previous multimodal sentiment analysis methods only explore the interactions between images and text from a single perspective,capturing the correspondence between image regions and text words,leading to results that are not optimal.In addition,data on social media is strongly personal and subjective,and the sentiment in the data is multidimensional and complex,which leads to the emergence of data with weak image and text sentiment consistency.To address the above two problems,a multimodal sentiment analysis model with interactive fusion of two perspectives under cross-modal inconsistency perception is proposed.On the one hand,cross-modal interaction of graphic and textual features from both global and local perspectives provides a more comprehensive and accurate sentiment analysis,which improves the perfor-mance and application of the model.On the other hand,the inconsistency scores of the graphical features are calculated to representthe degree of graphical inconsistency,as a way to dynamically regulate the weights of the unimodal and multimodal representations in the final sentiment features,thus improving the robustness of the model.Extensive experiments are conducted on two public datasets,MVSA-Single and MVSA-Multiple,and the results demonstrate the validity and superiority of the proposed multimodal sentiment analysis model compared to the existing baseline models,with F1 values increasing by 0.59 persentage points and 0.39 persentage points,respectively.