概述

      本项目为国家自然科学基金联合重点项目子模块,主要开展面向社交多媒体大数据的情绪感知与态势推演。特别地,当两岸热点事件发生后,在两岸彼此阻断的社交网络上各自发生发展,各自用户群体对该事件的观点倾向各有不同。随着时间的流逝,用户群的舆情也随之发生变化。随着智能手机等便携式移动设备的普及,以及微信等新媒体的出现,两岸事件的舆情传播呈现与以往不同的特征:文本、图像、视频和语音等多种媒体信息混杂。由于舆情的传播格局和生成演变机制的变化,传统的文本舆情分析和态势推演方法已难以应对两岸热点事件的分析与监管:首先,传统舆情分析仅基于文本内容,无法处理目前泛滥的多媒体多模态数据;其次,微信微博的短文本特性决定了文本分析技术无法精确捕捉文档舆情;再次,两岸社交网络用户由于繁简体字使用习惯偏差、用户习惯偏差、以及约定俗成的区别,往往对同一个热点事件产生了文本各异的描述和评论。

      本技术路线的整体思路如上图所示。首先,采取基于超图学习的跨模态的社交媒体舆情分类;随后,采用基于基于因果挖掘和基于模式分析的技术,从海量热点事件轨迹中预测新的两岸热点事件的潜在爆发趋势;最后,采用关联性分析技术和结构话题模型,以挖掘两岸针对同一热点事件的核心的不对称信息。这一部分的输入是两岸热点事件检测与跟踪结果,输出是针对每个热点事件的舆情、爆发趋势、和关键的不对称信息,以便进行平台可视化展示。

多媒体舆情分析系统Demo

代表论文

  • Rongrong Ji, Fuhai Chen, Liujuan Cao*, Yue Gao*.
    Cross-modality microblog sentiment prediction via bi-layer multimodal hypergraph learning. [pdf] [code] [bibtex]
    IEEE Transactions on Multimedia (TMM), 2018.
    
    							 
    							 @article{ji2018cross,
      title={Cross-modality microblog sentiment prediction via bi-layer multimodal hypergraph learning},
      author={Ji, Rongrong and Chen, Fuhai and Cao, Liujuan and Gao, Yue},
      journal={IEEE Transactions on Multimedia},
      volume={21},
      number={4},
      pages={1062--1075},
      year={2018},
      publisher={IEEE}
    }
    							 
    																	 
    							 
    																	 
  • Fuhai Chen, Rongrong Ji*, Jinsong Su, Donglin Cao and Yue Gao.
    Predicting Microblog Sentiments via Weakly Supervised Multimodal Deep Learning. [pdf] [project] [bibtex]
    IEEE Transactions on Multimedia (TMM), 2017.
    
    															
    															@article{chen2017predicting,
      title={Predicting microblog sentiments via weakly supervised multimodal deep learning},
      author={Chen, Fuhai and Ji, Rongrong and Su, Jinsong and Cao, Donglin and Gao, Yue},
      journal={IEEE Transactions on Multimedia},
      volume={20},
      number={4},
      pages={997--1007},
      year={2017},
      publisher={IEEE}
    }
    															
    															
    															
    															
  • Damian Borth, Rongrong Ji, Tao Chen, Thomas Breuel, Shih-Fu Chang.
    Large-scale visual sentiment ontology and detectors using adjective noun pairs. [pdf] [project] [bibtex]
    ACM international conference on Multimedia (ACM MM), 2013.
    
    		 
    		 @inproceedings{borth2013large,
      title={Large-scale visual sentiment ontology and detectors using adjective noun pairs},
      author={Borth, Damian and Ji, Rongrong and Chen, Tao and Breuel, Thomas and Chang, Shih-Fu},
      booktitle={Proceedings of the 21st ACM international conference on Multimedia},
      pages={223--232},
      year={2013},
      organization={ACM}
    }
    		 
    												 
    		 
    												 
  • Fuhai Chen, Yue Gao, Donglin Cao, Rongrong Ji*.
    Multimodal hypergraph learning for microblog sentiment prediction. [pdf] [code] [bibtex]
    IEEE International Conference on Multimedia and Expo (ICME), 2015.
    
    				 
    				 @inproceedings{chen2015multimodal,
      title={Multimodal hypergraph learning for microblog sentiment prediction},
      author={Chen, Fuhai and Gao, Yue and Cao, Donglin and Ji, Rongrong},
      booktitle={2015 IEEE International Conference on Multimedia and Expo (ICME)},
      pages={1--6},
      year={2015},
      organization={IEEE}
    }
    				 
    														 
    				 
    														 
  • Chao Chen, Fuhai Chen, Donglin Cao, Rongrong Ji*.
    A cross-media sentiment analytics platform for microblog. [pdf] [demo] [bibtex]
    ACM international conference on Multimedia (ACM MM), 2015.
    
    						 
    						 @inproceedings{chen2015cross,
      title={A cross-media sentiment analytics platform for microblog},
      author={Chen, Chao and Chen, Fuhai and Cao, Donglin and Ji, Rongrong},
      booktitle={Proceedings of the 23rd ACM international conference on Multimedia},
      pages={767--769},
      year={2015},
      organization={ACM}
    }
    						 
    																 
    						 
    																 
  • Damian Borth, Tao Chen, Rongrong Ji, Shih-Fu Chang.
    Sentibank: large-scale ontology and classifiers for detecting sentiment and emotions in visual content. [pdf] [project] [bibtex]
    ACM international conference on Multimedia (ACM MM), 2013.
    
    								 
    								 @inproceedings{borth2013sentibank,
      title={Sentibank: large-scale ontology and classifiers for detecting sentiment and emotions in visual content},
      author={Borth, Damian and Chen, Tao and Ji, Rongrong and Chang, Shih-Fu},
      booktitle={Proceedings of the 21st ACM international conference on Multimedia},
      pages={459--460},
      year={2013},
      organization={ACM}
    }
    								 
    																		 
    								 
    																		 
  • Lingxiao Li, Donglin Cao, Shaozi Li, Rongrong Ji*.
    Sentiment analysis of Chinese micro-blog based on multi-modal correlation model. [pdf] [bibtex]
    IEEE International Conference on Image Processing (ICIP), 2015.
    
    										 
    										 @inproceedings{li2015sentiment,
      title={Sentiment analysis of Chinese micro-blog based on multi-modal correlation model},
      author={Li, Lingxiao and Cao, Donglin and Li, Shaozi and Ji, Rongrong},
      booktitle={2015 IEEE International Conference on Image Processing (ICIP)},
      pages={4798--4802},
      year={2015},
      organization={IEEE}
    }
    										 
    																				 
    										 
    																				 
  • Jin Ye, Xiaojiang Peng, Yu Qiao, Hao Xing, Junli Li, Rongrong Ji.
    Visual-Textual Sentiment Analysis in Product Reviews. [pdf] [bibtex]
    IEEE International Conference on Image Processing (ICIP), 2019.
    
    											
    											@inproceedings{ye2019visual,
      title={Visual-Textual Sentiment Analysis in Product Reviews},
      author={Ye, Jin and Peng, Xiaojiang and Qiao, Yu and Xing, Hao and Li, Junli and Ji, Rongrong},
      booktitle={2019 IEEE International Conference on Image Processing (ICIP)},
      pages={869--873},
      year={2019},
      organization={IEEE}
    }