VideoForest: Interactive Visual Summarization of Video Streams Based on Danmu Data

VideoForest is a visualization system designed to convert an input video stream augmented with danmu data into a visual summary of its content with highlights along the storyline under users’ supervision.

Emerging online video-sharing websites such as YouTube allow users to access a huge number of videos and generate feedback via reviews and/or live comments (a.k.a. danmu), making it possible to summarize videos based on media and user responses collectively. A video summary produced by existing techniques may not fully capture an audience’s perception of and reaction to the source video, and thus may be less reflective. In this paper, we introduce VideoForest, a visualization system designed to convert an input video augmented with danmu commentary data into a treelike visual summary of content highlights under user supervision. The proposed visualization design employs a forest metaphor. The overall summary of different video sessions is illustrated as scene trees on top of the session timeline ground, with the roots depicting the corresponding danmu messages. VideoForest can also generate a detailed synopsis of user-selected video segment(s) as a compact storyline in the form of circle packing. We evaluate our system via case studies with real video data based on experts’ feedback. The results suggest the power and potential of the system.

Forest-Metaphor Visual Design


In this design, session-level key-frames of the video are clustered and visualized in scene trees (a) planted in a video timeline ground (b,c) with roots (d) showing the corresponding danmu posts. The video timeline ground consists of (b) a heatmap view that presents the contextual information extracted from danmu posts such as sentiments and volume and, (c) an interactive timeline view that enables users to specify the video sessions of interest.

System User Interface


The VideoForest system’s user interface consists of (a) the primary timeline summary view and (b) the session summary view.

Demo Video

Zhida Sun, Mingfei Sun, Nan Cao, and Xiaojuan Ma. 2016. VideoForest: interactive visual summarization of video streams based on danmu data. In SIGGRAPH ASIA 2016 Symposium on Visualization (SA '16). ACM, New York, NY, USA, , Article 10 , 8 pages. DOI: