A study on enhancing timeline-like visualization with verbal text

There has been a long-standing question of whether sound or audio annotations may assist in data visualization tasks. This paper presents our study focusing on enhancing timeline/storyline like visualizations with audio annotations. Timeline visualizations are widely used to illustrate interactions among different entities over time. This type of visualizations facilitate reviewing important activities and their associations. For a long timeline, however, it is difficult for the viewer to remember all the essential information found in the visualization process. Since hearing is another primary human sense for perceiving information, we conjecture that adding audio annotations to selected sections of a timeline visualization can help improve the viewer's recall of important events. We have designed a user study based on augmenting storyline visualizations with verbal text, and tested subjects with three different settings: visual cues only, verbal text only, and both. While our test results do not give a strong indication of the clear advantage of adding verbal text, the lessons learned in our study suggest directions for further study and will help us and others design audio-augmented visualization systems.