Dynamic video narratives

Carlos D. Correa, Kwan-Liu Ma

Research output: Contribution to journalArticle

53 Citations (Scopus)

Abstract

This paper presents a system for generating dynamic narratives from videos. These narratives are characterized for being compact, coherent and interactive, as inspired by principles of sequential art. Narratives depict the motion of one or several actors over time. Creating compact narratives is challenging as it is desired to combine the video frames in a way that reuses redundant backgrounds and depicts the stages of a motion. In addition, previous approaches focus on the generation of static summaries and can afford expensive image composition techniques. A dynamic narrative, on the other hand, must be played and skimmed in real-time, which imposes certain cost limitations in the video processing. In this paper, we define a novel process to compose foreground and background regions of video frames in a single interactive image using a series of spatio-temporal masks. These masks are created to improve the output of automatic video processing techniques such as image stitching and foreground segmentation. Unlike hand-drawn narratives, often limited to static representations, the proposed system allows users to explore the narrative dynamically and produce different representations of motion. We have built an authoring system that incorporates these methods and demonstrated successful results on a number of video clips. The authoring system can be used to create interactive posters of video clips, browse video in a compact manner or highlight a motion sequence in a movie.

Original languageEnglish (US)
Article number88
JournalACM Transactions on Graphics
Volume29
Issue number4
DOIs
StatePublished - Sep 13 2010

Fingerprint

Masks
Processing
Chemical analysis
Costs

Keywords

  • Graph-cut optimization
  • Image compositing
  • Interactive editing
  • Motion extraction
  • Video exploration

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Cite this

Dynamic video narratives. / Correa, Carlos D.; Ma, Kwan-Liu.

In: ACM Transactions on Graphics, Vol. 29, No. 4, 88, 13.09.2010.

Research output: Contribution to journalArticle

Correa, Carlos D. ; Ma, Kwan-Liu. / Dynamic video narratives. In: ACM Transactions on Graphics. 2010 ; Vol. 29, No. 4.
@article{32e8d3406e8a422cb559816584c65f34,
title = "Dynamic video narratives",
abstract = "This paper presents a system for generating dynamic narratives from videos. These narratives are characterized for being compact, coherent and interactive, as inspired by principles of sequential art. Narratives depict the motion of one or several actors over time. Creating compact narratives is challenging as it is desired to combine the video frames in a way that reuses redundant backgrounds and depicts the stages of a motion. In addition, previous approaches focus on the generation of static summaries and can afford expensive image composition techniques. A dynamic narrative, on the other hand, must be played and skimmed in real-time, which imposes certain cost limitations in the video processing. In this paper, we define a novel process to compose foreground and background regions of video frames in a single interactive image using a series of spatio-temporal masks. These masks are created to improve the output of automatic video processing techniques such as image stitching and foreground segmentation. Unlike hand-drawn narratives, often limited to static representations, the proposed system allows users to explore the narrative dynamically and produce different representations of motion. We have built an authoring system that incorporates these methods and demonstrated successful results on a number of video clips. The authoring system can be used to create interactive posters of video clips, browse video in a compact manner or highlight a motion sequence in a movie.",
keywords = "Graph-cut optimization, Image compositing, Interactive editing, Motion extraction, Video exploration",
author = "Correa, {Carlos D.} and Kwan-Liu Ma",
year = "2010",
month = "9",
day = "13",
doi = "10.1145/1778765.1778825",
language = "English (US)",
volume = "29",
journal = "ACM Transactions on Graphics",
issn = "0730-0301",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

TY - JOUR

T1 - Dynamic video narratives

AU - Correa, Carlos D.

AU - Ma, Kwan-Liu

PY - 2010/9/13

Y1 - 2010/9/13

N2 - This paper presents a system for generating dynamic narratives from videos. These narratives are characterized for being compact, coherent and interactive, as inspired by principles of sequential art. Narratives depict the motion of one or several actors over time. Creating compact narratives is challenging as it is desired to combine the video frames in a way that reuses redundant backgrounds and depicts the stages of a motion. In addition, previous approaches focus on the generation of static summaries and can afford expensive image composition techniques. A dynamic narrative, on the other hand, must be played and skimmed in real-time, which imposes certain cost limitations in the video processing. In this paper, we define a novel process to compose foreground and background regions of video frames in a single interactive image using a series of spatio-temporal masks. These masks are created to improve the output of automatic video processing techniques such as image stitching and foreground segmentation. Unlike hand-drawn narratives, often limited to static representations, the proposed system allows users to explore the narrative dynamically and produce different representations of motion. We have built an authoring system that incorporates these methods and demonstrated successful results on a number of video clips. The authoring system can be used to create interactive posters of video clips, browse video in a compact manner or highlight a motion sequence in a movie.

AB - This paper presents a system for generating dynamic narratives from videos. These narratives are characterized for being compact, coherent and interactive, as inspired by principles of sequential art. Narratives depict the motion of one or several actors over time. Creating compact narratives is challenging as it is desired to combine the video frames in a way that reuses redundant backgrounds and depicts the stages of a motion. In addition, previous approaches focus on the generation of static summaries and can afford expensive image composition techniques. A dynamic narrative, on the other hand, must be played and skimmed in real-time, which imposes certain cost limitations in the video processing. In this paper, we define a novel process to compose foreground and background regions of video frames in a single interactive image using a series of spatio-temporal masks. These masks are created to improve the output of automatic video processing techniques such as image stitching and foreground segmentation. Unlike hand-drawn narratives, often limited to static representations, the proposed system allows users to explore the narrative dynamically and produce different representations of motion. We have built an authoring system that incorporates these methods and demonstrated successful results on a number of video clips. The authoring system can be used to create interactive posters of video clips, browse video in a compact manner or highlight a motion sequence in a movie.

KW - Graph-cut optimization

KW - Image compositing

KW - Interactive editing

KW - Motion extraction

KW - Video exploration

UR - http://www.scopus.com/inward/record.url?scp=77956361188&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956361188&partnerID=8YFLogxK

U2 - 10.1145/1778765.1778825

DO - 10.1145/1778765.1778825

M3 - Article

VL - 29

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

SN - 0730-0301

IS - 4

M1 - 88

ER -