31. Related Work
Adobe Premiere is currently considered the state of the art low-end video editing system [Pre93]. Figure III.1 displays a sample screen illustrating the various features available in this application. The WYSIWYG-interface offers a very intuitive ways of arranging movie and audio segments, and of inserting transitions between segments or adding filters.
Figure III.1 Adobe Premiere sample screen
Adobe Premiere 4 also includes limited user programming capabilities. It allows to modify video segments using so-called transition and filter factories, to, e.g., make one video segment more red, amplify a channel, or mixing multiple channels. Additionally, the command palette (on the right in figure III.1) gives the user the capability to interactively define new commands. Unfortunately, the user is limited to adding only the commands to the command palette that are already available in the Premiere application. A real scripting language for defining new editing actions composed of predefined commands is missing.
Researchers have attempted to address such shortcomings, both in the wider domain of direct manipulation software and in the specific area of video editing systems. Eisenberg's SchemePaint system combines a simple painting program with a Scheme interpreter [Eis91]. The result is an immediately usable system that can also be programmed to produce results that are not possible with a strictly manual interface. Eisenberg argues for the incorporation of domain-enriched programming languages into a wide variety of direct manipulation systems, including video editing systems.
Ueda, Miyatake, and Yoshizawa's IMPACT system attempts to enrich the video editing interface from another direction. Rather than making it programmable in a general way IMPACT's designers supplement the direct manipulation interface with powerful functions that exploit image processing and analysis algorithms [Ued91]. The IMPACT system includes functions to identify "cuts" in a stream of video, to classify cuts (for example as a zoom-in, or pan-left), and to extract moving objects from scenes.
The VuStation system that was developed at MIT [Lin96] is "...a programming environment that facilitates the development of compute-intensive multimedia applications that combine intelligent media processing with traditional capture and display." Although the VuSystem was initially developed for proprietary ATM hardware, it runs now on a variety of commercially available systems. It is implemented as a UNIX application that interprets an extended version of Tcl [Ous94]. A collection of so-called in-band modules is linked into the application shell, providing extended functionality such as media capture, file storage, and media display. The application developer can either use existing modules from within Tcl or develop new modules in C++.
Figure III.2 Sample VuSystem Application
Figure III.2 describes a sample VuSystem application integrating predefined modules using Tcl. The VuSystem has also been extended with an interactive graphical environment to develop VuSystem applications based on a flow graph representation of the running program providing end user with programming capabilities [Wet94].
http://www.cs.cornell.edu/Info/Projects/zeno/rivl/rivl.html
Rivl is a Tcl language extension for audio, video, and image manipulation [Swa95]. Rivl extends Tcl with image and sequence (video, audio) data types. It allows for geometric manipulation, media assembly, conversion and transformation. The Rivl interpreter stores operations in a directed acyclic graph (DAG) whose edges correspond to images and whose nodes correspond to primitive operations such as scale or overlay. It is available on various UNIX system. Rivl is currently still under development at Cornell with the goal of building a rapid prototyping environment for exploring video contents processing.
Our effort, called VideoScheme, is similar to these examples. Compared to the VuSystem, our venture is much more restricted in scope, while the Rivl system, contrarily to our approach, omits the visual editing part. VideoScheme has been conceptually influenced mostly by the SchemePaint system in that we have built an extensible video editing system which provides the user with programming capabilities. By embedding a program interpreter into a direct manipulation video editing system we hope to achieve the flexibility and expressiveness demonstrated by SchemePaint. In particular, we hope to make it possible to implement the dedicated media-analysis functions of systems like IMPACT in this programming language, yielding advantages in both power and flexibility.