33. VideoScheme Language Overview

VideoScheme extends the Scheme language to accommodate digital media. In this section selected VideoScheme data objects and functions specific to video and audio editing and manipulation are described in order to illustrate the design of the program. In addition to the standard number, string, list, and array data types, VideoScheme supports objects designed for the manipulation of digital video, such as:

movie --

a stored digital movie, with one or more tracks.

track --

a time-ordered sequence of digital audio, video, or other media.

monitor --

a digital video source, such as a camera, TV tuner, or videotape player.

image --

an array of pixel values, either 24-bit RGB or 8-bit gray level values.

sample --

an array of 8-bit Pulse Code Modulation audio data.

These objects are manipulated by new built-in functions. Movies can be created, opened, edited, and recorded:

(new-movie)

; creates and returns a reference to a new movie

(open-movie filename)

; opens a stored movie file

(save-movie filename)

; saves a movie file

(cut-movie-clip movie time duration)

; moves a movie segment to the system clipboard

(copy-movie-clip movie time duration)

; copies a movie segment to the system clipboard

(paste-movie-clip movie time duration)

; replaces a movie segment with the segment on the clipboard

(delete-track movie trackno)

; removes a movie track

(copy-track movie trackno target)

; copies a movie track to another movie

(record-segment monitor filename duration)

; records a segment of live video from the monitor to a file

Images and sound samples can be extracted from movie tracks or monitors, and manipulated with standard array functions:

(get-video-frame movie trackno time image)

; extracts a frame from a video track

(set-video-frame movie trackno time duration image)

; inserts a frame into a video track

(get-monitor-image monitor image)

; copies the current frame from a video source

(get-audio-samples movie trackno time duration samples)

; extracts sound samples from an audio track

(get-color-histogram64 sub-pixels histogram

; compute color histogram of an array of color
; pixels using 64 buckets
; (i.e. 2 bits each for red, green, blue)

With this small set of primitive objects and built-in functions, we can rapidly build a wide variety of useful functions with applications in research, authoring and education.

One point to be made about VideoScheme is that it is a passive editing system. Edits are made by creating new reference lists of existing data or deriving new data. This structure is necessitated by the large volumes of data in a typical video segment. An advantage of this approach is that VideoScheme is an ideal test bed for information retrieval concepts since it does not directly modify files it manipulates.