William is the founder and the chief scientist of New Video Corp., designers of DVI hardware and software for the Macintosh. He can be contacted at 220 Main Street, Suite C, Venice, CA 90291.
Quicktime for the Macintosh and the Audio Video Kernel (AVK) for the PC both provide time-oriented, data-handling operating-system extensions. Consequently, there's considerable overlap in their functionality. And because both are modular and multilayered, there are ways to craft a marriage between them (even though there are some integration problems in the initial releases of both).
QuickTime, Apple's "media-integration architecture" extension to the Macintosh operating system, provides a uniform API so that Mac programs can operate on time-varying data like audio, video, and animation data. QuickTime is composed of over 500 calls and traps in three extensions: the Component Manager, which allows runtime binding of code modules into applications programs (similar to Windows DLLs); the Image Compression Manager, which handles image-data compression/decompression (similar to AVK's MVD); and the Movie Toolbox, the primary API which contains all the calls needed to record and play dynamic media (similar to the Windows MCI).
The primary QuickTime data type is the "Movie" (equivalent to the AVK's stream group), which is composed of zero or more "tracks" (substreams), each of which are associated (via pointers) with "media" (the AVSS file) that reference some digital data source of a single type (that is, a file that contains raw time-varying data). Each media is associated with just one track, and each track with one and only one media.
In QuickTime 1.0, each media is either an audio, video, animation, or still image, and has associated with it a built-in media handler that understands how to operate on this media when asked. A media also contains all the necessary housekeeping information to support these operations--media type, duration, time scale, quality, file references, and so on.
While there is a one-to-one mapping of track (pointers) and media, several media may be stored in or share a single raw-data file. It is these indirect references from tracks to media to raw data that allow the integration of AVSS files into the QuickTime environment as native "MooV" (pronounced movie) files.
The first way to integrate QuickTime and AVK is to port AVK to the Macintosh as a separate set of QuickTime components, loosely supplanting the Image Compression Manager, and to integrate the standard Macintosh APIs by (re)writing some of the most commonly used interface components (for instance, StandardMovieController or SequenceGrabThing) to use these new AVK image-compression components. The advantage of this approach is that not only are AVSS files fully and transparently supported, but all AVK facilities are supportable within the QuickTime framework, including the ability to schedule many simultaneous audio, video, and graphics streams and to multitask arbitrary microcode functions.
The disadvantages are that many of the higher-level QuickTime components need to be rewritten and maintained--and this is not a trivial task. You must also preserve full compatibility between these new components and their standard QuickTime siblings. There are also the problems associated with using non-standard APIs for things that aren't rewritten to support the AVK components.
Another, and by far the best, way to handle QuickTime/AVK integration is by writing an AVSS file media-handler component for the Movie Toolbox calls to use. This media handler would digest native AVSS files and decide what calls to what components (that is, CODECs) get made. In this form of integration, AVD driver routines can be integrated within the CODEC Component structure, as well as providing possible extensions to the current CODEC architecture. While this disposes of much of AVK's high-level functionality, it has the advantage of being totally transparent to all current and future QuickTime applications and handles the sticky QuickTime 1.0 problem with non-Macintosh sound streams.
The disadvantage of this approach is that QuickTime 1.0 doesn't support interchangeable, component-sized media handlers. The only media-handlers it supports are the built-in MooV file audio and video modules.
This notwithstanding, the next best approach is the way we did it at my company (New Video Corp., Venice, California)--port the AVD drivers and interfaces for New Video's EyeQ board and gain access to this functionality via the QuickTime CODEC API. This enables QuickTime to perform the stream synchronization and uses the lower levels of AVK (the MVD/DoMotion microcode engine), to perform the dirty work of decompressing, displaying, and playing audio from AVSS files. Again, the catch is that the only way to get data out of a native AVSS file is to figure out how to make the QuickTime built-in media handlers believe that the AVSS file is a MooV file.
Unlike AVK, QuickTime allows movie files to contain references (called "aliases") to media in other files. This facilitates very high-speed editing and manipulation of movies, because there's no copying of large amounts of data. It also allows media to reside on read-only devices like CD-ROM and still permit editing of movie tracks.
The key to our AVK/QuickTime integration is the use of these aliased MooV resources to convince QuickTime that the "raw" audio and video data of an AVSS file are, in fact, part of a standard QuickTime MooV file. The generation of these aliases is accomplished with the EyeQ Convert program. This program is similar to the standard QuickTime ConvertToMovie program used to convert various other Macintosh file types to QuickTime movies. EyeQ Convert reads an AVSS file frame directory and does an AddMediaSampleReference() call from the Movie Toolbox on each frame, using the raw AVSS file pointer, this frame's offset, and a pointer to the MooV resource file being created. The result of this program is a small--typically 10K--MooV resource that just points to a vanilla AVSS file, possibly on a read-only CD-ROM.
The most important ramification of supporting native AVSS files in the QuickTime environment is especially apparent in mixed computer installations. With this architecture, a PC or PS/2 application will see an undisturbed AVSS file and be able to use it in the standard way, while a Mac application looking at the MooV file associated with this AVSS file will see a vanilla QuickTime movie. When this file is played with QuickTime the "raw" video and audio data from the AVSS file is passed to the appropriate CODEC, which invokes AVD, which decodes the data and displays it.
The biggest problem with this last approach is that QuickTime 1.0 only knows about Macintosh audio streams, so the EyeQ Convert process must either: hide the AVSS audio frames in the video data so the CODEC can sort out the audio and video on-the-fly (playing the audio on the EyeQ board's DSP while displaying the video on its Intel i750); or generate a Macintosh 8-bit PCM audio stream from the 16-bit ADPCM4e AVSS audio stream and store this stream on a new track associated with the MooV file.
The latter approach has the advantage of providing complete Macintosh control of audio for this MooV, but the disadvantage of storing lower audio quality (8-bit mono at a 22-KHz sample rate vs. 16-bit stereo at a 32-KHz sample rate).
While AVK was designed with a hardware-assisted (i750) virtual-production studio model in mind, QuickTime was designed as a general-purpose multimedia extension to the Mac/OS, and did not presuppose the availability of hardware. Standard QuickTime movie files are typically compressed with either the AVC (Apple video compressor) CODEC component or the RLE (run length encoding) animation CODEC. In AVK, the compression algorithms (RTV or PLV), are implemented in i750 microcode and invoked by the DoMotion microcode scheduler. With QuickTime/AVK, these microcode functions are invoked from a generic Mac DVI CODEC so that they fit into the QuickTime environment. In addition to this MacDVI CODEC, New Video has implemented a software-only RTV decompressor CODEC that will decompress standard RTV AVSS files at lower resolution (128x12O) and at lower frame rate (10-15 FPS) than the EyeQ-assisted Mac DVI CODEC.
Copyright © 1992, Dr. Dobb's Journal