Dr. Dobb's Journal March 2001
Most Palm users crave access to the multimedia gadgets found in Windows CE-based handheld devices. Fortunately, PocketPyro (http://www.pocketpyro.com/), which makes accessories for the Palm, produces the "Pyro for Palm" (see Figure 1) that fulfills these multimedia cravings. Not only can the Pyro play MP3 files, but it provides filesystem management and is capable of speech recognition, text-to-speech, and other such multimedia applications. In this article, I'll examine the PocketPyro architecture, explore its SDK, and show how you can use it to create exciting multimedia applications.
Of course, most people are initially attracted by Pyro's MP3 playback capabilities. However, its expendability is what separates the Pyro from similar devices. This expendability is enabled by open interfaces, mass storage, and its processor. Much of Pyro's flexibility stems from its use of industry-standard interfaces and components. For example, multimedia content can be transmitted to and from its USB interface at 12 Mbps, rather than the paltry 35 kbps capacity of the Palm's serial port.
The Pyro ships with main memory configurations ranging from 64-192 MB of flash memory, and you can add additional storage with a Secure Digital (SD) memory card. While a significant advantage of SD cards is their ability to store multimedia content in minuscule form factors, the real reason for their popularity is security.
Content providers (aka Hollywood) have been hesitant to deliver digital content for portable devices. Fortunately, since each SD card contains hardware security features that minimize the risk of piracy, their fears have been pacified (see the accompanying text box entitled "The Secure Digital Music Interface").
A second reason for Pyro's flexibility is its ample supply of nonvolatile flash memory. By default, this memory is formatted as a large, removable drive that is accessible when you connect the Pyro to the USB port on a Windows 98/2000 machine.
The driving force behind Pyro's flexibility is its signal-processing engine, called "DragonThunder," which consists of a general-purpose Texas Instruments TMS320VC5402PGE-100 Digital Signal Processor (DSP) and intelligent multimedia management software. Since DragonThunder includes a general-purpose DSP, it is not limited to MP3 playback, and can be used for a variety of processing applications. The Pyro's DSP workload is assisted by an AT90LS8535-4AC RISC I/O processor, which controls the Pyro's power, monitors the DSP's status, performs serial and expansion port I/O, and monitors battery status.
FireStream, Pyro's multimedia API, gives you access to these hardware capabilities. The initial version of FireStream concentrates on providing basic multimedia building blocks that you can add to your Palm applications. These APIs are divided into two categories — enhanced file streaming and media APIs.
Since the Palm is a memory-constrained platform, its file I/O APIs are optimized to access and update information in place rather than copy data to and from a filesystem. The same principles are used to manipulate files on the Pyro's removable drive; see Table 1.
FireStream's file APIs are roughly equivalent to their Palm counterparts except they are prefaced with Dev and support multiple cards. To explain, the original Palm APIs require that the filesystem device be 0 since no other filesystems are supported. By contrast, the Pyro APIs not only work on the Palm's filesystem (device 0), but also on Pyro filesystems (device DEVICE_ID_POCKET_PYRO).
Unfortunately, Pyro developers had to replicate all of the functionality in Palm's file streaming APIs to implement multiple device support. Because the Palm OS does not offer installable filesystems, this is the only way to access nonPalm storage systems.
Unlike the DevFile functions, Pyro's media APIs have no functional equivalent in Palm OS. These functions not only let you manipulate multimedia content, but they offer access to logical multimedia devices running on the Pyro DSP.
To manipulate content to the Pyro DSP, you use the media streaming APIs. These functions can stream content to or from files, memory, or remote Internet-based sources (only file streaming and memory buffers are supported in the initial release). To open a file, you use the MediaFileOpen() API; see Listing One.
One interesting (and somewhat secretive) parameter of MediaFileOpen() is the MediaDescriptor, which describes file attributes such as file formats (MP3, WMA, or whatever). Theoretically, you can use them to retrieve the Digital Rights Management (DRM) information associated with the file. Traditional DRM attributes that could be contained in a MediaDescriptor are the number of times the file can be played or recorded. Should Pyro refuse to play your file due to DRM restrictions, these DRM attributes could be used to report a reasonable warning message to users rather than cryptic error numbers. Alas, the initial release of FireStream provides only a cursory description of MediaDescriptors. Therefore, we'll need to wait for the follow-up release to obtain the critical details on MediaDescriptors and DRM attributes.
Once you've created a media stream, use MediaFilePlay() to commence playback. Similarly, the MediaPause(), MediaResume(), and MediaStop() can be used to pause, resume, and stop a stream, respectively. When you're finished streaming, DevFileClose() releases any DSP resources you've allocated and destroys your stream object.
When you create a MediaStream object, Pyro configures a network of logical multimedia devices (such as amplifiers and speakers) necessary to stream the content (see Table 2). Since these devices are initialized with a vanilla set of audio attributes, you may alter them with the Media Device APIs.
You can get a listing of the Media Devices in a Pyro with MediaControlGetCount; see Listing Two. Most Pyros contain at least an AUDIO_OUTPUT_CONTROL. However, since Pyros are likely to be deployed by a variety of OEMs, it's possible that the AUDIO_OUTPUT_CONTROL may not be present, or that additional devices (such as AUDIO_INPUT_CONTROL) will be reported (Table 2).
Once you've resolved the device you wish to use, call MediaControlOpen() to open it. Before you blindly change the device's default settings (Table 3), you should call MediaControlGetAttr() to determine the device's original settings. For instance, to retrieve the current volume setting, pass the AUDIO_CONTROL_VOLUME attribute to MediaControlGetAttr() (see Listing Three).
When you're ready to change the volume, invoke MediaControlSetAttr(), which uses the same attributes as MediaControlGetAttr(). The only difference is that MediaControlSetAttr() updates a device's settings, whereas MediaControlGetAttr() retrieves the device's settings (see Listing Three).
Although MediaControlSetAttr() lets you modify the attributes of existing controls, you remain encumbered with the default network of multimedia devices that FireStream constructed for you. Thankfully, if your application requires a custom set of multimedia devices, you can create your own network of output devices with a MediaContext (see Listing Four).
MediaContexts are a grouping of logical devices that manipulate data. Contexts are created with the MediaContextCreate() API, which returns a MediaContext object that you can use to construct your multimedia network.
Although it is not obvious, the most important parameter associated with MediaContextCreate() is the callback parameter. This parameter lets FireStream send your application asynchronous notifications when the state of the MediaContext changes (see Table 4). To receive these notifications, your program must declare a function (or event handler) with the following calling convention:
typedef (*PFN_MediaEvents) (MediaContext hContext, MediaEvent *pEvent);
and pass the address of this function to MediaContextCreate(). Listing Five illustrates how to monitor the MediaPlayStop and MediaPlayPos events inside a MediaContext event handler.
MediaContextAddComponent() is used to add a device to the MediaContext (see Listing Six). Similarly, MediaContextRemoveComponent() extracts a device from the context. These functions are useful when you need to adjust audio output options. For instance, to transfer audio output from the stereo headphone jack to the expansion port, you would remove the headphone jack with MediaContextRemoveComponent(), and add the expansion port via MediaContextAddComponent().
Since a MediaContext produces or consumes content, it is worthless unless you attach it to a multimedia stream. This is accomplished with MediaPlayFile or MediaPlayBuffer(). In the previous MediaPlayFile example, I used the default context (0) to play the stream. However, to use a custom context rather than the default, the context parameter should be filled in with the value returned from MediaContextCreate() (see Listing Seven). If you attach an event handler to the context, the handler can be used to monitor the health of the MediaContext. Event handlers are particularly useful when streaming audio buffers to the DSP. Unlike MediaPlayFile(), which operates on files, MediaPlayBuffer() lets you stream audio buffers to a MediaContext. If you have more than one buffer to play, use the MEDIA_GENERATE_BUFFER_REQUEST flag.
MEDIA_GENERATE_BUFFER_REQUEST causes FireStream to send you a MediaBufferRequestEvent when it has consumed your buffer and is ready to accept an additional buffer (see Listing Eight). The primary reason for this event is underrun prevention. To explain, if you can't supply the DSP with data buffers in a timely fashion, it may belch out irritating noises before it finally ceases playback. MediaBufferRequestEvent alerts you that an underrun is imminent unless the DSP is either fed additional buffers or stopped.
When you're done with the MediaContext, call MediaContextDestroy() to release the resources associated with it. However, MediaContextDestroy() should be used carefully. For example, if you created a custom context and started playback with that context, you cannot delete the context until the playback stream is closed. If you try to close the context while it is actively attached to a stream, MediaContextDestroy() will fail. Furthermore, this may result in memory leaks since the resources associated with the context will not be released.
If you're a DirectX programmer who expects low-level access to audio hardware, it may surprise you that the initial revision of FireStream doesn't let you record or give you explicit control over buffer consumption. Fortunately, the Pyro developers realize these are critical features and intend to address these in updates to FireStream. Future enhancements to FireStream will include DSP documentation, recording, and enhanced streaming.
You may also be frustrated at the sparseness of the FireStream API when compared to the rich DirectX API set available on Win32 and WinCE. Programmers expecting an all-encompassing multimedia API will never be satisfied with the minimalist approach endorsed by FireStream. While DirectX is a wonderful solution for the PC, a Palm equivalent would have a huge footprint and devour battery power.
By contrast, FireStream attempts to provide a functional API while minimizing its memory footprint. Furthermore, FireStream does not shackle your applications to Pyro hardware. When other vendors write FireStream drivers for their hardware, your application will instantly be able to exploit the new devices.
The Pyro for Palm finally gives Palm developers access to multimedia capabilities. It not only supports MP3 playback, but can also be used for speech recognition, text-to-speech, and other multimedia tasks. This flexibility is enabled by the DragonThunder DSP infrastructure and the use of open interfaces such as USB and SD cards.
You can incorporate these multimedia capabilities in your Palm applications by using FireStream, the Pyro SDK. While the initial release of FireStream focuses on audio playback, it has specifically been designed to accommodate future technologies while minimizing the memory footprint.
Although the Pyro is an exciting device, it does have a few shortcomings. For instance, the FireStream SDK has limited streaming and no recording support. Furthermore, documentation on the DRM features are sketchy. Fortunately, Pyro has promised to address these software limitations in a follow-up release of FireStream. Despite these minor weaknesses, the Pyro for Palm's open interfaces, powerful DSP engine, and FireStream SDK is an evolutionary advance for the Palm platform.
DDJ
Err err;
FileHandle handle;
MediaFileDesc *mediaDesc;
mediaDesc.size = sizeof(MediaFileDesc);
handle = MediaFileOpen(libRef,
DEVICE_ID_POCKET_PYRO,
pCallback, // PFN_DEVFILE_EVENTS callback
&mediaDesc, // media description
0, fileModeReadOnly, &err);
// interesting info is returned in the MediaFileDesc
// for instance mediaDesc.pFileName has the filename
// open a file
handle = MediaFileOpen(libRef,
DEVICE_ID_POCKET_PYRO,
pCallback, // PFN_DEVFILE_EVENTS callback
&mediaDesc, 0, fileModeReadOnly, &err);
// start playing it
MediaFilePlay(libRef, hcontext, handle);
Int count;
Err rc;
// get the number of devices in for the default device.
rc = MediaControlGetCount(libRef,
-1, // default device
AUDIO_OUTPUT_CONTROL, &count);
if ( !rc )
{
// if rc == 0, then count contains the number of controls...
}
Err rc;
int value;
int id;
// get the control id, we'll use it to get/set attributes
rc = MediaControlGetId(libRef,
DEVICE_ID_POCKET_PYRO,
AUDIO_OUTPUT_CONTROL, // control type
0, &id);
// retrieve the current volume setting for the control...
rc = MediaControlGetAttr(libRef,
id, // control id
AUDIO_CONTROL_VOLUME, &value);
// modify the setting....
value++;
// change the current volume setting...
rc = MediaControlSetAttr(libRef,
id, // control id
AUDIO_CONTROL_VOLUME, &value);
// this handler is called when the state of the MediaContext changes
// pEvent->eType contains the type of event to monitor
Err MediaEventCallback(MediaContext hcontext, MediaEvent *pEvent)
{
// pEvent->eType is the type of event that occurred
// the switch illustrates the type of events you can check for.....
switch(pEvent->eType)
{
case mediaPlayStartEvent:
break;
case mediaPlayStopEvent:
break;
case mediaPosEvent:
break;
case mediaPausedEvent:
break;
case mediaResumedEvent:
break;
}
return ( 0 );
}
MediaContext hContext;
Err err;
// create a logical context...its empty be default.
hContext = MediaContextContext(libRef, pMediaEventCallback, &err);
if ( !err )
{
// indicate that we want audio output control...
err = MediaContextAddComponent (libRef, hcontext, -1,
AUDIO_OUTPUT_CONTROL,
0); // component #
}
MediaContext hContext;
Err err;
FileHandle handle;
handle = MediaFileOpen(libRef,
DEVICE_ID_POCKET_PYRO,
pCallback, // PFN_DEVFILE_EVENTS callback
&mediaDesc, 0, fileModeReadOnly, &err);
hContext = MediaContextContext(LibRef libRef, pMediaEventCallback, &err);
if ( !err )
{
err = MediaContextAddComponent (libRef, hcontext, -1,
AUDIO_OUTPUT_CONTROL, 0);
if ( !err )
{
// start playback to our media context....
err = MediaPlay(LibRef libRef, hContext, handle);
}
}
// this handler is called when the state of MediaContext changes pEvent->eType
// contains the type of event to monitor in this case, we're looking for
// MediaBufferRequestEvent so we can feed FireStream additional buffers
Err MediaEventCallback(MediaContext hcontext, MediaEvent *pEvent)
{
Err err;
UInt8 buffer[4096];
switch(pEvent->eType)
{
case MediaBufferRequestEvent:
{
MediaFormat format;
format.size = sizeof(MediaFormat);
format.eType = mediaFormatAudio;
format.data.audioFormat.codec = audioMP3
format.data.audioFormat.bitRate = 88200;
format.data.audioFormat.bitsPerSample = 16;
format.data.audioFormat.samplesPerSecond = 44100;
format.data.audioFormat.channels = 2;
// play another buffer and have an event generated when
// firestream is done with it..
err = MediaPlayBuffer(LibRef libRef, // ??? value ????
hContext, &buffer, 4096, &format,
MEDIA_GENERATE_BUFFER_REQUEST );
}
break;
}
return ( 0 );
}