Dr. Dobb's Journal July, 2005
Many people have become so emotionally attached to their Digital Video Recorders (DVRs) that they would suffer severe withdrawal pains if they couldn't pause or rewind live television. Yet despite this popularity, the software architecture of a DVR remains shrouded in mystery. In this article, I examine the core architectural features of a DVR, explain the differences between the DVR software on Linux or Windows PCs and DVR software optimized for set-top boxes, and explore future directions for DVRs.
A DVR is a combination of hardware/software that records live digital video content onto a storage device (such as a hard drive) so that the content may be played back in the future. To capture the live video, DVRs use a tuner to receive the cable, satellite, or over-the-air (OTA) signal. Tuners are typically part of DVR Set-Top Boxes (STBs), but they can also be located in external boxes attached to a DVR. Advanced DVR platforms contain two or more tuners so that you can record one program while watching or recording another.
Unlike a VCR, a DVR can only manipulate digital content. Consequently, many DVRs also contain an encoder to convert the analog video signal from your cable set-top box or television into a digital video stream (Figure 1). However, an encoder is not a requirement for a DVR. Since satellite and digital-only cable services only stream video in a digital format, an encoder is redundant. Therefore, to reduce costs, digital-only DVRs don't contain encoders.
The DVR feature that most excites viewers is the ability to pause or perform trick operations (fast forward and rewind, for instance) on live television. This is accomplished by recording content to a file (or buffer) as it's streaming from a tuner (typically, this file contains between 15-30 minutes of content).
If no trick operations are active, when the DVR hits the end of the buffer, it loops back to the beginning and continues to record over stale data (Figure 2). These types of buffers are called "circular buffers" because the DVR infinitely records over the same data. The moment you try a trick operation, the DVR continues to record to the circular buffer while playback continues from a point somewhere behind the record position (Figure 3). Because playback is occurring from the buffer, these buffers are also known as "time-shift buffers."
Although time-shift buffers enhance the viewing experience, their use may have unanticipated side effects, including slower channel changes and potentially lower picture quality for analog channels. One of the problems any digital video device must overcome is longer channel change times. To explain, most consumers are accustomed to analog televisions that offer virtually instantaneous channel changes. By contrast, it can take several hundred milliseconds for a digital video device to tune and start displaying decoded MPEG-2 video content. DVRs further aggravate the situation by needing to configure and use the time-shift buffer in addition to the normal MPEG-2 decoding overhead.
Another problem with time-shift buffers is their impact on analog picture quality from OTA or cable sources. In many circumstances, analog video is noticeably inferior to the equivalent digital video stream. Furthermore, analog content must be encoded into digital format before it can be stored in the time-shift buffer. Before analog content is encoded, it is typically streamed through a filter. While this filtering process can clean up analog artifacts, MPEG-2 encoding is a lossy process that is likely to degrade an already marginal analog picture.
The two predominant forms of DVRs are Home Theater PCs (HTPCs) and Set-Top Boxes (STBs). An HTPC is a turbo-charged PC that runs a customized version of Linux or Windows. Although Microsoft has been striving to make HTPC DVRs easier to use, the typical HTPC owner is a technically savvy individual who is capable of installing software drivers and patches. For example, HTPC owners must determine if they need a video capture card with a tuner or if an MPEG-2 encoder is necessary to convert analog video. Although these decisions may seem trivial to DDJ readers, they can be intimidating to average consumers.
Open-source, Linux-based HTPCs have achieved a cult following among DVR enthusiasts because of their flexibility. These projects permit viewers to compile only those features they want and eliminate features they don't need. For example, one feature that most users jettison is Digital Rights Management (DRM).
DRM technology is designed to prevent illegal copying of digital content. However, many users loathe it because it permits content owners to lock down the content and prevent viewers from making both legal and illegal recordings. While all are in agreement that illegal recordings should be prevented, consumer advocates are passionate that these restrictions on legitimate recordings are a blatant violation of copyright law.
As I mentioned in "HDTV & Broadcast Flags" (DDJ, November 2003), many of these DRM standards are still in the formative stages and currently are enforced by the honor policy. Consequently, an open-source HTPC can ignore certain DRM attributes without legal ramifications.
Unlike the immensely flexible HTPC DVR, a DVR-based STB is a prepackaged, easy-to-use solution designed for the mass market and is usually not customizable. Externally, an STB looks like a traditional cable or satellite set-top box super-sized to hold one or more hard drives. Internally, it contains one or more tuners to capture digital or analog audio/visual content. If it has analog inputs (a cable DVR or a standalone DVR such as a TiVo), then it contains at least one MPEG-2 encoder.
Yet, it is the software that distinguishes an STB DVR from its HTPC siblings. The primary software difference revolves around Conditional Access (CA) or encryption/copy protection technologies. In "The OpenCable Application Platform" (DDJ, June 2004), I pointed out that there are two predominant Conditional Access (CA) formats used to encrypt digital data on cable networks: PowerKey (for Scientific Atlanta platforms) and DigiCipher II (for Motorola platforms). Similarly, satellite vendors DirecTV and Dish (also known as "Echostar") use NDS and Nagra Conditional Access, respectively.
Because CA systems prevent users from making unauthorized digital copies of protected content, these boxes will never stream unencrypted digital content outside the box. Therefore, even if you use a DRM-free HTPC, the STB must decode the MPEG-2 content and transmit it in an unencrypted analog form so that your HTPC can capture it. The DVR then reencodes the content into a digital form for recording purposes. Because the analog picture often is inferior to the original digital content, and the decoding/reencoding process further degrades picture quality, this clearly is not a clean solution from a picture quality perspective.
By contrast, a cable or satellite DVR STB is able to record the digital video content directly without the extraneous digital-to-analog and analog-to-digital conversions. This results in digital recordings with optimal picture quality. However, because DRM attributes are preserved, some HTPC enthusiasts may not consider the improved picture quality worth the trade off.
A second software difference mentioned in "The OpenCable Application Platform" was the stranglehold proprietary operating systems have on the cable market. Fortunately, the emergence of DVR and OCAP is beginning to reduce Multiple Service Operators (MSOs) dependence on closed operating systems (thankfully, the American satellite market never bound itself to proprietary operating systems).
While OCAP boldly states that its mission is to eliminate proprietary operating systems and APIs, DVRs are silent killers of proprietary operating systems. To explain, before DVR, cable set tops operated in a closed world. These boxes only offered a single tuner and the tuner was the only way to obtain content. Consequently, you could concoct a highly customized operating system that coupled a conditional access system to proprietary hardware. By contrast, DVRs STBs are more akin to PCs. You can connect them to external hard drives and stream content to them from external IEEE 1394 devices. As these input and output sources multiply exponentially, it is virtually impossible to write proprietary drivers for all of these devices.
Given these ominous trends and the growing acceptance of OCAP, an increasing number of DVR STBs are abandoning proprietary operating systems and switching to Linux. Linux offers a plethora of proven and stable file systems, reliable IEEE 1394 and USB 2 drivers, and the ability to reuse code from Linux-based HTPCs. All of these factors combine to create significant time-to-market advantages for Linux-based solutions.
Again, DVRs rely on tuners, circular buffers, and storage devices for time shifting and digital video recording. However, there are significant variations on how these DVRs use these building blocks to enable other features. For example, there is no universal technique to handle the transition between time-shift buffer and live video content. To explain, if the viewer fast-forwards past the start of the time shift buffer, there are at least two ways of handling the situation.
One extreme is to always stream through the time-shift buffer (Figure 4). This alternative ensures that the viewer can instantaneously perform trick operations because the time-shift buffer is always available. Consequently, these DVR solutions never show live content—everything is streamed from a prerecorded time-shift buffer. Unfortunately, this may have the side effect of marginally decreased picture quality for analog sources since all content flows through an MPEG-2 encoder.
By contrast, when the viewer fast-forwards past the start of the time-shift buffer, other DVR solutions display the live video stream rather than the recorded content in the time-shift buffer. This is done by simultaneously recording the content to the time-shift buffer while displaying the live video (Figure 5). Although this provides maximum picture quality for analog video sources, there may be noticeable delays transitioning to/from time-shift mode.
First-generation DVR platforms (both HTPC and STB) use internal IDE hard drives for digital video storage. IDE drives dominated the DVR market because they are cheap and have significant storage capacity. However, newer DVR platforms are transitioning from IDE to "Serial Advanced Technology Attachment" (SATA), which has become the preferred DVR storage mechanism because of its increased data throughput, hot plug features, and massive external storage capabilities.
While SATA drives are faster, it is the hot plug and multiple drive support that makes them attractive to HD DVR vendors. Unlike IDE, SATA drives can be dynamically plugged into an external interface on the DVR. Furthermore, because these drives can be daisy-chained or placed in RAID configurations, even nontechnical users can easily add terabytes of storage to any DVR solution. This extra storage capacity is particularly crucial when recording HD content that can consume up to 20 megabits of data per second.
The initial generation of DVRs was designed for time shifting of Standard Definition (SD) content. The next generation of DVRs added the capability to time shift HD content. Because SD and HD content can be streamed through the same HD capable MPEG-2 decoder, you can create a primitive HD DVR by swapping out the SD MPEG-2 decoder for an HD MPEG-2 decoder. Unfortunately, the massive data rates of HD content make such a solution worthless if you use 20-60 GB hard drives that are viable for SD DVRs. Therefore, HD DVR solutions use significantly larger hard drives (at least 120-300 GB, for instance).
Longer term, HD DVRs will not only include larger hard drives, but will transition from MPEG-2 to MPEG-4 for archival purposes. MPEG-2 is the dominant digital video compression technology for today's cable and satellite boxes. Alas, since it was created long before DVRs became popular, it was optimized for picture quality and not data storage. As a result, a feature length HD movie compressed in MPEG-2 format consumes approximately 25 GB. By contrast, MPEG-4 (H.264) has been designed to produce similar picture quality at dramatically lower data rates (8 MB versus 20 MB). This means that the same movie compressed with MPEG-4 would only be 7 GB.
MPEG-4 technology excites satellite vendors because it lets them squeeze more HD channels onto a given satellite. Without the transition to MPEG-4, they would have to significantly increase the current satellite capacity to broadcast existing channels in HD format. Another benefit of MPEG-4 compression will be evident when massive SATA drives become available. When these two technologies are combined, it will be possible to create video jukeboxes with a breathtaking amount of HD content.
Beware, however, you may not be able to create these MPEG-4 HD jukeboxes with a nonDRM enabled HTPC. Movie studios and content owners are insistent that DRM embedded in the content be enforced. Without this guarantee, they will not permit blockbuster movies and other popular content to be broadcast in high-definition format. As a result, cable and satellite broadcasters will ensure that any DVR connected to their signals is compliant with FCC and industry standards or it will not be able to make digital recordings. Consequently, it will soon be virtually impossible (if not illegal) to create an HTPC that willfully ignores content protection when making digital copies.
Yet, the death sentence for HTPC DVRs may be delayed due to the analog hole. The analog hole describes a vulnerability in the current copy-protection schemes when the digital signal is converted into a high-definition analog format. If this HD analog content is transmitted on component video outputs, there is no standardized mechanism to protect it. Consequently, it is technically possible to reconvert it into a digital format to be recorded by a nonDRM DVR. Although this reconversion process does impact picture quality, the degradation isn't significant and you can still obtain a stunning HD signal after a digital-to-analog-to-digital conversion. Several different copy protection technologies have been proposed to close the "analog hole," but no consensus has been reached. As a result, given the immense number of analog HDTVs, it is likely that HTPC will be able to exploit this analog hole for the foreseeable future.
DVRs have transformed the American viewing experience. Viewers are now empowered to perform trick operations and pause live television. In this article, I examined how all DVRs contain core elements such as storage devices, time-shift buffers, and trick mode controls. Furthermore, there are two broad categories of DVRs, each appealing to a different audience. Technophiles and hackers will gravitate to HTPC DVRs, while consumers will be attracted to the ease of use of set-top box DVRS. In the future, DVRs will be transitioning to SATA drives with large drives and will cram more data on these larger drives by utilizing MPEG-4 compression.
At this point, you're probably eager to discover what industry standard, open APIs will let you control your DVR. Alas, the industry is stuck in a quagmire of proprietary APIs that require you to sign nondisclosure agreements to create applications. Furthermore, while there are open-source DVR projects, applications written to these APIs will not run on consumer-oriented DVR set tops. More specifically, even though many DVR vendors have transitioned to Linux, they have to maintain a closed API environment to maintain backward compatibility with legacy applications.
Thankfully, CableLabs released the "OCAP Digital Video Recorder (DVR)" specification (http://www.opencable.com/downloads/specs/OC-SP-OCAP-DVR-I01-040524.pdf). While this won't help you write applications for satellite DVRs, it will let you create DVR-enabled Java applications that run on any OpenCable-compliant DVR box. In a future article, I will examine the OCAP DVR API in more detail.
DDJ