Dr. Dobb's Journal March 2004
I think of it as the handwave. You remember it from your science classes in school, and maybe from some technical presentations you've attended—or, dare I say, given. The speaker lays out the big picture clearly enough, but skips over the tricky details. Engages in a little handwaving. Sometimes you're appreciative, sometimes you're just confused.
You know you've been handwaved when the speaker or writer says "obviously," or "the proof is left as an exercise for the reader." The classic portrayal of handwaving is that Sydney Harris cartoon—you've seen it, I know—where two scientist types are standing at a blackboard covered with abstruse equations, somewhere in the middle of which it says, "Then a miracle happens." One of the scientist types is saying to the other, "I think you should be more explicit here in step two."
Indeed.
But what if you didn't need to fill in the details? What if you could just assume the lemmas and get on with proving the theorem? What if the universe were imbued with intelligence, if the environment, or the infrastructure, or the luminiferous ether, or something out there, were densely populated with standard answers? What if all the annoying little nitty-gritty problems were solved, and the solutions—bug-free, standardized, and neatly wrapped in Esperanto APIs—were available for the price of a URI reference? Then you could just assume the answers, and delegate the details to the ether.
In such a fantasy, which just may be one of those things that past editors of this magazine have called a "Realizable Fantasy," the plan could actually be the product. Instead of producing and selling a piece of code, or a sandwich, you could just produce and sell a cleverly organized selection of links. I admit that the idea seems clearer as applied to software than as applied to sandwiches, but the sandwich industry has always lagged behind the software biz.
When the handwave works, it's a handoff.
As described above, it's a fantasy, but the idea is fundamental to programming. Throughout the history of computing this idea of handing off the details—of decentralizing or decoupling or deconstructing the computation—has taken many forms.
There's distributed computing. The SETIatHome project may not have found any off-planet intelligent life yet, but this model certainly proved itself recently when it found the latest largest prime number. P2P file-sharing systems and the structure of the Internet itself also demonstrate the power of the idea of distributing the intellectual load.
Then there are the various advances in the parallelizing of computations, from the temporal overlapping of instructions in microprocessors to modern multiprocessor personal computers, to supercomputers that implement SIMD or MIMD parallel algorithms. In all such modes of computing, the classic picture of a central processor that accepts instructions and data and churns out results like a vending machine spitting out candy bars is replaced by a picture of a network of nodes, each having the intelligence and the data required do its part of the job, on a kind of need-to-know basis.
And there are other threads in the history of computing that pull away from the vending machine model. Modern computing is suffused with the insights of early object-oriented programming, which extended the ideas of earlier structured programming, which codified the lessons learned from the use of subroutines in Fortran programs, which owed everything to the invention of certain assembly-language opcodes. All the way back to ENIAC there has been a pull away from the oversimplification inherent in that vending machine model that treats the computation as one chunk of code.
It was with the invention of object-oriented programming (OOP) that the idea of creating meaningful packages built of both information and processing power took shape. OOP is so embedded in computing today that it probably can't be thought of as a distinct paradigm any more. However it has evolved, the basic insight was revolutionary—the idea that this trick of packing together data and code in a way that has meaning really buys you something. That once you've created a widget, with the appropriate widget data and code, or properties and methods, then you can use that widget wherever a widget might be useful.
That was the promise of OOP—component software. Software engineering with interchangeable software parts. Pure OOP of the Smalltalk or Simula generation didn't really deliver on the promise, but it clarified the goal and put us on the road.
More recently, a different kind of deconstructing of computation has arrived, growing out of the intersection of the Internet and a technology so boring and simple-minded and pedantic that only a documentation writer for IBM could be interested in it: Structured Markup Language. The signposts at this intersection are XML, XML Schema, DTDs, Namespaces. The insights, at least the ones that I want to point out, are two:
The convergence of all these ideas, I suggest, puts us on the path to self-aware objects and an extreme distribution of code and data, with code and data being placed where they can do the most good. I say "on the path" because I think we are only at the beginning of this refactoring of the entire computing enterprise.
Anyway, that's how it looks to me. It may also be how it looks to the Content Reference Forum.
The Content Reference Forum (CRF), an organization formed by Universal Music Group, has come up with a standard for online music sharing that eliminates the concerns of the music industry by the shrewd expedient of eliminating the music.
This innovation is a real breakthrough, as shown by the fact that it would allow users of P2P file-sharing networks like Kazaa to swap music freely, with the blessing of the music industry. The music industry would give its blessing, but not its music. The standard that the CRF has announced is a standard for delivering everything about a piece of music except the content—in place of the content you get a reference to the content, hence the name of the group. This plan has the additional benefit of making the shared files much smaller.
Under the CRF scheme the shared file contains just metadata about the song. This metadata identifies and describes the song, tells how to resolve the reference to the song content, and names all the participants in the value chain, along with the nature of their participation. (You were able to parse that marketese, right? Old-fashioned physical goods had "owners," but intellectual property has a "value chain." I suspect that the term "chain" will soon become inadequate, if it is not already, and that the interested parties will have to traverse the value tree, or some other nonlinear data structure, to figure out to whom they owe what.) Using the metadata file, you can link to a Content Reference Server and download one copy of the actual song for your device, copy-protected like all get-out. Assuming that you have the rights.
The CRF hopes that technology companies will embrace the format, especially makers of music-playing devices. They suggest hat the music industry might reward people in some way who share lots of these music metadata files. Maybe by giving them more metadata files.
The CRF approach isn't limited to songs; it's intended to work with movies, too. One of the touted advantages is that it's context-sensitive. It fits the content to your device, location, and preferences. "It's the antithesis to one-size-fits-all mass market distribution models," the CRF says. I think that means that your broadband-connected computer gets a higher quality rendition of the tune than your cellphone does. Maybe it also means that if you're in Iran, you'll get the G-rated version of the movie, but I'm just reading between the lines. Interpolating. Antialiasing.
The CRF model isn't really full handwaving, since you eventually wind up downloading a file to store on your music-playing device or hard disk or somewhere. But it does allow small devices to convince themselves that they have more data than they could ever hold. Even if it's really just metadata.
I recently wrote in this magazine that Dr. Dobb's was still Dr. Dobb's. Tautologically true, of course, but readers wrote to point out that both this magazine and the programming enterprise have changed since the 1970s, when this magazine's subtitle was "Running Light without Overbyte." They certainly have. I now wonder how I can function with only 192 MB of RAM in the computer on which I'm writing this. Times have changed.
Maybe, to some extent, programmers engaged in embedded computing are keeping that Running Light without Overbyte spirit alive. But embedded computing is far from the limit of Computer Orthodontia.
The smallest relatively self-contained information-processing devices in widespread commercial use today are RFID tags.
They are also among the simplest, and it seems to me that they are very close to the ultimate in the dumbing-down of technology. The most basic RFID tags contain just about the least data and processing power, and actual power, for that matter, that you can put in a device and still have it do something interesting.
This kind of Tag just sits there doing nothing and consuming no power—it has no batteries or other source of power—until a Tag Reader signals it. The Reader produces a fluctuating magnetic field, which reaches a coil in the tag, generating a tiny electric current. A diode traps the current and causes it to accumulate in a capacitor, the voltage increases, and this triggers the Tag's IC to send out its ID code. That's all the IC knows how to do: Send out its one precious little item of data when a certain voltage is applied to it. It's Peppermint Patty from the "Peanuts" comic strip, who, asked any question in class, comes out of her daze just enough to say, "Present, sir!"
Fluctuations in the signal encode the ID. These fluctuations cause the Tag's transistor to trip on and off. This causes the Tag to generate a fluctuating magnetic field, which interacts with the Reader's magnetic field, and the Reader somehow extracts the Tag's ID code from this interaction of magnetic fields. I say "somehow" because the Tag has done its job when it sends out the ID code. It's the Reader that has to handle this information from the Tag, and that handling can be arbitrarily complicated. The Tag's role, though, is elegant in its simplicity.
The Tag contains only a coil, diode, capacitor, transistor, and a very dumb IC. In this model of RFID, called the "low-frequency model," the Reader and Tag need to be within one meter of one another and the Reader gets confused when more than one Tag is in the vicinity. A "high-frequency" model lets the Reader and Tag be farther apart and discriminates among Tags better, but the Holy Grail of being able to read all the Tags in a grocery cart as it flies through a checkout lane remains elusive.
Although it's very simple, I think there's something pretty amazing about this model of programming. All you need in one of these Tags is an ID and the passive ability to transmit that ID when signaled, the signal providing the power as well as the trigger. And what does that buy you?
That's the amazing part. Tags like these, or tags just slightly smarter, with the ability to receive and store a small amount of data, or equipped with a simple input device like a pressure sensor, can do all of the following:
They can find lost dogs, track lost kids, thwart kidnappers, catch crooks, and brand your cattle. They can let you zip through the tollbooth, unlock your car from a distance, buy your gas, read your tire pressures while your car is moving, monitor the engine temperature, and tie the service history and ownership history and accident history of your car to the car itself no matter where it goes, with no need for you to keep any records.
They can drive the entire cycle of events triggered by the purchase of a product, debiting the buyer's account, developing purchasing profiles on people, recording the ownership of this particular item, updating inventory records, initiating restocking of the shelf and reordering of the product, then tracing it throughout its shipping history, or blowing the whistle on a presumed shoplifter if the movement of the item isn't kosher.
They can time runners in races, find lost luggage and maybe even keep it from getting lost at all, track the movement of all currency, monitor stress in bridges, function as ID badges, assess when an Alzheimer's patient needs help, detect tampering with packages, sense hazardous materials, and link realtor signs on homes for sale with online photos and data about the home's last termite inspection report.
At least, in principle, they can do all these things. The tags themselves have all the smarts they need to do everything I just itemized. All that's needed is some smarts in the readers and associated programs and databases.
It is likely that all the information necessary to have predicted the terrorist attacks on September 11, 2001 was in the databases and files of American law enforcement and intelligence agencies well before 9/11. That in itself does not constitute a criticism of anyone in the government. Just making all the relevant data available to someone who could have seen the pattern may have been outside the realm of possibility at the time, and if anyone had actually had access to all the relevant data, it surely would have been impossible for one person to process it all and carry out the combinatorial task of considering every fact in the light of every other fact.
But not impossible for a program or for a lot of programs. I picture hundreds of specialized spiders that constantly traverse the Net looking for the particular kinds of coincidences that they know about. Expert systems that embody rules like IF threat to hijack airliner AND curious behavior of foreign student at flight school THEN increment(hijackingRiskValue). Neural net programs that evolve such rules.
But the data for such programs has to be out there in a form that can be used, and not just used by people. There's too much data, and there are too many kinds of patterns that could be significant. I really don't think that people are equal to the task.
What is equal to the task, I think, is a rich network of:
This network is not some idea that I am proposing; it is, it seems to me, the future that is actually unfolding before us right now.
And one of the things that I find intriguing about this developing future is what you might call the triumph of the dumb. (I know you think I must be talking about the current administration this time, but no, I refuse to be drawn into a political rant.) I mean the power of these remarkably dumb tags that know only how to say, "It's me, I'm here, and I'm OK." We may achieve that goal of a universe imbued with intelligence by putting the absolute minimum of smarts into everything, using RFID tags.
Recently, I got a lot of mail about a column discussing the curious properties of the endpoint of a beam of light. I don't want to stir up that brew again. But I would like to suggest that, with RFID tags, we may have reached the endpoint of Running Light.
DDJ