People should learn from their mistakes. Yes, I'm sure that's true. So should companies. And maybe computer programs. They should all learn from their mistakes, if they can't learn any other way. Experience, that jagged little pill, is a rough but effective teacher.
We'll start with a look at a book about games that learn, by two writers who are serious about playing and playful about words. Then we'll move on to Apple's World Wide Developers Conference, where the playful company tried to convince developers that it had learned a thing or two from its contact with the jagged edges of its own experience. And we'll wrap it up with some lessons from programming history.
But first, something about the vocabulary of computation.
People who create new products or open new areas of research are pretty much forced to neologize-they need to create new words to denote, and sometimes just to describe, the new things that they are bringing into the world.
Today, you toil in a field in which workers bring new things into the world every week, or every day (depending on the level at which you are willing to assign thingness). And it is the curious nature of these things that they often cannot even be used without being named. Often, in fact, using such a thing is mentioning its name. Probably not since the invention of language has any task required so much neologizing as computer programming does.
It's hard to come up with that many new words. Often it's easier and more mnemonic to repurpose or overload existing words. FORTRAN, SCSI, and SPARC are as legitimate coinages as Xerox and Kodak, but you wouldn't rather call a Lotus 1-2-3 document anything other than a spreadsheet-even though it really isn't the paper document to which that word formerly referred. Sometimes certain people feel that certain acts of terminological repurposing carry along too much of the words' original connotations. The word "memory" was once considered overly suggestive (back then, the preferred alternative would have been "core," though, not "RAM"). This criticism arises notably in the hubristically named field of Artificial Intelligence, and particularly when it comes to the repurposed terminology of wetware. "Memory" has become an accepted computer term, but words like "intention," "intuition," "introspection," and "instinct" are often considered to be evidence of the worst kind of fuzzy-minded anthropomorphism when applied to computer programs. (Maybe not the worst kind. That might be reserved for the use of terms like hope, fear, joke, promise, editor-at-large, and the like.)
Leonard Dorfman and Narendra Ghosh argue for some pretty suggestive anthropomorphisms in their book Developing Games that Learn (Manning Publications, 1996, ISBN 1-884777-15-5). Dorfman and Ghosh are particularly fond of the terms "program instinct" and "program consciousness." They also label their approach "objective artificial intelligence" (OAI) and imply that it is enough of a departure to deserve that name.
As a consumer (or more correctly, I guess, a retailer) of words, I care about this. I don't want to see the product cheapened or the market glutted with inferior goods. That was enough to get me into the book, but I suspect that most readers would be looking for something else in it, such as: What learning algorithms do D and G present?
There are many good algorithms for getting where you're going in the daylight when you know the way. The all-too-common and more pressing dilemma of being lost in the dark has so far yielded to only one algorithm, really, not counting variations on the theme: Proceed blindly, bouncing off obstacles. (You can make this sound more purposeful if you refer to the bouncing as "altering one's strategy to avoid unproductive avenues.") It's a brutal algorithm, and a ubiquitous one, popping up frequently in areas having to do with the survival and proliferation of species. It's Nature's way.
It's also the way of most computer algorithms that claim to learn, and Dorfman and Ghosh's approach is no exception. The authors claim that their approach is drawn from psychology, philosophy, and computer science, and while I don't doubt them, I wouldn't give the claim any real weight. The algorithms they've come up with seem to me to be pretty ad hoc and pragmatic. They also seem to be of limited generality, although I may have been misled by the fact that the authors only present examples from a limited domain of strategy games. The first algorithm they present keeps track of the moves made in a game and marks a move as unacceptable if it has led to a loss in the past. This is familiar to anyone who has looked into learning algorithms and is not very powerful.
The second algorithm does something similar but also analyzes moves backward from the end of the game to find the last move that wasn't forced, and changes that move. Again, not all that powerful.
The authors have given their third algorithm the awful name "three-filled marker disruption." That makes it sound specific to a particular situation, one that involved filled markers, whatever they are. As described, it is pretty problem-specific, but the general idea is powerful: Determine when a potentially dangerous pattern is being developed by the (human) opponent and disrupt the formation of that pattern.
The power of such an approach is evident: A pattern could be a chess opening (it isn't in D and G examples, though). Discerning what chess opening an opponent is employing and disrupting it is at least part of a good chess strategy. You'd also want to ensure that you were disrupting in a way that helped you to develop a good pattern of your own, and D and G's algorithm actually does a little of that.
The trick is developing the pattern-recognition function, of course, and the only help you'll get from D and G here is their clear example of how they developed the pattern-recognition function for the game they were analyzing. (It's "Drop Four," a game complex enough to be fun to play, but not on the order of chess.)
I have mixed feelings about Developing Games that Learn.
I don't really think that Dorfman and Ghosh's approach is distinctive enough to warrant the bombastic name of objective artificial intelligence; their work does not draw upon the extensive body of machine learning work in computer science, and the algorithms seem pretty specific to the games for which they were empirically derived. Although the authors suggest that you can use the OAI paradigm to derive algorithms for other games, and perhaps for other learning situations, I don't see it.
On the other hand, the book is very readable and they generously provide full source code both on the disk and in the book. Also, the three-filled marker disruption algorithm is powerful, and I do think that it is possible to study its derivation and its source code and to see ways to generalize what they have done. The way they discover patterns to avoid is, I think, clever and useful. Perhaps they should have introduced the book by saying, "Here are some programs that we wrote that play games and learn from their losses. Maybe you will find something useful in them for your own work." And skipped all that stuff about the OAI paradigm. Not everything is a paradigm.
In May, a humbled, though not necessarily humble, Apple Computer hosted its annual World Wide Developers Conference (WWDC). The event was well attended (I did my share, attending all five days). As usual, a great deal of technical information was imparted along with no small amount of cheerleading for the Macintosh Way. The latter sounded more defensive than usual in the wake of Apple's slipping ship dates for the much-ballyhooed Copland release of its OS, slipping market share, slipups in quality control, sloppy sales projections, sliding profitability....
Of course Apple is still a rich company with a larger market share than almost any other hardware vendor, unchallenged dominance in at least one major market, and fierce loyalty in a couple of others. And, of course, Apple has been as much a victim of press-piling-on as of its own gaffes. Nevertheless, every developer at WWDC knew, and even some Apple folks realized, that Apple's first mission at the conference had to be to answer the question, "Have you learned from your mistakes?" As it happens, Apple's new (as of February) Chairman of the Board and CEO, Gilbert Amelio, has actually written a book titled Profit from Experience, about his efforts in transforming National Semiconductor between 1991 and 1995. So I guess that pretty much answers the learning-from-mistakes question, huh?
Well, maybe not altogether. Particularly for those who have some concerns about the ways in which Amelio solved National's problems. The first variation on the learning-from-mistakes question is: Does Apple know what business it's in? Some bad answers:
Amelio outlined a new corporate structure that developers I talked to said made sense, with four divisions: Macintosh, Imaging (a very profitable area for Apple today), Alternative Platforms (the most intriguing), and Information Appliances (Newton and Pippin). Cutting across all of these will be AppleSoft, responsible for OS and application software (but no, Amelio assured developers, Apple will not compete with them-it will not develop products that are already being developed by third-party developers); AppleNet, charged with Apple's Internet initiatives; Apple Core Technology, the former Advanced Technology Group with a more-pragmatic focus; and AppleAssist, a revamped customer-service division that will also "help customers buy hardware and software," including third-party products. Having Apple help you sell your products might sound good, but there would presumably be a cost. One developer commented, "No, they won't compete with us, but they will cut into our margins."
The most striking change Amelio has made is that the product line will be radically simplified. The number of Mac models will be hacked down severely; the number of motherboards will be, in stages, cut from nine down to two or three; all Macs by year-end will ship with a minimum of 12 MB of RAM; and there will be just one version of the operating system for all Macs.
A returned Guy Kawasaki, beloved by all developers (wait a minute: Are these my notes, or a Kawasaki press release?), was much in evidence, signaling a return of an aggressive program of developer support and evangelism that has been missed of late. A more-concrete symbol is the $20 million that Amelio gave new VP for Developer Relations Heidi Roizen to spread around in helping third-party developers. It really is true that developers are crucial to Apple's future; maybe the company even gets it now. But Apple has always known what to say to developers; it's the follow-through that has sometimes disappointed. Only time will tell whether Apple really gets it.
Does Apple get it when it comes to the Internet? Let's check. I have the official WWDC-release t-shirt right here. It says, "The Internet. We're all over it." As an articulation of a corporate strategy it's a little vague, but it is all cotton. If the slogan means that Apple is on the Internet in a big way, that's true. The Web site and live Webcasts of events are well executed.
If it means that Apple knows what role a hardware and system-software vendor can play in the internetification of the world, I dunno, maybe nobody knows that. But I am surprised to find that Apple can lay claim to a pretty subtle and forward-looking vision of the Internetworked future. The pieces: There's the deal with Natural Intelligence to roll their Java compiler into the OS. There's the release of Cyberdog as a toolkit for internet-empowering apps; alls ya gotta do is embrace OpenDoc-not a small commitment. Spyglass is doing so and should, in the process, make its browser at least a distinctive bump on the boring Netscape-dominated landscape.
At WWDC, Netscape announced that it would be supporting OpenDoc. This took Apple executives by surprise. Netscape had been hemming and hawing about OpenDoc for some time, apparently, and Marc Andreesen finally decided to preempt the wrangling by announcing it onstage in front of several thousand developers. Apple PR was a little nonplused: It was a big news item and they didn't have any press releases to hand out. Netscape also announced that a QuickTime plug-in will be included in the next release of Navigator.
The Newton platform also got more Net-savvy with the Newton Internet Enabler, which, along with the backlit screen and other improvements in the past months, tended to evoke an even-more enthusiastic than usual "Not bad; I'd almost buy one" response from developers I talked to. These are pieces. But all of these pieces, I think, can be seen as elements of a vision of computing that is far more distributed and document-centered than today, a model of computing that undermines the importance of hardware, operating systems, and large applications in favor of the user-creator and the tool builder.
This is not at all the same thing as saying that these pieces are all elements of a particular strategy, or that anyone at Apple sees how these elements can be fit together (I certainly don't mean to suggest that I do), or how Apple benefits from them. I suspect that Apple is pulling together a lot of pieces that just seem like they have to be part of the future of computing in the era of the Internet, and is hoping that they will someday fit together to make something. And given the unpredictable nature of the Net, maybe that's a strategy.
Two readers took issue recently ("Letters," DDJ, July 1996) with some of my remarks (in March 1996) on the issue of who "really" invented the digital computer. Both Sy Wong and Phil Mitchell, I think, assumed that I was up to something more than commenting on a particular legal battle and the undeserved obscurity of Konrad Zuse. Certainly, a proper history of computing can't be written without giving credit to John von Neumann and to Alan Turing. I've mentioned both Andrew Hodge's biography of Turing (which, coincidentally, the same Phil Mitchell reviews in this month's "Programmer's Bookshelf") and his Turing Web page here before. One interesting take on the polymath genius von Neumann is Steve Heims's John von Neumann and Norbert Wiener: From Mathematics to the Technologies of Life and Death (MIT Press, 1980, ISBN 0-262-58056-X), which contrasts both von Neumann's theoretical legacy and his life with those of yet another whose name must be invoked in any proper computer history.
And speaking of history, Ed Yourdon recently reasserted his place as the Edward Gibbon of the American programming enterprise. His 1993 Decline and Fall of the American Programmer depressed, infuriated, or amused programmers in the United States and apparently had a big following among would-be Bill Gateses in Europe and Micronesia. Now we have Yourdon's Rise and Resurrection of the American Programmer (Prentice Hall PTR, 1996, ISBN 0-13-121831-X). But do American programmers need resurrection, except in the imagination of Ed Yourdon?
No matter. Those who obsessively search for the right software methodology will read this book for its circles and arrows, and those of us who are congenitally unable to work in teams will dip into it for tidbits such as this:
Java's [multithreading] approach is...modeled after the Cedar/Mesa systems implemented by Xerox PARC, which in turn are based on a formal set of synchronization primitives developed by Professor C.A.R. Hoare in the mid-70s.
I didn't know that.
And those of us who write for a living can only marvel at Yourdon's fine example of profiting from one's mistakes.