Last month I claimed that a new approach to user-interface design was needed, and cited developments like multimedia and virtual reality as the forces behind the need.
My inspiration in this was Brenda Laurel. Laurel is currently one of the principals in Telepresence Research, a research and development company focusing on technology that makes people feel physically present in remote or computer-generated environments. Before that, she was a software designer and programmer; marketeer; producer; researcher; protegee of Alan Kay; and human-interface consultant to Apple Computer, LucasArts Entertainment, and the School of Computer Science at Carnegie-Mellon University. Laurel edited a fat book on human-interface design, The Art of Human-Computer Interface Design (Addison-Wesley, 1990 and henceforth herein, TAOHCID), with contributions from most of the recognized and/or self-styled human-computer interface experts. Although the book was originally conceived as an in-house Apple project, by the time Laurel got through with it, half the contributors were from outside.
This month, I discuss that and another Laurel book, Computers as Theater (Addison-Wesley, 1991). But first, a quick look at yet another Addison-Wesley book on interface design, Tog on Interface (1992) by Bruce "Tog" Tognazzini.
Tog is an Apple veteran, employee number 66, and was, as of the publication of this book, Apple's Human-Interface Evangelist. Tog on Interface is largely drawn from the column he wrote for Apple's developer publication, Apple Direct, so if you've read those columns, you've read a lot of this book.
There is much new material, though, some drawn from other sources and some augmenting the columns. But while the columns may be augmented, they have not been updated: Some issues discussed here are no longer issues. This is the right approach, I think. The book is not intended to be the latest word on Apple's human-interface guidelines; there are better sources for that information.
The book is something more broadly useful: the thoughts of a human-interface expert who has thought long and hard about the issues, and who has actually spent time working with both users and developers. A lot of what's in the book should be useful to Windows developers or anyone who writes software.
I particularly liked Tog's advice on user testing on the cheap, his data on Fitt's Law and menus, and his critique of Ashlar Vellum.
The chapter on Vellum touches on a subject that will come up again later in this column: agents. Vellum is a CAD program that solves the user problem of locating specific targets, like the midpoint of a line, precisely. Vellum aids the user in this problem by the use of what Tog (but not Ashlar) calls an "agent." Ashlar calls it the Drafting Assistant, but Tog points out that it satisfies Alan Kay's basic definition of agents: "computer processes that act as guide, coach,and...amanuensis."
Brenda Laurel is an interface iconoclast.
In TAOHCID she immediately tosses up the prevailing notion of interface as "a discrete tangible thing that we can draw, design, implement, and attach to a bundle of functionality." Her intention, she announces, is "to explode that notion."
Not all the book's contributors contribute to the pyrotechnics; some of the contributions help to define the target. Thomas Erickson of Apple's Advanced Technology Group (ATG), for example, starts off the book with an illuminating explanation of how one of the more mystifying aspects of the Mac interface came to be. Countless critics have marvelled at the method Apple designers hit upon for the user to cause a disk to be ejected: by throwing it in the trash. Most of these critics have probably said or thought something like, "What could they have been thinking of?" Erickson tells all.
But the book quickly dives into the problems of interface design. Don Norman, author of The Design of Everyday Things (Doubleday, 1989), says that the best designs come when the necessary knowledge, which includes programming and graphic-design knowledge, is incorporated in the same person. Since programmers and graphic artists seem to think in different ways, this is not always practical; Erickson shows one way to fake it. Even if you don't think of yourself as creative, he says, there is hope in what he calls design by "symmetry."
Symmetry is a very powerful concept, but it isn't likely to turn a right-brained person into a left-brained one, or vice-versa. Scott Kim, who a few years ago set out quite deliberately to be the programmer-artist Norman imagines, probes into the thinking styles of software developers and graphic artists and comes up with a surprising conclusion: It isn't the big conceptual or goal-defining notions like problem-solving and communication that separate the programmers from the painters; it's the mundane survival skills of coding and making pictures. Laurie Vertelney of Apple's ATG pitted a programmer against a graphic artist on the task of redesigning the user interface of a familiar piece of software, and reports on the results.
Apple product engineer Annette Wagner gives a graphic artist's view of a conversation with the development team, and fantasizes about the ideal prototyping tool. It brought back my own experience when I used to sit in cover meetings for Dr. Dobb's. I would struggle to put my programming notions into some form that would evoke something for Art Director Mike Hollister. Mike, who had had less experience in dealing with programmers back then, would struggle to pull some useful idea out of me. Our prototyping tool was Associate Art Director Joe Sikoryak. Joe, an accomplished cartoonist, would quickly sketch what he thought I was talking about and show it to us. Usually it showed how graphically ridiculous my idea was, but even that moved the brainstorming process along. Wagner's prototyping tool seems to me like an attempt to bottle Joe after teaching him to program.
But the bulk of the book consists of articles that are not only about user-interface design but also at least partly relevant to developing interfaces for multimedia.
Game designer Chris Crawford shows that many of the innovations in user-interface design that will be heralded as multimedia takes off will have been implemented commercially years earlier by game designers. Sometimes the ease of use of game products doesn't translate to general-purpose computers: Don Norman compares Nintendo machines with current computers, to the detriment of computers, but it's not even clear how you would get to Nintendo-like ease of installation on general-purpose computers.
Apple Fellow Alan Kay interprets Marshall McLuhan. "Message receipt is really message recovery; anyone who wishes to receive a message embedded in a medium must first have internalized the medium so it can be 'subtracted' out to leave the message behind." "The medium is the message" means to Kay that, in order to decode the message, you have to become the medium. Scary thought.
The idea of computer as medium rather than as vehicle inspired Kay to introduce kids to computers before they take driver's ed. Maybe it's his work with kids that has kept him young and flexible; despite the fact that the Mac embodies a lot of his own ideas, Kay compares the continuing tweaking of the desktop interface with a biological organism attempting to live in its own waste products, a possibly apt but certainly unkind analogy.
Kay, an able analogist and metaphorist, is down on metaphor. "Metaphor," he says, "is a poor metaphor for what needs to be done." Agents now, he likes agents.
Agents run through the book like they do through an old Rocky & Bullwinkle episode, but movies also figure heavily in the plot. Ted Nelson explains "the right way to think about software design;" a way that touches on "the movie analogy." Joy Mountford of Apple's Human Interface Group reprises "the metaphor of filmmaker," and Paul Heckel's The Elements of Friendly Software Design (Sybex, 1991), with its copious analogies between filmmaking and software making, gets cited or quoted repeatedly in TAOHCID. These writers, though, are all talking about a movie metaphor or analogy. In fact, computers really are becoming tools for moviemaking. What about computer-controlled video and animation, and what about virtual reality? Laurel's contributors generally believe in these technologies; they think that multimedia has a lot to contribute to the user's experience.
Ronald Baecker and Ian Small of the Dynamic Graphics Project, University of Toronto, explain why animation should be thought of as a serious element in the design of the user's experience, rather than as a toy. Animation, they say, should be thought of as "movements that are drawn" rather than as "drawings that move." It's the dynamic aspect of the information that animation captures. One example they present is the animation of algorithms, showing how a sieve sieves, a bubble sort bubbles.
Gitta Solomon gives a corresponding argument for the virtues of color, and Gordon Kurtenbach, who was an Apple summer intern, and Eric Hulteen, ATG, do the same for gestures. No, we don't need to wave our hands to dismiss files, but gestures would provide a richness and naturalness of input that would be invaluable to composers, choreographers, sculptors, and designers of simulations. Joy Mountford and William Gaver, another summer intern, give the arguments for speech input and nonspeech audio output, and Chris Schmandt of MIT's Media Lab follows up with descriptions of several experiments in sound output: Phone Slave, the Conversational Desktop, and the Grunt system.
Grunt is interesting for what it shows about the possibilities of extremely limited speech recognition. Grunt responds strictly on the basis of the duration of the utterance, and in its domain of discourse this proves to be adequate. Users regard interaction with Grunt as remarkably natural, believe it or not.
As I mentioned, the themes of agents and multimedia run through this book, but they might not to be two distinct themes at all. Agents and multimedia could end up being aspects of a single approach to the design of the user experience, according to some of the contributors.
Nick Negroponte, director of MIT's Media Lab, talks about moving beyond direct-manipulation interfaces to interfaces that let users delegate tasks. "The desktop metaphor," he says, "is subject to serious change, soon." What Negroponte pictures is an interface populated with agents, but not necessarily the anthropomorphic bow-tied Phil of Apple's Knowledge Navigator videos. Negroponte sees the personalization of agents as a matter of user taste: "If you want your agents to wear bow ties, they will. If you prefer talking to parallelopipeds, fine."
Which sets Laurel up to give her views on agents. Laurel defends anthropomorphism, in a limited sense. Her argument is that as a species we have a large investment in some pretty subtle skills for predicting behavior from knowledge of character, and that it is entirely possible to give computer processes enough aspects of character that we can use these skills to predict their behavior.
There is a working example of some of this in Apple's Guides project. In this system, highly anthropomorphic entities serve as travel agents to aid users in navigating through a large educational hypermedia database. Tim Oren, ATG, discusses Guides in the book.
Beyond multimedia is the realm of virtual reality: datagloves, heads-up displays. Myron Kruger of Artificial Realities and Scott Fisher, Laurel's partner in Telepresence, sketch the benefits of VR and telepresence, while Autodesk president John Walker and author Howard Rheingold talk seriously about cyberspace (aka the Gibson Interface).
All of which leads--where? To the call for the creation of a new medium, according to Tim Oren. Or perhaps the recognition that what we already have is a medium, with all the associated problems like media bias.
In Computers as Theater, Laurel makes that assumption.
Ted Nelson has maintained for decades that software design has much in common with making movies. Since 1967 he has been pointing out techniques and concepts that software design can profitably borrow from film. Paul Heckel developed the parallels between the two fields in The Elements of Friendly Software Design, maintaining that software is mainly about communication, and the communication medium from which we can draw the most useful analogies is film. But another human-interface expert draws parallels with a more ancient art form: theater.
In Computers as Theater, Laurel presents a radically different way of looking at the interaction of humans and computers; so radical that she is entirely serious when she says that "the concept of interface itself is a hopeless hash" and that "we might to better to throw it out and begin afresh." Her model is theater, and she means to apply the model more seriously than Heckel or Nelson apply their movie metaphor. Because for Laurel, computers as theater is not a metaphor at all, but a model. Where Heckel presents a collection of D.W Griffith's moviemaking techniques and shows how they apply to computer-interface design, Laurel digs deeper, presenting a well-developed theory of the structure of drama and showing how it fits the structure of human-computer interaction.
The theory she presents is the theory of drama: Aristotle's poetics. Aristotle's theory has stood the test of time and is still used as a tool for understanding and criticizing the structure of drama. It has survived, the argument goes, because of its generality; its not a theory of acting or stage direction or any specifically dramatic activities, but a general theory of the structure of things interactive. What Laurel does with it is to derive a poetics of (computer-based) interactive form. That's her interface model, and it's extremely basic.
Because the model is so basic, it can be hard to see how it applies in specific cases, while in other cases can be too easy to slip into the obvious analogy. Laurel herself relies too heavily, I think, on examples that are too superficially cinematic, games in particular. But if I get her right, she's not talking about adding drama to software, but rather about a particular theory of the structure of interaction, a theory that ought to apply equally to the design, analysis, and criticism of software.
The example that Laurel is most prepared to offer is a rich source of data on interactivity and narrative techniques in human-computer interaction, but it is probably not ideal for exemplifying her theory. That's the Guides project she worked on at Apple, an experiment in the use of agents. In this project, the user is aided in exploring a rich database like a history of the American West by agents. The agents of the Guides project, unlike the frequently proposed on-line agents that would scan the nets for material of interest to you, represent their own interests. Each agent--the settler, the native American--is intensely personalized, down to the digitized photo of a person in appropriate costume. When consulted, they offer suggestions about next moves in the database and share anecdotes, both based on their own carefully construct biases and perspectives. When the user is examining material of little or no interest to a particular agent, its picture will change, slumping, falling asleep. Users can also construct their own agents.
The Guides project is an intriguing examination of subjectivity and point of view and the use of some very human and natural modes of interaction and the use of narrative structure--but I don't see that it says all that much about Laurel's theory. Possibly no current work in human interface can say much about the model, since it is a model of something that doesn't exit yet.
An interaction should be designed, according to Laurel, to have an identifiable beginning, middle, and end, and to take up no more than a few hours. These could be considered the minimum standards for getting into Laurel's game, and nobody is really designing software this way. In particular, adventure games traditionally violated this rule, packing stupid puzzles into the games to keep them going as long as possible. Games whose only exit is the symbolic death of the player violate another Laurel prerequisite: that an interactive session ought to leave you with something of value.
If nobody is playing Laurel's game yet, its not possible to point to good or bad examples. All current software is pretty much irrelevant. Computers as Theater, unlike the other books discussed here, is not immediately practical as a guide for developing software. It presents a new approach to thinking about software, and asks for a deeper intellectual commitment than the other books do.
But assuming she's on the right track, it should pay off big for those ready to make the investment.
Copyright © 1992, Dr. Dobb's JournalTog Soup
Blowing up Interface
Boris and Natasha
Bow Ties Optional
It's Just a Stage