I think that the practice of sending out "reviewer's guidelines" along with review copies of a complex product is a good one. Of course such guidelines are biased -- slanted toward the features of the product that the vendor thinks are most likely to impress the reviewer, engineered to draw the reviewer's attention away from the aspects of which the vendor isn't so proud. But I believe that they also serve two commendable purposes.
First, they tell the reviewer what the vendor had in mind in developing the product. This is sometimes far from clear. Reviewer's guidelines say, in effect, "Here's what we think we've accomplished; here are the features on which we want to be judged." With some products, it really does help a reviewer to know what the company had in mind; consider, for example, B&E Software's RagTime, an integrated software package for the Macintosh.
B&E is now marketing the third version of RagTime in the U.S. The European company faces the negative impression created by two less-than-impressive attempts to sell the product here through intermediaries. It faces the challenge of selling an integrated do-all application for the Macintosh at a time when Apple is pushing the idea of small, tool-like applications that share functionality. And it faces the problem of any integrated package: None of its included applications is the best in its class. If RagTime gets reviewed on the basis of past perceptions, or of Apple's vision of the future, or of its components, it will not fare well. Reviewer's guidelines would let B&E define the terms on which it wants RagTime to be evaluated.
The other virtue of reviewer's guidelines is that they virtually force marketing and engineering to communicate. As we have all seen demonstrated, a press release or advertisement can be written with almost no knowledge of the product. Not so with reviewer's guidelines. Before you start suggesting to reviewers what sort of benchmarks are appropriate in your product's category, you'd better know something about the product's performance.
I used the term "functionality" above with some reservations. When I was at InfoWorld in 1981, we argued at some length about whether or not there was such a word. We were not alone in disliking the word: Ted Nelson fumes about it in Computer Lib. But the word does exist and is used, because it's needed. It's ugly, but it's right.
I wonder, though, if software reviews today are using all the right words. In 1974, features and performance were preeminent. Design and depth are arguably more important today, but those words don't often appear as subheads in reviews.
Trip Hawkins of Electronic Arts has spoken eloquently about the need for software to be deep: Capable of being used for years, with new folds to discover, and thus a long shelf life. Naive users have different needs from experienced users, and a software product ought to have value for both groups. HyperCard is an extraordinary case study in depth: All the way from cut-and-paste application generation to the brink of writing utilities in C. No, HyperCard doesn't teach C, but everyone who gets deeply into HyperCard is strongly tempted to write an external command, and HyperCard's external command interface takes away a lot of the pain of learning to program the Mac.
Perhaps reviewers ought to give more attention to such subjective issues as depth and design, and less to those gargantuan tables of features and benchmark results.
Then again, objectivity is safer than subjectivity.
On June 21 of this year, the Supreme Court removed the opinion defense in libel suits. Writers of newspaper or magazine opinion columns or of letters to the editor, cartoonists, restaurant and software reviewers, can all be sued for libel if the opinions they express cause damage to someone. Negative software reviews, it goes without saying, cause damage to the company whose product is reviewed. The plaintiff has to prove that the damaging statement was not true, but legal experts predict that a lot of suits that would never before have been considered will now not only be filed, but will actually go to trial.
The same legal experts predict that this decision will cause commentators and reviewers to be much less outspoken in their views. The software reviewers of my acquaintance do not seem to be modifying their behavior in response to the ruling. They may be sorry.
Under the circumstances, I think I'll reserve my opinion of the present Supreme Court justices. I don't think we could print it anyway.