The Perl Journal June, 2004
Scripting languages fill an uncomfortable niche between simplistic macros and full-out application development. This can be a tough spot to call home: A scripting language has to accommodate a user base that can often have a high proportion of neophytes; it has to interact with the wide range of applications and data that everyday users are most likely to encounter; and it has to be highly extensible to avoid obsolescence. These requirements can, at times, seem mutually exclusive.
Similarly, programmers using such languages are often faced with the seemingly contradictory goals of making an application simultaneously very simple and highly functional. Of course, this is true for programmers in any language, but in languages like Perl, we're expected to do it quickly and produce applications that solve some very thorny problems. We're also expected to produce applications in spaces where commercial apps don't exist or aren't very good. Often, there's no commercial app to do the job because the problem domain is just too specialized.
Sometimes "specialized" is really just another word for "messy." Messy problems are the ones that have lots of cases that are exceptions to the rule. Messy problems are hard to encapsulate and hard to model an object around. But Perl and other scripting languages are the champions of messy situations. That's what they're bred for. To some extent, this is our fate as Perl programmers: clean up the mess.
But that's not all bad. Automating very messy processes can have huge benefits for users, precisely because messiness is what makes these processes time consuming and unpleasant when done manually. Taking away that burden can really make you someone's hero—but it does place special burdens on the programmer. It means a long development cycle, because shaking out all the weird exceptions and boundary cases can take a long time. It can mean a complex user interface, or a complex configuration file. It can require especially extensive documentation. It certainly means paying special attention to that trade-off between an application's ease of use and its power to handle everything the user might want. When that can encompass a host of niggling details, feature creep becomes more than just a delay—it can utterly slay a project.
Finding the right spot on the simplicity/functionality spectrum has effects long after the project is (supposedly) finished. It has a direct effect on your support costs. An application that is complex to use will lead to phone calls from users. On the other hand, an application that doesn't take into account the complex needs of your users will never be used.
Sometimes, the biggest challenge in automating a messy process is just getting users to realize why the problem is so messy. Questions like "Why do I have to tell the program to do that? Why can't it just figure that out for itself?" reveal that often, users don't even realize that they were making conscious decisions when doing tasks manually. If you do something enough, the various decisions involved become completely unconscious. It's easy to forget that actual input must be given at those points. Ironically, it often isn't until people are faced with an application that automates their tasks that they realize just how complex their tasks actually are.
Kevin Carlson
Executive Editor
The Perl Journal