Dr. Dobb's Digest March 2009
The Art of Application Performance Testing
Ian Molyneaux
O'Reilly Media
158 pp., $34.99
ISBN: 978-0-596-52066-3
Don't be fooled by the deceptively small page count of this book. Author Ian Molyneaux has distilled a wealth of hard-earned lessons into a set of checklists and bullet points that crystalize the majority of application performance testing principles in real-world practice today. Read on for a full review.
Coming in at just over 150 pages, The Art of Performance Testing is nevertheless a valuable read for any developer and/or tester bestowed with the responsibility of ensuring an optimal application execution experience. As is always the case in theoretically successful application construction and testing, performance expectations and realistic limits should be clearly defined before the first line of code is written. What distinguishes Ian's book apart from other, more verbose books on the subject is his advocacy for the performance testing business case and, in the book's third chapter, two case studies that show the lifecycle of such real-world testing outcomes.
The book opens with a chapter explaining the historical rationale behind performance testing, including the establishment of performance measures and outcomes, and why leaving performance testing to the last leg of an application development cycle is a really bad idea. This is followed by the largest chapter in the book and covers the fundamentals of application performance testing, the selection of performance measurement tools, constructing the test environment and setting realistic performance targets via business process owner consensus, key performance targets (such as availability/uptime, concurrency/throughput and response time), making sure an application is stable enough to test (obtaining a frozen code build, specifying the transaction list, and replay validation), and ensuring high-quality, accurate, and valid performance design and data capture.
Chapter 3 reviews best practice model approaches to performance testing via Proof of Concept (POC) checklists and form requirements via a six-step process. This chapter also reviews two real-world case studies, one for an online banking analysis and the other for a call center. Both of these are invaluable to see how the principles and techniques promoted by the author are put into practice.
The next chapter examines the recommended statistical and analytical procedures a tester should consider when interpreting the real-time and post-captured results. Linear, knee-spikes, standard distribution curves are some of the more frequent data views expected when monitoring Key Performance Indicators (KPIs). The chapter closes with helpful pre through post task checklists sure to make performance evaluations more meaningful and relevant.
The final chapter in the book presents a rapid overview of some of today's newest application frameworks and recommended ways for dealing with measuring their performance. Rich Internet Application (RIA) technologies like AJAX, Windows Presentation Foundation (WPF), Adobe's Flash/Flex and enterprise systems like SAP, Oracle, and SOA are very briefly touched upon. The book closes with 5 appendixes, spanning from transaction examples to KPI templates showing three screen grabs of performance trees, one which erroneously displays a duplicate of the Citrix MetaFrame tree instead of the intended Microsoft IIS tree. As of this review post, no replacement image has yet been posted on the O'Reilly errata web page for the book.
In summary, while The Art of Application Performance Testing is rather expensive for its short notebook on status, there are a couple of gold nuggets of hard-earned experience that can be uncovered in the book and practiced in your own application development lifecycles immediately. More cost-conscious readers may find plenty of free material with nearly as good checklists and recommendations by Googling the various performance testing blogs and even several articles on the Dr. Dobb's main website, but for those looking for a less time-consuming and more accurate, trustworthy source of practice and experience, this book delivers the goods in considerably less time and with greater impact.