Article nov2006.tar

Make Room on Your Bookshelf

Peter Baer Galvin

With all of the online information available these days, what is the purpose of books? I find that good books go into great detail on their topics, provide useful background information, and show the logical interconnection of their components. I find that the best books also link to information online to provide further information, related information, and to save typing (in the case of code). That is exactly what readers get with the new two-volume Solaris Internals books.

Contents

This two-volume set (Solaris Performance and Tools and Solaris Internals, Second Edition) does not actually replace the previously very useful Solaris Internals from 2001. In spite of the nearly 1500 pages in the two new books, they are not a superset of the old edition. Pre-Solaris 8 internals are available in the first edition, while Solaris 8 through Solaris 10 coverage is found in the new volumes.

The Solaris Internals, Second Edition volume is by far the larger of the two. It delves into the design principles and implementation details of Solaris versions 8 through 10, including broad coverage of:

  • Processes
  • Scheduling
  • Interprocess communication
  • Process rights
  • Zones
  • Resource controls
  • Memory management
  • File systems
  • Platform specific components
  • Networking
  • Kernel services
  • And kernel debugging with kmdb

There is even an appendix describing how to add a system call to Solaris.

For each of the topics, the authors provide detailed analysis of the algorithms and kernel implementation of those algorithms. They frequently show code fragments and appropriate header files. Where appropriate, they explore the kernel aspect using user-level commands, such as the proc tools and the mdb debugger.

Solaris Performance and Tools is more oriented toward exploring a system from outside of the kernel rather than inside of it as its bigger brother does. The book includes coverage of:

  • CPU
  • Processes
  • Disk Behavior
  • File Systems
  • Memory
  • Networks
  • Performance counters
  • Kernel monitoring
  • Dynamic tracing
  • Kernel statistics
  • MDB

The book also includes appendices on tunables, DTrace one-liners, Java DTrace scripts, and sample Perl Kstat utilities.

Its subtitle, "DTrace and MDB Techniques for Solaris 10 and OpenSolaris", is a bit misleading. As powerful as those tools are, the book certainly goes beyond them in showing how to analyze system performance and locate and diagnose performance issues.

Consider Chapter 4 on disk behavior and analysis. Here authors McDougall, Mauro, and Gregg discuss all aspects of disk performance. They show random and sequential I/O examples, detail response-time theory and fact, talk about maximum I/O sizes, and thoroughly explore the use of iostat in all its gory detail, including the all-important issue of what the various output fields really mean. Next, they show disk performance data from kstat, sar, and several DTrace scripts, including examples from the downloadable "DTraceToolkit" and the graphical "DTraceTazTool".

Utility

Some of the extraordinary usefulness that Solaris Internals, Second Edition brings is its interconnection to the Solaris source code that is now available at: http://www.opensolaris.org.

For each exploration of the internals of Solaris, there is a link to the online source code that is being described. As good as the first edition of this book was, the value of this new book is multiplied given the availability of the source and the correlation between the source and the book. Of course, there are also the expected links to appropriate documentation.

While all of the provided information is useful to kernel programmers, it is also useful to anyone who programs on Solaris 8, 9, or 10 systems. Knowledge of how the kernel does what it does is crucial to writing efficient code and to finding out what is inefficiently written. The book is also useful for systems administrators for its ability to shine a light into those dark system crevices where performance problems hide and to provide illumination of what the kernel is doing with respect to any given aspect of Solaris.

Solaris Performance and Tools is the best single book for analyzing system performance and debugging performance issues on Solaris 8, 9, and 10. Of course, it is especially useful on Solaris 10 because it shows how to use the new tools to gain insights that previously were difficult or impossible.

The books together come with impeccable pedigree, given the acknowledged expertise of the authors. And, although the text sometimes gets deep and technical, the authors never forget that they are writing for humans, and non-Solaris kernel engineer humans at that.

Every book leaves wishes unfulfilled, and these books are no exception, although the wishes are few and far between. Due to time constraints and deadlines no doubt, ZFS is not covered. ZFS is the revolutionary new file system/volume manager that shipped with Solaris 10 update 2. It was covered in previous Solaris Companion columns, which you can read online at: 

http://www.samag.com/documents/sam0602j/
and: 

http://www.samag.com/documents/sam0603h/
There are also quite a few useful documents at: 

http://www.opensolaris.org/os/community/zfs/docs/
including one describing the ZFS on-disk format. The new fsstat command is not included in the books either, which is no surprise given that it hasn't even shown up in an official Solaris release yet.

Final Thoughts

The Web site associated with the books is: 

http://www.solarisinternals.com
Here the authors provide links to their blogs, sample code, downloadable "cool tools", and even a podcast. There are future plans to allow wiki-based community contributions to the Solaris internals content, which would be a great addition to Solaris and OpenSolaris knowledge should it come to fruition.

In total, the two books reviewed here present a new level of knowledge about the internals of Solaris, what they do, how they behave, and how to analyze that behavior. The books are a must for developers, system programmers, and systems administrators who work with Solaris 8, 9, or 10. They are especially useful for users of Solaris 10 and OpenSolaris because of their exploration of the new tools in those releases. These books receive my highest recommendation.

Peter Baer Galvin (http://www.petergalvin.info) is the Chief Technologist for Corporate Technologies (www.cptech.com), a premier systems integrator and VAR. Before that, Peter was the systems manager for Brown University's Computer Science Department. He has written articles for Byte and other magazines, and previously wrote Pete's Wicked World, the security column, and Pete's Super Systems, the systems management column for Unix Insider (http://www.unixinsider.com). Peter is coauthor of the Operating Systems Concepts and Applied Operating Systems Concepts textbooks. As a consultant and trainer, Peter has taught tutorials and given talks on security and systems administration worldwide.