The Perl Journal November 2002
Extending and Embedding Perl, which covers both writing Perl extensions in C and calling Perl from within C, is a significant addition to the literature of one of the world's most widely used programming languages. Authors Tim Jenness and Simon Cozens demystify in precisely measured and carefully annotated terms the most arcane aspect of Perl, that of integrating Perl at the compiler level with other languages, primarily C, but also Fortran. If you already know how to script in Perl, have written modules or want to write your first module, and want to learn how to write modules in C or Fortran or call Perl from C routines, this economical volume is your best and pretty much your only written guide outside of the documentation bundled with the Perl interpreter.
The book is divided into two parts: The first part focuses on extending Perl, the second on embedding Perl. The reader's progress is methodical and orderly. The book starts with creating a Perl module in Perl, teaching just enough C to allow the same module to be written in C and called from Perl. After that, we're fully immersed in Perl internals and the tools Perl provides to match up those internals with the requirements of external languages. Everything is explained in text, with plenty of compilable examples and tidy diagrams that show the layout and relationships of data structures in a familiar and readable style of notation.
The chapters forming this extraordinarily self-contained book are:
1. C for Perl Programmers
2. Extending Perl: An Introduction
3. Advanced C
4. Perl's Variable Types
5. The Perl 5 API
6. Advanced XS Programming
7. Alternatives to XS
8. Embedding Perl in C
9. Embedding Case Study
10. Introduction to Perl Internals
11. Hacking Perl
followed by an appendix on Perl's typemaps, a bibliography, and an API index.
Authored by two individuals who have years of experience in writing published Perl modules and interfacing Perl as part of critical real-world applications, Extending and Embedding Perl is characterized by profound expertise, attention to detail, literate writing, and excellent editorial and production values. This book is so necessary that it would be a must-have for the problem domain, even were it less finely crafted; we're lucky that the one book to adequately address the most puzzling facets of Perl happens to be a masterpiece that will keep its value for years.
The publisher is making the source code for the examples available via the Web. The downloadable package is still being polished as I write, but the package provided for review was accurate, and the whole should surely be complete by the time you read this. The book's web site is http://www.manning.com/ jenness/, where you can buy the electronic version of the book online and discuss Perl with author Tim Jenness in a forum with readers.
Tim Jenness is 31. He got his Ph.D. from the University of Cambridge at the age of 24. We spoke with Tim by phone on September 26, 2002.
TPJ: What's it like being a computer programmer in Hawaii?
Tim: It's great. I work at the Joint Astronomy Centre in Hawaii (http://www.jach.hawaii.edu/JACpublic/index.html). I am the manager of the JAC high-level software group, responsible for all the high-level software (observation preparation, data processing) at the James Clerk Maxwell Telescope and at the United Kingdom Infrared Telescope, both on Mauna Kea. We have only three or four people in the group and you're allowed your own design, implementation, and testing.
TPJ: Between you and coauthor Simon Cozens, who wrote which parts of the book?
Tim: It's about 55 percent to 45 percent my favor, which is why I'm credited first author. I concentrated on extending Perl and Simon described embedding Perl. Simon got the idea, but he had already written a book, and there was no way he was doing that again on his own!
TPJ: I've done XS and I have never seen before an adequate book on this subject. Is there one other than yours?
Tim: The O'Reilly Associates book Advanced Perl Programming (http://www.oreilly.com/catalog/advperl/) had two chapters to cover everything, and that was four years old. Beside that, just the man pages for Perl. There was clearly a niche. I'm not sure if that was because no one had thought of it before. Maybe no one was brave enough before.
In 1995, I had to interface one of our esoteric data-reading I/O libraries. It was not practical to write a big C program just to list out headers and write a little table on the screen. I felt the pain as I learned how to do the interface. I tried to remember that pain when writing the book.
TPJ: In the book, you teach the Perl programmer just enough C to do the tasks in each chapter.
Tim: In the early stages, people asked, "Why are you doing this?" But if you are a Perl programmer, there are similarities that will hurt you when you go to C, that make you wonder why your code is not working.
The goal was that after Chapters 1 and 2, if you have a simple library with a C interface, you can build the Perl interface to it. Initially there was an objection to splitting C coverage as occurs in Chapter 2, but I felt I should give the reader enough C to do a simple interface without having to worry about pointers. I was careful that Chapter 2 doesn't use any of the things—arrays, pointers, strings—that are covered in Chapter 3.
TPJ: It's an exceptionally useful and readable book.
Tim: We could have written a book twice the size, but the theme is to get you in the door.
TPJ: It seems most people in the world are programming in C++, Java, and/or Perl.
Tim: We use Fortran here quite a bit—there are so many scientific libraries we have to use. The JCMT is from the 1980s and the UKIRT is from the 1970s. We still have control systems from 1988.
The users don't see this anymore. Everything the user sees nowadays is on Linux. The UK astronomical community has been using an object-messaging system called ADAM and later DRAMA that they wrote for themselves in the late 1980s. DRAMA can be downloaded off the Anglo-Australian Telescope web site (http://www.aao.gov.au/). You send messages out to tasks on remote systems and they come back and tell you what they are doing. It feels like CORBA. You provide arguments and get back data, synchronously or asynchronously.
In our new stuff we use XML and SOAP and translate to our low-level interface just before the stage where the telescope gets moved. From the user's point of view, you're coming in and using a Linux box with a Java or Perl app. You don't really notice the interface.
TPJ: Do you see much use these days of embedding Perl, of C calling Perl?
Tim: Simon did the embedding because XS is what I knew. But even in XS I find myself doing embedding. Our interface features callbacks, so the C bodies of methods launched by Perl XS have to call back into Perl.
TPJ: So C calling Perl is the yin to the yang of Perl calling C?
Tim: It's an XS module that farms the message off to an event loop in C, which calls back to Perl with the answer. It's a question of, "Who's embedding who?"
TPJ: What's the hot topic in Perl these days?
Tim: I suppose, "Should you be working on getting Perl 6 out the door or on maintaining Perl 5?" They're reengineering the internals so that the back end can be used to parse things like Python natively. Most people seem to be thinking, "Perl 6 sounds interesting, but we can't afford to worry about it yet."
TPJ