Dr. Dobb's Journal April 2001
The development of large-scale software projects across geographical boundaries requires rigorous configuration management. SCRAM, short for "Software Configuration, Release, and Management," is a flexible and scalable software-development tool designed for such a task. SCRAM provides configuration management, distribution, and software build services to ensure consistent and easy to use development environments that work in a multiplatform environment. This enables autonomous development groups to work together.
SCRAM is currently used in several projects here at CERN, the European Laboratory for Particle Physics. Projects range in size from one to more than 100 developers spread over some 50 different institutions in 35 countries. SCRAM (available at http://cmsdoc.cern.ch/cgi-cmc/scrampage/) is released under the GNU public license and is open to all. It is written mostly in Perl 5, but requires GNU make and a bash shell for the BuildSystem. System function calls are kept to a minimum, so porting to nonUNIX platforms, if required, should not be too big a problem.
One key function of configuration management is the art of keeping track of what you are working with. This means your compiler, operating system, external product libraries, environment, and the version of the source code. Only once all the components, known as "the configuration," are known is it possible to ensure that all developers are working on the correct thing and within the projects constraints.
In large projects, this can become a complex and tedious business and so SCRAM was developed to help out. The principle behind the tool is simple, if you don't explicitly ask for it, you don't get it. You create a toolbox with a well-defined set of tools, and then restrict yourself to that toolbox. In this way, nothing can sneak into your configuration (for example, because your system manager had installed something "useful" in /usr/lib) without you and your coworkers being able to reproduce it.
A source-code repository manager is a specialized tool for storing and versioning your source code. SCRAM is not a source-code repository manager. However, SCRAM requires that you have a source-code repository of some sort to define the source associated with a given release. The only manager SCRAM knows about is CVS (Concurrent Versioning System) (http://www.loria.fr/~molli/cvs-index.html), although it should be trivial to add others if required. (For more information on CVS, see "CVS Version Control and Branch Management," by Michael Simpson, DDJ, October 2000.) I run a CVS server with ssh access for my developers. It has been modified slightly to allow development group managers to manage their own repository access rights.
Figure 1 shows the main components required to define a SCRAM project. A project consists of a set of documents (see the accompanying text box entitled "SCRAM Documents"), which describe the structure and distribution of the project, external product requirements, and the BuildSystem configuration.
The BootStrap document is used to describe a project's contents, structure, and the servers from which source code and other components can be downloaded.
Using this BootStrap document as a guide, SCRAM can easily recreate a copy of the project on any site. Typically, there may be several BootStrap files for any release, depending on the target user group. One aimed at developers may contain instructions for all the source code to be downloaded from the repository, others may be aimed more at the end user and only export binaries and libraries.
Web-based click-to-install functionality is made available by attaching a mime-type when distributing the BootStrap document so that your browser can automatically route it through SCRAM. Currently, this is limited in that SCRAM cannot automatically install other products on which a project may depend. It can, however, give you information and URL pointers to point you in the right direction and tell you exactly what you need.
All the components that make up a software project such as source code, libraries, binaries, environment, and the like, are assembled together in a directory structure called a "Project Area." There are two types of Project Area — the central release area (CRA) and the developer area (DA); see Figure 2.
Typically, a single central release area, containing all the code, binaries, and so on, corresponding to a fixed release of the project is installed at each development site. It is a central release area that is created by SCRAM when it reads the BootStrap document. Once installed, the central release area remains untouched and serves as a reference point and source for all developers and users at the site.
Once the CRA is installed, developers are free to create a developer area based on a particular CRA in their own workspace. Each developer area provides an independent sandbox in which he can make incremental changes to every aspect of the base release, without affecting anybody else.
It is important to realize that the project area is the defining context for the SCRAM tool. This lets developers easily shift between different projects or different versions of the same project by simply moving to a different area. As each Project Area has its own environment, there's no need for messy and error prone editing of .cshrc or .profile to accommodate different or even conflicting project requirements.
A slight exception to the general rule is the run-time environment — the environmental variable settings required when you wish to run a binary generated by the project. These do not become a part of your environment by simply entering the Project Area, you have to ask for it explicitly. This allows you full control over your environment, depending on your task inside the directory. An administrator, for example, who just wants to delete files or perform other administrative tasks may well not take kindly to having their entire environment redefined.
A history mechanism exists to record the changes made by SCRAM to your environment. Should you switch to a different project area, the environment is easily purged of the effects from the previous area.
Run-time environments for specific applications or use cases can be described in special Runtime Documents. SCRAM can read these and convert to bash and csh forms as required. The document format should encourage proper documentation of these variables and their settings.
The Requirements Document, which is downloaded by the BootStrap document, is used to define the tools that go into your project toolbox. It can also contain information concerning distribution and access of the tools, such as special licensing arrangements and so on.
Figure 3 illustrates how you would fill your toolbox. Requirements are specified by the name and version of the product, each of which needs to be associated with a tool description document — a ToolDoc.
The ToolDoc, which is also automatically downloaded and parsed by SCRAM, contains information about the tool, such as:
This information is used by SCRAM to integrate the tool into the project environment as well as to provide information for the human user.
During the setup phase, SCRAM probes your system, looking for the specified tool. If it can't find it, it will ask you where it is. This information will be stored in the project area, so that it can be easily looked up whenever the tool is to be used. SCRAM is smart enough to check other central release areas it knows about for tool-specific information, so once you have told SCRAM about a tool, you rarely have to tell it again.
ToolDocs can be maintained independent of any given project, ideally by experts on the tool. By using well prewritten ToolDocs, you can easily integrate new tools into your build environment. You do not have to worry about platform specifics, dependencies, include paths, link lines, and so on.
It is a requirement in many of our projects that they use the same components, such as databases and other tools. What these should be has to be agreed upon and enforced across all the projects concerned.
To simplify the management of this constraint, a separate Requirements Document is maintained independently that specifies the agreed configuration. Each project includes this configuration in their own Requirements Document using the SCRAM document preprocessing functionality activated with a simple <INCLUDE url=...> tag. Functions exist to refine this selection to that appropriate for the project; that is, remove unused components.
As the Requirements are specified in an XML-like document format, it is relatively easy to write a CGI script that converts this to HTML. Example CGI scripts are provided in the SCRAM distribution; these check out the appropriate version of the Requirements Document from a CVS server, perform the HTML conversion, and display the requirements for any given release of a project. Figure 4 is an example of such a page.
There is no maintenance overhead to have such a page and the chances of a transcription error are greatly reduced. Such a page is useful as it lets people see what they need to have on their system before they try to install a project.
The SCRAM BuildSystem was developed primarily to support large C/C++ and Fortran projects. Its underlying philosophy is to separate the logical aspects of the build from the implementation details. As an example, you would simply ask for a library to be built from a set of sources; you do not need to worry about how to actually build a library on the various platforms.
I only intend to give an outline of the concepts and functionality currently provided by the BuildSystem. For more details and examples, see http://cmsdoc.cern.ch/cgi-cmc/scrampage/.
SCRAM, naturally enough, meets the needs of the projects in which it was born. There are many minor improvements to be made to facilitate use, but the main task is to revamp the BuildSystem, so that it becomes a much more flexible and powerful tool.
The plug-and-play interface between the BuildSystem and a ToolDoc for complex generation tools such as compilers has not yet developed to a very sophisticated level. All the hard-coded rules for the build classes such as libraries, binaries, and so on, should be replaced and moved into a ToolDoc description to improve modularity and flexibility.
Take, for example, your C++ compiler. You would ask for it as you would any other tool in SCRAM, through the toolbox mechanism. The C++ ToolDoc would, among other things, describe that it can make shared objects, and this is a type suitable for inclusion in the library build class.
By plugging in tools in this way, the functionality of a build class, and even the types of classes available can grow to whatever is required. All this happens without touching your BuildFiles as the logic of the build still remains the same. You still need libraries; it's just now possible to produce many different types, such as instrumented or debuggable versions.
Switching between compilers and adding new languages would become a trivial matter of loading the appropriate ToolDoc into your toolbox.
SCRAM versions have a major version of 0; for instance, V0_18_0. The major SCRAM Version, V1_0_0, is reserved for the SCRAM release that has the functions described earlier.
Other improvements to the BuildSystem will be autologging and active error reporting of builds. Automated statistics gathering, noting such things as average compilation times, number of errors in regression testing, and so forth should also be implemented for use in the software process improvement phase.
DDJ