Dr. Dobb's Journal November, 2004
Embedded projects share many of the same activities as traditional desktop or server projects. The differences, however, require special tools that are either not normally found or are poorly supported in traditional, general-purpose IDEs. The lack of specialized tools integrated into one environment forces some engineers to rely on command-line tools, effectively using the shell as the IDE. Others divide their efforts between general-purpose tools and the command line to get the job done.
Seeing the need for the productivity boost offered by integrated development environments, most companies offering embedded operating systems also sell tools to bridge the gap between general-purpose solutions and specialized needs of embedded engineers. Some companies rely on the command line as the integration point, creating special command-line utilities; others have gone the route of creating proprietary IDEs for their operating systems.
Companies that create with projects that use their own embedded operating systems, or opt not to use an operating system at all, gravitate toward roll-your-own tools. These tools are typically created as a side effect of the development effort on the OS itself. As a result, the tools created are usually very specialized and can't be used with other embedded OS solutions.
With the rise of Linux in the embedded space, the standardization of the operating system gives users more choice in selecting development tools. Even organizations that prefer roll-your-own operating systems look to Linux as the base of their systems, modifying it to suit their needs. In short, users are no longer bound to tools offered by vendors of an embedded solution or those created in house.
TimeSys, the company I work for, creates tools for engineers who use any distribution of Linux in their embedded projects. At first, TimeSys took the approach of building tools from a proprietary base. While developing from scratch gave TimeSys full control, too much time was spent reinventing the wheel in terms of project management, text handling, interacting with compilation, and code-generation tools.
TimeSys also found extensibility was important for customers. Not only did tools need to be easily extended, but a reasonably large pool of extensions needed to exist as well; otherwise, the extensibility wasn't particularly valuable. For a large pool of extensions to exist, the platform needs to have a large number of users so that extension developers can produce code appealing to the largest possible audience of potential users. So TimeSys directed its attention to platforms with a large user base or those deemed capable of attracting a sufficiently large number of users to sustain an ecosystem of extensions.
The need to minimize the recreation of commodity features and to provide an extensible environment led TimeSys to investigate tools that could serve as the foundation for the IDE. This foundation needed to be open enough to let TimeSys customize to customer needs, but contain enough features so that TimeSys did not have to write/recreate basic features. The platform also had to be used by enough people so that a reasonably sized extension ecosystem existed or would be likely to form.
While evaluating new technologies for a next-generation IDE, TimeSys found the Eclipse project. At the time, Eclipse was newly released to the open-source community and offered a great Java IDE. However, the CDT project for C/C++ development was still in its infancy. Nevertheless, TimeSys saw the potential of the tool and determined that the project would eventually attract the critical mass necessary for success, primarily because of its:
After carefully evaluating Eclipse with the eye of an embedded engineer, TimeSys determined the new features to add to solve customer problems. Our goal was to provide tools serving the full spectrum of Linux development, from board bring-up to packaging and deployment. To provide these features, we built the TimeStorm development toolsuite on top of Eclipse and CDT. (CDT provides a C/C++ development environment, analogous to the Java Development Toolkit, in the Eclipse framework.)
Embedded development almost always involves cross compilation; that is, compiling code for a system is different than that used for development. Using cross compilers introduces another layer of complexity when creating a makefile: Users must override many of make's standard macros and rules to invoke the cross compiler with the correct parameters. Engineers must also ensure that the tools are on the system path or specify the absolute name of the compilation tool.
When TimeSys first evaluated CDT, it did not contain any build facilities—users were responsible for maintaining a project's makefile. Considering the effort involved in maintaining a makefile, TimeSys created an Eclipse plug-in that introduced the concept of tool chains and build configurations (Figure 1). Tool chains encapsulate a set of programs necessary to compile and manipulate a program; for example, a full installation of gcc includes a compiler, preprocessor, linker, archive, and debugger along with some other helper programs, typically referred to as "binutils."
After registering a tool chain in TimeStorm, users can refer to it in a build configuration. The build configuration contains a tool chain and the compilation options. In conjunction with the files in the project, TimeStorm generates a makefile with references to the appropriate tools and options from the build configuration. All of this work is hidden from users; TimeStorm treats the makefile as an implementation detail.
Users can specify as many build configurations as necessary for a project, and switch between the build configurations at will. This is an important feature for embedded engineers, who frequently switch between targets during development to avoid the performance overhead of remote debugging.
Engineers spend most of their time debugging. While the base CDT package includes Launch Configurations for debugging, they only accommodated debugging on the development host. Consequently, TimeSys added specialized Launch Configurations to Eclipse that collected information from users such as:
With all of this information, TimeStorm can do the drudge work of downloading the files to the target and starting a debugging session without user intervention. Because Eclipse has a function in the user interface for reusing the last Launch Configuration, users can begin a remote debugging session with just a single mouse click.
Under the covers, TimeStorm relies on much of the CDT framework for the debugging functionality, which in turn, relies on GDB as a debugging engine. When debugging starts, CDT executes GDB in a separate process and interacts with GDB via the machine interface (MI). When running GDB in MI, interaction still occurs through the console. However, the commands and results have been changed to be easily parsable by a computer rather than a person. (If you're curious, you can see what this looks like by starting GDB with the --interpreter=mi parameter.)
TimeSys created its own launch delegate class (based on AbstractLaunchDelegate) so that the correct commands are sent to GDB during the start of a debug session to make contact with a remote host and used this in place of the CDT launch delegate. After making the connection, GDB uses the same commands as debugging locally.
In the absence of TimeStorm's Eclipse plug-ins, users can download the necessary files to the target, usually via a script, and launch the debugger. Most users opt for a graphical front end to GDB such as DDD or Insight. These tools work well with alternate GDB engines, but making the connection with the remote host requires access to the GDB console.
Kernel and root filesystem configuration are important parts of creating a working embedded project. Configuring the kernel involves removing functionality not necessary for the application to minimize the memory resources required by the kernel. Root filesystem configuration has pretty much the same goal—removing what's not necessary. Engineers working on desktop or server applications rarely need to worry about the root filesystem configuration, other than checking at installation time that any dependencies are satisfied.
Eclipse and CDT do not offer any features for making a root filesystem or doing kernel configuration. While robust tools exist for editing the kernel configuration, none exist for creating a root filesystem. Without root filesystem configuration tools, users must perform this tedious and time-consuming task by hand.
As a matter of convenience, users can tie together a root filesystem and a kernel configuration into a system configuration. With system configurations, users can easily recreate a system configuration from scratch. TimeStorm offers an editor (Figure 2) that lets users pick and choose the files that should appear in the root filesystem.
When users are satisfied with the kernel and filesystem configuration, it can then be built into one of several formats suitable for distribution on the board. Distribution for embedded software usually means "burning" the image into flash or other nonvolatile memory on the board. With TimeStorm's RFS configuration utility, the root filesystem can be built as a Journaling Flash File System, Version 2 (JFFS2) image. Users can then place that image on the flash media via the utilities included with the target board or by running Linux on the host and accessing the flash media through a driver.
The root filesystem configuration editor lets users select the files that appear in the root filesystem of the embedded Linux target. The root filesystem consists of the files and directories mounted under the "/" mount point in Linux. Depending on the resources of the target computer, the root filesystem can hold the contents of a typical desktop system. For memory constrained targets, engineers take care to make sure only the necessary software gets copied into the root filesystem.
Creating a root filesystem from scratch can be a complicated process, as users must include the files and directories expected by the system initialization process; for instance, the first program the kernel executes is /sbin/init, and /sbin/init looks in a file /etc/inittab to figure out what processes to start running. While some embedded systems bypass this standard start program, the root filesystem still must contain any directories used for mount points as well as utilities and libraries required by the application.
TimeStorm includes an integrated editor that allows users to pick and choose "packages" to be placed in the root filesystem. After selecting the packages, users can then reduce the root filesystem size by excluding files not necessary for the embedded system. For instance, many of the packages include some documentation (typically man pages) that can be removed depending on the anticipated usage patterns of the device. TimeStorm keeps track of the dependencies between packages and warns users if a file has been removed that another package will need in order to function properly.
The architecture of TimeStorm lets users create their own packages as well. Being able to create packages means users can use the root filesystem configuration editor with any software. Users take advantage of this feature when they need to add software from the open-source community or customer software developed in house or by a systems integrator. Customers also use this feature to create packages containing the binaries of their project, so they can use the tools to include their binaries into a root filesystem with the tools.
With the advent of the Linux 2.6 kernel configuration tools, TimeStorm integrated kernel configuration directly into the Eclipse environment. TimeStorm's kernel configuration editor presents the user with an interface matching that of other multipane editors in the Eclipse environment. Linux kernels prior to 2.6 continue to use the text-mode kernel configuration editor.
Because the kernel configuration tools are robust, most users could probably use them outside of the TimeStorm environment with adequate results. TimeStorm offers users a convenient way of launching the tools from within the IDE. However, the fully integrated kernel configuration editor for 2.6 kernels offers users the advantage of staying in a familiar environment.
Eclipse with CDT can be used to as a platform for doing kernel development in its present condition. The code-browsing and project-management features make it easy to get a high-level view of the kernel project and easily navigate around the project. Having an IDE for kernel development greatly reduces the barriers for engineers not accustomed to working from the command line. This is one area where the lack of makefile management does not matter: Because the kernel comes with a sophisticated make system that works rather well, all users need to do is ensure that the build file calls the targets in the right sequence, the cross-compilation tools are in the path, and the RAM disk image is in the right place (if one is to be used) before building.
To reduce this complexity, TimeStorm contains feature projects where users can specify the cross compiler and, optionally, an initial RAM disk with a filesystem. The extensions use this information to build the kernel by executing make with the correct targets in the correct order.
The Eclipse and CDT projects provide an excellent C/C++ development environment. However, by themselves, these projects don't address the specialized requirements of the embedded engineering community. But by using Eclipse and CDT as a starting point, tool vendors can provide embedded engineers with an outstanding IDE, thereby concentrating efforts on the unique needs of embedded developers.
DDJ