Cross-Platform Development

Using Shared Libraries across Platforms

Amal Shah and Hong Xiao

Shared libraries are an important technology with an unfortunate lack of standardization among various compilers.

Introduction

Many operating systems today provide a mechanism for linking binary modules to an executable at run time, and for sharing them with other executables. These modules are known as shared libraries, or dynamic link libraries (DLLs). A variant of the shared library is the Dynamic Shared Object (DSO). Shared libraries have become fairly common in large application development. Applications are often assembled from components that provide different functionalities, and these components more often than not reside in shared libraries. Hence, whether you are developing these components or are building an application containing several shared library components, it's important to understand how to use shared libraries across platforms, and to be aware of the issues you may encounter using them.

This article presents several issues related to using shared libraries across platforms. While this article is Unix-centric, we do provide comparisons with platforms such as Win32, OpenVMS, and the Unix OS/390 (Open Edition). (In this article, we refer to the PC platform using Windows NT and Windows 95 as Win32.) The issues we cover can be split into compiler issues, related to compiler flags that may impact shared library creation; linker issues, dealing with the plethora of options that control shared library behavior, such as exporting symbols, versioning, linking with an executable, etc.; and run-time issues, related to environment variable settings, debugging tools, and performance issues. Table 1 (divided into parts a and b to fit formatting constraints) details the platform-specific options.

Advantages and Disadvantages of Shared Libraries

The output of compiled source code is object files. The linker may convert the file into either an executable, a static archive, or a shared library. Archives and shared libraries provide a good way to partition your application code based on its logical structure. However, linking object files to an executable or using static archives results in large executables for larger applications, which may impact startup times. Also, multiple copies of this application may be loaded concurrently into memory, thus impacting system performance.

Advantages of shared libraries include:

Disadvantages of shared libraries include:

Thus, in many cases a self-contained executable linked with archive libraries may be preferable. Libraries such as the C library, and other system libraries, are often available in both forms.

Compiler Issues

On most Unix systems, code used in a shared library must be explicitly compiled in a position-independent code (PIC) manner, typically through a -PIC flag. On some systems, such as Win32 and Digital Unix, this is the default (table entry 1). PIC compilation causes each function call to be resolved through a linkage table or a global offset table, and results in a small performance penalty. Code compiled with the -PIC flag will not mix with non-PIC code. On the OS/390, shared libraries must be compiled with the -DLL flag, because the compiler inserts special code sequences for exported data and functions. Because of this special code, there are restrictions in making calls via function pointers and passing data between DLL and non-DLL code on OS/390 systems.

Linking a Shared Library

The linker creates the shared library. It inputs object files, shared libraries, or archives, and outputs the shared library. Typically, shared libraries contain two parts: a shared area and a private area. The shared area contains shared code and read-only data, which is shared among all processes that use the shared library. The private area contains read-write data and internal bookkeeping information such as the linkage table and a list of other shared libraries. When creating an executable to be linked with shared libraries, the linker does not copy the shared library code or data into the executable. Instead, it binds the executable's references to symbols in the shared library to entries in a linkage table, and the loader performs the complete resolution at run time.

Linker flags play an important role in how the shared library is created and how it will behave at run time. Most linker flags are unique to that platform's linker. The most important flag, of course, is the flag that tells the linker to create a shared library instead of an executable or an archive library (table entry 2). Typically, there are flags that control:

Symbol resolution can be a tricky part of creating shared libraries. Most Unix linkers by default allow unresolved symbols in shared libraries. Unresolved symbols can lead to surprises (i.e., crashes) at run time if a particular symbol is not defined at all, or is defined differently in the application. Similarly, symbols with multiple definitions between libraries can lead to side effects such as unintended functions being called at run time. Most systems provide flags to allow, disallow, or warn about multiple symbol definitions and undefined symbols during link time (table entry 6), which is especially important if your application links with several third-party libraries. Although not always possible, using a strict linking model to create self-contained shared libraries can help prevent several hard-to-debug run-time problems resulting from symbol collisions between libraries. We'll discuss this issue later in the run-time section of this article.

Unlike most Unix linkers, the linkers on VMS, Win32, AIX, and OS/390 are stricter, in that every symbol must be resolved through import files or libraries. On these systems, linking a shared library creates two files: a binary shared library and an import file or library (a .LIB file on Win32 or a side-deck .x file on OS/390). An ASCII text export file (such as the .def file on Win32 or the .exp on AIX) allows you to control the symbols exported from your library. Most Unix systems provide the equivalent of the Win32 .def file via linker options to export or hide symbols from the library (table entry 7). Porting large applications from forgiving linker systems such as Solaris to strict systems such as VMS or Win32 can be painful if you have not properly managed symbol dependencies and exports between libraries.

For example, suppose you write a function foo(int) in C:

long gData;
int foo(int i)
{
   int j;
   j = i +5;
   return j;
}

If you want to export both function foo and variable gData from a Win32 shared library, you must include a .def file which contains the following EXPORT foo statement:

library myLib
exports
foo    @1
gData    data

But exporting symbols is a little different on Unix. Unlike Win32, most versions of Unix do not require a .def to define export symbols (functions and data). You define export symbols within the link command, where you can also specify whether to export all the symbols in the shared library, or to export all the symbols while hiding some symbols (table entry 7). The VMS linker allows you specify symbol vectors containing data or procedure symbols which can be exported from the library.

Another way to export symbols from a shared library under Win32 — either data, functions, or an entire C++ class — is to use __declspec(dllexport) right before the function or class you intend to export. Note that this feature is not supported on any other platform except OS/390, which has the _Export keyword.

Because the C++ compiler uses name mangling, you must be careful when using .def to define exported symbols, because the name mangling is platform-dependent. On Win32, the best way to export a C++ shared library is to use __declspec(dllexport). When using __declspec(dllexport), you needn't list all the mangled symbols in the C++ class in a .def. On most Unix platforms, however, there is no option such as __declspec(dllexport). When a shared library is generated on such systems, all the symbols are exported from the library by default.

In addition, mutually referencing libraries (i.e., libraries that import symbols from each other) can be problematic on strict linker systems such as Win32 or VMS, because each library requires the import file or library of the other to complete the link. This is usually resolved by using a utility (such as LIB on Win32) that generates the import file or library, which can then be used in the link step of the other DLL.

You can provide some shared libraries with a version number at link time. On Unix, you can specify this version number by adding an extension to the library name, such as libfoo.so.1. You should change the internal library name to reflect this version number (table entry 3) in case the shared library's interface changes. Some linkers allow you to link an executable to require an exact version match of shared libraries at run time, or to ignore library version information.

Note that shared libraries or executables containing C++ code should be linked via the C++ compiler, so that static constructors within the library get called when the library is loaded.

Run-Time Issues

When an executable is linked, the shared libraries on which it depends are recorded in a library list section. However, the actual location of these shared libraries, as well as the binding symbols from the libraries, is determined at run time by the dynamic loader (table entry 12). At startup, control is passed to the dynamic (run-time) loader, which traverses the executable's library list section and recursively maps each shared library into memory, usually resolving the library's location using the system's runtime library path environment variable (e.g., LD_LIBRARY_PATH for Solaris or PATH for Win32, as shown in table entry 9). The symbols referenced by the executable and the shared libraries are then resolved based on the linkage table and the location at which these shared libraries have been loaded into memory. Finally, control is passed to the program's startup routine. Typically, the search order for a shared library is:

1. the directory in which the executing process resides

2. the path, if any, contained in the library name itself

3. the path specified at the executable's link time using a linker option (table entry 5)

4. the path in the system's runtime library path environment variable (table entry 9)

5. the default library search path (usually /lib, /usr/lib, etc. on Unix; or on Win32, the Windows directory, the Windows system directory, and the executing process' directory)

While the order above is often the default behavior, it can be tweaked on many systems using link options. These could include ignoring the runtime library path for security reasons, or delaying loading a library until it is actually accessed, rather than loading it at startup time.

Using link flags, you can control the order in which the system searches for symbols at run time. On most versions of Unix, the search for symbols typically starts by default from the executable, and then traverses the list of shared libraries in the order they were linked with the executable. At the library's link time, you can specify an option (such as -Bsymbolic) to change this bind behavior to look first in the shared library, then in the executable, and finally in other shared libraries (table entry 10). Search order can be important in applications that depend on a large number of libraries. In a simple case, as we have demonstrated in our example, search order can generate different results on different platforms. In the real world, search order could cause different dialog boxes to pop up, for example, or it could cause a core dump.

Consider the example below. It uses two shared libraries: one in C, the other in C++. It also includes one executable linked with the two libraries, and another executable that loads the shared library at run time. You can use makefile.sol2 (Listing 1) to generate the shared libraries and the executable. The two shared libraries and the executable all define the same symbol GlobalSymbol. First, define a header file slC.h to declare CGlobalSymbol that is shared from main.cxx and slC.cxx:

class CGlobalSymbol { };

The snippet below is main.cxx, used to generate the main executable. The executable prints the address of the CGlobalSymbol object.

#include <stdio.h>
#include "slC.h"
CGlobalSymbol GlobalSymbol;
extern void sl1_printSymbol();
extern "C" void sl2_printSymbol();
int main(int argc, char *argv[])
{
  printf("%s:%d: Address of GlobalSymbol
         " in executable = %#x\n",
         __FILE__, __LINE__, &GlobalSymbol);
  sl1_printSymbol();
  sl2_printSymbol();
  return 0;
}

The following snippet is slC.cxx, used to generate a C++ shared library. This file also declares a CGlobalSymbol object. Note that slC.cxx and main.cxx use the same symbol name.

#include <stdio.h>
#include "slC.h"
CGlobalSymbol GlobalSymbol;
void sl1_printSymbol()
{
  printf("%s:%d, Address of GlobalSymbol in DLL1 = %#x\n",
         __FILE__, __LINE__, &GlobalSymbol);
}

The following is slc.c, used to generated a C shared library. This file also declares a variable CGlobalSymbol and shares the same symbol name with main.cxx and slC.cxx:

#include <stdio.h>
int GlobalSymbol = 3;
_printSymbol()
{
  printf("%s:%d, Address of GlobalSymbol in DLL2 = %#x\n",
         __FILE__, __LINE__, &GlobalSymbol);
}

To generate a shared library on Solaris 2, use makefile.sol2 (Listing 1) , and type

make ARCH=sol2

When run on HP-UX, this program prints:

main.cxx:12: Address of GlobalSymbol in executable = 0x40001100
slC.cxx:10, Address of GlobalSymbol in DLL1 = 0x40001100
slc.c:8, Address of GlobalSymbol in DLL2 = 0x40001100

The correct and desireable behavior is that the addresses of all these global symbols would be different. The executable should print the address of the CGlobalSymbol residing in the executable; sl1_printSymbol (in the C++ shared library) should print the address of the CGlobalSymbol residing in the C++ shared library; and sl2_printSymbol (in the C shared library) should print the address of the global variable CGlobalSymbol residing in the C shared library. Unfortunately, shared libraries do not always behave the way you would expect across platforms, as shown by the HP-UX example above.

As previously mentioned, one workaround for resolving symbols within an internal shared library on Unix is to use a linker option to change run-time symbol binding. Another workaround is to hide these symbols inside their respective libraries (table entry 8). For example, if your application is written in C++, you can provide your users with only a C interface. It is generally preferable to hide all the symbols except a list of C APIs. This would be true even if all it did was help you avoid C symbol linkage collision with other libraries. But using C APIs also lets you hide all the details of all the C++ objects, classes, variables, and many other symbols to avoid collision with other shared libraries, hence improving portability across platforms. In general, it is good practice to avoid global data, to hide or avoid exporting symbols that are not used outside the library, and to use the correct run-time binding mode to search for symbols within the library first.

Programmatic Loading Control

While shared libraries can be loaded at program startup, they can also be loaded under application control at run time using interfaces exposed by the dynamic loader. Typical Unix functions are dlopen (table entry 16), to load a library; dlsym (table entry 17), to obtain the address of a symbol in the library; and dlclose, to unload the library. On Win32, the equivalent functions are LoadLibrary, GetProcAddress, and FreeLibrary. The two forms are very similar, except that the Unix functions do not involve calling a shared library initialization routine such Win32's DllMain.

The following sample code is load.c. It will load a shared library using dlopen, get a pointer to function sl1_printSymbol using dlsym, run that function, then close the shared library.

typedef void (*sl2_printSymbol)();
main(int argc, char **argv)
{
    void *h = dlopen("libslc.so", 1);
    sl2_printSymbol s = (sl2_printSymbol)dlsym(h,
        "sl2_printSymbol");
    s();
    dlclose(h);
}

Use the modified version of makefile.sol2 shown in Listing 3. You can try the generated executable loader to test load a shared library dynamically, and call the functions inside a shared library dynamically at run time.

Symbol conflicts may cause a performance hit when loading libraries. Some linkers provide options (see table entry 13) to identify these conflicts at link time.

Debugging issues related to environments with shared libraries can be fairly tricky. A useful utility on most systems (ldd on Solaris, as shown in table entry 18) is one that lists dynamic dependencies, which essentially lists all the shared libraries linked with a particular executable or another library. Many systems have useful environment variables to trace the run-time loader's actions (table entry 11), listing resolved symbols and other handy information.

Summary

We've provided a number of general concepts about shared libraries, and how they exist and differ on various platforms. Although the basic features of shared libraries are similar on most platforms, implementation and behavior details vary widely across platforms. The systems referred to in this article are Solaris 2.5, HPUX 10.11, Digital Unix 4.0, IRIX 6.2, AIX 4.1, OpenVMS 7.1, OS/390, and Win95/NT 4.0. OS and compiler versions are always evolving, so many of the details in this article could change in newer OS and compiler releases. We recommended you consult your system's man pages or help files for the latest information.

Acknowledgments

Thanks to the engineers at Bristol Technology, who contributed to various ideas and details in this article.

Listing 2

Amal Shah is a principal engineer at Bristol Technology, where he has been involved in porting MFC and OLE to various Unix platforms.

Hong Xiao is Bristol Technology's principal software engineer. Hong has over 10 years experience in software design and development. He is involved with porting the Microsoft foundation classes from MS-Windows to Unix and OpenVMS, and is a key contributor to OLE development on Unix.