Dr. Dobb's Journal January, 2005
Last month, I introduced the new version of C++ that will be available in Visual Studio .NET 2005 (Whidbey). In this month's installment, I continue my examination of the new language, covering new compiler and linker switches.
The C Runtime Library (CRT) contains standard functions used in millions of lines of C and C++ code. It is not possible to throw away this code; indeed, that has never been Microsoft's intention. The first version of Managed C++ had interoperation with native code at its core: The IJW technology (It Just Works!) was provided to let you compile native code as Intermediate Language (IL) without any changes. However, as the product was used, issues arose because the managed and unmanaged worlds are two very different places.
The first version of Managed C++ demonstrated a bug in the interoperability code: When a call was made from managed code to unmanaged code, the interoperability thunk did not record the application domain identifier. This meant that there was a possibility that when the call returned in a process with more than one application domain, the flow of execution could continue in an application domain other than where the call was initiated. This was fixed in 1.1 of the runtime, but interoperability is fraught with subtle issues. Application domains present a problem to the CRT because it's a concept that does not exist in the unmanaged world. Indeed, the CRT was designed for a single-threaded world—UNIX developers would fork a new process that would have its own copy of the CRT global variables. Microsoft did a good job of making the CRT multithreaded, but .NET app domains bring up another problem because a .NET thread is not constrained to a single application domain.
Version 1.1 of the runtime was not without its issues. One issue (also present in 1.0) manifests a problem with the operating system. When Windows loads a DLL, it calls the library's entry point (usually DllMain) to let the library initialize itself. However, while doing this, the operating system obtains and holds onto various synchronization objects, so the initialization code should be restricted to only a few APIs to ensure that there is no possibility of a deadlock. Managed C++ lets you write a DllMain, but more importantly, if you use the CRT, the compiler provides a version for you to initialize the CRT and call constructors of static or global unmanaged objects. Visual C++ .NET 2003 provided a workaround to let you use unmanaged code in managed libraries and avoid the so-called "DLL mixed-mode loader problem."
Another issue with C++ was that if you used unmanaged code, the library would automatically be unverifiable. What this means is that when the library is loaded, the runtime would give the library minimal permissions if the assembly came from a source other than the local hard disk. Thus, C++ libraries could not be downloaded over the Internet or intranets. Again, 1.1 provided a workaround for this issue to make it more likely that your library would be verifiable; however, it was messy to implement.
The Whidbey compiler recognizes these issues and provides switches to take them into account. The /clr switch has three options: /clr, /clr:pure, and /clr:safe. The idea is that you determine the verifiability requirement of the assembly and also the details about how unmanaged code is treated, then the compiler generates appropriate code and, in some cases, issues errors if the code does not meet the criteria. This means that it is no longer necessary to use the workarounds created for .NET 1.1 to get around the loader lock bug, or to create a verifiable assembly.
The first option, /clr, is for so-called "mixed-mode" modules—those that can contain unmanaged code compiled as x86; so by definition, the module is not verifiable. When you compile with this switch, it is assumed that you will use the CRT and so the compiler links with msvcmrt.lib (an import library for msvcm80.dll). In addition, the compiler adds COMIMAGE_FLAGS_32BITREQUIRED to the .corflags member of the CLR header in the assembly to indicate that the assembly must be run in a 32-bit process because the assembly contains function fixups. A fixup is essentially an unmanaged function pointer and is determined at runtime; for the 32-bit compiler, the fixup is always 32 bits!
Using unmanaged libraries is a problem because unmanaged libraries have no concept of .NET application domains. In native code, the module entry point is used to initialize global variables and call constructors of static objects. However, this can cause a problem if the constructors of these objects call code that is not safe in DllMain. To get around this, the Whidbey compiler adds a module static constructor to perform initialization. In .NET, a static constructor is guaranteed to run before the first use of the constructor's type. By creating a module static constructor, the initialization code is called sometime after the library's entry point has been called—after any .NET initialization has occurred, but before any managed code is executed.
Consequently, if you look at an assembly compiled with /clr using ILDASM, you see a static method called .cctor and various other static methods used to access the application domain and initialize the native code. Many of these methods are native; hence, the compiler marks the assembly with SecurityPermissionFlag.SkipVerification to indicate that the assembly can be loaded only if it has this permission (which usually means that the assembly is installed on the local hard disk). By default, in mixed mode, all global variables are global to all application domains in the process; that is, one single copy of the data exists regardless of the application domain used to access the data. If you want a native, global object to be global to only the application domain (in effect, a copy of the object for each application domain), then you can use the __declspec(appdomain) modifier.
When you use the /clr compiler switch, your code is automatically linked with the msvcmrt.lib static import library. You cannot use the statically linked CRT (the /MT or /MTd options) with /clr.
The C++ compiler can be used to create .NET modules, and previous versions of the compiler used the /clr:noassembly switch. However, in the new version of the compiler, there are three types of code generated (according to its verifiability and whether x86 code is permitted) and /clr is used to determine the type of code generated. Because of this, the /clr:noassembly switch has been replaced with /LN, which is used to produce a .NET module. In addition, the compiler expects the C++ code to use the new language syntax by default. If you want to compile a source file that uses the old managed C++ syntax, then you can use the /clr:oldSyntax switch. This means, of course, that the code is compiled as mixed mode.
Mixed-mode assemblies contain x86, unmanaged code. By definition, this is not verifiable. Mixed-mode assemblies are great if you want to use existing native C++ static libraries, a native .obj file, or native code that does not compile to IL. There are two other options for compiling an assembly—pure and safe. Both types only contain IL and contain no x86 code. However, of these two, only "safe" is verifiable code.
You use /clr:pure if you want to use unmanaged types in your code that are included through source files that are compiled by the managed compiler; for example, template files. The native code is compiled to nonreference types; that is, they are not created on the managed heap, but still compiled to IL and run under the .NET runtime (this is achieved because the types are essentially compiled as value types). However, because these types are not managed, it means that the code is not verifiable.
Again, static and global unmanaged objects have to be initialized before they are used, so pure assemblies also have a module static constructor. The difference between the two is that pure assemblies maintain a copy of each global object for each application domain. If you prefer a global native object to have a single value for all application domains in the process, you can mark the variable with the __declspec(process) modifier. Pure assemblies can contain native C++ classes, but not x86 code. This still means that the assembly is not verifiable; hence, a pure assembly is marked as requiring the SkipVerification permission.
Pure assemblies can call native code exported from a DLL through platform invoke. The CRT is provided through the msvcm80.dll DLL, and a pure assembly has an external module reference to this in its manifest. The linker accesses the CRT methods through the msvcurt.lib import library. Whereas mixed assemblies statically link CRT functions to the assembly as embedded x86 code, pure assemblies access the CRT functions exported from msvcm80.dll through platform invoke. This way, pure assemblies can ensure that they only contain IL. Indeed, if you look at the manifest using ILDASM, you'll see that a pure assembly has a .corflags value of COMIMAGE_FLAGS_32BITREQUIRED | COMIMAGE_FLAGS_ILONLY; that is, it only contains IL, but will call functions through platform invoke and so must be run in a 32-bit process.
The final type of assembly you can create is a verifiable assembly, created with /clr:safe. This is the most restrictive type from a C++ point of view because you cannot have native types, x86 code generated from your code or linked from a static link library, nor can you access any native code through a platform invoke. The advantage of code compiled with /clr:safe is that it is verifiable; that is, the compiler performs a verifiable check and issues errors if any of the code is not verifiable. Verifiability will be an important issue in the future. For now, it appears only to be a security check to make sure that code that you have downloaded from the Internet does not try anything out of the ordinary. However, as .NET code finds its way into the operating system or enterprise services, such as SQL Server, then being verifiable as being safe code starts to become an important issue. Visual C++ provides this functionality in Whidbey—ready for the operating systems and services of the future.
A verifiable assembly can only call components in other assemblies that are also verifiable, even though msvcm80.dll is compiled as a .NET library assembly and still contains embedded x86; hence, the assembly demands the SkipVerification permission. So safe code cannot use the CRT at all.
The compiler switches can be used to compile individual source files without linking using /c, then an assembly is created by linking the .obj files together. The overall verifiability of an assembly is determined by the lowest verifiability of the individual .obj files that are linked together. If you decide that you want a lower level of verifiability, then you can specify this using the /CLRIMAGETYPE linker switch. The options are IJW, PURE, and SAFE, which correspond to mixed mode, pure, and safe.
Win32 code does not use exceptions to report problems because few languages could handle structured exception handling (SEH) exceptions. (There are a few Win32 APIs that generate SEH exceptions, but only if specific flags are passed to the function.) Instead, Win32 functions indicate that an error occurred through a return value (usually a FALSE value) and the caller can obtain details about the error by calling GetLastError to get a 32-bit value. Platform invoke has always had the ability to obtain such error codes, but it was an optional facility. The managed code calling the native function would use the [DllImport] attribute and set the SetLastError field to true. Then, when an error occurs, the runtime calls GetLastError and caches the result, so your managed code would then have to call Marshal.GetLastWin32Error to get the error code.
The new linker has a switch called /CLRSUPPORTLASTERROR. When used without a parameter, this switch means that all calls through platform invoke will be accompanied by a call to GetLastError, regardless of whether the SetLastError field is set, and the result is cached for a subsequent call to Marshal.GetLastWin32Error. Clearly, this is a performance issue, so you can use the SYSTEMDLL option to restrict this behavior to just system DLLs. Finally, you can use the NO option to turn off this facility.
The linker also provides the switch /CLRTHREADATTRIBUTE, which indicates the default COM apartment attribute of the entry point. This attribute is only used if the code calls a COM object (directly or indirectly). Since every COM object must be run in an apartment, the runtime makes the thread join a COM apartment just before it makes the first call to COM on the thread. By default, the runtime makes the thread join the process's multithreaded apartment (MTA), but if the COM object is marked as being single threaded, then the COM object is created in an STA apartment and the current thread uses COM marshaling to access the object. Even worse, if the thread is the GUI thread in a Windows Forms application, then the thread must join an STA, because if it enters the MTA, it can cause the UI to become unresponsive.
In earlier versions of the runtime, you used the [STAThread] or [MTAThread] attributes on the entry point to indicate the apartment that you would prefer the main thread to join. If you use one of these attributes, then the value supplied through /CLRTHREADATTRIBUTE is ignored. Otherwise, the switch lets you specify the apartment for the main thread of the process (but not for other threads, where the default will still be MTA).
When you compile a mixed mode or a pure assembly, you do it so that you can use native C++ code in your assembly. Such assemblies will use the CRT and (for mixed assemblies) this might be embedded x86 code or (for mixed and pure assemblies) it will be a platform invoke call to the msvcm80.dll library. Even though this library is managed and shared, it is not installed in the GAC. Instead, it is installed as a shared Win32 library in the WinSxS folder (a so-called side-by-side assembly). When your code uses a side-by-side assembly, it declares the version of the assembly in a manifest file. This is an XML file with a manifest extension. By default, the linker will generate a manifest file for you for mixed and pure assemblies, but if you do not want it to do this you can use the /MANIFEST:NO switch. The default name for a manifest file is the name of the assembly suffixed with .manifest, but you can change this name with the /MANIFESTFILE switch. By default, the linker adds the msvcm80.dll library to the manifest file's dependency list, but you can add other entries using the /MANIFESTDEPENDENCY switch.
One of the problems that the new version of the compiler has sought to solve is the so-called "double platform invoke problem." When you create a native function in your code, the compiler creates two entry points—one managed and the other unmanaged. The unmanaged entry point for the function provides a thunk that performs the necessary transition between unmanaged and managed worlds. It is used when a pointer to the function is passed to unmanaged code or is called by code that cannot be compiled to IL (for example, it has varargs or if the function is compiled with #pragma unmanaged). In some situations (vtables, for example), the entry point must be obtained at runtime. But this creates a dilemma for the compiler, because at compile time there is not enough information to determine whether the code requiring the entry point is managed or native. Consequently, the compiler for Version 1.1 of the runtime chose the unmanaged entry point. This meant that if the code requiring the entry point was managed, it would get an unmanaged entry point; hence, there would be another transition from the managed to the unmanaged world. This is the source of the name of the problem because a call through the managed vtable entry would require a transition from the managed world to the unmanaged world to the unmanaged entry point, and then from the unmanaged world to the managed world to get to the actual function: a double transition.
The new version of the compiler provides a solution to this issue. The __declspec(__clrcall) can be used on methods in mixed and pure code to indicate that the method should not have an unmanaged entry point. Because the method will only have a managed entry point, it means that it can only be called by managed code, but it also means that the call will always be through the managed entry point. It's worth pointing out that because pure and safe code do not have to accommodate embedded x86 code, they do not suffer from the double platform invoke problem.
The final topic I cover also concerns platform invoke, namely calling native code. Earlier versions of Managed C++ could use platform invoke through the [DllImport] attribute, but this assumed that the code had been rewritten as managed C++. If you used existing C++ code and compiled it with the /clr, then the code would call DLL exported code through a static import library. In effect, this would add a thunk from the managed world to the unmanaged call to the DLL exported function. The thunk would effectively wrap a JMP to a memory location in the import address table (IAT) that would be fixed up at runtime. This is essentially how calls to DLLs are made in native code. If you ran dumpbin /imports on such an assembly, you would find that the IAT would contain an entry for the DLL functions you called.
In Whidbey, this behavior still occurs for mixed-mode assemblies. However, when you compile pure code, such embedded native code is not allowed and the compiler replaces the call with a platform invoke call. The code still looks as if it has a thunk because it still links to the DLL's import library. However, a call to dumpbin /imports shows that the IAT is empty. To see the effect of this, compile Example 1 with /clr, then with /clr:pure, and for both assemblies, use dumpbin /imports to show the IAT. Furthermore, use ILDASM and examine the method the compiler generates for MessageBeep: For mixed code, this method has the pinvokeimpl(lasterr stdcall) modifier indicating that a thunk is created, whereas the pure assembly has the pinvokeimpl("USER32.dll" lasterr stdcall) modifier indicating that platform invoke is used directly.
Next month, in the final installment of this series about the new version of Managed C++, I'll examine new features in the language: generics, array declarations, and the new pointer operators.
DDJ