September 2002/The .NET Managed Extensions to C++

C# and .NET

The .NET Managed Extensions to C++

Stanley Lippman

Pay attention to the Man Behind The Curtain — he knows how to do C++ in .NET, and more importantly, why.

It is sometimes difficult to remember that Visual Studio.NET provides a .NET version of C++ as well as C#. Maybe we didn’t give it a snazzy enough name: the Managed Extensions to C++. Managed is meant to indicate its direct support of the CLR (Common Language Runtime). Extensions is meant to suggest that we have added to the existing ISO C++ rather than attempting to either create a new language or to limit support to a runtime library. Our approach was to introduce keywords with an ISO C++ compliant double underscore (__interface, __gc) to set off support for features unique to the CLR. A frequently asked question is: Why did we do that? I’ll briefly address that in the second section of this article.

Ok. So why should you use the Managed Extensions for programming rather than, say, C#? After all, C# is very familiar to a C++ programmer. (Some would claim it is actually more familiar.) Another reason is that C# is currently the best language for illustrating elements of the CLR — it helps simplify the complexity of what is actually going on. Of course, this simplification is also an area of controversy. Not surprisingly, C++ programmers complain when they look at C# code that they are being prohibited from accessing the underlying representation and that this makes system programming... well, it presents difficulties. So, not surprisingly, under the Managed Extensions, you get that access. I cover this in the last section of this article, since it is the most esoteric.

The primary reason you want to use the Managed Extensions is for interoperability support. Essentially, you can mix in any managed class or class hierarchy within our Standard C++ program and it just works. There are constraints however in the places a managed class can be placed. This is largely because of the garbage-collected nature of the managed type. Additionally, you can declare a pointer member to an existing native C++ class within a managed class. This permits you to provide a managed class wrapper around any C++ class or class hierarchy. This is the topic of the first section.

Accessing Your Native C++ Code Base

So, you have these really nifty C++ classes and class hierarchies you’ve used and reused for whomever knows how long. Now here comes .NET, and you want to expose these class interfaces to .NET applications. Or perhaps you still choose to write your performance critical components using the native platform or not give up the expressive power of the STL for certain domain solutions. How can you make these native classes available to a .NET application?

In the early uses of C++, one of the most effective was wrapping an external function API into a set of classes. Examples of this include Doug Schmidt’s ACE, which first provided a class abstraction of the Berkley Sockets API, and Microsoft’s MFC, which first provided a class hierarchy to encapsulate Win32 GUI programming. The solution to exposing a native C++ class interface to a .NET application is by wrapping a pointer to a native C++ class object in a managed __gc class.

The example I typically use is the TextQuery application I implemented to illustrate the STL and inheritance in my C++ Primer, Third Edition [1]. Its implementation uses maps, lists, vectors, and so on, as well as supporting a Query language inheritance framework. Here is a very simple wrapping of the native type:
// this is the CLR equivalent of a header file!
#using <mscorlib.dll>

// the native C++ header file
#include "TextQuery.h"

// open the .NET System namespace ...
// this allows us to directly refer to System::String, below
using namespace System

// ok: the __gc keyword introduced a CLR "reference" type
__gc class TextQueryManager {
private:

    // a pointer member to the native C++ class
    TextQuery *pquery;

    // a pointer instance of a System::String ...
    // these are always declared as pointers
    String * ps;

public:

    // a default constructor
    TextQueryManager() : pquery( new TextQuery ){}

    // one argument constructor -
    // the CLR does not support default arguments ...
    TextQueryManager( TextQuery *pq ) : pquery( pq ){}

    // ok: the actual API
    // inlined stub functions dispatch call
    void query_text(){ pquery->query_text();}
    void build_up_text() { pquery->build_up_text(); }
};
What We Did and Why

I learned two general programming heuristics at Bell Labs that have always held me in good stead. One is that whenever there is a hard problem to solve, add a layer of abstraction. That is, see if providing a programmable hook in the space between the request for and access of an object can heuristically provide a solution. In C, that meant adding a pointer. With C++, a class represents the abstraction layer.

The CLR adds a layer of software abstraction that serves as a virtual execution system. This allows for a much smarter and dynamic runtime environment and has been successful in supporting the component-programming paradigm. In particular, it solves many of the implementation problems of COM.

In many ways, the C++ and CLR object models represent the physics of Kansas and Oz. Dorothy can move between the two worlds, but only successfully when she learns the rules that operate in each separate world domain. For example, you can only get from here to there by clasping your hands together, squeezing your eyes tightly shut, and wishing under the Oz semantics. These semantics, however, just don’t make sense in Kansas. How do you get the C++ Dorothy to understand that, but still feel as if she is speaking C++? This is the fundamental challenge of providing the Managed Extensions.

A simple example of this is the difference in how information about your programs is known. During compilation, lots of information about the program is stored along with the actual translation of the program source. This metadata allows the CLR to provide a non-sequential execution and compilation environment. For example, under the CLR, forward declarations are not required because the metadata is available. In C#, this metadata is attached to the project either through a command-line argument or through the IDE Project menu. In the Managed Extensions, we chose to make it available through a using statement:
#using <mscorlib.dll>
mscorlib.dll contains the definitions of all the core types, such as String, Enum, Object, Type, etc. To access one of these types, you must include two bits of information:

The namespace within which it can be found. (Yes, I believe fully qualified names are an abomination.) All C++ programmers know this by now.

The assembly within which the metadata for the types is stored. In this case, it is mscorlib.dll. This is analogous to the header file, but this is where Oz and Kansas go their separate way.

We could have used the #include notation, but it would damp down or dismiss the real difference in semantics between textually pulling in source code and having the full description of all program types available before processing the source code or at run time when executing the program. It is analogous to the difference between the fire-hose model of, say, parsing an XML document and the in-memory DOM model.

Because the native platform represents a fire-hose model of compilation, the order of module execution and the collection of type definitions within modules other than the one currently being compiled can’t be known to the compiler. In its most trivial aspect, it requires forward declarations, whereas the CLR does not require these declarations.

More seriously, this has resulted in one of the weakest aspects of C++: the order of global data requiring static initialization is undefined and a constant problem. The programming tricks to solve that, such as Schwarz Counters, don’t scale. Under the CLR, all initialization/access dependencies are resolved automatically.

This is a significant difference. The native solution, a header file, is a textual insertion that must be processed at each point of inclusion. There is little you can do with it but parse it. (I know — you can pre-compile it. This allows us to add to the compilation state without storing a processed representation.) The CLR loads and has execution access to abstract descriptions of each type in the program assembly. It is capable of providing a set of support and service facilities that require manual labor in the native world (e.g., serialization).

So, we had to decide how to do that, and in a way that “fit in” with native C++ as much as possible. C#, for example, does not have the user specify the necessary assemblies. The metadata contains that information, but that may not be obvious to the reader of the program. And so we chose to require that the user specify each assembly holding required program metadata.

The obvious choice, of course, was #include. But its use conveys nothing of the actual CLR processing. There is no textual insertion. The source stream of tokens is not disrupted. In short, we are not in Kansas anymore. #using is intended to serve as our pair of ruby red slippers.

Another major decision point was how to represent a CLR common type system. This is two-fold, of course: a) the definition of the type, such as TextQueryManager defined above, and b) the allocation and use of objects of that type.

For the definition of a managed type, we chose to introduce a keyword, a marker to alert the user that they have left Kansas. In the case of the reference type, we chose __gc, for garbage collected. The double underscore is an attempt to be ISO compliant. Nobody particularly likes it.

A reference type is garbage collected. It exhibits shallow copy semantics. It supports static constructors, properties, delegate, and event members. The System::Object class is always its super base class. A reference type fully participates in run-time type reflection.

How about its use? We felt that because of the different extent and copy semantics of a reference type from that of the native C++ class object, we couldn’t use the same syntax to represent both. We felt that would be too confusing and lead to confounding errors.

How should it be indicated? The reference type is ... well, it is a reference. That is, it is a handle/object pair in which the named object holds either null or the address of an actual unnamed object allocated on the managed heap.

Unfortunately, the C++ reference is not sufficient to support the necessary semantics. We need to be able to initialize or assign null to an object of a managed reference type. We need to be able to assign it to refer to a different object, and so on. C++ references exhibit deep copy semantics. They can only be initialized, not assigned to. You cannot, in short, separate a C++ reference from the object to which it refers.

We could have introduced a flexible reference syntax. For example:
// hypothetical flexible reference syntax ...

Matrix ^ mat = new Matrix;
mat = a * b;

Matrix ^ old_mat = mat;
mat = scale( mat[ 0, 1] );
But this strays beyond the officially sanctioned mechanisms for C++-compliant platform extensions, and there has been a great deal of hesitation in moving off in that direction, even if it does solve a number of otherwise thorny issues.

And so, we decided to represent the managed reference type as a pointer. After all, the copy semantics of a pointer are the same as for the managed reference type. Moreover, it feels more natural to be assigning the return value of the new expression to a pointer. Wherever possible, we attempted to conform if not to the letter then to the spirit of the C++ Standard.

Until a Standard Library Comes Along

The .NET class framework can be thought of as providing multiple levels of programmer support. There is core language support for the fundamental data types, such as int, bool, and string, as well as the enum and array types. In addition, the framework provides the same kind of program support you expect from a standard library: I/O, collection classes, threads, sockets, regular expressions, and so on. In addition, the framework provides application domain support in such areas as XML, ASP.NET, and Web Services.

To make use of the .NET class framework, you simply declare instances of the object using a pointer. For example, here’s a small piece of code to iterate over all the directories on each available logical drive of the machine. The String and DirectoryInfo classes are part of the .NET class framework.
// get an array of strings of the form: C:\
// this is a 'managed' array ... it has self knowledge

String *logical_drives [] = Directory::GetLogicalDrives();


for ( int ix = 0; ix < logical_drives->Length; ++ix )
{
    String*    dir_path = logical_drives[ ix ];
    DirectoryInfo *dir= new DirectoryInfo( dir_path );
        
    if ( ! dir->Exists ){ ...; continue; }
    
    DirectoryInfo *sub_dirs [] = dir->GetDirectories();
    int dir_cnt = sub_dirs->Length;
    
    // ... etc ...
}
Don’t Box Me In

Probably the hardest part of designing the CLR common type system was in unifying value and reference types. A value type is a local object. It stores its value directly within itself and reflects bit-wise copy semantics. There are a number of constraints imposed on it in order to insure its correct and safe behavior [2]. A reference type, on the other hand, is a tightly coupled pair: a handle through which to refer to a garbage-collected object, and the object itself allocated on the managed heap.

Since all types under the CLR are derived from System::Object (Object is a reference type), all values can be assigned to an Object entity. Thus, you can write, in C#,
object o = 1;
Console.WriteLine( o.ToString() );
and have 1 appear on your terminal.

Now, since System::Object is a reference type, in order for it to refer to an instance of the constant integer literal 1, it must create a temporary. This is the same thing we originally did in C++ with references:
// permitted until cfront 3.0
int &ri = 1;
That is, in order for ri to refer to the literal integer constant, you generate an integer temporary and assign it the value 1.

With the CLR, a temporary is also created, but it is slightly more involved. It must allocate a reference type on the managed heap. It does so by defining a shadow reference type for each value type. So, when the programmer subsequently writes:
o = 2;
an integer reference type must be allocated on the heap and the literal value 2 assigned to it. Any previous object referred to by o now has one less reference. If there are no references to the object, of course, it is marked as subject to garbage collection. This generation of the shadow value reference type is called boxing.

So, if I wanted to be silly and write a loop like the following in C#:
for ( int ix = 0; ix < int.MaxValue(); ++ix )
    o = ix;
I would be creating quite a lot of shadow integer reference types. Most C++ programmers blanch even at the thought of what this must be like. What you want is to have access to the object referred to by o and update it in place. This saves a great deal of memory thrashing. Moreover, you would probably be smart enough to know that you could simply assign o int::MaxValue() - 1.

So, the primary difference between the Managed Extensions for C++ and C# is the assumptions the two languages make about their programmers. And this is handily illustrated in their decisions on how to present the underlying CLR boxing mechanism to the programmer.

C# has chosen to hide the mechanism completely. On the plus side, this simplifies the language both in terms of the syntax and in terms of the amount of concepts one has to explain or expect the programmer to understand. This was also a goal of COBOL. The point similarly is to allow a less technically savvy person to write the necessary code. This is not in itself bad, although those of us who have put in the time, so to speak, tend to feel threatened and dismissed as a result. But this has been the movement of computer programming ever since Grace Hopper introduced assembly language to the protests of the very few comparatively who could program in hex machine code. If C# can become the Visual Basic of the .NET platform, or the new COBOL of the enterprise, then I think we have come a great way in our attempt to make the profession more accessible to the average person.

In the Managed Extensions, we have chosen to move in the opposite direction. We expose the entire mechanism. We provide an explicit value reference type, so that you can directly manipulate it within your program. This may save you a ton of temporary objects being allocated and freed from the heap, but it means you will have to take special cautions because this guy is on a managed heap. If you mess up, you are really messing up. We want to see your credentials. This is why we support interior pointers as well. It gives you that 20% performance boost that you cannot get with any of the domesticated languages.

We are still trying to understand how we can best adapt to this strange new Oz-like environment and, to be honest, it is going to take a few iterations. C++ has always been like that. The hard part now is that there are so many people watching. Back in those early days, it was much easier to stumble and be awkward.

Notes and References

[1] Stanley B. Lippman and Josee Lajoie. C++ Primer, Third Edition (Addison-Wesley, 1998).

[2] Most C++ programmers initially grump and complain about the constraints. The truth is that in this managed Oz, many of our basic assumptions as C++ programmers are considered bad habits of an obsolete technology that need to be corrected. (That is putting it in very stark terms.) If the native Kansas is our home, with shared beliefs and specialized support from the environment, then we are strangers in the strange Oz-like land of the CLR. It is a sometimes uncomfortable feeling, as if certain basic values one has always esteemed are suddenly discarded or dismissed with an annoying smugness. I like to think of the CLR as Smalltalk’s revenge. The battleground remains performance and accessibility to the underlying implementation.

Stanley B. Lippman is IT Program Chair with you-niversity.com, an interactive e-learning provider of technical courses on Patterns, C++, C#, Java, XML, ASP, and the .NET platform. Previously, Stan worked for over five years in Feature Animation both at the Disney and DreamWorks Animation Studios. He was the software Technical Director on the Firebird segment of Fantasia 2000. Prior to that, Stan worked for over a decade at Bell Laboratories. Stan is the author of C++ Primer, Essential C++, and Inside the C++ Object Model. He is currently at work on C# Primer for the DevelopMentor Book Series for Addison-Wesley. He may be reached at stanleyl@you-niversity.com.