Visual C++ omits the darndest things in its support for Windows/CE. Too bad memory tracking is one of those things.
Visual C++, under Windows 9x/NT, provides memory-tracking functions that are extremely useful to detect memory leaks when running an application in debug mode. MFC (Microsoft Foundation Classes) takes advantage of that implementation by dumping unfreed objects upon application termination. Those features, essential to developers to track memory leaks, are not implemented under Windows CE. (They were among those items deemed unnecessary in order to reduce the footprint.) This causes a real problem for developers, since Windows CE is typically used in memory-restricted environments, where memory leaks can truly exhaust system resources. This article presents a simple solution to track those leaks. The source code can be downloaded from CUJ's ftp site (see p. 3 for downloading instructions).
VC++ Memory Tracking Under 9x/NT
Visual C++'s C library includes special memory-allocation functions (_malloc_dbg and _free_dbg) that keep information about each allocated block, such as the filename and the line number where the allocation was done. I first explain how MFC takes advantage of those functions.
MFC (specifically, Afx.h, included via StdAfx.h, which is itself typically found in any MFC project) defines the DEBUG_NEW symbol as a call to a new-expression that takes two arguments [1]: THIS_FILE and __LINE__. This definition, in itself, does nothing, but consider that AppWizard always generates the following at the beginning of each source (.cpp) file:
#ifdef _DEBUG #define new DEBUG_NEW #undef THIS_FILE static char THIS_FILE[] = __FILE__; #endifIn debug mode, this translates a statement such as:
char * p = new char[10];into:
char * p = new(THIS_FILE, __LINE__) char[10];THIS__FILE and __LINE__ are two macros that the compiler automatically translates into the current filename and the current line number. Consequently, the previous statement could become:
char * p = new("E:\MyApp\MyApp.cpp", 41) char[10];This new-expression results in a call to an overloaded global operator new, which internally (in AfxMem.cpp) invokes _malloc_dbg, passing the filename and the line number. Upon program termination, the information passed to _malloc_dbg is dumped for every memory block that hasn't been released. For instance, the memory block allocated in the previous statement, if never released, results in the following dump in the Output window, in debug mode:
Detected memory leaks! Dumping objects -> E:\MyApp\MyApp.cpp(41) : {69} normal block at 0x00420030, 10 bytes long. Data: < > CD CD CD CD CD CD CD CD CD CD Object dump complete.Additionally, MFC redefines operator new and delete for its CObject object (the root class of the MFC hierarchy), to invoke _malloc_dbg and _free_dbg with special parameters. This gives MFC the ability to dump the object of a class. For instance, the following statement:
CWnd * pWnd = new CWnd;invokes CObject::operator new() and ultimately results in the following dump:
Detected memory leaks! Dumping objects -> E:\MyApp\MyApp.cpp(50) : {87} client block at 0x004210D0, subtype 0, 64 bytes long. a CWnd object at $004210D0, 64 bytes long Object dump complete.Note that in addition to the filename and line number, the class type (a CWnd here) is provided.
Windows developers have been relying on such useful information to track memory leaks in their applications. Unfortunately, all this is gone under CE: _malloc_dbg and _free_dbg are not available in the SDKs (only malloc and free are), the DEBUG_NEW macro simply translates to new, and although CObject declares its own operator new, it doesn't take any extra arguments. Hence allocating memory but not releasing it remains undetected. Although Windows CE (and all other Windows operating systems) reclaims memory upon process termination, most applications on CE never really stop, as typical CE devices are merely suspended, not powered off. A memory-leaking application will eventually swallow all memory available, and applications will start to crash.
The solution to this problem is simply to reintroduce memory tracking and leak detection. Whatever approach is adopted, a few requirements must be met: the solution must work with C++ new- and delete-expressions (not only malloc and free, which do not invoke object constructors and destructors), changes to the actual code (if any) must be minimized, and all memory leaks should be dumped automatically upon program termination. Also, the solution disallows changes to the actual C and MFC libraries, so as to keep up with new versions. Ditto for some hacks that could fail on future versions of the compiler or libraries. As a relief, the solution can be limited to MFC applications for CE developed with Visual C++ and running in debug mode. As you're about to see, this can quickly turn into a challenging exercise.
Tracking Memory
Before revealing the final solution, I explore a couple different scenarios.
The first scenario is to replace malloc and free by some functions with the same names (prefixed by the "extern C" linkage directive) that would additionally track memory, hoping that new and delete would call those functions. This approach must be abandoned, since adding those functions results in compiler errors in C++.
The second scenario is to override the global operator new and operator delete functions by declaring new implementations. That works, but not for CObject-derived objects, since CObject provides its own operator new and operator delete. Hence, assuming that CMyObject is derived from CObject, executing:
CMyObject * pMyObject = new CMyObject;does not invoke the global operator new, but rather CObject's. As a result, the allocation of CObject-derived objects will go undetected. However, redefining CObject::operator new does work. For instance, if you add the following code in your application, it will be called to allocate memory:
void* CObject::operator new(size_t sz) { return malloc(sz); }But such an implementation is not useful, since extra parameters are required (the filename and the line number). And since CObject declares only the above form of operator new (under CE), the compiler will not accept any declaration with extra parameters.
To make things worse, the compiler will accept a custom implementation of CObject::operator delete, but that implementation doesn't get called. Never. Only the original implementation is invoked. It turns out that Visual C++ uses an optimization when deleting an object via delete. This optimization is done via the object's vtable and results in the original CObject::operator delete being invoked. Thus, it is impossible to know which objects get released.
The third scenario is to use the second approach and hack all vtables to use the address of the custom delete implementation. But this approach is not good because 1) vtables are read-only (writing into them crashes the application); 2) this hack that may not work in future versions of the compiler; 3) it still doesn't solve the CObject::operator new problem.
The only scenario that works is the second: re-implementing the global operators new and delete, but it fails at tracking CObject-derived objects. However, a closer look at the CObject's allocation operators reveals that under CE, they simply invoke the corresponding global operators. Hence calling either makes no difference under CE. For instance, the following code:
CWnd * pWnd = new CWnd; // invokes CObject::operator new delete pWnd; // invokes CObject::operator deleteis functionally equivalent to:
CWnd * pWnd = ::new CWnd; // invokes ::operator new ::delete pWnd; // invokes ::operator deleteSince a new-expression can accept arguments using placement syntax [1], the previous code can be rewritten as follows:
CWnd * pWnd = ::new(THIS_FILE, __LINE__) CWnd; ::delete pWnd;The above code will result in calls to the custom versions of operator new and operator delete and allows us to track all memory allocations, including CObject's.
The issue suddenly becomes enforcing the use of the custom global operators (without changing the actual application code) to make sure that CObject's operators are never called. Luckily, this is something that can be easily achieved through some #define directives, as presented in Figure 1. Note that the DEBUG_NEW macro is redefined in debug mode. All that remains at this time is to actually track memory.
Keeping Track of Memory
Keeping track of memory is simple enough: every time a block is allocated (via new), a simple information structure is allocated (via malloc) and initialized with the specified parameters (the size, the filename, and the line number). The requested memory block is also allocated via malloc. But before the custom new returns that address, it stores it in a map and associates it with the information structure (see Figure 2). When that block is released (via delete), the associated entry is looked up in the map and discarded. The associated information structure is also released. When the program terminates, the entries that remain in the map are the memory blocks that haven't been freed — the memory leaks. They simply have to be dumped to inform the developer.
MFC provides type-safe templates for arrays, lists, and maps. Any of those data structures could be used, but maps offer the fastest access, and they are ideal to associate an address to a pointer to a structure. Figure 3 presents the information structure and the map type.
The map is defined as a global variable and accessed within the custom implementation of the global operator new and delete. A more object-oriented approach is to encapsulate the map into an object (declared global) and provide simple methods to access it (see Figure 4).
Dumping the Map
The map can be dumped by simply listing its content. By formatting each line using the special format filename(line number) : message [2], Visual C++ will open the file and kindly position the cursor on the related line when the user double-clicks on it for a quick identification of the source of the leak. Here is an example of such a dump:
1 memory leak(s) found: E:\MyApp\MyApp.cpp(19) : block at 0xef0d5c, 10 byte(s) longThe CMemTrack object destructor looks like the ideal spot to dump the map. By making the object a global variable, its destructor would be invoked at the very end of the program execution. Unfortunately, it doesn't work like that: in debug mode, the destructor is not called at all. Dumping the map then requires adding a statement in the application to dump the code, but it is best not to have to do that, although a static method is provided to do so if required.
DLLs (dynamic-link libraries) come to the rescue: they are always invoked upon program termination for cleanup. The CMemTrack object can then be implemented in a DLL, which dumps the map when it is invoked for cleanup.
All in all, the class object and its methods reside in a DLL (MemTrack.dll) and the associated library file (MemTrack.lib), whereas all the definitions are stored in a header file (MemTrack.h).
Adding Memory Tracking to an Existing MFC Project
The only modification required to use the above implementation of memory tracking in an MFC project is to include the header file (MemTrack.h) in the source files. To make things easier, this header file can be included as the last #include in StdAfx.h, making the file visible throughout the entire MFC project. Since the header file (MemTrack.h) also includes a directive to automatically include the library (MemTrack.lib) at link time, the project's settings don't even have to be changed. Every call to new and delete will invoke the inline versions that track memory and produce a list of memory leaks (if any) at the end.
There are three limitations to this solution: 1) AppWizard's generated code contains new-expressions that are tracked (such as the IMPLEMENT_DYNCREATE macro), but the corresponding delete-expressions are buried in the MFC's library and go undetected, resulting in a few false memory leaks (that are fortunately easy to recognize); 2) explicit references to the global operator new and delete (i.e. ::new and ::delete) produce compiler errors; and 3) any object's implementation of operator new and delete will not be called. Fortunately, most applications are not impacted. Otherwise, memory leaks do not go incognito anymore.
Notes
[1] A C++ new-expression can take any number of arguments via the use of placement syntax:
T *ptr = new(arg1, arg2, ...) T;The arguments are provided to the new-expression immediately after the new keyword. When the compiler encounters this new-expression, it will call the corresponding allocator function (operator new) that has been overloaded to accept the extra arguments.
The term "new-expression" should not be confused with operator new. The former refers to an expression in the source code (such as the above) that invokes both the allocation and construction of an object. The latter refers to either a global or class-specific function that allocates memory. operator new does not invoke a constructor.
[2] This format is also the one used by the compiler to output errors.
Jean L. Gareau is Principal Engineer at Annasoft Systems in San Diego, CA, and the author of the book Windows CE from the Ground Up (Annabooks, 1999). He welcomes your comments at jeangareau@yahoo.com.