Dr. Dobb's Journal May 2000
As GUIs have become dominant on desktop systems, they have started taking hold in embedded systems as well. But even with many new UI innovations, the command line is the best solution for many situations. Windows CE is a perfect example. One of its greatest features is the ease of creating GUIs for embedded systems, even though the tools to build and download the operating system to a custom platform are all command-line driven. With the release of Platform Builder 2.12, the build process is becoming a lot more graphical in nature, but the main tool for downloading and communicating with the device remains the command-line CESH utility.
As well known and widely used as CESH is, there are some very useful features that are barely, if at all, mentioned in the documentation. Used correctly, these techniques let you completely automate a test suite to download and run on the device while logging results on the desktop machine. Because most devices have limited storage capacity for storing test-run results and unattended test runs save hours of tedious work, this represents a huge gain in productivity when developing a CE application. Also, since these features do not require Windows CE Services or ActiveSync 3.0, they work over a plain CESH connection -- a big win for custom platforms that don't have networking support. This is just the tip of the proverbial iceberg. In this article, I'll demonstrate many other uses of these features as well. In particular, I'll focus on three hidden techniques of CESH: remote file I/O with desktop machines, utilizing command lines from programs, and retrieving network address information about an Ethernet connection.
Remote file I/O lets programs running on Windows CE devices manipulate files on desktop machines connected via CESH. This is accomplished with five functions: U_ropen, U_rclose, U_rread, U_rwrite, and U_rlseek, the prototypes of which are in Listing One. If you've ever done any file I/O using the Standard C libraries, you already know how to use these functions. U_ropen and U_rclose are used to create and destroy a file handle, respectively. U_rread and U_rwrite transfer data to and from a file, and U_rseek moves the location for the next file operation to a specific offset. (The leading U in the function names indicates that the filename is always a Unicode string and leading r means that they perform remote file I/O.)
The fact that the functions perform remote file I/O is the key to their usefulness. When you call U_ropen, the parameters are marshaled into a data structure and sent over the CESH connection to the desktop machine. The desktop then unpacks the parameters, passes them to the Win32 CreateFile API, and sends the resulting file handle or error code back to the device. CreateFile is called from the running CESH process, so files without directory paths are created in the current working directory and can be specified before you run CESH by setting the value of the _FLATRELEASEDIR environment variable. The other four remote I/O functions work the same way, mapping to the CloseHandle, WriteFile, ReadFile, and SetFilePointer APIs.
The second parameter to U_ropen is a set of bit flags that specify the type of operations allowed on the file handle. Unfortunately, the Windows CE header files do not define these flags. Instead, you must grab the defines that are meant to be used with the desktop function open from the Visual C++ fcntl.h header file. Older DOS values used with the file open interrupt (0x21 function 0x6c for the curious) are supported as well. Both types of flags are conveniently defined in Listing One so you don't have to go searching all over creation for them. Missing from Listing One are flags to specify text or binary mode -- this is not an accidental omission. Using the _O_TEXT or _O_BINARY flags with U_ropen will cause it to fail for a simple reason: The CreateFile API doesn't support similar flags. Because U_ropen ultimately ends up in a desktop call to CreateFile, U_ropen has to limit its features to what can be supported with CreateFile. To this end, U_ropen mirrors the CreateFile behavior and does all file I/O in binary mode.
The first parameter to U_ropen doesn't have to be just a filename. It can include a relative or full directory path with or without a drive letter. It can also be a UNC filename that specifies a network resource such as "\\andrewt\drivec\foobar.txt." You can also take advantage of the fact that CreateFile allows access to resources other than files -- named pipes, mailslots, communication ports, consoles, and disk devices can all be created and accessed with the remote I/O functions. This opens up a world of possibilities. Error messages can be printed to the CESH prompt on the desktop by opening and writing to the console (via the filename CON). Communication between a desktop and device process can utilize a mailslot or named pipe, which is several orders of magnitude faster than using a temporary file. Other than being able to access overlapped I/O, anything that can be achieved with the desktop file I/O APIs can be achieved with remote file I/O.
To demonstrate how to use remote file I/O, I wrote RegDump, a utility to dump the device registry to a text file on the desktop (see Listing Two; the complete source code to RegDump is available electronically; see "Resource Center," page 5). RegDump recursively iterates through all the values of each registry key, formats them according to their data type, and uses remote file I/O to write the output to the desktop file regdump.txt. All error messages are printed to the CESH command prompt by opening and writing to the file CON. This may seem like a toy program, but it can be a useful (and sometimes the exclusive) method of viewing the device's registry on a headless system that has no visual method of interacting with the user. Platform Builder has some additional examples of using remote file I/O in some of the sample code in the public directory.
Once you can coax a program running on the device to manipulate files on the desktop, how does that process get launched in the first place? For a human interacting with the device, it's easy enough to double tap the filename, use the Run dialog, or take advantage of the s command from the CESH command line. What if you want a program to launch the device process instead of a human? Luckily, there's a way to talk to the CESH command line programmatically.
While CESH is waiting for interactive input at the command-line prompt, it also has a thread that is polling the file FSAUXIN for input. Any text written to FSAUXIN will be executed by CESH just as if it was typed interactively, except it is not echoed on the command line. By simply opening FSAUXIN and writing "s foo.exe" to the open file handle, you can programmatically launch foo.exe on the device. It's interesting to stop for a moment and consider how CESH implements the s command -- it uses remote file I/O.CESH to simply forward the command to the device, and the device launches the executable. The CE loader first looks in the local file system and then on removable media, such as Compact Flash or PCMCIA cards. If the file is still not found, the loader uses remote file I/O to read it from the desktop system. This also explains why it is much slower to run a program residing on the desktop rather than the device -- it is paged in over the communication link rather than being read from the device's local file system.
FSAUXIN is located in CESH's working directory, which can be customized by setting the FLATRELEASEDIR environment variable before starting CESH. Although manipulating FSAUXIN is no different than any other file, there are a couple of quirks that aren't readily apparent. First off, FSAUXIN must be opened for writing in append mode. Any other mode will result in the command text not getting executed at all. The other thing to be aware of is that buffering of any writes to FSAUXIN will cause the command to not be executed until the writes are flushed to disk. This is easy to work around by using the unbuffered I/O functions (open, write, and so on), turning off buffering with a call to setvbuf, or manually calling fflush on the file pointer after each write. Finally, trying to read from FSAUXIN will result in nothing but frustration. As the name implies, it is a one-way write-only channel to CESH.
Listing Three presents the CESHEXEC utility (also available electronically), which shows how to use FSAUXIN. The code assumes that the _FLATRELEASEDIR environment variable points to the current directory that CESH is running -- you don't have to be in that directory to use it. Any arguments passed on the command line are concatenated together, with spaces in between each one, and passed to CESH as a single line. I chose to write CESHEXEC in C, but it could easily be rewritten in Perl or any other language that has file I/O capabilities. A utility like this can be extremely useful for automating a test suite to run on the device. For an explanation of all the accepted commands, type "?" at the CESH command line or execute "ceshexec ?" while CESH is running.
There are some drawbacks and limitations to programmatically executing commands through CESH. There is no way to detect when a command fails or is not syntactically correct. The only clue that something went awry is the message at the CESH command line. If the command is successfully executed, there is no way to determine when it is completed. For example, if you use s to launch a program, CESH returns as soon as it verifies that the executable exists and sends the command to the device -- it does not wait for the remote program to start running. The only method of getting feedback from a command is to build it in yourself using a named pipe or temporary file techniques. Another issue with FSAUXIN is that writes are not serialized. If multiple writes are interleaved, whether from two programs or two threads in the same program, the input may be scrambled. If any of your writes to FSAUXIN have the possibility of overlapping, you should wrap them with a critical section or a mutex.
If you have been working on Windows CE for a while, you may recall that CESH started out as PPSH, short for "parallel port shell." PPSH had much the same capabilities as CESH, but could only communicate via the PC's parallel port. With the release of CE 2.1, PPSH was renamed to CESH to indicate transport-layer independence and support was added for Ethernet. This support adds the ability to connect multiple devices to a single PC and greatly improves the performance of OS image downloads and CESH command-line execution.
The other feature that is indirectly added with support for Ethernet connectivity via CESH is the ability to communicate over the network with sockets. This provides an alternative to the named pipe and temporary file solutions. The problem with sockets is that you have to know the IP address or machine name in order to establish a connection. Although this can be entered manually, it's much less problematic to have the program configure itself automatically. Luckily, CESH gives us a method for doing this.
Hidden deep in the header files included with Platform Builder is IOCTL_ HAL_GET_IP_ADDR. If you don't already know, an IOCTL (which stands for "I/O control") is simply an integer used in a call to the KernelIoControl API to read and write data that is not exposed by other APIs. Typically, an IOCTL provides an escape hatch for system integrators and device-driver writers to provide custom information and features without adding new APIs. For CESH, IOCTL_HAL_GET_ IP_ADDR lets the device retrieve IP and Ethernet MAC address information about the desktop machine that it is connected to. Using this information, you can easily start up a socket connection between device and desktop machines and stream bits back and forth. If you are familiar with the ppp_peer hack that works with Windows CE Services, you may recognize this as the Ethernet flavor of the same technique.
When used in Ethernet mode, CESH actually provides several different channels for kernel debugging, downloading, debug messages, and other services.
Listing Four shows how to use IOCTL_ HAL_GET_IP_ADDR. You simply call KernelIoControl with the IOCTL code, a DWORD containing the IPINFO_ DOWNLOAD constant, and an initialized IP_INFO structure. If the call succeeds and the IP_INFO structure's dwIP field is nonzero, then the address information was successfully retrieved. It is not sufficient to only check the return value from KernelIoControl. When CESH is connected over a parallel port KernelIoControl returns True, but no address information is actually contained in IP_INFO. The easiest way to detect this is to initialize IP_INFO before the call and check the value after a successful return from KernelIoControl. If the value is nonzero then the structure contains valid IP and MAC address values.
After you have the IP address of the desktop machine, communicating over a socket connection is straightforward. I don't have the space to cover the specifics here, but the code from my article "Using WinSock with Windows CE" (Windows Developer Journal, June 1998) could very easily be adapted to work in this situation.
I've shown how to manipulate files on the desktop over a CESH connection, programmatically execute commands from the CESH command line, and query for the IP address of a machine connected over an Ethernet CESH connection. My motivation has been to test suite automation, but this is certainly not the only possibility -- that's only limited by your creativity and imagination.
Thanks to Glenn Davis at Microsoft and Dave Orvis at BSQUARE for their helpful feedback on early drafts of this article.
DDJ
/* Function declarations and defines for CESH remote I/O functions */
#ifdef __cplusplus
extern "C"
{
#endif
/* stolen from pkfuncs.h */
int U_ropen(const WCHAR *, UINT);
int U_rread(int, BYTE *, int);
int U_rwrite(int, BYTE *, int);
int U_rlseek(int, int, int);
int U_rclose(int);
#ifdef __cplusplus
}
#endif
/* defines stolen from the desktop VC++ header file io.h */
#define _O_RDONLY 0x0000 /* open for reading only */
#define _O_WRONLY 0x0001 /* open for writing only */
#define _O_RDWR 0x0002 /* open for reading and writing */
#define _O_APPEND 0x0008 /* writes done at eof */
#define _O_CREAT 0x0100 /* create and open file */
#define _O_TRUNC 0x0200 /* open and truncate */
#define _O_EXCL 0x0400 /* open if file doesn't exist */
/* sequential/random access hints */
#define _O_SEQUENTIAL 0x0020 /* file access is sequential */
#define _O_RANDOM 0x0010 /* file access is random */
/* Simple utility to dump a the registry on a Windows CE device to a desktop file via the CESH remote file I/O APIs */
#include <windows.h>
#include "ceshio.h"
/* print a debug message to the CESH console on the desktop machine */
void DbgOut(LPSTR psz)
{
int fh = U_ropen(L"con", _O_CREAT | _O_WRONLY | _O_TRUNC );
U_rwrite(fh, (PBYTE)psz, strlen(psz));
U_rclose(fh);
}
/* Dump a single registry value by type */
#define LONIBBLE(x) ((x) & 0x0f)
#define HINIBBLE(x) (((x) & 0xf0) >> 4)
BOOL DumpRegValue(HKEY hkey, int nIndentSize, LPSTR pszValueName,
DWORD dwType, PBYTE pbValueData, DWORD dwDataSize, int fh)
{
while ( nIndentSize-- )
U_rwrite(fh, (PBYTE)" ", 1);
U_rwrite(fh,(PBYTE)pszValueName, strlen(pszValueName));
LPSTR pszType;
LPSTR pszData = NULL;
switch( dwType )
{
case REG_BINARY:
{
pszType = "(BINARY)";
const char *pcszHex = "0123456789abcdef";
pszData = (LPSTR)malloc((dwDataSize * 3) + 1);
for ( DWORD i = 0; i < dwDataSize; i++ )
{
pszData[i*3] = pcszHex[HINIBBLE(pbValueData[i])];
pszData[i*3+1] = pcszHex[LONIBBLE(pbValueData[i])];
pszData[i*3+2] = ' ';
}
pszData[dwDataSize*3] = 0;
}
break;
case REG_DWORD_BIG_ENDIAN:
case REG_DWORD: // same value as REG_DWORD_LITTLE_ENDIAN
{
if ( dwType == REG_DWORD_BIG_ENDIAN)
pszType = "(DWORD_BIG_ENDIAN)";
else
pszType = "(DWORD)";
wchar_t wchBuf[16];
pszData = (LPSTR)malloc(16);
wsprintf(wchBuf, _T("0x%x"), *(DWORD*)pbValueData);
wcstombs(pszData, wchBuf, wcslen(wchBuf)+1);
}
break;
case REG_MULTI_SZ:
{
pszType = "(MULTI_SZ)";
LPWSTR pwsz = (LPWSTR)pbValueData;
pszData = (LPSTR)malloc(dwDataSize+1);
pszData[0] = 0;
LPSTR pszIter = pszData;
while ( *pwsz )
{
wcstombs(pszIter, pwsz, wcslen(pwsz)+1);
pwsz += wcslen(pwsz) + 1;
if ( *pwsz )
{
strcat(pszIter, "\\0");
// make sure you overwrite previous zero terminator
pszIter += strlen(pszIter);
}
else
strcat(pszIter, "\\0\\0");
}
}
break;
case REG_EXPAND_SZ:
case REG_LINK:
case REG_RESOURCE_LIST:
case REG_SZ:
{
if ( dwType == REG_EXPAND_SZ )
pszType = "(EXPAND_SZ)";
else if ( dwType == REG_LINK )
pszType = "(LINK)";
else if ( dwType == REG_RESOURCE_LIST )
pszType = "(RESOURCE_LIST)";
else
pszType = "(SZ)";
if ( dwDataSize && pbValueData )
{
//NOTE: have to handle specially be
// cause string is not zero terminated?
LPWSTR pwsz = (LPWSTR)pbValueData;
int numchars = (dwDataSize/2)+1;
pszData = (LPSTR)malloc(numchars);
wcstombs(pszData, pwsz, numchars);
pszData[numchars-1] = 0;
}
}
break;
case REG_NONE:
pszType = "(NONE)";
break;
default:
pszType = "(UNKNOWN)";
break;
}
U_rwrite(fh,(PBYTE)": ", 2);
U_rwrite(fh, (PBYTE)pszType, strlen(pszType));
if ( pszData )
{
U_rwrite(fh,(PBYTE)" ", 1);
U_rwrite(fh,(PBYTE)pszData, strlen(pszData));
free(pszData);
}
U_rwrite(fh,(PBYTE)"\r\n", 2);
return TRUE;
}
/* Dump reg key values and recursively dump all subkeys */
BOOL DumpRegKey(HKEY hkey, int fh)
{
BOOL bSuccess = FALSE;
DWORD dwMaxSubkeys;
DWORD dwMaxValues;
DWORD dwMaxSubkeyLen;
DWORD dwMaxValueNameLen;
DWORD dwMaxValueLen;
static int s_iCurIndent = 0;
if ( RegQueryInfoKey(hkey, NULL, NULL, NULL,
&dwMaxSubkeys,&dwMaxSubkeyLen,
NULL, &dwMaxValues,&dwMaxValueNameLen,
&dwMaxValueLen,NULL,NULL) == ERROR_SUCCESS )
{
// dump this key's values...
DWORD dwValueNameLen = (dwMaxValueNameLen+1)*sizeof(wchar_t);
LPTSTR pszValueName = (LPTSTR)malloc(dwValueNameLen);
LPSTR pszValueNameA = (LPSTR)malloc(dwValueNameLen);
PBYTE pbValueData = (PBYTE)malloc(dwMaxValueLen);
DWORD dwIndex = 0;
while ( dwIndex < dwMaxValues )
{
DWORD dwSize = dwMaxValueNameLen+1;
DWORD dwDataSize = dwMaxValueLen;
DWORD dwType;
if ( RegEnumValue(hkey, dwIndex, pszValueName, &dwSize,
NULL, &dwType, pbValueData, &dwDataSize) != ERROR_SUCCESS )
break;
wcstombs(pszValueNameA, pszValueName, wcslen(pszValueName)+1);
if ( !DumpRegValue(hkey, s_iCurIndent, pszValueNameA,
dwType, pbValueData, dwDataSize, fh) )
break;
dwIndex++;
}
free(pszValueName);
free(pszValueNameA);
free(pbValueData);
// only succeeded if we did all the values
bSuccess = ( dwIndex == dwMaxValues );
if ( bSuccess )
{
// ...then enumerate other keys
DWORD dwKeyNameLen = (dwMaxSubkeyLen+1)*sizeof(wchar_t);
LPTSTR pszKeyName = (LPTSTR)malloc(dwKeyNameLen);
LPSTR pszKeyNameA = (LPSTR)malloc(dwKeyNameLen);
dwIndex = 0;
while ( dwIndex < dwMaxSubkeys )
{
DWORD dwSize = dwMaxSubkeyLen+1;
if ( RegEnumKeyEx(hkey, dwIndex, pszKeyName, &dwSize,
NULL, NULL, NULL, NULL) != ERROR_SUCCESS )
break;
strcpy(pszKeyNameA, "[");
wcstombs(pszKeyNameA+1, pszKeyName, wcslen(pszKeyName)+1);
strcat(pszKeyNameA,"]\r\n");
if ( s_iCurIndent )
{
int i = s_iCurIndent;
while ( i-- )
U_rwrite(fh, (PBYTE)" ", 1);
}
if ( U_rwrite(fh, (PBYTE)pszKeyNameA,
strlen(pszKeyNameA))!=(int)strlen(pszKeyNameA) )
break;
// recurse into the key if we can open it
HKEY hSubkey;
if ( RegOpenKeyEx(hkey, pszKeyName,
0,0,&hSubkey) == ERROR_SUCCESS )
{
s_iCurIndent += 4;
BOOL b = DumpRegKey(hSubkey, fh);
s_iCurIndent -= 4;
RegCloseKey(hSubkey);
if ( !b )
break;
}
else
{
DbgOut("ERROR: could not open registry key '");
DbgOut(pszKeyNameA);
DbgOut("'\n");
}
dwIndex++;
}
free(pszKeyName);
free(pszKeyNameA);
// only succeeded if we did all the keys
bSuccess = ( dwIndex == dwMaxSubkeys );
}
}
// do an extra crlf if we're finishing the top-level key
if ( !s_iCurIndent )
U_rwrite(fh, (PBYTE)"\r\n", 2);
return bSuccess;
}
int WINAPI WinMain (HINSTANCE hInstance, HINSTANCE hPrevInstance,
LPWSTR lpCmdLine, int nCmdShow)
{
int fh;
fh = U_ropen(L"regdump.txt", _O_CREAT | _O_WRONLY | _O_TRUNC );
if ( fh == -1 )
{
DbgOut("error creating file\r\n");
}
else
{
#define RKENTRY(x) { #x "\r\n", strlen(#x) + 2, x }
struct RKEntry
{
LPSTR psz;
int len;
HKEY hkey;
} RegKeys[] = { RKENTRY( HKEY_LOCAL_MACHINE ),
RKENTRY( HKEY_CURRENT_USER ), RKENTRY( HKEY_USERS ),
RKENTRY( HKEY_CLASSES_ROOT ),
};
int nEntries = sizeof(RegKeys)/sizeof(RKEntry);
// iterate the list of top level reg keys and dump each one
for ( int i = 0; i < nEntries; i++ )
{
if ( (U_rwrite(fh, (PBYTE)RegKeys[i].psz,
RegKeys[i].len) != RegKeys[i].len) ||
!DumpRegKey(RegKeys[i].hkey, fh) )
{
DbgOut("ERROR dumping key\r\n");
}
}
U_rclose(fh);
}
return 0;
}
/* Example code to show how to programmatically pass cmds to CESH */
#include <windows.h>
#include <stdio.h>
#include <io.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
char szFSAUXIN[_MAX_PATH];
char szCmd[_MAX_PATH] = {0};
// catenate the cmd into a single string up to the max length
if ( argc > 1 )
{
int iCurCmdLen = 0;
for ( int i = 1; i < argc; i++ )
{
if ( (iCurCmdLen + strlen(argv[i]) + 2) > _MAX_PATH-1 )
{
printf("Error: cmd string is too long. "
"Max size is %d\n", _MAX_PATH-1);
exit(EXIT_FAILURE);
}
sprintf(&szCmd[iCurCmdLen], "%s ", argv[i]);
iCurCmdLen += strlen(argv[i]) + 1;
}
strcat(szCmd, "\r");
}
else
{
printf("syntax: ceshexec <cmd>\n");
exit(EXIT_FAILURE);
}
// make sure we have a _FLATRELEASEDIR
LPCSTR pcszFRD = getenv("_FLATRELEASEDIR");
if ( !pcszFRD )
{
printf("Error: Environment variable "
"_FLATRELEASEDIR must be set\n");
exit(EXIT_FAILURE);
}
// build path to FSAUXIN, avoiding double slashes
if ( pcszFRD[strlen(pcszFRD)-1] != '\\' )
sprintf(szFSAUXIN, "%s\\fsauxin", pcszFRD);
else
sprintf(szFSAUXIN, "%sfsauxin", pcszFRD);
int fh = open(szFSAUXIN, _O_WRONLY | _O_APPEND);
if ( fh != -1 )
{
write(fh, szCmd, strlen(szCmd));
close(fh);
}
else
printf("Error: could not open '%s'\n", szFSAUXIN);
return 0;
}
/* Sample code to show how to get the IP and MAC address of the
desktop machine connected via Ethernet to CESH */
#include <windows.h>
#include "ethdbg.h"
#include "halether.h"
int WINAPI WinMain (HINSTANCE hInstance, HINSTANCE hPrevInstance,
LPWSTR lpCmdLine, int nCmdShow)
{
IP_INFO IpInfo;
DWORD dwAddrType;
DWORD dwLen = 0;
dwAddrType = IPINFO_DOWNLOAD;
memset(&IpInfo, 0, sizeof(IpInfo));
if (KernelIoControl(IOCTL_HAL_GET_IP_ADDR, &dwAddrType, sizeof(dwAddrType),
&IpInfo, sizeof(IpInfo), &dwLen))
{
if ( IpInfo.dwIP == 0 )
{
NKDbgPrintfW(L"No peer address information available. "
"Probably connected over a parallel port " "link\n");
}
else
{
NKDbgPrintfW(L"CESH peer address info: IP:%hs, "
"MAC:0x%02X:%02X:%02X:%02X:%02X:%02X\n",
inet_ntoa(IpInfo.dwIP),IpInfo.MAC[0],
IpInfo.MAC[1],IpInfo.MAC[2], IpInfo.MAC[3],
IpInfo.MAC[4],IpInfo.MAC[5]);
}
}
else
NKDbgPrintfW(L"Error getting peer address info: %u\n",
GetLastError());
return 0;
}