Leor Zolman has been involved with microcomputer programming for 15 years. He is the author of BDS C, the first C compiler targeted exclusively for personal computers. Leor's first book, Illustrated C, is now available from R&D Publications, Inc. Leor and his family live in Lawrence, KS.
This is the second installment in a series of columns describing the CMENU menu compiler system. Last time I introduced the CMENU specification language's syntax, and described the data structures used by the cmenu translator (pre-compiler). This month I'll present the entire procedural section of the cmenu program.
Doing Files
cmenu's main function passes the name of each file to be processed, in sequence, to the dofile ( ) function.dofile( ) (beginning at line 40 of Listing 1) processes a cmenu specification file from start to finish. The first few lines chop off the .mnu extension in case the user included it in the filename. Two copies are then made of the base filename. One (src_name) gets the source file extension appended onto it, and the other (obj_name) gets the object file extension.
If opening the source file succeeds, lines 62-65 initialize some global modes and counters. The file is entirely processed within the token loop described earlier.
After an EOF is encountered and the main loop terminates, the file is closed and several checks for possible error conditions are performed. In line 85 we make sure that at least one menu was defined, or there wouldn't be much point in writing an output file.
Lines 88-93 check if the last menu terminated correctly. If the value of global in_menu is TRUE, then there was an endmenu keyword missing. At this point, we also call the itemcheck( ) function to see if the last item in the last menu was similarly incomplete. (An item isn't complete until both text and action clauses for that item have been processed.)
The last piece of error checking is performed in lines 95-101, a test for unresolved forward menu references. The MInfo array is scanned for entries having a Processed flag still set to FALSE. Menu entries are created as soon as they are first referenced (unlike item entries, for the reasons given above). It is, therefore, possible that a menu entry referenced in an lmenu statement was never defined or the identifier was misspelled somewhere. Either way we get an unresolved menu reference.
Finally, if there weren't any fatal errors encountered, the object file for the current menu is written to disk and dofile( ) returns.
Utilities, Utilities
The remaining functions in cmenu1.c provide support for creating entries in the menu and item info tables, searching for those entries by label name, checking for item completeness, writing the output file, string matching, and reporting errors and warnings.Menu info table management is performed by create_menu( ) and find_menu( ). Because MENU structures are stored directly, there is no need to worry about memory allocation when creating a new MINFO entry. create_menu( ) defines a local MINFO structure, initializes it appropriately, and returns it by value to the calling routine. To find a given MINFO entry, find_menu () iterates through the MInfo array, comparing the name of each registered entry with the name string supplied. When a match occurs, a pointer to the structure having the matching name is returned.
The item info table is managed by create_item( ) and find_item( ). find_item( ) works like find_menu( ). Because IINFO structures are stored by reference (to reduce memory requirements), create_item( ) must allocate a memory block for the new IINFO structure and return a pointer to that block. Provided there wasn't any problem with obtaining the memory, both the IINFO structure and its member INFO structure get initialized with all the default values appropriate to an unprocessed item entry.
The itemcheck( ) function is called from a few places in cmenu to make sure both required portions of an item definition, the text and an action clause, have been specified.
When a menu specification file has been completely processed and no fatal errors occurred, the write_file( ) function is called to create the output file on disk. After creating the output filename and opening the file for writing (in binary mode), the first thing write_file( ) actually writes is the value of the global menu count, n_menus. Each menu is then written to the file in a format consisting of the MENU structure first, then each associated ITEM structure. There is no need to write an explicit item count for each menu because that information is already part of the MENU structure, and rmenu can obtain that value dynamically when the .mnc file is loading for execution. To insure portability across machines with different storage requirements for various variable types, all reads and writes involving the .mnc file are performed using the sizeof operator to determine how many bytes to transfer. Even writing a simple integer value, such as the menu count (line 273), should employ sizeof rather than a constant byte count value such as 2 or 4.
After each item is written to disk, its memory block is freed (line 299). If there are many menu definitions in a single menu specification file, then many blocks of item memory will end up having been allocated and freed repetitively. Since, however, cmenu is only run occasionally, the extra overhead this repetition entails isn't enough to have a noticeable effect on performance.
In rmenu, realtime efficiency is far more critical because many processes may be active simultaneously. (I often see between ten and fifteen rmenu processes running concurrently at the office.) Later we'll see how rmenu employs a slightly more complex allocation strategy to avoid the memory thrashing problem.
Error Reporting
The next set of functions in cmenu1.c are warning( ), error( ), and fatalerr( ) (lines 307-394). These represent variations on a single theme: display a diagnostic message on the standard error device (usually the user's terminal). All three functions take a variable number of arguments, allowing format conversions like printf( ) to be supported. Since pre-ANSI and post-ANSI C differ in their provisions for functions that accept a variable number of arguments, I found it necessary to include two different versions of the starting sequence in each of the three error reporting functions.ANSI C function headers allow the specification of ellipses (...) to indicate a variable number of arguments in a function header or prototype. The macros va_list and va_start, used in the processing of the variable length argument list, are taken from the <stdarg.h> standard header file (see line 16).
Pre-ANSI C, on the other hand, cannot handle ellipses in function definitions. Before the existence of ellipses and <stdarg.h>, functions taking a variable number of arguments were processed with the macros defined in <stdarg.h>'s predecessor, <varargs.h>. <varargs.h> contains alternate versions of the macros va_list and va_start, plus an additional macro named va_alist that is used in function headers as the placeholder for optional arguments.
The usages of <varargs.h> and <stdarg.h> differ only up to (and including) the appearance of the va_start macro. After that, the methods are equivalent, so the conditional portions of the three error reporting functions end at that point.
Each diagnostic function calls fprintf( ) to send the name and current line number of the source file involved to the standard error stream. Then they call vfprintf( ) to send the specific error information suppled in the call to the standard error stream. The only difference between the three functions is in the way they affect the global error flags. warning( ) sets no flags. error( ) sets only err_flag. fatalerr( ) sets both err_flag and fatal. Setting the fatal flag causes processing of the current source file to be terminated immediately upon return to the main processing loop, while err_flag is sampled only after the current source file has been completely processed (without fatal errors) to determine if an object file should be written.
String Matching
The matchkey( ) function searches the keyword table to see if a keyword exists matching the given string, and returns the token value for that keyword if a match is found.The strstr( ) function tests if one given string is a subset of a second given string. strstr( ) is supplied in case your library does not already include it. Compilation of strstr( ) is conditionally controlled by the NEEDSTR symbolic constant defined in the makefile.
Parsing The Input Stream
The two major components of cmenu remain to be discussed: token parsing, and token processing. Because tokens must be recognized before they can be processed, I'll begin with a description of the token parsing process. I'll then tackle token processing to tie everything else together.Tokens are the basic syntactic objects manipulated by cmenu. Each token is represented by an integer value, sometimes in conjunction with the value of an auxiliary variable. If the token represents a keyword or special condition, then the integer value alone is sufficient to qualify it. If the token represents a string or a numeric value, then the token value T_STRING or T_VALUE in conjunction with the text in global array tparam (for strings), or the integer value of the global vparam (for values) is needed to fully qualify the token. You'll find definitions for tparam and vparam in ccmenu.h (see CUJ, January, 1992). The token values are defined in lines 41-99.
The token parsing code resides in cmenu3.c (Listing 3) . There are three functions in this source file, two of which are called from other parts of the program: gettok( ) to get the next token, and ungettok( ) to "unget" a token. The last function, getword( ), is called only by gettok( ).
getword( ), The Workhorse
getword( ) does all the grunt work in the token parsing process. It recognizes and skips over comments and whitespace, keeps track of the current line number, recognizes both quoted and unquoted text strings, and maps unquoted text strings into lowercase. getword( ) returns a pointer to a statically allocated text string containing the text of the next token from the input file without making any attempt to distinguish a keyword string from straight text. (That job is handled by gettok( ).) Starting in line 126 (of Listing 3) , getword( ) checks for all the special cases just mentioned. Newlines cause the lineno global to be incremented, while other whitespace is ignored (including commas and semicolons). The only non-alphabetic keyword, the colon, requires a special case. This simplifies the code to recognize other tokens (lines 146-147).Comments are handled by getword( ) in lines 149-162: all characters after the leading # are ignored until a newline is encountered (or EOF, should the trailing newline be missing). The line count is then incremented.
Lines 164-200 process quoted strings. First, the quoted_text flag is set to TRUE so gettok( ) will know a string has been found. Then the string is collected into the tok array, with appropriate action being taken when certain special characters are encountered. Because multiline strings are not supported, a newline in a string is treated as an error, as is an EOF condition. If a double quote is encountered, the string is terminated with a zero byte and a pointer to the string is returned to gettok( ).
Finally, an unquoted string (probably a keyword or an identifier, but possibly a short action string or a pathname) is processed in lines 202-211. Any of the usual separators or a colon terminate such a string. While it is being collected up into the tok array, the string is converted to lowercase. As soon as an illegal character is found, it is "ungotten" and a pointer to tok is returned.
Back To gettok( ) And ungettok( )
At the top of cmenu3.c (lines 14-91), several items of static data are defined to support the unget feature for tokens.This is a necessary feature for a simple language processor like cmenu, because in some cases the code must scan past the end of a clause to determine that the clause has come to an end. The unget feature allows any token processing function to give back the extra token. Then, next time the gettok( ) function is called to get a token from the input stream, the ungotten token will be returned again. Only a single level of ungetting is needed to process the CMENU syntax.When gettok( ) is called it first checks for an active ungotten token. If found, all the saved static token detail variables are copied into th eir global counterparts and gettok( ) returns the saved token value.
If there wasn't an active ungotten token, it is time to get a new one. The global detail variables tparam and vparam are cleared, and getword( ) is called to fetch the next syntactic object from the input stream. Lines 85-98 handle the easy kinds of tokens: EOF, quoted string, colon, or simple reserved word. (In the case of quoted text, the text needs to be copied into tparam.) Lines 100-104 process integer values by setting vparam if a leading digit is detected and returning T_VALUE. If we reach line 105, the token is treated as an unquoted string: the text is copied into tparam, and T_STRING is returned.
The ungettok( ) function (lines 29-49) first checks to make sure there isn't already a token pushed back. (There shouldn't ever be if the program is working correctly; this test was put here only to catch possible development bugs.) Then all the token detail values are saved in the static variables.
Token Processing
The cmenu code discussed up to this point all plays a supporting role to the actual token processing functions in cmenu2.c. The call to each token processing function is placed indirectly through the function pointers stored in the keywords table. The dispatching takes place in the main processing loop in cmenu1.c.As each token processing function receives control, it can count on the availability of certain information through global variables. We've already seen how most of these global variables are managed. There's one more, named token, that contains the most recently scanned token value (responsible for arrival at a particular function in cmenu2). This variable is assigned in the main processing loop, immediately before the dispatch is performed. We need to know this value, because some token processing functions can handle more than one token, and they examine the value of token to determine which token they have been called to process.
The Menu Clause
To start at the beginning, the do_menu( ) function (Listing 2, line 20) handles the start of a menu definition. The first thing that happens is a check to see if the previous menu in the program was properly terminated with endmenu. If it wasn't, then do_endmenu ( ) is called and a warning is issued. If it was just a missing endmenu statement, the compilation will still (begrudgingly) succeed.do_menu ( ) then checks for the existence of a menu label by calling gettok( ) and seeing which token turns up next. If it is not a string, then no label was given. This is only tolerated at the start of the file, because the omission of a label in subsequent menus would prevent that menu from ever being accessible (via the lmenu action). If the first menu is missing a label, a dummy label is stuffed into tparam.
If the label is too long it is truncated (lines 38-43), and then a check is made to see if the name appeared previously. If not, then we have the simple case of a new definition, and lines 47-49 create an entry for it in the MInfo array.
If the name has been used before (i.e., there's already an entry in MInfo for it), then there are two possible explanations. Either it was used in a forward lmenu reference, which is legitimate, or the label has already been used in the definition of a previous menu, constituting an error (line 55). We can tell which case it is by examining the Processed flag associated with that MInfo entry.
When we get to line 57, MIp points to the MInfo entry for the new menu. We now need to modify a group of elements of the Menu structure that is itself an element of the MInfo entry. So, we assign the address of the Menu structure to the pointer Mp, and then initialize everything in the Menu structure through that pointer. Mp remains a valid pointer to that Menu structure after return from do_menu ( ); several other functions will take advantage of it.
After all the attendant mode and structure initialization, we check for a trailing colon in lines 68-69 and, if found, ignore it.
Menu Options
The next six functions (lines 75-257) deal with all the possible menu option statements that, if present, must appear before the initial item clause.do_title( ) and do_path( ) are very similar, each verifying the existence of its required text string parameter, making sure the option hasn't appeared before, and stuffing the string into the appropriate element of the current Menu structure. (Mp comes in handy here.)
do_path( ) performs all the steps above, plus one more: it deletes any trailing path delimiter character (slash or backslash) found at the end of the path text. This prevents two consecutive path delimiter characters from appearing in a path string when incremental paths are glued together.
The do_align( ) function was originally intended to support alternative ways of aligning the item text on the screen. I later changed my mind about the usefulness that option and never taught rmenu how to recognize it. The cmenu code for processing the option remains, if someone thinks of a reason for rmenu to use it.
do_spacing( ) and do_columns( ) are similar to do_title( ) and do_path( ), but they take an integer instead of a text string for their parameter. I don't think they require any further annotation.
The do_escape( ) function handles both the escape and noescape options, checking for the usual error conditions and then setting the escape flag in the Menu structure to either YES or NO. If this option does not appear, the default value for the escape flag ends up being DEFAULT, which has a very different meaning from both YES and NO.
The do_endmenu( ) function pulls clean-up duty after a menu definition. If no items were found, that merits an error message. The forward reference table is then checked to make sure all references have been resolved. If an unresolved reference is found, then a little shuffle-play is performed involving the global line number variable, lineno. This forces the error message function to report the line number where the unresolved reference was made, instead of the current line number.
Finally, global modes are reset in lines 284-287 with values indicating a "not in menu" status. The Processed flag is set to show that the menu has now been defined, and the number of items found is stashed away in the Menu structure (squeezing some final mileage out of old venerable Mp).
The Item Clause
There is a lot of parallelism between the set of functions that process item related statements and the set we just saw for handling menu related ones. I suppose one could even describe the relationship between menus and items as fractal, because the structure somewhat repeats itself. There is an item clause with an optional identifier, followed by a set of item options, some of which are required and some are optional. In the case of item definitions, however, there is no requirement that the optional clauses precede the required ones (the way menu options must precede all menu items).There is no explicit keyword to mark the end of an item definition. The first thing, therefore, the do_item( ) function (lines 291-357) must do is check that the last item, if any, defined in the current menu included its required clauses. The call to itemcheck( ) in line 303 does this.
Next, we test if a label is attached to the item definition. If not, a dummy name is constructed; if so, the identifier name used is checked for legality in lines 311-326.
Lines 328-329 make sure that the label name has not been used before in the current menu. Even if a forward reference has been made to this label, it will not show up in a call to find_item( ) because the forward reference information has all been segregated into the fwd_refs table (to be examined shortly).
Now we can create an entry for the new item, and place a pointer to it in the Items array. The create_item( ) function (Listing 1, lines 160-192) allocates memory for the new item info structure, initializes its members, and returns a pointer to the structure. If there is a problem creating the array, create_item( ) returns NULL and the process is aborted.
Having survived to line 335, the global mode in_item is set and a handy ITEM pointer, Ip, is set to point to the ITEM structure element of the IINFO structure just created. Ip may then be used throughout the remainder of this item's processing.
Lines 338-340 resolve any possible forward references to the new item. The fwd_refs table is scanned for an entry whose name matches the new item's. If one is found, the lmenunum element of the item where the reference was made is set (indirectly, through the refp pointer) to reflect the new item's index value. This overwrites the previous item's lmenunum value of UNDEF_FWD, thereby resolving the forward reference.
The last part of the do_item() function checks to see if the item text is present as part of the item clause. To prevent ambiguity, I've required a colon to precede any item text included directly in an item clause. Without this restriction, the parser couldn't know if text found after the item keyword was meant to be a label or an item text string. At this point in do_item() (line 344), anything other than a colon terminates the item clause processing.
Even if a colon is spotted, there may not be any item text given. Lines 350-351 check for the item text and register it via do_text2( ) if present.
Item Options
The next six functions handle the various item options and the required action clause. By now the pattern of these token processing functions should be familiar. I'll skip the routine details and point out the highlights.The do_opts( ) function handles a whole slew of trivial binary options by setting Item flags to values triggered directly by the option tokens. There are three functional categories processed by do_opts( ): prompting, pre-clearing and post-clearing. Only one option from each of those three categories is permitted to appear within any one item definition. The flag associated with each category is initialized to the value DEFAULT. It keeps that value until the appropriate statement forces the value to either YES or NO.
The do_nextitem( ) function processes the nextitem clause, so the user may alter the menu system's flow of control. One of four variations must be used. The last is to supply a label for the next item to be highlighted. If the next token after nextitem is a string, then find_item( ) is called to see if the item has been defined. If it has, its index is inserted into the nextitem element of the Item structure and no forward referencing is involved. If find_item( ) couldn't find it, lines 432-437 install the item reference into the forward reference table, for resolution when the item definition with the given label finally appears. If you missed it, see "The Item Clause" section above for related details.
If the screen text for the menu item was not specified in the item clause, it must appear in a subsequent text clause. The do_text( ) and do_text2( ) functions process this option. After do_text( ) has checked the basics, do_text2( ) tests whether the string is too long to store in the available space. If it's too long, a message is printed and no attempt is made to copy the string into the ITEM structure.
After copying a reasonable length string in through the Ip pointer, the flag named widest (in the MENU structure) is updated to reflect the length of the longest item text seen so far. This value will be used by rmenu when it comes time to decide how all the menu items will be arranged on the screen.
Springing Into Action
There must be exactly one action associated with each menu item, from a choice of four possibilities. The do_action( ) function is set to process all four actions, each using a different token sequence.The exit action may be specified either with or without the action keyword preceding it. Both forms are valid.
The basic action clause is specified by the action keyword and a text string. The string represents an operating system command (or sequence of commands separated by semicolons). All do_action( ) does is copy the string (through Ip) into the action element of the current ITEM structure (line 527). The emenu action, which calls an external menu, is processed in the same way, except that the acttyp element of the ITEM structure is set to identify the action string as an external menu name rather than a command to be passed to the system's command interpreter.
The fourth type of action is the lmenu clause, specifying a local menu to run. If no menu by the given name is found in the file, a menu structure is created and added to the MInfo array, and the index number of the new menu entry is plugged into lmenunum element of the item being processed.
There is nothing new to say about the do_help( ) function, so I won't.
The final function in the module, do_err( ), gets called whenever a keyword is encountered unexpectedly (for example when the first keyword is seen without a leading nextitem). In the keywords table array, any keyword that does not represent the beginning of a legitimate option sequence is assigned do_err( ) as its processing function. By calling the fatalerr( ) function to print its error message, do_err( ) forces compilation to be terminated immediately after the error message is printed, allowing the user to fix the syntax without having to deal with additional spurious error messages.
Intermission: dmenu
This concludes my illustration of the cmenu program. Before delving into rmenu in the next installment, I'd like to introduce a little diagnostic utility named dmenu. I wrote this short program after eliminating the simple syntax errors from my initial stab at the cmenu code, but before rmenu was written. At that point, I needed a quick, easy way to examine cmenu's output files for debugging purposes. dmenu reads an .mnc file and disassembles it into human readable form, providing a complete picture of a compilation job performed by cmenu. With the help of dmenu, I was able to debug cmenu quickly, without even needing to have menu available. The code for dmenu.c is shown in Listing 4.