Columns


Standard C

A Matter Of Interpretation

P.J. Plauger


P.J. Plauger has been a prolific programmer, textbook author, and software entrepreneur. He is secretary of the ANSI C standards committee, X3J11, and convenor of the ISO C standards committee.

Introduction

I have just returned from a meeting of X3J11, the committee that developed the ANSI standard for C. That committee is now charged with answering questions about the standard. Anyone who needs some help interpreting the C standard can write a letter to the X3 Secretariat. The committee is to meet from time to time to review these requests for interpretation and to draft responses. This was the first interpretation meeting, since the C standard has just been approved by ANSI.

A certain amount of committee time went to learning a new way of doing business. Through most of its history, X3J11 concerned itself with writing the standard for C. If we found something confusing or unpopular, we focused on finding better wording to replace what was there. For the last year or so, the focus was on achieving closure. We were quick to make changes that clarified the intent of the standard or fixed obvious typos. We were much slower to make changes that were substantive (that changed the definition of the C language), unless the problem was too severe to ignore.

Now the committee is in a different position. It cannot change a single word of the ANSI C standard. It doesn't matter how confusing it might be. It doesn't matter if the committee had a different intent in the past. It doesn't matter if it has a really neat idea for how to do things better now. The C language is defined by what the ANSI standard now says. And it's likely to stay that way for at least the next five years.

What the committee can (and must) do is help others understand the ANSI C standard. A request for interpretation through official channels must be answered. The answer must be approved by a roll call vote. At least two thirds of those voting must vote in favor of the answer. The answer is then forwarded to the X3 Secretariat for further processing.

The X3 Secretariat places each answer in one of three classes:

  • Class A — Either the answer is found in one or more cited sections of the standard or the answer is outside the scope addressed by the standard.
  • Class B — The committee acknowledges that the standard contains a typographical error which could lead to an ambiguity. The ambiguity is resolved by citing earlier X3J11 documents to defend the interpretation put forth by the committee.
  • Class C — The committee must interpret the standard by "interpolation or extrapolation."
  • If X3 deems that the response belongs in Class A, it simply forwards the response to the inquirer. Otherwise, X3 must approve the interpretation by a 30-day letter ballot among its members. Approved responses are disseminated to the general public in the form of Technical Information Bulletins. Note that even a Technical Information Bulletin does not change the meaning of the ANSI standard. It merely supplies additional information to help people interpret the standard.

    The Easy Stuff

    The hardest part about shifting gears is realizing that the standard can no longer be changed. No matter how many years people spend reviewing and refining a technical document, infelicities still filter through. The old reflex of rewording to fix must now be stifled. We have to live with what's there, not rephrase our way out of problems.

    We also have to stop worrying about how we want the C language to be. That's cast in concrete now. We can discuss what we think the standard says. If there is ambiguity, we can debate whether it can be read the way we intended. Our choices are rather limited.

    These constraints simplify in many ways the job of interpretation. Many readers of the standard have the same attitude that the committee is trying to outgrow. They see an area that is confusing or surprising and immediately focus on how to fix it. What should be a simple request for interpretation turns into a suggestion for change.

    The committee has a simple answer to any such suggestions: "Thank you for sharing, but the time for changes has passed. You'll have to wait for the next revision of the C standard to address those issues." By X3's criteria, that's a solid Class A response. It is outside the scope of the current standard.

    Other responses are a little harder to formulate, but are still relatively easy. So long as a reasonable reading of the standard supports the committee's interpretation, X3 will still consider them Class A. The committee mainly has to point to supporting words within the document.

    Here is a fairly simple example. One of the committee members asked for a clarification of what an implementation can do with #pragma preprocessing directives. Many vendors use pragmas as the basic mechanism for altering the semantics of a C program to provide a variety of extensions. But some people on the committee have long felt that a strictly conforming program can safely contain pragmas. These two viewpoints lead to a fundamental conflict.

    If a strictly conforming program can contain pragmas, then what you can do with them must be severely circumscribed. You can turn listings on or off. You can turn debugging control on or off. But you can't change any behavior dictated by the C standard for a strictly conforming program.

    Whatever the intent of the members who voted for pragmas in their current form, the standard is now clear. Section 3.8.6, page 94, line 34, says that a #pragma preprocessing directive "causes the implementation to behave in an implementation-defined manner." It does not restrict what parts of the language may change. A strictly conforming program, however, may not use any implementation-defined language features that can alter the output it produces. Ergo, no strictly conforming program can contain a pragma (at least not one that survives conditional preprocessing).

    If an implementation doesn't have to worry about changing strictly conforming programs, it has considerable license. A conforming implementation can tolerate all sorts of extensions within conforming programs. It just can't alter the behavior of a strictly conforming program in nonstandard ways.

    This was an easy interpretation for the committee to make. It mostly involved reading the standard and drawing obvious conclusions. Those conclusions need not support the understandings agreed to in past committee meetings. They merely have to follow directly from a reading of the C standard. That's the easy part of interpretation.

    A Little Harder Case

    Now let's look at a slightly tougher case. Another one of the committee members asked for clarification of the strtoul function. We added it to the Standard C library as a companion to the older strtol. The formatted input (or scan) functions must now call strtoul for unsigned integer conversions. An implementation can no longer just call strtol and convert the result to unsigned. Even if the bits come out right, as they usually do on most two's complement machines, the overflow checking does not.

    You can get overflow when converting an input field with strtoul. Type in a 12-digit number, for example, on a machine where unsigned long int is 32-bits and the function will set errno to ERANGE. You can't get overflow when you perform arithmetic involving unsigned operands, but you can when converting to an unsigned type. Converting a floating type to an unsigned type can also overflow.

    Fine. All that is clear enough. Now let's add a small wrinkle. You can convert a field that contains a negative number with strtoul. The committee explicitly added this latitude because so many people habitually write -1 as shorthand for the largest unsigned value. (They probably shouldn't, for a variety of reasons. But they do.)

    So the question is, can adding a minus sign to a field cause strtoul to report overflow? If so, what is the lowest value that causes overflow? The standard doesn't explicitly state what happens. The result is a matter of interpretation.

    One interpretation is that you can't convert any negative field to unsigned without causing overflow. That is certainly true of floating point values. Any value less than or equal to -1.0 in principle causes overflow. (Since this is undefined behavior, an implementation has certain latitude. It can, for example, just go ahead and produce the low-order bits of the negative number, as some folks would prefer.)

    A less extreme view is that a negative value must be representable as a signed long int. Overflow occurs, then, only if the field value is less than LONG_MIN. That would give strtoul the same behavior for negative values as strtol.

    The most tolerant view is that adding a minus sign to a field can never change whether it overflows. Any magnitude between zero and ULONG_MAX converts without overflow. Negating the resulting unsigned long int is a well defined operation that never overflows.

    So which of these is the proper interpretation? (Or is there yet another?) The description of strtoul (Section 4.10.1.6) contains only two statements that bear on the treatment of negative field values. On page 153, lines 40-41, it says, "If the subject sequence begins with a minus sign, the value resulting from the conversion is negated." And on page 154, lines 3-4, "If the correct value is outside the range of representable values, ULONG_MAX is returned, and the value of the macro ERANGE is stored in errno."

    Now, this suggests that writing a minus sign in the field is not completely silly. Otherwise, it would say so. That rules out the first interpretation. And the value resulting from the conversion can be any value that can be stored in an unsigned long int. That rules out any notion that strtol rules should somehow apply.

    Negation is well defined for unsigned quantities. It never produces a value outside the range of representable values. A reasonable reading of the text, then, leads to the third interpretation. Overflow occurs only when the magnitude is too large to represent as an unsigned long int.

    You can't really falut the standard for not spelling this behavior out in detail. A standard that tries to do so for all cases would, in fact, be unreadable. For precision, you have to depend upon an economy of statement. Never repeat a requirement if you can avoid it, lest you say the same thing two slightly different ways. Rely on orthogonality of concepts to specify combinations too numerous to describe separately.

    Admittedly, the description of strtoul could have been better. Had we anticipated any confusion over the use of minus sign, we could have chosen our words more carefully. Just a slight rephrasing would have emphasized how overflow is determined and how negation is performed. We didn't get is perfect, but we got it close enough to support a reasonable interpretation.

    A Still Harder Case

    Now let me describe a real interpretation problem. This one occupied considerable air time at the last X3J11 meeting. It remains unresolved. Two rather different interpretations are supported by the standard. Neither can be clearly favored by recalling the intent of the committee. The matter wasn't addressed in sufficient detail in past meetings.

    What started the discussion was a request for interpretation from the Free Software Foundation. It seems that the GNU C compiler wants a certain latitude in how it returns a value from a function returning a structure type. Depending on your philosophy, that latitude may or may not be a good idea.

    To ensure that generated code is reentrant, each call to a function returning a structure type must provide the storage for the return value. That storage can be a secret temporary on the stack. Often, however, the return value is immediately stored in a data object. It seems silly to have the function copy the return value into the temporary, then have the caller copy the value again to its final destination.

    An obvious optimization is for the caller to pass the address of the final destination to the function as the place to store the return value. It eliminates a double copy which, for large structures in particular, can be expensive. It looks, on the surface, like nothing but a good idea.

    One problem can arise, however. Say the data object you are assigning to overlaps the structure from which the function obtains its return value. If it is exactly the same data object, the copy operation is redundant but should work correctly. Now say that the overlap between source and destination is only partial. Then, depending on how the bytes are copied, you can get a curdled result.

    A partial overlap of structures is rare but not impossible. A union can contain two members each of which contains an instance of the same structure type. If the instances do not lie at exactly the same offset within each member, they can overlap partially. For example,

    struct x {.....};
    union y {
        struct x member1;
        struct {
            char c2;
            struct x s2;
            } member2;
        } u;
    u.member1 = u.member2.s2;
    It so happens that the standard directly addresses assignments of this form. Section 3.3.16.1, page 54, lines 33-35 says, "If the value being stored in an object is accessed from another object that overlaps in any way the storage of the first object, then the overlap shall be exact and the two objects shall have qualified or unqualified versions of a compatible type; otherwise the behavior is undefined."

    That clearly states that this assignment is chancy. It warns the programmer not to indulge in such practices, lest the program do something unexpected and undesirable. It also gives implementors considerable latitude. They can go ahead and get the "right" answer anyway. Or they can allow such practices to generate bad code. In either case, an implementation has no obligation to warn the programmer. This is strictly a case of caveat emptor.

    But what if you take the same declarations as above and add:

    struct x frs(struct x);
    u.memberl = frs (u.member2.s2);
    The function may choose to return its argument as the value of the function. If it does so, is it covered by the same caveat as for direct assignment? In other words, is the result undefined if the source of the function value happens to overlap the eventual destination for that value?

    You can see why an implementor would want the same caveat to apply. It gives the translator permission to make the obvious optimization without worrying about any subtle consequences. Should the function return a value that overlaps its destination, the copying operation is allowed to screw up. It is up to the programmer to avoid situations where an unfortunate overlap might occur.

    On the other hand, you can see why a programmer might want the caveat not to apply. It is one thing to be restricted in how you write assignments. At least both left and right hand operands are more or less obvious. But function calls can obscure any connection between left and right hand side. It is a greater burden to check code for subtle situations where partial overlap might occur.

    Two fundamental philosophies are at war here. On the one hand we have the folks who believe that C should execute as fast as possible. The translator should err on the side of efficiency. Leave it up to educated users to avoid dangerous case. On the other hand we have the folks who believe that C should be made into a safer language. Don't permit subtle errors to arise just to make the language easier to optimize in all areas.

    The caveat I quoted above was added to assignment statements purely for the sake of the former camp. We decided that structure assignment should not have to generate fail safe copying code. It could assume no overlap or exact overlap of operands.

    It is not clear, however, that the committee intended the caveat to extend to implicit assignments. Returning a value from a function is such an implicit assignment. The committee was clear that functions returning structures must be reentrant. That prevents them from returning values by storing them in a static data object. (Some existing C compilers generate code that does this.) Requiring reentrancy does not exactly require copying the value to a temporary.

    At least not to everybody. It is a matter of interpretation.

    And that's not the end of it. The issue may have been raised in terms of functions returning structures, but it goes much further. If a function call can muck up the copying of a value, why not another operator as well? It is hard to extend the license to fail a little bit without extending it a lot.

    And there is nothing in the words I quoted that is peculiar to structure types alone. I have certainly written C compilers for small machines that copy double operands around the same way that structures are copied. If structure copies can work incorrectly, so too can double copies. So, in fact, can copies of practically any data type.

    So the request from the GNU folks raises serious issues. Whatever the committee decides will likely fall well outside Class A. It will take more discussion, careful thought, and clear reasoning to develop an acceptable interpretation. It won't be easy.

    Conclusion

    As always when I criticize the C standard, I feel the need to put matters in perspective. Focusing on the things that aren't nice makes them loom much larger than they really are. Great expanses of the standard read just fine. They describe a language that is largely unchanged from the days of Kernighan and Ritchie's first description. They describe the language with a laudable precision.

    Remember that people request interpretations only when they get confused or surprised. Confusion and surprise arise in those rare places where the wording of the standard is spongy. With all the people proofreading drafts these last couple of years, you can bet that the spongy places tend to describe second order aspects of the language.

    X3J11 was complimented by ANSI on the relative cleanliness of the final draft. ANSI found few places where they required typographical fixes or changes in style. For all the standard has been critically examined over the past year or so, it has generated relatively few requests for interpretation. I think it's in pretty good shape.

    Please keep that perspective in mind as you digest this column. Try to keep it in mind for at least another month. I will probably remind you next month anyway. That's when I discuss what's really wrong with the ANSI C standard.