One of my major motivations in writing the Standard C library was to prove in the multibyte functions. It's nice to know whether they can be written efficiently, or whether they can be written at all. I was pleased to find that the basic specification in the C Standard is indeed viable.

What's hard is to write the critical functions to be usable with multiple character sets. Even within Japan, several multibyte encodings are popular. I didn't want to write multiple specialized versions of these functions. And I didn't want to require lots more coding to support Chinese, Korean, Arabic, or other large character sets around the world. That proved to be the fun part.

The functions I present here are the ones declared in the header <stdlib.h>. They are the bare minimum needed to support multibyte characters (sequences of one or more bytes to represent each character), wide characters (fixed-size integers that can represent all possible characters), and conversions between these two forms. Both the ISO C and C++ standards will soon incorporate a richer set of additional functions for manipulating large character sets.

#include <stdlib.h>
int mblen(const char *s, size_t n);

mbtowc((wchar_t *)0, s, n);

Forward references: the mbtowc function (7.10.7.2).

#include <stdlib.h>
int mbtowc(wchar_t *pwc, const char *s, size_t n);

The implementation shall behave as if no library function calls the mbtowc function.

In no case will the value returned be greater than n or the value of the MB_CUR_MAX macro.

#include <stdlib.h>
int wctomb(char *s, wchar_t wchar);

The implementation shall behave as if no library function calls the wctomb function.

In no case will the value returned be greater than the value of the MB_CUR_MAX macro.

#include <stdlib.h>
size_t mbstowcs(wchar_t *pwcs,
const char *s, size_t n);

No more than n elements will be modified in the array pointed to by pwcs. If copying takes place between objects that overlap, the behavior is undefined.

#include <stdlib.h>
size_t wcstombs(char *s, const
wchar_t *pwcs, size_t n);

No more than n bytes will be modified in the array pointed to by s. If copying takes place between objects that overlap, the behavior is undefined.

Footnotes

131. If the implementation employs special bytes to change the shift state, these bytes do not produce separate wide character codes, but are grouped with an adjacent multibyte character.

132. The array will not be null- or zero-terminated if the value returned is n.

#include <stdlib.h>

int mbcheck(const char *s)
   {  /* return zero if s is valid */
   int n;

   for (mblen(NULL, 0); ; s += n)
      if ((n = mblen(s, MB_CUR_MAX)) <= 0)
         return (n);
   }

mbtowc — Use this function much the same as you would mblen, described above. Two differences exist between the functions mbtowc and mblen:

• If the first argument to mbtowc is not a null pointer, the function returns the wide character it converts. Thus, you can translate a single wide character at a time, unlike mbstowcs which translates the entire string at once.

wcstombs — Use this function to convert an entire wide-character string to a multibyte string. You needn't worry about whether locking shifts occur, since the function processes the entire wide character string. You also needn't worry that the resultant multibyte string is too long, since the third argument n limits the number of elements stored. If the function returns n, the conversion was incomplete. If the function returns a negative value, the wide-character string is invalid.

wctomb — Use this function to convert a wide-character string to a multi-byte string one wide character at a time. Here, for example, is a function that checks whether a wide-character string has a valid encoding:

#include <limits.h>
#include <stdlib.h>
int wccheck(wchar_t *wcs)
   {  /* return zero if wcs is valid */
   char buf[MB_LEN_MAX];
   int n;

   for (wctomb(NULL, 0); ; ++wcs)
      if ((n = wctomb(buf, *wcs)) <= 0)
         return (-1);
      else if (buf[n - 1] == '\0')
         return (0);
   }