Tim is the author of The Working Programmer's Guide To Serial Protocols (Coriolis Group, 1995) and can be contacted at kientzle@netcom.com.
While Internet mail is a wonderful tool, it currently has a major shortcoming. Because it was developed to handle 7-bit text messages, Internet mail is unsuitable for transferring binary data such as word-processor files, audio, graphics, and other useful data. In particular, the 8-bit character sets now in use in most of the world aren't supported by the standard mail transport protocol (SMTP).
Clearly, the handling of Internet mail needs to change, but there are many obstacles. Any major change to the Internet mail transport would take years. During that interval, "islands" of enhanced mail capability would be unable to exchange mail over the sea of existing 7-bit mail transport.
In lieu of an enhanced mail capability, savvy users have for years been encoding their binary data into a form compatible with existing 7-bit mail systems. The Multipurpose Internet Mail Extensions standard (MIME) formalizes and automates this process, augmenting existing mail programs to automatically encode and decode data with a minimum of user intervention. Since only the programs used to read and compose messages require modification, MIME provides enhanced mail facilities without rewiring the Internet.
The basic definition of Internet mail is contained in RFC822, which states that a mail message consists of header lines followed by a message body. While RFC822 describes the syntax of header lines in considerable detail, it is less precise about the body: "The body is simply a sequence of lines containing ASCII characters." MIME augments this by adding five new headers which, among other things, specify the precise format of the message body; see Table 1.
MIME specifies the format of the message body in three layers. The first is a broad type, which identifies the general kind of data. By itself, the type doesn't provide enough information for the reader to do anything useful, but it does help the reader select a default handling for certain classes of messages (for example, text formats might be simply listed to the screen, while unrecognized image formats would not).
The second layer is the subtype. The type and subtype together specify the exact kind of data in the message; for example, image/gif. The third layer specifies how the data is encoded into 7-bit ASCII for transfer.
The Content-Type header contains a type and subtype separated by a "/" character, followed by a list of keyword=value pairs. For example, the Content-Type text/plain; charset=iso-8859-8 might be used for a plaintext file containing characters in the ISO Roman/Hebrew character set. If the display supported Hebrew characters, the mail reader could (after decoding) display the text as it was intended by the sender.
As Table 2 illustrates, there are currently seven defined types. The first four types in Table 2 indicate a single data file in a single format; their subtypes are listed in Table 3. These basic content types are an improvement over text-only mail, allowing messages to contain graphics, sound, or other data types. They are also easy to support; mail readers only need to parse the Content-Type and Content-Transfer-Encoding headers, decode one of two simple data formats, and pass the result to a separate viewer program.
The message and multipart types provide features that can reduce mail-delivery costs and allow single messages to combine different kinds of data.
The message content type provides three important capabilities. Message/rfc822, for instance, allows an RFC822-compliant message (which may also be a MIME message) to be embedded within a MIME message. This provides improved support for returning or forwarding messages. The message/external-body type saves on transfer costs by specifying that the actual message body is contained elsewhere. Keywords define exactly how the message body can be retrieved (for example, via anonymous ftp or as a local file). Figure 1 gives some examples. The message/partial type allows a single large message to be split and sent as several smaller messages. This is important when dealing with mail systems that limit the size of messages. The message/partial has three keywords:
The multipart content type allows a single message to contain several pieces, each in a different format. The most common multipart message is multipart/mixed, which indicates that the message consists of multiple pieces, each with its own separate Content-Type header. There are also multipart/alternative, in which the parts are alternative formats of the same information (such as a plaintext and a word-processor file with the same content); multipart/parallel, in which the parts are intended to be displayed simultaneously (such as an audio recording and a photograph of the speaker); and multipart/digest, which is the same as multipart/mixed except that the default content type for each part is message/rfc822 rather than text/plain.
All message and multipart types allow (indeed, often require) the embedded data to have its own headers. Technically, the embedded data is not an RFC822 message (for example, it may lack a From header), even though it has the same general format. For example, if a message has type message/external-body, the body contains a series of lines that look like RFC822 headers, including Content-Type, Content-Transfer-Encoding, and Content-ID (required for message/external-body). Like RFC822, a blank line indicates the end of the headers.
Multipart messages must have some way to separate the different parts. The boundary keyword specifies a string that does not occur anywhere else in the message. The actual separators consist of the specified string preceded by "-". The end of the multipart message is marked by the boundary string preceded and followed by "-". Figure 2 shows this mechanism in action. This displays a text message while retrieving and playing audio data from a local file. A minimal MIME-compliant mail reader would show the text part and inform the user of the type and location of the external-file data.
Transparent handling of binary data is one of the primary goals of MIME. It does this by specifying the encoding in the Content-Transfer-Encoding header field. Table 4 lists the five currently defined encodings. The first three indicate that the data is unencoded. The 8bit and binary types are used primarily for parts contained in message/external-body and occasionally with mail systems that do support 8-bit messages.
The Quoted-Printable encoding is intended for data that is primarily 7 bit, with occasional 8-bit values. For example, text messages in ISO character sets are often predominantly 7 bit. Quoted-Printable allows most 7-bit text characters to represent themselves. The remaining characters are encoded as three-character sequences consisting of "=" followed by a hexadecimal number. In particular, "=" is encoded as "=3D".
The advantage of the Quoted-Printable encoding is that it allows any part of the data that is in 7-bit US-ASCII to be read without decoding. However, for raw binary data, it can introduce excessive overhead. The preferred encoding for raw binary data is Base64, which encodes each three bytes of binary data as four characters. The 24-bit value is considered four 6-bit numbers, which are then encoded from the characters A--Z, a--z, 0--9, +, and /. Thus, "the" becomes "dGhl". The result is padded with "=" to a multiple of four characters and broken into 72-character lines. Listing One presents a simple encoder/decoder program for this encoding method. This encoding is similar to the one used by the popular uuencode utility, but avoids using punctuation characters that are lost or altered by certain mail gateways.
In some cases, no encoding is necessary. In particular, the multipart type always uses 7bit, as does message/partial and message/external-body. Under certain circumstances, other message types can use binary or 8bit. The remaining content types can use any available encoding. The point of these restrictions on message and multipart is to avoid nested encodings, which can unnecessarily bloat the message. Remember that a Content-Transfer-Encoding of 7bit for a multipart message means that the individual parts have all been encoded for 7-bit transport.
Many projects have used mail to transfer scripts that are automatically executed on the receiving machine. MIME's application/postscript is one example, and other such content types are being proposed. Any system that allows a received program to be executed automatically is a potential security risk. PostScript includes the ability to modify files, and even without that, it is possible to crash many systems by consuming excessive memory or disk space. Security-conscious systems may need to restrict the handling of these content types. For example, it is usually more secure to send PostScript files to a printer than to interpret and display the data on the host machine.
The current MIME specification, RFC1521, is available from the mail server at RFC-INFO@isi.edu. When requesting the spec, be sure to include the lines
retrieve: RFC
doc-id: RFC1521
in the body of the message. Other RFC documents can be retrieved in a similar fashion.
MIME does not extend RFC822 to allow the use of non-ASCII characters in mail headers, but a related proposal, documented in RFC1522, does. An extended text subtype text/enriched is described in RFC1563. This replaces the text/richtext type proposed in an earlier MIME draft (the name change was to reduce confusion with Microsoft's RTF).
You can obtain the specification--in both text and PostScript form--and the free MetaMail implementation of MIME at ftp://thumper.bellcore.com/pub/nsb.
Table 1: MIME headers.
Copyright © 1995, Dr. Dobb's Journal
Header Description
Content-Type Specifies the type of data contained in the
message. For example, a Content-Type of
audio/basic indicates a
particular audio format that the mail reader
should decode and play.
Content-Transfer- Specifies how the (binary) data is encoded
Encoding into 7-bit text.
MIME-Version Indicates MIME compliance. Was omitted from
early drafts of MIME, so isn't yet used by
all encoders.
Content-ID Uniquely identifies the body of the message.
Content- Provides an additional human-readable
Description description.
Table 2: MIME content types.
Type Description
text Human-readable text, possibly with
textual markup. Any file with type text
should be intelligible if simply listed to the
screen. In particular, binary word-processor
formats are not text.
audio Sound data.
image Still image.
video Movie or animated image.
application Application-specific data file.
Includes script files in certain text languages.
message Wrapper for an embedded message.
multipart Multipart message. Each part may be
in a different format. Subtypes indicate
relationships between different parts.
Table 3: Simple MIME data types.
Type/Subtype Description
text/plain Plaintext with no special formatting.
The key charset is used to
specify US-ASCII or one of the ISO-8859
character sets.
text/enriched An alternate format specified in
RFC1563.
application/ Binary data of an unspecified format.
octet-stream The type key can be
used to give additional, human-readable information.
The padding key can be used to specify
0--7 bits of padding added to round a
bit-oriented file to a whole number of 8-bit
bytes.
application/ A PostScript file.
postscript
image/gif A still image in GIF format.
image/jpeg A still image in JPEG format.
audio/basic A single-channel 8000-Hz audio file in
8-bit ISDN m-law (PCM) format.
video/mpeg A video image in MPEG format. Video
images may or may not contain an associated
soundtrack.
Table 4: MIME encoding types.
Encoding Description
7bit Unencoded 7-bit text.
8bit Unencoded 8-bit text.
binary Unencoded binary data.
Quoted- Most 7-bit characters are unencoded;
Printable other characters are represented as
"=" followed by a hexadecimal number.
Base64 Encoded in Base64 using digits A-Z,
a-z, 0-9, +, and /.
Figure 1: Examples of message/external-body Content-Type headers. As with all RFC822-compliant headers, these are single lines.
Content-Type: message/external-body; access-type=local-file; name="/pub/LargeFile"
Content-Type: message/external-body; access-type=anon-ftp; size=12345678;
site=somehost.com; name=LargeFile; directory=pub/other; mode=image
Figure 2: Sample multipart message.
From: tim@humperdinck (Tim Kientzle)
To: tim@humperdinck
Subject: A Sample Multipart message
MIME-Version: 1.0
Content-Type: multipart/parallel; boundary=SoMeBoUnDaRyStRiNg
Any text preceding the first boundary string is ignored
by MIME-compliant mail readers. This area usually holds
a short message informing a person using a non-compliant
reader that this is a MIME message that they may not be
able to read.
-SoMeBoUnDaRyStRiNg
The preceding blank line ends the headers for this part.
Since there were none, this is assumed to be plain text
in US-ASCII. The boundary cannot occur in the actual
text, so that mailers can quickly scan the text to
locate the boundaries.
-SoMeBoUnDaRyStRiNg
Content-Type: message/external-body; access-type=local-file;
name=/pub/file.audio
Content-Transfer-Encoding: 7bit
Content-Type: audio/basic
Content-Transfer-Encoding: binary
This text is ignored, the actual audio comes from the
file /pub/file.audio. Both blank lines above are
important. Also note the different encodings.
The 7bit encoding means that this embedded message is
in 7bit (which is mandatory for message/external-body),
while the actual audio data is stored in binary in the
local file.
-SoMeBoUnDaRyStRiNg-
This text follows the closing boundary marker above,
and is therefore ignored by compliant mail readers.
Listing One
/****************************************************************************
MIMECODE - encode/decode binary data using MIME's base64 method
Definition: ``radix encoding'' is the process of encoding data
by treating the input data as a number or sequence of numbers
in a particular base. The most common example is base-16 (hexadecimal)
encoding, although other bases are possible.
This program encodes and decodes data using the base 64 encoding
used by MIME. Output is broken into lines every 72 characters.
Decoding ignores control characters. Base 64 encoding adds 33% to
the size of the input file.
Usage: mimecode <options>
Description: reads from stdin and writes encoded/decoded data to stdout.
Options: -e Encode
-d Decode
****************************************************************************/
#include <stdio.h>
/* This digit string is used by MIME's base-64 encoding */
/* MIME also deliberately ignores `=' characters */
#define BASE64DIGITS \
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
static unsigned long bitStorage = 0;
static int numBits = 0;
/* Masks for 0-8 bits */
static int mask[] = { 0, 1, 3, 7, 15, 31, 63, 127, 255 };
/****************************************************************************
ReadBits: reads a fixed number of bits from stdin
If insufficient bits are available, the remaining bits are
returned left-justified in the desired width.
*/
unsigned int ReadBits(int n, int *pBitsRead)
{
static int eof = 0;
unsigned long scratch;
while ((numBits < n) && (!eof)) {
int c = getchar();
if (c == EOF) eof = 1;
else {
bitStorage <<= 8;
bitStorage |= (c & 0xff);
numBits += 8;
}
}
if (numBits < n) {
scratch = bitStorage << (n - numBits);
*pBitsRead = numBits;
numBits = 0;
} else {
scratch = bitStorage >> (numBits - n);
*pBitsRead = n;
numBits -= n;
}
return scratch & mask[n];
}
/****************************************************************************
WriteChar: output character to stdout, breaking lines at 72 characters.
*/
static count = 0;
void WriteChar(char c)
{
putchar(c);
count++;
if (count >= 72) { /* Chop after 72 chars */
putchar('\n');
count = 0;
}
}
/****************************************************************************
PadMimeOutput: pad output for MIME base64 encoding
*/
void PadMimeOutput(void)
{
while ((count % 4) != 0) {
putchar('=');
count++;
}
}
/****************************************************************************
ReadChar: Get next non-control character from stdin, return EOF at
end-of-file
*/
int ReadChar(void)
{
int c;
do {
c = getchar();
if (c==EOF) return c;
} while ( (((c+1) & 0x7f) < 33) ); /* Skip any control character */
return c;
}
/****************************************************************************
WriteBits: Write bits to stdout
Note: assumes `bits' is already properly masked.
*/
void WriteBits(unsigned bits, int n)
{
bitStorage = (bitStorage << n) | bits;
numBits += n;
while (numBits > 7) {
unsigned scratch = bitStorage >> (numBits - 8);
putchar(scratch & 0xff);
numBits -= 8;
}
}
/****************************************************************************
Base64Encode: encode stdin to stdout in base64
The encoding vector used here is the one used by MIME.
*/
void Base64Encode(void)
{
int numBits = 6; /* Encode 6 bits at a time */
int digit;
const char *digits = BASE64DIGITS;
digit = ReadBits(numBits,&numBits);
while (numBits > 0) { /* Encode extra bits at the end */
WriteChar(digits[digit]);
digit = ReadBits(numBits,&numBits);
}
PadMimeOutput(); /* Pad to multiple of four characters */
putchar('\n');
}
/****************************************************************************
Base64Decode: decode stdin to stdout in base64
The `decode' array specifies the value of each digit character.
-2 indicates an illegal value, -1 for a value that should be
ignored. ReadChar() already ignores control characters.
Ignores parity.
*/
void Base64Decode(void)
{
int c, digit;
int decode[256];
{ /* Build decode table */
int i;
const char *digits = BASE64DIGITS;
for (i=0;i<256;i++) decode[i] = -2; /* Illegal digit */
for (i=0;i<64;i++) {
decode[digits[i]] = i;
decode[digits[i]|0x80] = i; /* Ignore parity when decoding */
}
decode['='] = -1; decode['='|0x80] = -1; /* Ignore '=' for MIME */
}
c = ReadChar();
while (c != EOF) {
digit = decode[c & 0x7f];
if (digit < -1) {
fprintf(stderr,"Illegal base 64 digit: %c\n",c);
exit(1);
} else if (digit >= 0)
WriteBits(digit & 0x3f,6);
c = ReadChar();
}
}
/****************************************************************************
Usage: print usage message to stderr
*/
void Usage(char * progname)
{
fprintf(stderr,"Usage: %s <options>\n",progname);
fprintf(stderr,"Options: -e Encode\n");
fprintf(stderr," -d Decode\n");
}
/****************************************************************************
main: parse arguments, call appropriate encode/decode function
*/
int main(int argc, char **argv)
{
int encode = 1;
if (argc < 2) { Usage(argv[0]); exit(1);}
while (argc > 1) {
char *p=argv[--argc];
switch(*p) {
case '-':
{
switch(*++p) {
case 'e': case 'E': encode = 1; break;
case 'd': case 'D': encode = 0; break;
default:
fprintf(stderr,"Unrecognized option: %s\n",p);
Usage(argv[0]);
exit(1);
}
}
break;
default:
fprintf(stderr,"Unrecognized option: %s\n",p);
Usage(argv[0]);
exit(1);
}
}
if (encode) Base64Encode();
else Base64Decode();
exit(0);
}