Internet Programming

HTML Document Templates for CGI Applications

Carlos Moreno

Get the power of C/C++ programming with the flexibility of HTML by keeping the two at arms length.

Introduction

CGI (Common Gateway Interface) is an important tool in the field of Web and Internet Programming. It enables web servers to execute programs in response to requests made by remote users (clients) via web browsers.

CGI is not needed for simple websites, which serve as a mere collection of documents ready to be displayed whenever a client requests one. However, this "collection of documents" approach is often insufficient, and the web server has to perform some processing in response to a client's request. Examples of this are online information systems, online stores, search engines, etc. In an online store or an online information system, clients request information based on certain parameters; the web server has to search through a database — or through other servers providing certain online services — to obtain the information requested.

In this article, I present a set of classes and functions that greatly simplify the implementation of CGI applications in C++. Generally speaking, the main advantage of C/C++ over the other CGI programming options (mainly PERL and Unix shell scripts) is efficiency. However, in my opinion, the main advantage of my approach is the convenience and ease of development of complex CGI-enabled websites.

This article does not require prior knowledge of HTML (Hypertext Markup Language). I present a very brief introduction to HTML basics. However, I strongly advise that you use any book or online reference to learn more about it.

Understanding HTML

Simply put, HTML is the standard formatting language for web pages. It provides a set of tags that define and format the various sections of a web page. It is based on standard ASCII characters.

Thus, a web page is simply a text file containing HTML code, which is stored on a web server. When a remote client requests a specific web page, the web server locates the file containing the requested document and transmits it back to the client's browser. The browser receives the text stream, decodes it, and generates the graphical display encoded in the HTML file.

HTML is based on the use of tags. Tags are represented by keywords enclosed by angle brackets. HTML tags are case insensitive. Some tags, such as <br>, are stand-alone, but most tags must be provided in pairs: the opening tag and the closing tag. The closing tag uses the same keyword as the opening tag, preceded by a forward slash. Examples of these are text formatting tags, such as the <b> and </b> tags; the text between these tags is displayed in bold characters. Also, some tags indicate sections of the HTML document, such as the <title> and </title> tags. These last two tags enclose the document's title, which is the text displayed in the browser's title bar.

The anchor tag (<a> and closing tag </a>) specifies two kinds of elements: links and targets. In the case of links, the tags must include the href attribute, as shown in the example below:

The code of this article is available
on the
<a href="http://www.cuj.com">C/C++
 Users Journal</a>'s web site.

This fragment of HTML code defines a link to the www.cuj.com website. The section of the document enclosed by the <a> and </a> tags is the link. In general, it will be displayed by most browsers as underlined text in a specific color that changes when the link has been visited. The href attribute specifies the URL (Uniform Resource Locator or, informally, the web address) of the link.

Anchor tags can also mark a section or a location in the document. These locations are known as targets. In the example below, anchor tags are used to implement a table of contents at the beginning of the web page. The first three anchors serve as links; the last three serve as targets:

...
Table of Contents<br>
<br>
<a href="#introduction">Introduction</a>
<br>
<a href="#html_intro">Understanding
HTML</a> <br>
<a href="#cgi_intro">Understanding
CGI</a> <br>
<br>
<br>
<a name="introduction"></a>
<b>Introduction</b>
<br>
<br>
CGI is an important tool ....
...
<a name="html_intro"></a>
<b>Understanding HTML</b>
<br>
<br>
...
<a name="cgi_intro"></a>
<b>Understanding CGI</b><br>

Notice that in the targets, the <a> and </a> tags don't enclose any text or use an href attribute; they just mark a specific document location and identify that location with a name. As you will see, I take advantage of anchor tags in my implementation of the CGI toolkit.

Understanding CGI

The web server executes CGI applications in response to remote-user requests. The way information flows in a CGI-enabled web transaction is similar to that of a normal HTML document request. In the case of a CGI application, the difference is that the HTML file is not stored on the web server; instead, the CGI application is executed, and it dynamically generates the stream of HTML text to be sent back to the client's web browser. Remote users send their requests by pointing their web browsers to the executable CGI file instead of pointing to an existing HTML file.

Thus, if the remote user selects http://www.example.com/cgi-bin/example.cgi, the web server will execute the file example.cgi that is stored in the directory /cgi-bin, and the CGI application will generate the HTML code that will be sent back to the client's web browser. The CGI application responds to the client's request by sending the HTML text stream to stdout (e.g., using cout).

A simple example of a CGI application is the following program, which responds to clients' requests with a fixed message:

#include <iostream>
using std::cout;

int main()
{
    // header; CGI applications must
    // send this line first
    cout << "Content-type:"
         << "text/html\n\n\n";
         << "This is the text"
         << "displayed by this"
         << "CGI application.\n";
    cout << "<center><b>CGI Demo</b>"
         << "</center><br><br>"

    return 0;
}

This program must be compiled, and the executable should be placed on a CGI-enabled directory of the web browser (typically /cgi-bin). When remote users request http://www.example.com/cgi-bin/example.cgi, the web server executes example.cgi, reads the output that this program sends to stdout, and sends it back to the remote user's browser.

Most CGI applications are much more complex than this simple example. They typically read or update information stored in databases on the web server or other machines accessible from the web server. Based on this information, the CGI application builds the HTML-formatted stream of output text.

Additionally, the CGI application may require parameters or other information from the remote user. A typical example would be a search engine — the user has to indicate the search keywords and criteria, and the CGI application uses that information to execute the search. This is typically done using HTML forms. Forms can be included in an HTML document using the <form> tag. An example of a form to request user ID and password is shown below:

<form action="/cgi-bin/login.cgi" method="post">
User Name: <input type="text" name="user_name"> <br>
Password:  <input type="password" name="password"> <br>
<input type="submit" value="Ok">
</form>

In this form, the action attribute indicates which CGI application will be invoked when the user clicks on the submit (OK) button. In this case, the browser will point directly to login.cgi (on the same site, since no absolute URL is indicated) and will pass the query according to the information that the user entered. If the user enters "John Smith" as name and "C++" as password, then the query sent to the CGI application will be user_name=John+Smith&password=C%2B%2B.

Notice the URL encoding. & separates the different fields of the query. Spaces are replaced by +, and non-alphanumeric characters are replaced by an "escape sequence" consisting of % followed by the ASCII code of the character in hexadecimal format.

Also note the method attribute in the <form> tag. The CGI application reads the query from stdin if the method used is POST, or from the environment variable QUERY_STRING if the method used is GET. Other pieces of information are available from other environment variables. (See one of the references for more details.)

Writing CGI Applications in C++

One of the main inconveniences in writing complex CGI applications (in any language) is maintainability. The C++ programs (or PERL or shell scripts) must include sections of code that build the HTML document. In other words, the CGI program code must include HTML code. This represents a major drawback concerning website maintainability. Changing the graphical look of your website would involve painful modifications to the C++ code. (Keep in mind that web pages are often designed using graphical, point-and-click tools, where even minor changes may lead to tens or hundreds of lines of HTML code that change.)

Also, including HTML code in your C++ programs implies that either the C++ programmer must be able to write HTML code, or the person in charge of the web page design should have access to the C++ code.

One of the main advantages of the toolkit that I present is that it allows you to completely isolate the web page design from the process of writing the CGI applications.

The key idea is to take advantage of the fact that virtually every CGI application responds with a pre-defined HTML with fields that need to be dynamically generated. This allows you to design your web pages with "fill in the blank" fields that the CGI application will complete (typically with information coming from databases.)

These "fill in the blank" fields are provided via anchor tags. To specify such a field in an HTML file, the web designer includes an empty anchor tag with the name attribute, as shown below:

<a name="user_name"></a>

The CGI application reads the file and replaces the anchor tags with the appropriate data. If the application (accidentally or not) fails to replace the anchor tag, it will be disregarded by the web browser when displaying the document.

A Class for Dynamically Generating HTML

The ideas discussed in the previous section are conveniently encapsulated in the HTMLstream class (see Listings 1 and 2). The constructor for this class takes a string argument indicating the filename of the HTML template document. When an HTMLstream object is constructed, the contents of the file are read into the data member html_code. Alternatively, you can invoke the constructor with no arguments and later call the open method to read the HTML code from the specified file. Also, you can use the stream insertion operator to append arbitrary data to the HTML stream.

HTMLstream provides a send method, which sends the HTML stream to the output stream parameter, prepended with the header (Content-type: text/html). For increased reusability, the method receives an ostream object as a parameter, with a default to cout:

HTMLstream reply ("user_not_registered.html");
reply << "<br><center>An extra line added at the end</center>";

reply.send ();  // equivalent to reply.send (cout);

HTMLstream provides a set_field method that replaces the anchor tags with a given string (the "fill in the blank" fields in the HTML code). This method receives two parameters, indicating the field to be set and the contents to store. An example is shown below:

HTMLstream reply ("invalid_password.html");
// name is a string variable that contains the
// user name (presumably from the CGI query)
reply.set_field ("user_name", name);

reply.send ();

In this case, the HTML template document invalid_password.html should contain a fragment similar to the following:

Sorry <a name="user_name"></a>, the password you entered is 
incorrect.

The set_field method in the example above will replace the anchor tag with the content of the variable name (the second parameter in the call to set_field).

Listing 2 shows the implementation details of these methods.

HTMLstream also provides the add_option method, to dynamically fill the options in a pull-down menu, and the remove_table_row method, to dynamically size a table. Instead of adding table rows as required, it is easier and more convenient to include the maximum number of rows in the HTML template document and then remove the extra ones from the HTMLstream object. (An example of this is provided in sample code with this month's online archives. See www.cuj.com/code/.)

Utility Functions for CGI Programming

The string and container classes available in C++ and the C++ Standard Library are of great help to CGI applications. I took advantage of these tools to provide a set of utility functions that greatly simplify the processing of the queries. These functions are shown in Listings 3 and 4.

Function cgi_parameters returns a map<string, string> containing all the parameters of the user's query. In the example above, where the query string was user_name=John+Smith&password=C%2B%2B, you could directly use the following:

map<string, string> parameters = cgi_parameters ();

string name = parameters ["user_name"];      // stores "John
                                             // Smith" in name
string password = parameters ["password"];   // stores "C++" in
                                             // password

cgi_parameters receives an argument indicating the method used. This argument can be get, post, or any. It defaults to post. cgi_parameters calls function get_cgi_string to read the query string, and then uses function tokens to split the string into individual parameters. For each parameter, tokens is called again to break them into name, value pairs. Finally, decode_cgi_string is used to decode the URL-encoded strings (that is, it replaces + with spaces and the escape sequences with the corresponding characters). After decoding, this pair is inserted into the map<string, string> returned by cgi_parameters.

Listing 4 shows the details of all these functions and a few other utility functions as well. Typically, you will only have to use cgi_parameters, which decodes the parameters and breaks them into ready-to-use pieces.

A Concrete Example

The following example shows part of a login process authorized by a CGI application. To make the example simple, I will not use databases. Instead, I use text files containing the following information: user name, password, and a list of options authorized for the user. The fields are separated by tabs, and the records are separated by newline characters (i.e., one user per line). The options will be used to fill a pull-down menu of selections in the welcome page, once the user is authorized.

The login page contains the following form:

<form action="/cgi-bin/login.cgi" method="post">
User Name: <input type="text" name="user_name"> <br>
Password:  <input type="password" name="password"> <br>
<input type="submit" value="Ok">
</form>

Users enter their information and click on the OK button. The application responds with one of two possibilities: a welcome page, with a pull-down list to select the next action, or an error page indicating that either the user name is not registered or the password is incorrect.

The HTML templates are stored in welcome.html and login_failed.html. welcome.html contains, among other things, the following:

Hello <a name="user_name"></a>, welcome
back.  Please select
one of the following options: <br><br>
<select name="options">
</select>

Here is a portion of login_failed.html:

Login failed. Either the user name that
you supplied (<a name="user_name"></a>)
is not registered in our database,
or the password is incorrect.

login.cpp (which should be compiled to the executable login.cgi) is shown below. To keep the example as simple as possible, I omit the exception-handling code, and I assume that no errors will occur (e.g., files will be successfully opened and read, all the fields in the HTML templates will be correctly set up, etc.):

// #include files

int main()
{
   map<string, string> parameters = 
      cgi_parameters ();

   string line;
   bool login_accepted = false;

   ifstream users ("users.txt");
   while (!login_accepted && getline (users, line))
   {
      vector<string> user = tokens (line, "\t");

      if (user[0] == parameters["user_name"] &&
         user[1] == parameters["password"])
      {
         login_accepted = true;
      }
   }

   if (login_accepted)
   {
      HTMLstream reply ("welcome.html");
      reply.set_field ("user_name",
         parameters["user_name"]);

      for (int i = 2; i < user.size(); i++)
      {
         reply.add_option ("options", user[i]);
            // fill the pull-down menu according to the user
      }

      reply.send ();
   }
   else
   {
      HTMLstream reply ("login_failed.html");
      reply.set_field ("user_name", parameters["user_name"]);
      reply.send ();
   }

   return 0;
}

Security Issues

Before trying out your first CGI application, make sure that you read the section on security in a reference book. Keep in mind that you are executing a program on your machine. Malicious hackers may take advantage of carelessly written CGI applications to steal information from your machine, gain privileged access, or perform destructive actions.

One of the well-known tricks is to cause a CGI's input buffer to overflow when reading the query string. This does not happen if the application is carefully written and definitely can not happen using this toolkit, since I use getline to read into a string object, which will be dynamically resized to accommodate a string of virtually any size.

There are many other security issues that I do not discuss in this article. Please remember this advice (even though I don't think I can repeat it enough times): Do read the section on security in a good reference book before trying out your first CGI application!

Summary

CGI applications are at the heart of dynamic websites. They provide a convenient and portable way of enabling web servers to perform certain processing in response to remote requests.

The C++ language is an excellent choice for developing CGI applications. Though C++ does not provide built-in tools oriented to CGI applications, this article shows that with C++ it is easy to write highly reusable and powerful tools, making CGI programming easier than in any other programming language.

However, extreme caution should be taken when writing CGI applications, regardless of the choice of language or tools. Carelessly written CGI applications can easily compromise the security and stability of web servers.

References

Bjarne Stroustrup. The C++ Programming Language, 3rd Edition (Addison, Wesley, 1997).

Eric Ladd and Jim O'Donnell, et al. Using HTML 4, XML, and Java 1.2 (MacMillan Publishing Company, 1999).

Mark Felton. CGI Internet Programming with C++ and C (Prentice Hall, 1997).

Carlos Moreno has a Bachelor's Degree in Electronics Engineering and a Master Engineering diploma in Computer and Electrical Engineering. He has 12 years of experience in the development of low-level applications and currently works as an instructor/trainer in C, C++, Object-Oriented Programming, and Visual Basic, and as an independent consultant and software developer. His main interests are digital signal processing, audio and image processing applications, communications, data encryption and compression, and (of course) computer games development. He can be reached at moreno@mochima.com or on the Web at http://www.mochima.com.