Padamadan’s classes make managing state in a client-server environment a sweet experience.
Introduction
When a web browser requests information from a web server, the server may send a piece of state information along with its response. This state object is called a cookie. Web servers use cookies to store information on the client’s system; the next time the client contacts the server, it will send the cookies back to the server, enabling the server to retrieve the information stored. Users can configure the browser settings to ignore or accept the cookies. In order to maintain the state information between sessions, the client should send the cookies back to the server.
A web browser extracts the cookie information from the server responses. Internet Explorer uses the WinInet SDK [1] calls to retrieve the cookie header information. A custom browser can use either the WinInet SDK API or TCP/IP socket functionality to extract the cookie headers from server responses.
There are a several aspects to cookies that make them potentially difficult to manage: there are several different kinds of cookies, they may contain varying amounts of data, and they may or may not have an expiration date. In this article, I’ll present two classes: a CookieManager class, which manages cookies for a specific domain or web server, and the Cookie class, which encapsulates most of the features of the original Netscape specification as well as RFC 2109-based cookies. I will also compare the original Netscape specification (the Version 0 specification) and the new RFC 2109 (Version 1 or later) and discuss how these versions are managed in those classes.
I tested the CookieManager and Cookie classes against cookies of both specifications using a Windows Console application. The sample application with the source code is available for download from the CUJ website. I compiled the source code using Visual C++ 6.0. The typical users of the classes presented here would be custom browser applications, which directly talk to the web servers for specific information.
Kinds of Cookies
Persistent Cookies vs. Per-Session Cookies
Cookies with an expiry time associated with them, set by the web server, are known as persistent cookies. The browser client discards the cookies once the cookies expire. The state information set by the web server resides on the browser until the cookies expire. Web servers may send cookies without an expiry time and may later set an expiry time for those cookies through new responses.
If there is no expiry time associated with a cookie, it’s a session cookie. Browser clients use session cookies until the Internet session between the browser and the server ends. Browser clients keep these cookies in cache and discard them as soon as the user closes the browser.
Client-Side vs. Server-Side Cookies
Cookies generated on web servers are called server-side cookies. Server-side cookies are generally used to store the state information of the server on the client side, whereas client-side cookies, generated by the scripts on the client side, store the user-interface-related information for the client. The scripts on the browser client may let the user customize look-and-feel of the HTML pages as per the user’s choice and keep them in the cache. Next time the user visits the same site, the user will get the same look-and-feel of the pages that the user opted for before. Client-side cookies do not generally affect the business logic of the server-side applications. Client-side and server-side cookies can be of the session or the persistent nature.
Specifications for Cookies
Web servers and the browser clients should follow certain specifications for the cookies to work well. Netscape Communications introduced the first specification for cookies [2]. Most of the current browsers and web servers support the cookies as specified by the original Netscape specification. The W3 Consortium has proposed a new cookie specification [3], called RFC 2109, for the HTTP/1.1 protocol. These cookies are considered to be Version 1 or later, and the original Netscape specification cookies are Version 0 cookies. Cookies based on the new proposal will have the version information in them so that they won’t interfere with the old specification cookies. Browser clients should be able to interpret the cookies of both versions and act accordingly.
How Cookies Are Used
Web servers send cookies to browser clients by including a Set-Cookie header as part of their HTTP response to an HTML request from the browser. Below is an example of a response header, which contains a Set-Cookie header (lines broken to fit column).
Original Netscape (Version 0) specification:
HTTP /1.1 200 OK Server: XYZ Webserver 1.0 Date: Fri, 20 Oct 2000 18:45:52 GMT Content-type: text/html Set-Cookie: PRDUCTID=7387AD1212; domain=.acme.com; path=/; expires=Thu, 19-Oct-2000 00:00:00 GMT Set-Cookie: POSITION1232=5445ADS Set-Cookie: TRACK=<121232>; domain=.acme.com Set-Cookie: Redirect=none; path=/; secure Location=/start/MenuRFC 2109 (Version 1 or later) specification:
HTTP /1.1 200 OK Server: XYZ Webserver 1.0 Date: Fri, 20 Oct 2000 18:45:52 GMT Content-type: text/html Set-Cookie: PRDUCTID="7387AD1212"; domain=".acme.com"; path="/"; Max-Age="3000"; Version="1" Set-Cookie: POSITION1232="5445ADS"; Version="1" Set-Cookie: TRACK="<121232>"; Version="1"; domain=".acme.com" Set-Cookie: Redirect="none"; path="/"; Version="1"; secure Location=/start/MenuWhen the client requests a URL from an HTTP server, the server includes the cookies it received from the web browser in previous requests. (The client sends them with a "Cookie" header as described below.) The web server may send the same cookies again to the browser clients with different attribute values. It’s the responsibility of the browser clients to update the values of the cookies in the cache. The following is a typical HTTP request sent by a browser (lines broken to fit column).
Version 0 specification:
GET /start/moredata.html HTTP/1.1 Cookie: POSITION1232=5445ADS; TRACK=<121232>; Redirect=noneRFC 2109 specification:
GET /start/moredata.html HTTP/1.1 Cookie: $Version="1"; POSITION1232="5445ADS"; TRACK="<121232>"; $Domain=".acme.com"; TRACK="<121232>"; Redirect="none"; $Path="/"Parts of a Cookie
The Set-Cookie HTTP Response Header as per the Netscape specification contains one required cookie-name/value pair followed by a set of optional attribute-name/value pairs and a flag:
Set-Cookie: <cookie name>=<value> [; <attribute name>=<value>] ... [; expires=<date in GMT format>] [; domain=<name>] [; path=<path_value>] [; secure]Here the <cookie name>=<value> pair specifies the cookie name and the value assigned to it. The other name/value pairs are attributes of the cookie just named. The braces ([ and ]) in the snippet above indicate optional attributes. (The secure flag is really an attribute as well, but its <value> part is optional and is usually omitted.) The header may contain one or more <cookie name>=<value> pairs. Since the Netscape specification lists just four possible attribute names (domain, path, secure, and expires), a name/value pair that doesn’t start with one of these four represents another cookie.
The expires flag, if present, specifies when the cookie will expire in GMT (Greenwich Mean Time) format so that the browsers can discard it once its expiry time passes. path specifies the subset of URLs in a domain for which the cookie is valid. The domain attribute specifies the Internet domain name of the host from which the URL is fetched. If the secure flag is present, it means that the client should send this cookie back only over HTTPS (Hypertext Transfer Protocol over Secure Sockets Layer).
RFC 2109 specifies the following
Set-Cookie syntax:
Set-Cookie: <cookie name>=<value> [; Comment="<value>" ] [; Domain="<value>" ] [; Max-Age="<value in seconds>"] [; Path="<path_value>"] [; secure] [; Version="<a single digit>"]Here <cookie name>=<value> and Version="<a single digit>" are the only required name/value pairs. Max-Age defines the lifetime of the cookie in seconds. All other attributes have the same meaning as in the Version 0 specification. As in the Version 0 specification, an RFC 2109 Set-Cookie header can contain more than one cookie. A cookie name is easily recognized as not being one of the attribute names Domain, Path, Secure, Max_age, Comment, or Version.
Once the Max-Age seconds elapse, the client should discard the cookie. I’ll discuss the attributes in detail while explaining the Cookie class.
Using a Cookie Class
A cookie contains a group of name-value pairs. The Cookie class uses an STL map object to store all the name-value pairs. The map takes care of not allowing more than one name-value pair with the same name attribute for the same cookie.
The Cookie class (Listing 1) accepts the cookies of both the specifications mentioned above. The Cookie class uses its private member function constructNameValueMap to separate the name-value pairs from the cookie string and map each attribute name to its corresponding value by inserting them into the map object. Cookie::getCookieString converts the map object back to a cookie string to be sent back to the server.
If the Version attribute is present in the cookie string and its value is greater than or equal to one, it is an RFC 2109-based cookie. Any action on the cookie name-value pair is taken based on the version information. As per the RFC 2109 specification, the Version attribute can appear anywhere in the cookie string after the cookie-name/value pair. There is some potential for ambiguity here. Note that Version is not one of the attributes specified for Version 0 cookies. So it could appear as a valid cookie-name as the first name/value pair of a Version 0 cookie. constructNameValueMap searches for Version="1" right after the first cookie name-value pair. If the web server sends a header of the form Set-Cookie: ID=2; Version=1; Path=.foo.com, the cookie class interprets it as an RFC 2109-based cookie named “ID”. (However, it would be possible to interpret this header as defining two Version 0 cookies: one named “ID” and one named “Version”.) It is important to realize this, because even though the new cookie specification is meant to work with the HTTP/1.1 protocol, most of the websites using HTTP/1.1 have not started supporting the RFC 2109 specification.
You can query the Cookie object to get the state of the cookie. It’s an invalid cookie if there are no name-value pairs, or if the domain name it contains is not the domain that the URL was requested from. As per the specifications, the domain string of the (as of this writing) seven top-level domains (.com, .edu, .net, .org, .gov, .mil, and .int) should have at least two periods in it. Thus domain=foo.com is invalid whereas domain=.foo.com is a valid attribute pair. Any other domain requires at least three dots (i.e., domain=.abc.va.us). RFC 2109 specifically states that all domain values should start with a dot. The old specification does not insist on this. isDomainValid checks if the value part of the domain attribute is per the specifications. The doesDomainTailMatch(const string& strReqDomain) method verifies if the domain the CookieManager object was instantiated with matches the domain name in the cookie.
If the cookie does not have a domain name attribute, it’s treated as a valid cookie for the default domain. As per the Version 0 specification, a domain attribute of acme.com would match host names anvil.acme.com as well as shipping.crate.acme.com. RFC 2109 is more stringent. If the fully qualified domain name of the server ends with the value of the domain attribute, then the beginning part of the fully qualified domain name must not contain any dots. For example, a cookie containing the attribute Domain=.foo.com sent by a server at y.x.foo.com would be rejected, because the beginning part of the fully qualified domain name is y.x, which contains a dot. doesDomainTailMatch implements these rules.
After a cookie passes the domain validation, a client-side application should perform another check. It should ensure that the path the cookie was received from matches the path to which it will be sent. If the path is not specified, the application should assume it is the same path as the document’s path, whose header contains the cookie. The browser application should not forward a cookie if its path value is not a prefix of the request URI (Uniform Resource Identifier). For example, a URI containing /foo/base would match the path value /foo, because /foo is a prefix of /foo/base. Thus all cookies having the path value /foo would be forwarded to /foo/base as well by the application.
Further, if a client packages more than one cookie in a request header, the name-value pair for the cookie having the more specific path attribute /foo/base should come before /foo when a request is sent to the server. Both specifications allow the same cookie names to appear more than once in the cookie header to be sent to the server. This is because a cookie name, path name, and the domain name form a unique string. doesPathMatch(const string& path) matches the cookies as per the path rules.
Let’s say the client received the following cookies during a session:
Set-Cookie: PRODUCT=CAR; Path=/foo/start Set-Cookie: PRODUCTID=2343; Path=/ Set-Cookie: SALESID=3222; Path=/home Set-Cookie:CUSTID=3453; path=/foo/If the client then requests the file /foo/Start/update.html, the client’s request header will contain the following cookie:
Cookie: PRODUCT=CAR; CUSTID=3453; PRODUCTID=2343The Cookie object can tell you if the cookie is a session-based or persistent cookie. It basically checks for the existence of the Expires or the Max-Age attribute based on the version. The Expires attribute represents the time when the cookie expires in the GMT format. But RFC 2109 cookies do not have an Expires attribute. Instead, RFC 2109 recommends a Max-Age attribute, which specifies the expiry time of the cookie in seconds. The hasExpired method of Cookie class returns true if the cookie is expired. It validates the value of Expires or the Max-Age attribute based on the cookie version.
The Cookie class uses the WinInet API time functionality to manipulate the expiry and Max-Age values. This is the only place where Windows-specific calls are used. All other methods are generic and portable to any operating system. A client-side application should discard all the session cookies when an Internet session with a particular domain ends. The Cookie class supports isSessionCookie and isPersistentCookie methods for this purpose.
The Cookie::isSecure method verifies whether the cookie contains the flag secure or not. Applications should send secure cookies only through HTTPS (HTTP over SSL - Secure Socket Layer) protocol. The CookieManager class (described below) uses this method of Cookie class to decide whether to send a cookie to the web server or not, based on the protocol it’s using.
The Cookie class supports two forms of comparison between Cookie objects. The implementation of operator== checks that each object has the same domain and path attributes. It also checks that all name-value pairs that aren’t predefined (such as Expires) match. The isEqual method compares the objects for the domain and path values and the names of the cookies. If both the objects are of the same domain and path, and if the cookie names are equal, they represent the same cookie. The CookieManager class uses isEqual to update the cookies in the cache with the new values received from the server.
Cookies in a Client Request
Browser clients send cookies to web servers along with their URL requests with the syntax shown below.
Cookie syntax as per Version 0 specification:
Cookie: NAME1=value; NAME2=value...Here the browser returns only cookie name and values to the server. I’ve noticed that even if you add the path and the domain information along with each cookie, most of the web servers accept the cookies. However, I encountered some issues when I added the expiry and the secure flags to the cookies sent to the server.
Cookie syntax as per RFC 2109:
Cookie: $Version="<1 or greater>"; NAME1="value"; NAME2="value"; $Path="value"; $Domain="value"The only required item as per the old (Version 0) specification is the name-value pair. But RFC 2109 requires the Version attribute as well along with the cookie name-value pair. Even though comma separators are also allowed instead of semicolon separators per RFC 2109, it’s recommended using a semicolon for backward compatibility. Max-Age, Secure, and Comment attributes are ignored here. Even if you send more than one cookie in the URL request, there should be only one Version attribute, and it should appear as the first name-value pair in the cookie header. Further, cookie names should not use “$” as the first character. It is reserved as the first character of attribute names.
The browser clients should send only non-expired cookies that originated from the request host, relevant to the request URL. These rules for the cookie: header as per the old and the new specifications are taken care of in the Cookie and CookieManager classes.
Managing Cookies with CookieManager
The CookieManager class (Listing 2) uses the Cookie class to decompose cookies sent by web servers. The browser application should instantiate a CookieManager object with the domain name the application is going to send requests to. The application instantiates a CookieManager object per domain. That way, the application would not a send a cookie that originated from an a.foo.com web server to a b.foo.com web server or vice versa.
The CookieManager class maintains a list of Cookie class objects in the form of an STL list. A cookie string can have more than one cookie name-value in it. A web server can send a cookie of the form "Set-Cookie: Product=1; Volume=50; Path=/start;" in response to a URL request and later can tell the browser client to discard one of these cookies by setting an expiry time with the following cookie: "set-cookie: Volume=50; expires=Friday, 27-Oct-2000 00:00:00 GMT;". Here the "Volume=50" cookie has an expiry time associated with it, but the "Product=1" cookie does not have an expiry time attached.
For handling these kinds of cookies, the CookieManager class instantiates a Cookie object with the cookie string, separates the cookie name-values into individual cookies using updateCookieList(const Cookie&), and updates the cookie list. The Cookie class has a method getCookieNames(NAME_MAP&) to retrieve only the cookie name-values without those reserved attribute name-value pairs. If a cookie is expired, the CookieManager removes it from the list. If the cookie is not relevant to the current domain, the CookieManager rejects it. To add cookies to the CookieManager’s cookie list, a client application can use addCookie(const string&).
A client application can use the getCookieString(...) of CookieManager to convert the entire cookie list into a cookie string ready to be sent to a specified server and specified path.
Browser clients can use the clearSession method of CookieManger to clear the session cookies pertinent to a session. This method also removes the expired and the invalid cookies from the cookie list. The CookieManager class does not serialize the cookies in the list to a disk drive. Implementation can be easily changed to save and restore the persistent cookies from a disk drive.
Listing 3 shows how to use the CookieManager class in an application. It uses the test cases given in the specifications. The listing shows the output of the test cases as well.
Conclusion
The cookie specifications specify some limits for the number of cookies that can be kept on the client system. These limits can be enforced at the application level. CookieManager and Cookie implement most of the rules of the current specifications. However, as the RFC 2109 is still in the development stage, the syntax and attributes may change at any time. Please refer to the URLs mentioned below for the latest updates.
References
[1] WinInet SDK on Microsoft Developers Network (MSDN), <http://msdn.microsoft.com>.
[2] Netscape Specification, <http://www.netscape.com/newsref/std/cookie_spec.html>.
[3] RFC 2109, <http://www.w3.org/Protocols/rfc2109/rfc2109> or <http://www.cis.ohio-state.edu/htbin/rfc/rfc2109.html>.
Babu George Padamadan is a technical manager with IT Solutions Pvt. Ltd., Bangalore, India. He specializes in C++/COM+. He has a B Tech in Computer Science. He can be reached at babu.george@itsindia.com or babu_george@hotmail.com.