Dr. Dobb's Journal February, 2005
A web site can see, potentially, millions of Internet users in a day. This makes performance and scalability a big issue in web-site development. Web-site performance is measured in response time, or how long it takes for a user's browser to receive HTML after a request. Scalability is the capacity to maintain good performance even when large numbers of users simultaneously—and continuously—make requests. To achieve scalability, the hardware and software that make up a web site are typically segmented into tiers like those in Figure 1.
The web tier is where user requests are first handled and queued. The application tier is where the business logic—and code—behind the web site resides. The data tier, sometimes called the "enterprise information system" (EIS) tier, contains databases or legacy systems as data sources. Depending on the web site, some tiers can be combined or segmented even more. Sun, for instance, defines further subtiers with its model-view-controller architecture (MVC).
Web services are often used to expose application functionality to paying customers, or to integrate legacy systems with newer applications. However, web services can become part of an application's design to help achieve component isolation, gain greater scalability, and ease development. In this article, I propose adding a new tier to the traditional multitiered architecture—the service tier.
An architecture without a service tier has a huge flaw—the lack of data hiding or abstraction. Without a service tier, application code must have intimate knowledge of the data sources and associated details, such as database schemas or low-level software APIs. Adding a service tier to your architecture—one that contains web services that abstract the data sources from the core business logic code—solves this problem.
A service tier should be designed to achieve the following goals:
In general, a service tier that achieves these goals can provide other benefits to application developers. First, there is greater potential for software reuse as the service tier is accessed through a generic interface. Second, a more efficient development process is achieved as the application tier and the service tier can be developed independently. For example, developers strong in JSP or ASP can concentrate on building applications, while database developers (or developers familiar with an external API) can build portions of the service tier. This helps to avoid maintenance problems that come from mixing database queries in application-tier code. Third, system interdependencies (and associated complexity) are greatly reduced. A well-designed service tier lets backend data systems change and evolve independently and transparently from the software components that use them. Finally, true implementation independence is achieved, as the service tier components can be written in a language or environment different than that of the application.
The Simple Object Access Protocol (SOAP) is perhaps the best choice when implementing components of a service tier. SOAP not only achieves the architectural goals and benefits, but yields other practical benefits as well. For example, a SOAP web service can be deployed across multiple servers, and load balanced using existing HTTP load-balancing technology. The SOAP protocol itself defines a strict contract for communication between layers. In addition, SOAP—being based on XML—provides the right level of abstraction, allowing for implementation independence.
In Figure 2, a web application architecture enhanced with a service tier, the web tier and application tier have been combined for simplicity. The elegance of the design comes from the degree of separation and abstraction achieved across the tiers. The application code becomes simpler as it no longer deals with the details of individual data sources. SOAP adds consistency to the way data is requested from and returned to the application code.
Java has long been the leading choice for web application development. Sun positioned Java as a powerful platform with which to build web services when it introduced the Java Web Service Developer Pack (WSDP) about two years ago. Now at Version 1.4, the Java WSDP provides all of the APIs, tools, and infrastructure you need to build web services that support the important standards such as:
For more information on the Java WSDP (along with software downloads), see http://java.sun.com/webservices/index.jsp. There are download bundles available for Sun's J2EE 1.4-compatible application server, Sun's Java web server, and Apache's Tomcat web server/Servlet container. I suggest starting with the Tomcat container, which is an open-source project from Apache-Jakarta (http://jakarta.apache.org/tomcat/). It's free, works well, and is available on many platforms.
The Java WSDP integrates XML parsing using JAXP; WSDL and low-level SOAP messaging using JAX-RPC; XML to Java binding using JAXB; UDDI using JAXR; and high-level SOAP messaging with attachments using SAAJ. For a detailed overview, see the Java Web Service's Tutorial (http://java.sun.com/webservices/docs/1.4/tutorial/doc/index.html).
The sample web application (available electronically; see "Resource Center," page 5) I present here is a basic financial portal. The portal displays stock quotes, company fundamental data, and company financial data (balance sheet, income statement, and cash flow statement) when a company's ticker symbol is entered. I've chosen to use the Java WSDP bundle that comes with the Tomcat 5.0 web container. The code uses the SOAP with Attachments API for Java (SAAJ) to build two SOAP servers and one client, as well as JAXP for parsing the XML payloads of the SOAP messages. Figure 3 illustrates the basic architecture for the application.
The application contains two SOAP web services—a Quotes Service and a Fundamentals Service—which are written using SAAJ. To use SAAJ, your Java code must extend the class, com.sun.xml.messaging.soap.server.SAAJServlet, contained within the file saaj-coms.jar. Doing this, your class basically becomes a Java Servlet capable of receiving SOAP requests by overriding the method, onMessage. This method provides the SOAP request as a parameter, and lets you return a SOAP response.
The Quotes and Fundamentals services both wait for a SOAP request that looks like the XML in Example 1. The contents of the SOAP body are in bold, and contain the company's stock symbol, name, and the central index key (CIK) as assigned by the Securities Exchange Commission (SEC).
When the Quotes Service receives a SOAP request, it is parsed and checked to ensure it contains all of the data needed. Listing One shows the code that receives and parses a simple SOAP request message. After the request is parsed, the Quotes Service makes an HTTP request to Yahoo using the company's stock ticker. The result from Yahoo is comma-delimited quote data, which is parsed and formatted into an XML SOAP body to be sent back to the SOAP client. The code for generating the SOAP response is shown in Listing Two. It is important to note that using Yahoo for quote data is for illustrative purposes only, and should not be distributed in any way.
The Fundamentals Service works in a similar fashion. SOAP requests are received and parsed to gather the needed data, which includes the company's ticker symbol and CIK. Next, the service makes an FTP request to the SEC's FTP servers to obtain the latest financial filings for the given company. The FTP response contains basic company information—such as address and business classification—as well as the financial filings in HTML form. Next, the service creates a SOAP response that contains the company's fundamental data in the SOAP body, with the HTML filings added as a SOAP attachment. The code for this can be seen in Listing Three.
SOAP requests are made to both web services (Listing Four) from a Java Servlet that contains the application code called the "Financial Servlet." This Servlet generates an HTML form for users to enter company ticker symbols, as in Figure 4. When a symbol is submitted, a SOAP request is made to the Quotes service as well as the Fundamentals service. The SOAP responses are received and parsed, and an HTML web page (Figure 5) is generated containing all of the quote, fundamental, and financial data from the SOAP messages and the attachment.
This application illustrates how segmenting retrieval into a service tier can be a major design improvement. For example, the web services in this application can be distributed across multiple servers, where the work of gathering quote and fundamental data can occur in parallel instead of serially. Future modifications, such as a change in the source for quote data, can be made without changing any deployed applications. Only the Quotes service would need to change. The SOAP interface completely hides the implementation details.
The sample financial portal application assumes that the application will be deployed and run within the Tomcat 5.0 web container, although it should run within any J2EE web container without major changes.
The Java WSDP comes with the Apache Ant build tool (http://ant.apache.org/). To use Ant, you must write XML scripts that build and deploy your Java software. An Ant script for the financial portal application is included with the sample code. The script contains paths to the JAR files needed during compilation, such as those for SAAJ, JAXP, and J2EE. You may need to modify these paths for your system. It is assumed that the application code exists in the path c:\dev\FinancialWebApp\FinancialPortal, with the directory structure in Example 2, which is a standard J2EE WAR structure.
The source code files exist within the directory, WEB-INF/src, where the complete path matches each component's package name. The same subdirectory structure is used for the compiled class files within WEB-INF/classes. The Ant script contains the target, "compile," which looks for the source files in the proper source paths, and places the compiled class files in the proper output paths.
The WEB-INF directory contains the deployment descriptor, web.xml, for the entire web application, containing the Quotes web service, the Fundamental web service, and the financial portal Servlet (Listing Five). The Ant script contains the target, "war," which uses the JAR utility to package up the entire directory structure into a file named "FinancialPortal.war." This archive contains the class files, the application deployment descriptor, and other miscellaneous files that make up the financial portal application.
The next step is to inform Tomcat of the existence of the sample web application. First, locate the path, tomcat-jwsdp-1.4/conf/Catalina/localhost, in the directory where you installed the Java WSDP/Tomcat bundle. In this location, add an application context file named "financialportal.xml," which tells Tomcat where to find the application's WAR file. The contents of this file are simple and can be seen in Listing Six.
Once these steps are complete, the web application is ready to run. When Tomcat is started, it will automatically read the file, financialportal.xml, and attempt to deploy the web application (the financial portal) referenced within. When loaded, the application reads an index file that tells it where to locate the financial filings for all U.S. public companies. This step requires around 30 seconds, after which the application will be ready for requests.
The index file that the application uses is located on the SEC's FTP servers at ftp://ftp.sec.gov/edgar/full-index/master.idx/. Companies file with the SEC periodically, so you will need to download this file occasionally to keep up to date. The Fundamentals Service can be enhanced to download this file automatically based on a schedule.
Finally, open a browser and type http://localhost:8080/ financialportal to display the company ticker request page. If you chose a port other than 8080 when you installed Tomcat, you must modify this URL accordingly. Enter a valid U.S. company ticker, such as EBAY, and around one minute later, you will see that company's quote data, fundamentals, and financial filings.
Building a SOAP web service is ideal when you need to expose data and/or functionality to customers. Even if customers are never directly exposed to it, building a service tier (an entire layer of web services within your application) adds value to an application's architecture by limiting the coupling between components and by creating the potential for future reuse.
Conceivably, the service tier can be extended beyond the boundaries of one application. The service tier can become a company-wide deployment of web services available to all applications in your organization. Thus, the need for a metadata service—a service that describes all available web services—may arise.
The Java Web Service Developer Pack contains all of the tools and support necessary to build web services in Java. Combined with the open-source web container, Tomcat, you can build web services to be deployed on a Windows or UNIX system with little or no modification. If you choose to deploy to Tomcat on Linux, for example, your web service infrastructure costs will be nothing more than the hardware to run on—with no OS or application server licenses to purchase. That makes Java a powerful choice when building web services.
DDJ
public SOAPMessage onMessage(SOAPMessage message)
{
try {
// The envelope contains the message header, body, and attachments
SOAPEnvelope env = message.getSOAPPart().getEnvelope();
SOAPBody bdy = env.getBody();
// Get the ticker symbol
Iterator iter = bdy.getChildElements();
SOAPElement node = getNode("GetData", iter);
if ( node != null )
{
iter = node.getChildElements();
symbol = getNodeValue("symbol", iter);
}
// Get the quote data from Yahoo
String data = getQuoteData( symbol );
// ...
}
catch(Exception e) {
return null;
}
}
Back to article// Create the SOAP reply message
SOAPMessage replyMsg = msgFactory.createMessage(); // part of SAAJ
SOAPEnvelope env = replyMsg.getSOAPPart().getEnvelope();
SOAPBody bdy = env.getBody();
// Add the quote data to the message
bdy.addChildElement(env.createName("symbol")).addTextNode( symbol );
bdy.addChildElement(env.createName("last")).addTextNode( last );
bdy.addChildElement(env.createName("date")).addTextNode( date );
bdy.addChildElement(env.createName("time")).addTextNode( time );
bdy.addChildElement(env.createName("change")).addTextNode( change );
bdy.addChildElement(env.createName("open")).addTextNode( open );
bdy.addChildElement(env.createName("low")).addTextNode( low );
bdy.addChildElement(env.createName("high")).addTextNode( high );
bdy.addChildElement(env.createName("volume")).addTextNode( volume );
return replyMsg;
Back to article// Create the SOAP reply message SOAPMessage replyMsg = msgFactory.createMessage(); SOAPEnvelope env = replyMsg.getSOAPPart().getEnvelope(); SOAPBody bdy = env.getBody(); // Add the fundamental data to the SOAP body ... // Add the financial HTML as a SOAP Attachment AttachmentPart ap = replyMsg.createAttachmentPart(financialHTML,"text/html"); replyMsg.addAttachmentPart(ap); return replyMsg;Back to article
// Create a SOAP request message
MessageFactory msgFactory = MessageFactory.newInstance();
SOAPMessage soapMsg = msgFactory.createMessage();
SOAPEnvelope envelope = soapMsg.getSOAPPart().getEnvelope();
SOAPBody bdy = env.getBody();
// Add the request data to the SOAP body
SOAPBodyElement bdyElem = bdy.addBodyElement(
envelope.createName("GetData"));
bdyElem.addChildElement(envelope.createName("symbol")).addTextNode( symbol );
bdyElem.addChildElement(envelope.createName("name")).addTextNode(companyName);
bdyElem.addChildElement(envelope.createName("CIK")).addTextNode( companyCIK );
SOAPConnectionFactory scf = SOAPConnectionFactory.newInstance();
con = scf.createConnection();
// Send the request to the Quote Service
URL urlEndpoint = new URL( "http://localhost:8080/quotes" );
SOAPMessage quoteReply = con.call( soapMsg, urlEndpoint );
// Send the request to the Fundamentals Service
urlEndpoint = new URL( "http://localhost:8080/fundamentals" );
SOAPMessage fundReply = con.call( soapMsg, urlEndpoint );
Back to article<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE web-app
PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.2//EN"
"http://java.sun.com/j2ee/dtds/web-app_2_2.dtd">
<web-app>
<display-name>Financial Portal</display-name>
<description>
Displays company quote and financial data
</description>
<servlet>
<servlet-name>
fundservlet
</servlet-name>
<servlet-class>
com.fundamentals.FundServlet
</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet>
<servlet-name>
quoteservlet
</servlet-name>
<servlet-class>
com.quotes.QuoteServlet
</servlet-class>
<load-on-startup>2</load-on-startup>
</servlet>
<servlet>
<servlet-name>
FinancialServlet
</servlet-name>
<servlet-class>
com.financialportal.FinancialServlet
</servlet-class>
<load-on-startup>3</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>
fundservlet
</servlet-name>
<url-pattern>
/fundamentals
</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>
quoteservlet
</servlet-name>
<url-pattern>
/quotes
</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>
FinancialServlet
</servlet-name>
<url-pattern>
/financialportal
</url-pattern>
</servlet-mapping>
</web-app>
Back to article<Context
path="/financialportal"
docBase="c:/dev/FinancialWebApp/FinancialPortal/FinancialPortal.war"
debug="1">
<Logger className="org.apache.catalina.logger.FileLogger"
prefix="finance_log."
suffix=".txt"
timestamp="true"/>
</Context>
Back to article