Dr. Dobb's Journal September 2001
The Enterprise JavaBeans (EJB) architecture is the standard component architecture for building distributed object-oriented business applications in Java. EJB makes it straightforward to write applications, because you do not have to understand low-level transaction and state management details, multithreading, connection pooling, and other complex low-level APIs. Plus, EJB application development follows Java's write-once-run-anywhere philosophy — once an Enterprise Bean is developed, it can be deployed on multiple platforms without recompilation or source-code modification.
The EJB architecture addresses the development, deployment, and run-time aspects of an enterprise application's life cycle. EJB technology hides the details of the underlying middleware plumbing, letting you focus on writing pure business logic that does not contain server-specific code. The code can then execute on top of any server implementing the EJB specification. Sun provides a J2EE-compliance (Java 2 Platform, Enterprise Edition) program to guarantee portability across servers.
There are a number of available EJB application servers, including:
Since each application server has its strengths and weaknesses, it is essential that you select an application server based on the requirements of the project at hand. In this article, we present criteria for evaluating and selecting application servers. For instance, one criterion is J2EE compatibility. Sun gives the J2EE logo/brand to products that pass the J2EE Compatibility Test Suite (J2EE CTS), thereby guaranteeing compatibility, portability, and reliability. Other criteria factors include:
Benchmarking must be carried out using two main strategies — CPU-bound testing and database-bound testing.
In the CPU-bound testing model, CPU-intensive EJB components are deployed on each application server, and EJB clients make calls to the server. This tests specific functionality and efficiency of the application server under study. Using CPU-bound testing reveals bottlenecks, internal interactions, and thread management of each application server under study. The main differentiating factors are the JNDI access mechanism, thread management, and connection pooling.
In the database-bound testing model, database-intensive EJB components are deployed on each application server and EJB clients make calls to the server. The purpose of this model is to accurately simulate the application's production environment.
These testing models must be conducted in either nonclustering operating mode (in which the application components are tested in a single-server mode), or clustering mode (in which the application components are tested in a cluster of two members of the same application server instances, used on two different machines).
In sample tests, we capture benchmarking metrics on both the client and the server simultaneously. Captured data were synchronized together for each testing case. Before each test began, a snapshot of the process running on the application server machine was captured. The same was done at the end of each test to ensure that no other processes were running in between.
The metrics on the client were measured by eLoad from RSW (http://www.empirix.com/). The list of metrics, along with the unit of measure, includes:
These metrics are recorded by eLoad, and each value represents the average value of all the virtual users participating in each run. (The Average Response Time for 20 virtual users with different scripts or profiles measures the average response time for the overall number of 20 users.)
The metrics on the server side were measured using Sun Solaris commands (vmstat, iostat, sar -ug, and the like). Both the perfmeter and proctool commands were used for interactively monitoring all other parameters such as CPU utilization, load, network traffic, memory, and disk utilization. A snapshot of the machine state and all processes running on both the application server and Oracle server were captured using the ps command. The metrics recorded for the benchmarking results include Application Server CPU Utilization (in percentage), and Oracle Server CPU Utilization (in percentage).
At the start of each run triggered by the eLoad controller machine, a concurrent process was manually initiated to capture the server-side metrics. During each run, all other external parameters were monitored using the perfmeter tool.
Planning the testing environment consists of deciding upon:
A sample testing configuration consists of two main environments: the stress testing client and the deployment (simulated production) environment. As Figure 1 illustrates, a typical client testing environment might consist of two NT machines to create the stress test or load — one machine acting as the controller and the other an agent. All test runs are conducted from the agent, keeping the controller free to monitor other metrics and resources. The stress testing tool we used was eLoad in running the test scripts, and eTester to develop these scripts.
In the deployment environment, using a nonclustering mode, a single EJB application server is used. In clustering mode, two members of the same application server instances are used on two different machines. Only homogeneous hardware is used in the application tier configuration. Both the web and the database tiers stay unchanged in all the test runs conducted for all the application servers under study, and in all modes of operations.
One of the first things you do is develop a prototype application to be used as a basis for testing across the application servers. This prototype could be a representative piece of the main application. The prototype consists of the following components:
Based on the version of the application server, it may be necessary to port the EJB and client code to different application servers. The changes are mostly in the areas of JNDI lookup, database lookup, and deployment descriptors.
The prototype should address the user pattern that needs to be tested. The user pattern is the profile of queries or requests from all client machines. It represents both the number of all users requesting information, and how they request this information. Since this is just a benchmarking between two application servers, the user pattern can be created with no time delay between requests. This, of course, has a dramatic increase on the load and can achieve fast results. In this benchmarking effort, the virtual user is the unit of load to be applied with no delay. Each user is represented as a separate thread and remains active for exactly five minutes. All user patterns are identical for all servers in all modes and models.
For example, in the CPU-bound application model, a user pattern with a mix of 2 to 10 percent data-bound components and 90 to 98 percent CPU-bound components was used. The test could start with 15 users, with an increase in the load linearly by 15 users to a desired maximum number of users.
These tests could be repeated for clustered and nonclustered configurations as well. The client and server metrics could be plotted for a clustered configuration, as in the sequence of graphs in Figure 2.
Figure 3 is a summary of these results in the spider graph. This type of graph provides a snapshot view in comparing all the application servers.
In the database-bound model, we observed that the database server soon reached a saturated high load with very small impact on the application server. The test could start with one user, with an increase in the load linearly by one user to a desired maximum number of users. These tests could be repeated for clustered and nonclustered configurations as well.
The client and server metrics could be plotted as in the case of CPU-bound testing.
The spider graph in Figure 3 summarizes the results from the CPU-Bound/CPU-Bound Cluster, and Data-Bound/Data-Bound Cluster tests. The application server that occupies most of the area in the graph is better than the other application server(s). If it encloses the polygon of another application server(s), it clearly dominates the other(s).
We learned several lessons from the benchmarking efforts:
Application servers are at the heart of e-commerce and distributed computing environments. In some cases, performance of a high-volume transactional system is more important than integrating with legacy and middleware components. In this article, we only focused on the strategy and criteria for selecting the appropriate application server. We also described a methodology for performing a benchmarking study with the tools recommended for obtaining effective results. The results shown here are merely for demonstration purposes.
DDJ