Dana is a member of the networking group at Wind River Systems. He can be contacted at dana@wrs.com.
AA network connection to a remote computer can cause a performance bottleneck for many real-time, embedded applications, especially those that move large amounts of data. To avoid such bottlenecks, every ounce of network throughput must be squeezed out of the system. Most factors that affect network throughput are beyond the application developer's control, but the overhead incurred by making multiple copies of data can be avoided using a "zero-copy interface." A zero-copy interface allows a real-time application to send to and receive from the network stack without copying data. The data buffer sent or received at the application layer is also used by the device driver to send or receive data.
In this article, I will examine the concepts, requirements, advantages, and disadvantages associated with the use of a zero-copy interface. Currently, two popular real-time operating systems support zero-copy TCP--Wind River's VxWorks and Integrated Systems' pSOS. I'll use the VxWorks zbuf facility as an example implementation. The zbuf facility provides calls to perform zero-copy operations on VxWorks' 4.3 BSD-derived TCP/IP protocol stack. I will also present sample benchmarking code and explain the potential performance benefits of using a zero-copy interface.
To introduce the zero-copy-interface concept, I'll trace the flow of received data through VxWorks' TCP/IP protocol stack. Figure 1 shows a typical data-reception scenario. First, the network chip automatically fills a single device-driver buffer with received data. The driver code then copies the data from the device-driver buffer into a chain of network buffers and hands the chain off to the network stack. The stack processes the buffer chain and passes it up to the socket layer, which holds onto the data. When the application issues a receive call, a user-specified amount of data is copied from the socket's network-buffer chain into an application-provided buffer. Although no data copies are performed within the TCP/IP stack itself, every byte of data is still copied twice: once at the device-driver-layer interface and once at the socket-layer interface. This approach stems from UNIX's process model, where applications running in "process space" cannot access network buffers, which reside in "kernel space." VxWorks' flat memory scheme does not impose such a division of memory, but rather allows network and application buffers to coexist in the same memory space, and the zero-copy interface capitalizes on this.
Figure 2 shows the same data flow, with the addition of a zero-copy interface. As before, the network chip fills a single device-driver buffer with received data. Instead of copying the data, however, the driver code makes a system call to transform the device-driver buffer into one large network buffer. The driver then supplies a pre-allocated, empty device-driver buffer to the chip and passes the filled network buffer to the protocol stack. In effect, the driver is "loaning" its filled receive buffer to the stack for processing, and expects to be notified through a callback routine when the buffer is free to be filled again. Once the data is passed to the protocol stack, the network code performs the exact same functions as before. When the data arrives at the socket layer, the application may issue a zero-copy-interface receive call to access the network buffer. In much the same way that the driver loans its device-driver buffer to the stack, the zero-copy receive call allows the network to loan its network buffer to the application, thereby avoiding the socket-layer copy. When finished with the data, the application must make a zero-copy-interface call to return the network buffer to the stack, at which time the stack returns the buffer to the device driver via the driver callback routine.
Note that no specific mention is made of transport-layer functionality. This layer is not affected by zero-copy issues; therefore, UDP can be used just as well as TCP. The aforementioned scenarios follow data reception, but data transmission is just as easily traced. In this case, the application loans a data buffer to the socket layer via a zero-copy interface send call. The socket layer processes the application's buffer and passes it down the stack, performing the usual stack manipulations along the way. At the device driver, the data is sent to the physical media directly from the network (application) buffer. Once the data has been sent, the driver returns the data buffer to the network stack, which in turn gives it back to the application through an application-specified callback routine.
You have now seen how data may move through VxWorks' TCP/IP protocol stack without being copied. At the socket layer, zero-copy-interface calls allow buffers to be shared between the application and the stack. The driver may also share buffers with the stack, but because of network-chip-architecture constraints, many device drivers still copy data. Some devices use on-chip buffer space, which necessitates data copies, and not all chips can send directly from outgoing buffer chains. Although VxWorks provides system calls to help avoid data copies, the device-driver developer must determine whether data copies are necessary and write the driver accordingly. Since device-driver internals are usually hidden from real-time application developers, I'll concentrate now on the socket-layer portion of the zero-copy interface.
VxWorks offers a socket-layer zero-copy interface with the zbuf facility, a buffer-abstraction library that allows you to manipulate and share buffers by copying pointers to the data instead of the data itself. The facility's basic unit, the zbuf, has three essential properties, as follows:
By itself, the zbuf facility provides a mechanism for sharing data between separate modules. zbufs may be used as the buffering scheme for new protocols, middleware, messaging or communication applications, as well as for sharing data between different programs or software layers.
The socket layer of VxWorks' TCP/IP stack was modified to provide an interface to zbufs. The zbuf routines in Table 2 let the application send zbufs to and receive zbufs from the network stack without copying data. The routines in Table 1 may be used to create and build zbufs before they are sent, and to access and delete zbufs after they are received; the routines in Table 2 perform the actual sending and receiving.
While zero-copy interfaces bypass data copying, they are not always faster than the original method of copying. The time to create, build, maintain, and delete a zbuf must be weighed against the time it takes to copy the data; unfortunately, these are different on every system. They depend heavily upon both hardware (CPU architecture, clocking speed, caching issues, memory-access times) and software (buffer size, task interactions, throughput of network peer, and so on). No simple formula can unequivocally dictate the use of zbuf socket calls instead of the regular BSD socket calls, but the following guidelines will likely indicate the appropriate interface.
The zbuf socket interface is most useful when your real-time application is sending or receiving large amounts of data over the network. By this, I mean not only that a large total quantity of data is sent, but also that each call is sending or receiving a large data buffer (at least a few hundred bytes). If only a few bytes are transferred in each call, the zbuf overhead will outweigh any savings, so applications that have small or infrequently transferred buffers should probably continue to use the standard BSD socket calls.
zbuf socket calls are useful when network throughput, or the time spent in the network code, is limiting your application. If the CPU is idle much of the time, zbuf savings will not be significant.
To use zbuf socket calls, your application must adhere to several requirements:
As previously stated, the zbuf socket interface is most useful for transferring a lot of data over the network. Figure 4(a) presents a few lines that appear in many applications that perform such transmissions. Figure 4(b) is the same, except that it was recoded to use zbufs. The appBufGet() and appBufRetn() references are fictitious, application-specific routines. In most applications, they will just manipulate a list of free, fixed-length application buffers.
Figure 4 illustrates two important points: the strong correlation between the BSD and zbuf socket calls, which makes for an easy migration to zbufs; and the application's need to maintain a pool of buffers with zbuf, as opposed to one allocated buffer with BSD fragment. The pool of buffers allows the application to send multiple data buffers to the network stack--a feat that BSD accomplishes by copying data.
Listings One and Two can be compiled and used to benchmark the zbuf socket interface. Listing One (tcpBlaster.c) repeatedly sends TCP packets across a network connection to a remote computer. Listing Two (tcpBlastee.c) is used to receive the TCP packets sent by tcpBlaster. Neither program looks at the data being sent; they simply track the network throughput and periodically print out performance statistics for the user. The code can be run between a VxWorks target and a UNIX host, or between two VxWorks targets. To utilize VxWorks' zbuf socket interface, simply define the macro INCLUDE_ZBUF_SOCK when compiling the code.
The benchmark code in Listings One and Two was run in a controlled environment to measure VxWorks' TCP/IP protocol stack throughput, with and without the zbuf socket interface. Four test cases were selected:
The test environment consisted of an isolated two-node LAN comprised of a 20-MHz, 68030-based Motorola MVME147 and a 33-MHz, 68040-based Motorola MVME167 networked together via an Ethernet link. Both single-board computers were running VxWorks Version 5.2 and had the benchmark code loaded. The MVME167 was faster and did not limit network throughput, so saving data copies on this board was of little interest. The speed of the MVME147, on the other hand, created a performance bottleneck. Using the zbuf interface to improve the protocol-stack performance on this slower board directly affected overall system throughput.
Table 3 shows that using the zbuf socket interface increased network throughput in all four test cases. Performance improvements ranged from a 3.9 percent speedup when receiving 1K buffers, to a 20.0 percent gain when sending 15K buffers. As the transfer buffer grew, percentages increased, and the fixed zbuf overhead costs paled in comparison.
Figure 1: Data flow through a TCP/IP stack.
Figure 2: Data flow through a TCP/IP stack equipped with a zero-copy interface.
Figure 3: (a) A simple zbuf; (b) complex zbufs.
Figure 4: (a) BSD code fragment; (b) zbuf code fragment.
Copyright © 1995, Dr. Dobb's Journal
(a)
pBuffer = malloc (BUFLEN);
while ((readLen = read (fdDevice, pBuffer, BUFLEN)) > 0)
write (fdSock, pBuffer, readLen);
(b)
pBuffer = malloc (BUFLEN * BUFNUM); /* allocate memory */
for (ix = 0; ix < (BUFNUM - 1); ix++, pBuffer += BUFLEN)
appBufRetn (pBuffer); /* fill list of free bufs */
while ((readLen = read (fdDevice, pBuffer, BUFLEN)) > 0)
{
zId = zbufCreate (); /* insert into new zbuf */
zbufInsertBuf (zId, NULL, 0, pBuffer, readLen, appBufRetn, 0);
zbufSockSend (fdSock, zId, readLen, 0); /* send zbuf */
pBuffer = appBufGet (WAIT_FOREVER); /* get a fresh buffer */
}
Table 1: The zbuf API. (a) Creation and deletion routines; (b) data-copying routines; (c) operations; (d) segment routines.
Routine Description
(a)
zbufCreate() Create an empty zbuf.
zbufDelete() Delete a zbuf and free any associated segments.
(b)
zbufInsertBuf() Create a zbuf segment from a buffer and insert into a zbuf.
zbufInsertCopy() Copy buffer data into a zbuf.
zbufExtractCopy()Copy data from a zbuf to a buffer.
(c)
zbufLength() Determine the length in bytes of a zbuf.
zbufDup() Duplicate a zbuf.
zbufInsert() Insert a zbuf into another zbuf.
zbufSplit() Split a zbuf into two separate zbufs.
zbufCut() Delete bytes from a zbuf.
(d)
zbufSegFind() Find the zbuf segment containing a specified byte location.
zbufSegNext() Get the next segment in a zbuf.
zbufSegPrev() Get the previous segment in a zbuf.
zbufSegData() Determine the location of data in a zbuf segment.
zbufSegLength() Determine the length of a zbuf segment.
Table 2: zbuf socket API.
Routine Description
zbufSockSend() Send zbuf data to a TCP socket.
zbufSockSendto() Send a zbuf message to a UDP socket.
zbufSockBufSend() Create a zbuf and send it as a TCP socket data.
zbufSockBufSendto() Create a zbuf and send it as a UDP socket message.
zbufSockRecv() Receive data in a zbuf from a TCP socket.
zbufSockRecvfrom() Receive a message in a zbuf from a UDP socket.
Table 3: zbuf socket interface benchmark results. Tests run from tcpBlaster to tcpBlastee. zbuf socket interface used on MVME147, not MVME167. (a) Receive: MVME167-->MVME147; (b) Transmit: MVME147-->MVME167.
bufSize Throughput Throughput zbuf Speedup
sockBuf BSD Interface zbuf Interface Over BSD
(a)
1K,5K 494 KB/sec 513 KB/sec 3.9%
5K,15K 644 KB/sec 717 KB/sec 11.0%
15K,45K 682 KB/sec 770 KB/sec 13.0%
(b)
1K,5K 491 KB/sec 518 KB/sec 5.5%
5K,15K 684 KB/sec 798 KB/sec 17.0%
15K,45K 704 KB/sec 844 KB/sec 20.0%
Listing One
/* tcpBlaster.c - test code to send TCP data to a remote target */
/* Copyright 1995 Wind River Systems, Inc. */
/* modification history: 01a,26may95,dzb cleaned-up from
original blaster src. Added zbuf support.
*/
/* includes */
#ifdef UNIX
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <stdio.h>
#include <errno.h>
#else /* UNIX */
#include "vxWorks.h"
#include "sockLib.h"
#include "errnoLib.h"
#include "zbufSockLib.h"
#include "ioLib.h"
#include "inetLib.h"
#include "in.h"
#include "stdlib.h"
#include "stdio.h"
#include "string.h"
#endif /* UNIX */
/*******************************************************************************
* tcpBlaster - continuously send TCP data to a socket
* This routine opens a TCP socket connection to a remote target, which
* should be running tcpBlastee. This routine then enters a continuous loop
* where TCP data is sent to the connected socket. The remote target,
* remote port number, send buffer size, and send and receive socket
* buffer sizes must be set through parameters to this routine.
* Compile with UNIX if running on a UNIX system, else VxWorks is assumed.
* Compile with INCLUDE_ZBUF_SOCK for VxWorks zbuf socket interface support.
* RETURNS: N/A
*/
#ifdef UNIX
main (argc, argv)
int argc;
char * argv [];
{
struct hostent * targetHost; /* remote target hostname */
int targetPort; /* remote target port number */
int sendSize; /* size of buffer to send */
int sockBuf; /* size of socket buffers */
#else /* UNIX */
void tcpBlaster /* VxWorks entry point */
(
char * targetAddr, /* remote target IP address */
int targetPort, /* remote target port number */
int sendSize, /* size of buffer to send */
int sockBuf /* size of socket buffers */
)
{
#ifdef INCLUDE_ZBUF_SOCK
ZBUF_ID zbufSend; /* zbuf for sending */
ZBUF_ID zbufSave; /* zbuf for duplication */
#endif /* INCLUDE_ZBUF_SOCK */
#endif /* UNIX */
struct sockaddr_in sin; /* socket address struct */
int sFd; /* socket fd */
char * pBuffer; /* buffer to send */
bzero ((char *) &sin, sizeof (sin));
#ifdef UNIX
if (argc < 5) /* check args */
{
printf ("usage: %s remoteName remotePort sendSize sockBuf\n",argv [0]);
exit (1);
}
targetHost = gethostbyname (argv[1]); /* process args */
targetPort = atoi (argv [2]);
sendSize = atoi (argv [3]);
sockBuf = atoi (argv [4]);
/* set remote target address values */
if (targetHost == 0 && (sin.sin_addr.s_addr = inet_addr (argv [1])) == -1)
{
fprintf (stderr, "%s: unkown host\n", argv [1]);
exit (2);
}
if (targetHost != 0)
bcopy (targetHost->h_addr, &sin.sin_addr, targetHost->h_length);
#else /* UNIX */
sin.sin_addr.s_addr = inet_addr (targetAddr);
#endif /* UNIX */
sin.sin_port = htons (targetPort);
sin.sin_family = AF_INET;
/* allocate buffer to be sent to remote target */
if ((pBuffer = (char *) malloc (sendSize)) == NULL)
{
printf ("cannot allocate buffer of size %d\n", sendSize);
exit (1);
}
if ((sFd = socket (AF_INET, SOCK_STREAM, 0)) < 0) /* open socket */
{
printf ("cannot open socket\n");
free (pBuffer);
exit (1);
}
/* set socket buffer sizes to user-specified value */
if (setsockopt (sFd, SOL_SOCKET, SO_RCVBUF, (char *) &sockBuf,
sizeof (sockBuf)) < 0)
{
printf ("setsockopt SO_RCVBUF failed\n");
free (pBuffer);
exit (1);
}
if (setsockopt (sFd, SOL_SOCKET, SO_SNDBUF, (char *) &sockBuf,
sizeof (sockBuf)) < 0)
{
printf ("setsockopt SO_SNDBUF failed\n");
free (pBuffer);
exit (1);
}
#ifdef INCLUDE_ZBUF_SOCK
/* create master zbuf and enroll send buffer into created zbuf */
if ((zbufSave = zbufCreate ()) == NULL)
{
printf ("zbufCreate failed\n");
free (pBuffer);
exit (1);
}
if (zbufInsertBuf (zbufSave, NULL, 0, pBuffer, sendSize, NULL, NULL) ==
NULL)
{
printf ("zbufInsertBuf failed\n");
zbufDelete (zbufSave);
free (pBuffer);
exit (1);
}
#endif /* INCLUDE_ZBUF_SOCK */
/* connect to remote target */
if (connect (sFd, (struct sockaddr *) &sin, sizeof (sin)) < 0)
{
printf ("connect failed: host %s port %d\n", inet_ntoa (sin.sin_addr),
ntohs (sin.sin_port));
free (pBuffer);
exit (1);
}
for (;;)
{
#ifdef I
CLUDE_ZBUF_SOCK
/* duplicate master zbuf - duplicate will be sent */
if ((zbufSend = zbufDup (zbufSave, NULL, 0, sendSize)) == NULL)
{
printf ("zbufDup failed\n");
break;
}
/* send data to remote target */
if (zbufSockSend (sFd, zbufSend, sendSize, 0) < 0)
#else /* INCLUDE_ZBUF_SOCK */
if (write (sFd, pBuffer, sendSize) < 0)
#endif /* INCLUDE_ZBUF_SOCK */
{
printf ("tcpBlaster write error: %d\n", errno);
break;
}
}
close (sFd); /* cleanup */
free (pBuffer);
#ifdef INCLUDE_ZBUF_SOCK
zbufDelete (zbufSave);
#endif /* INCLUDE_ZBUF_SOCK */
printf ("tcpBlaster exit\n");
}
Listing Two
/* tcpBlastee.c - test code to receive TCP data from a remote target */
/* Copyright 1995 Wind River Systems, Inc. */
/* modification history: 01a,26may95,dzb cleaned-up from
original blastee src. Added zbuf support.
*/
/* includes */
#ifdef UNIX
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <stdio.h>
#include <sys/time.h>
#include <signal.h>
#include <errno.h>
#else /* UNIX */
#include "vxWorks.h"
#include "logLib.h"
#include "sockLib.h"
#include "zbufSockLib.h"
#include "sysLib.h"
#include "ioLib.h"
#include "stdlib.h"
#include "stdio.h"
#include "string.h"
#include "wdLib.h"
#include "socket.h"
#include "in.h"
#endif /* UNIX */
/* globals */
#ifndef UNIX
WDOG_ID tcpBlasteeWd = NULL;
#endif /* UNIX */
int tcpBlasteeRecv = 0;
int tcpWdIntvl = 10;
/* function declarations */
void tcpBlasteeRate ();
/*******************************************************************************
* tcpBlastee - continuously receive TCP data from a socket
* This routine opens a TCP socket and waits for a remote target to conenct.
* The remote target should be running tcpBlaster. This routine then enters
* a continuous loop where TCP data is read from the connected socket. The
* local port number, receive buffer size, and the send and receive socket
* buffer sizes must be set through parameters to this routine.
* Compile with UNIX if running on a UNIX system, else VxWorks is assumed.
* Compile with INCLUDE_ZBUF_SOCK for VxWorks zbuf socket interface support.
* RETURNS: N/A
*/
#ifdef UNIX
main (argc, argv)
int argc;
char * argv [];
{
int localPort; /* local port number */
int recvSize; /* size of buffer to receive */
int sockBuf; /* size of socket buffers */
#else /* UNIX */
void tcpBlastee
(
int localPort, /* local port number */
int recvSize, /* size of buffer to receive */
int sockBuf /* size of socket buffers */
)
{
#ifdef INCLUDE_ZBUF_SOCK
ZBUF_ID zbufRecv; /* received zbuf */
#endif /* INCLUDE_ZBUF_SOCK */
#endif /* UNIX */
struct sockaddr_in sin; /* local socket address */
struct sockaddr_in from; /* remote socket address */
char * pBuffer; /* buffer to receive into */
int fFd; /* master socket fd */
int sFd; /* slave socket fd */
int numRead; /* number of bytes read */
int len = sizeof (from); /* length of remote sock addr */
bzero ((char *) &sin, sizeof (sin));
bzero ((char *) &from, sizeof (from));
#ifdef UNIX
if (argc < 4) /* check args */
{
printf ("usage: %s localPort recvSize sockBuf\n", argv [0]);
exit (1);
}
localPort = atoi (argv [1]); /* process args */
recvSize = atoi (argv [2]);
sockBuf = atoi (argv [3]);
signal (SIGALRM, tcpBlasteeRate); /* set up stats timer */
alarm (tcpWdIntvl);
#else /* UNIX */
if (tcpBlasteeWd == NULL && (tcpBlasteeWd = wdCreate ()) == NULL)
{
printf ("cannot create tcpBlastee watchdog\n");
exit (1);
}
wdStart (tcpBlasteeWd, sysClkRateGet () * tcpWdIntvl,
(FUNCPTR) tcpBlasteeRate, 0);
#endif /* UNIX */
sin.sin_port = htons (localPort);
sin.sin_family = AF_INET;
/* allocate buffer into which data will be received (non-zbuf) */
if ((pBuffer = (char *) malloc (recvSize)) == NULL)
{
printf ("cannot allocate buffer of size %d\n", recvSize);
exit (1);
}
if ((fFd = socket (AF_INET, SOCK_STREAM, 0)) < 0) /* open socket */
{
printf ("cannot open socket\n");
free (pBuffer);
exit (1);
}
if (bind (fFd, (struct sockaddr *) &sin, sizeof (sin)) < 0) /* set local */
{
printf ("bind error\n");
free (pBuffer);
exit (1);
}
if (listen (fFd, 5) < 0) /* set listen queue */
{
printf ("listen failed\n");
free (pBuffer);
exit (1);
}
/* wait for incoming socket connections */
while ((sFd = accept (fFd, (struct sockaddr *) &from, &len)) == -1)
;
/* set socket buffer sizes to user-specified value */
if (setsockopt (sFd, SOL_SOCKET, SO_RCVBUF, (char *) &sockBuf,
sizeof (sockBuf)) < 0)
{
printf ("setsockopt SO_RCVBUF failed\n");
free (pBuffer);
exit (1);
}
if (setsockopt (sFd, SOL_SOCKET, SO_SNDBUF, (char *) &sockBuf,
sizeof (sockBuf)) < 0)
{
printf ("setsockopt SO_SNDBUF failed\n");
free (pBuffer);
exit (1);
}
tcpBlasteeRecv = 0; /* reset bytes received */
for (;;)
{
#ifdef INCLUDE_ZBUF_SOCK
numRead = recvSize; /* set desired receive size */
/* read data from socket */
if (((zbufRecv = zbufSockRecv (sFd, 0, &numRead)) == NULL) ||
(numRead == 0))
#else /* INCLUDE_ZBUF_SOCK */
if ((numRead = read (sFd, pBuffer, recvSize)) <= 0)
#endif /* INCLUDE_ZBUF_SOCK */
{
printf ("tcpBlastee read error: %d\n", errno);
break;
}
#ifdef INCLUDE_ZBUF_SOCK
zbufDelete (zbufRecv); /* delete zbuf - return buf */
#endif /* INCLUDE_ZBUF_SOCK */
tcpBlasteeRecv += numRead; /* track of total bytes */
}
close (fFd); /* cleanup */
close (sFd);
free (pBuffer);
#ifndef UNIX
wdCancel (tcpBlasteeWd);
#endif /* UNIX */
printf ("tcpBlastee exit.\n");
}
/*******************************************************************************
*
* tcpBlasteeRate -
* RETURNS: N/A
*/
void tcpBlasteeRate ()
{
/* print stats for user's benefit */
if (tcpBlasteeRecv > 0) /* incoming data ? */
{
#ifdef UNIX
printf ("%d bytes/sec tot %d\n", tcpBlasteeRecv / tcpWdIntvl,
tcpBlasteeRecv);
#else /* UNIX */
logMsg ("%d bytes/sec\n", tcpBlasteeRecv / tcpWdIntvl, 0, 0, 0, 0, 0);
#endif /* UNIX */
tcpBlasteeRecv = 0;
}
else /* no data in last interval */
{
#ifdef UNIX
printf ("No bytes read in the last 10 seconds.\n");
#else /* UNIX */
logMsg ("No bytes read in the last 10 seconds.\n", 0, 0, 0, 0, 0, 0);
#endif /* UNIX */
}
/* re-schedule stats timer for another interval */
#ifdef UNIX
signal (SIGALRM, tcpBlasteeRate);
alarm (tcpWdIntvl);
#else /* UNIX */
wdStart (tcpBlasteeWd, sysClkRateGet () * tcpWdIntvl,
(FUNCPTR) tcpBlasteeRate, 0);