Tim, who is a team leader at AmeriData Consulting, has developed flight-simulator technology for the U.S. Air Force and is author of Photo-Based 3D Graphics in C++ (John Wiley & Sons, 1995).
Many memorable movie scenes have been created using a graphics technique known as "blending." In Jurassic Park, for example, computer-generated dinosaurs were blended into existing live-action footage.
This article describes a powerful graphics technique known as "alpha blending" (sometimes referred to as "image compositing"). Careful application of this algorithm permits two or more images to be composited in such a way that viewers can't detect that the resulting image is a composite. I'll present an abridged version of the Porter-Duff alpha-blending algorithm (described by T. Porter and T. Duff in their paper "Compositing Digital Images," in the SIGGRAPH '84 Proceedings) and show several applications, including how blending can be used to create realistic shadows.
The fundamental idea behind blending is that a third channel or image can be used to drive a blending process, which combines the image of the object to be blended (the "cutout image") and background images. The blending techniques I present here combine the cutout and background images using the equation Bij=Cij Aij+(1-Aij) Bij, where i and j are image column and row indexes, respectively, and Aij is a factor (called "alpha") that has a value between 0 and 1 inclusive. Bij is a pixel in the output image and Cij is a pixel in the cutout image. As Figure 1 illustrates, you implement blending by applying the blending equation to three image objects: the cutout image, the corresponding alpha image, and the output image. Each pixel (i,j) of the cutout image is assumed to be "lined up" or colocated with pixel (i,j) in the cutout's alpha image. Each pixel in the alpha image contains a number that can be interpreted as an alpha factor. The alpha factor acts as a translucency indicator: A value of 0 implies transparency and a value of 1 implies complete opaqueness.
From another perspective, the blending equation replaces each background-image pixel with a weighted sum of itself and the corresponding cutout-image pixel. The weights are provided by alpha-image pixel values. It then follows that if each alpha factor in the alpha image is set to 1, the cutout image will replace the background pixel over which it is superimposed. If the alphas are all 0, then blending the cutout image into the background image will have no effect since each pixel in the cutout image will have effectively been made transparent. Even more interesting things start happening when you use alphas between 0 and 1.
An alpha image can be generated during the process of making a cutout image. The overall process of creating an alpha image is as follows:
To convert the mask image into an alpha image, the edges of the white area in the mask image are smoothed. In smoothing, the mask-image alpha factors will be calculated in pixels along the edges of the tree that are greater than 0 and less than 1. When this mask image is then used to blend the cutout and background images together, an effect will be produced whereby the outermost edges of the tree are made translucent. The end result is that the normal effects of aliasing are reduced because pixels along the edges of the cutout tree image (where one form of aliasing occurs) are being combined with the background.
To get the proper effect, you need to be very particular about performing edge smoothing in an alpha image. The top part of Figure 2 shows three sideview plots of the intensities in one horizontal line of the mask image shown in the bottom half of Figure 2. The lower-intensity plot is labeled "Unsmoothed." This plot shows the edge profile of the mask image. If you smooth the edges of the mask image by applying a simple, sliding-window average (or "block") filter, the result is the situation diagrammed in the middle-intensity plot labeled "Conventional Block Filter" in the upper part of Figure 2. In this case, the block filter will "smear" the mask image into areas for which there are no corresponding cutout-image pixels. The parts of the image affected are indicated by the small white triangular areas that lie outside the vertical lines denoting the edges of the cutout image. Nonzero alphas in these triangular regions will cause 0-valued pixels from the cutout image to be mixed into the background image; the result is that the blended image exhibits a dark halo. What you want instead is to smooth the mask image so that the alpha factors corresponding to 0 pixels in the cutout image remain 0. In other words, you only want to calculate alpha factors where the mask-image pixels are greater than 0. This approach will produce the result labeled "Desired" in the uppermost portion of Figure 2, where the alpha factors along the edges of the tree are less than 1 and all alphas outside the edges of the tree are 0. Alphas lying in the interior of the cutout remain 255.
Figure 3, a visual-effect scene created by compositing two maple-leaf models based on the same image, illustrates the blending effect. Figure 3(a) is blended, while Figure 3(b) is not. Figure 4(a) is a close-up of the blended leaf border composited using alpha blending. Figure 4(b), on the other hand, is a close-up of the same portion of the leaf border, but unblended. The image in Figure 4(a) exhibits less contrast than its unblended counterpart. This lack of contrast will cause the model's edges to remain unnoticed by your eye. In this case, what you don't see makes the difference. A cutout image and its corresponding alpha image can be used during scene generation to composite models into the final scene. Blending the cutout into the final scene minimizes the effects of aliasing.
This edge-smoothing algorithm is implemented as a two-pass process in Listing One. In particular, the smoothX3NN function is applied to the three nearest neighboring pixels in the image's horizontal lines (which are oriented along the x axis of the world-coordinate system); the smoothY3NN function is applied to the three nearest neighboring pixels in vertical columns in the image. To smooth the mask image in both the x and y directions, a call must be made to each of the smoothing functions. The order in which the calls are made will not appreciably change the outcome.
The smoothX3NN function makes a pass over the image, line by line. As the function traverses the pixels in each line, it calculates the average of the current pixel and one pixel on either side of it. These three pixels make up an "averaging window." Function smoothX3NN writes an output pixel into the output alpha image (in the location corresponding to the center of the averaging window) only if the pixel in the center of the averaging window has a corresponding nonzero cutout-image pixel. Larger averaging windows make for greater smoothing effects. For example, a smoothX4NN and smoothY4NN could easily be constructed and applied to mask images. Listing One contains the edge-smoothing functions that convert a mask image into an alpha image.
Suppose you want to uniformly vary the opaqueness of all the alpha-scale factors in an alpha image. Instead of creating a new alpha image containing the scaled alpha factors, you can incorporate a scale factor into the blending equation as follows. First let f equal the new alpha factor, which incorporates the alpha-scale factor s, as in f=sAij/m, where m is 255, the maximum 8-bit pixel value. Since Aij ranges from 0 to 255, the alpha-scale factor Aij/m ranges from 0 to 1. The blending equation can then be rewritten as Bij=f Cij+(1-f) Bij. The new equation enables the contribution of the alpha image to the final, blended result to be scaled by varying s. By default, the alpha-scale factor is set to a value of 1 in the scene file. Setting it to 0.5 would cause a cutout-image pixel with a corresponding alpha-image pixel value of 255 (the maximum possible) to be combined with the output-image pixel in a ratio of 1:1. In other words, the output pixel would be a 50 percent mixture of both cutout-image pixel and background-image pixel. Figure 5 shows a visual-effect scene in which the same model is blended into a background image with four different alpha-scale factors. From left to right, the alpha scale factors are: 0.8, 0.6, 0.4, and 0.2. Varying the alpha-scale factor during generation of a sequence of images can result in morphing effects.
Suppose, however, that you wish to add a shadow to a model. Using a variation of the alpha-scale factor idea, you can simply add to the scene file another model that uses the same image file as the original model. To add shadows, you first include another copy of each model in the scene file, renaming the model to indicate that it is a shadow. Now you interactively rotate and position each shadow model using the scene preview tool until they appear in the desired perspective.
Once the shadow models have been suitably positioned, how do you make the shadows dark? Actually, you want to subtract an amount from the area of the background image upon which the shadow is cast. The amount to be subtracted can be determined from the alpha values and the alpha-scale value itself. You can cause such a subtraction to occur by making the alpha scale negative. In the case of a negative alpha-scale factor s, the blending equation is altered to the form Bij=Bij+fCij. Since f will be negative if s is negative, a subtraction is performed as desired. Obviously, a darker shadow requires a greater negative alpha-scale factor. A good starting point is to make alpha scale -0.2; this will subtract 20 percent of the alpha-image value from the background pixels. A lower bound is placed on Bij so that it cannot have a calculated value of less than 1.
The two leftmost shadows shown in Figure 6 were created by blending the alpha image itself into the background. The shadow of the spruce tree on the right side of Figure 6 was created by using the spruce-tree image itself as a source of values to subtract from the background image. The result is a more complex and interesting looking shadow. Since the shadow of an object can be occluded by the model from which it is derived, the shadow model is always rendered first.
Listing Two implements the blending equation and incorporates the modifications for positive and negative alpha-scale factors. Note that the blend function accommodated the possibility that the blended output pixels may be offset by an amount specified in the function arguments xOffset and yOffset. Listing Two contains the function blend.
When you understand how alpha blending works, you can use it to smooth edges of cutout images, add shadows to existing models, alter the opaqueness of any model, and other special effects. You can also combine alpha blending with other graphics techniques, such as digital image warping ("morphing"); Figure 7 is an example of this.
Figure 1: Implementing the blending equation.
Figure 2: Generating a mask image from a cutout image.
Figure 3: (a) Blended model; (b) unblended model.
Figure 4: (a) Closeup of unblended model; (b) closeup of blended model (composited using alpha blending).
Figure 5: Blending into a background image with different alpha-scale factors.
Figure 6: Creating shadows by blending the alpha image itself into the background.
Figure 7: Combining three-dimensional image warping with alpha blending for special effects.
Copyright © 1995, Dr. Dobb's JournalListing One
void memImage::smoothX3NN(){
int x, y;
BYTE HUGE *myTemp = bytes;
for (y = 1; y <= imageHeight; y++){
for (x = 1; x < imageWidth; x++){
if(x == 1 && *myTemp > 0){
*myTemp = (*(myTemp)+ *(myTemp+1))*0.5;
}
else
if(x == imageWidth && *myTemp > 0){
*myTemp = (*(myTemp-1) + *(myTemp))*0.5;
}
else
if(x > 1 && x < imageWidth && *myTemp > 0){
*myTemp = (*(myTemp-1) + *(myTemp)+ *(myTemp+1))*0.33333;
}
myTemp++;
}
myTemp+=pads;
}
}
void memImage::smoothY3NN(){
int x, y, y1, y2, y3, result;
for (x = 1; x <= imageWidth; x++){
for (y = 1; y <= imageHeight; y++){
if(y > 1) y1 = getMPixel(x, y - 1);
y2 = getMPixel(x, y);
if(y < imageHeight) y3 = getMPixel(x, y + 1);
result = 0;
if(y == 1 && y2 > 0)
result = (y2 + y3) * 0.5;
if(y > 1 && y < imageHeight && y2 > 0)
result = (y1 + y2 + y3) * 0.33333;
if(y == imageHeight && y2 > 0)
result = (y1 + y2) * 0.5;
setMPixel(x, y, (BYTE)result);
}
}
}
Listing Two
short blend(memImage *inImage, memImage *alphaImage, memImage *outImage,
float alphaScale, short xOffset, short yOffset){
//
// Blend over the common area in input and mask images
//
short inputRows = inImage->getHeight();
short inputCols = inImage->getWidth();
short maskRows = maskImage->getHeight();
short maskCols = maskImage->getWidth();
short commonRows = min(inputRows, maskRows);
short commonCols = min(inputCols, maskCols);
//
// each memImage is assumed to be opened for random access
short x, y;
BYTE maskPixel, inPixel, outPixel, addedPixel;
float inWeight, outWeight;
for(y = 1; y <= commonRows; y++){
for(x = 1; x <= commonCols; x++){
maskPixel = maskImage->getMPixel(x, y);
if(maskPixel > 0){
inPixel = inImage->getMPixel(x, y);
outPixel = outImage->getMPixel(x + xOffset, y + yOffset);
inWeight = (float)maskPixel / 255.0 * alphaScale;
outWeight = 1.0 - inWeight;
if(alphaScale > 0.0)
addedPixel = (inWeight * (float)inPixel) +
(outWeight *(float)outPixel) + 0.5;
else{
addedPixel = (float)outPixel + (inWeight *(float)inPixel) + 0.5;
// make certain shadows won't produce negative values
if (addedPixel > outPixel) addedPixel = outPixel;
}
if (addedPixel < 1) addedPixel = 1;
if (alphaScale == 0.0) addedPixel = 0;
outImage->setMPixel(x + xOffset, y + yOffset, addedPixel);
}
}
}
return 0;
}