Dwayne Phillips works as a computer and electronics engineer with the U.S. Department of Defense. He has a Ph.D. in Electrical and Computer Engineering from Louisiana State University. His interests include computer vision, artificial intelligence, software engineering, and programming languages.
Introduction
This is the eighth in a series of articles on images and image processing. The first seven articles discussed reading, writing, displaying, and printing images (TIFF format); histograms and histogram equalization; edge detection; and spatial frequency filtering. This article will discuss a collection of image operations (all source code is available through the C Users Group).In the past seven installments, I discussed various topics in image processing. In this installment, I will address image operations that allow you to have fun with images. These operations are: adding and subtracting images, rotating and flipping images, cutting and pasting parts of images, zooming and shrinking images, and creating blank images. As the photographs will show, these allow you to edit images by piecing them together. I'll list the code to perform the operations, integrate them into the C Image Processing System (CIPS), and give application programs that use them.
Addition and Subtraction
Figure 1 illustrates the first operations: image addition and subtraction. In image addition, you add each pixel in one image to each pixel in another and place the result in a third image. If the sum is greater than the maximum pixel value, you set it to the maximum value. Image subtraction is the same. If the difference is less than 0, then you set it to zero.Image addition and subtraction allow you to place objects into and remove objects from images. In one application, you subtract two images in a sequence to see what has changed. For example, take an aerial photograph of a factory two days in a row, subtract the two, and you can detect what has moved from day to day.
You can also subtract images to remove features. Photograph 1 shows a house. Photograph 2 shows the output of an edge detector applied to Photograph 1 (Phillips Nov. 1991, Jan. 1992). Photograph 3 shows the result of subtracting the edges from the original image. Note how the edges in Photograph 3 are whitened out or removed.
Listing 1 shows the functions that implement image addition and subtraction. The add_image_array function adds two image arrays. Given the names of the image files (in1_name and in2_name) and the line and element parameters, it reads the arrays, adds them, and stores the result to an output image. You can use the same image for the two inputs and the output, that is, you can add an image array to itself and write the result on top of itself. If the output image does not exist, add_image_array will create it by calling create_allocate_tiff_file. If the sum of two pixels is greater than the maximum pixel value, you set it to the maximum value. The subtract_image_array function is the same as add_image_array except it subtracts the corresponding pixels from two image arrays.
These simple functions may seem insignificant. We are only talking about adding and subtracting, but did you ever do anything useful by adding and subtracting numbers? Think of the possibilities and experiment.
Rotation and Flipping
The next operations are image rotation and flipping. Figure 2 and Figure 3 show three rotations and two flips. I define rotation as pinning the lower right-hand corner pixel of an image array and rotating the image array about that pixel, clockwise 90 degrees. Figure 2 shows the three rotations possible. If you rotate four times, the result is the same as the original.I define two image flips shown in Figure 3 : horizontal and vertical. The horizontal flip exchanges the vertical columns on opposite sides of the center of the image array. The first column swaps with the last column; the second column swaps with the next to last column, and so on. The vertical flip exchanges horizontal rows on opposite sides of the center of the image array. The top row swaps with the bottom row, the next to top row swaps with the next to bottom row, and so on.
Rotating images is useful for effects and making wide images narrow and narrow images wide. Image flipping can turn objects upside down and create spatial negatives (right-handed people become left-handed).
Photograph 4 shows the result of rotating the image in Photograph 1. Note, the code in Listing 2 only rotates one image array at a time. Photograph 1 is not the result of rotating every array in the image. It is the result of taking every array, rotating it, and pasting the result into special places in another image. This requires the application program given later in Listing 8.
Listing 2 shows the function needed for rotating and flipping image arrays. rotote_flip_image_array will perform all three rotations shown in Figure 2 and both flips shown in Figure 3. It requires the names of the input and output image and the line and element parameters. The input and output image can be the same. If the output image does not exist, rotote_flip_image_array will create it. A single rotate operation (type = 1) rotates the array once. Type 2 and 3 rotations require the same the operation two and three times. If you request a type 3 rotation, the function will execute the first three if statements and rotate the array three times. This is not the most efficient method, but it is the simplest to implement and understand. The if statements type == 4 and type == 5 perform the horizontal and vertical flips. These are straightforward swapping of rows and columns.
Cut and Paste
The next operations are image cut and paste. They allow you to cut and paste rectangles from one image to another. Figure 4 shows a cut and paste example where a section of image B was pasted into image A by reading from one image and writing into another one. Photograph 5 shows the result of pasting parts of the image in Photograph 2 into the image in Photograph 1. This demonstrates how you can paste processing results into images to better judge their effect.Listing 3 shows the two new functions. cut_image_piece is a wrapper around the existing read_tiff_image. It checks for existence of the image and then reads it. In the same manner, the function paste_image_piece is a wrapper around write_array_into_tiff_image. It checks for the existence of the output image and creates it if required. Then, it writes the image piece into the target image file. The last function of Listing 3, check_cut_and_paste_limits, ensures the line and element parameters are not too large. Calling programs should use this before using the cut and paste routines.
Image Scaling
The next operation is image scaling. This includes zooming (enlarging) and shrinking images. You use image enlarging to zoom in on a part of an image for closer examination. Image shrinking is useful for saving disk space, fitting a large image into a small display, and pasting several images into one image of the same size.I chose two techniques for image zooming: replication and interpolation. Figure 5 shows these techniques for a 2x enlargement. You enlarge an image by repeating a single pixel several times in both directions. The replication method uses the simplest technique; it just copies the pixel. This is easy to do, but the enlarged image appears jagged.
The interpolation technique is harder to implement, but the result is smoother. Note in Figure 5 how the 1 in the upper left corner of the original enlarges in the two techniques. Replication copies it, but interpolation smoothes the transition from 1 to 4, 6, and 9. The extra pixel values are half way between the 1 and the values of its neighbors. The 9 in the original image does not change in the final result. This will always be the case for the pixels on the bottom row and far right column of the image.
Photograph 6 shows a 4x enlargement of an image array using the replication technique (part of the boy image of previous articles). Photograph 7 shows a 4x enlargement using the interpolation technique. The curves in the replication technique are more jagged than the interpolation case.
I implemented three techniques for image shrinking: corner, average, and median. Figure 6 shows these three techniques for a 2x shrinking. In shrinking you will replace an nxn (2x2, 4x4, etc.) pixel area with one pixel. How do you choose the one pixel value to represent the nxn area? The corner method uses the pixel in the upper left-hand corner of the nxn area. The average method uses the average pixel value and the median method uses the median pixel value. Figure 6 shows how each method returns a different result. Which should you use? Experiment — try all three and use the one that works best for the given image. The corner method is fastest since it does not perform any calculations. The median method is slowest, especially for a large nxn area, because it takes the pixels in the nxn area, sorts them, and finds the median value.
Photograph 8 shows an aerial image. Photograph 9 shows 2x shrinking of that image using the corner technique. Most image shrinkings give a good appearance and it is difficult to discern any differences among the three techniques.
Listing 4 shows the functions that implement the image zooming and shrinking. I executed these functions for a factor of 2 and 4. You can repeat these over and over in combinations to achieve about any factor you want. Usually, it is not practical to shrink by more than 4x because you would not be able to see anything. You can modify the code to use other factors. Just ensure your factor will divide into ROWS and COLS (both = 100) evenly. If not, you will have a blank area along the bottom and right edge of the image arrays.
The functions zoom_image_array and interpolate_pixel perform image zooming. zoom_image_array reads a ROWSxCOLS image array from the input image file, enlarges it by 2 or 4, and writes the resulting image arrays to an output file. The output image file will be 200x200 or 400x400. The function will do replication or interpolation depending on the value of the method parameter. Both methods are similar except interpolation calls the interpolate_pixel function to calculate the interpolated values.
The trick to zoom_image_array is the six levels of nested for statements. These allow you to use any factor for the zooming. You read one input image array before looping and zooming. The outside loops (over A and B) loop over the zoom factor because you are dividing the input image array into factor*factor parts. The loops over i and j loop over the individual pixels in the input image array. Again, you divide ROWS and COLS by factor because you are dividing the input into parts. The loops over a and b replicate each pixel factor*factor times. This structure may appear odd, but it works and it will work for any factor which divides into ROWS and COLS evenly.
The functions shrink_image_array, average_pixel, median_pixel and blank_image_array implement image shrinking. This works for a factor of 2 or 4 so shrink_image_array reads in either a 200x200 or 400x400 image, shrinks it down to a 100x100 area, and writes this to an output file. It uses either the corner, average, or median technique depending on the method parameter. The average_pixel and median_pixel functions determine the output pixel value for those techniques.
shrink_image_array also uses the six levels of nested for loops. The computation is backwards, but the loop structure is basically the same as in zoom_image_array. It reads an input image array factor*factor times, loops through the shrinking, and writes out one image array.
If the desired image array is out of the limits of the input image file, then you call blank_image_array to create an empty array instead of trying to read from disk. This out-of-limits condition will occur many times. If the input image is 300x300 and you want to perform a 4x reduction, you will need a 400x400 input image. blank_image_array allows you to shrink the 300x300 image. The output will have a blank area along the bottom and right edges. That is much better than the garbled area you would have had otherwise. The final function in Listing 4 is get_scaling_options. The main routine of CIPS will call this so the user can enter the scaling parameters.
Blank Images
The final operation is creating an image. You use a blank image as a bulletin board to paste other images together. Photograph 10 shows a composite made up of four images that I pasted onto a blank image. The upper left shows the image in Photograph 9. The lower left shows the result of running a low-pass filter on the original image. The upper right shows the edge-detector output, and the lower right shows the result of a high-pass filter. I created this mosaic by running the original image through the three operations. Then I created a blank image and pasted the four smaller images into it.Listing 5 shows an application program that will create a blank image. This interprets the command line, sets up the image header, and calls create_allocate_tiff_file.
Now let's integrate all these new functions into CIPS. Listing 6 shows the changes made to the main program of CIPS. The case sections handle image addition and subtraction, rotation and flipping, cutting and pasting, zooming and shrinking, and creating a blank image. Each case section obtains parameters from the user and calls the desired function.
The final four code listings are application programs that use the new features added in this article. Listing 7 shows the mainas program. It adds or subtracts two entire image files and stores the result in a third file. All three files can be the same if desired. This program checks the command line, checks the size of the input files, and loops over them either adding or subtracting the image arrays. This program produced the image in Photograph 3.
Listing 8 shows the mainr program. This program produced the rotated house shown in Photograph 4. It is hard-coded to take a 300x300 image and rotate it 90 degrees by rotating each part of the image and pasting them into the correct places on the output image. Each of the nine calls to rotate_flip_image_array does the rotation and writes the result to the specified location. A little thought and a nested for loop construction would allow you to generalize this to any size input image.
Listing 9 shows the side program. This program will take two input images and combine them into one output image. It can place the inputs side by side or top to bottom in the output image. It uses the create_allocate_tiff_file function to create the output image as a bulletin board. It loops through the input images and uses cut_image_piece and paste_image_piece to place the two input images together. Photograph 11 shows part of the house image placed side by side with a high-pass filter output. Photograph 12 shows the same two images placed top to bottom. The side program allows you to display and compare two complete image files. If the total image is too large to display, shrink it.
Listing 10 shows the half program. This program shrinks an input image by a factor of 2. It produced the image in Photograph 9 by halving the image in Photograph 8. It loops over the input image and shrinks each 200x200 area into a 100x100 area by calling shrink_image_array.
Conclusion
This installment of the series described several image operations. The major benefit of these is the ability to edit images by adding, rotating, flipping, scaling, and cutting and pasting. These operations are fun because they allow you to place original images and processing results together in combinations and display them all at once. Enjoy and experiment. These are low level tools that you can combine in an endless variety of ways.
References
Karl, John H. 1989. An Introduction to Digital Signal Processing. San Diego: Academic Press.Levine, Martin D. 1985. Vision in Man and Machine. New York: McGraw-Hill.
Phillips, Dwayne. March 1991. "Image Processing, Part 1: Reading the Tag Image File Format." The C Users Journal. 92-100.
Phillips, Dwayne. May 1991. "Image Processing, Part 2: Displaying Images and Printing Numbers." The C Users Journal. 117-126.
Phillips, Dwayne. June 1991. "Image Processing, Part 3: Displaying and Printing Images Using Halftoning." The C Users Journal. 89-104.
Phillips, Dwayne. August 1991. "Image Processing, Part 4: Histograms and Histogram Equalization." The C Users Journal. 59-70.
Phillips, Dwayne. November 1991. "Image Processing, Part 5: Writing Images to Files and Basic Edge Detection." The C Users Journal. 75-102.
Phillips, Dwayne. January 1992. "Image Processing, Part 6: Advanced Edge Detection." The C Users Journal. 47-63.
Phillips, Dwayne. October 1992. "Image Processing, Part 7: Spatial Frequency Filtering." The C Users Journal. 71-92.