Dr. Dobb's Journal September 1999
WiT, a visual programming package from Logical Vision (http:// www.logicalvision.com/), was created for designing algorithms. With WiT, you rapidly develop solutions by building dataflow-based visual programs called "igraphs" that are based on icons and links. The icons represent operators to input, process, and output data objects that travel along the links of the igraph. Figure 1 is an example igraph, where the execution starts with the operator rdObj, which reads in an image file. Through the first link, the image object flows to the next operator, where it is zoomed to a particular size. After an additional contrast enhancement, the processed image is displayed. An optional WiT toolkit lets you create custom graphical user interfaces for turnkey end-user applications. The toolkit offers Visual Basic support as well as a proprietary platform-independent, object-oriented GUI toolkit for ANSI C.
WiT is primarily a tool for developing image-processing applications. In particular, its built-in operators (grouped into libraries, see Figure 2) and support for frame grabbers allow for a rapid prototyping of industrial inspection algorithms, such as the optical quality assessment of product parts in a production line. Other examples include inspecting printed-circuit boards for completion or determining whether pen caps were attached correctly. Applications for optical character recognition of license plates and barcode reading also exist.
Scientific institutions use WiT to develop solutions to analyze the tracings of electrophoresis gels, classify and count blood cells from images taken from visible light microscopes, and much more. Complex solutions to challenging image-processing problems, such as the measurement of nonrigid organs in X-ray imaging sequences, can also be developed with WiT. Such research projects, however, may reach the limits of the system.
Apart from being useful to develop real-world applications, WiT can also be used as an educational tool. Many textbook image-processing operators are readily available, so images demonstrating their effects can be prepared without much effort. Students quickly learn its basic visual-programming features. Without the need to bother with the details of a particular programming language, the courses can focus on image-processing functions and algorithms. As WiT supports the visual programming of modular and well-documented code as well as customized GUIs, it is also used to teach more abstract concepts of software development and project management.
With any programming language, well-documented code does not only contain comments and meaningful names for procedures and variables. It is also accompanied by an abstract description of the implemented algorithm, for example, flow-charts or Nassi-Schneiderman diagrams. WiT's visual igraphs integrate these features to obtain self-contained visual programs. In small projects, a hardcopy may be all the documentation required. An annotation function lets you insert arbitrary geometrical objects and text directly into the executable igraph. In Figure 3, for instance, the operators are grouped into the main building blocks input, processing, and output. These modules are highlighted using colored rectangles. Text fields are used to explain the overall function as well as local details. Operators and links may have arbitrary names in an igraph (instance names). Together with the distinct icon images, such an igraph can be understood at a glance.
WiT igraphs follow a dataflow paradigm. Unlike other such systems, it is possible to realize complex nonlinear and parallel algorithms instead of just performing a linear sequence of operations. The operator library therefore includes control structures such as if branches and for loops. WiT's scheduler is able to handle these control structures and even automatically distribute operator execution on multiCPU machines or other dedicated hardware. Consequently, operators are executed in a time-efficient order, which is not necessarily the logical order the programmer had in mind. Hence, you have to ensure that your igraph executes properly under different circumstances. To make this happen, WiT includes a number of synchronization operators, such as switches or gates waiting for an object to arrive, before allowing another object to pass. Figure 3 uses such a gate to block the processing of the next image of a sequence. After users acknowledge a dialog box, the gate receives a synchronizing object and lets the next image file name pass on to an input operator.
The dataflow concept is a good choice for a visual-programming environment that lets you create executable block diagrams. Certain code structures, however, require additional effort to work the way you expect it; even more if you are used to work control driven or object oriented. Figure 4 gives you an idea of the difficulties. Both igraphs perform a number of low pass operations on an input image, subtract the filtered image from the original, and display the result. With just a fixed number of iterations given as a parameter to the for operator, the igraph in Figure 4(a) realizes this loop effectively. If the loop limit is created by another operator, however, a data object containing the value is needed as an input to the for operator. As this input is required each time for executes, an overhead is needed to regenerate the limit value and synchronize it with the loop; see Figure 4(b).
With complex igraphs, the handling and synchronizing of data objects can become rather complicated, and before spending too much time on the peculiarities of the dataflow concept you should consider the control-driven model. A simpler code and a more efficient execution would be possible if the igraph was implemented in (for example) C; see Listing One. Luckily, WiT allows the combination of both paradigms. You can write your own functions in C and use them in your igraphs like any built-in operator.
The WiT Manager component handles the creation and administration of user-defined C functions and data structures. Adding your own function is easy. All you have to do is define the operator interface in a simple language. The Manager then creates a C file containing a WiT-compatible function prototype as well as a project file for the Microsoft Developer Studio where the actual function code is added and compiled into a DLL.
An alternative method for extending WiT's functionality without writing any C code involves "hierarchical operators" (or "hops" for short). You can add new functions or improve the modularity of complex igraphs by simply encapsulating an igraph into a hierarchical operator. All you need to do is to define the hop's interface (name, inputs, outputs, and parameters) and assign it to an existing operator library. In Figures 5(a) and 5(b), for example, the igraph in Figure 1 appears as a hop called my_zoom. The concept of a hierarchical operator provides many benefits of a subroutine. Hops can be nested to an arbitrary depth. It is even possible to promote a parameter from within a hop and make it a parameter of the hop itself. With my_zoom, for example, the scaling parameters of the built-in zoomSize operator were made hop parameters called width and height; see Figure 5(c).
Technically, hops are little more than a graphical grouping of operators. Normally, WiT's scheduler acts as if all hierarchical operators were expanded to a single top-level igraph. Unlike operators written in C, a hierarchical operator is not initialized each time it is called, but only once when the top-level igraph is started. So hops, unfortunately, cannot solve synchronization problems. Despite this drawback, hierarchical operators are one of the most important features of WiT. Without them, complex software could not be developed.
Because it is easy to define a hop, you will likely have a lot of them soon. Particularly when working in a team where each member develops certain software modules, it is essential to properly organize all the operators (C or hierarchical). Consequently, the WiT Manager lets you create projects containing one or more operator libraries.
A project directory is created by the Manager that contains in different subdirectories the C code, help file, and icon image for each operator. Hierarchical operators are also assigned to a library and hence are part of the project the library belongs to. Consequently, the hop's interface definition is stored in a project subdirectory as well. The WiT Manager, however, does not handle hops, which is why you can store the hop's igraph files wherever you want. In practice, hop files should be kept in their related project directory. When testing a hierarchical operator (my_zoom, for example), you create a top-level igraph (such as test-my_zoom) for executing the hop. These test igraphs should also be stored in the project directory or in a user-created subdirectory therein. Then your project directory is self-contained and can be copied to a computer where all the team member's modules are integrated into the overall solution. Figure 6(a) shows a part of such a project directory, called "Tools." For simplicity, the only library of this project was also called Tools, and Figure 6(b) shows how the hierarchical operators of this little toolchest appear in WiT.
If your task is to integrate several modules with, say, more than 150 hierarchical operators in about 10 levels into a single image-processing solution, then an important limitation of WiT becomes evident. WiT has no overview display -- just single igraph windows -- making it hard to find a hop's place in the overall algorithm or to see the algorithm's complete structure. This results in a number of problems:
To address these problems, I created HopTree -- a Visual Basic application (project files and executable) that recursively scans any igraph file and creates a tree view of the hierarchical operators used (available electronically; see "Resource Center," page 5). Its features include:
Central to HopTree is the tree window, which is an ActiveX element called "TreeView," available in both the Professional and Enterprise editions of Visual Basic 5. TreeView provides all the functionality you know from the directory window of the Windows Explorer, for example. Before the tree view of an igraph can be built, HopTree needs to know which hierarchical operators are defined and where to find their igraph files. The hop's definitions are stored in library-related files within a project directory (<projectpath>\def\<libraryname>.hop), so HopTree has to know the paths of each project. WiT also needs this information, which is why they are stored in a configuration file (*.wrc). Luckily, all relevant files are stored in ASCII format and can hence be parsed easily. In the config file, for example, project paths are preceded by the keyword <projpath, while paths to igraph files are labeled <igrpath (see Listing Two).
After startup, users enter the path and name of the configuration file to be used. It can also be given on the command line. The "Scan this..." button then calls the ScanConfig_Click() procedure, which first builds a list of all the project paths found in the config file, expands environment variables (if any), and displays the full paths in HopTree's tree view window. Now the *.hop files for each project are scanned for the keyword operator (Listing Three), the hierarchical operator's names are stored in a list and displayed. Finally, the igraph paths are read from the config file, stored, and displayed as well; see Figure 7(c). HopTree is now ready to scan an igraph file through the recursive GetOprec procedure. It takes as input the node of the current hierarchical operator in the TreeView as well as its file name. When GetOprec scans an igraph file, it looks for lines beginning with an N character denoting an operator entry. The following two words represent the operator name as well as its instance name (Listing Four), which are displayed in the TreeView as requested. If the operator is listed in the internal hop list, GetOprec checks if it can be found in any of the stored igraph paths. In this case, the procedure calls itself with the parameters of the new hierarchical operator. The recursive procedure returns when the end of the current hop's igraph file is reached, that is, when all deeper-level hops have been processed and the current branch is complete; see Figure 7(a). Listing Four shows parts of the igraph from Figure 3.
DDJ
image = rdObj("saturn");
limit = 10;
image2 = image;
for(i=0; i<limit; i++)
image2 = lopass2D(image2, 9, 9);
image = aluOp(image, image2, "-");
display(image, "hipass");
#
# Generated by WiT Manager
#
<objpath .>
<objpath ${WITHOME}\images>
<objpath ${WITHOME}\images2>
<igrpath .>
<igrpath ${WITHOME}\demo\eg>
<igrpath ${WITHOME}\demo\sub>
<igrpath ${WITHOME}\demo2\sub>
<projpath ${WITHOME}\shared>
...
"Demo"
operator alignAngle
{
derived;
input object "in" (0,20);
output object "out" (60,20);
}
operator alignImage
{
derived;
input object "in" (0,40);
output object "offset" (80,20),
object "angle" (80,40),
object "corrected image" (80,60);
}
...
wit_ig_version_5.2 761 484 0 0 761 484 4 5 0 1.000
N rdObj "read input image" 335 100 n p filename 3 n ;
N alignImage alignImage 435 80 n ;
N overlayData "_ #93" 660 235 n p name 1 s original p useColor 2 ...
N overlayData "_ #92" 660 170 n p name 1 s corrected p useColor 2 ...
N display "_ #84" 660 100 n p name 1 s angle p Placement 2 d 5 p X ...
N display "_ #90" 660 35 n p name 1 s offset p Placement 2 d 5 p X ...
N rdObjSync "Cuts Graphic" 570 290 n p
filename 2 s $WITHOME\\images2\\alignCuts ;
N sequencer sequencer 185 150 n p count 0 d 0 ;
N prompt prompt 305 230 n p Type 2 d 1 p Message 1 s Next ;
N gate gate 335 170 n ;
N if "_if last" 225 220 y p cond 3 n ;
N dirSync Filenames 115 150 n p filename 2 s $WITHOME\\images2\\align?.wit ;
N startSync startSync 45 150 n p type 2 d 1 ;
L "_ #9" 0 0 3 0 2 415 120 435 120 ;
...
L "trigger next image" 0 0 1 0 6 720 190 740 190 740 350 185
350 185 240 225 240 ;
...
G 6 1 1 0 0 0 0 0 0 0 45 60 Verdana r 11 "These operators read
in the sequence of ...