R. Colin Johnson is Advanced Technology Editor for EE Times, author of the book Cognizers -- Neural Networks and Machines That Think (John Wiley & Sons, 1988), editor of the monthly newsletter Cognizer Report, and outhor of the Cognizer Almanac, a yearly report. Colin can be contacted at 10075 Barbur Blvd., #405, Portland, OR 97219.
Cognitive computing denotes an emerging family of problem-solving methods that mimic the intelligence found in nature. These methods draw from diverse areas of scientific research and embrace various yet-to-be-proven theories about naturally occuring phenomena. The common goal of these methods is to crack tough problems that have resisted straightforward analytic solutions, such as intractable problems caused by combinatorial explosions.
Recognition problems of all sorts--handwriting, speech, object--involve the real-time processing of many related inputs. Just tracking objects from one video frame to the next involves thousands of computations performed on thousands of picture elements (pixels). Control problems of all sorts also involve juggling many simultaneous constraints. Robotics controllers present particularly thorny problems, since they must compute hundreds of transcendental functions just to lift a finger accurately. Forecasting and other statistical analysis methods also involve hundreds or thousands of simultaneous equations. For instance, using an exhaustive search to find the optimal route for a traveling salesman visiting a dozen cities takes more computer time than it is worth. How is it that naturally intelligent entities routinely solve such massively combinatorial problems without straining? Even the little honeybee can plot complex routes that would challenge a supercomputer to match. By copying the way that nature solves these problems, cognitive computing instills that intelligence into your algorithms. For many programming problems, cognitive computing can result in smaller, faster-running programs that don't take as long to write and debug.
Natural intelligence abounds almost everywhere you lookin nature. Neural networks extract principles from brain science to model recognition, learning, and planning processes. Fuzzy logic more closely resembles the way humans reason with approximate rules-of-thumb than does traditional logic, according to its inventor, electronic engineer Lotfi Zadeh. Genetic algorithms draw directly from insights gathered in genetics research--modeling Darwin's principles of natural selection. It seems anywhere nature exhibits intelligent solutions, some programmer somewhere is copying them. Fractals, for instance, vaguely resemble the manlier in which living systems repeat patterns while growing. Biologist Aristid Lindenmayer invented a specialized variation--called "L-systems" after their author--which mimic the fractal-like mechanisms of plant growth. Chaos theory uses mechanisms derived from mathematical simulations of temporally repeating processes that are never exactly the same twice--from dripping faucets to the weather. Cellular auto-mata model a two-dimensional universe of "living" cells and a simple set of ways they may interact. And the list goes on. Most of these methods mimic solutions found in nature by creating a software simulation of that process and plugging the parameters of real-world problems into it. The accuracy with which the simulation models nature is usually secondary to solving those real-world problems. Accordingly, there are a wide variety of development tools to help build such simulations; see Table 1. Experienced programmers sometimes begin with a specialized development tool with the option of generating source code in a traditional language, say C. After studying the code for a while, they catch on to encoding neural learning methods, fuzzy logic, or genetic algorithms directly into their own programs.
Cognitive computing substitutes more-intelligent processing techniques for the long, complex, and often brittle programming techniques with which we are all familiar. Models based on cognitive computing can often be as accurate as closed-form analytic equations, but apply to a wider range of situations since they are not limited, for instance, to being linear. However, one disadvantage of cognitive computing techniques is that they often cannot be proven stable by anything other than extensive field testing. Traditional analytic proofs of stability are impossible to obtain for an approach that does not use an analytic methodology.
Instead of an ordinary top-down, divide-and-conquer approach, cognitive computing techniques seemingly jump to conclusions without going through all the intermediate steps. The common element here is the notion of intelligent interpolation. All three core cognitive computing technologies--neural-, fuzzy- and genetic-based--derive their generality from interpolating the solutions to problems with which they have not previously been faced from the solutions to ones with which they are familiar. The intelligence with which they perform this interpolation--sometimes called "generalization"--is key to their success.
To be sure, if a reasonable number of closed-form equations can accurately solve a problem, then there is no need to resort to cognitive computing techniques. But if not, then cognitive computing offers more economical alternatives to brute-force approaches like exhaustive searches through the space of all possible solutions. The solution space for problems that resist closed-form expression is usually too large to search in a reasonable amount of time. But cognitive computing techniques sidestep that problem by converging on an optional solution from relatively few working examples. ("Near optimal" may be more appropriate here, since again, these solution cannot be proven to be optimal for the same reason they can't be proven stable--no analytic method is available.)
Intelligent interpolation in a neural network, for instance, employs all neurons simultaneously, each comparing its results to its neighbors in order to combine them into an intelligent solution by consensus. A fuzzy system performs a similar operation by evaluating all applicable rules in parallel and intelligently combining their results. Likewise, genetic algorithms create increasingly better solutions by mutating and splicing together the best-so-far solutions until an optimum is reached.
When simulated on a standard computer, of course, you have to perform supposedly parallel operations sequentially and keep track of intermediate results. However, while sequential simulations of parallel processes may squander hardware resources according to those building parallel analog microchips, their operation appears to be no less effective than truly parallel hardware (as long as your computer and/or accelerator board is fast enough for a given application).
Deployed applications of cognitive computing abound--from signal processing to pattern recognition, feature extraction, industrial inspection, business forecasting, credit rating, securities picking, medical diagnosis, speech processing, natural-language understanding, constraint satisfaction, robotics control, and adaptive process control. The single most widely used technology from cognitive computing is the adaptive filter, first named the "adaptive neuron" by its inventor, electronic engineer Bernard Widrow. The adaptive filter changes its characteristics in response to changing noise patterns in the signals it receives. All modern high-speed communications rely on such adaptive filters to cancel echo. [Editor's note: For more information on adaptive filtering, see "Finding Significance in Noisy Data," by Roy E. Kimbrell, DDJ, June 1992.]
Pattern recognition is at the heart of many neural-network applications. The neural network learns the feature set from historical data. Such classification systems work best when there is a large database of typical examples from which to learn. Software simulations of neural networks have learned to recognize everything from the underwater calls of whales to the radio-waves of distant galaxies. For instance, Nestor (Providence, Rhode Island), the oldest public neural-network company, got the banking industry started with neural-based systems back in 1987. Now the whole banking industry uses neural networks, according to BancTec (Dallas, Texas), to read the numbers on bank checks and credit-card charge slips. Just last year (1992), Synaptics (San Jose, California) announced the first microchip-based neural network to read the numbers on checks and credit-card slips. Its I-1000 electronic retina is a parallel analog implementation that emulates the neural networks of the eye-brain system (marketed to banks by VeriFone, Redwood City, California). General-purpose neural and fuzzy microchips are already available from Adaptive Solutions (Beaverton, Oregon), American NeuraLogix (Sanford, Florida), Inform (Aachen, Germany), Intel (Santa Clara, California), Omron (Schaumburg, Illinois), and Togai InfraLogic (Irvine, California), with Motorola (Austin, Texas), NEC (Kawasaki, Japan), National Semiconductor (Santa Clara, California), and Ricoh (Tokyo, Japan) planning or releasing microchips in 1993.
Adaptive Solutions, Inform, NeuraLogix, and Omron wrote their own software for programming their respective chips, but Intel has turned to independent developers--BrainMaker from California Scientific Software (Nevada City, California) and DynaMind from NeuroDynamX (Boulder, Colorado). Intel is also working with Nestor on the development of an all-digital neural microchip using a second-generation neural model based on radial-basis functions (to be announced in 1993). The most successful neural-network startup company--HNC (San Diego, California)--has had enough success with its pattern-recognition capabilities for optical character recognition (OCR) to spin off that operation to Mitek Systems (San Diego, California) last year. Neural approaches to OCR have been successful enough at tough recognition tasks to attract even the mainstream vendors. The first was Calera Recognition Systems (Sunnyvale, California). The company has kept its use of neural technology quiet, but last year (1992) it came out of the closet with its FaxGrabber product for automatically converting incoming faxes to text. Calera's major competitor in OCR, Caere (Los Gatos, California), announced it was using neural networks with its AnyFax technology, prompting Calera's admission that it had already been using neural networks.
AbTech 804-977-0686 AIMQAutomated Intelligent Modeler Machine-learning software using proprietary polynomial network technology. $1495 Adaptive Solutions 503-690-1236 BuildNet A comprehensive X-Window-Motif environment for conducting neural-network experiments and developing finished applications. $5000 CNAPS-1064 General-purpose, all-digital microchips with 64 processing nodes running all popular neural-network algorithms can be programmed into this device. CodeNet Consists of software-development tools, including an assembler, a graphical-interface builder, a debugger, and libraries of common neural-network algorithms for CNAPS architecture. $15,000 CNAPS-C C-language compiler extended for parallel processing and fixed-point math targeted at the CNAPS architecture. $3000 AI Ware 216-421-2380 N-NET EX Neural Network Development System Neural-net package for developing applications based on AI Ware's patented functional-link architecture. $1995 N-NET 500 Series Neural Net Development System Same as above, for Sun Microsystems workstations. $2995 American NeuraLogix 407-322-5608 Fuzzy Pattern Comparator, NLX110 Fuzzy-data comparator using 32-bit precision with automatic configuration circuitry. Neural Processing Slice, NLX420 800 MIPS neural-slice architecture running at 300 million connections per second (CPS). FPC Applications System, ADS110 Fuzzy-pattern comparator applications system based on FDC device on PC card with menu-driven software. $395 NPS Development System, NPS application system with PC card containing none or four NPS devices. $595 (ADS420-1), $1495 (ADS420-4) Fuzzy MicroController, NLX230 Eight-input fuzzy microcontroller with on-chip storage for 64 fuzzy rules sharing 16 membership functions. FMC Development System, ADS230 Fuzzy-logic controller development system based around FMC devices on PC card with menu-driven software. $395 Applied Intelligent Systems 313-995-2035 NetRead Software using neural-network learning method for reading semiconductor-wafer codes. $5000 Aptronix (Marketed by Motorola) 408-428-1888, 408-428-1883 FuzzyNet BBS FIDE, Fuzzy Inference Development Environment Software-development environment for embedding fuzzy-logic-based solutions in applications. $1495 Axcelis 206-624-2446 Evolver Genetic-learning add-on for Wingz spreadsheet program. $345 California Scientific Software 916-478-9040 BrainMaker High-speed software simulation of back-propagation-of-errors learning method. Does not use floating-point operations. $195 BrainMaker Professional Expanded version of the BrainMaker software simulator with hooks for calling a trained network from another program. $795 NT5000 Neural Control System BrainMaker real-time data collecion and process-control interface. $7950 Intel 80170NX Neural Network Development System Integrated-circuit neural-network control systems, with complete development environment allowing BrainMaker neural net to be created and trained on user's PC, then downloaded to a chip. 80170NX: $940 each; iNNTS: $11,800 Epic Systems Group 818-355-2988 Neuralyst A neural-network add-on for Microsoft's Excel spreadsheet. $165 (PC or Macintosh versions) Run-time Library Runtime library for the Macintosh or the PC. $495 HyperLogic 619-746-2765 CubiCalc User-programmable fuzzy-logic expert-system shell for Microsoft Windows; optional runtime compiler and library. $495 up CubiCalc RTC CubiCalc with runtime compiler and C-language object library allows programs to incorporate fuzzy logic into applications. $795 CubiCard Hardware input-output board for connecting fuzzy spreadsheet to real-time data sources. $1495 The OWL Neural Network Library C library of 20 neural-network simulations for PC with Microsoft or Turbo C or Macintosh with Think C. $499 up Inductive Solutions 212-945-0630 NNetSheet 1.2 Neural-network algorithms implemented as spreadsheet formulas. The runtime system is the spreadsheet. $495 Induce-It A case-based reasoning tool for building expert systems. $995 NNetsheet-C A fast C implementation of many popular neural-network learning algorithms. $895 GenSheet Uses Microsoft Excel as the user interface in implementing genetic algorithms as fast C-coded dynamic link libraries for IBM-compatible computers, and as code resources for Macintosh computers. $895 Inform +49 2408-6094 Fuzzy 166 chip Fuzzy microchip built for Inform by Siemens AG (Munich, Germany). $100 FuzzyTECH Fuzzy-logic development tools, including a graphical input system, precompiler, compiler, and online development tool. Price ranges from $1600 stand-alone to $9200 fully configured. FuzzyTECH Shell Rule editor addition to fuzzyTECH that allows engineers to graphically adjust the values of running fuzzy systems. $1900 FuzzyTECH NeuroFuzzy Module Adds the capability of permitting a neural network or other learning method such as a genetic algorithm. Intel 408-765-9235 80170NX Neural-network chip using an all-analog circuitry plus EEPROM for synapses. $940 iNNTS, Intel's Neural Network Training System Development system including two 80170s, interconnection board, and all software. $11,800 EMB Board for users wishing to quickly prototype multichip setups. The multichip board can house up to eight 80170s (comes with two). $9750 Mind's Eye 314-921-8433 GA-lib C source code genetic-learning library. Nestor 401-331-9640 Nestor Development System (NDS 1000) Neural-network development system for DOS-based PCs and Sun Microsystems workstations. NeuralWare 412-787-8222 NeuralWorks Professional II-Sun (Sun-3, Sun-4, Sun-386i) Neural-network development system for professional engineers and researchers. $2995 ($1495 PC, $4995 Silicon Graphics) Explorer-PC Introductory program for learning to use neural networks. $299 NeuralWorks Designer Pack Converts networks developed under Professional II into C source code. (IBM PC or Sun)) NeuralWorks User Defined Neurodynamics Allows customers to write their own learning rules and functions for Professional II. $995 (IBM PC), $1999 (Sun) NeuroDynamX 303-442-3539, 800-747-3531 DynaMind A diagnostic tool for creating, training, and implementing neural networks. $145 DynaMind Developer Bundles DynaMind with NeuroLink, a library of C-code routines for embedding runtime neural networks. $495 iDynaMind Software trains networks on the Intel 80170NX ETANN. Norrad 603-434-0047 NET-Link+ Genetic-learning Bridge Bridges gap between Nexpert Object, from Neuron Data, with several other packages, including: NeuralWare's neural-network simulator, NeuralWorks; AIM, the abductive-reasoning tool from AbTech Inc.; and TILShell, the fuzzy-logic development system from Togai InfraLogic, of Irvine, California, GA-Lib from Mind's Eye. $295--$595 (depending on configuration) Omron 708-843-7900 FP 3000 Digital Fuzzy Processor Fuzzy-logic microchip designed for control applications. FS 10AT Digital Inference Software Fuzzy-logic development software using a fill-in-the-blank method to generate FP 3000 object code. Promised Land Technologies 203-562-7335 Braincel Add-on neural network for the Excel spreadsheet. $249 Synaptics 408-434-0110 I-1000 Neural OCR microchip Analog neural-network used in the VeriFone Onyx check reader. Performs sensing and recognition of E13B at 1000 CPS. Ward Systems Group 301-662-7950 NeuroShell A production neural-network shell program designed to function in the domain of expert systems. $195 NeuroSheet A NeuroShell option that imports spreadsheets, learns from them, and exports results back to spreadsheets. $99 Database Option for NeuroShell Option for NeuroShell that processes dBase III files directly without conversion. $69 NeuroWindows Attaches neural capabilities to programs written in Microsoft's Visual Basic, C, or other languages. $369 Togai InfraLogic 714-975-8522 Fuzzy C Development System Toolkit for developing fuzzy-logic expert systems. Generates C source code. $2490-$3900 MicroFPL Development System Software toolkit for developing fuzzy-logic expert utilizing a runtime kernel for the target CPU. $2000-$17,000 TILShell A computer-aided software engineering (CASE) tool for building fuzzy expert systems. $2300-$3300 TILGen Automated rule-base generation tool. Uses neural-network learning to analyze inputs and generate its output. $975-$1100 FC110 Development System Converts fuzzy-logic knowledge base to FC110 DFP-based products. $750-$900
Calera's OCR engine learned from a huge training set resulting in over a million different recognized variations of the standard alphanumeric characters. Since its first product shipped in 1986, it has continually updated its neural technology and is currently relying on an advanced learning model derived from radial-basis functions. Caere won't reveal the exact neural methodology used with its AnyFax technology, but did admit that it took over 100,000 examples of various types of fax documents from which to learn. Audre Recognition Systems (San Diego, California) does not confine its OCR to standard alphanumerics, but teaches its neural networks to read the specialized symbols on engineering drawings, too. The company uses a learning method that is a proprietary modification of the popular back-propagation-of-errors approach. After learning a specific set of symbols--say those used to build airplanes--the system will automatically convert whole drawings to a standard computer-aided design (CAD) description language and create a custom Postscript laser font for its special symbols.
Neural networks are also helping recognize patterns on the shop floor. For instance, Applied Intelligent Systems (Ann Arbor, Michigan) built neural-recognition capabilities into its vision computers for quality control in factories. Other companies are also adding neural-recognition capabilities to their industrial inspection systems. For instance, AI Ware (Cleveland, Ohio) has enhanced traditional infrared spectroscopy verification systems with neural networks for its customers. Ordinarily skilled experts read the plots produced by spectroscopy, but AI Ware has trained a neural network to automatically validate product quality on-the-fly. The system trains on known good and bad parts after which it can spot the difference without human intervention.
The National Science Foundation (Washington, D.C.) sponsored one of the more unusual pattern-recognition applications, developed using BrainMaker. This system uses a non-destructive testing method that can assess damage to bridges and other steel and concrete structures, such as those found on highways, without damaging them. The NSF neural network learns from the reflected sound waves transmitted through concrete. Called the "impact-echo method," hardened steel spheres drop on the surface of the concrete. The impact propagates a wave through the material that reflects off cracks and voids in the structure. A transducer measures displacement at the surface caused by the reflected waves, and the neural network compares them with known solid-structure reflections which it has already learned.
Neural networks recognize many other patterns, too. For instance, HNC's software is being used to classify EKG signals in the presence of noise, as well as to detect suspect pap smears at Neuromedical Systems (Suffern, New York).
Fuzzy logic is seldom used for pattern recognition, but for process-control applications it is being adopted by every major industrial controller vendor. Startups are also popping up that combine cognitive computing techniques. For instance, Pavilion Technologies (Austin, Texas)--a spin-off of the Microelectronics and Computer Consortium (MCC, Austin, Texas)--has created adaptive software for optimization and control in the materials-processing industry. Called Process Insight, it combines neural networks, fuzzy logic, and chaos theory to optimize processes and adjust their parameters for minimal waste and highest yields. AI Ware also serves the process-control industry with a neural-based technology that optimizes cost and product performance while minimizing trial-and-error experimentation. Its machine-health diagnostic system takes real-time measurements from machines at a plant and correlates them with product-quality measurements. After learning these correlations, the system can predict eminent failures and cue an embedded expert system to schedule machine maintenance.
AI Ware also has a custom formulation system that optimizes processes for producers of rubber, plastics, chemicals, metals, and products based on these materials. After learning from in-house formulations, the system becomes an automated expert chemist. Just specify a new formulation and the system automatically determines the optimal set of ingredients and cookbook-style processing steps. The ability of cognitive computing techniques to deal with unforeseen circumstances makes them amenable to all sorts of forecasting applications. The financial industry was one of the earliest converts. NeuralWare (Pittsburgh, Pennsylvania) has its NeuralWorks software installed throughout the financial industry, as do many other vendors. Besides just predicting which securities will rise or fall in value, financial forecasting also includes automated mortgage- and credit-approval systems, demand and sales forecasting, and many industry-specific forecasting applications. The diversity of these forecasting needs prompted HNC to create its DataBase Mining Workstation (DMW). The DMW learns interdependent relationships and logical sequences in any database, thereafter forecasting future contingencies and answering "what-if" queries.
HNC has also crafted several vertical applications packages with the DMW. For instance: SkuPLAN is a demand forecasting system that predicts sales for chainwide stores; Automated Real Estate Appraisal System (AREAS) forecasts property valuation based on multidimensional modeling; Automated Mortgage Processing Systems (AMPS) forecasts good and bad risks for underwriters. Fuzzy logic is also entering the financial markets. In 1991, "fuzzy" was the "word of the year" in Japan followed by a flood of consumer items using it--from cameras and video recorders to washing machines a vacuum cleaners. Now fuzzy logic has penetrated the Japanese business arena, too, with several portfolios being successfully managed by fuzzy- and neural-based systems.
In the U.S., every major consumer electronics maker and industrial controller makes is integrating fuzzy logic into their wares. Business users are also building fuzzy logic into their software base, but few are sharing their secret. One company, FuziWare (Knoxville, Tennessee), is repackaging some of the software it has written under contract for specific business users. Its FuziQuote, for instance, is a bid-automation package that provides quick turnaround quotes for complex jobs. Its FuziQuery aims at accurately finding sales prospects. Its FuziCost is a modeling tool that accepts intuitive knowledge about cost, profit, and strategic performance to provide what-if scenarios. And its FuziCell is an area-management system for flexible factory managers.
HyperLogic (Escondido, California) has created a fuzzy-Logic expert-system shell it calls CubiCalc. Like a spreadsheet, you configure CubiCalc for anything from business users to industrial control. Other companies are offering add-ons for spreadsheets that incorporate cognitive computing techniques. For instance, Inductive Solutions (New York, New York) offers spreadsheet add-ons for genetic algorithms (GenSheet), neural networks (NetSheet), and case-based reasoning (Inducelt). Axcelis (Seattle, Washington) adds genetic algorithms to spreadsheet with its Evolver. Epic Systems Group (Pasadena, California) adds neural capabilities to spreadsheets with its Neuralyst. Promised Land Technologies (New Haven, Connecticut) adds neural capabilities to spreadsheets with its Braincel. Ward Systems Group (Frederick, Maryland) can read spreadsheets, perform neural-based learning with them, and return the learned values transparently with NeuroSheet bridge to its expert-system-like NeuroShell.
For a cheap neural-network subroutine library, try a somewhat oudated book called Parallel Distributed Processing (MIT Press, 1986) which comes in a version with a free disk. For an even cheaper fuzzy-logic library, try Motorola's free one (download from 512-891-3733). The only genetic-algorithm subroutine library of which I am aware is GA-lib from Mind's Eye (Florrisant, Missouri). If you want to glue together some of these technologies in a hurry, you might check out NET-Link+ from Norrad (Nashua, New Hampshire). NET-Link+ hacks together the expert-system building environment Nexpert Object, from Neuron Data (Palo Alto, California), with several other packages including NeuralWorks, GA-lib, AIM--the abductive-reasoning tool from AbTech, and the TILShell, a fuzzy-logic development system from Togai InfraLogic.
Copyright © 1993, Dr. Dobb's Journal