1.1 WHAT IS A NEURAL NETWORK?

 Work on artificial nerual networks, commonly referred to as "neural networks," has been motivated right from its inception by the recognition that the human brain computers in an entirely different way from the conventional digital computer. The brain is a highly complex, nonlinear, and parallel computer(information-processing system). It has the capability to organize its structural constituents, known as neurons, so as to perform certain computations (e.g, pattern recognition, perception, and motor control) many times faster than the fastest digital computer in existence today. Consider, for example, human vision, which is an information-proceessing ttask (Marr, 1982; Levine, 1985; Churchland and Sejnowski, 1992). It is the function of the visual system to provide a representaion of the environment around us and, more important, to supply the information we need to interact with the environment. To be specific, the brain routinely accomplished perceptual recognition tasks(e.g., recongnizing a familiar face embedded in an unfamiliar scene) in approximately 100-200 ms, whereas tasks of mush lesser complexity may task days on a conventional computer.

For another example, consider the sonar of a bat. Sonar is an active echo-location system. In addition to providing information about the relative velocity of the target, the size of the target, the size of various features of the target, and the azimuth and elevation fo the target(Suga, 1990a,b). The complex neural computations needed to extract all this information from the target echo occur within a brain the size of a plum.

Indeed, an echo-locating bat can pursue and capture its target with a facility and success rate that would be the envy of ragdar or sonar engineer.

How, then, does a human brain or the brain of a bat do it? At birth, a brain has great structure and the ability to build up its own rules through what we usually reffer to as "experience." Indeed, experience is built up over time, with the most dramatic development(i.e., hard-wiring) of the human brain taking place during the first two years from birth; but the development containues well beyyond that stage.

A "Developing" neuraon is synonymous with a plastic brain: Plasticityy permits the developing nervous system to adapt to its surrounding environment. Just as plasticity appears to be essential to the functionning of neurons as information-processing units in the human brain, so it is with neural networks made up of artificial neurons. In its most general form, a neural network is machine that is designed to model the way in which the brain performs aparticular task or function of interest; the network is usually implemented by using electronic componecnts or is simulated in software on a digital computer. Our interest in this book is confined largely to an important class of neural networks that perfom useful computations through a process of learning. To achieve good performance, neural networks employ a massive interconnection of simple computing cells referred to as "neuraons" or "processing units." We may thus offer the following definition of a neural network viewed as an adaptive machine;

A neural network is a massively parallel distributed processor made up of simple processing units, which has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects:

1. Knowledge is acquired by the network from its environment through a learning process.

2. Interneuron connection strengths, known as synaptic weights, are used to store the acquired knowledge.

The procedure used to perform the learning process is called a learning algorithm, the function of which is to modify the synaptic weights of the network in an orderly fashion to attain a desired design objective.

The modification of synaptic weights provides the traditional method for the design of neural networks. Such an approach is the closest to linear adaptive f ilter theory, which is already well established and successfullyu applied in many diverse fields(Widrow and Stearns, 1985;Haykin, 1996). However, it is also possible for a neural networks to modify its own topology, which is motivated by   the fact that neurons in the human brain can die and that new synaptic connections can grow.

Neural networks are also referred to in literature as neurocomputers, connectionist networks, paralled distributed processors, etc. Throughtout the book we use the term "neural networks"; occasionally the term "neurocomputer" or "connectjionnist network" is used.