Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.


  1. Advanced Patent Search
Publication numberCA2067217 C
Publication typeGrant
Application numberCA 2067217
PCT numberPCT/US1990/005868
Publication date23 Feb 1999
Filing date10 Oct 1990
Priority date10 Oct 1989
Also published asCA2067217A1, DE69033681D1, DE69033681T2, EP0495901A1, EP0495901A4, EP0495901B1, US5136687, WO1991006055A1
Publication numberCA 2067217, CA 2067217 C, CA 2067217C, CA-C-2067217, CA2067217 C, CA2067217C, PCT/1990/5868, PCT/US/1990/005868, PCT/US/1990/05868, PCT/US/90/005868, PCT/US/90/05868, PCT/US1990/005868, PCT/US1990/05868, PCT/US1990005868, PCT/US199005868, PCT/US90/005868, PCT/US90/05868, PCT/US90005868, PCT/US9005868
InventorsGerald M. Edelman, George N. Reeke, Jr.
ApplicantGerald M. Edelman, George N. Reeke, Jr., Neurosciences Research Foundation, Inc.
Export CitationBiBTeX, EndNote, RefMan
External Links: CIPO, Espacenet
Categorization automata employing neuronal group selection with reentry
CA 2067217 C
An apparatus capable of sensing the presence of objects in its environment, categorizing these objects without a prior description of the categories to be expected, and controlling robotic effector mechanisms to respond differentially to such objects according to their categories. Such responses include sorting objects, rejecting objects of certain types, and detecting, novel or deviant objects. The invention includes a device called a "classification n-tuple" (of which a "classification couple" is a special case) capable of combining signals from two or more sensory modalities to arrive at the classification of an object.
1. An apparatus for categorizing objects in a physical environment according to sensory input data relating to those objects and for sorting the objects in accord with such categories comprising one or more sensory means for sensing input signals, each of said sensory means identified with a specific sense function, processing means for receiving said input signals, for categorizing objects according to said input signals and for generating output signals in response to said input signals, output effector means for receiving said input signals and for sorting said objects in response to said output signals, each of said output effector means identified with a specific motor output function, said processing means comprising a plurality of cells, each of said cells characterized by a state of activation determined by a response function, a plurality of synapses, each of said synapses comprising a unidirectional connection between one of said cells and one of said sensory means, output effector means or another of said cells and each of said synapses having a strength capable of differential modification determined by a selective learning rule, a plurality of groups of cells, each of said groups comprising a collection of cells connected more strongly among themselves than they are connected to cells in other groups, a plurality-of sensory repertoires each corresponding to one of said sense functions and each comprising collections of said groups, interconnected by mappings comprising synaptic connections, a plurality of motor repertoires each corresponding to one of said motor output functions and each comprising collections of said neuronal groups interconnected by mappings composed of synaptic connections, a plurality of value repertoires, each connected to one or more of said sensory repertoires or to other cells and capable of responding differentially to changes in the environment signalled by said input signals caused by the actions of the said output motor function, and each comprising collections of said neuronal groups interconnected by mappings composed of synaptic connections, wherein said groups of cells comprise one or more primary repertoires of variant, overlapping response selectivities prior to selection by heterosynaptic input from said value repertoires, and comprise after selection secondary repertoires of such selectivities adapted to perform a particular categorization task and to perform particular output actions upon the categorization of certain types of objects, a plurality of processing repertoires, each connected to one or more of said sensory, motor and value repertoires by synaptic connection to form mappings, a plurality of reentrant signalling means between said sensory repertoires wherein during operation of said apparatus each sensory repertoire receives signals derived from at least one of said sensory means and outputs signals to at least one of said output effector means and the modifications of said synaptic strengths alters the contributions of one or more neuronal groups to behaviour providing integrated sensory and motor behaviour, said sensory repertoires connected by reentrant signalling comprising classification n-tuples, wherein the apparatus is adapted to carry out categorization of the objects.
2. The apparatus of claim 1 for sorting the said objects in accord with categories established from characteristics of input data and wherein said repertoires comprise vision system means, reaching system means, touch system means, reentrant categorization system means, and response system means.
3. The apparatus of claim 2 for establishing categories of objects and sorting the objects in accord with such categories wherein said vision system comprises a scanning visual input device, and a foveation and fine-tracking oculomotor system, said reaching system comprises a multi-joined arm having a set of movement means and neuronal repertoires subserving the control of said arm causing it to reach out to such objects in order to trace or grasp them for sorting, and said touch system comprises a tactile system means using a second set of movement means in said arm.
4. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein each of said repertoires comprises cells having connections selected from among the following connections, connections chosen by a specific rule, and individually enumerated, such as connections forming a topographic mapping, connections having a specified density-distance relationship, in which all cells in any group lying in a square band at a certain distance from a given target cell are connected with a given equal weight to said target cell, and connections receiving input corresponding to the average activity of all cells in a specified source layer.
5. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein each of said synapses has efficacies capable of differential modification dependant upon the state of synapses on the same cell.
6. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein each of said synapses has efficacies capable of differential modification dependent upon the strength of a reentrant response.
7. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein each of said synapses has efficacies capable of differential modification for the selection of connections receiving temporally correlated inputs.
8. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein each of said synapses has efficacies capable of differential modification that includes a rule selector factor to generate value-dependent synaptic modifications for different connections.
9. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein said repertoires are connected by pathways of signals and said reentrant signalling means comprises backwards connections from a repertoire to prior repertoires in one of said pathways.
10. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein said repertoires are connected by pathways of signals and said reentrant signalling means comprises parallel connections between repertoires in different pathways.
11. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein said reentrant signalling means comprises reciprocal connections each exchanging cell activity signals in one direction between two repertoires.
12. The apparatus of claim 11 for establishing categories of objects and sorting the objects in accord with such categories wherein repertoires are connected by pathways of signals and said reentrant signalling means comprises reciprocal connections between two repertoires in different sensory pathways.
13. The apparatus of claim 1 for establishing categories of objects and sorting the objects in accord with such categories wherein said value repertoires include sensory afferents or afferents from other parts of the nervous system, both topographic and non-topographic mappings and efferents that heterosynaptically influence large populations of synapses.
14. An apparatus for establishing categories of shape and patterning of physical objects and sorting the physical objects in accord with such categories comprising optical sensor means to visually sense said objects and generate input signals in response thereto, tactile sensor means to sense said objects by touch, said means being installed on a jointed arm capable of reaching out to bring said tactile means into contact with said objects, and generating tactile signals in response thereto, kinesthetic sensor means to sense the angular positions and motions of joints in the said jointed arm and generating kinesthetic signals in response thereto, processing means for receiving input data, for categorizing said input data and for generating output data in response to said input data, output means being adapted to receive said output data and to manipulate said objects in response to said output data, said processing means comprising a plurality of processing elements and memory registers configured in such a way as to constitute a plurality of synapses, each of said synapses having efficacies capable of differential modification of the strength of connections between pairs of said processing elements, said efficacies determined by an amplification function, a plurality of groups of neurons, each of said neuronal groups comprising a repertoire of neurons and including said neuron's associated axonal and dendritic aborization patterns a value repertoire adapted to increase a value parameter when the optical sensory means moves towards regions having predetermined optical characteristics and fixates upon them, whereby said repertoire provides heterosynaptic input to synapses thereby modulating the modification of connections from an SC repertoire having excitatory cells connected to ocular motor neurons OM, a VR visual repertoire of said neuronal groups containing excitatory and inhibitory layers of neurons, for the purpose of forming a neuronal mapping of visual signals produced by the said optical sensor means an SC repertoire having excitatory cells connected to ocular motor neurons OM adapted to cause motion of said optical means, an NC repertoire adapted to cause gestural motions by said arm spontaneously or from input from vision and arm kinesthesia, an IN intermediate repertoire adapted to pass signals from said MC repertoire to an SG repertoire which controls said arm, and being adapted to block outputs from MC which do not lead to desired motions of the arm upon receiving inhibitory signals, a GR repertoire adapted to correlate configurations of the arm in space with target positions, a PK repertoire connected to said GR secondary repertoire, adapted to cause inhibition of incorrect gestures of the arm by sending said inhibitory signals to the IN
repertoire an SG repertoire connected to said IN secondary repertoire, adapted to integrate control signals from the reaching and tracing subsystems, producing coordinated motions of the said arm in response to signals from either source, a plurality of reentrant signalling means between said repertoires, said VR repertoire being mapped to SC, said SC repertoire being mapped to OM, said IN intermediate secondary repertoire adapted to send signals to said SG repertoire, and to receive sensory input and primary gesture signalling from MC, said GR secondary repertoire connected to said PK
secondary repertoire, via repertoire IO wherein the position of the arm and the physical object are associated with signals corresponding to primary gestures that arise from MC.
15. The apparatus of claim 14 for establishing categories of shape and patterning of physical objects and sorting the physical objects in accord with such categories further comprising an LGN (lateral geniculate nucleus) repertoire having ON and OFF type neurons, said LGN ON neurons being adapted to respond strongly only to spots of light surrounded by a relatively dark area, and said LGN OFF neurons being adapted to respond strongly only to spots of darkness surrounded by a relatively light area, an R repertoire containing groups of neurons having a field of view and adapted to respond to vertical, horizontal, or oblique line segments, or to line segments ending within the field of view of one of said neurons, or to line segments which change direction within the field of view of one of said neurons an R2 repertoire connected to cells in common with cells in R, wherein response signals in R2 represent combinations of elementary visual features detected by R, an RM secondary repertoire having inputs from kinesthetic receptors in the touch-exploration motor system, wherein RM responds to two textures, smooth and rough, an ET secondary repertoire having inputs from R2 and from RM and adapted to enhance activity in response to combinations of visual and tactile sensory signals as represented in R2 and RM, said combinations corresponding to various categories of objects to which the system may, from time to time, have been trained to respond, a triggering network adapted to end tracing by detecting novelty (and its absence) in the RM responses and integrating the appearance of novelty over time to recognize the completion of a trace and producing a response upon the termination of said appearance of novelty comprising two layers of neurons stimulated by signals indicative of rough or smooth units by R M but having refractory periods that prevent the resumption of activity after such activity has become depressed until a time interval long enough for a motor response to occur after stimulation, a third layer of neurons, stimulated by either of the first two or both, and a fourth layer of neurons, having inhibitory connections with the first, having a high level of varying activity, said triggering network being coupled to R2 and R M, to re-excite groups previously stimulated during examination of the stimulus causing said response signals, wherein activation of R2 and R M by neural events occurring independently in the two repertoires constitutes reentry and brings about categorization, and wherein physical objects of a particular class are sorted by virtue of the generation of a response in the arm upon triggering only when it is the case that responses accumulated in the R2 and R M repertoires during the period of visual and tactile examination corresponding to a particular category previously established by selective amplification of synaptic connections between R2 and R M on the one hand and repertoire ET on the other.
16. An automaton to analyze critical problems involving the acquisition and change with time of integrated sensory and motor behaviour comprising an input array on which two-dimensional patterns or visual scenes are represented, an assembly of repertoires of differential responding elements interconnected by mappings comprising synaptic connections that transform input patterns, an arrangement for coupling these networks to specified motor-output functions, means for detection of motion, comprising a connected assembly of cells, multiple inputs from the input array or directly from other sensory means, or from the outputs of groups of cells in the same or different repertoires, a single time-dependent scalar variable, which characterizes the state of each cell, and which is dependent upon the strengths of the inputs to that cell, each input multiplied by a synaptic strength, means for enabling selection in both sensory and motor control portions, wherein (a) mutual training is achieved of both sensory and motor control portions through encounter with an environment and (b) signal combinations are automatically selected during said training, an amplification rule to alter the synaptic strength, of a connection according to the activity of pre- and postsynaptic groups, said rule providing for the weakening of connections between pairs of units of which one, but not both, are active, said rule providing for the strengthening of connections between pairs of units of which both, or neither, are active, said rule providing for modulation of the amount of synaptic change according to a heterosynaptic input which signals the success or failure of recent behavioral activity of the apparatus by increasing or decreasing the strength of the synapses as determined by a value repertoire.
17. The invention of claim 16 further comprising value repertoires, which through reentry favour the learning of activities of value, said value repertoires further comprising connectivities which predispose their constituent groups to respond to the sequelae of adaptive behaviors, sensory afferents, both topographic and non-topographic mappings, and efferents that heterosynaptically influence large populations of synapses.
18. An apparatus according to claim 16 comprising a processing means comprising neurons, each of which is characterized by a state of activation determined by a response function, and synapses, each of which has a unidirectional connection between two of the neurons, each of the synapses having an efficacy capable of differential modification of the strength of said synapses determined by a response function according to a selective learning rule.
19. An apparatus according to claim 16 having groups of neurons connected more strongly among themselves than they are connected to neurons in other groups, and neural maps, corresponding to one of the sense functions or one of said motor output functions, and comprising repertoires of neuronal groups, the input connections to which are so arranged that a correspondence exists between either locations in space or other properties sensed by the sensory means on the one hand, and locations in each of said neural repertoires, such that responses to different objects or to the same object in different locations tend to occur in different locations in each of said neural maps.
20. An apparatus according to claim 16 in which the modification of synaptic efficacies of cells alters the contribution of selected neuronal groups to behavior, thereby providing integrated sensory and motor behavior.
Description  (OCR text may contain errors)

, , , ~ 91/06055'. . PC~r/US90/05868 1 Categorization Automata Employing 2 Neuronal GrouP Selection With ~eentrY
i 3 Background of The Invention 4 This invention relates to neuronal network simulation and in particular to the 6 use of simulated neuronal networks in computerized 7 apparatus called "automata" for performing basic 8 intellectual and physical tasks. Such tasks may 9 include as an example the recognizing, discriminating, lo and sorting of various objects or inputs.
ll The simulated networks and the 12 automata of the present invention are distinguished by 13 their ability to learn as opposed to the mere training 14 of sensorimotor components. During use they develop or improve the criteria by which they recognize and 16 discriminate between input signals. They are capable 17 of categorization, association, and generalization and 18 capable of adaptive behavior based on these abilities.
19 Thus, in use the networks and the automata of which they are a part do not require pre-programming that 21 anticipates all possible variants of the input data 22 they will receive, nor do they have to be pre-23 programmed with information anticipating the relation 24 of the input data to the output operations of the automaton.
26 The foregoing features are 27 observed in natural creatures and it has long been the 28 goal to develop neural network simulations that would 29 exhibit them. However, nervous system function is not currently accessible to detailed experimental analysis 31 at the level of adaptive behavior. Prior attempts to 32 simulate nervous system function have relied upon 33 analogy with certain features found in natural neural 34 systems but have been limited in their success. To that end there has been extensive study of the ~UB5~ITUT~ St1~T


5~ 2C~7~2~ PCT/US90/0586~_ 1 physical characteristics of neural networks in 2 organisms. At present two things are undeniably clear 3 about such systems. First, the physical 4 characteristics of the naturally occurring systems (e.g. the neurons or synaptic junctions) are extremely 6 complex and the number of parameters that are 7 necessary completely to describe such a system is 8 vast. The selection therefore of a group of 9 characteristics that might enable operation of an artificial neural system on a useful level is an 11 extremely complex problem that could hardly be carried 12 out without some kind of automated method. Second, 13 the sheer number of components in any animal is huge 14 compared even to the number of components that are available with the largest of present day computers.
~ Nature therefore provides 17 examples that display a level of performance that 18 would be desireable in a computerized automaton, but 19 also offers an overabundance of possibilities in-how this may be effected and no guarantee that with 21 present hardware it is even possible that an activity Z2 of interest can be simulated in a useful manner.
23 For example, in a preferred 24 embodiment of the invention to be described below, a total of 153,252 simulated synaptic connections are 26 made among 5,747 simulated neurons of 62 different 27 types. In an alternative embodiment of the visual 28 system, also described below, there are 8,521,728 29 synaptic connections among 222,208 simulated neurons, for an average of 38 connections per unit. In 31 contrast, it is estimL~bed that the human brain has 101~
32 neurons and 1015 synapses, with an average density of 33 120,000 neurons/mm3. The density of synapses is on the 34 order of 4X108 per mm3, for an average of approximately 4000 synapses per neuron.

~ IRs~lTuTE ~tt~Fr ~91/0605~ 2~ 7~ ~ ~ PCT/US90/05868 1 It has been surprisingly 2 discovered that despite the relative paucity of 3 connections received by units in the simulation, which 4 undoubtedly reduces the variety and subtlety of their responses, if a careful selection of characteristics 6 is made, sufficient complexity remains to generate 7 useful automata capable of learning and executing 8 tasks of interest.
9 Others have suggested ways to model integrated cortical action. There has been 11 proposed a hierarchical model in which the visual 12 cortex computes a series of successively abstracted 13 "sketches" of the visual scene. This model, unlike 14 the present invention, is aimed at producing a symbolic description of objects in a scene, and does 16 not incorporate means for categorizing objects or 17 responding to them. Connectionist models for cortical 18 function have also been proposed which incorporate 19 simplified abstract neurons connected to form networks. Systems based on these models have been 21 used to accomplish a number of tasks, including 22 recognition of shapes, pronunciation of written texts, 23 and evaluation of bank loan applications. Most such 24 systems incorporate a "learning algorithm", which adjusts the connections of the network for optimal 26 performance based on the presentation of a 27 predetermined set of correct stimulus-response pairs.

28 A model of sensorimotor -29 coordination has been reported that claims an attempt to replicate real neuronal structures and to utilize 31 neural maps in sensorimotor coordination. However, 32 the model has only limited utility: It is not capable 33 of categorization of the incoming data but merely 34 permits vicual signals to drive the position of an arm after training of the system.


WO91/0605~ Zc~ ~ PCT/US90/0586~-1 Another neurally based model for 2 nervous system function is primarily concerned with 3 visual pattern recognition. When a new stimulus is 4 presented to that model, it searches sequentially in its memory for a recognition template that matches the 6 stimulus; if such a match is not found, the system is 7 able to create a new template which then becomes 8 available for matching in subsequent searches. The 9 present invention, on the other hand, relies upon selection among preexisting, variant recognizing 11 elements to provide responses to novel stimuli. The 12 concept of reentry, used in the present invention to 13 integrate the responses of multiple sensory 14 modalities, is also lacking in the visual pattern recognition model.
16 The present inventors have also 17 described predecessors of the present invention. The 18 present invention differs from its predecessors by, 19 inter alia, its ability to interact with the environment through motor output, which enables 21 responses that affect sensory input. This feature is 22 termed "reentry". The responses have degrees of 23 adaptive value leading to more complicated behavioral 24 sequences and the possibility of learning. Such learning is accomplished by selection, operating 26 through a new synaptic change rule.
27 Brief Description of the Invention 28 An automaton constructed 29 according to the principles of the present invention comprises devices for sensing the state of its 31 environment, including an input array on which 32 patterns or visual scenes are captured, (for example, 33 by use of a television camera), an assembly of 34 interconnected networks or "repertoires" of recogniz-ing elements that transform input patterns, and an 36 arrangement for coupling these networks to specified TIJTE SH~ET

~091/0605~ Z~ ~7 PCT/US90/05~8 1 motor-output functions. Patterns represented on the 2 input array correspond to objects in the real world 3 which may move; mechanisms for detection of motion and 4 for development of translational and a partial degree ~- 5 of scale invariance has been provided in a preferred 6 embodiment of the automaton. Each recognizing 7 element, called a "group" (as a short form of 8 "neuronal group"), is a component of a repertoire and 9 implements a connected assembly of neuron-like units ("cells"). Cells have multiple inputs that may come 11 variously from the input array or other senses such as 12 touch or kinesthesia or from the outputs of cells in 13 the same or different repertoires. The state of each 14 cell is characterized by a single time-dependent scalar variable, sj(t), variously referred to as the 16 state of cell i at time t, or the output of cell i at 17 time t. It is dependent upon "synaptic strengths", 18 cjj, also referred to as the "connection strengths".
19 The term cjj refers to the strength of the jth input to cell i (cjj > 0, excitatory; cjj < 0 inhibitory). The 21 pattern of connections is specified by a matrix with 22 elements ljj.
23 The present invention achieves 24 its performance in part because of the arrangement of neuronal repertoires to form an overall system, in 26 part because of the choice of initial values of the 27 connection strengths, and in part because of a novel 28 amplification function, which is the rule controlling 29 the alteration of the "synaptic strength", cjj, of a connection according to the activity of the pre- and 31 postsynaptic groups.
32 The rule utilized in the present 33 invention provides, among other possibilities, for the 34 weakening of connections between pairs of units of which one, but not both, are active. This scheme 36 provides for the strengthening of connections if both ~- IR~TITUTE SH~Et 2~
WO91/06055 ~ PCT/US90/058~_ 1 presynaptic and postsynaptic levels of activity are 2 low. In addition, and most importantly, it provides 3 for modulation of the amount of synaptic change 4 according to a second input, known as a "heterosynaptic" input, which signals the success or 6 failure of recent behavior to the synapse undergoing 7 modification as determined by a "value repertoire".
8 This more elaborate, heterosynaptic amplification rule 9 enables the present invention to achieve learning. As a result the invention, aside from its direct utility, 11 provides an apparatus with which to analyze critical 12 problems involving the acquisition and maturation of 13 integrated sensory and motor behavior.
14 The present invention has value repertoires, which favor the learning of activities of 16 "value". A value repertoire has connectivities which 17 predispose them to respond to the sequelae of adaptive 18 behaviors, but their constituent neuronal groups may 19 be normal in all other respects. Characteristic features of the value repertoires include the presence 21 of sensory afferents, a relative lack of internal 22 order and topography, and diffuse and widespread 23 efferents that heterosynaptically influence large 24 populations of synapses.
Selection is a major feature of 26 the present invention. Common selectional mechanisms 27 are used to implement learning in both sensory and 28 motor control portions of the invention. This has 29 several advantages: (a) common training of both through encounter with a common set of real-world 31 situations, (b) no need to design codes for 32 communication of information between the two parts of 33 the robotic system -- meaningful signal combinations 34 are automatically selected during the training process. The use of selection against a preexisting 36 repertoire of variant recognizing elements has also ~ IR5a~1-rUTE ~i~EFr '~091/06055 2~ ~7 PCT/US90/05868 l been developed to train a recognition system. This 2 allows natural selection to be imitated in a machine 3 in order to eliminate the need for precise programming 4 for a particular recognition task. Programming is ~- 5 replaced by training based on experience with stimuli 6 similar to those that will be encountered later by the 7 device.
8 A method has been devised 9 permitting a digital computer (serial or parallel) to simulate the activity of any number of neurons ll connected together in any desired anatomical 12 arrangement. There is no programmed limit on the 13 number of cells or connections between them, nor on 14 the number of different kinds of cells or kinds of connections between them. This has entailed 16 representing in computer memory the geometry of the 17 network being simulated and the means o-f correlating 18 generic parameters contained in linked record l9 structures with the specific parameters of a given cell.
21 A means for simulating desired 22 anatomical arrangements of neurons with specified 23 realistic biophysical properties has also been 2 4 developed. It includes a means for representing the 2 5 connectivity of neuronal segments in matrix form so 26 that a computer simulation can efficiently traverse a 27 list of simulated elements and compute the response 28 voltage of each in a cyclical manner.
29 Among the advantages of the present invention are the following: The construction 31 of a recognition machine from a large repertoire of 3 2 variant recognition elements avoids the need to 33 specify precisely the characteristics of each element 34 and the detailed way they are connected together.
The invention is intrinsically 36 reliable against failure of its individual components.

~IR~TlT~JTE SHfFr W O 91/0605~ 7 -8- PC~r/US90/0586~_ 1 The "degeneracy" of such a system covers the space of 2 possible inputs with units having overlapping response 3 specificities. This degeneracy differs from 4 redundancy. Degeneracy in this invention is the presence of multiple, nonisomorphic but functionally 6 interchangeable units, whereas redundancy is the 7 duplication of isomorphic structural units to achieve 8 fault tolerance. Once the permissible "envelope" of 9 design parameters is determined, the individual elements are constructed with random variation, 11 greatly simplifying the mass production of computa-12 tional elements for the invention.
13 A further advantage of the 14 selective learning mechanism employed in the present invention, is its ability to adapt automatically to 16 different ~nvironmental conditions, such as, for ~F~e~en~
17 ~ example, difficult frequency distributions of objects 18 which the system is required to sort, different 19 distinguishing features of those objects, or dif-ferent mechanical characteristics of its sensory and effector 21 devices.
22 It is an object of the present 23 invention to provide an automaton having sensory and 24 motor systems and which is capable of learning.
It is a further object of the 26 present invention to provide such an apparatus capable 27 of establishing categories of input objects and 28 sorting the input data in accord with such categories.

29 It is a further object of the present invention to provide such an apparatus having 31 one or more sensory means identified with specific 32 sense functions for sensing input data and having 33 processing means for receiving the input data, for 34 categorizing the input data and for generating output actions in response to said input data and having ~ T~UT~

~ 91/0605~ Z ~ PC~r/US90/05868 1 output effector means for receiving said output data 2 and for sorting objects in response to said output 3 data, each of the output effector means being 4 identified with a specific motor output function.
It is a still further object of 6 the present invention to provide such an apparatus 7 where the processing means comprises simulated 8 neurons, each of which is characterized by a state of 9 activation determined by a response function; and synapses, each of which has a unidirectional 11 connection between two of the neurons. Each of the 12 synapses has an efficacy or strength capable of 13 differential modification determined by an 14 amplification function according to a selective learning rule.
16 It is a still further object of 17 the present invention to provide the aforesaid 18 apparatus having groups of neurons of one or more ls types connected more strongly among themselves than they are connected to neurons in other groups, and 21 neural maps comprising repertoires of neuronal groups, 22 corresponding to one of the sense functions or one of 23 said motor output functions.
24 It is a still further object of the present invention to provide the foregoing 26 apparatus with reentrant signaling means between the 27 neural maps.
28 It is still a further object of 29 the present invention to provide a further set of neural maps, corresponding to the various functions 31 which the automaton is designed to perform and 32 comprising value repertoires constructed from neuronal 33 groups.
34 It is yet a further object of the present invention to provide such apparatus in which 36 the modification of the synaptic efficacies alters the ~ c ~ u ç ~rr WO91/0605~ - PCT/US90/0586~_ Z~ 7 1 contribution of selected neuronal groups to behavior, 2 thereby providing integrated sensory and motor behav-3 ior.
4 Brief Description of the Drawings Figure 1 is a top level schematic 6 of the present invention.
7 Figure 2 is a schematic drawing 8 of repertoires showing the constituent groups and 9 their manner of interconnection.
Figure 3 is a drawing depicting 11 degeneracy in the responses of groups in the present 12 invention.
13 Figure 4 is a drawing of 14 generalized input means in the overall process of the present invention.
16 Figure 4.1 is a drawing of input 17 vision means in the overall process of the present 18 invention.
19 Figure 4.2 is a drawing of input touch and kinesthesia means in the overall process of 21 the present invention.
22 Figure 5 is a drawing depicting 23 reentry in classification n-tuples in the overall 24 process of the present invention.
Figure 5.1 is a drawing depicting 26 reentry in a classification couple of the overall 27 process of the present invention, and how a 28 classification couple is formed using reentry 29 connections.
Figure 5.2 is a drawing of a 31 classification n-tuple in the overall process of the 32 present invention. r ' 33 Figure 6 is a drawing of output 34 means in the overall process of the present invention.


2C~ 7 ~091/0605~ PCT/US90/05868 --11-- . .
P ~
1 Figure 7 is a drawing depicting 2 differential modification in the overall process of 3 the present invention.
4 Figure 8 is a drawing of the , 5 overall view of the automaton in a preferred 6 embodiment of the present invention. Figure 9 7 is a drawing of the visual system in the preferred 8 embodiment of the present invention.
9 Figure 10 is a drawing of an alternative (RCI) visual system for an automaton in an 11 alternative embodiment of the present invention.
12 Figure 11 is a drawing of the 13 oculomotor system for an automaton in the preferred 14 embodiment of the present invention.
Figure 12 is a drawing of the 16 kinesthesia system for an automaton in the preferred 17 embodiment of the present invention.
18 Figure 13 is a drawing of trace 19 system for an automaton in the preferred embodiment of the present invention.
21 Figure 14 is a drawing of 22 reaching system for an automaton in the preferred 23 embodiment of the present invention.
24 Figure 15 is a drawing of the output system showing reaching and object manipulation 26 for an automaton in the preferred embodiment of the 27 present invention.
28 Figure 16 is a drawing of the 29 overall flowchart stages of operation of a preferred embodiment of the present invention. Figure 31 17 is a drawing depicting the generation of diversity 32 for the simulation of the automaton of the present 33 invention.
34 Figure 18 is a drawing depicting the simulation of the changing environment for the 36 automaton of the present invention.

~1 I~S~TITUT~ Stt~FI

WO91/06055 PCT/US90/058~_ 2~7~7 -12-1 Figure 19 is a drawing of the 2 evaluation of kinesthesia for the automaton of the 3 present invention. ~' 4 Figure 20 is a drawing of the evaluation of touch for the automaton of the present 6 invention.
7 Figure 21 is a drawing of the 8 evaluation of values for the automaton of the present 9 invention.
Figure 22 is a drawing depicting 11 the evaluation of neural repertoires for the automaton 12 of the present invention.
13 Figure 23 is a drawing depicting 14 the evaluation of geometrically defined connections for the automaton of the present invention.
16 Figure 23.1 is a drawing 17 depicting the steps in evaluation of geometrically 18 defined connections.
19 Figure 23.2 is a drawing depictlng the layout of geometrically defined 21 connections.
22 Figure 24 is a drawing of move 23 effectors of the automaton of the present invent on.
24 Detailed Description Of A
Preferred Embodiment Of the Invention 26 As shown in the figures, the 27 preferred embodiment of the present invention is an 28 automaton having a visual sensing means which detects 29 objects moving in a two dimensional plane, (for example, objects restricted to a table top or a 31 television image of the three dimensional world) and 32 reaches for them with a effector arm that both senses 33 the object by touch and engages it when appropriate to 34 move the object. The automaton comprises a cortical network which resides in a digital computer or other 36 processor. The cortical network comprises neuronal 2 ~ 5 7?~ 7 ~091/060~5 PCT~US90/05868 1 cells, which are associated with each other during the 2 operation of the automaton into various neuronal 3 groups and maps.
4 Figure 1 is an overall schematic diagram of the organization of the preferred 6 embodiment. As depicted in the figure, a collection 7 of repertoires entitled Repertoires for Modality (1) 8 thorough (n) are created by a process termed the 9 Generation of Diversity. Each of these repertoires receives Sensory Data from an associated sensory input 11 device designated as Sensory Data (1) through (n).
12 Examples of such devices are television cameras and 13 pressure transducers. The conversion of such sensory 14 data to signals that are interpretable by a computer is well known to persons schooled in the relevant 16 arts.
17 At a next level is a 18 classification n-tuple of repertoires called Higher 19 Level Repertoires for Modality (1) through (n) each of which receives data from the repertoires for 21 modalities (1) through (n). The higher level 22 repertoires, also termed neuronal maps in the 23 following, are connected by reentrant signalling means 24 to form classification couples or classification n-tuples, which categorize stimulus objects and in turn 26 provide output data to a motor system that controls 27 the action of effectors for such functions as sorting 28 and robotic control.
29 Within the system, signals which are exchanged between the higher level repertoires are 31 termed reentrant signals and are shown passing between 32 all of the classification n-tuple pairs. The 33 connections between the various elements are shown as 34 arrows in the figures. The asterisks on the figure indicates the differential amplification of connection ~"~ ~1'~ Cu~Fr W O 91/0605~ ~ 6 7 ~ ~ ~ PC~r/US90/0586~_ 1 strengths biased according to values assigned by a 2 value scheme.
3 As shown in Figure 2, each '-4 repertoire consists of a collection of variant groups, with more or stronger connections between units in a 6 group than between units in different groups. For 7 example in the preferred embodiment the strengths of 8 connections, cjj, between units within a group is 9 typically 0.5 - 1.0 (arbitrary units) while the strength of connections between units in different ll groups is typically 0.3. As shown in greater detail 12 in figure 2 the repertoires are linked by reentrant 13 connections and each repertoire comprises a collection 14 of groups of cells having intragroup connections as lS well as intergroup connections within the repertoire.
16 The output connections are associated with individual 17 cells, although it is the intent of the construction 18 (inasmuch as groups are collections of cells and not 19 entities in and of themselves) that it is not important which of the cells in the group has the 21 output connection, by virtue of the high correlation 22 of activity levels among cells of a group. This 23 provides a system in which cells could change their 24 group membership or groups could fragment or reorganize themselves without sudden and dramatic loss 26 of functionality of the repertoire as whole. Although 27 this implies a degeneracy that appears to sacrifice 28 the efficiency of the construction as a whole, it is 29 more than made up by the flexibility that is achieved in the functioning of the preferred embodiment as a 31 whole. Indeed, portions of the construct could fail 32 without any overall loss of functionality of the 33 entire system.
34 Figure 3 shows a multidimensional parameter space describing input stimuli, in this case 36 a two dimensional space. In the illustration; a tJ~E SH~

2~ ?~ 7 ~091/06055 ~ PCT/~S90/05868 1 stimulus object X stimulates three groups termed 2 groups 1, 2, and 3, but not the group marked 4. The 3 groups shown have overlapping response specifications 4 illustrating the principle of degeneracy. Sensory signals arising from the stimulus are transferred via 6 direct input and intergroup connections to all groups 7 in the sensory repertoire; however, as depicted in the 8 figure only groups whose input specificity matches the 9 input signal sufficiently well are actually excited.
A specific input means associated 11 with vision is shown in figure 4.1. Here the input 12 relates to vision and involves an input array that 13 constitutes the retina of the automaton. The external 14 environment contains an object that produces an image upon the input array. The input array is 16 topographically mapped onto a primary visual 17 repertoire of groups that respond to the image of the 18 stimulus. Other neuronal groups that are not 19 responding are also shown. The term topographic refers to a mapping that preserves geometrical 21 relationships. One of the discoveries of the present 22 invention is that it is necessary to have both 23 topographic and non-topographic mappings in order to 24 provide sufficient functionality to the device.
Other specific input means, 26 depicted in figure 4.2, are associated with the senses 27 of touch and kinesthesia. Here the real object in the 28 environment is contacted by touch sensors comprising 29 transducers such as are known to a person of ordinary skill in the relevant art and connected at the end of 31 a multijointed arm. The touch sensors map onto the 32 primary touch repertoire having groups that respond to 33 such stimulus. An additional primary kinesthetic 34 repertoire is associated with each of the joints' angular ranges.


, . , !, W O 91/06055 ~ 7 PC~r/US90/0586~_ . -16-1 Figure 5.1 depicts reentry in a 2 typical classification couple, In this case, reentry 3 between the visual association area repertoire (~) and 4 the trace association area repertoire ~ (to be defined below) is shown. As depicted, the visual association 6 area repertoire receives inputs from a primary visual 7 repertoire and has outputs to later processing areas.
8 The trace association area repertoire (denoted MT) 9 receives its inputs fro~; the primary kinesthetic repertoire and also outputs to later processing areas.
11 The reentrant connections are those shown between the 12 ~ and ~ repertoires.
13 Figure 5.2 is an example of a 14 classification n-tuple (in this case with n=3). Input signals are received by a repertoire for a visual 16 submodality, such as texture. outputs from this 17 submodality are sent by reentrant connections to two 18 other submodalities, in this example for color and 19 direction of motion. Each submodality has direGted outputs, and in addition bidirectional reentrant 21 connections with each of the other members of the n-22 tuple.
23 Figure 6 shows the arrangement of 24 a t pical output means. Shown are the oculomotor repertoire implementing two opposing pairs of movement 26 means similar to muscles that move the eye or TV
27 camera left-right and up-down respectively, and the 28 arm motor repertoire implementing flexor and extensor 29 movement means at the various joints of the arm. The various motor components receive signals from neuronal 31 groups. In the case of the arm motor repertoires 32 there are separate neuronal groups whose output 33 activates flexors and others to activate extensors.
34 Training by selection is depicted in figure 7. Sensory inputs relating to the 36 consequences of behavior are connected to a value ~1 IRCTI~U'rE ~H~T

91/06055 z~ 7 t ~ ' PC~r/US90/05868 1 repertoire. The value repertoire is arranged in such 2 a way that its response to these sensory inputs is 3 larger when the action has been more successful in 4 attaining a particular goal, and smaller when the action has been less successful. This arrangement 6 allows differential synaptic modifications based on 7 behavior. The value repertoire has a single output 8 which is connected as a heterosynaptic input to cells 9 in a repertoire along the motor pathway between the input and output means. Unlike the case with other 11 methods for training neural networks, only a single 12 input (in the present case, an input from a value 13 repertoire) is needed to regulate synaptic 14 modification at all synapses in a particular motor system, because value input is required solely to 16 indicate the relative degree of success of past 17 behavior, and not to indicate in detail the amount of 18 correction required at each synapse to generate better 19 behavior. The normal input of cells in the motor pathway is joined with the value input in the sense 21 that both are factors in the determination of 22 modifications in the strengths of synapses.
23 Figure 8 is an overall diagram 24 showing the component neuronal subsystems of the preferred embodiment. These subsystems are shown in 26 more detail in the following figures. The following 27 conventions are used in figures 9 - 15.
28 0 A cell 29 ~ An excitory connection ¦ An inhibitory 31 connection 32 An ambiguous connection 33 (excitory or inhibitory) 34 (Member of a bidirectional set.) ~ CU~Fr WO91/0605~ ~ A PCT/ US90/0586~

2C~67 ~ 7 -18-1 ~ An ambiguous connection 2 (excitory or inhibitory) 3 (Member of unidirectional set.) ~ A connection biased by 6 value.
7 Figures 9-11 show an object to be 8 categorized within a "window" of visual attention that 9 is moved by the oculomotor system as depicted in figure 6. The object is imaged on an input array of 11 2~ x 2kY pixels. The output from the array provides 12 input to an R repertoire of FD (feature detector) 13 cells. These cells may be of several different kinds, 14 responding, for example, to line segments in different orientations (vertical, horizontal, or oblique), to 16 bent lines, or to line ends. The choice of these 17 types may be made when the automaton is set up, to 18 ~;r;ze its sensitivity to features of input objects 19 that are likely to be relevant for identifying and sorting them. In the preferred embodiment described 21 here, four such kinds of FD cells are used, which 22 respond to line segments oriented vertically, 23 horizontally, and at the two possible 45~ positions 24 (northeast -- southwest and northwest -- southeast).
Whatever choice of feature-detecting cells is made in 26 a particular embodiment, one cell of each type is 27 placed at each position in the R repertoire. In the 28 preferred embodiment described here, there are 14x14 29 positions in the R repertoire. Each response of an FD
cell thus indicates both the type and position of a 31 corresponding feature in the input array, and 32 therefore also in the object of attention. These 33 responses are sent as data and combined with locally 34 reentrant connections in an R2 repertoire of llxllxl E2 cells and llxllxl RX cells for categorization. The 36 output pattern is then sent (see Figure 15) to the ET

SUR~TITUT~ St~m 2~7~ 7 ~ 91/060~5 PC~r/US90/05868 1 repertoire and read out to an RC repertoire while 2 interacting with reentrant connections from an 3 repertoire. Figure 10 illustrates an 4 alternative version of the visual system which A 5 utilizes reentrant connections among three visual 6 areas to obtain a unified visual response to more 7 complex objects which may contain contours defined by 8 moving points and which may contain contours which are 9 occluded by other objects lying between the object in question and the eye of the automaton. This 11 alternative visual system consists of three sections, 12 or areas, each of which has several repertoires which 13 are described in detail later. The VOR section is a 14 visual orientation area containing groups that respond predominantly to the orientation of visual contours.
16 This section receives input from the input array via 17 an intermediate "LGN" repertoire that responds to 18 visual contours. The VHO section is a visual motion 19 area containing groups that respond predominantly to the direction of motion of visual contours. This 21 section receives inputs from VOR and also provides 22 reentrant inhibitory connections to VOR that sharpen 23 the specificity of the directional responses.
24 Finally, the VOc section is a visual occlusion area containing groups that construct responses to 26 invisible, occluded contours by the action of 27 connections from the other two sections, along with 28 reentrant connections back to VOR .
29 Figure 11 depicts the oculomotor system used to direct the visual sensor (television 31 camera or other visual means) toward a target object 32 and to follow moving target objects. Visual 33 repertoire VR receives topographically mapped inputs 34 from the input array, and repertoire SC receives similarly mapped inputs from VR. An oculomotor 36 repertoire OM in turn receives inputs from SC, so ~ ~D~ rl IT~ S ~;~T

WO91/06055 z ~ t~f~ PCT/US90/058~_ 1 arranged that each cell of OM receives inputs from 2 cells covering the entire area of SC. The OM
3 repertoire is divided into four areas, indicated by R, 4 U, L, and D in the figure, which are specialized to move the visual sensor respectively to the right, up, 6 left, or down. The right-left and up-down pairs are 7 mutually inhibitory. FO is a value repertoire which 8 responds weakly to objects in the peripheral visual 9 field of the automaton, and strongly to objects in the central visual field. Outputs from FO provide 11 heterosynaptic bias to the selection of connections 12 between SC and OM such that motions of the visual 13 sensor that bring objects into the central visual 14 field are selectively enhanced.
The arrangement of the 16 kinesthetic system used in categorization is depicted 17 in figure 12. Connections from position and/or motion 18 sensors in arm joints are received by the KE
19 repertoire of lx12xl cells. Outputs are sent in-turn to an MT repertoire of 12XlXl M1 and M2 cells and 21 12XlX4 MS and MB cells. The MT repertoire also has 22 suppressive connections from an RC repertoire. An RM
23 repertoire having 12xlx16 RM cells and an equal number 24 of RX cells receives output from the MT repertoire, readout connections from an RC repertoire and 26 reentrant connections from E2 cells in the R2 27 repertoires. Its outputs go to the RX cells in the R2 28 repertoire and to the ET repertoire.
29 The trace system of the preferred embodiment is shown in figure 13. The object to be 31 categorized is detected in the environment by vision 32 and the arm is brought to the object by the reaching 33 system, then moved into its straightened posture.
34 Connections from the touch receptors are input to the TH repertoire and from there to the TC repertoire.
36 From the TH repertoire signals are received by four 2~5~?~
~091/06055 PCT/US90/05868 1 edge repertoires El-E4 which are constructed with a c 2 pattern responsive respectively to right, bottom, left 3 and top edge positions. These in turn input to the TM
4 trace-motor repertoires which output in turn to the motor drives that move the tracing finger in its four 6 directions.
7 The reaching system of the 8 preferred embodiment is shown in figure 14. A
9 repertoire MC receives both visual input (from repertoire WD) and kinesthetic input (from repertoire 11 KE, which is the same repertoire as repertoire KE in 12 figure 12). These inputs are arranged in such a way 13 that activity of groups in MC corresponds to arbitrary 14 combinations of positions of objects in the environment (signalled by WD) and positions of the arm 16 joints (signalled by KE). Groups in MC are connected 17 in a dense, overlapping fashion to cells in an 18 intermediate repertoire IN. These cells in turn are 19 connected to motor control repertoire a (see figure 15). A value repertoire (lower left) views the 21 position of the distal end of the arm (vial~visual 22 repertoire HV) and the position of the stimulus object 23 (via visual repertoire WD). These inputs are 24 topographically arranged such that cells in the value repertoire respond most strongly when the hand is near 26 the stimulus object. Activity in the value repertoire 27 biases the selection of input connections to MC.
28 Also shown in figure 14 are 29 repertoires GR, IO and PK, which are responsible for inhibiting unproductive motions of the arm. Cells in 31 repertoire GR receive both visual input (from 32 repertoire WD) and kinesthetic input (from repertoire 33 KE). Cells in repertoire IO receive mixed excitatory 34 and inhibitory connections from MC, and their activity is modulated by excitatory input from the value 36 repertoire. Cells in repertoire PK receive large ..~Q~t~lTF ~

WO91/060~ 2~ ~ 7 PCT/US~/0586 1 numbers of input connections from randomly selected 2 cells in GR, and sparse but strong connections from 3 cells in IO. Cells in PK inhibit activity in IN.
4 Figure 15 together with Figure 6 depicts the output system for reaching and object 6 manipulations. The ET repertoire receives input from 7 RX cells in repertoires ~ and ~ o~L~uLs its signals 8 to the OP repertoire, which also receives signals from 9 the RX cells in repertoire ~ and outputs to the RG
(reflex generator) repertoires. These have reentry 11 with each other and output to the flexor and extensor 12 systems of cells in repertoire SG. The latter control 13 the individual joints of the arm. The flexor system 14 comprises subrepertoires SG-1 and SG-2 of lx4x8 AF
cells. The extensor system comprises similar 16 subrepertoires SG-2 and SG-l repertoires of AE cells.
17 The SG-1 repertoires also receive inputs from KE
18 receptors (for inhibition at the ~imits of motion), 19 ~; from touch receptors, and from RN repertoires related to reaching.
21 Figure 16 is a flowchart showing 22 the stages of the simulation of the preferred 23 embodiment of the present invention. The stages 24 comprise the following: a timer is started and default parameters are established that pertain to functions 26 not explicitly set in the control file. Then a 27 control file is interpreted to establish the 28 particular embodiment of the automaton to be 29 simulated. This file includes so-called Group I
input, which pertains to sensors and effectors, and 31 Group II input, which pertains to repertoires, cell 32 types, and connections. Parameters from these inputs 33 are stored in so-called "control blocks" for later 34 use. Consistency checks are performed, flags are set requesting allocation of memory for optional variables 36 and statistics, the storage requirements for these UTE s~n 2Q6~
rO91/06055 ~ s ~' PCT/US90/05~8 1 items are calculated, then the required storage is Z actually allocated. The display windows are located.
3 Then the repertoires are generated and the environment 4 in which the automaton will operate is established.
Group III input is interpreted to 6 provide for control of stimulus presentation, printing 7 and plotting options, value scheme parameters, reset 8 options, and cycle parameters. A series of trials is 9 then begun, each of which involves the presentation of one or more stimuli. Group III parameters may be 11 charged at any time during the course of a simulation.
12 At the beginning of each trial, 13 cells and effectors may be reset to a standard state 14 if desired (for example, so that reaching may always be performed from a standard starting posture). The 16 state of the environment is updated including the 17 presentation of a new stimulus. Then the kinesthesia 18 is evaluated (see d~~ +' ~~23~\1OW), value is 19 evaluated (see details below) and the neural repertoires are evaluated (see details below). The 21 effectors are then moved in a manner that will be 22 described in greater detail. According to the input 23 parameters, the results are then printed and plotted.
24 The status of the repertoires after training can be saved for later reuse as a performing automaton. The 26 trial series continue until no more input is present.
27 Figure 17 shows the steps in the 28 generation of repertoires. First space is allocated 29 for all repertoire data. A repertoire and a cell type are begun. Pointers to arrays for current and 31 previous cell data are calculated and stored.
32 Geometric connection types are initialized by 33 calculating needed geometric constants and by 34 allocating space for temporary storage and for normalization and falloff tables. Normalization 36 tables contain values needed to adjust the strength of : '~
W091/0605~ 2C ~ 7%~7 PCT/USgo/oS~

1 geometric connection types when the geometric area in 2 question lies partly outside the boundary of the input 3 cell layer. In such cases, the input to the geometric 4 connection type may be multiplied by the ratio of the area of the complete geometric region to the area of 6 the region that is inside the input cell layer, thus 7 normalizing the input to that which would have 8 occurred if the entire geometric area had fallen 9 within the boundary of the input cell layer. Falloff tables contain values needed to adjust the strength of 11 geometric connection types according to the distance 12 of the geometric area in question from the center of 13 the input cell layer. Falloff tables may be used when 14 it is desired to favor stimuli that fall near the center of a sensory cell repertoire by reducing the 16 amount of lateral inhibition produced by stimuli that 17 are farther from the center. Both normalization and 18 falloff corrections are calculated at this time and 19 stored in tables for later use. This is repeated for each geometric connection type.
21 Next, the specific connection 22 types are initialized. As each specific connection 23 type is begun, various constants needed for the later 24 calculations are initialized, and files are opened if connectivity patterns or connection strengths are to 26 be read from external files. If a previously saved 27 run is to be utilized as a starting point, the saved 28 states of the repertoires are retrieved from external 29 storage at this point. File headers are checked to determine if the repertoire names and parameters match 31 the current run. The saved states, consisting of s;
32 values for the last two time steps, and the saved 33 connection data (cjj and ljj) are read in.
34 If previously stored s; values are not being used, then all cells in a cell type are 36 initialized one-by-one. For each cell, group number ~ IR~T~ JT~ S~

ZC~6'~ ~7 0 91/06055 ~ ~ '. P(~r/US90/0~868 1 and x, y coordinates are set and all connection data 2 are zeroed. A loop over connection types is then 3 executed. For each connection type, setup code 4 appropriate to the methods of cjj and ljj generation specified by the group I or group II input is 6 executed: in the case that cjj values are set from 7 externally supplied matrices, the appropriate matrix 8 is located; in the event that cjj values are set in a 9 gradient pattern known as a "motor map", the orientation of the motor map is established; in the 11 event that cjj values are chosen randomly, a generating 12 seed is selected.
13 For each new connection, the cjj 14 generation code is executed. Depending upon the type of generation, a cjj value is chosen: For a motor map 16 an increment is applied to the previous value; for 17 random cjj, a random value is generated; for matrix c 18 a value is picked up from a file and when all the 19 input values have been used the list of values is recycled from the beginning. The ljj generation code is 21 then executed to define the connectivity matrix.
22 These steps continue until all 23 cell types and repertoires have been generated. The 24 various options allow generation of diversity in many forms within the repertoire. This diversity is 26 central to the construction and operation of the 27 present invention.
28 Figure 18 depicts the stages in 29 the updating of the state of the environment and of r 30 the retinal cells of the input array. This is for 31 test purposes only; in an actual device sensors in the ' 32 real environment are used. The steps are as follows:
33 All pixels of the input array are set to the 34 background density; then a loop is executed over all active (i.e. visible) objects. Each object is moved 36 according to its pre-established pattern of motion, ~UB~TITUTE ~

W O 91/0605~ ~ P(~r/US90/0586 ~ '7 -26-1 or, alternatively, an object may be removed from the 2 environment and replaced by a newly selected object.
3 Motions that may be ge ~rated in this simulation 4 include rotation, random jumping, linear motions and oscillation in various patterns. In addition, an 6 object may be moved because it has been hit by the arm 7 of the automaton. When an object reaches the edge of 8 the input array, it may be caused to undergo various 9 boundary condition operations: it may disappear, it may be reflected back by a mirror reflection, it may 11 reenter at the opposite edge (toroidal boundary 12 condition), or it may remain fixed at the edge.Pixels 13 covered by the object are then set to appropriate 14 brightness values. These values are calculated to match the values that would be measured by a 16 television camera viewing a corresponding real object 17 at the same position.
18 Figure 19 shows the steps in the 19 evaluation of kinesthesia. This is for test purposes only; in an actual device stretch or velocity sensors 21 in the real effector arms are used. For each arm, the 22 cells responding to kinesthetic sensors in that arm 23 are located, and for each joint the following values 24 are calculated: a = the current joint angle, ~a = the joint range/number of KE cells, and a = ~a * KE half 26 width. Then for each of the cells responding to each 27 such joint the following values are calculated:
28 s; = Min(l.O, exp(-( e-amjn)2/a2) 29 a = a + ~a If there is a universal joint 31 (the special type of shoulder joint used in tracing) 32 the following values are calculated instead:
33 ~a = 2~/number of KE cells 34 a = ~a * KE half width ST~TlJT~

~O9l/0605S Z~ ~7 PCT/US90/05868 AMPL = I(x shift + Y2shift) ~ (This is the 2 distance the universal joint has moved from its 3 starting position.) 4 EJL = Max(EJL, AMPL) - (This is the maximum distance the universal joint has moved since 6 the start of the current run.) 7 AMPL = AMPL/EJL (This is the amplitude 8 of universal joint motion as a fraction of the largest 9 motion which has so far occurred.) These operations may be carried 11 out using for x shift, y shift either the absolute 12 joint angle or the change in the angle since the 13 previous time step, whichever works better for a 14 particular categorization task. Then for each cell responding to the joint the following is calculated:
16 ARG = ~ - T (if ~ - T<~) else 2~ -(~- T) 17 s; = Min(1.0, exp-(ARG2/a2)) 18 ~ = ~ + ~a.
19 Then for each of the visual sensors of the automaton having kinesthetic sensors, 21 the following is calculated:
22 T = current position of eye muscle in 23 question 24 ~ = Range of eye motion/number of KE
cells 26 a = ~ * KE half width 27 Then for each cell responding to the 28 eye muscle the following is calculated:
29 ~ = O
sj = Min(l.0, exp(-(~-T)2/ a2) 31 ~

32 ~igure 20 is a flowchart for the 33 evaluation of touch by the present invention. Touch 34 receptors may be arranged in a rectangular pattern on any of the joints of a given arm, but usually they are 36 placed only on the distal end. As with the case of SUB5~TITUTE ~H~E~

WO91/0605~ z ~ ' PCT/US90/0586~_ 1 vision and kinesthesia discussed above, the simulation 2 of touch rectors is for test purposes only; in an 3 actual device, pressure sensors on the real arm are 4 used. For each of the arms with touch the sj value of all touch cells is first cleared to 0. Then x,y is 6 picked up at the attachment point. For each joint of 7 the arm the position of that joint in the environment 8 is calculated. If the joint has touch receptors, the 9 location of the upper left corner of the tactile box (the collection of touch receptors on that segment) 11 and the orientations of the proximo-distal and lateral 12 axes are calculated. Then the axial and lateral 13 dimensions of the individual touch cell receptive 14 fields are computed. The projection of each such cell onto the input array at the current orientation is 16 then computed. (In order to optimize the speed of 17 this calculation, the four corners of the receptor box 18 are located and the smallest and largest x of these 19 four is found. Only x coordinates in this range are examined. For each such x, the range of pixels 21 covered in the y direction by receptors at this x 22 value is then determined and all pixels in this y 23 range are examined.) For each pixel so examined that 24 contains a non-background value, the corresponding touch receptor is activated.
26 Figure 21 shows the evaluation of 27 input to a value repertoire. There are two generic 28 types of value scheme, environmental or internal rpe.
29 Environmental value schemes base their activity on some feature sensed in the environment, usually one 31 that is changed by the activity of the automaton;
32 internal value schemes base their activity on the 33 state of some other repertoire in the automaton and 34 are used for homeostatic purposes. For the internal type, the value may increase in proportion to the 36 number of cells firing, reach a maximum when a ~UE~STITUTE SH~E~

2~ Z~
0 91/06055 ,~ PC~r/US90/0~868 1 particular number of cells in the source repertoire 2 "fires", and then decline as more cells fire (this is 3 known as a "tent" function) having the shape of a 4 rising and falling ramp.
- 5 Alternatively, the value may remain maximal when more 6 than the optimal number of cells fire (this is known 7 as an "N FIRE" function), having the shape of a rising 8 ramp function that levels off.
9 For the environmental type a value for VC is calculated depending on the 11 characteristic of interest, for example, the distance 12 between the hand and an object being grasped. For 13 both types of value scheme, VC is converted to Vb, 14 which is the change in VC since the previous time step, multiplied by a fixed scaling factor. vb is then 16 subjected to a "sliding window" averaging process to 17 smooth it, and the resulting value, va, is assigned to 18 that particular value scheme.
19 Figure 22 shows the steps in the method for the evaluation of neural repertoires. This 21 consists of the evaluation in turn of modulatory 22 connection types and then specific connection types.
23 Modulatory connection types are described by "MODVAL"
24 control blocks, which contain parameters for the calculation of modulation values, and by "MODBY"
26 control blocks, which contain parameters for the 27 modulation of particular cell types by such modulation 28 values. Similarly, specific connection types are 29 described by "conntype" control blocks, which contain parameters dictating the calculation of inputs from 31 specific connection types. These parameters are used 32 in conjunction with the cjj and ljj values that were 33 stored during the repertoire generation of diversity 34 stage. These calculations are described in detail following the descriptions of the remaining figures.

~UE~TITUTE SH~El W O 91/0605~ P(~r/US90/0586~_ 2C~ 30-Figure 2 3 shows the steps in the 2 method for the evaluation of the geometrically defined 3 connections. Figure 23.1 is a flow chart showing the 4 steps in the method. Each set of geometrically defined connections is described by parameters 6 contained in an "i nhihhlk~ control block. A method 7 has been developed for calculating the contribution of 8 the geometric connections for one i nhihhlk in time 9 proportional to the number of cells in the source cell type plus the number of groups in the target array 11 multiplied by the number of bands of groups 12 surrounding each target cell from which geometric 13 connections are taken. This method is significantly 14 faster than the obvious method of simply adding up the contributions of the inputs to each cell, which 16 requires time proportional to the number of groups in 17 the target array multiplied by the square of the 18 number of bands of groups surrounding each target cell 19 from which geometric connections are taken. This new method requires the use of intermediate storage known 21 as "boxsum" (BxSum) arrays, horizontal strip (HSTRIP) 22 arrays, and vertical strip (VSTRIP) arrays. These 23 arrays are used to build up the needed sums of input 24 activities in the form of horizontal and vertical strips of cells which can be combined in various ways 26 to form the square bands that make up each set of 27 geometric connections, as shown in Figure 23.2. For 28 each inhibblk the boxsum array is cleared and then for 29 each group and for each cell in that group the sum of the activities of the cells in that group, less a 31 t~ eshold amount, is calculated and stored in the 32 b~xsum array. Because geometric connections are 33 evaluated on the basis of a series of rings of cells 34 surrounding the target cell, it happens when a target cell is positioned near an edge of the repertoire in 36 which it is contained that portions of these rings may ~1 IB~Tl~uTE S~

2 ~ J r ~ _IL7 ~091/06055 PCT/US90/05868 1 fall outside the boundaries of the source cell 2 repertoire. Values are assigned to these missing '~ 3 inputs according to a "boundary condition" which may 4 be chosen for each ;nh;h-hlk according to the function of the particular geometric connections in question.
6 The possible boundary conditions include: (1) noise 7 boundaries (the missing cells are assigned noise 8 values), (2) edge boundaries (the missing cells are 9 assigned the values of the nearest cells inside the edge of the source repertoire, (3) mirror boundaries 11 (the missing cells are assigned values of cells in the 12 interior of the source repertoire at positions 13 corresponding to the positions of the missing cells 14 mirrored across the nearest boundary), (4) toroidal boundaries (the missing cells are assigned values of 16 cells in the interior of the source repertoire as if 17 the left and right edges and the top and bottom edges 18 of the source repertoire were joined to form a 19 hypothetical repertoire with toroidal geometry,-and (5) normalized boundaries (the missing cells are not 21 considered; instead, the input value obtained for the 22 source cells that do exist is multiplied by the ratio 23 of the area of the full set of geometric bands to the 24 area of the bands that lies inside the boundaries of the source repertoire; this normalization is carried 26 out with the aid of normalization tables prepared 27 earlier). A pre-computed falloff correction may also 28 be applied at this time.
29 Once all the boxsums have been evaluated, taking into account the boundary 31 conditions, the ring summations are carried out as 32 outlined in Figure 23.1. The first ring consists of 33 the single group at the position of the target group.
34 The second ring consists of the eight groups surrounding the target group, and so on. Each ring is 36 calculated as the sum of four strips; two vertical ~t~ IR5Tl~uT~ SHEE~

WO9l/06055 PCT/US90/0586~_ z 9~ r B 1 7 --32 1 strips forming the left and right edges of the ring;
2 and two horizontal strips forming the top and bottom 3 edges of the ring. For the second ring, the vertical 4 strips consist of single groups and the horizontal strips consist of three groups (see Figure 23.2); for 6 each successive ring, the vertical and horizontal 7 strips are elongated by adding a group at each end.
8 The calculation is arranged in such a way that the 9 strip sums already calculated are reused, after augmentation, for the next set of rings, but each 11 strip is paired with a different set of more distant 12 strips as the rings P~pAn~. This method of combining 13 strips is the critical innovation that makes possible 14 the great speed of the present method of calculating geome~rical connections.
16 The completed ring sums are 17 multiplied by the coefficients ~ (defined below) and 18 stored for later use.
19 Figure 23.2 illustrates some of the features of the method of evaluating geometric 21 connections. The boundaries of a source repertoire 22 for a set of geometric connections are indicated by a 23 large square. These boundaries are e~pAn~ed, using a 24 boundary condition as described above, to fill a larger rectangle. Squares labelled A, B, and C
26 represent the areas covered by single groups. The sum 27 of the activities of all the cells in each such group 28 constitute the first r~ng of inhibition for the 29 corresponding target cells. Additional rings are defined as the sums of values in horizontal and 31 vertical strips, indicated by numbers 2, 3, 4, 32 The series of rings at A illustrates how the boundary 33 area is used to provide values for areas that do not 34 exist in the source repertoire proper (stippling).
The series of rings at B and C illustrate how a single 36 horizontal or vertical strip (for example, strip 3, ~UBSTITUTE SHI~ET

Z~7,~7 ~ O 91/060~5 PC~r/US90/05868 .

1 cross-hatched) contributes to rings of inhibition 2 around two y~OU~S (B and C) simultaneously.
- 3 An alternative embodiment of the 4 visual part for the categorization subsystem is described in the following paragraphs. This 6 embodiment may be used to replace the preferred 7 embodiment when it is desired to respond to visual 8 fields that contain overlapping objects or objects 9 defined by contours, parts of which may be moving.
The alternative embodiment uses a more complex set of 11 reentrant connections to achieve a unified response to 12 such input patterns.
13 Figure 10 presents a simplified 14 schematic of the overall network connectivity with further details given in Table 2. The connections 16 between VOR~ VOC~ and VMO follow a particularly simple 17 reentrant system.
18 Stimuli of various sizes and 19 shapes excite the elements of the Input Array (64x64 pixels) which corresponds to a visual sensor. There 21 is a single pathway from the Input Array through the 22 "LGN", to "4C~". "4C~" projects to "4B" which 23 contains several separate populations of units. "4B-24 Dir" units are directionally selective, and are - reentrantly connected to VMO~ "~B-Orient." units are 26 orientation selective, and provide input to "4B-Term"
27 units which are specialized for detecting line 28 terminations, and are reentrantly connected to V0C.
29 "4B-Orient." units also project to Reentrant Conflict units, which as discussed below, respond to conflicts 31 in the responses to real and illusory contours (these ~ 32 units signal an internal inconsistency in the 33 determined occlusion relationships of several adjacent 34 surfaces).
The "LGN" and "4C~": The "LGN"
36 contains ON-center, OFF-surround units which receive ~- IR~lTuTE SH~Fr 7~7 1 inputs from 3x3 pixel regions of the Input Array. No 2 OFF-center LGN units are included, and for that 3 reason, light stimuli on dark backgrounds have been 4 used exclusively.
The orientation selectivity of ~
6 "4C~" units arises from the spatial pattern of the 7 connections from the "LGN". Each "4C~" unit receives 8 excitatory inputs from an elongated (3xl unit) region 9 of the LGN, and inhibitory inputs from two elongated (3xl unit) region of the LGN, and inhibitory inputs 11 from two elongated (5x2) flanks. The connection 12 strengths of the inputs are adjusted so that units can 13 be partially activated by lines whose orientations are 14 within 45 degrees of the preferred orientation. The square geometry of the underlying pixel matrix 16 necessitates different inhibitory surrounds for 17 obliquely orientated versus horizontal or vertically 18 oriented units.
19 Directional selectivity in ~'4C~"
is achieved through a mech~nicm involving temporarily 21 delayed inhibition. Inhibitory inputs from the "LGN"
22 produce a signal which leads to hyperpolarization of 23 the unit through the opening of simulated ion 24 channels. The temporal delay in the inhibition is controlled by the time constants of the production and 26 decay of this signal. Each "4C~" unit receives 2 sets 27 of inputs from the "LGN", one excitatory and one 28 inhibitory. Directional selectivity dep~n~ upon the 29 fact that the inhibitory units are shifted with respect to the excitatory inputs (the direction of the 31 shift will turn out to be the null direction of the 32 "4C~" unit). In these simulations, the 2 sets of 33 inputs were always shifted by 1 unit, and the temporal 34 delay was always 1 cycle; however, the same mechanism can be used to generate a range of velocity 36 sensitivities.


2C~ ~7 ~n~ 91/06055 ~-. PC~r/US90/05868 1 The V~0 PathwaY
2 "4B-Dir" units: Starting with the "4B-Dir" units, the 3 VMO pathway transforms the selectivity of unit 4 responses from a primary selectivity to orientation (found in "4C~") to a primary selectivity to direction 6 (found in the Direction repertoires). There are 2 7 types of "4B-Dir" units. One type receives excitatory 8 inputs from two adjacent "4C~" units. Using a 9 temporal delay mechAn;sm similar to that in "4C~", the "4B-Dir" unit is activated only if one of these inputs 11 occurs within a fixed time window before the other 12 input. Each "4B-Dir" unit also receives direct-acting 13 inhibitory inputs from a 5x5 unit region in the "4C~"
14 repertoire whose directional selectivity is in the null direction; this null inhibition greatly enhances 16 the directional selectivity.
17 Inputs from 3 of these units with 18 orientation preferences spAnn;n~ 90 degrees are then 19 subjected to a threshold and summated by a second type of "4B-Dir" unit. Summation over the 3 directions 21 (NE, E, & SE) assures that "4B-Dir" will detect lines 22 moving Eastward even if they are not of the preferred 23 orientation for a given "4C~" repertoire. Due to the 24 non-linearity of unit properties, this local circuit scheme is not equivalent to a single unit that 26 receives, thresholds, and sums the inputs from "4C~".
27 For example, in this scheme, excitation of units in 28 different "4C~" repertoires will not give rise to 29 activation of "4B-Dir".
ComParator Units: The first 31 repertoires in VMO are the comparator repertoires.
32 There are 4 comparator units for each direction (e.g.
33 North); each unit is inhibited by motion in one of the 34 4 adjacent directions (e.g. NE, NW, E and W). Each comparator unit receives excitation from a 5x5 unit 36 region in the corresponding "4B-Dir" repertoire, and ~ IR5~TlTlJTE SH~

W O 91/0605~ 2 Ct~ 7~L 7 P(~r/US90/0586~_ 1 inhibition from a 5x5 unit region in one of the 2 adjacent "4B-Dir" repertoires. Thresholds are 3 adjusted so that each comparator unit is activated 4 only if the responses to motion in its preferred direction eYcee~ those to motion in the adjacent 6 direction.
7 Direction Units: The final stage 8 in the V~0 pathway consists of units that sum inputs 9 from the 4 comparator units selective for motion in a particular direction. Thresholds are arranged so that 11 inputs from at least 3 of the 4 sources are necessary 12 to fire a unit. This represents a majority vote on 13 the differential comparisons carried out by the 14 comparator repertoires, and signals that the response to motion in a given direction is stronger than in the 16 adjacent directions.
17 VMO - V~ Reentry: Outputs from 18 each direction unit are reentered back to "4B-Dir" in 19 VOR~ Each direction unit inhibits a 3X3 unit region in all "4B-Dir" repertoires except the one with the same 21 directional preference. This arrangement tends to 22 suppress activity in "4B-Dir" repertoires that does 23 not correspond to the active V~ repertoire.
24 We have found several alternative schemes of reentrant connectivity that help generate 26 directional selectivity. For instance, excitatory 27 reentrant connections to "4B-Dir" units can be coupled 28 with cross-orientation inhibition. Alternatively, 29 reentry can originate from the comparator units instead of the directional units. Several such 31 schemes generate roughly equivalent results.

32 The V0c Pathway 33 The V0c pathway detects and 34 generates responses to occlusion boundaries. An 3~ occlusion boundary can be defined by the presence of ~R~T~Tl~F s~r ~091/0605~ ZC~7 PCT/US90/05868 -37- - ~

1 textures or lines of various orientations (either 2 stationary or moving) that terminate along its extent, 3 and by the absence of textures or lines that extend 4 across the boundary. The VOc pathway initially responds to local cues consistent with an occlusion 6 boundary, but continued responses depend upon the 7 global consistency of these local cues (i.e., whether 8 multiple local terminations can be linked up along an 9 extended discontinuity or "fracture" line). The same local cues which indicate the presence of occlusion 11 boundaries are responsible for the generation of 12 illusory contours.
13 The VOc system also checks 14 (through reentry) that the occlusion boundaries it discriminates obey a number of self-consistency 16 relationships. For example, two occluding boundaries 17 should not cross each other, nor should an occluding 18 boundary cross a real boundary. These physical 19 inconsistencies are reflected by internal conflicts in the system which must be resolved to yield a 21 consistent pattern.
22 "4B-Term" Units: As local cues 23 to occlusion, the V0c pathway uses both line 24 terminations and the differential motion of textures.
"4B-Term" units detect line terminations due to an 26 inhibitory end-region in their receptive fields. End-27 stopped receptive fields are found in simple cells of 28 layer 4B as well as in complex cells of layers 2 and 29 3. However, unlike end-stopped cells in the striate cortex, "4B-Term" units have only one end-inhibitory 31 region, and are thus sensitive to the polarity of the 32 termination, (i.e., at which end of the line the 33 termination is found).
34 Wide Angle Units: Since lines can terminate at an occlusion boundary with a variety 36 of orientations with respect to that boundary, Wide ~UB~TITUTE S~IEFI

W O 91/060~ PC~r/US90/0586.~_ 2~ %~ 7 -38-1 Angle units sum inputs from "4B-Term" units whose 2 preferred orientations span 90 degrees.
3 Termination Discontinuity Units:
4 Wide Angle units project to Termination Discontinuity S (TD) units which detect local cues to occlusion.
6 These local cues consist of any of several terminating 7 lines that approach the presumptive occlusion boundary 8 from either side. In order to be activated, a TD unit 9 must be activated by at least 3 inputs from the Wide Angle repertoires -- and at least one of these inputs 11 must correspond to line terminations of an opposite 12 polarity to that of the other inputs. All 3 types of 13 inputs must, in addition, come from units distributed 14 along a line in a Wide Angle repertoire (and this is assured by the geometry of the connections).
16 A separate population of TD units 17 (referred to as Direction ~iscontinuity units) carries 18 out a similar operation upon inputs from V~ that 19 signal the presence of a discontinuity in motion. Use of a single population of TD units to receive inputs 21 from both V~ and V~ is not possible because activation 22 of a TD unit requires a combination of inputs, and a 23 single unit (of the simple type used here) cannot 24 distinguish a valid combination which arises from one set of sources from partial combinations which arise 26 from two different sets of sources. Direction 27 Discontinuity units have a slower time decay for 28 voltages allowing them to maintain responses to moving 29 stimuli that have recently passed through the receptive field of the unit.
31 Occlusion Units: Occlusion units 32 respond to the actual location and course of an 33 occlusion boundary. Each Occlusion unit receives 34 connections, in a bipolar fashion, from two sets of TD
units distributed in opposite directions along a 36 common line. To be activated, an Occlusion unit must ~llR~TlT~lTF ~ r O 91/06055 2~ L~ PC~r/US90/05868 -39- ' - .

1 receive inputs from both bipolar branches (i.e., both 2 sets of TD units). This connection scheme ensures 3 that a string of adjacent Occlusion units will be 4 activated by an occlusion boundary.
The remaining repertoires and 6 connections in the V2~ pathway (described below) deal 7 with the elimination of false cues, the resolution of 8 internal conflicts in generated responses, and with 9 the reentrant "recycling" back to V~ of responses to occlusion boundaries determined in V~.
11 Common Termination Units: Common 12 Termination units respond to configurations in which 2 13 or more lines terminate at a common locus. These 14 units sum inputs from Wide Angle units with identical receptive field locations and adjacent orientation 16 preferences. Common Termination units directly 17 inhibit TD units.
18 Reentrant Conflict Units:
19 Reentrant Conflict units respond to locations at-which illusory contours cross real contours or other 21 illusory contours. Reentrant Conflict units receive 22 connections from 3 "4B-Orient." repertoires having 23 orientations spanning 90 degrees (in exactly the same 24 manner as Wide Angle units), and in addition receive excitatory reentrant connections from Occlusion units.
26 To be activated, each Reentrant Conflict unit requires 27 at least one input from an Occlusion unit (illusory 28 line) and one input from a "4B-Orient." unit (real 29 line) with an overlapping receptive field. Reentrant ~30 Conflict units are also strongly inhibited by 31 corresponding units in the Wide Angle repertoires:
32 since illusorY contours alwaYs join real contours at 33 their terminations, conflicts at a termination are not 34 to be counted.
Occlusion Conflict Units:
36 Occlusion Conflict units receive connections from ~UE~TITIITE St1E~T

1 Reentrant Conflict units in exactly the same manner 2 that Occlusion units receive co~ections from the TD
3 repertoires, and they generate responses to (illusory) 4 contours between the points of conflict. The Occlu~ion Conflict units directly inhibit Occlusion 6 units, thereby-cAnceling (through reentry) L2 ~_ e~s 7 to any segment of a generated illusory contour which 8 was in conflict.
9 Recursive Synthesis by Reentry:
A final V~ reentrant pathway allows signals generated 11 by illusory contours and structure-from-motion to be 12 reentered back to V~, and treated a_ if they were 13 signals from real contours in the periphery entering 14 via "4C~n. This recursion is a key property of reentry. A separate population of ~4B-Term~ units is 16 used to receive inputs from Occlusion units. These 17 "4B-Term" units then project to the Wide Angle units, 18 thereby merging with the signal stream of the normal 19 asc~n~ing V~ pathway. This reentrant pathway allows contours generated through structure-from-motion from 21 V~ inputs to be used a_ termination cues for the 22 generation of additional illusory contourQ. This is 23 the basis of the me~h~nism underlying the recursive 24 syn aesiQ simulation experiments to be ~ieCllQee~

WO 91/06~K~ PCI'/US90/OS868 Motion Effectors 7 Figure 24 ~how~ the ~tep~ taken in 8 implementing motion effectors. For each arm, each 9 joint angle (jangle) is adjusted according to the formula (modulated by forcing jangle into a specified 11 range accounting for physical motion limitations on a 12 real arm):
~ 13 jangle - old angle + angle scale *
14 change The ~change~ is calculated as the 16 sum of activity in the motor neuron~ controlling the 17 joint in question (changes are ~n~Ye~ by a pointer 18 "jptrn); the angle scale is a fixed parameter.
19 If the ar~ i~ in it~ special exten~e~ posture for object tracing, termed a 21 ~CA~lorl ~ cal tracing position", then it i8 driven by 22 angular ~hA~es at a "universal joint~, which permits 23 motion in both a horizontal and a vertical plane, in 24 the manner of a shoulder ~oint. In this ca~e, the vertical and horizontal rotations are driven by 26 separate neuronal repertoirQs, a~ indicated in figure 27 24, and the remaining joints are typically held fixed.
28 The kinesthetic calculation for a universal joint 29 differs fro~ that for a normal joint, aQ shown in figure 19.
31 The calculation~ outlined in 32 figures 22-23 are repeated a fixed number of times.
33 These repetition~ constitute ~inner cycle~. The 34 larger set of calculation~ outlined in Figs. 18-24, including these "inner cycles~, are repeated once for W O 91/~ PCT/US9O/

1 each time the environment is updated, a total of NTR
2 trials constituting one trial series.

3 Definitions And Verbal Description 4 Of The Automaton As shown in figure 2, a neuronal 6 group is a collection of units, v~riously termed cells 7 or neurons, that are strongly connected to one 8 another. Stronqly connected refers to having a 9 greater number or strength of intragroup connections relative to intergroup connections. (As a typical 11 working definition, cells in a group receive more than 12 half of their input from other cells in the same 13 group.) 14 Neuronal y~OU~ form in any embodiment of the invention as a result of correlated 16 stimulation of sets of cells which have appropriate 17 preformed interconnections, inasmuch as such-18 correlated stimulation leads to 5tLen~l h~ning of 19 intercellular connections (nsynAp~es~) under operation of the rule for synaptic modification to be described.
21 These ~LVU~S continuously refor~ at a rate which is 22 relatively slow compared to the time scale of physical 23 operations of the automaton.
24 Pec~use of the
interconnection of cells in ~Lvu~s, the overall 26 operation of the automaton may be described in terms 27 of the average respon~ of its neuronal ~G ~ ~, as if-28 they, and not the individual cell~, were the 29 fundamental units of the cortical network. In this way the cortical network i~ ~ade les~ ~ensitive to the 31 life history and proper functioning of any individual 32 unit. In order to simplify the calculation~ required 33 to maintain the automaton, ~-~u~onal ~.ou~g may often WO 91/060SS PCI'/US90/OS868 1 be replaced by simplified entities that do not contain 2 individual neurons.
3 In operation, the neuronal groups 4 become associated with one another into a plurality of S neuronal maps. A neuronal map is a functionally 6 defined structure of interconnected neuronal groups, 7 usually tuyo~.aphically arranged that s~oJ~ly respond 8 to related inputs. The signals that pass between one 9 neuronal map and another may be termed reentrant signals.
11 The preferred embodiment of the 12 present invention is implemented by assigning ~ 13 appropriate parameters to control the operation of a 14 general-purpose cortical network simulator (CNS) program. The resulting embodiment is a large-scale 16 network consisting of multiple types of units with 17 detailed specification of the connectivity between and 18 among them, as well a~ of the specific properties of 19 the interunit connections, which parallel certain electrical and chemical properties of biological 21 synapses. The embodiment also allows stimulus ob;ects 22 of various sizes and shapes, which may be detected by 23 a TV camera moving across the environment and feeding 24 images to the input array.
The preferred embodiment that is 26 described has been implemented on a digital 27 supercomputer architecture in a combination of FORTRAN
28 and Assembler language. The y~G~m provides a user 29 interface that permits a large ~e~r~e of control over the ~tructure and size of the simulated network~ at 31 the level of ~repertoires~, cells, and connections.
32 Control ~tatements are used to define named entities 33 of each of these ty-ye~. Th~ ordering of the control 34 ~tatements e~tablishes a three-layered tree structure in whlch the node~ at the top level are repertoire~, 36 thosQ at the intermediate level are cell type~, and WO 91/0605~ , PCT/US90/

1 those at the bottom are co~nection classes. The nodes 2 at each level are allocated dynamically and linked by 3 pointers. Each node (also known as a ~control block") 4 contains parameters that define the properties of the co~bpo.. ding objects, as well as pointers to the 6 arrays containing the ob~ects themselves. In 7 addition, connection nodes contain pointers to the 8 cell-type nodes defining the cells where the 9 connections originate. There are three kinds of these connection nodes, for specific (nconntype~ blocks), 11 geometrically defined (usually inhibitory) connections 12 (~inhihhlk'l blocks), and modulatory connections 13 ("MODVAL" and 'IMODBY" blocks). The statements 14 defining the connection nodes contain codes to select any of several methods of connection generation -16 uniform connectivity, topographic map, or a list read 17 from an external file. Other codes are provided to 18 control the generation of the co ~ tion strengths and 19 the particular re~ponFe function and method of amplification to be used.
21 The actual data objects (cells 22 and co~ctions) are allocated after all the nodes 23 have been defined and the total me~ory requirements 24 calculated. These requirements can amount to as little as one byte per connection plus two bytes per 26 cell. To minimize virtual memory page faults, the 27 output state variables of the cells, wh$ch ~ust be ~8 acc~s~ed randomly for input to other-cells, are 29 allocated in one block. All the other var~ables, which occupy significantly more spacQ but are -ccesse~
31 only sequentially, are allocated in another block.
32 Because various types of cells require different 33 variables at the CQ11 and connection 1QVQ1a~ the 3~ offset of each variable from the beg~nninq of its cell or co~ections record is also variable. ThesQ offsets 36 are calculated and stored in a table associated with WO 91/060S~ PCr/US90/OS868 1 each cell-type node. For maximum speed in the $nner-2 loop Assembler code, the nec~ss~ry offsets are moved 3 into the displacement fields of the relevant 4 instructions during the initialization for each cell type. Looping then requires only th- updating of a 6 record pointer for each new cell or connection to be 7 processe~. In the less critical FOR$RAN code, the 8 variable offsets from the tables are used along with 9 the record pointers to generate array subscripts.
The simulation program makes use 11 of dynamically allocated storage and record data 12 structures with FOR$RAN programming, neither of which 13 is a feature of the FOR$RAN language. $he scheme 14 depends on the absence of execution-time subscript range chec~ ing in the ob~ect code produced by the 16 FOR$RAN compiler. The record structures are defined 17 as lists of variables in common blocks, which both 18 enforces the desired storag- ordering and make~ the 19 shared base addresses available in all the subroutines that refer to them. Each variable is declared as an 21 array of dimension one, and the offset to the actual 22 dynamic storage is provided by a call to an Assembler 23 routine that obtains the de~ired storage from the 24 operating system. Individual array elements are 2S accessed by subscripts which are sums of these dynamic 26 array offsets with the customary ~nAeYe~.
27 To loop over an array of such 28 dynamic records, the s~h~cript variabl- i~ simply 29 increased by the record length aft-r each iteration.
Processing of an entire network involves execution of 31 an outer set of loops that traverse all the nodes of 32 the tree structure (incidentally providing base 33 address for access to the parameters ~tored in the 34 nodes), together with inner~loops that ~Lc~ the cells and connections belonging to ~ach nod-.

WO91/~XK~ PCT/US~O~K8 1 In a further preferred embodiment 2 the model is structured for parallel execution. For 3 purposes of carrying out parallel exe-u~ion in a 4 machine with local memory, s~(t) and silt+1) arrays containing the activities of the cell~ at times t and 6 t + 1, respectively, are kept separately for each cell 7 type, permitting synchronous updating of all t~e 8 activities at the end of each cycle by communication 9 of the s~(t+1) arrays from each node to all other lo nodes, where they replace the current si(t) arrays.
11 This arrangement permits s~(t) to be read in a 12 consistent way by all processors while sj(t+1) is being 13 calculated and stored. Synchronization of the 14 processors is required only at the co~pletion of an entire cell layer, when sl(t+~ ubstituted for 16 sj(t). With the modestly sized s arrays of thi~
17 embodiment, it is possible to broaAc~et copies to all 18 the processors to avoid communications bottlDnec~ for 19 random access from other cells during the evaluation of s~(t+l).
21 In operation, during each unit 22 time interval, new states are calculated for all cells 23 in all repertoires in turn. Co~.. F~-ion ~en~ hs are 24 modified in accordance with an ~mplific~tion rule immediately after each connection has been used for 26 the calculation of the new activity value for its 27 cell. A number of such cycles is typically carried 28 out before a new stimulus is presented.
29 The system i~ opierated both for the ~L~ose of "trainingn, during which a selQction of 31 and pathway~ take~ place 80 that th neural map 32 structure i8 established, and thereafter for 33 performance (wherein further training may continue 34 indefinitely). A few cycles are usually sufficient for the syetem ou~u~ to reach conve~ 9 for a new 36 stimulus after the selection ~tag- has been co~pleted.

W091/~KS PCT/US~/0~K~

1 ~When an ob~ect is being traced for tactile 2 identification, the time for tracing control~ the 3 overall response time.) 4 To simplify the description of the preferred embodiment, it will be divided into 6 three sections. The first ection de~cribes in 7 general terms the cell types, connection types, and 8 certain other parameters that are u~-d in all 9 repertoires. The second section defines the various repertoires that are used in the automaton, both in 11 terms of the cell types and connections that are 12 utilized and in terms of the functions that they 13 perform. The third section describes the manner in 14 which the activity value of a cell i8 calculated and ~ 15 the additional parameters that are involved in selec-16 tion.
17 A. Cell Types. Connection Ty~es and Other Parameters 18 The anatomical specifications of 19 the preferred embodiment comprise repertoires, cell types, and connection types. There are three broad 21 classes of connection types: specific, geometric, and 22 modulatory. Specific connections are described by 23 listing individually the cells which are 24 interconnected (by use of a matrix [lij]), permitting any desired array of connections to be specified.
26 Geometric co~nections are arranged in a series of 27 concentric rlngs around a given target cell: all the 28 cells in each ring may have a common scale factor, but 29 the scale factor may be different for each ring.
Modulatory co~nections derive their input fro~ the 31 total level of activity of all the cells in a given 32 source repertoire, each cell being included with equal 33 weight.
34 The various cell type~ are defined in tQrms o~ several numerical paramQter~ that 36 distinguish their properties. Each cell type may be W091/~S PCT/US90/O~K8 specified to have CQn ?ctions of any or all of the 2 three broad classes just described. For each cell 3 type, a positive threshold value above which the sum 4 of all inputs is effective for excitation i5 specified. A negative threshold i~ ~imilarly set for 6 inhibitory input~. A hit threshold i8 set, such that 7 a cell's output is con~idered active only when its 8 o~puL exceeds this value.
9 A cell may be sub~ect to depression, whereby its response is reduced as a ll function of its past activity. Furthermore, a 12 depression threshold may be set, such that if the 13 amount of depression exceeds a specified value, the 14 cell enters a refractory period, during which input to the cell has no effect. A refractory period parameter 16 is ~et to define the length of this time. In 17 addition, a refractory decay li~it ~ay be set which 18 defines an alternative kind of refractory period in l9 which a cell becomes refractory when it fire~ above the hit threshold and remains refractory until its 2l output decays below the said refractory decay li~it.
22 A sustaining threshold may also be ~et, such that the 23 total inputs above the said sust~ining threshold 24 excite the cell even when it is in a refractory period.
26 A long term potentiation (~LTP") 27 threshold may be set. LTP refer~ to a long-la~ting 28 Pnh~nc~ment of the effect~ of a given cla~s of 29 synapses which oe~ in the preferred embodiment by lowering their effective thresholds. In a biological 31 context it ha~ been defined as a phenomenon in which a 32 brief series of biochemical events gives ris- to an 33 PnhAncement of ~ynaptic efficacy that i- extraordi-34 narily long-la~ting. The LTP thre~hold allow-contributions fro~ an individual connection typ- to 36 the build-up of LTP only if the threshold i8 ~Ycee~le~

WO 91/060S5 PCT/US90/O~K~

1 by the input to that connection type at that instant 2 of time.
3 Several parameters may be 4 specified for each connection type that inputs to a particular cell type. These identify the source of 6 the co~ections (e.g. another repertoire of cells, the 7 input array, or "virtual cells" that signal sensory 8 inputs, namely sight, touch, ~in~ hesia, or, finally, 9 a particular value scheme). Further parameters specify the number of connections of each type and the 11 rule for initializing the strengths of these 12 connections during repertoire generation. Available ~ 13 rules either generate a gradient (also known as a 14 motor map), obtain values from a stored matrix, or use random values.
16 Each specific connection type 17 also requires specification of the manner of 18 generation of the identitiea of the cells from which 19 the co~ections originate within the specified source cell type. Separate rules may be given for selection 21 of the first connection of each type and for the 22 selection of subsequent connections. Rules available 23 in the CNS simulator for selection of first 24 connections are "external" (wherein the identities are read from a connection list); "float" (wherein a - 26 selection is made uniformly from all cells in a 27 specified box on the input source); ~group" (wherein 28 the selection is made uniformly from the same group of 2g which the target cell is a member; ~joint~ (selection is made uniformly from s~lccessive subdivisions of a 31 repertoire arranged to ~ sqrve the needs of the 32 s~ccessive joints of a l.mb in turn); ~normal~
33 (selection is made from cells distributed normally 34 around the location of the target cell); ~other"
(~election is made from ~o~ other than the group 36 the target cell is in); "t~G~.~phic" (connections are 1 mapped from a rectangular box on the input ~ource 2 assuming stationary visual sensor~ scanned topo-3 graphic" (same as topG~aphic except map move~ with a 4 specified "window" embodying the field of view of a specified eye); ~uniform~ (connections are selected 6 uniformly from all cells in th- source repertoire);
7 "systematic" (successive cells are displace~d by a 8 constant distance).
9 The. connection type also specifies the relationship of subsequent connections 11 to the first connection selected. The values for this 12 relationship are "ad~acent~ (each connection is spaced 13 by a fixed stride from the previous connection);
14 "boxed" and ~diagonal~ (col-~.e~-Lions are arranged in a determinate matrix); ~crow'~ foot~ h~esuent 16 connections are uniformly but randomly distributed in 17 a rectangular box centered about the first position 18 chosen); "indepen~ent~ (all connections are rhos~n 19 independently); and "partitioned~ (duplicate-connections are avoided by partitioning the source 21 cells into subsets and choosing one from each subset).
22 Two parameters specify the 23 dynamical behavior of each connection type, a 24 threshold value below which each input is ineffective, and a scale factor which determines the relative 26 contribution of the particular connection type 27 relative to other connection types incident upon the 28 same cells.
29 With Le ye~ to the modification of conn~ction strengths according to the selective 31 learning rule given below,-certain parameters may be 32 specified for each connection type. The~e include an 33 amplification factor, which ad~usts the overall rate 34 of synaptic change, an ampli~icAtion thre~hold relating to post synaptic activity, an amplification 36 threshold relating to presynaptic activity, an WO91/~KS -51- PCT/US~/O~K~

1 amplification threshold relating to heterosynaptic 2 activity, and a rule selector. Selective modification 3 of connection strengths also involves the formation of 4 a "modifying substance" at each synapse, with respect to which a production rate, a decay constant, and a 6 maximum decay rate de~cribed b-low may be specified 7 for each ~on~ection type.
8 There aro ~any other numerical 9 parameters and control parameters relating to the screen display which could be chosen by a person of 11 skill in this art.
12 B. The Re~ertoires 13 The preferred embodiment 14 described here is an automaton specialized to the tasks of detecting ob~ects by a visual sense, re-ching 16 toward said objects with an arm, examining said 17 ob;ects by vision and by touch using said arm, 18 categorizing said ob~ects based on said examination, 19 and either accepting or rejecting said ob~ects based on their category, using the arm as a means of 21 rejecting. The repertoires to be described are each 22 associated with one or more of these functions, and 23 accordingly the descriptions are divided into the 24 following five subsystems: The oculomotor (saccade) subsystem, the reaching subsystem, the tracing 26 subsystem, the categorizing subsystem, and the 27 re~ection subsystem. The principles of the present 28 invention may be used to construct other automata for 29 different purposes and such automata would consist of similar repertoires inte.~G e ted in way~ appropriate 31 to a particular task that would be evident to a person 32 skilled in this art. AS an example of such an 33 application, a e~co,d embodiment of the visual system 34 is described, which is able to ~e~ond to so-called ~illusory" contours, thereby detecting and WO 91/060~S PCr/US90/OS868 1 categorizing objects, the visual image of which is 2 occluded by other objects.
3 The repertoirea of the preferred 4 embodiment are described in detail in Table I, and those of an alternative embodiment of the visual part 6 of the categorizing subsystem in Table II.

W091/~XK~ PCT/US~/O~K8 2 PRO~ S OF ~ k.OIRES

4 Meth-Gen2;
S Reper- CellNUM-CNNS
6 toire TyDes:e Afferentsl e(k~: n~k~ Ffferents3 7 VR RV O.2 Input Array RS 1 SC
8 0.1 1.0 9 RI 0.0 VR RI RG 1 VR RV -0.1 0.35 11 SC M2 o.o VR RV RTI 4 OM
12 0.5 1.5 SC M2 -13 IN 0.0 SC M2 RG
14 0.1 0.35 FO FO 0.2 Input Array RSI 3 Vl 16 0.1 0.5 17 OM OM 0.1 SC ~Vl~ RXA 256 move 18 0.25 0.2 visual 19 sensor ReDertoire Details of Unit DYn~mics and Connectivity 21 VR Layers RV and RI: 841 excitatory and 841 22 inhibitory units (29x29 grid). Every 23 excitatory unit receives one 24 to~c~Laphical connection from the portion of the environment currently viewed 26 through the visual sensor. Local 27 excitatory and lateral inhibitory 28 connectivity provides sharpeni ng of 29 responses and attentional bias.
Excitatory unit~ have depression, with 4 31 refractory cycles after 8 con~ tive 32 cycles of maxim~l firing.
33 SC Layers M2 and IN: 256 excitatory and 256 34 inhibitory units (16x16 grid). Every WO 91/060Ss PCT/US90/O~U~

1 Repertoire Details of Urit DynA~ics An~l Col-nectivitY
2 excitatory unit receive~ four 3 toyo~aphical connections. Local 4 connectivity a- in VR.
FO 121 excitatory units (llxll grid).
6 Receive one excitatory to~G~Laphically 7 mapped conn~ ~ion from th- entirQ visual 8 field, as well as additional connections 9 from the central 15% and 3% of the visual field, respectively. Global average of 11 activity in the repertoire is used for 12 heterosynaptic value input into 13 connections from SC to OM.
14 OM 36 motor units (4 ~ou~8 of 9 units).
Each unit receives 2S6 excitatory 16 connections from the entire array of SC
17 units. Two $nhibitory connections from 18 opposing units provide lateral inhibition 19 (sharpening r~s~ol~e).

WO9l/~XK5 ~ r~NG SYST~

2 Reper- Cell M th-Gen;
3 toire TYpes:e Afferents NUM-CONNS Ffferents 4 HV HD 0.1 Input Array RT 1 Value 0.0 0.4 6 WDWD 0.2 Input Array RS 1 Value 7 0.1 0.35 8 WI 0.0 WD WDRG 1 WD WD -9 0.1 0.35 Value H2 0.91 HD RGC 30 Value 11 0.1 0.5 Scheme ~2 13 0.1 0.5 14 KEKE 0.0 VJ RX 1 MC, GR
0.0 0.35 16 MC ME,MF WD (V2) RUC 16 I0, IN
17 0.3 0.1~ 0.25 19 0.1 0.7 RE (V2 ) RJI 8 21 0.1 0.15 23 0.1 0.15 24 I0 IE,IF MC RTI 5 PK
0.5 0.1 0.15 26 GRGl 0. 6 WD RUC 9 E~
27 0.25 0.9 29 0. 25 0.25 PK PE,PF GR RTI 72 ~ -31 0.3 0.1 1.2 33 ' 0.1 0.6 0.1 0.25 WO 91/060SS PCT/US90/O~

1 Reper- Cell Meth-Gen;
2 toire Ty~es :e Affer~nts NUM-CONNS ~fferents 3 IN RE,RF MC RXC 3 SG
4 0.3 0.2S 0.5 6 0.2S 0.25 8 0.1 0.7 9 SG AE,AF ~ RJI 16 move arm 0.1 0.1 0.85 joints 12 0.2 0.1 13 Details of Unit Drnamics anA Conn~rtivity 14 Repertoire HV 2S6 units for hand vision (16 x 16 grid).
16 One excitatory connection from the visual 17 field.
18 WD 841 units (29x29 grid) 1 excitatory 19 tu~o~Laphic con~ector from input array. Cell type WI provides latral inhibition to sharpen 21 responses.
22 Value 256 units (16x16 grid), 30 excitatory 23 tG~G~-aphical colu.Qctions fron an llxll 24 region in HV, 40 excitatory topG~aphical connections from a 17x17 region in WD. Both 26 inputs are reguired to elicit a ~e,yonoe.
27 Global average of activity in the repertoire 28 is used for hetero~ynaptic ~alue input into 29 conne~tions from WD and RE to MC.
KE 12 units p~r arm ~oint (12x4 grid). Units 31 are tuned usinq a Gaussian function to 32 .~e~EJG.. -i preferably to a part~)lar ~oint 33 position (angle). ~
34 MC Layer ME: 192 units (1 . 4 grid), predominantly moving ext~ or muscles with 16 WO 91/060S~ PCT/US90/O~U~

Details of Unit Dyrami C8 an-l Co~nectivitv 2 ReDertoire 3 MC connections (e/i ratio 2.33) from 13 x 13 4 regions of WD, 16 conn-ctions (-/i ratio 1.5) S from the entire array WD (these connections 6 only to the first ~oint), 16 co~n~ctionQ, 7 mapped and unmapped, (e/i ratio 1.0) from all 8 joint levels in RE, 6 ~xcitatory connections 9 from MC ~oint level n-1 to n, and 18 inhibitory connections from MC ~oint level n 11 to n-l. If value i~ positive connections 12 from WD and KE to MC are streng~h~ned if pre-13 and postsynaptic unit are coactive; if value 14 is negative these co~nections are wea~ene~.
In addition, 24 inhibitory connections from 16 MF cells, and 16 inhibitory conne tions from 17 TH cell~.
18 Layer 2: 192 flexor units (lx4 grid). Same 19 con~ectivity as Layer 1.
I0 Layer 1: 96 flexor units (lx4 grid), 5 21 connections (e/i ratio 4.0) from HC. Unit 22 activity is modulated by valu- ~2 -- cells 23 fire only if positive value is present.
24 Layer 2: 96 extensor units (lx4 grid).
Connectivity as in Layer 1.
26 GR 288 units (12x24 grid), 6 rc --e_--ions (e/i 27 ratio 1.22) from ~ , and 9 c-ol~r.e_-~ions (e/i 28 ratio 4.0) from 8 x 8 regions in WD. Both 29 inputa required to fire unit.
PK Layer PE: 96 flexor units (lx4 grid), 4 31 strong excitatory tG~c~Laphical cr~ e ~ions 32 from I0 flexor unit-, 216 initially w-ak 33 co e tions (e/i ratio l.S) from GR units.
34 These connections are strengt~-r~1 if pr--and post-synaptic unit~ ar- coactive, and 36 weakened if presynaptic unit is activ- but Wo 91/060SS PCI'/US90/OS868 1 Details of Unit Dy~Amic~ An~ Connect~vity 2 Repertoire 3 PK post-synaptic i~ not. Pg units remain active 4 for several cycle~ after excitation, before entering a refractory period of several 6 cycle~s.
7 Layer PF: 96 ext~n~or unit~ (lx4 grid).
8 Connectivity a- in ~ayer 1.
9 IN Layer RE: 192 flexor units (lx4 grid), 6 excitatory connections from MC flexors, from 11 topographically corresponding and neighboring 12 joint levels, 24 inhibitory connections from 13 the corresponding joint level in PK, both 14 extensors and flexors. These connections become less inhibitory if pre- and post-16 synaptic units are coactive, and become more 17 inhibitory if pre- synaptic unit is active 18 but post-synaptic is not.
19 Layer RF: 192 extensor units. Connectivity as in Layer 1.
21 SG Layer AE: 128 flexor units (lx4 grid), 16 22 connections from IN flexors or extensors (e/i 23 ratio 1.86), all-or-none inhibitory 24 connections from TH to joints 1 and 2. These co,u~e~ions inhibit gross arm moveoent when 26 touch is establi~hed.
27 SG Layer AF: 128 extensor units (lx4 grid).
28 Connectivity as in Layer 1.

WO 91/060S5 PCT~US90/O~K8 TACTIT.~ SySTF~I

2 Reper- Cell Meth-Gan;
3 toire TvDes :e Affer~nts NUM-CNNS Efferents 4 TH TH .01 VT TC,El,E2, from s RX 1 E3,E4 touch 6 0.0 receptors 7 0.5 8 TC TC 0.1 TH MTB 9 TM
9 0.0 2.0 El,E2, E1,......... TH MTB 9 TM
11 E3,E4 0.1 0.0 1.33 12 TM UD,LR TC LXB 36 move 13 0.1 0.1 0.18 shoulder 14 El,E2, RJA 1 ~oint E3,E4 0.1 0.22 16 Details Of Unit Drnamics An~ Connect~vitY
17 TC, E1, E2, E3, E464 units (8x8 grid), 1 18 excitatory connection from grid o~ touch receptors on 19 last arm joint.
36 units (6x6 grid), 9 connections arranged 21 in an on-center off-su~o~.d matrix.
22 36 units each (6x6 grid), 9 connections 23 arranged in a matrix allowing edge detection.
24 16 units, 4 each for up, down, left, and right motion. 36 excitatory co,u-e_Lions from all 26 positions in TC. Csnne~tions to each Or the four 27 motor neuron ~ou~s have one-dimensional gradients in 28 their connection strength. Additional connections 29 from El, E2, E3, and E4 terminate on their respective groups of motor neurons to further bias joint motion.

91/060SS PCr/US90/05868 1 CAT~GORTZ~TION SYST ~

2 Reper- Cell Meth-Gen;
3 toire Types :e AfferentQ NUM-CNNS ~fferents 4 LG~ LN,LF Input Array MSB 16 R
S 0.1 0.12 2.5 6 RFD 0.35 LGN MTB 2S R2 7 0.1 0.4S
8 RZ E2 0. 4 R RUI 3 2 ~7", RF
0.1 0.7 11 0.3 1.2 12 RX 0.91 E2 RX
13 0.15 0.35 0.3 0.5 17 0.2 1.0 18 MT Ml 0 .1 MT M2 RG 1 . R, 0.01 0.5 M2 0.1 XE RJ
21 0.08 1.5 22MS 0.91 MT Ml RG
23 0.3 0.4 0.3 0.24 27 0.3 0.4 29 0.3 0.4 RC CR RX
31 0.2 5.0 3 2MB O.91 MT M1 RG
33 0.3 0.24 3 4 MT Ml ROI 9 0.3 0.24 WO 91/060S~ Pcr/usso/oss6s 1 Reper- Cell Meth-Gen;
2 toireTypes:e Afferents NUM-CNNS Efferents 4 0.3 0.24 6 0.3 0.24 8 0.2 0.5 9 R~RM 0.1 MT MB RTI 2 R2, RF
0.2 0.5 12 0.2 1.0 13 RX 0.91 RM RM RX
14 0.1 0.35 16 0.3 2.5 18 0.2 1.0 19 RCTS 0.1 MT MS RX 1 R2, R~
0.2 0.5 21 TB 0.1 MT MB RX
22 0.2 0.5 23 CE 0.1 RC TS RXA 48 24 0.2 1.0 26 0.2 1.0 27 CR 0.3 RC TS RX 48 28 0.1 1.0 O.l l.O

32 0.1 1.0 33 ET ET 0.91 RM RX RXA 32 OP
34 ~~0.2 1.5 36 0.2 1.5 WO91/~X~5 -62- PCT/US~/~&~

1 Reper- Cell Meth-Gen;
2 toire TY~es;~ Afferents NUM-CNNS Efferents 3 OP OP 0.91 E$ ET RUP 8 RX
4 0.2 1.5 6 0.2 l.S
7 RG X2 0.5 RG X1 RXA 4 SG
8 0.1 1.0 9Xl 0.5 OP OP RUP 4 0.1 0.7 12 0.1 1.0 13 X3 0.5 RG X2 RXA 4 14 0.1 1.0 Details of Unit Dynamics And Connectivity 16 Layers LN, LF: 324 ON-center units and 324 17 OFF-center units (18x18 grid), receiving 16 inputs 18 each in a to~o~Laphic map from the input array.
19 784 units each of 4 types (14x14 grid), receiving 9 to~o~Laphically mapped connections 21 arranged in a matrix to produce orientation selective 22 units. Each position in R contains 4 units responding 23 optimally to horizontal, vertical and diagonal lines.
24 Layer E2: 484 units (llxll grid), 57 excitatory connections spread out over the entire R
26 array.
27 Layer AX: 484 units. 1 to~o~Laphically 28 mapped connection from layer E2. Units will not fire 29 if only these connections are active. They also receive connections from the trigger unit TR and-48 31 reentrant con~ections from F~. Activity in these 32 connections can lower the excitation threshold (~LTPn) 33 of the unit.
34 Layer Ml: 12 (12xl grid), receiving a temporally delayed connection from Layer 2.


1 Details of Unit DYna2ics An~ Connect~vity 2 Layer M2: 12 unit~ (12xl grid), 1 3 topographically mapped excitatory connection each from 4 M1 and M2, 9 inhibitory connections from non-corresponding position~ in Ml and H2. 1 inhibitory 6 connection fro~ trigger rep rtoir-. Units detect 7 correlation of motion in one direction ("smooth 8 edgesn).
9 Layer MB: 48 units (12xl grid), 1 excitatory topographical connection from M2, 1 inhibitory 11 topographical connection from Ml, 9 excitatory non-12 mapped connections from M1, 9 inhibitory non-mapped 13 connections from M2. 1 inhibitory connection fro~
14 trigger repertoire. Units detect absence of correlation in motion (~bumpy edges~).
16 Layer RM: 192 unit~ (12xl grid), 2 excitatory 17 tu~G~aphical co~nections from MT, Layer MB. Units 18 also receive input from RC to re-excite units that 19 have been recently active.
Layer RX: 192 units (12xl grid), 1 excitatory 21 mapped input from Layer 1. This input alone will not 22 fire the unit, which also receive 24 re-entrant 23 connections from ~. An additional input from RC can 24 influence firing threshold of unit by "LTP~.
Layer 48 units (12xl grid), 1 to~o~aphical 26 excitatory co~n-ction from MT, Layer MS. Units are 27 active for several cycles after activation and then 28 enter a refractory period. They detect novel smooth 29 contours -- the Ahsençe of firing in this Layer indicates the ahsence of such features. Inhibited by 31 RC CR.
32 Layer TB: 48 (12xl grid), 1 to~,~J~aphical 33 excitatory c~ ~ection from MT, Layer MB. Re, 34 like Layer TS, but for ~bumpy~ contour~.
Layer CE: 1 unit, 48 excitatory connections 36 each from Layer TS and TB. Novelty detector.

WO 91/060Ss PCT/USgO/

Details of Unit DynAmicg .n~ Corr~ectivitY
2 ~ayer CR: 1 unit, ~8 inhibitory connection~
3 each from Layer TS and TB, 1 excitatory connection 4 from Layer CE. Fires if no novel stimuli features are detected.
6 16 units (lxl grid), 32 connections each from 7 widespread regions in ~ and F~. Inputs from both 8 and ~ are required to fire a unit.
9 16 units (lxl grid), 32 excitatory connections from ET, 8 excitatory connections from ~, 11 Layer RX. Two inputs are required to fire a unit.
12 12 unit~ arranged in 3 Layers. Form 13 oscillatory circuit.
14 ~ote:
lS The c~ ion strength~ of afferents in 16 underlined type are modifiable, under the 17 heterosynaptic influence of a value scheme if one is 18 listed. The e/i ratios for certain connection types 19 give the ratio of the number of excitatory to the number of inhibitory conn-ctions. An excitatory 21 connection cannot become inhibitory by amplification, 22 or vice-versa.
23 1. The following abbreviation-~ are used in 24 the listing of afferent conn~ction~:
VJ Kinesthetic ~--~or~ from ar~ joints 26 VW Kinesthetic ~Qn~ors of visual 27 sensor motions 28 VT Touch 29 Vn Value scheme n All other na~e~ or repertoires, or, where 31 a repertoire ha~ more than one cal} type, and the 32 connections di~tinguish betwe~n these cell type~, the 33 name of the repertoire is followed by the naoe of the 34 cell typ~.

WO 91/~K5 PCT/US90/O~K~

1 Geometrical and ~odulatory connection~
2 are described in the comment~ in the right-hand column 3 Details of Unit DYn~_ics ~ Connectivitv 4 or in the main text 2 Hethod of generation and n~mher of 6 connections $he following ~bbr-viations are used to 7 describe the method of gen-ration of C~J for each 8 connection type 9 L gradients of ~motor maps" are lo generated ll M specified matrices are used 12 R random numbers are used 13 $he following abbreviations are used to 14 describe the method of generating the first 1~ of each connection type to each call (the meanings of these 16 terms have been given above) 17 E external O other 18 F float T to~G~dphic 19 G group S ~Ann~
toyo~aphic 21 J ~oint U uniform 22 N normal X systematic 23 The following abbreviations are used to 24 describe the method o~ generating ljj after the first one for each connect type ($hese abbreviations are 26 omitted where there is only one connect of the given 27 type) 28 A adjacent D diagonal 29 8 boxed I in~ependent C crow's foot P partitioned 31 e(k), o~(k) ~paren same a~s ~ s~ript]
32 represent the threshold and ~cale factor described in 33 the text 34 3 -- Indicate~s i nh i hAtory CG ~_ eion-- +
indicates mixed excitatory and inhihatory WO 91/~S PCT/USgO/O~K8 1 TARTF Il 2 PRQP~RTIF~ O~ RCI MODFT. ~r~ F~

3 Major 4 Repertoire Property Afferents FfferentS
"LGN" ON-Center, Input Array "4C~"
6 Off-Surround 7 n4c~" Orientation ~LGN~ ~4B-Dir"
8 Selectivity ~4B-Orient~
9 "4B-Dir" Directional "4C~ Comparator Selectivity Direction 11 "48-Orient~ Orientation "4C~n n4B-Term"
12 Selectivity Rentr. Confl.
13 Occlusion 14 "4B-Term" Orientation ~4B-Orient~ Wide Angle and polarity 16 of line 17 terminations 18 "4B-Term~ Same as Occlusion Wide Angle 19 (reentrant) n 4B-Term"
Reentrant Responds to Occlusion Occlusion 21 Conflict crossings "4B-Orient" Angle 22 of real and Wide Angle 23 illusory 24 contours Wide Angle Bro~en~ ~4B-Term~ Term. Disc.
26 orientation Common Term.
27 selectivity 28 Common Term. Detects Wide Angle Term. Disc.
29 Detector lines with orientations 31 within 90-32 terminate 33 at a common 34 locus.

WO 91/060SS PCT/USgO/OS868 1 Ma;or 2 ReDertoire Pro~ertv Afferents FfferentS
3 Repertoire Major Afferents Efferents 4 Property Termination Responds to Wide Angl- Occlusion 6 Discontin- lins ter- Comoon Term.
7 uity minations 8 consistent g with an occlusion 11 boundary 12 Direction Responds to Direction Occlusion 13 Discontin- different (motion) 14 motion con-sistsnt with 16 an occlusion 17 boundary 18 Occlusion ~esFon~ to Term. Disc. Reentr. Confl.
19 real con- Occl. Confl.
tours, ~4B-Orient.
21 occlusion 22 borders, 23 and illusory 24 contours occlusion Responds to Dir. Disc. "4B-Term"
26 (motion) occlusion (reentrant) 27 borders 28 based on 29 structure-from-motion 31 Occlusion Generates Re~.~ant Occlu~ion 32 Conflict illusory Conflict 33 contours be-34 tween con- ~' flicting 36 points found WO 91/060SS PCr/US90/OS868 1 Major 2 ReDertoire Property Afferents Ffferents 3 by Reentrant 4 Conflict repertoire 6 Comparator Compares ~48-Dir~ Direction 7 motion in 8 adjacent 9 directions Direction Direction Comparator n4B_Dirn 11 Selectivity Dir. Disc.

12 Repertoire Connectivitv Details 13 "LGN~ 1 pixel excitatory center, 3x3 pixel 14 inhibitory a~ ~ ow-d -"4C~" 5xS connection matrix for Horiz. ~
16 Vertical orientations: 7x7 matrix for 17 obliques. Temporally delayed inhibition 18 in null direciton.
19 "4B-Dir" Two types: first type gets excitatory inputs from 2 adjacent "4C~" units, one 21 input is temporally delayed and displaced 22 in preferred direction. Also receives 23 inhibition from 5x5 units in the "4C~"
24 repertoire selective for null direction.
Second type sums inputs from 3 such units 26 whose orientation preferences span 90-27 and reentrant connections from Direction 28 repertoires.
29 "4B-Orient" Excitatory inputs from 4 adjacent - colinear units in ~4C~" and inhibition 31 fro~ su~o~.d. All 4 excitatory inputs 32 are arequiared to fire unit.
33 "4B-Term" ~ocal circuit eXcited by ~4B-Orient~ unit 34 and inhibited by adjacent ~4B-Orient~.

WO 91/060SS PCr/USgO/OS868 1 ReDertoire Connectivity De~ails 2 "48-Term" Similar to 4B-Term but on connections 3 (reentrant) from Occlusion repertoires inst-ad of 4 ~4B-Orient~.
Reentrant Excitatory connections from Occlusion and 6 Conflict from the ~4B-Orient~ repertoires in the 7 three most nearly orthogonal directions.
8 Inhibitory inputs from orthogonally 9 oriented "4B-Orient~ repertoires and from Wide Angle repertoire.
11 Wide Angle 1 excitatory input from 3 "4B-Term~
12 repertoires with adjacent directional 13 preferences (e.g. N, NE, ~ NW) and by 14 "4B-Term" (reentrant).
Common Term. Connections from 2 Wid- Angle repertoires 16 Detector with adjacent orientation preferences.
17 Both inputs required to fire unit.
18 Termination Connections from linear strips (2x87) of 19 Discontin- units in each of 2 wide angle rçpertoires uity with opposite polaritie~, and from a 21 single unit at CG.~ ing position in 22 one of the two Wide Angle repertoires.
23 inhibitory connection from Common 24 Termination.
2s Direction Similar scheme to Ter~ination 26 Dlscontin- repertoires but with input~ from 27 uity Direction repertoires. Time constant of 28 voltage decay is longer to allow short-29 term persistence of ~spgn3~9 to moving objects.
31 Occlusion 60 bipolar excitatory connections from -32 units distributed along a lin- in TD
33 repertoire. Single inhibitory connection 34 from Occlusion_Conflict rep rtoire.
Excitatory connections from ~4B-Orient~.

WO 91/060SS PCr/US90/OS868 1 Repertoire Connectivity Details 2 Occlusion Similar connectivity to Occlusion 3 (motion) repertoires on inputs from Direction 4 Discontinuity repertoire~.
Occousion Similar connectivity to Occlusion 6 Conflict repertoire but on inputs from Reentrant 7 Conflict repertoire~.
8 Comparator 5x5 unit excitativity and 5x5 unit 9 inhibition from "4B-Dir~ repertoires with adjacent preferred directions. For each 11 direction, there are 4 comparator units 12 (e.g. N vs NE, N vs NW, N V8 E, and N vs 13 W).
14 Direction Sums inputs from 4 comparator repertaires with same preferred direction: 3 inputs 16 needed to fire unit.

WO 91/060S~ PCr/US90/0~868 l Oculomotor SubsYstem 2 The vision repertoire used 3 for oculomotor control is known by the abbreviation 4 VR. It comprises cell types RV and RI. RV cells have connections from the input array IA and geometrical 6 connections from both RV and R~ c-118.
7 Th- following abbreviations 8 are used to describe the origins of the various g connections:
IA Input array ll VJ Xinesthetic 12 sensors from arm joints 13 VW Kinesthetic 14 sensors of visual sensor motions VT Touch 16 Vn Value scheme n 17 All others are names of 18 repertoires and cell types also described in this 19 section.
The RI cell type receives connections 21 only from RV cells. The VR seconA~ry repertoire 22 therefore contains both excitatory and inhibitory 23 layers of neurons. The inhibitory cells act to 24 stabilize the overall level of activity in~epen~t of the size or brightness of the stimuli falling on the 2 6 input array.
27 A colliculus-like repertoire SC is also 28 defined to act as an intermediary between the visual 29 repertoire VR and the oculomotor repertoire OM. It comprises cell types M2 and IN. ~2 cells receive 31 excitatory connections from RV cells and inhibitory 32 geometrical connections from both H2 and lN cells.
33 The IN cell type receives conn~ ~ions from M2 cells 34 only. The inhibitory IN cells act to permit M2 cells to respond to only a single stimulus at any one time 36 in a "winner-take-all" fashion. Thi~ inhibition is WO 91/~X~ PCT/~iS90/

1 modulated by a falloff function which tends to favor 2 stimulus objects whose image falls toward~ the center 3 of the retina over objects located farther off-center.
4 The SC repertoire has excitatory cells connected to ocular motor neurons OM adapted to cause motion of the 6 optical sensor that visually sen~es ob~ects and 7 generates input data in response thereto.
8 Modifications of these connection~ during training is 9 heterosynaptically influenced by a value repertoire lo F0, which is arranged to respond more strongly a the 11 amount of stimulation of the central area of the 12 visual field of the visual sensor increases.

13 Reaching Subsystem 14 We now describe the repertoires csncerned with reaching movements of arms. The motor cortex MC
16 repertoire imitates motions of the single arD in the 17 preferred embodiment. It comprises cell types MF
18 (motor flexor cells) and ME (motor exten~or cells).
19 80th cell types receive excitatory connections from WD
(object vision) and XE (joint kinesthesia). The two 21 cell types are mutually inhibitory, and, in addition, 22 they are inhibited by primary touch cells, reducing 23 arm movement when an object is touched. The MC
24 repertoires drive opposing ~muscle system~" and are adapted to cause gestural motions by the arm initiated 26 by noise or by input from vision and arm kinesthesia.
27 By selecting gestural motions from an a priori reper-28 toire, these robotic control syste~s avoid the 29 necessity to make a detailed mathematical analysis of the kinematics and dynamics of robot joint ~otion and 31 to program each and every motion of each ~oint.
32 A mechanism based on the ~tructur- of the 33 cerebellum of the brain is ~sed to ~filter out~
34 inappropriate motions in such a sy~tem, l~ n~ to selection of the most useful motions. Such sy~tems ' CA 02067217 1998-09-16 .

WO 91/060~ PCr/USgO/OS868 1 will automatically optimize their motions for the 2 particular mix of motions which they are most commonly 3 asked to perform. ~y using value scheme~ that include 4 a term that reduces value in proportion to the energy consumed in driving a limb, the system can find 6 optimal motion strategi~ that minimize energy 7 consumption.
8 Output from the MC repertoire~ is pAsE~
9 to a similarly opposing pair of cell types, RF and RE, lo in a repertoire IN, which represents an intermediate 11 nucleus which corresponds in function to basal ganglia 12 in the brain. The RF and RE cell type~ form a 13 mutually inhibitory pair. This IN "intermediate"
14 repertoire is adapted to receive ~xcitatory ~ignals from the motor cortex HC and inhibitory signals from 16 the PF and PE cells of the model cerebellun, which are 17 responsible for the inhibition of ineffective gestures 18 that are initiated from time to time by the MC.
19 The model cerebellun consists of repertoires GR, I0, and PX. The ~granule 21 cell" repertoire GR comprises cell type Gl (granule 22 cells). This repertoire is adapted to correlate the 23 configuration of the arm in space (se~-e~ by 24 connections from KE) with the position of a target object (senee~ by connections from HD).
26 The "inferior olive~
27 repertoire IO comprises cell types IE and IF, which 28 receive inputs from the motor cortex, modulated by the 29 value repertoire to be described. I0 cell~ arQ
adapted to provide drive to PR cells at ~n ~arly state 31 of training, before appropriate specific connections:
32 from GR cells to PK cells have be~n sQlQctQd. The I0 33 inputs to PK cells do not in themselves carry 34 information concerning the conditions ot a particular reaching event, but rather they provid- undirected WO91/~5 PCT/US~/~K~

1 initial activity which provides a basis for operation 2 of the selective learning mechanism of this invention.
3 The "Purkin~e cell"
4 repertoire PK comprises cell types PF (associated with arm flexors) and PE (associated with arm extensors).
6 Both cell types receive mixed excitatory and 7 inhibitory connections from Gl cells, excitatory 8 connections from IO cells, and PF and PE are 9 themselves mutually inhibitory. This repertoire lo implements a portion of the cerebellum and acts to 11 inhibit inappropriate signals passing through the IN
12 repertoire.
13 Finally, signals controlling 14 the reaching motions of the arm pass from the IN
repertoire to the SG repertoire, rep~ ing spinal 16 ganglia. This repertoire also comprises two cell 17 types, SF and SE, which are mutually inhibitory. The 18 SG repertoire provides a point where motion-con~ol 19 signals relating to tracing motions and swatting motions of the arm may be combined with signals 21 relating to reaching. As in MC and IN, the cell types 22 relating to flexor and extensor motor mean~ are 23 mutually inhibitory, in order that contradictory 24 motions of flexor/extensor muscle pairs may be suppressed and not transmitted to the arm muscles.
26 A value scheme for the 27 training of reaching motions is provided by 28 repertoires HV, WD, and VALUE. HV is adapted to 29 respond to visual images of the automaton's hand, WD
is adapted to respond to visual images of objects in 31 the environment, and VALUE combines input~ fro~ these 32 two visual areas by means of overlapping mappings and 33 the use of a high firing threshold in such a way that 34 output is maximized when thè arm is near the ob~ect.
3 5 VALUE output provides heterosynaptic bias for the 36 selection of connections between WD and MC, between RE

WO 91/0605S PCI'/US90/OS868 1 and MC, and between PK and IN and also provide~
2 modulatory input to I0, as shown $n figure 14 and in 3 table I.
4 Tracing S~hsystem Touch sensors are used to 6 guide ex~loration of ob~ect~ in th- onvironment by the 7 arm, pro~lding kinesthetic ~ignal~ which are the 8 second input (along with vision) to tho cla~sification 9 couple to be described below. Under the control of this subsystem, the arm traces the edges of 11 arbitrarily shaped objects.
12 The arm assumes a 13 straightened "canonical" exploratory position (see 14 figure 13) when touch sensors signal that it has contacted an object. In this position, all joints 16 except the shoulder are immobilized, and the shoulder 17 acts as a universal joint, permitting motion~ in 18 vertical and horizontal direction~.
19 Exploratory motions are generated initially in random directions by 21 spontaneous neural activity in motor repertoire T~
22 (figure 13). These random motions are biased by touch 23 signals in two ways to produce coordinated tracing:
24 (1) Touch receptors are responsivo to varying pressure 2s across the receptive sheet at the end of the aro. A
26 pressure gradient sensed in a particular direction by 27 one of the repertoires El-E4, receiving connections 28 from TH, acts to enhance motor activity in 29 perpendicular directions, thus bia~ing the aro notions to trace along the edges of objects. (2) Whon 31 pressure decreases, repertoiro TC acts along direct 32 connections to T~ to inhibit the current direction of 33 motion and enhance its opposite, endinq to bring the 34 arm back into contact with the ob~ect when it ~~ ~Ers away.

WO 91/~S PCT/US90/~

l Categorization Subsystem 2 We now proceed to a 3 description of the repertoires involved in the 4 categorization of stimulus objects by the reentrant combination of visual and kinesthetic cells.
6 The LGN (lateral geniculate 7 nucleus) repertoire has ON and OFF type neurons, such 8 that the LGN ON neurons are adapted to respond only to 9 regions of the visual field where a central spot of light (light ON) is surrounded by a dark area;
ll conversely, LGN OFF neurons respond to a central point 12 with light OFF surrounded by a lighted area.
13 The R repertoire comprises 14 cell type FD (feature detectors), which receives input from LGN ON and OFF cells. The R repertoire is 16 adapted to respond to vertical, horizontal, or oblique 17 line segments.
18 The F~ repertoire comprises 19 cell types E2 and RX. The E2 cell type receives connections from R FD cells and has excitatory 21 reentrant connections from itself. The RX cell type 22 receives connections from E2; from RM RM (the key 23 reentry for categorization); and from RC CR trigger 24 cells. The R2 repertoire is connected to overlapping regions in the R repertoire so that E2 cells respond 26 to combinations of features in different positions of 27 the input array.
28 The MT (~motion trace~) 29 repertoire comprises cell types Ml, M2, Modifying substance, and MB. Ml and M2 cell~ pro~ide, 31 respectively, delayed and prompt r~r~lJol~-e~ to 32 kinesthetic signals from the univer~al joint, relayed 33 via primary kinesthetic repertoire KE. Hodifying 34 substance cells have excitatory ~Q'~ e_~ion~ from a common direction of motion signalled by both Ml and M2 36 cells and are inhibited by other directiona o~ motion;

WO 91/~5 PCT/~S90/O~U~

1 they accordingly respond to smooth contours of an 2 object being traced. MB cells have similar 3 connections, but with the excitatory and inhibitory 4 contributions from the delayed kinesthetic Ml cells reversed, so that MB cells respond most strongly to 6 "bumpy" contours of an object being traced. Both 7 Modifying substance and MB cells are inhibited by the 8 trigger repertoire so that motion trace output is not 9 generated during the stage of active response to an object after it has been traced.
11 The ~ repertoire comprises 12 cell types RM and RX (re-entry cells). R~ cells are 13 adapted to respond to various combinations of rough 14 and smooth contours signalled by their inputs form MT
cells. This activity builds up LTP in these cells so 16 they can be fired more easily by later input from the 17 RC triggering repertoire. RX cells receive input form 18 ~ cells, as well as reentrant input form Rz E2 cells.
19 It is the combined action of these tow inputs in RX
(as well as the symmetrical combination of R~ RX and R2 21 E2 inputs in the RX cells of the E2 repertoire) that 22 generates neural firings that signal the category of 23 an object to the output response system to be 24 described next. This output is permitted only when input form the RC trigger repertoire is also present 26 at RX cells.

27 Reiection SubsYstem 28 A triggering network is 29 provided to end tracing by detecting novelty in the R~
responses and integrating the appearance of novelty 31 over time to recognize the completion of a trace. The 32 triggering network repertoire RC comprises cell types 33 TS, TB, CR, and CE. The TS and TB layers ~re 34 stimulated respectively by smooth or rough R~ units but have long refractory periods that prevent them from WO 91/060S~ PCr/US90/OS868 1 resuming activity until some time after stimulation.
2 In such an embodiment, the tracing apparatus depicted 3 in figure 13 is omitted, and the MT repertoire is 4 equipped with inputs from an alternative visual repertoire, designed similarly to the R repertoire 6 already described, but containing feature-detecting 7 cells with larger visual fields that are capable of 8 responding to contours rather than short segments of 9 contours, and other cells capable of respon~n~ to contours that are joined or otherwise correlated in 11 various ways. This alternative visual system provides 12 inputs to the classification couple that is of a ~ 13 similar nature to that provided in the preferred 14 embodiment by the kinesthetic trace system. Thes-arrangements lead to firing in the CR layer only when 16 there is no activity in the CE layer, a situation 17 which occurs when no novel tactile features have been 18 detected by MT cells for some time. The ou~ of the l9 triggering network is coupled to Rz and R~, where it re-excites units previously stimulated during 21 examination of the stimulus. As a result, activation 22 of R2 and R~ by neural events occurring indep~n~ently 23 in the two repertoires - a so-called reentry - brings 24 about categorization. As a result, in the preferred embodiment rough-striped physical objects are sorted.
26 Repertoires ET, OP and RG
27 complete the rejection subsystem. Cells in ET receive 28 input from RX cells in both R2 and ~ repertoires.
29 These inputs are active only when triggering ha~
occurred, as just described. Various ET cells ~ n~
31 to various combinations of F~ and F~ activity, and thus 32 enable the automaton to respond to a variety o~
33 categories. OP cells receive inputs from ET, and thus 34 can be tuned to respond to one category or another by selection based on a value scheme. The RG repertoire, 36 when triggered by inputs form OP, produces an W091/~K5 PCT/US~/~&~

1 oscillatory motion of the arm which may be directed to 2 reject objects of a particular class which the user of 3 this invention might wish to have rejected, for 4 example, an object that is visually striped and bumpy.
The simulation of the 6 present invention incorporateR reentrant signaling.
7 Reentry refers to parallel and recursive ongoing 8 signaling between two or more mapped reqions along 9 ordered connections. Reentry is a mode of interconnection and signalling along such connections 11 that permits mappings between sensory signals and 12 neuronal responses to objects and events in the 13 environment to organize spontaneously. Reentry 14 further provides a means for the correlation of repr~ entations in diverse sensory modalities, 16 permitting consistent responses to be established and 17 maintained without specific programming.
18 Classification n-tuples are collection~ of n neuronal 19 repertoires joined by reentry to give classification of stimuli based on correlations of signals in all of 21 the component repertoires. Such classifications are 22 more powerful than can be accomplished by any one 23 repertoire alone because they take into account 24 combinations of features represented in the various elements of the n-;~ple. Classification n-tuples 26 could be based on data from diverse sensors that are 27 normally difficult to combine, e.g. optical, sonar, 28 radar sources.
29 The automaton is designed so that sensory signals triggered by the stimulus remain 31 distributed among multiple, functionally se~Le~ated 32 areas. Integration of these signals is achieved by 33 reentry. This controlled form of interaction through 34 reentry, as opposed to direct connection of signals from different sensory modalities to a common sensory 36 repertoire, permits each modality to retain the W09l/~5 PCT/US~/05&~

1 distinctive features of it~ responses despite the 2 possible presence of confounding ~uxtaposition~ of 3 features in other modalities. Each modality thus 4 retains the ability to distinguish stimuli to which it has a distinctive response.
6 C. Calculatina Cell Activity Values 7 A relatively small number of 8 parameters controls the properties of the simulation 9 in the preferred embodiment. Each unit is a sim-plified model neuron or cell which nonlinearly sums 11 inputs from other units. The output of a unit, which 12 generally corresponds to the average firing rate of a 13 single neuron, is given by:
14 sj(t) - ((A + G +
M)~(Is))~(D) + N + W
16 where (Greek letters are used for adjustable 17 parameters; Roman letters for dynamic variables):
18 sj(t) = state of cell i at time t 19 A = total input from specific connections = ~ cjj(sl , eE)~ c~J
21 strength of~connection from 22 input j to cell i (cjj > 0, 23 excitatory; cjj ~ 0, 24 inhibitory), ljj = index number of cell connected to 26 input j of cell i, eE ' 27 excitation threshold (sl <
28 eE ignored), k = index o3e~r 29 connection types, j = index over individual connections, 31 G = total geometrically defined input -32 ~ (sg - ec), ~ -33 strengthl~f cnnn~ ~ions from 34 ring k around cell i, gji -index number of cell 36 connected to geometrically . CA 02067217 1998-09-16 WO 91 /060S~; PCr/l,TS90/OS~K8 l defined input j of cell i, ec 2 - activity threshold for 3 geometric inputs (sg < e, ignored 4 M - total modulatory input, define~
~imilarly to G except all 6 c-lls in t~e source layer 7 are included with equal 8 w-ight~, 9 I, 5 total shunting inhibition, sum of all specific and geometric 11 inputs designated as 12 shunting inputs (shunting ~ 13 inhibition multiplies the 14 excitatory terms (A + G + M) lS and is thu~ abl- to o;~
16 any amount of excitatory 17 input to a group 18 Accordingly it is of 19 critical importance in assuring the stability of 21 repertoires), 22 D = depression 5 ~DSj(t-l) + noD(t~l) "~D =
23 growth coefficient for 24 depression, nD ~ decay coefficient for depression 2 6 When D > eD, where eD is a 27 refractory threshold, then 2 8 ~(D) is set to O for a 29 specified number of cycles, after which D is sQt to O
31 and ~(D) returns to 1 0, 32 N = noise, which may be shot noise or 33 Gaussian noise, 34 W ' decay term - nsj(t-l), and ~(X) ' sigmoidal function, approxi~ted 36 as ~(x) - 1 - 2x2 + x~

WO 91/060S~ PCr/US90/OS868 1 The entire collection of 2 terms ((A + G + M)~(I,))~(D), as well as the input from 3 each individual connection type (which may be thought 4 of as the input to a local region of a dendritic tree), must exceed a given firing threshold or it is 6 ignored. These connection typ- threshold~, e~, ar-7 modulated by long-term-potentiation (LTP) according to:
8 e"~ ~ e,~ - aLL
g L - nLL(t - l) + /~L (S~ (t-l) -k 11 where:
12 e~, = modified value of connection type 13 threshold e~
14 OL ~ LTP scaling factor, L - LTP value, QL
~ decay coefficient for LTP, 16 ~L ' ~L ' homo- and 17 hete~osynaptic growth 18 factors for LTP, eL.,e 19 homo- and heterosynap ~ c LTP
action thresholds, A~ ~ total 21 input from connection type 22 k. aL may be negative to 2 3 implement long-term 24 depression; unlike the D
(normal depression) term, 26 the LTP term may have 27 different effects on 28 different afferent 29 connection types.
This method of calculating 31 cell responses incorporate~ several advances over the 3 2 prior art. First, the MAX and MIN connection-type 33 specific constraints make it possible to design cell 34 types with multiple input classes while any one of the inputs from dominating the respo~ of the WO 91t060SS PCI'/US90/OS868 1 cell type as a whole. Suitable ad~ustment of these 2 parameters makes it possible to design cell types 3 which have properties which are a generalization of 4 the well-known electronic devices known as "AND" and "OR" gates, in which, respectively, more than one 6 input or any one input must be activ- for the unit as 7 a whole to be activated. The generalization referred 8 to here is that each single input to an ~AND" or ~OR~
9 gate is here replaced by the combined input of an entire class of connections, each weighted and 11 thresholded as given in the equation just presented.
12 Second, the provision of "shuntingn-type inhibition 13 makes it easier to design networks which are 14 intrinsically stable. Third, the depression-like term makes it possible to design automata which have a 16 selective form of attention in which response 17 automatically shifts from one stimulus to another as 18 the depression term takes effect to reduce any 19 response which is maintained for a certain length of time. Fourth, the division of inputs into specific, 21 geometric, and modulatory classes re~ces the burden 22 of computation significantly for those inputs which 23 meet the more restrictive geometric conditions of the 24 non-specific classes. (Mathematically, all three types could be expressed by the equation for the most 26 general type, namely, the "specific" connection type.) 27 In a preferred embodiment, 28 the strength of a synapse or connection from cell j to 29 cell i (denoted cjj) is modified during the course of training of the apparatus of this invention in accord 31 with the following equation:
32 cjj (t+1) 5 Cl; (t) + ~-~(c~j)-(<s~> - e~)-(m~l - e~) (v 33 - ev) R
34 where:

WO 91/060S~ PCI'/USgO/05868 1 ~ - amplificat$on factor, a parameter 2 which adjusts the overall 3 rate of synaptic change, 4 <s;> = time-averaged activity of cell i, calculated according to 6 <sj(t)> - d s~(t) + (d-l)<si(t-l)>
7 where d - damping constant 8 for averaged activity, g e, ~ amplification threshold relating to lo postsynaptic activity, 11 mij Z average concentration of 12 postsynaptic "modifying 13 substance" produced at a 14 connection made on cell i by cell j according to 16 mjj (t) = mjj (t-l) + U~-Sj - Min(T~-m~j (t-l),T~~), 17 where u~ ~ production rate 18 for m~;, T~ - decay constant 19 for m~j, T~l~ ~ maximum decay rate for m~j (m~J may be re-21 placed simply by sj if no 22 time lapse occurs between 23 activity and selection), 24 e, (k) = amplification threshold relating to presynaptic activity, 26 v(k) = magnitude of heterosynaptic input 27 from relevant value scheme 28 neurons, 29 ev = amplification threshold relating to value, and 31 R - rule selector. R may be set to +1, 32 0, or -1 in~e~enA~ntly for 33 each of the elght 34 combination~ of the signs of the three thresholded terms 36 in the ampllflcation WO 91/~K~ PCT/US90/OS868 1 function, giving a total of 2 3~ ~ 6561 possible amplifi-3 cation rules. Positive 4 values of R lead to enhancemQnt of connections 6 with corr-lated pre- and 7 po~t-synaptic activity 8 (--lection); n-gativ- values 9 of R lead to suppression of such connections 11 (homeostasis). By choosing 12 a particular rule, it i~
13 possible to simulate any of 14 a wide variety of different kinds of synApse~, with 16 properties corresponding, 17 for example, to thos- that 18 might be seen with different 19 neurotransmitters. Typi-cally, we rhoose a rule in 21 which ~ is +1 when (v - ev) , 22 0 and either of (s~ - e, ) or 23 (m~j - e~) > 0, i.e. when a 24 value signal is present, a synapse i~ strengthened when 26 both presynaptic and 27 postsynaptic cells are 28 active, but W4?Akt:-' wben 29 on- is active and the other is not.
31 The synaptic modification 32 rules used in the present invention, and represented 33 by the above equation, deviate from the prior art and 34 particularly from the well-known Hebb rule, in several significant ways: First, the heterosynaptic factor 36 (v-ev), which is tied to a value scheme, is introduced.

WO 91/060SS PCr/US90/OS868 1 This term allows the synaptic change in one part of a 2 network to be influenced by events elsewhere. It 3 allows a selective neural network to learn, as opposed 4 to merely to train. A system havinq this property can improve a performance that occurs spontaneously, or, 6 given an appropriate conditioning paradigm, it can 7 learn to keep what is presently re~ected and vice 8 versa. Second, the use of the time averaged post 9 synaptic activity s; in place of the current activity sj(t), and the use of the "modifying substance", m~, 11 in place of the current synaptic weight, cjj, make~ it 12 possible for the system to learn in the normal 13 situation in which the completion of an action, and 14 its evaluation by a value scheme, occur after the cessation of the neural activity which caused that 16 action. The si and mjj variables provide a localized 17 "memory" of neural firing condition~ so that synaptic 18 modification may be applied to connectionC that played 19 a causative role (excitatory or inhibitory) in a particular behavior after that behavior has been 21 evaluated. These delayed evaluations are important to 22 selective learning of behaviors that involve se~uences 23 of actions, for example, reaching followed by 24 grasping. Third, the uses of the rule selector 'R' work with the newly introduced value factor (v-ev), 26 providing either enhancement or repression of 27 responses associated with various combinations of 28 presynaptic activity, postsynaptic activity, and 29 value.
Operation of the Repertoires Comprisinq ~h~ Auto~ on 31 Further details of the 32 operation of the individual repertoires implemented in 33 the preferred embodiment are as follows:
34 The preferred embodiment in operation depends on motions of the visual ~ a~
36 means for target location and selection. As shown in WO 91/060S~ PCI'/US90/OS868 1 figure 11, these motions are controlled by an 2 "oculomotor" subsystem. VR, a retinal-lik- visual 3 repertoire, contains two layers o~ cells, excitatory 4 and inhibitory. It is mapped to the SC repertoire, which controls visual sensory movements. SC has its 6 excitatory cells connected directly to four 7 collections of ocular motor neurons, OM, with random 8 strengths, and with inhibitory co~ e ~ions between 9 opposi: motions. The value scheme for visual sensor motion responds weakly to light in the periphery and 11 more strongly to light in the central region, thereby 12 implementing in a simple fashion the behavioral 13 criterion "bring the sensor towards bright ~pots and 14 fixate upon them". This value schem~ provide~ a heterosynaptic input that ~odulates the modific~tion 16 of connections from SC to OM. Activity in thes-17 connections simulates the formation of a 810wly 18 decaying modifying substance. Connections, m~l, that 19 have been active during any ~ind of motion are labelled by this modifying sub~tance until it decays 21 as specified by the parameters T~ and Tl,~. Co--~e_-Lions 22 so labelled are amenable to undergo long-lasting 23 changes. As a result of activity occurring shortly 24 before centering on an object and consequent activation of the value repertoire, these cr ~ tions 26 are selected and strengthened. In this way, selection 27 acts on neuronal populations after their activity has 28 produced an effect.
29 The MC repertoire generates primary gestural mc~ion sponr~nesll~ly or in ~ pon~e 31 to sensory input from vision and arm kinesthe~ia. Its 32 output is transmitted to IN (intermediate nucleus).
33 IN sends connections via SG to four sets of motor 34 neurons, one for each joint~in the arm, organized in extensor/flexor pairs.

WO91t~K~ PCT/US~/~

l Target vision and 2 kinesthetic inputs give rise to diffuse and fast-3 changing firing patterns in GR units. Each G~ pattern 4 correlates an actual configuration of the arm in space with a target position.
6 GR units connect densely to 7 PK units. Connection frou GR to PK associate 8 positions of the arm and target with patterns 9 corresponding to primary gestures that arise fro~ MC
and reach PR via repertoire I0, which is patterned ll after the inferior olive. Activity in PK inhibits 12 activity in IN and filters out inappropriate gestures 13 from patterns transmitted to IN from MC. Thus, the 14 combined MC, GR, I0, PK, and IN network~ e~bed a lS reentrant signalling loop.
l6 The reaching subsystem has a 17 value system so that the neurons Le~und more actively 18 as the moving hand approaches the vicinity of the l9 foveated target object. These neurons receive input from two areas responsive to objects in the 2l environment and to the hand of the automaton. These 22 inputs arborize in overlapping fashion over the 23 surface of the value network. Therefore correlated 24 activity, indicating nearness of the hand to the object, is required for a vigorous L~yo~o. The 26 response of the value network increases as the hand 27 approaches the target and the degree of overlap in the 28 mapped inputs increases.
29 The value repertoire activity is carried to the IO network. Bursts in the 3l value repertoire associated with gestural motion-32 bringing the hand near the target activate I0 units 33 that have already received sybthreshold excitation 34 from MC. Thus activity in I0 ~p~n~ on rc~
activity in MC. IO activity is carried to the P~
36 cells.


1 The invention in this 2 particular embodiment allow~ connections from 3 "parallel fibers" converging on PR units to be 4 amplified when PK cells are excited from the IO due to a gesture that is in the process of being selected.
6 After repeated amplification, the ~parallel fibers~
7 are capable of exciting PK unit~ on their own, and 8 thus acquire the ability to ~preset~ the pattern of PK
9 cell activity even before a gesture is initiated in MC. The output of these PK cells is available for 11 "filtering" gestures in IN just before they happen.
12 Activity in the value 13 repertoire is also carried to MC. Active connections 14 relating the position of the visual target to particular gestures are amplified according to value.
16 Synaptic populations whose activity is associated with 17 motions closer to the object are selectively favored 18 in these modifications.
19 Once the arm has reached a particular object, tracing motion begins with the 21 objective of providing supplementary information which 22 is combined with visual signals in a ~classification 23 couple" for the purpose of categorizing the object.
24 The arm assumes a canonical exploratory position when it touchss an object, in 26 which all joints except the shoulder are immobilized.
27 This is done in the simulations to reduce the burden 28 of training the motor system to generate tracing 29 motions using all the degrees of freedom available to the model arm: in a real robot, there i8 no cAnonical 31 exploratory position and the arm is trained to perform 32 tracing in the same way that it i~ trained to p-rform 33 reAching. The arm traces the edges of arbitrarlly 34 shAped ob~ects, making fine; non-ballistic motions.
The edges are sence~ by the kinesthetic ~_ep~ors.
36 Exploratory motions are generated in random WO 91/~K~ PCT/US90/O~U8 1 directions, biased by touch signals to produce 2 coordinated tracing. That i8 to say they are biased 3 to move in direction~ parallel to edges seno6~ by 4 touch to trace along such odge~; and they are biased to change direction when the pressure drops. Tracing 6 proceeds along the edges until interrupted by a burst 7 of activity in the reentrant categorization sy~tem.
8 A~ an alt-rnative embodiment 9 vision may be used alone without the arm. In such an embodiment, the tracing apparatus depicted in figure 11 13 is omitted, and the MT repertoire is equipped with 12 inputs from an alternative visual repertoire, designed 13 similarly to the R repertoire already described, but 14 containing feature-detecting cells with larger visual fields that are capable of respon~ing to contours 16 rather than short segments of contours, and other 17 cells capable of responding to contours that are 18 joined or otherwise correlated in various ways. This 19 alternative visual system provides inputs to the classification couple that is of a similar nature to 21 that provided in the preferred embodiment by the 22 kinesthetic trace system.
23 The repertoires involved in 24 categorization comprise the following:
An LGN (lateral geniculate 26 nucleus) repertoire has ON and OFF cells. LGN ON
27 cells have excitatory center-inhibitory surround 28 receptive field structures. They are connected 29 topographically to an R network, which ~ Qnds to io vertical, horizontal, or oblique line ~egment~, and 31 thereby forms an image that emphasize~ edges of 32 objects. The R network signals to R2 which receives 33 connections from large overlapping regions in R, 34 therefore losing details of the ~ppearance of tho object, but R2 is ~._ponsive to combination~ of fea-36 tures that may be used to characterize an ob~ect.

.' wo 91/060SS Pcr/usso/oss6s R~ i8 a repertoire dealing 2 with motor patterns. In the version de~cribed, it 3 responds to two shapes, smooth and rough. Inputs come 4 from kinesthetic receptors in the touch-exploration motor system. Smooth-sensitive cells respond strongly 6 when tracing continues in a ~ingle direction and are 7 inhibited when the direction of trac- changes. Rough-8 sensitive cells respond strongly when tracing 9 continually changes direction and are inhibited when lo the direction remains constant. Cells of both types ll are provided with maximal responses for each of eight 12 principal directions of tracing.
13 A triggering network, which lg ends tracing, detects novelty in the ~ ~e-l-onse~ and integrates the appearance of novelty over time to 16 recognize the completion of a trace. Four layers are 17 implemented. The first two are stimulated by rough or 18 smooth ~ units but have long refractory periods that 19 prevent resuming activity until some time after stimulation. These units are combined in the third 21 layer, and the third layer inhibits the fourth, which 22 has a high level of spontaneous activity (noise).
23 Cells in the fourth layer thus become active, and 24 trigger the overall response of the whole system, when novel activity ceases to be detected in the first 26 layers.
27 The trigger ~e_~onse is 28 coupled back to ~ and ~. It acts there to re-excite 29 units previously stimulated during examination of the stimulus. Activation of ~ and ~ by neural event~
31 G~ Ling in~epon~ontly in tha two repertoires 32 constitutes reentry and is the decisive step in 33 categorization. Only upon coactivation of a~op.iate 34 visual ~.ou~s in R2 and corr~lated lriresth-tic ~ou~3 in ~ after a trace of the ob~-ct ha~ been co~pleted is 36 a categorical response elicited. A rough-striped W091/~ PCT/US90/O~K~

1 object generates a reflex oscillation that swats the 2 object away. The system could be trained at will to 3 recognize other categories of objects for rejection.
4 It should be understood that although the invention has been presented in detail 6 for a particular embodiment it is not so limited and 7 the full scope of protection afforded by this patent 8 is determined by the following claims.

International ClassificationG06G7/60, G06N3/04, G06F15/18, B25J9/22, G06N3/00, G06K9/62, G06N99/00
Cooperative ClassificationG06N3/04, G06K9/6267
European ClassificationG06N3/04, G06K9/62C
Legal Events
21 Apr 1992EEERExamination request
10 Oct 2006MKLALapsed