Pattern Recognition Using Chaotic Neural Networks

Reprints available directly from the publisher Photocopying permitted by license only (C) 1998 OPA (Overseas Publishers Association) N.V. Pattern recognition by chaotic neural networks is studied using a hyperchaotic neural network as model. Virtual basins of attraction are introduced around unstable periodic orbits which are then used as patterns. Search for periodic orbits in dynamical systems is treated as a process of pattern recognition. The role of synapses on patterns in chaotic networks is discussed. It is shown that distorted states having only limited information of the patterns are successfully recognized.


I. INTRODUCTION
Recently, studies of chaos in neural networks have drawn attention from workers in diverse fields [1][2][3][4].There are speculations that chaos plays an important role in processes such as pattern recognition in neural networks.However, a definitive study on the role of chaos in neural networks is still missing.The aim of this note is to present a plausible scheme by which chaotic networks can be used in pattern recognition.
First, we recall the following basic features of chaotic systems: (1) In a chaotic system many chaotic attractors may coexist.The basins of these attractors may be riddled.(2) A chaotic attractor is topologically transitive, meaning that every point in the state space can be reached from every other point through dynamical evolution.The frequency at which a randomly initiated trajectory visits the regions of an attractor depends on the dynamical properties of all of the embedded unstable periodic orbits.(3) Periodic orbits are dense in chaotic systems.(4) Chaotic systems are sensitive to initial conditions.This sensitivity implies that chaotic systems lose memory rapidly.Pattern recognition using chaotic neural net- works with such features appears counter-intuitive from the point of view of the usual pattern recognition methods in neural networks.The usual way of pattern recognition in neural networks involves two main steps: (a) the network is taught a certain number of patterns (this determines the synapses) and (b) unknown states which contain partial information of the learned patterns are Corresponding author.recognized as the fixed points reached by running the dynamics.In hard chaos, there are no stable fixed points with attracting basins.Hence, dynam- ics as such will not drive an arbitrary state to any unstable fixed point as it does in the usual neural network pattern recognition.The fact that chaotic systems have a huge collection of periodic points, each of which may be used as stored patterns, makes chaotic neural networks worth studying.
However, one has to be careful.It is possible that the dense periodic points of a chaotic system are clustered in a small region creating practical dif- ficulties to use them.The riddled basin boundaries may also cause problems.We focus on the class of systems in which there is a large number of well distributed periodic orbits, specially those of short periods.Pattern recognition in such chaotic systems can be realized if we augment the dynamics by a scheme that seeks the periodic points and vanishes after the periodic points are found.The procedure (see below), is meant to find the periodic points but not to alter their number, location or nature.

THE MODEL DYNAMICAL SYSTEM
For the purpose of demonstration, we consider the "maximum hyperchaotic system" which has recently been studied in Ref. [5].This model consists of N neurons whose states are described by Si(t), i= 1,..., N. The N neurons are connected to one another through the synapses J;.j, i, j 1,..., N. The dynamics of the states is described by Si with f(z) tanh(cz)e -/:e, where c, and are constants.Depending on J,../and the parameters ( and/3, the system presents chaotic or nonchaotic behaviors.For example, when N 4, c 3 and/3 2, the system is hyperchaotic which can be verified by checking the Lyapunov expo- nents in the phase space.For the synapses 0.01 1.50 0.03 0.01 1.60 0.01 0.10 1.00 J-0.03 0.02 0.01 1.50 (3) 0.00 1.00 1.70 0.01 all the four Lyapunov exponents of this 4-dimen- sional system are positive [5].The neural network is thus "maximum hyperchaotic".
This hyperchaotic system with N 4, c 3, 2 has infinitely many unstable periodic points embedded in a finite phase space.However, only the ones with short periods are better suited for pattern recognition.For pattern recognition, those chaotic systems that support a large number of well distributed periodic orbits of short periods should be preferable.The model system of this work is not claimed to be ideal for this purpose.We have chosen this model to show that the process of pattern recognition by a chaotic system is not affected by its number of positive Lyapunov exponents.Since there are no attracting periodic points, it is unlikely that randomly chosen initial state variables will exactly match those of a periodic point for the dynamics to end in a persistent periodic behavior.Thus for pattern recognition, we need to create "virtual basins of attraction" of the periodic points by augmenting the dynamics in a way that does not change the number, location or nature of the periodic points.This is easily done by an algorithm that finds the periodic point that lies in the nearest vicinity of a given initial point.All the states in this virtual basin of attraction will be identified as the pattern represented by this periodic point.If a periodic point cannot be reached from an initial state, then that state will be categorized as "I do not know".There are algorithms in the literature for finding periodic points of dynamical system.For the purpose of illustration, we use here the classic Newton's root finding method (NRFM) [6].Using NRFM with N=4, c=3, /3=2, we have found 25 fixed points of Eq. ( 1) (roots of Si- f(._lJiSj(t))-O,i-1,...,4).These fixed points are denoted by the vectors Yi- ./ ,j-1,2,3,4are the four fixed-point values of the state variables for the ith fixed point.Table !contains 3 of these fixed points correct to four significant figures.All the fixed points were generated by a double precision routine.Twelve other fixed points are found by changing the signs of these 12 fixed points (except Ya).These fixed points are used below for pattern recognition.We have also found a large number of periodic points of higher periods but they are not discussed in this work.

III. PATTERN RECOGNITION
The phase space, in the presence of NRFM, is visualized as disjoint regions of fixed points with virtual basins of attraction.When the dynamics is run in this space with an arbitrary initial state, one of the three things happens: (a) the state evolves into a known fixed point and the pattern is recognized; (b) no fixed point is reached (this is the "I do not know" behavior of the network); (c) a new fixed point different from the known ones is found.This new fixed point is added to the memory.In this way all the fixed points that are accessible by the algorithm can be found if the initial list of fixed points is not complete.One may say that the phase space has been partitioned into regions of dynamic persistence (fixed points with virtual basins).
We now discuss the pattern recognition by considering the fixed points as the patterns them- selves.Our task is to retrieve the fixed points.The initial state, e.g.near 2 (denoted by Y2 y2, y22, Y, y42)), is defined by Y2 Y2 q-e(R)r, ( where, e(R) is a unit vector whose direction varies according to a random number R and r is a real number.The second term in (4) thus gives the distortion of the fixed point Y2 in the form of a 4- dimensional hypersphere with radius r. Figure illustrates statistically the retrieval rates for the fixed points at various radii.The numbers on the concentric circles are the ratio (percent) of number of successful recognitions to the total probes for recognizing the fixed points.First, the retrievability depends on the noise (radius r) determining how far Y2 is from Y2.Then, it depends on the patterns.For example, the retrievability for r 0. This is a reflection of the location of the fixed points and their virtual basins generated by the NRFM.It may be reminded that the synaptic connection matrix of our model is asymmetric.We would like to stress that it is the root finding method (in this paper NRFM) that determines the fixed points and retrieves a fixed point from a state which has only partial information on it.Without using this approach, the fixed points cannot be found, although we know that they exist.The scheme presented above for pattern recogni- tion in chaotic systems is different from that used in nonchaotic models.In the usual nonchaotic models patterns are stored in synapses (e.g.[7][8][9]) such that the dynamics alone drives an initial state having partial knowledge of a pattern to a fixed point (pattern).For chaotic networks, we need an additional algorithm such as the N RFM to drive a state to a fixed point (pattern).Thus, with the help of schemes such as the NRFM, chaotic systems can

FIGURE
The retrieval of the fixed point (a) 1" (b) Y2; (c) Ys; (d) 10" The radii of the circles from inner to outer are 0.1, 0.2, 0.3 and 0.4.
be used for pattern recognition.Compared to non- chaotic systems, the role of synapses on patterns is less direct.At this stage, we can only say that the synapses (along with other parameters) influence the number and locations of the distinct fixed points which we have considered as patterns.Search for fixed points (periodic orbits) in dynamical systems is treated in this work as a process of pattern re- cognition.One advantage of using chaotic systems for pattern recognition is that the storage capacity can be increased enormously.With suitable higher dimensional chaotic systems pattern recognition using chaotic systems may be very useful for practical purposes.In this paper, we have catego- rized the fixed points of our model system as patterns.The question of mapping a given geo- metric shape into the fixed points remains to be addressed.

Call for Papers
This subject has been extensively studied in the past years for one-, two-, and three-dimensional space.Additionally, such dynamical systems can exhibit a very important and still unexplained phenomenon, called as the Fermi acceleration phenomenon.Basically, the phenomenon of Fermi acceleration (FA) is a process in which a classical particle can acquire unbounded energy from collisions with a heavy moving wall.This phenomenon was originally proposed by Enrico Fermi in 1949 as a possible explanation of the origin of the large energies of the cosmic particles.His original model was then modified and considered under different approaches and using many versions.Moreover, applications of FA have been of a large broad interest in many different fields of science including plasma physics, astrophysics, atomic physics, optics, and time-dependent billiard problems and they are useful for controlling chaos in Engineering and dynamical systems exhibiting chaos (both conservative and dissipative chaos).We intend to publish in this special issue papers reporting research on time-dependent billiards.The topic includes both conservative and dissipative dynamics.Papers discussing dynamical properties, statistical and mathematical results, stability investigation of the phase space structure, the phenomenon of Fermi acceleration, conditions for having suppression of Fermi acceleration, and computational and numerical methods for exploring these structures and applications are welcome.
To be acceptable for publication in the special issue of Mathematical Problems in Engineering, papers must make significant, original, and correct contributions to one or more of the topics above mentioned.Mathematical papers regarding the topics above are also welcome.
Authors should follow the Mathematical Problems in Engineering manuscript format described at http://www .hindawi.com/journals/mpe/.Prospective authors should submit an electronic copy of their complete manuscript through the journal Manuscript Tracking System at http:// mts.hindawi.com/according to the following timetable: 2 is as high as 0.99% in Fig. (a), while it is only 8 1% in Fig. (c).