非線形結合振動子系における位相パターン情報処理に関する研究

(1)

九州大学学術情報リポジトリ

Kyushu University Institutional Repository

非線形結合振動子系における位相パターン情報処理に関する研究

松野, 哲也

https://doi.org/10.11501/3123166

出版情報：Kyushu University, 1996, 博士（工学）, 論文博士

(2)

(3)

/

Study of Information Processing of Phase Pattern in Nonlinear

Coupled-Oscillator System

Tetsuya Matsuno

Kyushu University

Graduate School of Information Science and Electrical Engineering Department of Electronic Device Engineering

ftffl��bMgM��M il-T-Tr\-1 �I�W.�

1996

�fiX: 81F

(4)

Abstract

Nonlinear dynamics in a system of coupled phase-oscillator is studied. The system treated in this study is a simple model which is expected to be analyti

cally tractable and is for aiming at extracting certain essentials in phase-dynamics observed in biological neural networks. One of the significant features of neural networks is their capabilities of learning, which means adaptively changing of variable parameters such as connectivities among neurons. The parameters char

acterizing the coupled-oscillator systems are natural frequencies and coupling pa

rameters. It can be considered that changes of the connectivities among neurons in learning processes are corresponding to changes of the natural frequencies and the coupling parameters in the model system of coupled oscillators.

Here, we treated three kinds of coupled-oscillator models, one of which is ar

ranged for learning temporal signals by changing distribution of natural frequen

cies of oscillators, one is for learning input-output relations by changing coupling strengths among oscillators, and the last one is the associative memory system in which phase patterns are embedded into coupling strengths. The last system is not for simulating learning processes, but for investigating effects of coupling nonlinearity (coupling amplification factor) on the retrieval dynamics.

The systems concerned here are composed of phase oscillators of the discrete

time version, i.e., composed of circle maps.

In the case of the system arranged for learning temporal signals, it was ob

tained numerically that the system exhibited macroscopic periodic behavior as the output of recognition, which became chaotic when unlearned temporal sig

nal was applied to the system. It was also found that the macroscopic output changes its complexity by decreasing the similarity between the learned signal and the input signal unlearned.

In the case of the system arranged for learning input-output relations by changing coupling strengths, it was found that the input (phase pattern)-output (coherency) relations specified were realized by the modified Boltzmann-machine learning rule. The learning mechanism was described by the thermodynamic method. We found the self-organized interpolation characteristics, which made the system possible to response to unlearned inputs adequately. It was also found that the distribution of natural frequencies induced the chaotic response, which could be regarded as the indication of that the input signal was unlearned.

In the case of the associative memory system, the statistical mechanical method described effects of the natural frequency distribution on the storage capacity and a stationary retrieval state. The dynamical statistical analysis pro

vided the description of retrieving processes in terms of a set of reduced macro

scopic equations, which was relevant in relatively wide regime in the parameter space spanned by the coupling nonlinearity and the storage ratio. The enhance

ment of the storage capacity by the coupling nonlinearity can be explained an

alytically and confirmed numerically. It was found numerically that a retrieval process was oscillatory and chaotic when the coupling nonlinearity was sufficiently large.

(5)

Conclusions

79

0 Acknowledgements

83

Appendices

A Supplement for Introduction

84

A ·1 Physiological Background . . . 84

A· 2 Circle Map and Coupled Circle Maps 85

B

Details of Calculations in Chapter

3 88

B ·1 Analysis of the Learning . . . 88

B·2 Definition of the Distance and the Orbit 90

C Details of Calculations in Chapter

4 93

C·1 Stationary Analysis . . . 93 C·2 Derivation of the Effective Frequency Distribution 97 C·3 Derivations of

FfJ(m)

and

GfJ(m)

. . . 99 C·4 Time Evolution Equation of

0"2(t)

. . . 99 C·5 Macrodynamics with the Distribution of Natural Frequencies 104

0 References

105

(7)

1 Introduction

Various kinds of spatio-temporal patterns are often observed in different kinds of nonlinear spatially extended systems1-3)

(

or nonlinearly interacting many-body systems

)

. A collective oscillatory phenomenon in such a coupled-system is one of the most interests in the fields of physics, chemistry and biology.4)

A biological neural network in animal brain can be regarded as a nonliner coupled-oscillator system. Recent studies5-21) suggested that oscillatory and chaotic electrochemical activities in neurons and brains were playing an important role for information processings

(

see Appendix A

·1).

The deterministic chaos, 22) which will be simply referred to as chaos hereafter, could be an important key concept for understanding complicated dynamical behavior in n?nlinear systems, such as animal brains.

1.1 Significance of the Present Study

The aim of the present study is:

1)

to construct a pattern recognition and learning system using coupled oscillatory elements,

2)

to clarify a dynamics in a certain class of high-dimensional nonlinear oscillatory systems

(

to characterize the dynamical structure on a high-dimensional torus

)

_,

3)

to understand essential features of fundamental mechanisms of oscillatory

(

_and

chaotic

)

information processings in animal brains.

Recent electrophysiological studies concerning visuaP2· 13) and olfactory per

ceptions14· 15) in animal brains showed that synchronizations and chaotic oscil

lations in neurons could play an important role for recognition and learning of stimuli from external world.

An adequate simple model is quite useful to understand fundamental mech

anisms of the oscillatory information processings. Here, a circle map23-26) is adopted as an elementary interaction among oscillators which constitute a math

ematical model of the nonlinear coupled-system. A circle map is a discrete-time version of the phase-oscillator

(

_rotator

)

model,4) which is one of the simplified models for describing oscillating quantities under perturbational forces from other oscillators

(

see Appendix A·2

)

_.

Some simplified coupled-oscillator models with capabilities of learning and recognition have been proposed. 27-32) Some numerical studies have been carried out by using very specific models strongly reflecting the specific biological struc

ture concerned. 33-35)

However, there have not been any analytically-tractable coupled-oscillator models which can learn a mapping from input of a phase pattern to output of coherency of phases of oscillators. That is, self-organization processes and characteristics of such input-output relations in coupled-oscillator systems have scarcely been studied in detail until present, although there are many kinds of investigations for systems of coupled oscillators with fixed parameters which do

(8)

not change their values. Hence, we investigated dynamics in certain types of coupled-oscillator systems in which internal parameters are varied adaptively by the learning rules introduced in this study. We adopted the natural frequencies as the variable parameters in the system designed to learn periodic temporal pat

terns, while the coupling strengths were adopted as the variable parameters in another type of the system designed to learn input-output relations supervised.

Various kinds of nonlinear oscillatory dynamics including complex

(

or chaotic

)

behaviors were found in the systems concerned here.

Furthermore, a statistical mechanical analysis clarified the mechanism of macrodynamics of retrieving phase patterns in the system constructed as the associative memory model. Phase patterns to be memorized were randomly spec

ified and embedded into the coupling parameters by obeying the coupled-oscillator version of the Hebbian rule. In this study, we derived the set of time evolution equation of macroscopic variables in a system composed of oscillators. It was found that the introduced amplification factor for coupling strengths induced complex oscillatory retrieving dynamics, the high-dimensional chaos, clustering, and more complicated nonlinear behavior.

Here, we would like to emphasize our standpoint of the present study. The important feature of the model treated here is its simplicity, which will make an

alytical

(

thermodynamical or statistical

)

study possible, while this simple model can exhibit highly complex behaviors when the nonlinearity of the system is suf

ficiently strong. This simple nonlinear model must be a good tool to investigate complex behaviors because there exist both the rich complex dynamics in the strongly nonlinear regime and the analytical tractability in not-so-strong nonlin

ear regime. The above will help us to establish the basis or regime

(

^{in which}

analytical descriptions are relevant

)

as the starting point from which we can em

bark on making systematic study of the complex dynamics towards the strongly nonlinear regime in the phase

(

^state

)

space spanned by parameters characterizing behaviors of the system concerned.

1.2 Reviews of Previous Works

1.2.1 Coupled-Oscillator Models

Various spatio-temporal dynamics can be observed in differnt kinds of physi

cal, chemical and biological systems under the condition far from equilibrium or far from linearity.1-3) Some of the systems exhibit oscillatory behavior in an ade

quate regime of control parameters; i.e., a certain nonlinearity makes the system bifurcate from a simple stationary equilibrium state to oscillatory stable states.

In this case, it is empirically known that a certain perturbative consideration can simplify a basic equation describing the system.4) One of the simplified resultants is the coupled-oscillator model.4)

There exist many intensive studies on a system of many coupled-oscillator models in the field of physics. There are studies on a system of coupled phase-

2

(9)

oscillators (sometimes called phase-rotators) with mean field coupling,36-42) with various types of local couplings,43•4 4) with random coupling,45•46) with external fields 47) and with time-delay.48) There are studies on a system of coupled limit

cycle oscillators,49-51) a system of relaxation oscillators52) and a system of pulse

coupled oscillators. 53) There exist some studies explicitly aiming at modelling neural networks: the type of phase-oscillator27• 28) and the type of limit-cycle oscillator. 29-31• 5 4• 55)

In this study, however, we are focusing our attention only on systems composed of a lot of oscillators, although there exist some studies aiming at modelling animal locomotions (or a certain neural network called the central pattern generator which makes the locomotions possible) by using a simple mathematical model composed of a small amount of (a few or several) coupled oscillators. 56-58)

In most cases, a system composed of N (equivalent) oscillators ususally have been described by a set of coupled differential or difference equations given as follows:

�I i ( t) ⁼F ( 11 ( t)' ... ' IN ( t); ¢>1 ( t)' ... ' ¢> N ( t); JL)' i ⁼1 ' ... ' N' ( 1.1)

�</>i(t) ⁼ G(11(t),···,IN(t);</>1(t),···,¢>N(t);JL), i ⁼1,···,N, (1.2) where li(t) and </>i(t) are amplitude and phase of an i-th oscillator, respec

tively. The symbol � denotes differential operator with respect to continuous

time (dfdt), or difference operator with respect to discrete-time (t: integer), e.g.,

�</>i ( t) ⁼ </>i ( t + 1) ^-

</>i

( t). Functions F and G are reflecting characteristics of a system concerned. A symbol JL denotes a set of control parameters which determins the dynamical behavior of the system.

The above equations can be reduced to a set of closed coupled equations concerning only phases when time-variances of amplitudes of the oscillators can be negligibly small in adequately introduced coordinate systems. Moreover, in the case that the effect of amplitudes of oscillators on the dynamics of their phases is negligibly small, (in other words, in the case that decoupling condition between phases and amplitudes can be assumed to be fairly satisfied, ) the concerning system may be given by

N

�<l>i(t) ⁼Wi +

�

1<ij9( <l>i(t), </>j(t); 'Pij), i ⁼1, ... 'N, (1.3) j=1

where Kij represents a coupling strength or interaction force exerting from the j

th oscillator to the i-th oscillator. A parameters 'Pij, together with Kij, determin the coupling characteristics in the system (see Fig. 1.1). A coupling function g is 27r-periodic with respect to

<Pi

and <l>i· A parameter

wi

is a natural frequency of the i-th oscillator. The system described by eq. (1.3) is called a coupled phase-oscillator model. 4)

It should be noted that a discrete-time model cannot be considered as an ap

proximation for a continuous-time model, especially in strong nonlinear regime.

Hence, the discrete-time model had better be considered as a fundamentally dif

ferent system rather than the approximation. In the language of the nonlinear

(10)

Network of Oscillators

j- th oscillator /�

Interaction

Kij g( (jJ ^i, (jJ _i _{; qJ} ij)

Fig. 1.1: Schematic illustration of a simple coupled-oscillator model. Variables

¢>i

^and

¢>i

are the phases of the i-th oscillator and the j-th one, respecively. The parameter

Kij

reflects the coupling strength of the interactive force exerting from the j-th oscillator to the i-th oscillator. The parameter

'Pij,

together with

Kij,

determins the coupling character through the coupling function g.

4

(11)

dynamical theory, the discrete-time model should be the model which is obtained by introducing the Poincare plain which cutts the continuous trajectory corre

sponding to the continuous-time dynamics of the system concerned.22) From the practical point of view, discrete-time models have great advantage especially for numerical investigations of strong nonlinear dynamics. Hence, many researchers use a discrete-time model

(

often called a map system

)

in numerical studies, al

though there is some ambiguity in a mathematical relation between the model and the differential

(

continuous-time

)

equation describing the real physical system under consideration.

The coupled-oscillator models can be classified according to their structures of coupling between elementary oscillators. The coupling structures are classified into categories of the global coupling and local coupling. The mean-field coupling and the random coupling are examples for the global coupling. The nearest

neighbor coupling is a example for the local coupling. Moreover, symmetry is an important factor for coupling, i.e., there exist many symmetric coupled models especially in the filed of physics because interactions between elements such as spins of electrons and atoms by physical forces are symmetric in general. Fur

thermore, the symmetry is often assumed for mathematical simplicity, although couplings between neurons could be asymmetric in general cases.

The symmetric assumption here means

(1.4)

which represents a relation between a force exeriting on the i-th oscillator from the j-th oacillator and a force exerting on vice versa, and which is conceptually corresponding to the Newton's kinetic law of action and reaction.

In many cases in studies on oscillatory systems using phase-oscillator models, the coupling function g is assumed as

(1.5)

The above coupling function can be considered as the simplest one in the class of 21r-periodic functions. In this study, we will also adopt the above function for simplicity. The effect of higher harmonics of a coupling function on nonlinear dynamics in not studied here, although it may be important in certain cases.59)

1.2.2 Neural Network Models

There exist a lot of kinds of neural network models60) for pattern recognition, learning, associative memory, motion control, self-organization and so on. In these network models, the so-called linear threshold gate is introduced as a typical basic element. The i-th element, corresponding to a neuron, receives N input signals

(

^x1,

· • �, XN)

with specified synaptic weights

(Kil, ···,KiN).

Here, let input signal

Xj

take only '+1' or

'-1'

for simplicity. The element emitts an output signal

Yi

⁼ +

1

only when a value of a summation of the input signals with the weights

(12)

Kij

exceeds a certain threshold value

hi;

Otherwise, it emitts

Yi

input-output relation of the i-th element can be simply given by

N

Yi

== sgn[

L f{ijXj -hi ], j=l

-1.

The

(1.6)

where sgn[ ^·

]

is the sign function. There exist some modifications60) of the above element. For example, there are a multi-value (three states or larger) element, an analog element (it is a limitting case given by taking infinite states in a multi

value element), a stochastic element (including effect of noise), an element of an arbitrary transfer function (instead of the sgn), and so on.

Conventional elements like the above versions do not have oscillatory nature.

Hence, network models composed of the above non-oscillatory elements exhibit relatively simple dynamical behavior. For example, in a conventional model of associative memory, the state of the system converges to a stable fixed point corresponding to a memorized pattern to be retrieved. Actually, the dynamical structure of the retrieving process is not so simple, and careful statistical consid

eration is necessary.61)

Network models composed of elements which are given by eq.

(1.6)

or its modified versions can oscillate if an adequate interaction with time-delay is in

troduced between the elements. However, oscillatory behaviors of this kind can be regarded as a side-effect which is not included in the purpose of the present study.

1.2.3 Remaining Open Problems

Unclarified points concerning oscillatory information processings are as fol

lows.

1: Concerning the information processings of pattern learning and recog

nition, there are some studies for reproducing observed oscillatory behavior in animal brains in terms of nonlinear oscillatory dynamics. 32-35• 27-29• 31) However, considerations on the relation between information processing and oscillation are not sufficient to show advantages of oscillatory information dynamics compared to non-oscillatory information dynamics operated by e.g., the linear threshold gate as a neural element.

2: The macroscopic oscillation, synchronization and especially the high

dimensional chaotic behavior are the main features of the dynamics observed in many-body strongly nonlinear systems composed of coupled oscillatory elements, such as biological neural networks. In spite of that, the essential role of chaos in animal brains has not yet been clarified by means of neuro-physiological experi

ments. There are not any theories that can show how to do neuro-physiological experiments to find explicitly the essential meanings of the neuro-chaos.

3: In a coupled-oscillator system as a neural network model, optimal learning rule or optimal embedding rule for phase-patterns is unknown. The dynamics of

6

(13)

learning of spatial or temporal phase-patterns have not yet been systematically investigated. There is no analytical study on nonlinear dynamics in coupled

oscillator system as an associative memory machine.

4: In order to improve the above unsatisfactory situations, it is necessary to investigate the detailed dynamics in strong nonlinear coupled oscillatory systems with arbitrary configurations of coupling types. However, we have few knowledge on high-dimensional strong nonlinear oscillatory systems. Whereas there exist intensive studies on coupled nonlinear

(

_chaotic

)

maps,62) one can find only one study on coupled circle maps with the mean field coupling.63)

1.3 Outline of the Present Paper

In this study, we investigated the three kinds of coupled-oscillator systems as a first step of understanding the essential features of oscillatory information processings in biological neural network systems. These three versions were the system for learning and recognition of temporal pattern, that for learning and recognition of spatial static pattern and that for associative memory. Two kinds of learning rules was applied to the first and the second versions of the systems, respectively, for studying learning dynamics and the capability of pattern recog

nition in the resultant systems. A statistical analytical method was applied to the third version of the system of associative memory machine to clarify retrieving processes of memorized phase patterns and to evaluate the storage capacity and information content.

The chosen model was a discrete-time type because of the advantage in study

ing strong nonlinear dynamics, e.g., strong nonlinear dynamics such as high

dimensional chaos can be relatively easily generated by the discrete-time systems.

The coupling function described by eq.

(1.5)

was adopted for mathematical sim

plicity.

The present paper is organized as follows for describing the results of the present study.

Chapter 2 deals with the first version which is designed for learning a temporal signal

(

temporal phase pattern

)

by adaptively changing a distribution of natural frequencies of elementary oscill.ators. All the coupling strengths between oscilla

tors are fixed to be equal. It is shown that the system has capability of learning a periodic signal, while the complexity of the oscillatory response increases with decreasing similarity between the input signal pattern and the learned pattern. In other words, there is a kind of bifurcation with increasing correlation dimension, which reflects the effective degrees of freedom of the system concerned when the similarity is regarded as a bifurcation parameter. It can be regarded that the system has an ability to recognize the distance between a learned pattern and an input pattern by the complexity of the oscillatory output.

Chapter 3 deals with the second version which is designed for learning static spatial phase pattern by changing the coupling strengths between oscillators using a modified version of the Boltzmann-machine learning rule. The effect of the

(14)

distribution of natural frequencies on the dynamics is also discussed. The desired output is the coherency of a group of oscillators which are defined as an output unit. The learning process changed coupling parameters so as to establish the input-output relation specified. In the case of that there exists the distribution of natural frequencies, the system which had completed the learning exhibited oscillation of the coherency and increased its complexity by increasing the distance between a learned pattern and an input pattern.

Chapter 4 is concerned with a statistical mechanical analysis for the third version: a system of associative memory. Here, randomly specified phase pat

terns were embedded into coupling parameters so that the system may work as an associative memory machine. A parameter, which is a global amplification factor multiplied to all the coupling strengths, was introduced for investigating the effect of the coupling nonlinearity on the macroscopic dynamics in the sys

tem concerned. The statistical method provided a set of time evolution equations, from which a storage capacity was obtained. It was shown that the storage capac

ity was considerably enhanced by the amplification factor. It was also shown that the amplification factor induced macroscopic chaotic oscillations and clustering behavior. The effect of distribution of natural frequencies of the oscillators on the storage capacity and on the behavior of the system is also discussed theoretically.

Chapter 5 summarizes the resultants, clarifies the significance of the present study, and we will discuss about unclarified points remaining as future tasks.

Speculations for engineering applications will be presented.

Appendices provide supplements for introduction and detailed mathematical calculations leading some important results.

8

(15)

2 Learning and Recognition of Periodic Tem

poral Signals in Coupled Oscillators

As a basis of understanding nonlinear cooperative complex dynamics in neural systems, which have been shown, e.g., the olfactory bulb of a rabbit14• 15) and the primary visual cortex of a cat12• 13), it may be important to clarify the fundamental character of the behavior of the oscillatory system composed of many elements with the input of temporal signals. It is also important to provide a simple model of oscillatory and complex cooperative phenomena in neural systems. For this purpose, here we used the coupled oscillators whose behavior is described only by phases of the elements.

The model adopted here is a discrete-time version of the phase oscillators with mean-field coupling. This model receives an external temporal input, and behaves in obeying a learning rule which changes adaptively the distribution of the natural frequencies of the elementary oscillators.

In biological neural systems, there exist many kinds of variable parameters, which are adaptively changed by learning processes, and which can be expected to be corresponding not always to the change of coupling strengths among os

cillators but also to the change of natural frequencies of the oscillators in the simplified model which will be treated here. All of the conventional studies on neural models have been devoted to investigating dynamics dependent only on the variable coupling strengths. The effect of the adaptively-changing of distribu

tion of the natural frequencies on the dynamics has never been studied, although there exist many studies about the effect of the distribution itself on the dynam

ics. The present study is the first trial in which the learning rule, which induces the change of the natural frequencies, is incorporated to the system of coupled oscillators.

It is found that the behavior of the mean field, which is the order parameter or the coherency, is periodic when the input signal to the system is sufficiently similar to the learned signal, where the signal has previously been applied to the system to learn. The order parameter oscillates randomly or chaotically when the input signal is far from the learned signal, and the degree of freedom of the behavior, which is measured by the correlation dimension, increases by decreasing the similarity between the input signal and the learned signal.

Note that we will use the term 'learning' in a wide sense in this chapter. The learning here means that the changing of the distribution of natural frequencies so that the 'high-coherency regular state' (in which phases of oscillators are almost aligned and behave regulary) will be realized when a certain periodic temporal signal (in which its frequency is specified) is applied as an input to the system concerned. The learning rule is somewhat intuitively introduced, although the rule works, as will be shown in later sections. The learning rule (as a general term) should be introduced by means of minimizing a certain estimate function describing distance between a desired input-output relation and a real one. A certain kind of statistical or thermodynamical analysis may become relevant if the

(16)

estimate function for learning is adequately introduced. However, to construct the estimation function for the coupled-oscillator system treated in this chapter is belonging to a future task, at present.

2.1 The model

The explicit expressions of the model investigated here is coupled

N

phase oscillators as follows:64)

</>;( t

+

1)

⁼

</>;( t)

₊

!1;( ^t)

+

� R( t) sin[27r( <p( t) - </>;( t) )]

+

��) ^sin[27r( ^,P( t) - </>;( t) )];

i

⁼

1, .

^.

.

N,

R(t) exp(i27r<p(t))

⁼

� ^t exp(i27r</>j(t)),

;=1

(2.1)

(2.2)

where <Pi(t) is a phase renormalized by 27r of the i-th oscillator at the discrete time t ( t: integer). Note that phases and (angular) frequencies are normalized by 21r only in this chapter 2, while they are not normalized in other chapters.

The i-th oscillator stated here is supposed to represent a cluster of some neurons which may be equivalent to the oscillator.

A

natural frequency of the i-th oscillator at time t is denoted by Oi(t), which will vary in according with a learning rule mentioned later.

A

complex quantity, R(t) exp(i27rcp(t)), is a complex order parameter defined by eq. (2.2), which stands for a coherency of a dynamic state of the system. The amplitude R( t) of the order parameter takes the value from

0

to 1, which represents the degree of an alignment of phases of the oscillators. Since R( t) is the amplitude of the mean field of the oscillators, it is supposed to represent an electric field outside of neural cells which is an average of neuronal activities. The phase of the order parameter cp(t) means the average of the phases of oscillators. The last term of the right-hand side of eq. (2.1) stands for a contribution or perturbation of a signal input into the system, where A(t) and 'lj;(t) represent an amplitude of the input signal and its phase, respectively . Here the strength of the input signal is considered to affect all of the oscillators equally. The system described by eq. (2.1) and eq. (2.2) is a discrete-time version of the phase oscillator model with mean-field coupling studied well by Kuramoto4• 36) and several researchers,37• 38• 40• 47• 63) except for the external force (input) term: (A(t)/27r) sin[27r('l/;(t)- <Pi(t))].

In this study, we investigate responses to periodic signal inputs. It can be demonstrated by introducing the learning rule, which will be mentioned later, that the coupled-oscillator system can recognize periodic input patterns in the sense that a learned pattern induces a coherent phase motion, where R(t) takes the value close to unity, or R( t) oscillates regularly (periodically); on the con

trary, an unlearned pattern induces a disordered behavior, in which R(t) oscillates

(17)

randomly and its average value is close to zero. The system concerned here is schematically illustrated in Fig. 2.1.

2.2 Learning rule

The entrainment

(

synchronization

)

can be seen in the coupled oscillators. Let us consider the coupled-oscillator system under a periodic external force

(

_input

)

_.

When the frequency of the external force is nearly equal to the average of the natural frequencies of the elementary oscillators, i.e., the natural frequency of the system, phases of all the element oscillators rotate at the rate

( 'l/;( t

⁺

1) - 'l/;( t))

of the external force. In this state, phases are almost aligned relatively; it means the ordered state. Therefore,

R( t)

takes the value close to

1.

When a deviation of the frequency of the applied external force from the natural frequency of the system becomes larger, the ordered state mentioned above becomes unstable and transits into a state in which

R( t)

oscillates with a relatively lower value, ^as will be mentioned later in detail.

Note that the word ordered

(

or coherent

)

state is used in broad sense here.

We will regard the system ^asthe ordered state when

R(t)

takes nonzero value, which is necessary to be sufficiently larger than its amplitude of fluctuation.

Regarding the coupled oscillators as a pattern-recognition system, an ordered

(

_coherent

)

state is naturally considered to be a "recognized state" which means the system exhibits that an input pattern is learned, while a disordered

(

_incoher

ent

)

state is considered to be an "unknown state,"15) where the system exhibits that an input pattern is unlearned. That is, the system what we want to construct here is expected to work ^asa certain kind of detector which can discriminate a learned signal from unlearned signals by using macroscopic synchronization dy

namics, and the output of the detector is its coherency of the constitutent oscil

lators. The degree of this coherency is expected to reflect the similarity between the learned signal and an input signal.

The input patterns used here are restricted to periodic temporal signals, and the ordered state arises from a kind of synchronization with the input. Thus, the learining rule adopted here. should work to change the natural frequencies of oscillators so that the coupled-oscillator system might exhibit the ordered state resulting from the synchronization with the periodic signal input. The explicit expression which realizes the frequency shift is as follows:

- L

ni(t

⁺

1)

⁼

ni(t)

⁺

cni

⁺27!" sin

[

_27r

(

_'l/;

(

_t

)

_-

<Pi(t))],

i ₌ 1,···,N,

(

_2.3

)

where

!1i(t)

is a natural frequency of the i-th oscillator at time

t,

^C_and

fii

_are

constants to prevent an excess condensation of frequencies of oscillators, and

fii

affects a stationary distribution of the frequencies after a learning process. In this study, the values of them was set and fixed as

fii

=

!1i(O)

_forⁱ=

1,

^{· ·}·, N. _The

(18)

Input Output

A(t) ei21tl/f(t) R(t) ei2ncp (t)

Oscillator

Mean Field

Fig. 2.1: Schematic illustration of the structure of the coupled-oscillator system concerned in this chapter. All of the oscillators are coupled to each other with equal coupling strengths,!{, i.e., connected to the mean field generated by all the oscillators. An input signal is applied to all the oscillators with equal weights.

Note that phases and frequencies are normalized by 21r only in this chapter 2, while they are not normalized in other chapters.

12

(19)

strength of the effect of the external force (input signal) is determined by a con

stant L. The learning rule is intuitively introduced, but should be derived by min

imizing a certain kind of estimation function which is describing distance between the desired input-output characteristics and the real characteristics. However, to construct an adequate estimation function is left as a future task.

The learning rule defined by eq.

(2.3)

works basically as follows: the natural frequency of the i-th oscillator is increased when the phase of the input signal advances on that of an oscillator, and the natural frequency of the i-th oscillator is decreased when the phase of the input signal is late for that of the oscillator. A lot of simulations confirm that if an input signal is periodic a time evolution of the natural frequencies always goes into the stationary state, where the distribution of the natural frequencies is fixed. Learning is considered to be completed when the distribution becomes stationary, and the distribution is fixed during investigation on responses to various inputs.

The learning rule that the distribution of the natural frequencies changes according to the difference between the phase of the input signal and the those of the oscillators may seem to be strange. The learning rule, which causes to modify constants of coupling strength, is widely used in many kinds of neural network models. It is easy to understand the correspondance between the coupling strength in some kinds of neural network models and the coupling strength which correlates neuronal activities in real biological neural systems; the correspondance between the natural frequency of the oscillator and some constants to characterize real neural systems cannot be understood straightforwards. However, we cc:.n explain the reason for using the learning rule of eq.

(2.3)

for the coupled oscillators by considering that one oscillator in the system concerned is corresponding to or equivalent to a cluster composed of some neurons. Some coupled neurons with an adequate strength can be considered to be one oscillator. Its natural frequency is dependent on the coupling strength in some conditions.43) Hence, the shift of the natural frequency of the oscillator can be attributed to the change of the coupling strength. The rule of the shift of the natural frequencies is an assumption introduced for an attainment of the synchronization with an input signal which is learned by the system; the rule adopted here should be confirmed or justified by correlating simulation results with data from real neural systems in the future.

2.3 Results

In this study, the number of the oscillators N was fixed at 40. The dependence of the dynamic behavior of the present system on N had been investigated pri

marily, and concluded that the 40 oscillators were sufficient to exhibit the drastic synchronized-desynchronized transition. A coupled-oscillator system composed of extremely small number of oscillators cannot conserve a disordered state with small fluctuation; i.e.,

R(t)

cannot keep a lower value near to zero. To construct a pattern-recognition system utilizing the synchronized-desynchronized transition of the system, we use the relatively small number of oscillators which can exhibit

(20)

the desynchronized state with sufficiently small fluctuation as well as the synchro

nized state. The initial values

</>i(O)

for

i

⁼

1, · · · N

were set randomly, because it had already been confirmed that the system dynamics was independent of the initial values.

2.3.1 Input-output characteristics

A typical response of the present system to a periodic input signal was studied.

A uniform distribution of

ni,

with zero average, was adopted, and the width of the distribution � was set at

0.1

and fixed:

C

⁼

0,

L ⁼

0.

The coupling strength K was set at

0.4.

In this case, the system is expected to show non-chaotic states, because of K <

1.

In spite of the non-chaotic condition, the system exhibits considerably complex

(

turbulent

)

behavior because of the distribution of the natural frequencies of the oscillators. Responses of

R( t) (

output

)

to the periodic input signal with various frequencies were studied, where the input signal was expressed as follows:

A(t)

¢(t+1)

0.5,

1/;(t) +

^wo.

(2.4)

Stationary behaviors of

R( t)

is depicted in Fig.

2.2(

a

)

, which is a bifurcation diagram of

R( t)

along the axis of the parameter ^w0

(

the frequency of the input signal

)

. The coupled oscillators keep an ordered

(

coherent

)

state in which

R( t)

takes the value close to

1

when ^w0 is sufficiently close to zero, which can be regarded as the natural frequency of the whole system. The ordered state becomes unstable when ^w0becomes larger than a threshold value dependent on](,� and

A.

In the destabilized state, the behavior of the system becomes complex in the sense that the degrees of freedom of the behavior increases by increasing ^w0far from zero. This effective degrees of freedom of the dynamics can be measured by the correlation dimension.65-68)

The correlation dimension is the exponent ^v in the correlation integral

C1(r) (

^,...._,

r11)

which is defined in the d-dimensional reconstructed phase space as65)

. 1

C1(r) = N2 L u(r- II �i- �i II), s i#j

where

Ns

is the number of sampling points, and

(2.5)

xi= (R(i),R(i + m),R(i +2m),··· ,R(i +

(d

- 1)m)). (2.6)

The delay time

m

was chosen as

1

in the calculation here. The function

u(

^·

)

is the unit function defined as

u(

^x

)

⁼

1

if ^x �

0

and

u(

^x

)

⁼

0

if ^{x <}

0.

The expression

II �i - �j II

means the distance between the vector

�i

and

�i

in the d-dimensional phase space. Then, the correlation dimension ^vcan be obtained formally as

v

=

lim ln

C1( r)

. r-+0 ln ^T

14 (2.7)

(21)

The dependence of the correlation dimension of the amplitude of the order parameter

R( t)

on the frequency of the input w0 is shown in Fig. 2.2

(

b

)

, in which a very narrow region around the threshold

( ""0.0675)

is only depicted because a large amount of data and calculation are needed66) when the dimension or effec

tive degrees of freedom is large. To avoid a difficulty of calculating the higher dimension, an entropy will be used later to characterize the complexity or ran

domness of the behavior of the system. Figure 2.2

(

b

)

shows the bifurcation from the state of one degrees of freedom by increasing the parameter w0 quantitatively by four kinds of scales. Since an attractor in a state of many degrees of freedom has a complex structure whose dimension is dependent on the measuring scale, dimensions of various scales are investigated. It is shown that the dimension of any scale increases by increasing w0 above the threshold.

Figures 2.3

(

a

)

-

(

c

)

depict the amplitude of the order parameter,

R(t),

its orbit in the

R( t )

^-

R( t

+

1)

space, the instantaneous velocity

(

defined as

cp( t)

^_

cp( t

+

1)

cp( t))

of the phases of the order parameter, the orbit of

cp( t)

in the

cp( t )-cp( t

+

1)

space and the orbit of the order parameter in the complex plane at some values of w0 slightly above the threshold, respectively. The increase of the complexity of the behavior of the order parameter occurs by increasing w0. When w0 is slightly above the threshold, the amplitude

R(t)

exhibits an orbit of a closed loop implying a limit cycle, which means that almost all the oscillators behave like one oscillator, and the corresponding correlation dimension is nearly unity. Increase in w0 far away from the threshold increases the effective degrees of freedom of the system. It reflects the higher correlation dimension, and the behavior of the system becomes complex as seen in the

R(t)-R(t

+

1)

return maps of Figs. 2.3

(

b

)

,

(

c

)

, in which the orbit of

R(t)

looses its low dimensionality and shows a dispersed at tractor. However, the

cp( t)-cp( t

+

1)

return maps in Figs. 2.3

(

a

)

-

(

c

)

indicate that the mean rotating velocity of the phases of the oscillator is equal to that of the input. The orbit of the order parameter in the complex plane shows that the fluctuation of the macroscopic behavior mainly comes from the fluctuation of the amplitude of the order parameter.

2.3.2 Learning and recognition of a temporal pattern

We propose a learning rule which is considered to be suitable for the coupled

oscillator system to learn a periodic temporal-pattern

(

signal

)

, and to recognize the learned signal by exhibiting high-ordered

(

coherent or collective

)

dynamics implying entrainment

(

synchronization

)

to the input signal.

Figure 2.4 shows redistribution process of ni obeying eq.

(

2.3

)

at various values of the coefficients C under the input of periodic signal which is expressed by eq.

(

2.4

)

with ^w0⁼⁼

0.12.

It is clear that the degree of the condensation of the distribution is low at a high value of the coefficient C. We should avoid an excess condensation because such a condensation induces the coherent behavior whether the signal is input into the system or not; i.e., the system always exhibits the coherent motion of the phases of the oscillators, implying uselessness to recognize

(22)

(

^a

) _1.0

0.8 R(t) ^0.6

0.4 0.2 0.0

0.0 0.1 0.2

(b)

^.Î Î Î ^l ^-I Î Î Î

�

⁰

A 0

� A

0

B

'Uj

�

2f-

Q) 0 ₀ ^-

s

0

�

^... ⁰

�

6

_{0 0}

A A

.9 _... 0

�

₀ ₀ A

Cl>

� � � ⁰

�

1r--

^-

0 0 0 ₀

0

D: _1o-3·5

0

6: 10-3·0

0

0: _10-2·5

0

^{I-f) 0} ⁰ ⁰

0: _10-2·0

^-

I I I I I I I I

0.066 0.068 0.07 0.072

Wo

Fig. 2.2: Stationary behavior of the amplitude of the order parameter,

R(t).

The dependence of

R(t)

on the input parameter, w0, is depicted in (a). The dependence of the correlation dimension of

R(t)

on the input parameter, wo, is shown in (b).

16

(23)

(_a)

U":>

0.95

-

d [Z]

�

^0.90

! �

^{� o}

'

^v

/

^I

2000 ₂₅₀₀t 3000 _0.90R(t) _0.95 ⁰

D

�fili l!ll llll!lllll� ⁴ '0

0.066 1-..--��--_�-..J

2000 2500 3000 0

cp(t) {b)

� �- :� i:f ^�: I

0.8 L-...---_j

(c)

2000 ₂₅₀₀ 3000 _0.8 _0.9 _1.0

2000 2500 3000

2000 ₂₅₀₀ 3000

R(t)

0

cp(t)

0.7 0.8 0.9 1.0

R(t)

0 cp(t)

-1 0

-1 ₀

-1 0

Fig. ^2.3: Macroscopic behavior of the system under the external input. The amplitude

R(t)

of the order paremeter is depicted at the top left, the phase velocity

<{;( t)

of the order parameter at the bottom left, the return map of

R( t)

at the top right, and the return map of

cp( t)

is depicted at the right to

<{;( t).

The orbit of the order parameter is depicted at the extreme right, in which the set of quantities

R( t)

and

cp( t)

is considered to be a polar coordinate. The input parameters are ^asfollows: (a) ^wo⁼⁰.⁰⁶⁸², (b) ^wo⁼^0.0700,(c) ^wo⁼^0.0720.

(24)

a temporal pattern by the synchronized-desynchronized transition. Conversely, we should avoid an excess dispersion of the distribution because the system always exhibits the desynchronized or incoherent behavior at relatively low values of the coupling constant K, while always exhibits the coherent behavior at relatively high K. An adequate value of C selected here is 0.03, and the coupling constant K is set at 0.4; these values are suitable for constructing the expected characteristics mentioned above. The strength L of the effect of the input signal on the learning process is 0 .05. These three values of C, K, L are fixed hereafter, and the initial width of the distribution is 0.3.

Figures 2.5

(

a

)

and

(

b

)

show responses of the amplitude

R(t)

of the order parameter to the input signal which is obeying eq.

(

2.4

)

at various frequencies

w0 before the learning and after the learning, respectively. The length of a bar in these figures indicates an amplitude of a fluctuation of

R(t).

There is no synchronized region before the learning, while after the learning, a synchronized state arises in which

R(t)

takes a constant and relatively high value. The similar characteristics seen in the previous section are also observed in the characteristics constructed by the learning rule; i.e., the effective degrees of freedom of the oscillation of

R(t)

increases by increasing the distance from the synchronized reg1me 1n wo.

Next, we consider the slightly complicated periodic pattern which means:

A(t)

�(t

⁺

1)

0.5,

�(t)

⁺w1,

t:

even

�(t)

⁺w2,

t

:odd.

(

2.8

)

The redistribution process of the natural frequencies under the learning of the input signal expressed by eq.

(

2.8

)

is depicted in Fig. 2.6. In this figure, it is found that some of values of ni move towards the average of w1 and w2, and others move towards the average subtracted by 0.5. Simulations with various values of w1 and w2 showed the similar behavior. However, the ratio of the number of ni moving towards the average value to that of ni moving towards the average subtracted by 0.5 is also found to be dependent on the values of w1 and w2

themselves. The coupled oscillators discriminate various input signals expressed by eq.

(

2.8

)

in this way.

Responses of the system after the learning to the input signals obeying eq.

(

2.8

)

is shown in Figs. 2.7

(

a

)

-

(

d

)

. The pair of parameters

(

w1,w2

)

of the learned pattern is set at

(

0.02,0.38

)

, and the amplitude of the signal

A

is 0.5. The figures show the amplitude

R(t)

of the order parameter, the instantaneous velocity

<j;(t)

^_

cp(t

⁺

1)- cp(t)

of the phase of the order parameter the return map of

R(t)

and the return map of

cp(t)

and the orbit of the order parameter in the complex plane at various input parameters

(

w1, w2

)

. The coupled oscillators exhibit a regular response to the learned pattern, while an irregular response is observed when an unlearned signal is input into the system.

Here, we stress that a quantitative estimation of the temporal irregularity of the order parameter is necessary to distinguish between the learned pattern and

1

8

(25)

(a)

(b)

(

^c

)

(d)

0

- 0. 5 c:.___...._..._.._�___._...._....--J.... ... �� ... ---..._._ ... -'-o-.---3

0 .5 ;.---,--��--....,....�--r-... ...,.---r----

0

-0.5--�--��--��

0. 5 -=-� · ---,------rlX[m�"T""-lr-�---,.-.--_._..,. ... --r�--.

0

-0.5 �----'-...-l..l.!�l...l-..o...:lt...L-�..._...._...-'---�_.____.�...__�

0. 5 _�---^-_,.-,^,.,..t"ll"[,..-y-n�,....,....,.,..,..,�r-:T"Trrr"I'"....,.,...,.'T'T""'T"..-:-r.-r-IT""� ...

0

0 200 400 600 800

t

Fig. 2.4: Redistribution process of the natural frequencies ni. The learing process starts at t ⁼ 100 with the input parameter ^wo ⁼ 0.12. The parameter C is as follows:

(

^a

)

^C⁼^0.01,

(

^b

)

^C⁼^0.03,

(

^c

)

^C⁼^0.05,

(

^d

)

^C⁼0.07. The parameter L is 0.05. The coupling strength ]{ among the oscillators is 0.4 and the initial width of the distribution is 0.3.

(26)

(a) R(t)

(b) R(t)

1

0 -0.2

1

0 -0.2

!111111111111

0 0.2

wo

. · · · · ··

1'1 1'1111

0 0.2

wo

Fig.

2.5:

Responses of the amplitude of the order parameter are shown. The length of a bar indicates an amplitude of a fluctuation of

R(t).

The dependence of

R( t)

on

w0

is depicted before the learning

(

a

)

and after the learning

(

b

)

. The learned input parameter

wo

is

0.12.

0

-0.5 �--��--��

0 200 400

t

600 800

Fig.

2.6:

Redistribution process of the natural frequencies of ni. The learning pro

cess starts at

t

==

100

with the set of the input parameters

(w1,w2) = (0.02, 0.38).

20

(27)

(a)

��

�b::d.,

^-

... ... ,:0�·� i·o.

^.

. ·_0

�

^0.5 ^�

0

s:

..

2700 2800 2900 3000 0 I

'P(t) (b)

�·:·��·�

²⁷⁰⁰ ²⁸⁰⁰ _t ²⁹⁰⁰ ³⁰⁰⁰ ⁰ _R(t) ^I

�·:b:Ji'[J

^-I

2700 2800 2900 3000 0 I

¥'(t) (c)

-I

(d)

�·:�!'[;] .

::::- . �· ·:

^-1 ⁰ ^I

�

^0.5 ^{� �}^s: ^{.. : .·}^�r

_· _,�-$.

0

2700 2800 2900 3000

'P(t)

Fig.

2.7:

Macroscopic behavior of the system under the external input. The configuration of the figures is the same as Fig.

2.3.

The sets of input parameters are as follows:

(

^a

) (

^wt,w2

)

⁼

(0.02,0.38), (

^b

) (

^w1,w2

)

⁼

(0.2,0.2), (

^c

) (

^wbw2

)

⁼

( -0.3, 0. 7), (

^d

) (

^{wb w2}

)

⁼

(0.3, 0.3).

(28)

the unlearned pattern. Only from the temporal-average value of the order param

eter we cannot discriminate the learned pattern from the unlearned patterns as seen by comparing Figs. 2.7(a) and (b), in which the temporal-averaging values of

R(t)

are almost the same. However, if the input pattern is sufficiently different from the learned pattern, the temporal-average of

R( t)

itself takes a lower value as seen by comparing Figs. 2.7(a) and (d).

In order to obtain a quantitative description of the regular behavior and the irregular (turbulent) behavior of the system as shown in Figs. 2.7(a)-(d), the negative of entropy (denoted by

-H)

is more useful to describe the irregularity than the correlation dimension, because an extremely large amount of data is needed to calculate the correlation dimension when the resultant dimension takes a considerably high value. The expression of

(-H)

is as follows:

- H = LPi

lnpi, (2.9)

�

where

Pi

is the probability of finding the value of the quantity concerned (which may take

R( t))

in an i-th cell. Here, the i-th cell means the i-th area defined by dividing the range of the quantity into sufficiently small size of areas and num

bering them. The input parameters

(

_w1,w2

)

dependence of the negative entropy is shown in Fig. 2.8. It is shown quantitatively that the system is in the ordered state when the input pattern is sufficiently similar to the learned pattern. The measure of the similarity of the temporal signal is necessary. Here, we point out that we can measure the similarity between temporal signals by the quantitative characterization of the regular or the irregular behavior of the coupled-oscillator system.

2.3.3 Transient dynamics

Figure 2.9 shows transient responses of

R(t)

to input signals. A signal is input into the system during some period which is indicated by a bar in the figure. The figure shows that a quick response is observed after the start of a signal input.

It means that a rapid transition from a disordered state to a stationary state (a coherent or an incoherent state) is observed at any parameters of the input signals.

After the stop of a signal input, the system exhibits a considerably long period of a transient response which means a sequence of pulses with constant intervals.

The height of the pulses gradually decreases and goes to the stationary disor

dered state. It should be noted that the height and the sharpness of the pulse depend on the parameters ( w1, w2

)

of the input signal. It is found that the rate of losing the height and the sharpness of the pulse is strongly dependent on the similarity between the learned signal and the input signal; it means that the quick transition (or return) to the initial stationary disordered state is observed with less similarity between them, while the slow transition to the disordered state is observed with high similarity. The periodic synchronization among the oscillators

22

(29)

(a)

^-H

0

-2

-4 0.5

(b)

^0.5

0.4

0.3

0.2 -0.1

• 0

0 0.1 0.2

Fig. 2.8:

(

a

)

The dependence of the negative entropy on the input parameters

(

^{w1, w2}

)

with A= 0.37. The number of the cells is 256.

(

^b

)

The contour lines of the negative entropy.

(30)

(

a

)

R(t)

(b)

R(t)

(c)

R(t)

(d)

R(t)

1

0 1

0

0 2000 4000 6000

t

Fig. 2.9: Transient responses of R(t) to input signals. The sets of input param

eters

(w1,w2)

in (a)rv(d) are the same as those in Figs. 2.7(a)rv(d), respectively.

The bar indicated in the top shows the period in which a signal is input into the system.

24

非線形結合振動子系における位相パターン情報処理 に関する研究