Non-Uniform Cellular Neural Network and its Applications

様 式 6 壬日間ム 文 自 _録 〈甲 工〉 報 告 番 号 ￨乙 工 第 2 6 下E=7: _{￨氏 名 ￨ 可イ} 辰 工 修

学 位 論 文 題 目 ~on-Uniform Cellular Neural ¥etwork and its Applications

論 文 の 目 次

Chapter 1. General Introduction

Chapter 2. Discrete-timeCellular Neural Network

Chapter 3. A Modified Tracing Curve Algorithm forCNN Chapter 4. Associative Memory with DTCNN

Chapter 5. Application in ImageProcessing Chapter 6 Overall Conclusion

文 文 A_{仏 国} 山 ム 問 考 主 参

1 . “Iterative middle rnapping learning algorithm for cellular neural network" ， Chen HE and Akio USHIDA， Insti tute of Electronics， Inforrna

-tion and Communication Engineers Trans.， ¥"01.E-77A， ~o. 4， pp.706-715， 1994

2. "Arnodifiedpredictor-corrector tracing curve algorithm fornonlinear

resistive circuits". Chen HE and Akio lSHIDA. Institute of Elec -tronics. Information and Communication Engineers Trans.， Vol.E-74， No.6， pp.1455-1462， 1991.

副 論 文

1 . “Convergence analysis ofsynchronous-updating CNNand related DTCN~" ，

Chen HEand Akio USHIDA， Proc. of19931nt. Symp. on Nonlinear Theor.

and i tsApp1.， pp.29-34， Hawaii， American， 1993. じ れ H T 4 m 川 ハ υ ι H 山 W +し+L ・ 1 ム ρ u r よ M N H ハ U σ b ? l ム 1 i n d ndri H U σbρ し V 円 弓 U nuM 聞 け 円 u s u -ょ n u u n n 1 i F i ハ U a ' 凸U ・ n d 1 i n ド 門 U m ・ 1 g y h n Q U 円 し . ー ム p . ， n ド +LHU 9unu ハ u m T l ム l n ワ L ρ u σ b 1 i n J n A u n H U 円 G AunHunu --1i 円 U m 凶 戸 + l ム ' ﹂ H U ハ U ﹁ h u + し 円 h u - - _t ム ・ 唱 11 ム w w ハ ﹂ 一 ハ U ハ リ υ vvJT ム 円 hU TADi--ハ U m 川 n ド ρ しv ' n ド m 川 A A H h υ ' 凸 ﹂ T i σ b vvH 門 unu - I n o -4 十 し Hunb a s ， 可 よ ハ U 凸 ﹂ ハ し ・ τ よ p u n u L k u ハ υ 円 、 U A 八 T よ s p ndJU n 1 i r a a a n -p u u b H U H H U -ょ 1 i n ¥ U 11 ム 門 U ρuρ し } ， パ u p し 仏 川 口 P し a η f M

3. “An efficient algorithm for solving nonlinear resistive circuits" ，

Chen HE andAkio USHIDA， Proc. of 1991 1EEE 1nter. Symp. onCircuit andSystem， pp.2328-2331. Singapore， 1991. 備 考 l 論 文 題 目 は ， 用 語 が 英 語 以 外 の 外 国 語 の と き は 日 本 語 訳 を つ け て ， 外 国 語 ， 日 本 語 のJI聞 に 列 記 す る こ と。 2 参 考 論 文 は ， 論 文 題 目 ， 著 者 名 ， 公 刊 の 方 法 及 び 時 期 を 順 に 明 記 す る こ と。 3 参 考 論 文 は ， 博 士 論 文 の 場 合 に 記 載 す る こ と 。

様 式 7 吾肩ゐ関、 _{文 内} 容 要 ヒ日ニ

匂

二

三

〉

報 告 番 号 ￨ 乙 工 第

26

τ~ ￨氏 名 ￨ 何 辰 工 修 学 位 論 文 題 目 on-UniforrnCellularNeural~etwork and its Applications 内 容 要 旨 セ ル ラ ー ニ ュ ー ラ ル ネ ッ ト ワ ー ク (C N ~ ) には連続時間的な も の と ， 離 散 時 間 的 な も の が あ り ， 本 研 究 は 主 に 後 者 に つ い て 議 論 する. C 0:Nは1988年にカリフォルニア大学ノ〈ークレ校のL.O.Chua 教 授 ら に よ っ て 提 案 さ れ ， 現 在 ， ア メ リ カ ， ヨ ー ロ ッ パ を 中 心 に 盛 ん に 研 究 が 進 め ら れ て い る .C N Nは 従 来 の ニ ュ ー ラ ル ネ ッ ト ワ ー ク と 異 な り ， 近 傍 の セ ル と の み 結 合 し て い る た め 集 積 回 路 と し て の 実 現 が 容 易 で あ り ， 画 像 処 理 用 Ci¥Nと し て 注 目 さ れ て い る 第 一 章 で は ， ニ ュ ー ラ ル ネ ッ ト ワ ー ク に 関 す る 研 究 の 動 向 ， お よ び ， 人 間 の 目 と 同 様 な 処 理 機 能 を 持 つ 連 続 時 間 C N Nに 関 す る 研 究 の 動 向 と ， こ の 論 文 で 議 論 し て い る 離 散 時 間 C て 簡 単 に 述 べ て い る の 背 景 に つ い 第 二 章 で は ， 離 散 時 間 的 な 非 均 一 C N Nと し て3 二 相 同 期 信 号 の 回 路 モ デ ル を 提 案 し ， そ の 安 定 性 等 に つ い て 議 論 し で あ る . こ の モ デ ル は 各 セ ル に つ い て 二 相 同 期 信 号 l個 で 実 現 で き る た め ， VLS1 の 実 現 が 容 易 で あ る と 云 う 特 徴 が あ る . ま ず ， モ デ ル の 動 作 原 理 か ら 状 態 電 圧 ， 出 力 電 圧 の 動 作 領 域 を 明 か に し た . こ の こ と は 物 理 的 に 実 現 可 能 な C N Nを 設 計 す る た め に 重 要 で あ る. つぎ に ， 安 定 性 を 議 論 す る た め に エ ネ ル ギ 一 関 数 か ら リ ア フ ノ フ 関 数 を 定 義 し ， そ の 関 数 の 時 間 単 調 減 少 の 条 件 を 利 用 し て ， 大 域 的 な 安 定 性 を 持 つ 離 散 時 間 CNNの 設 計 方 法 を 明 ら か に し た 第 三 章 で は ， 非 線 形 シ ス テ ム に お け る 平 衡 点 の 求 解 法 に つ い て 議 論 し て い る .連 想 記 憶 に 用 い ら れ る C N Nは 多 く の 平 衡 点 を も ち ，

入 力 信 号 に よ っ て ど の 平 衡 点 に 到 達 す る か が 決 定 せ ら れ る . ロバス トな連想記憶用 CN);を 設 計 す る た め に は ， こ の よ う な 平 衡 点 を 調 べ る こ と が 必 要 で あ る . こ こ で は ， 解 曲 線 追 跡 法 に 基 づ い た 複 数 解 の 求 解 ア ル ゴ リ ズ ム を 提 案 し て い る . こ の ア ル ゴ リ ズ ム は 急 激 な 解 曲 線 の 変 化 を 効 率 よ く 追 跡 で き る よ う に ， エ ル ミ ー 卜 予 測 子 と BDF 積 分 公 式 に 基 づ い て い る . ま た ， 大 規 模 系 に 適 用 で き る よ う に ニ ュ ー ト ン ・ ラ フ ソ ン 法 の 代 わ り に ブ ラ ウ ン の 反 復 法 を 採 用 し て い る こ の よ う な ア ル ゴ リ ズ を 採 用 す る こ と に よ り ロ バ ス ト な C););の 設 計 が 可 能 と な る . 第 四 章 で は ， 離 散 時 間 C N Nに よ る 連 想 記 憶 に つ い て 述 べ て い る . 連 想 記 憶 は 人 間 の 脳 の 基 本 的 な 機 能 で あ り ， ニ ュ ー ラ ル ネ ッ ト ワ ー ク 応 用 研 究 の ー っ と し て 古 く か ら 盛 ん に 研 究 さ れ て い る.本 章 で は ， 離 散 的 な C Nl ¥ を 用 い た 外 積 学 習 ア ル ゴ リ ズ ム と 中 点 写 像 ア ル ゴ リ ズ ム の 2種 類 の 記 憶 方 式 を 提 案 し ， そ の 性 質 を 解 明 し て い る. まず，前者は，入力ノぞターンに対して，エ ネ ル ギ 一 関 数 の 値 が 最 少 に な る よ う に ニ ュ ー ロ ン 聞 の 接 続 を 表 す 重 み 行 列 を 設 定 し よ う と 云 う も の で あ り ， こ れ はHebbの 理 論 に 基 づ い て い る . ま た ， 上 の よ う な 手 法 で 学 習 さ れ た パ タ ー ン を 連 想 記 憶 で き る 条 件 に つ い て 議 論 し た.中 点 写 像 ア ル ゴ リ ズ ム は 重 み 行 列 の 設 定 方 法 に 対 し て ， い ま 考 え て い る 中 心 セ ル か ら の 近 傍 を 定 義 し，近傍に存在するセルの状態、 を ベ ク ト ル 表 示 す る.こ れ を 全 て パ タ ー ン に つ い て 実 行 し ， こ の よ う に し て 決 定 さ れ た 行 列 に よ っ て 写 像 さ れ る セ ル の パ タ ー ン が ， 元 の 中 心 セ ル と 同 一 の パ タ ー ン を 持 つ よ う に 重 み 行 列 を 設 定 し よ う と い う も の で ， 数 学 的 に は 一 般 化 逆 行 列 の 理 論 に 基 づ い て い る.この よ う な 学 習 方 法 の 特 徴 は 入 力 さ れ た 画 像 が 全 て 連 想 さ れ る と 云 う こ と で あ る . 本 章 で は ， さ ら に ， こ の こ と を 応 用 例 に よ っ て 実 証 し た 第 五 章 で は ， 画 像 処 理 へ の 応 用 と し て ， 輪 郭 抽 出 ， 雑 音 除 去 ， 視覚ノぞターンの認識に対する離散的な C N Nに つ い て 述 べ て い る . 多 く の 結 果 か ら 処 理 時 間 は 従 来 の も の と 比 較 し て 極 端 に 短 縮 さ れ る こ と が 分 か っ た . ま た ， 不 均 一 離 散 時 間 Cf¥Nに よ っ て ， 一 つ 画 面 中 に 多 数 の 異 な る 視 覚 ノ ぞ タ ー ン を 同 時 に 認 識 で き る こ と も 分 か っ た 。 第 六 章 で は ， 不 均 一 離 散 的 な C N Nの 特 徴 と 今 後 の 問 題 点 に つ い て 述 べ て い る

様 式 9 論 文 審 査 の 結 果 の 要 旨

を

二

t

報 告 番 号 ￨ 乙 工 第 26 τEコ7 _{￨氏 名￨ 何 辰} 工 修 主 査 牛 田 明 夫 教 授 審 査 委 員 ￨ 副 査 為 貞 建 臣 教 授 ;Ij 査 木 内 陽 介 教 授 学 位 論 文 題 目

Non-Uniforrn Cellular Neural Network and its Applications

審 査 結 果 の 要 旨 セ ル ラ ー ニ ュ ラ ル ネ ッ ト ワ ー ク (C州 ) に 関 す る 研 究 は1988年 に カ リ フ ォ ル ニ ア 大 学 の し

o

.Chua

ら に よ っ て 提 案 さ れ ， ア メ リ カ ， ヨ ー ロ ッ パ を 中 心 に 盛 ん に 研 究 さ れ て い る . CNNに は ア ナ ロ グ 型 と 離 散 的 な も の が あ り ， 彼 の 研 究 は セ ル 聞 が 不均一に結合した二相同期信号を利用した離散時間 C~N (DTCNN)につ い て で あ る 本論文では， DTCNNの 安 定 性 対 す る 十 分 条 件 を 導 出 し ， 2種 類 の 連 想 メ モ リ ー の 設 計 手 法 を 提 案 し た . 一 つ はHebbの理論に基づし1た 外 積 学 習 ア ル ゴ リ ズ ム を 利 用 す る も の で あ り ， 他 は 写 像 さ れ る セ ル の パ タ ー ン が 元 の 中 心 セ ル と 同 ー の パ タ ー ン を 持 つ よ う に 重 み 行 列 を 設 定 し よ う と い う 中 点 写 像 ア ル ゴ リ ズ ム を も ち い る も の で あ る . こ れ に 関 す る 論 文 は 電 子 情 報 通 信 学 会 の 英 文 誌 ， 中 国 ， ハ ワ イ で 開 催 さ れ た 国 際 会 議 に 報 告 さ れ て い る 一 般 に ， ニ ュ ー ラ ル ネ ッ ト ワ ー ク は 多 く の 平 衡 点 を 持 っ て お り ， そ の 性 質 を 解 明 す る た め に は ， 演 算 効 率 の よ い 平 衡 点 の 求 解 法 が 必 要 で あ る.本 論 文 で は 予 測 子 - 修 正 子 法 を 用 い た 一 求 解 法 を 提 案 し て い る . こ れ に 関 す る 論 文 は 電 子 情 報 通 信 学 会 の 英 文 誌 に 報 告 さ れ て い る さらに， DTCNNの 画 像 処 理 へ の 応 用 と し て ， 輪 郭 抽 出 ， 雑 音 除 去 ， 視 覚ノぞター ン の 認 識 用CI0Nに つ い て 述 べ て い る 彼 の 提 案 し たDTCNNは と く に 演 算 処 理 時 間 と 安 定 性 の 点 で 優 れ て お り ， VLSIに よ る 実 現 が 期 待 さ れ て い る 以 上 の よ う に 本 研 究 は ， 学 会 ， 国 際 会 議 で も 評 価 さ れ て お り ， 本 論 文 は 博 士 ( 工 学 〉 の 学 位 授 与 に 値 す る も の と 判 定 す る.

Non-Uniform C

e

l

l

u

.

l

a

r

Neural

N

etwork and i

t

s

A

]

;

>

p

l

i

c

a

t

i

o

n

s

D

i

s

s

e

r

t

a

t

i

o

n

Submitted

i

n

Candidacy f

o

r

t

h

e

Ph. D. D

e

g

;

r

e

e

by

Chen H E

Docto

γ

Course

fo γ Syste'fTj~

E

れ

g

z

n

e

e

γ

z

n

g

Graduα

，

t

e

S

c

h

o

o

l

，

En~linee竹ng

Tokushima Uni

悦 γ

s

i

t

y

，

J

α

p

αれ

September 1

9

9

4

Contents

Acknowledgements 6

1 General Introd uction 1.1 Background . 1.1.1 The neuron 1.1.2 Development of artificial neural networks . 1.1.3 Cellular neural networks 1.2 Purpose of this study . References . ORunRuoδQU ハ U 1 i A 生 司 l ム 11 ム T S_ム

2 Discrete-time Cellular Neural Network 2.1 Introduction. 2.2 Discreもe-timecellular neural network 2.3 Dynamical range of our DTCNN 2.4 DTCNN with binary output . 2.5 Stability analysisof DTCNN . 2.6 Conclusion. References . . R U ハ o n U つ j U F U ケ toonud 1 1 2 2 2 2 3 3

3 A Modified Tracing Curve Algorithm for CNN 3.1 Introduction. 3.2 Predictoralgorithm . 3.3 Corrector algorithm. . . 3.4 Choice ofthe step sizes. 3.5 Computational Algorithm 3.6 Illustration examples 3.7 Conclusion.. 0 0 1 5 6 7 0 3 4 せ A せ A せ A せ A 生 A 笠 に υvo 1

CONTENTS 2

References . . 55 4 Associative Memory with DTCNN

4.1 Introduction. 4.2 Outer productlearningalgorithm 4.2.1 Storingobject patterns. . . 4.2.2 Stationary ch乱racteranalysis . 4.2.3 Illustrationexample 4.3 Middle mapping method . 4.3.1 Middle-mapping learning algorithm 4.3.2 Iteration learningalgorithm 4.3.3 Illustration example 4.4 Conclusion. References . 6 6 8 9 3 7 5 6 0 3 9 0 V O H U F O F O R U ハ O 円 i 円 i Q U Q U Q u n u 4 1_よ 5 A pplications in Image Processing 5.1 Introduction. 5.2 Featureextraction 5.3 Noiseremoval 5.4 Visual pattern recognition 5.5 Multiple visual pattern recognition 5.6 Conclusion. References . . . 102 102 104 107 115 118 124 126 6 Overall Concl usion 127 A A List of the Related Papers by the A uthor

A.1 Publications. A.2 International Conferences A.3Technical Reportsand Other Presentations. 131 131 132 133

L

i

s

t

of Figures

1.1 The structure of the neuron . . . .. 8 2.1 A continuous-time CNN cell . . . .. 17 2.2 Some typical 2-D regular grids . . . .. 17 2.3 A synchronous-updating CNN cell. . . " 20 2.4 The equivalentcircuit of SCNN cell during

o

η . . . .. 21 2.5 The equivalent circuitof SCNN cellduring

O

T2 • • • • • • • • • • • • • • 22 2.6 The equivalent cell circuit of SCNN ... 24 2.7 Uncontinωus monotone decreasingE(

t

)

curve ... 29 3.1 Solution curve wi七hour algorithm. . . .. 50 3.2 Solution curve for 6-cells Hopfield net. in a=O.l . . . .. 52 3.3 Four convergent states for CCD with CNN ... 53 4.1 Stored sample patterns . . . " 68 4.2 The A matrix for Example 1 . . . .. 69 4.3 Associative memory with outer product learning rule . . . .. 71 4.4 Stored sample patterns in Example 2 . . . .. 73 4.5 The A matrix for Example 2 . . . .. 73 4.6 Associative memory with outer product learning rule . . . .. 75 4.7 One sub-prototype does not satisfy the stationary condition and can not be recurred in this case . . . .. 76 4.8 Stored 4 prototypes for Example 1 ... 83 4.9 The feedback coefficients matrixA ... 84 4.10 The retrieval process for one pro be . . . " 85 4.11 4 probes and associative memory results . . . .. 87 4.12 4 prototypes are stored . . . " 88 4.13 The probes and retrieval results . . . .. 90 3

LIST OF FIGURES 4 4.14 8 prototypes are stored. 4.15 Input probes and the associative results. . 4.16Stored prototypes and initial probes with Gaussian noises. つ ん 円 。 只 U Q d Q u o d

5.1 An image composed by one diamond and foursquares ... 105 5.2 The extracted edge from the inputimage in Fig. 5.1. . . . 105 5.3 A picture of Chinese characters (Zhou SHEN 1427-1509) ... 106 5.4 The extracted edge from the input image in Fig. 5.3. . . . 106 5.5 A Chinese picture "Listening Bamboo" ( Zhen-mi時 Wen1470-1560) 108 5.6 The extractededge from the inputimage in Fig. 5.5. . . . 109 5.7 A photocopy by the satellite"Landsat"with σ = 0.8 m = 0 Gaussian white noise ... 110 5.8 Result image from Fig. 5.7乱fternoise removing. . . 111 5.9 A Chinese picture"YellowMountain

吋ZhouSHEN 1

427-1509) with σ = 0.7 and m = 0 Gaussian white noise . . . 112 5.10 Result image from Fig. 5.9 after noise removing. . . 113 5.11 The noise removing for a 16x16 Chinese character with σニ 0.2and m

=

0 Gaussian white noise . . . 114 5.12 A probe pattern composed by"

r

"

and

"ノ"

shaped elements . . . 116 5.13 A probe pattern composed by"

r

"

and "T" shaped elements. . . . 116 5.14 Classified resultforthe probe image in Fig. 5.12 . . . 117 5.15 Classified resultforthe probeimagein Fig. 5.13. . . 118 5.16 A probe pattern composed by three types of elements . . . 119 5.17 Two types of textons are picked out . . . 121 5.18 A probe image composed by three types elements . . . 122 5.19 Two distinc七typesofelements are picked ou t ... 123

L

i

s

t

of Tables

3

.

1

Some c

a

l

c

u

l

a

t

e

d

d

a

t

a

f

o

r

E

x

a

m

p

l

e

1

.

5

1

4

.

1

The s

i

m

u

l

a

t

i

o

n

r

e

s

u

l

t

s

f

o

r

2

6

E

n

g

l

i

s

h

c

h

a

r

a

c

t

e

r

s

s

t

o

r

i

n

g

i

n

a

c

e

l

l

u

l

a

r

a

s

s

o

c

i

a

t

i

v

e

memory b

y

t

h

e

m

i

d

d

l

e

m

a

p

p

i

n

g

a

l

g

o

r

i

t

h

m

.

9

8

5

.

1

The T

e

m

p

l

a

t

e

f

o

r

t

h

e

e

d

g

e

e

x

t

r

a

c

t

i

o

n

1

0

4

5

.

2

The T

e

m

p

l

a

t

e

f

o

r

t

h

e

n

o

i

s

e

r

e

m

o

v

a

l

1

1

0

5

.

3

T

e

m

p

l

a

t

e

1

f

o

r

p

a

t

も

e

r

nr

e

c

o

g

n

i

も

i

o

n

1

1

7

5

.4

T

e

m

p

l

a

t

e

2

f

o

r

p

a

t

t

e

r

n

r

e

c

o

g

n

i

t

i

o

n

1

1

7

5

.

5

T

e

m

p

l

a

t

e

3

f

o

r

m

u

l

t

i

p

l

e

v

i

s

u

a

l

p

a

t

も

e

r

nr

e

c

o

g

n

i

t

i

o

n

1

2

0

5

.

6

T

e

m

p

l

a

t

e

4

f

o

r

m

u

l

t

i

p

l

e

v

i

s

u

a

l

p

a

t

t

e

r

n

r

e

c

o

g

n

i

t

i

o

n

1

2

0

5

.

7

T

e

m

p

l

a

t

e

5

f

o

r

m

u

l

t

i

p

l

e

v

i

s

u

a

l

p

a

t

t

e

r

n

r

e

c

o

g

n

i

t

i

o

n

1

2

4

5

.

8

T

e

m

p

l

a

t

e

6

f

o

r

m

u

l

t

i

p

l

e

v

i

s

u

a

l

p

a

t

t

e

r

n

r

e

c

o

g

n

i

t

i

o

n

1

2

4

5

Acknow

ledgements

This presentthesisis a co11ectionof studiescarrieclout unclerthedirection and guiclance of Professor Akio Ushida of Tokushima University during 1991-1994. Therefore

，

1 wou1d 1ike to take this opportunity to thank a11 thosewho he1ped me and provided me to accomp1ishmy work.

Four years ago

，

1 came to Tokushima University乱sa foreign visiting scholar. Since

October of 1991う1have been workingfor Ph. D. degree ine1ectronics engineering. 1 wou1d like to express my great gratitude to ProfessorAkio Ushida for supplying me with this research and study chance

，

and a1so forhis guidance

，

he1pfu1 advice and continuous encouragen1entduringthis period.

1 wish to gratefu11y acknow1edgemy indebtednessto Professor Kiyoshi Kohno

，

Di -rector of Faculty of Engineering

，

and His wife 1¥1rs. Nayoko Kuhno who is my五rst J apanese Language Lec七urerafter1 came to Japan. As very nice teachers

，

seniors and friends

，

they give me and my wife with many hellps and very warm encouragements. 1 never forget their important supports and great he1ps forever.

1 sincere1y thank Professor Akio Sakamoto

，

Dr.Takashi Shimamoto and Dr.Yoshi -fumiNishiofor their he1pfu1 advice and kind en:couragement throughout the work.

1 wish to express my appreciation to Professor Takeomi Tames乱daand Professor Yoshisuke Kinouchi of Tokushima University

，

the member of my thesis committee

，

for their comments and suggestions

，

and particu1ar1y for their carefu1 and critica1 reading of this dissertation.

1 am very much grateful to Mr.Yoshihiro Yamagami of Tokushima University for his very usefu1 helps and warm encouragements.From him 1 learned many computer techniques during these fouryears.1 believe he.1S one of the most exce11ent computer technical expert who 1 know until now.

1 haveenjoyedan interesting research life with rny co11eagues of U shida and Sakamoto Laboratory and 1 am deeply indebted to them. 1 am very gratefu1 to Mr.Shinzao Liu

，

ACKNOWLEDGEMENTS 7 Mr.Zhen Hong

，

Mr.Youichi Tanji

，

Mr.Yoshinobu Setou

，

Mr.Motoshi Miyamoto

，

Mr.Takashi Ikoma

，

Mr.KoujiN akai

，

Mr.Kazuhisa Hirat乱 乱ndother peoples for their

friendly helps during the period of this research. 1 also wish toもhankall members in Ushida and Sakamoto Laboratory

，

Department ()f Electrical and Electronic Engineer -ing

，

Tokushima University

，

for a warm and inter:esting research environment which 1 have enjoyed in U shida and Sakamoto Laboratory.

1 would like to express my deep appreciation to my wife Ling-ge Jiang for her affec -tionate continuous encouragement and great help.

At last

，

but no the least

，

1 would like to express my gratitude to my father

，

my mather

，

my wife's father

，

my wife's mather and my son who helped and encouraged me greatly

，

and gave all kinds of assistances to obtain my ed ucational aims under the most comfortable atmosphere and to make this study a great success.

Department Electrical and Electronic Engineering Facul ty of Engineering

TOKUSHIMA University

C

んし件

ε

C

hapter

1

General Introduction

1

.

1

Background

1

.

1

.

1

The neuron

The human brain is one of the most complicated things that we have studied in detail

，

and in the same time

，

is poorly understood on the whole.

The neuron is the basic unit of the brain

，

it is shown in Fig.1.1.

synapse

)--、 W_J ~

come from another neuron 3

Xj

¥一一γ___; soma

come from another neuron 1 X J

-y Figure1.1:The structure of the neuron

CHAPTER 1. GENERAL INTRODUCTION 9

The soma is the body of the neuron. Attached to the soma are long

，

irregularly sharped五laments

，

called dendrites. The dendrites act as connections through which all the inputs to the neuron arrかe.Another type of nerve process attached to the soma is called an axon. This is electrically active and serves as the output channel of the neuron. The axon terminates in a specialized contact called a synapse that couples the axon with the dendrites of another cells.

The dendrites can perform addition on the inputs. The axon is a non-linear device

，

producing a monoもoneincreasing output voltage when the resting potential within

the soma varies over a certain critical threshold. The contact strength between the dendrites and other neuron 's synapse is different from another.

1

.

1

.

2

Development of a

r

t

i

f

i

c

i

a

l

neural networks

The year 1943 is often considered as the initial year in the development of arti五cial neural neもworks[l].McCulloch and Pitts[2] outline the first formal model of an elemen

-tary computing neuron. The model included all necessary logic computing element. Although the implementation of this model was not technologically feasible in that era

，

their model laid the groundwork for future developments. Donald Hebb[3] first proposed a learning scherne for updating neuron's connections that we now refer to as the Hebbian learning rule. He stated that the information can be stored in connections

，

and postulated the learning technique that had a pro -found impact on future developments in this field. Hebb's learning rule m乱deprimary contributions to neural networks theory.

During the 1950s

，

the first neuron computers were built and tested[4]. Theyadapted connections automatically. During this stage

，

the neuron-like element called a percep -tion was invented[5].Itwas aもrainablemachine capable of learning to classify cert乱ln patterns by modifying connections to the threshold elements. The idea caught the imagination of engineers and scientists.

Despite the successes and enthusiasm of the early and mid-1960s

，

the existing learn -lngもheoremsin that time wereも00weak to support more complex problems. Mean-while

，

the artificial intelligence area emerged as a dominant and promising research field

，

which took over

，

among others

，

many of the tasks that could not be solved by neural networks of that time. the research activity in the neural network field had sharply decreased.

CHAPTER 1. GENERAL INTRODUCTION 10

乱handfu1of researchers. The study of 1earning in in networks of thresho1d e1ements

and of the mathematica1 theory of neural networks was pursued by S.Amari[6，7]. Also in Japan

，

K.

Fu

kushima developed a class of neural network architectures called as neocognlもrons[8]. The neocognitron is a model for visual pattern recognition and is

concerned with biological plausibi1ity. The network emulates the retina1 images and processes them using two-dimensiona11ayers of neurons.

Associative memory research has been pursued by

，

among others

，

T. Kohonen[9・11]

and J.A.Anderson[12]. Unsupervised learning networks were developed for feature map-ping into regular arrays of neurons[10]. S.Grossberg and G.Carpenter have introduced a number of neural architectures and theorems and developed the theory of adaptive resonance networks[13，14]. During the period from 1982 until1986

，

several seminal publications were published that significantly furthered the potential of neural networks. The era of renaissance started with J.J .Hopfield[15ヲ16]introducing a recurrent neural network architecture for associative memories. His papers formu1ated computationa1 properties of a fully connected network of uniもs.

Another revitalization of the field came from the publication in 1986 of two volumes on parallel distributed processing

，

edited by J .M[cClelland and D .Rume1hart[17]. The new 1e乱rningru1e and other concepts introduced in this work have removed one of the

most essentia1 network training barriers that grounded the mainstream efforts of the mid-1960s.

Beginning in 1986-87

，

many new neural networks research programs were initiated. Among of them

，

the researchs of cellular neural network theorems and applications are very activity and developed in surprising speed.

1

.

1

.

3

Cellular neural networks

Cellular neural network( CNN) is a non1inear dynamica1 analog processing array having a 2-

，

or 3-dimensional grid architecture[18

，

19]. There are on1y finit loca1 connections from each processing cell to their adj acent elenaentsう sothat it is very suitable for the tasks where signal values are placed on a regular 2-D or 3-D geometric grid and the direct interactions between the signals are linaiもedwithin a local neighborhood[20]. Differing from general neural networks

，

CNN cells capture the geometric

，

nonlinear

，

andjor delay-type properties in the interaction weights. Also differing from Hopfield network， due to their loca1 connectivity， CNN can be easily realized with VLSI tech

-CHAPTER 1. GENERAL INTRODUCTION 11

nique. Meanwhile

，

the range of dynamics and the connection complexity are inde -pendent of the toもalnumber of processing cells

，

the implementation is reliable and robust.

Since 1988

，

just in a short period

，

it has given rise to wide interests in theもheo -retical researches for various generalizations and their applications in the areas like as image processing

，

pattern recognition

，

robot vision

，

motion detection and oth -ers. T.Roska and L.O.Chua presented a strueture with nonlinear and delay-type もemplates[21]

，

H.Harrer and J.A.Nossek extended the eontinuous model to diserete-time arehitecture[22]. At the same time

，

many other rese乱rchersalso make significant contri -butions to the CNN paradigm

，

which have been documented in some proceedings[23ぅ24]

and special issues[25

，

26].

1

.

2

Purpose of t

h

i

s

study

In Chapter 2う first

，

we will show the cell model of the continuous-time CNN

，

and some typical types of 2-D array structures briefly. After introducing a two phases synchronous-updating signal into a continuous-time CNN

，

we obtain a synchronous -updating CNN

，

we call it as SCNN. By extracting the v乱luesof state variations Viand output variationsYiat the updating momenもst

=

kTう k -0

，

1

，

2.

・

・

・

，

we derive a

discrete-time CNN which topology and output fumction are distinct from the DTCNN presented by Harrer and Nossek. With the dynamic route method

，

the dynamical properties of out DTCNN are analyzed. Moreover

，

the generalized energy functions for our SCNN and DTCNN are presented respectively. After then

，

two convergent theorems for our DTCNN are described. Since these convergent theorems are suitable for gener乱lizednon-uniform DTCNN

，

they provide the potential to apply our DTCNN more widely

，

for examples

，

to multi-types of visual patterns recognition

，

associative memories and others.

In Chapter 3

，

we present a modified BDF curve tracing method. The results shows this algorithm could be used efficiently to trace those solution curve with some sharp turning points. Specially

，

we want to point out that the Brown method is a kind of the Gauss-Seidel algorithm to be used for nonlinear乱1gebraicfunctions.Itis known that the convergence ratio is second order near to the solution.Furthermore

，

a number of the function evaluations is (N2

₊

_{3N)j2 when the function consists of N functions.} Observe that that the Newton method takes N2 _{evaluations of the partial derivatives}

CHAPTER 1. GENERAL INTRODUCTION 12

乱ndN eva1uations of functions. Thus

，

the Brown method is efficient1y applied to trace solution curve

，

such that the approximate solution is obtained by Hermite po1ynomial.

The a1gorithm presented here can be usefu1 in the ana1ysis of neura1 networks

，

e.g. during the design of temp1ates for cellu1ar neural networks.1七canbe乱ppliedto 1arge

networks provided that the extreme spariもyand the structure of the coefficients are exp1oited. The method can be app1ied for some types of neurons with smooth non -linear output functions or piecewise linear output functions. In genera1

，

there does not seem to be much hope for an e伍cientway to filnd a11 equi1ibrium points in a given neura1 network un1ess appropriate guide1ines are fo11owed during the synthesis process. In Chapter 4

，

first

，

we describe the outer product 1earning approach to set up the weights with suitab1e va1ues which is re1ated to the object patterns information， it is ca11ed as storing object patterns into a ce11ular associative memory. Meanwhile

，

some analyses about the stationary property of the ce11ular associative memory with outer product 1earning ru1e are taken. A condition is:presented which ensure the stored pa七ternsas the stab1e states of a ce11u1ar associative memory. After then

，

a midd1e

-mapping 1earning a1gorithm for cellu1ar associative memory is presented

，

which makes full use of the properties of七hecellular neural network so that every stored pro七0もype can be guaranteed as an equilibrium point of our memory. At the s乱metime

，

it has ability of iterative 1earning. This kind of eomputation is typica1 of a learning process: once the synaptic matrix has been cornputed from a given set of prototype vectors

，

the addition of one extra item of know1edge does not require that the who1e computation is performed again. One just h乱sto carry out one iter乱tion

，

st乱rtingfrom the previous matrix

，

so that the computational e伍ciencycan be improved. Besides

，

its implementation with circuiもsis more feωible because the weight matrix is not symmetric.

Since the synchronous updating ru1e is used in both of them， their associative speeds are very fast compared to the Hopfield associative memory.

In Chapter 5うfirstうwe乱pplyour DTCNN to the feature extraction and noise remove forもheimage processing. Some rea1 image are chosen as our processing object and then

，

input to DTCNN乱sboth input signa1s and initial states. After a few times iterative operations

，

desired resu1ts are obtained. Although the same function can a1so be carried by continuous-time CNN

，

time consuming differentia1 operations are taken during the procedure and more iterative operations are required， Contrasting it

，

our DTCNN rea1ized by software simulation can do them on1y with 5% or 10%

CHAPTER 1. GENERAL INTRODUCTION 13 computing cost

，

so it is faster and effi.cienter tha，n continuous-time CNN in this case. After then

，

we illustrate the potential of DTCNN for the visual pattern recognition. From a prototype composed by over two types of elementsうwecan detect desired visual patterns successfully. When there exist obvious differences between these two types of element8

，

it is easily recognized by human vision system. But for some similar composed elements

，

it is said to be very diffi.cult and time consuming for human vision system. For our DTCNN

，

after suitable template is designed

，

it is easily and quickly to pick out our desired patterns from a prototype in both cases. This technique can be applied for robot vision. Finally

，

based on our convergent analysis result in Chapter 2

，

we design space-varying non-uniform DTCNN for multiple visual patterns recognition. 1n a non-uniform DTCNN

，

two or more templates are used for the cells lying in differenも region of 2-D processing array. Two examples are given toshow the ability of non -uniform DTCNN to detect multiple visual patterns from a prototype at the same time

，

which have distinct geometrical charac七er80 th句rcan not be picked out by unique template at once.Itextented the application region of our DTCNN more over.Since the weight matrix A and B contributedby two or more distinct templates are not symmetrical matrixes

，

or

，

AijヲtAjtand Bij 子~ l~ji generally

，

the stability analysis of non-symmetric continuous-time CNN is still open problem and dose not been solved， the similar application by continuous-time CNN has not been reported until now.

CHAPTER 1. GENERAL INTRODUCTION 14

References

[1] J .M.Zurada

，

Introduction to: Artificial neural systems

，

West Publishing Co・?

1992.

[2] W.S.McCulloch and W.H.Pitts

，

"A logicalcalc山lSof the ideas imminent in nervousactivity"

，

Bull. Math. Biophys.ヲvo1.5

，

pp.115-133

，

1943.

[3] D.O.Hebb

，

The orgαnization of behavior

，

a neuropsychological theory

，

New York: J ohn Wiley

，

1949.

[4] M.Minsky

，

"Neural nets and the brain"

，

L)octo叫 Dissertatio伐 PrincetonU ni

-versity

，

NJ

，

1954.

[5] F.Rosenblatt

，

"The perception: a probabilistic model for informationstorageand organization in the brain"

，

Psych. Rev.

，

voL65

，

pp.386-408

，

1958. [伊例6引]S.Ama紅ri

，

"Leω悶a紅r凶I threshold eleme凶nも1刊Sダ

，

"

IEEE1干αηs.Computers

，

vol:C-21

，

pp.1197-1206

，

1972. [7] S.Amari， 市euraltheory of association and concept formation"， Biol. Cybern.

，

vo1.26

，

pp.175-185

，

1977. [8] K.Fukushima and S.Miyaka

，

"Neocognitron:a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position"

，

Biol. Cybern.

，

vol:36( 4)

，

pp.193-202

，

1980.

[9] T. Kohonen

，

Associative memory: a system~-theoreticαl approach

，

Berlin: Spring -Verlag

，

1977.

[10] T. Kohonen

，

"A simple paradigm forもheself-organized formation of structured feature maps"

，

inCompetition and cooper・αtionin neural nets.

，

ed. S.Amari

，

M.Arbib. vo1.45

，

Berlin: Spring-Verlag

，

1982. [11] T. Kohonen

，

Se

が

organizationandαssociative memory. Berlin: Spring-Verlag

，

1984. [12] J.A.Anderson

，

J.W.Silve凶 ein

，

S.A.Rite and R.S.Jones

，

"Distinctive features

，

categorical perception

，

and probability learning: some applications of neural model"

，

Psych. Rev.

，

vo1.84

，

pp.413-451

，

1~177.

CHAPTER 1. GENERAL INTRODUCTION 15 [13]S.Grossberg

，

"

Classicaland instrumentallearning by neuralnetworks

ぺ

inProgress inTheoretical Biology

，

vo1.3

，

New-York: Academic Press

，

pp.51-141

，

1977. [14] S.Grossberg

，

Studies of mind and brain: neunαl principles of learning perception

，

development

，

cognitionαnd motorcontrol

，

Boston: Reidel Press

，

1982. [15]J.J.Hopfield

，

"Neuralnetworks and physicalsy仰 mswith emergent collective computationalabilities"

，

Proc. N，αtl. Sci.， vo1.79

，

pp.2254-2258

，

1982. [16] J.J .Hopfield， 市euronswith graded response havecollective computationalprop -ertieslikethoseof two stateneurons"

，

Proc. N，αtl.Acad. Sci.

，

vo1.81

，

pp.3088 -3092

，

1984. [17] T.L.McClellandand D.E.Rumelhart

，

Pαrallel distributed processing

，

Cambridge:

MIT Pressand thePDP Research Group

，

1986.

[18] L.O.Ch凶 andL.Yang

，

"Cellularneuralnetwork:theory"

，

IEEE Trans. Circ Syst.

，

vol.CAS-35

，

pp.1257-1272

，

1988.

[19] L.O.Chua and L.Yang

，

"Cellularneural network: application"

，

IEEE ηαηs Circ. Syst.

，

vol.CAS-35

，

pp.1273-1290

，

1988.

[20] L.O.Chua and T.Roska

，

"The CNN paradigm

ぺ

IEEETr，αns. Circ. Syst.-I

，

vol.CAS-40

，

pp.147-156う1993.

[21] T.Roska and L.O.Chua

，

"Cellularneural networks with nonlinear and delay

-typetemplate elements"

，

inProc. IEEE Int. Wor士shopon CNN and Their Applications

，

pp.12-25

，

1990.

[22] H.Harrer and J.A.Nossek

，

"Discrete-timecellular neural networks: Architecture

，

applications and realization"

，

Int.J. of Circuit Theory and Applicαtions

，

vo1.20

，

pp.453-467

，

1992.

[23] Proc. IEEE Int. Workshop on CNN and Their Applicαtions (CNNA-90)

，

Bu-dapest

，

Oct. 1990. [24] Proc. 2nd IEEE Int. Workshop on CNNαnd Their Applicαtions (CNNA-92) Munich

，

Sept. 1992. [25] Inter. J. Cir. Theor. Appl.

，

Special Issue on Cellular Neural Networksぅvo1.20う 1992. [26] IEEE 1干ans.Circ. Syst.

，

Special Issue on Cellular Neural Networks

，

vo1.40

，

1993

Chapter

2

Discrete-time Cellular N

eural

Network

2

.

1

Introduction

Inthis chapter

，

自附

，

a continuous-time ce11ular neural ne凶twor吋比k

仁

叩

NぜN)presented by Chua and Yang[山1]

，

and the network grid structure are introduced

，

theirb紛icprop

-erties ofcontinuous-time CNN are briefly described here. After then

，

as ourstudy

，

a synchronous-updatingclocksignalis introduced into an original continuous-time CNN

，

by sampling the output values at a series of updating moments

，

we obtain another type of discrete-time CNN which circuit topologyis different from that by Harrer and Nossek[2]. After then

，

some detail analyses aboutthedynamical property and stability of out DTCNN are performed. The results ShO"T that

，

if the parameters in the tem -plates are designed carefully so that the convergent sufficient conditions are met

，

the generalized energyfunction is monotone decreasing and the stability of DTCNN can be guaranteed. Cellular neural network( CNN) is a locally connected

，

nonlinear dynamical analog processing array having 2-

，

or 3-dimensional grid architecture. One processingelement

，

called as a cellうwithpiecewise linear output function七emplateis shown on Figure 2.1. In general

，

a11 ce11s are arranged on a 2-D geometrical regular grid( one layer)

，

but this layer can be duplicated to form 3-D multilayer CNN if it is required. Some typical 2-D regular grids are shown in Figure 2.2. For simplicity

，

in this study

，

we just consider the case in which a 2-D rectangular regular grid with M rows and N columns

，

as Figure 2.2(a)

，

is used. In this grid

，

each squarerepresents a CNN cell. The c(i

，

j) denotes a celllying in ith row and jthcolumn.

CHAPTER 2. DISCRETE-TIME CELLULAR NEURAL NETWORK U.. U 、1/u .. '事1・J. R _Y

1/

!

(a)A cell circuit (b)Piecewise linear function Figure 2.1:A continuous-timeCNN cell ， 、 、 ， (a)Rectangular grid (c)Hexagonal grid (b)Triangular grid : ， ， Figure 2.2: Some typical 2-D regular grids 17

18 Every cell just only connects directly with near cellsうwhichconstitute a neighborhood

Nrぽoundthat cell， and the neighborhood of c( i，

j

)

i

s denoted by N_r( i，

j

)

.

The radius

ofもheneighborhood Nr isdenoted by r

，

the number of the cells in

N

r is equal to

(2r+1)x(2r+

1

)

For the cell circuit shown inFigure 2.1うastate equation and an output equation of

a continuous-time CNN are written as follows:

DISCRETE-TIME CELLULAR .NEURAL NETWORK CHAPTER2. (2.1a) (2.1b)

二

一

干

↓

L

い

υ

叫附

tり3 A む切X c(k州，，1り)ε Nr(

i

υ

ω

しρ，j)

+ 乞

B(i

，

j;k

，

l

)

U

k

l

(

t

)

+

1 c(k，l)εNr(i，j) =;(￨Ut

仲

c

色辿

dt Yij

(

t

)

Vjε{1?2F- N} Viε{1

，

2γ ・

，

.

M}

，

In order to analyze system characもereasily

，

we rearrange all cellsintoone-dimensional vectorform inthe order ofrows. Then， the cell is denoted by c(i)

，

iε{1，え・・・

，

n

}

and

n=λ1/

x

N. (2.2) Corresponding to this description style

，

we defilne a matrixSεRnxη乱S S三

{

S

;j: Inthis way

，

the continuous-time CNN is described by followingequations. 1 i ハ U 一 一 一 一 ィ J . 7 4 Q U Q U (2.3a) (2.3b)

-

A

v

(

t

)

+

A

y

(

t

)

+

Bu

+

1

Sα

t

(

v

(

t

)

)

伽一の川 W C 3 where -1 ~ Yi三1

，

i = 1γ・1n}; with入 ]ι>0 .L"X AεRη×η三

{A

_ij;1三t三仏 1三jざη}; v ε R

ぺ

u ε R

ぺ

yε Dn三 {YiεRη: Aε Rnxη 三 diag[入・・・入]

CHAPTER 2. DISCRETE-TIME CELLULAR NEURAL NETWORK 19

Bε Rnxη

三 {B_ij;1三iS n

，

1三j三η}

Here

，

both A and B are sparsematrixes. Theirelements satisfy thefollowing condi -tions:

Aij= Aij. Sij (2.3c)

Bij

=

Bij . si~í (2.3d)

It means that

，

when Sij= 0

，

Aijand Bijare equal to zero

，

but in other case

，

they

are equal to arbitrary real number decided by the particular purpose of CNN. Iε Rn三 {11ん .

.

.

In} s α

t

(

v

(

t

)

)

= [5α

t

(

V

1

(

t

)

)

5

a

t

(

v

2

(

t

)

)

.

.

.

α5

t

(

V

n

(

t

)

)

]T ; 1 i l u ー 一 fl く

l

t

一 一 、_、 B ， ， ノ U J ' ' E ‘_、 4 2U α c u 叫 >1 -1

S

V

i

S

1 叫 <-1 (2.3e)

When the next two conditions are met

，

IVi(O)1三1

，

IUilS 1

(

2

.

4

)

the range of dynamics is bounded by a single number M which can be calculated in terms of the cloning templates:

M

=

max{lvil}

=

max{l

+九

111

+ 九 乞

(

I

A

ij1

+

I

B

ijI)} (2.5)

c(j)εNr(i)

Moreover

，

if the following condition is satisfied

，

Aii>

土

(

2

.

6

)

for symmetric continuous-time CNN， its stability can be proved. Then，もheconver

-gent results can be derived as follows:

民

1

V

i

(

t

)

1

>

1

i

E

q

i

u

t

(

t

)

=

土l

(

2

.

7

)

(

2

.

8

)

CHAPTER 2. DISCRETE-TIME CELLULAR NEURAL NETWORI<

2

0

2

.

2

Discrete-time c

e

l

l

u

l

a

r

n

4

B

u

r

a

l

network

In this section

，

we build up a model for our discrete-timeCNN. First

，

a stateupdating signalis introduced into a cellular neural network

，

so that a synchronous-updating cellularneural附 work(SCNN)is obtained. which means that

，

at the kthupdating timet

=

=

kT

，

the statesof all cells are alもeredsimultaneously.Here

，

T describesthe updating period in our SCNN

，

According to this rule

，

a cellof SCNN isshown as Figure 2.3(a). Y i (t) Buu i 己~) Biju j .1

~

"

"

"

1

YCj(t~ (b) Piecewise-linear output function

Y I Yj(k)

kT <k+l)T (a)A cellc(i)of SCNN (c) Clock signalfor SCNN

Figure 2.3: A synchronous-updating CNN cell

φand φin Figure 2.3(a) are a clock signal and :its inverse

，

they control two updating switchs respectively φis shown as Figure 2.3(b). Cx and Cy are two sample-hold

capacitors. DuringゆTlphase of the kth clock periodうtε (kT

，

kT+T

，

]

l

k=O

，

1

，

2

，

・・1φ=1

，

φ = 0

，

the terminal voltageVi(t)inCx is kept as its initial valueVi(t)= vi(k)

，

k = kT The voltage-controlled voltage sourceYi(t)

=

=

sat(vi(t)) is also a constant during this phase

，

denoted by Yi(k). The capacitorCy ischargedby the voltage sourceYi(k) through the resistanceRy.Sinceもhevalue ofRJ~ is a very small and in the order of the internal resistance of the voltage sourceYi(t)

，

this charge is finished quickly in very shorttime

，

we can writeYcJt)= Yi(k)after the transient response

，

about2.3RyC

い

is completed.The voltageVRi

(

t

)

is determined by the resistanceRx

，

the current source

CHAPTER 2. DISCRETE-TIME CELLULAR NEURAL NETWORK 21

Iiand the voltage-controlled current sourcesAijYcj (t)and Bijuj

，

jε Nr(i). The equivalent circuitis illustrated inFigure 2.4，itisa one-order nonlinear dynamic circuit. Figure 2.4:The equivalent circuitofSCNN cell during

ゆ

'

T

l A state equation and an output equation of this circuit model are

c生必

ν dt 、_、 BE ， r 、_、 EE ， J 4 ' u 〆 ' e a 、_、 U 〆 ' a t 、_、 ふ し

a

Q U 、_、 E ， ， ， 4F U 〆 ' ' t 、_、 c u u l

一 九

_(2.9a) 1 . 1 :::.. ，.!!: 仇

ω

州

4パぷ

(

例

刈

t

の

t

)

二

一石ト瓦句叫州州

tパぷ

例

(

刈

t

t

の

)

H

+

子

F

E

プ

(

l

?

A

ん川

i j ヲ手t:i

Then

，

we considerもhe

ゆ

T2phase of the kth clock period

，

t

ε(

kT

+

T₁

，

(k

+

l)T] and φ = 0 φ = 1 inFigure 2.3. The piecewiselinearvoltage-controlled voltage source

Yi(t)is varying with the voltage Vi(t)

，

but sinceφ = 0

，

it has no feedback effectionも0

Vi(t)during thisphase. The one-order dynamic circuitisconsistsof_Cx

，

Rx

，

Ii乱n

the voltage-controlled current sourcesAんtりijY釣りCj(

例

tt

の

)

a 凶 広Bρij川i刈刈刈的U川Lりj

，

here

，

YCj (t)isthe terminal voltage inCνof the jth cell.Sincethe terminal voltageYCi

(

t

)

=

ぉ(k)?t=1?2?...?n is held as a constant here

，

thisdynamic circuitisequivalent to a linearRC dynamic circuit

，

the initialvalue ofthe terminal voltageinCx is determined by Vi(t)in previous clockupdating moment t = kT

，

i.e.，的(k).Obviously， after2.3RxCx

，

the transientre

-sponse issettled to zero

，

the circuitmust convergenももoiもssteady state.The equivalent circuitisshown in Figure 2.5.

Corresponding to this circuitmodel

，

we can derivea state equation and an outpuも

equation asfollows: ー ノ -1 I J 4b 一 }4 b ' /l¥-4L/l 、 向 一 d 仇 JU 一 z

c

L

+

乞

_同

n B u

+

L

ん A n

玄

同

+

仙

w

リ

叫 イ 川 、 z u l 一 R

州

-Q U (2.10a) (2.10b)

22

DISCRETE-TIME CELLULAR NEURAL NETWORK

CHAPTER2. VRi (t) V i (t) -l u Figure 2.5: The equivalent circuit of SCNN cell during

ゆ

'

T

2 In contrast to continuous-time CNNs

，

Yj(k)in the state equation (2.10a) is a sampled state of output variant at the kth updating timet =

kT

and is held in the capacitor Cy. The feedback strength

'

2

:

/

]

=

1

AijYj(k)from the neighbor cells to a cellc(i)remains a constant value fort

ε

(kT

，

(k

+

l)T

，

]

but the variantsVi(t)and Yi(t)in the equations are varying continuously with timet. The equations (9) and (10) describe the state a，nd output equations of SCNN in

O

T

l

and

ゆ

T2phases respectively. Combining them together

，

we get a set of equations to

describe SCNN in a whole clock period. (2.11a) (2.11b) When t

ε(

k

T

，

kT +T

I

]

か

Ci

(

t

)

一

S削乱剖t

い

一土(卜一土句川似

ω

t

パぷ

(

t

)

十

￡

む

A

i

j

Y

C

i

(

t

)

+

土

B

i

j

U

j

十

I

i

]

Att R z j = 1 1 3 c j j=1 j手t

c

坐必

y dt YCi

(

t

)

(2.11c) When t

ε(

kT

+

T

1

，

(

k

+

l

)

T

]

古

川

(t)

+

L

Aij Yj(k)

+

L

Bij Uj

+

Ii ハ dVi(t) '-'X dt (2.11d) Viε{1

，

2γ・.

，

n

}

sat(Vi(t)) Yi(t) k = 0

，

1

，

2

，

'

In order to derive a DTCNN

，

extracting the state variableViand the output variable

Yi at a series of updating moments t

=

kT

，

k

=

0，1，2.・." and assume the updating

23 DISCRETE-TIME CELLULAR lvEURAL NETWORK

CHAPTER2. decayed to zero

，

dVi( t) / dt= 0 is kept ateveryupdating moment. Inthis case

，

we can o btainthefollowi時 discrete-timeequationsfrom theequation(11). (2.12a)

玄

AijYj(k)

+

L

Bij Uj

+

Ii sat(vi(k

+

1

)

)

士

山

(2.12b) Viε{1

，

2γ・.

，

n

}

Yi(k

+

1

)

k

=

0

，

1

，

2

，

.

They describethestate and the output of our discrete-time CNN. Here

，

we derive a discrete-もimeCNN which topology is distinct from the DTCNN presented by Harrer

and Nossek

，

but has more tighter correspondingrelation withcontinuous-timeCNN.

(2.13a) (2.13b) Moreover

，

we canwriもethem as vectorequationsas follows. Ay(k)

+

Bu

+

1 sat(v(k+1))

土

v(k+1)

y(k

+

1) A

，

1~ε Rnxn

DTCNN

u

，

1ε R

ぺ

of our

v

，

y

，

Dynamical range

where k

=

0

，

1

，

2γ

，

・

2

.

3

In order to implement physical DTCNNヲweneed to investigate its dynamic range. In

an continuous-time CNN) the topology of the network is unvaried

，

its dynamic range has been proved for the initial state

I

V

i

(

O

)

1

<

1.But in our network

，

the topology is time-variant

，

the initial value of Vi in each updating period is obtained from the steady-state value of the last updating period

，

so that its initial value may be changed within its dynamical range. To get the dynamic range of synchronous-updating CNN

，

let us consider the equivalent circuit shown as Figure 2.6. In Figure 2.6

，

Ii

= E

j

=

l

Aij Yj( k)

+

乞j=lBij Uj・ WhileinゆT2of the kth clock period

，

φ = 0

，

φ = 1

，

Vi二 V_Ri

，

assumingT2 ~ 2.3RxCx) when t = (k + l)T

，

the circuit converges to iもssteady state. Analyzing the equivalent circuit in steady state

，

we can get the maximum value ofVi

(

t

)

in steady stateωfollows. Rx (Ii

+

Ii ) Rx [

L

Aij Yj (k)

+

L

Bij Uj

+

Ii 、_{、 ‘ ， ， ノ} 4 ' u 〆 ' E 目 、 _、 - a ， . U

CHAPTER 2

.

DISCRETE-TIME CELLULAR lvEURAL NETWORK

here

，

we define U i Figure 2.6: The equivalent cell circuit of SCNN

<

<

Rx[

L

:

IAijIIYj(k)1

+玄

1

Bij

1

1

Uj

1

+

1

Ii

1

]

Rx

[

2

:

1

Aij・

1

+乞1

Bij

1

1

Uj

1

+

1

Ii

1

]

υ m ω

=

Rx { max[

L

:

1

Aij

1

+乞

1

Bij

1

1

Uj

1

]

+

1

川 }

Next

，

in

C

T

T

l

phase of the next clock period

，

t

ε( (k

+

1 )T

，

(k

+

l)T

+

T

I

]

，

1

，

φ = 0

，

Vi(t)is held asvi(k + 1)

，

but from (2.11b)

，

we have VRi(t)

=

Rx [

乞

AijYj(t)

+

L

:

Bijuj

+

Ii Obviously

，

りん

(

t

)

三Vmax tε( (k

+

l)T

，

(k

+

l)T

+

T1] 24 (2.14) (2.15) φ =

The value of the voltage sourceYi(t)depends on Vi(t)

，

so that during this phase it is also a constant. We denote it asYi(k

+

1). By charging toC_y

，

it is stored inC_y. In the followingゆT2phase

，

tε( (k

+

l)T

+

T1

，

(k -+-2)T

，

]

争 =0 andる =1 againぅthe dynamical character of this cell is described as Vi(

t

)

G7=τWHEAtj

仙

where tε((k

+

l)T

+

T1

，

(k

+

2)T]. In order to analyze the transient response

，

we solve this equation

，

the initial voltage inCx isVi( k

+

1) obtained previously

，

thus we get

CHAPTER 2. DISCRETE-TIME CELLULAR NEURAL NETWORK 25 、_、 t z ， ノ 4 g h v 〆 ' f s 、_、 a b u tー(k+l)T-Tl_ 1 r::'" ， _ ::..._ _， r 叫州仙(伏

糾

k

い

山

山

+

刊

刊

り

1

巾

山

)μe一 RxG白:r:;r; + 五瓦町;

5

ご

戸瓦

;

[

E

?

A

山

+1

咋

1玖鳥

パ 川

j.ιu t ト一(μk+l吋)T一T町1 1 η vi(k+1)e RxGx +~[乞 Aijyj(k

+

1)

+乞

BijUj

+

Ii] RxCx [1-e . RxCx ] Cz j=1 的(k+ 1)e - Rx Gx + Rx [

玄

AijYj(k+ 1)

+

主

BijUj

+

山

1-e(ktJ71-t) tー(k+l)T-TJ j=l j=l where t

ε(

(k+ l)T+ T1

，

(k+ 2)T] Then

，

we obtainthemaximum value ofVi(t)during wholeゆT2phase. tー(k+l)T一五 九 IVi(t)

I

:

:

;

IVi((k+1)T)le- 'RxCx

-

A

-

+

九

￨

乞

AijYj(k+1) j=l T -z + 一 円 T 一 z り 一 R + 一 ρ U

L

+

u B

η

Z

M

パ

+

m

u < Inthisway，もhebiggest dynamical range of our DTCNN is illustrated as

I

Vi(t)

I

三

I

V

m ω

I

for allt and Vmω

=

Rx {

I

Ii

I

+ max[

L

:

I

Aij

I

+乞

I

Bij

I

I

Uj

I

]

}

(2.16) Here

，

the maximum value ofVi(t)is less than that of the continuous-time CNN

，

in addition of that

，

the required initial condition

I

Vi(O)

I

三1is alsoeliminated

2

.

4

DTCNN

with binary ou

.

t

put

From previous analysis

，

we proved the dynamical range of state variables in SCNN. Since a DTCNN is derived from a SCNN by extractingもhestates and outputs at a series of discrete timet

=

0

，

T

，

2T

，

・." in general

，

the values of the state variables in DTCNN is a set of arbitrary numbers which amplitudes are less thanIvmaxl.Then

，

the outputYi(k+ 1) in DTCNN is a variable from -1 to +1

，

its value is determined by the statevariablevi(k+ 1) with (2.12b). But for SOllle applications

，

the binary output is required.In this section

，

we give two theorems to describe the sufficient condition and necessarycondition respectively to guarantee a set of binary outputs in our DTCNN.

DISCRETE-TIME CELLULAR lVEURAL NETWORK 26 Theorem 2.11f the following condition is satisfied

，

the output ofDTCNN must be equα1 to十1or -1. CHAPTER2. (2.17)

L

+

u B η

ヤ 山

一

戸

+

A n

乞出

+

1 一 九 > 一 A Proof:From (2.12a)

，

we have

土

州

+1)==

主

A

川

When the output amplitudeIYi(k

+

1)1 is equal to 1

，

the amplitude of state variable must be greater than 1 or equal to1

，

IVi(k

+

1)1ど 1.so that we have (2.18) lbd3(k)+

土

BijUj + 1il

三

土

I

L

Aij Yj(k)

+乞

Bt3U1+L￨ Since さ￨乞AijYj(k)I-I

I

:

Bijujl-11i (2.19) 三IAiiYi(k)

卜乞

IAijYj(k)1

一

￨

乞

BijUjl-11il jヲI:i どIAiiYi(k)

卜乞

￨Atj￨-lZBtjUj￨-lL￨ j戸

From (2.18) and (2.19) we can五ndthat when the equation (2.17)is met

，

IVi(k+1)1ど 1

，

then

，

IYi(k

+

1)1 == 1

，

theoutput isa binaryvalue 口 Inthenexttheorem

，

we give the necessary conditionfora binary output in DTCNN. Theorem 2.21ftheoutputYi(k+1)ofDTCNN isαbinary vαlue

，

thefollowi旬 relαtion must besatisfied. (2.20)

L

U B n て ム - F A n

乞 一

間 同

1 一 九 > 一 A

27

DISCRETE-TIME

CELLULAR NEURAL NETWORK

CHAPTER2.

When the outputYi(k

+

1)ofDTCNN isa binary value

，

we must have Proof: (2.21 )

￨

主

AijYj(k)

+

土

Bij

_い

Iil

三

土

IVi(k

+

1)1三1so that

￨

乞

AijYj(k)

+

玄

BijUj+L Since (2.22) L .

+

u B n

乞

州

+

A n

ヤ ム

一 信

+

A < 一 Considering the equation (2.21)

，

it can be foundwhen

い

1

叫

Vj (ρ2.2

却

刈

O

的

)

mus剖tbe s 乱剖tisf五ie吋d. 口

From above two theorems， it is known when the templatesfor DTCNN are designed to meet the equation (2.17)

，

DTCNN can be used to realize the mapping from RnもO

Bn so that the equation (2.13) can be written as (2.23a) (2.23b) Ay(k)+Bu+I sat(v(k

+

1)) 一

十

(k

+

1) y(k

+

1) A

，

B

ε

Rnxn. y ε B

ぺ

u

，

1

ε R

ぺ

where k

=

=

0

，

1

，

2γ・.

，

ofDTCNN

v

，

analysis

S

t

a

b

i

l

i

t

y

2

.

5

In other chapters

，

we will give the applications of our DTCNN to image processingう include associative memory and visual pattern recognition. In these applications，五rst， a probe image with multiple gray level is inputted into DTCNN as its initialstateat

t

=

=

O. Then

，

by some times of updating

，

final stablestateis obtained

，

which means

that the subsequentsetofstatearethe same totally

，

no statechange is risen by a clock

signal.This final stable state is corresponding to an equilibriumpoint distributed in DTCNN's dynamical space. The output in the finalstable stateis an object image.

CHAPTER 2. DISCRETE-TIME CELLULAR lVEURAL NETWORK 28 and the matrix

B

aredesignedtomeet the sufficient condition (2.17)

，

the pixelinthe objectimage is only binaryvalue+ 1 and -1. Itis known thatone of the mosもeffectivetechnique for analyzing theconvergence proper七iesof dynamic nonlinear circuits is Lyapunov's method. This method isalso used by N.Fruehauf

，

L.O.Chua and E.Lueder forconvergenceanalysis of reciprocal DTCNN withcontinuousnonlinearities[4]， but their result is justsuitablefor reciprocal DTCNN

，

not for general case. Inthis sectionう五rst

，

Lyapunov energyfunctionsaredefinedtoSCNN and DTCNN respectively. Then

，

the convergence conditionisanalyzedfor SCNN. Since our DTCNN is obtained by extractinga seriesof updating moments from SCNN

，

all convergence analyzingresults areeasily extendedto DTCNN. Our basic object isconcentratedon general case， i.e.， nonuniform and nonreciprocal DTCNN， whichcoversa reciprocal DTCNN's convergence conditionby N.Fruehauf

，

L.O.Chua and E.Lueder just as a special case. Firstうwedefinea generalized energyfunction for SCNN asfollows: -izzAjUt(t)Uj(k)一EEPB玖

川

ijj + 主主占

5

;

t

E?Ud山山?穴灼削(

り

作t

)

例 一

Z

L

Ut (t) where t

ε

(kT

，

(k+1)T

，

]

k=0

，

1

，

2

，

'

E(t) = (2.24) Yj(k)in the equation (2.24) is a constant

，

Yi(t)is a variable duringゆT2'but inゆTl phase，抗(t)is kept as a constant so thatE(t)is invariant in this ph乱se.At theupdating moments oft

=

(k+ l)T

，

k

=

0

，

1

，

2γ・"the value ofYj(k+l) is substituted intoYj(k)ぅ

sothatE(t)may jump at those moments

Meanwhile

，

a gener乱lizedenergy function for our DTCNN is defined as follows: n n n η E(k+1)= -

L2

二

AijYi(k

+

1) Yj(k) -

乞乞

BijYi(k

+

1) Uj i=1 j=1 i=1 j=1 1 i

+

' k

u u

r れ η ヤ μ M 1E よ

+

' k q L ・t

u u

n

乞 同

l 一 丸

+

(2.25) Since the energy function denoted in (2.24) is consists of four sums of finite items， o bviously， iもisbounded乱，ndthen， we will provethatif the convergent conditions are met

，

this energy function is a uncontinuous monotone decreasing function

，

so that the differentialofE(t)to timeT is equal to zero when T is tending toward infinite

CHAPTER2. DISCRETE-TIME CELLULAR lVEURAL NETWORK 29

1~__ dE(

t

)

ハ

ー … 一

-t→ゐ

d

t

Differing from the continuous-time CNN

，

here

，

theE(t)is an unco凶 nuousmonotone

decreasing function so that this equation has two meaningsう五凶，iもdenotesthatE ( t)is

kept as a constant d uringゆTlphase

，

but it is monotonely decreasing withinゆT2phase.

Second， around an updating instantaneous， the value ofE(t)may be suddenly changed，

but its monotone decreasing property is still remained

，

so that七heinstantaneous value ofE(t)under a updating must less than or equal to the previous value before this updating. This situation is illustrated in Figure 2.7. During every period

，

E(

t

)

is co凶 nuousdecreasing， but at some updating mornents， for examples， att

=

(k -l)T and t = (k

+

l)T， it is reduced uncontinuously (2.26) E(。 一--:T t-..一一一一一一一一T^1-一ーー一一一一'・ (k・l)T kT (k+l)T Figure 2.7: Uncontinωus monotone decreasingE (

t

)

curve Aもthesame time

，

from (2.24)

，

we can find E 一_d J U 一

d

Y

i

(

t

)

~ ~ D n.

d

Y

i

(

t

)

I 1 ~ n. f4-¥

d

Y

i

(

t

)

~ T

d

Y

i

(

t

)

一 吾 川 (

(2.27) where t

ε

(kT

，

(k+l)T] so that if the convergence conditions are met

，

the differe凶 alof

Y

i

(

t

)

to time

T

is also equal to zero when T is tending toward infinite.