Towards Maximum Geometric Margin Minimum Error Classification

(1)

_________________________________

*Graduate School of Engineering, Doshisha University, Kyoto, E-mail: skatagir@mail.doshisha.ac.jp, Telephone: +81-774-65-7567

**NTT Communication Science Laboratories, NTT Corporation, Kyoto

***Mastar Project, National Institute of Information and Communications Technology, Kyoto

Towards Maximum Geometric Margin Minimum Error Classification

Kouta YAMADA*, Shigeru KATAGIRI*, Erik MCDERMOTT**, Hideyuki WATANABE***, Atsushi NAKAMURA**, Shinji WATANABE**, Miho OHSAKI*

(Received July 28, 2009)

The recent dramatic growth of computation power and data availability has increased research interests in discriminative training methodologies for pattern classifier design. Minimum Classification Error (MCE) training and Support Vector Machine (SVM) training methods are especially attracting a great deal of attention. The former has been widely used as a general framework for discriminatively designing various types of speech and text classifiers; the latter has become the standard technology for the effective classification of fixed-dimensional vectors. In principle, MCE aims to achieve minimum error classification, and in contrast, SVM aims to increase the classification decision robustness. The simultaneous achievement of these two different goals would definitely valuable. Motivated this concern, in this paper we elaborate the MCE and SVM methodologies and develop a new MCE training method that leads in practice to the best condition of maximum geometric margin and minimum error classification.

-G[YQTFU㧦minimum classification error training, geometric margin, functional margin, support vector machine

ࠠ࡯ࡢ࡯࠼㧦ᦨዊಽ㘃⺋ࠅቇ⠌㧘ᐞ૗ࡑ࡯ࠫࡦ㧘㑐ᢙࡑ࡯ࠫࡦ㧘ࠨࡐ࡯࠻࡮ࡌࠢ࠲࡯࡮ࡑࠪࡦ

ᦨᄢᐞ૗ࡑ࡯ࠫࡦᦨዊಽ㘃⺋ࠅቇ⠌ᴺࠍ⋡ᜰߒߡ

ጊ↰ ᐘᄥ㧘 ᩿ ṑ㧘ࡑࠢ࠳࡯ࡕ࠶࠻࡮ࠛ࡝࠶ࠢ㧘ᷰㄝ ⑲ⴕ㧘 ਛ᧛ ◊㧘ᷰㇱ ᤯ᴦ㧘ᄢፒ ⟤Ⓞ

ߪߓ߼ߦ

⛔⸘⊛ࡄ࠲࡯ࡦ⹺⼂ߩⓥᭂߩ⸳⸘⋡ᮡߪ㧘ࡌࠗ࠭

࡝ࠬࠢ⁁ᘒ㧘ߔߥࠊߜ⺋ࠅ₸߇ᦨዊߣߥࠆࠃ߁ߥ⁁

ᘒࠍ㧘ಽ㘃ེࠍ⸠✵ߒߡታ⃻ߔࠆߎߣߢ޽ࠆ 㧚ߎ ߩࠃ߁ߥⷰὐ߆ࠄ㧘ฎౖ⊛ߥ⼂೎ቇ⠌ᴺߢ޽ࠆᦨዊ

ੑਸ਼⺋Ꮕ㧔Minimum Squared Error: MSE㧕ቇ⠌ߦઍࠊ ࠅ 㧘 ᦨ ዊ ಽ 㘃⺋ ࠅ Minimum Classification Error:

MCEቇ⠌㧘ࠨࡐ࡯࠻࡮ࡌࠢ࠲࡯࡮ࡑࠪࡦSupport

Vector Machine: SVM㧘᧦ઙઃ߈࡜ࡦ࠳ࡓ࡮ࡈࠖ࡯

࡞࠼Conditional Random Field: CRF)⁴⁾㧘ࡉ࡯ࠬ࠹ࠖࡦ

ࠣBoostingߥߤߩ㧘᭽ޘߥᚻᴺߩ⎇ⓥ߇ㅴ߼ࠄࠇ ߡ޿ࠆ㧚ߎࠇࠄߩ⼂೎ቇ⠌ᴺߩ᦭ലᕈߪ᣿ࠄ߆ߢ޽

ࠅ㧘ᄢⷙᮨߥታ਎⇇ߩಽ㘃໧㗴ߢ߽㧘ᮡḰ⊛ߥ⸠✵

ᚻᴺߣߒߡ⏕┙ߐࠇߡ޿ࠆ㧚

(2)

ߎࠇࠄߩᚻᴺߩਛߢ߽㧘MCEߣSVMߪ․ߦᵈ⋡

ߔߴ߈․ᓽࠍ஻߃ߡ޿ࠆ㧚MCEߪಽ㘃⺋ࠅᢙࠍᦨዊ ൻߔࠆߎߣࠍ⋥ធ⊛ߦ⋡ᜰߔቇ⠌ᚻᴺߢ޽ࠆ㧚ಽ㘃

ེߩ⸠✵⋡ᮡߣߒߡ㧘ಽ㘃⺋ࠅᢙߩᦨዊൻߪ⥄ὼߢ ℂߦ߆ߥߞߚ߽ߩߢ޽ࠅ㧘ಽ㘃♖ᐲะ਄ߦ⋥ធ⽸₂ ߔࠆߎߣ߇ᦼᓙߢ߈ࠆ㧚৻ᣇ㧘SVMߪಽ㘃Ⴚ⇇ߣߘ ࠇߦᦨ߽ㄭ޿ࠨࡦࡊ࡞ࡄ࠲࡯ࡦߣߩᐞ૗ࡑ࡯ࠫࡦࠍ ᦨᄢൻߔࠆߎߣࠍ⸠✵⋡ᮡߣߒߡ޿ࠆ㧚ᐞ૗ࡑ࡯ࠫ

ࡦᦨᄢൻߦࠃࠅ㧘ᧂ⍮ࡄ࠲࡯ࡦ߳ߩಽ㘃⠴ᕈࠍะ਄

ߐߖࠆߎߣࠍ⋡ᜰߒߡ޿ࠆ㧚

MCEߣSVMߣߩ㧘ߎࠇࠄ ⒳㘃ߩ⸠✵ᚻᴺߢ↪

޿ࠄࠇߡ޿ࠆᦨㆡൻၮḰࠍหᤨߦḩߚߔࠃ߁ߥ⸠✵

߇ߢ߈ࠇ߫㧘ࠃࠅ⦟޿ಽ㘃ེࠍታ⃻ߢ߈ࠆߣ⠨߃ࠄ ࠇࠆ㧚ߒ߆ߒਔ⠪ߩቯᑼൻ߿ߘߩዉ಴ߩᨒ⚵ߺߦᄢ ߈ߥ㓒ߚࠅ߇޽ࠆߎߣߥߤ߇ේ࿃ߒߡ߆㧘ߘߩࠃ߁ ߥ⛔วߐࠇߚ⸠✵ᚻᴺߪߎࠇ߹ߢታ⃻ߐࠇߡ޿ߥ޿㧚 ᧄ⺰ᢥߢߪߎߩࠃ߁ߥ⢛᥊߆ࠄ㧘MCEߣSVMߩ㐳 ᚲߣ⍴ᚲࠍ␜ߒߚ਄ߢ㧘MCEቇ⠌߳ᐞ૗ࡑ࡯ࠫࡦᦨ ᄢൻߩ᭎ᔨࠍዉ౉ߔࠆߎߣࠍᬌ⸛ߔࠆ㧚MCEߦ⌕⋡

ߔࠆℂ↱ߪ㧘ၮᧄ⊛ߦ ࠢ࡜ࠬߩ࿕ቯᰴరࡌࠢ࠻࡞

ߩಽ㘃↪ߦቯᑼൻߐࠇࠆSVMߣᲧߴߡ㧘MCE߇㧘㖸ჿ⹺⼂⺖㗴╬ߩታ਎⇇ߩࡄ࠲࡯ࡦ⹺⼂⺖㗴ߢⷐ⺧

ߐࠇࠆ㧘ᄙࠢ࡜ࠬߩนᄌ㐳ࡄ࠲࡯ࡦߩಽ㘃ߦㆡߒߚ ቯᑼൻߩᨒ⚵ߺࠍ᦭ߒߡ޿ࠆߎߣߦ޽ࠆ㧚

/KPKOWO%NCUUKHKECVKQP'TTQT ቇ⠌

ಽ㘃࡝ࠬࠢߣ᳿ቯೣ

MCE ቇ⠌ߪరޘ㖸ჿࡄ࠲࡯ࡦ⹺⼂ಽ㊁ߢ↪޿ࠆ ߎߣࠍᗐቯߒߡ㐿⊒ߐࠇߚ߼㧘࿕ቯ㐳㧔࿕ቯᰴర㧕 ࡌࠢ࠻࡞ߛߌߢߪߥߊ㧘นᄌ㐳ࡌࠢ࠻࡞೉ࠍᛒ߁ࠃ ߁ߥࡄ࠲࡯ࡦಽ㘃໧㗴ߦ߽ㆡ↪ߢ߈ࠆ㧚ߒ߆ߒߎߎ ߢߪ⼏⺰ࠍන⚐ߦߔࠆߚ߼ߦ㧘࿕ቯ㐳ߩࡄ࠲࡯ࡦࡌ

ࠢ࠻࡞xࠍಽ㘃ߔࠆ໧㗴ࠍ⠨߃ࠆ㧚ߎߎߢxߪJ୘ ߩࠢ࡜ࠬߩਛߩ৻ߟߢ޽ࠆࠢ࡜ࠬC_jߦዻߒߡ߅ࠅ㧘 ಽ㘃ེߩ⸳⸘↪ߦ㧘ߎࠇߣห᭽ߥࡄ࠲࡯ࡦ߆ࠄᚑࠆ 㓸วF_N( { ,x₁,x_N})߇ਈ߃ࠄࠇߡ޿ࠆ߽ߩߣߔࠆ㧚

߹ߚ㧘ಽ㘃ེߪ⸠✵น⢻ߥࡄ࡜ࡔ࡯࠲_ȁߦࠃߞߡ᭴

ᚑߐࠇߡ޿ࠆ߽ߩߣߔࠆ㧚

⛔⸘⊛ߥࠕࡊࡠ࡯࠴ߦࠃࠆಽ㘃ེߩ⸳⸘ߪ㧘ၮᧄ

⊛ߦࡌࠗ࠭࡝ࠬࠢߩᦨዊൻߣ޿߁᭎ᔨߦၮߠ޿ߡⴕ ࠊࠇࠆ㧚ߎߎߢࡌࠗ࠭࡝ࠬࠢߣߪ㧘୘ޘߩࡄ࠲࡯ࡦ ߩಽ㘃⚿ᨐߦᔕߓߡ⺖ߐࠇࠆ࡝ࠬࠢ㧘ߔߥࠊߜ޽ࠆ

⒳ߩ៊ᄬ୯ࠍⓍಽߒߚ߽ߩߢ޽ࠆ㧚៊ᄬߩㆬᛯߣߒ ߡᦨ߽⥄ὼ߆ߟၮᧄ⊛ߥ߽ߩߪ㧘ಽ㘃⚿ᨐ߇⺋ࠅߢ

޽ࠆ႐วߦ୯߇ ߣߥࠅ㧘ᱜߒߊಽ㘃ߢ߈ߚᤨߦߪ ߣߥࠆ߽ߩߢ޽ࠈ߁㧚ߎߩ៊ᄬߪ㧘ࡠࠫࠬ࠹ࠖ࠶

ࠢ៊ᄬߣ߽๭߫ࠇࠆ㧚਄⸥ߩಽ㘃໧㗴ߢߪ㧘ࡠࠫࠬ

࠹ࠖ࠶ࠢ៊ᄬߪએਅࠃ߁ߦቯ⟵ߢ߈ࠆ㧚 1 if , ( | )

0 otherwise,

i j

i j O D C _®

¯

z

ߎߎߢD_iߪಽ㘃⚿ᨐ߇ࠢ࡜ࠬC_iߢ޽ࠆߣ޿߁ߎߣ ࠍ⴫ߒ㧘O D( _i|C_j)ߪࠢ࡜ࠬC_jߩࡄ࠲࡯ࡦࠍࠢ࡜ࠬ

Ciߦಽ㘃ߒߚߣ߈ߦ⺖ߐࠇࠆࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬ ࠍ⴫ߔ㧚ߎߩߣ߈ಽ㘃ེߩ⸳⸘⋡ᮡߪ㧘એਅߩᑼߢ

⴫ߐࠇࠆ࡝ࠬࠢߩⓍಽ୯ࠍᦨዊൻߔࠆࠃ߁ߥ㑐ᢙ D( )x ࠍ⷗ߟߌࠆߎߣߢ޽ࠆ㧚

1

( ( ) | ) ( , )

J

j j

j

R

¦³

_FO D ^x C p C ^{x x}d ߎߎߢFߪࡄ࠲࡯ࡦⓨ㑆ో૕ࠍ⴫ߒߡ߅ࠅ㧘D( )x ߪ

޽ࠆࡄ࠲࡯ࡦxࠍಽ㘃ߒߡ㧘ࠢ࡜ࠬ࡜ࡌ࡞ࠍ⚿ᨐߣ ߒߡ㄰ߔ㑐ᢙߢ޽ࠆ㧚ߎߩᑼߢ⴫ߐࠇࠆ࡝ࠬࠢߩⓍ

ಽ୯ߪ㧘⠨߃ᓧࠆోߡߩࡄ࠲࡯ࡦࠍಽ㘃ߒߚ႐วߩ ಽ㘃⺋ࠅᢙࠍࠞ࠙ࡦ࠻ߒߚ߽ߩߣߥߞߡ޿ࠆ㧚 એਅߩࠃ߁ߥ᳿ቯೣࠍታ⃻ߢ߈ࠇ߫㧘ᑼߩ࡝ࠬ

ࠢࠍᦨዊൻߢ߈ࠆߎߣ߇଻⸽ߐࠇߡ޿ࠆ㧚

| ) ( | ) for all decide C_i if (P C_i x !PC_j x jzi ߎߩࠃ߁ߦ㧘੐ᓟ⏕₸߇ᦨᄢ୯ࠍขࠆࠢ࡜ࠬߦࡄ࠲

࡯ࡦࠍಽ㘃ߔࠆ᳿ቯೣࠍ㧘ࡌࠗ࠭᳿ቯೣߣ๭߱㧚ߒ ߆ߒߎߎߢ૶ࠊࠇࠆ੐ᓟ⏕₸ߩᱜ⏕ߥ୯ߪ৻⥸ߦ᳞

߼ࠆߎߣ߇࿎㔍ߢ޽ࠅ㧘ታ㓙ߦߪࡌࠗ࠭᳿ቯೣߪએ ਅߩࠃ߁ߦ⟎߈឵߃ࠄࠇࠆ㧚

for all decide C_i if ( ; )g_i x ȁ !g_j( ; )x ȁ jzi ߎߎߢg_j( ; )xȁ ߪ್೎㑐ᢙߣ๭߫ࠇ㧘౉ജࡄ࠲࡯ࡦ

xߩࠢ࡜ࠬC_j߳ߩᏫዻᐲࠍ૗ࠄ߆ߩ⸘▚ߦࠃߞߡ

⴫ߔ㑐ᢙߢ޽ࠆ㧚ታ↪⊛ߥ႐㕙ߢߪ㧘ߎߩ್೎㑐ᢙ

(3)

߆ࠄ᭴ᚑߐࠇࠆ᳿ቯೣᑼࠍ↪޿ߡᑼߩ࡝ࠬࠢ

ߩᦨዊൻ߇⋡ᜰߐࠇࠆߎߣߦߥࠆ㧚

⺋ಽ㘃᷹ᐲ

ᑼߢ⴫ߐࠇࠆ᳿ቯೣߪ㧘ࠢ࡜ࠬߏߣߦਈ߃ࠄࠇ ࠆోߡߩ್೎㑐ᢙg_j( ; )x ȁ j 1,,Jߩ⸘▚ߣ㧘ߘ ࠇࠄࠍᲧセߔࠆᠲ૞߆ࠄᚑࠅ┙ߞߡ޿ࠆ㧚ಽ㘃ེߩ ࡄ࡜ࡔ࡯࠲ᦨㆡൻߦߪ㧘ᑼߩࠃ߁ߥᒻᑼߩᲧセᠲ

૞ߪߘߩ߹߹ߢߪㆡߐߥ޿㧚ᢙ୯⸘▚ߦࠃࠆᦨㆡൻ ᚻ㗅ߩਛߦߎߩ᳿ቯೣࠍ⚵ߺㄟ߻ߚ߼ߦߪ㧘Ყセᠲ

૞⥄૕߽ᢙ୯⸘▚ߦ⟎߈឵߃ࠆߎߣ߇ᔅⷐߣߥࠆ㧚

⺋ಽ㘃᷹ᐲߣ๭߫ࠇࠆએਅߩ㑐ᢙ߇ߘߩ⟎߈឵߃ࠍ ታ⃻ߔࠆߚ߼ߦ↪޿ࠄࠇࠆ㧚

( ; ) ( ; ) max ( ; )

j j i

i j

d g g

z

ȁ ȁ ȁ

x x x

ߎߎߢxߪࠢ࡜ࠬC_jߦዻߒߡ޿ࠆ౉ജࡄ࠲࡯ࡦߣ ߔࠆ㧚⺋ಽ㘃᷹ᐲߪ㧘ߘߩᱜ୯ߦࠃߞߡ౉ജࡄ࠲࡯

ࡦߩ⺋ಽ㘃ࠍ⴫ߒ㧘⽶୯ߦࠃߞߡᱜಽ㘃ࠍ⴫ߔ㧚߹

ߚ㧘ߘߩ⛘ኻ୯ߪኻᔕߔࠆಽ㘃᳿ቯߩ⏕߆ࠄߒߐࠍ

⴫ߒߡ޿ࠆ㧚හߜ㧘⛘ኻ୯߇ᄢ߈޿߶ߤ㧘ᱜಽ㘃޽

ࠆ޿ߪ⺋ಽ㘃ߩᐲว޿߇ᒝ޿ߎߣࠍᗧ๧ߔࠆߎߣߦ ߥࠆ㧚ߎߩ⏕߆ࠄߒߐߩ⒟ᐲߪ㧘ಽ㘃᳿ቯߩ㗎ஜᕈ robustnessߣኒធߦ㑐ଥߒߡ޿ࠆ㧚

ᑼߩ⺋ಽ㘃᷹ᐲߪ㧘ᑼߩ᳿ቯೣߦ߅ߌࠆᲧ セᠲ૞ࠍᢙ୯⸘▚ߦ⟎߈឵߃ߡߪ޿ࠆ߇㧘ᑼਛߦᦨ ᄢ୯ㆬᛯߩṶ▚ሶ߇↪޿ࠄࠇߡ߅ࠅ㧘ߎߩ߹߹ߢߪ ࡄ࡜ࡔ࡯࠲_ȁߦ㑐ߒߡᓸಽਇน⢻ߣߥࠅ㧘ᦨ߽ၮᧄ

⊛ߥᦨㆡൻᚻᴺߢ޽ࠆ൨㈩ᴺࠍㆡ↪ߔࠆߎߣ߇ߢ߈ ߥ޿㧚ߎߩᦨㆡត⚝ᚻ⛯߈ߦ߅ߌࠆਇචಽߐࠍ⸃᳿

ߔࠆߚ߼㧘MCE ߩቯᑼൻߪᑼࠍએਅߩࠃ߁ߥᓸ ಽน⢻ߥᒻᑼߦ⟎߈឵߃ߡ޿ࠆ㧚

1

( ; ) ( ; )

log exp(1 ( ; )) 1

j j

J

i i j

d g

J g

\ \ z

ª º

« »

¬

¦

¼

ȁ ȁ

ȁ

x x

x

ߎߎߢ\ ߪᱜߩቯᢙߢ޽ࠆ㧚\ ߩ୯߇චಽᄢ߈ߊߥ ࠆᤨ㧘ฝㄝߩ╙ 㗄ߪࡄ࠲࡯ࡦx߇ታ㓙ߦዻߒߡ޿

ࠆࠢ࡜ࠬࠍ㒰޿ߚࠢ࡜ࠬߩ್೎㑐ᢙߩਛߢ৻⇟ᄢ߈

ߥ୯ࠍߣࠆ್೎㑐ᢙߩㄭૃ୯ߣߥࠆ㧚ߔߥࠊߜ㧘ߘ ߩᤨᑼߪᑼߩఝࠇߚᓸಽน⢻ߥㄭૃᑼߣߥࠆ㧚

⺋ಽ㘃᷹ᐲߦࠃࠆಽ㘃࡝ࠬࠢ

ᦨዊൻߩኻ⽎ߢ޽ࠆᑼߩࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬ ߪ㧘᛽⽎⊛ߥ⴫⸥ߢ޽ࠆD_iߦࠃߞߡಽ㘃᳿ቯࠍ⴫ߔ ߎߣߦࠃߞߡቯ⟵ߐࠇߡ޿ࠆ㧚ᑼߩಽ㘃⺋ࠅᢙ࡝

ࠬࠢࠍᢙ୯⸘▚ಣℂߢᦨㆡൻ㧔ᦨዊൻ㧕ߔࠆߚ߼ߦ ߪ㧘D_iࠍౕ૕⊛ߥᢙᑼߢ⴫ߔᔅⷐ߇޽ࠆ㧚 ▵ߢ ቯ⟵ߒߚ⺋ಽ㘃᷹ᐲࠍ↪޿ߡ㧘ᑼࠍd_j( ; )x ȁ !0 ߩߣ߈ߦ ߦߥࠅ㧘d_j( ; )x ȁ 0ߩߣ߈ߦ ߦߥࠆᜰ

␜㑐ᢙ1(d_j( ; )x ȁ !0)ߢᦠ߈឵߃ࠆ㧚ߔࠆߣᑼߩ ಽ㘃࡝ࠬࠢߪ㧘એਅߩࠃ߁ߦᦠ߈឵߃ࠄࠇࠆ㧚

1

( ) 1( ( ; ) 0) ( , )

J

j j

j

R d p C d

F !

¦³

ȁ x ȁ x x

ߎߩᣂߒߊቯ⟵ߒߚ࡝ࠬࠢߪ㧘ಽ㘃ེߩࡄ࡜ࡔ࡯࠲

ȁߩ㑐ᢙߣߥߞߡ޿ࠆ㧚ߒ߆ߒ㧘ߎߎߢ↪޿ߚᜰ␜

㑐ᢙ1(d_j( ; )x ȁ !0)ߪ㧘 ߆ ߩ୯ߒ߆ขࠄߥ޿

ᑼߩࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬߣᧄ⾰⊛ߦหߓ߽ߩߢ޽ࠆ㧚 ߒ߆ߒߎߩ߹߹ߢߪࡄ࡜ࡔ࡯࠲ȁߦ㑐ߒߡᓸಽਇ น⢻ߥߩߢ㧘MCE ߢߪ૗ࠄ߆ߩᒻߢᐔṖൻߐࠇߚ ࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬࠍ↪޿ࠆ㧚ߘߩࠃ߁ߥᐔṖൻࡠ

ࠫࠬ࠹ࠖ࠶ࠢ៊ᄬߩౖဳ⊛ߥ଀ߣߒߡ㧘એਅߩࠃ߁ ߥࠪࠣࡕࠗ࠼ᒻᑼߩ㑐ᢙࠍ᜼ߍࠆߎߣ߇ߢ߈ࠆ㧚

( ; ) ( ( ; )) 1

1 exp( ( ; ) )

j

d

ad b

O O

ȁ ȁ

ȁ

x x

x

ߎߎߢaߪᱜߩቯᢙ㧘bߪᱜ߆⽶ߩቯᢙߢ޽ࠆ㧚ᐔ Ṗൻࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬߪ㧘ࡄ࡜ࡔ࡯࠲_ȁߦ㑐ߒߡ ᓸಽน⢻ߣߥߞߡ޿ࠆ㧚߹ߚ㧘ߎߩ㑐ᢙߦ߅ߌࠆᐔ Ṗᕈߪ㧘ቯᢙaߦࠃߞߡࠦࡦ࠻ࡠ࡯࡞ߐࠇࠆ㧚ߒߚ ߇ߞߡಽ㘃⺋ࠅᢙ࡝ࠬࠢߩᐔṖᕈߪ㧘ࠪࠣࡕࠗ࠼㑐 ᢙߦ߅ߌࠆᐔṖᕈ㧘ߘߒߡᑼߩ⺋ಽ㘃᷹ᐲߦ߅ߌ ࠆL_pࡁ࡞ࡓߩᜰᢙߩਔᣇߢ᳿߹ࠆ㧚

ࡄ࡞࠷ࠚࡦផቯߦࠃࠆቯᑼൻ

ߎߎ߹ߢ⼏⺰ߒߚࠃ߁ߦ㧘㑐ᢙߦᐔṖᕈࠍขࠅ౉

ࠇߚరޘߩ⋡⊛ߪ㧘ࡄ࡜ࡔ࡯࠲ߦ㑐ߔࠆᓸಽࠍน⢻

(4)

ߦߔࠆߎߣߦ޽ߞߚ㧚ߒ߆ߒ㑐ᢙߩᐔṖᕈߪ㧘ਈ߃ ࠄࠇߚ⸠✵↪ࡄ࠲࡯ࡦߩ๟ㄝ㗔ၞߦ઒ᗐ⊛ߦࡄ࠲࡯

ࡦࠍሽ࿷ߐߖࠆലᨐࠍᜬߜ㧘⚿ᨐ⊛ߦᧂ⍮ࡄ࠲࡯ࡦ ߦኻߔࠆ⸠✵⠴ᕈࠍะ਄ߐߖࠆߎߣࠍᦼᓙߐߖࠆ㧚

ߒ߆ߒ㧘ߎߩᐔṖᕈߣ⠴ᕈߩ㑐ଥߪ㧘੕޿ࠍ⚿߱

⋥ធ⊛ߥቯᑼൻ߇᣿ࠄ߆ߦߐࠇߡߎߥ߆ߞߚߎߣߥ ߤߦ⿠࿃ߒߡ㧘චಽߦ⸃᣿ߐࠇߡ޿ߥ޿㧚ᐔṖᕈࠍ ᄌᢙߣߒߡ⠴ᕈࠍ೙ᓮߔࠆߚ߼ߩᯏ᭴ߩ⼏⺰ߪ㧘ෳ

⠨ᢥ₂ ߦߺࠄࠇࠆࠃ߁ߦ㧘ẋㄭ෼᧤ℂ⺰ࠍᒁ↪ߒ ߚቯᕈ⊛ߥ᳓Ḱߢὑߐࠇߡ޿ࠆߩߺߢ޽ࠆ㧚ℂ⺰⊛

ߥᨒ⚵ߺ߽㧘ታ↪⊛ߥ೙ᓮᚻᴺ߽㧘ᧂߛචಽߥ߽ߩ ߪ㐿⊒ߐࠇߡ޿ߥ޿ߩ߇ታᖱߢ޽ࠆ㧚

ߎߩ໧㗴ࠍ✭๺ߔࠆ⠨߃ᣇߩ৻ߟߣߒߡ㧘ᓥ᧪߆ ࠄሽ࿷ߔࠆࡄ࡞࠷ࠚࡦផቯߩ᭎ᔨࠍ೑↪ߒߚ MCE ߩౣቯᑼൻ߇ⴕࠊࠇߡ޿ࠆ㧚એਅߢߪ㧘ߎߩቯᑼ ൻߩᚻ㗅ࠍ⺑᣿ߔࠆ㧚

߹ߕೋ߼ߦ㧘ᑼߢቯ⟵ߒߚಽ㘃⺋ࠅᢙ࡝ࠬࠢࠍ㧘 ࡄ࠲࡯ࡦⓨ㑆Fߩㇱಽ㗔ၞߦ㑐ߔࠆⓍಽࠍ↪޿ߚᒻ ߦᦠ߈឵߃ࠆ㧚

1

( )

( ) 1( ( ; ) 0) ( , )

( | )

j j

J

j j

j J

j j

P C

R d p C d

p C d

F

¦³

!

¦ ³

ȁ x ȁ x x

x x

ߎߎߢF_jߪF_j {xF|d_j( , )x ȁ !0}ߣߥࠆ㓸วߢ

޽ࠆ㧚xߘߒߡg₁( , ),x ȁ ,g_j( , ),x ȁ ,g_J( , )xȁ ߪ㧘 ㅪ⛯߆ߟ⏕₸⊛ߥᄌᢙߣ⷗ߥߔߎߣ߇ߢ߈ࠆߩߢ㧘

( ( , ))

j j

m {d x ȁ ߽߹ߚㅪ⛯ߥ⏕₸ᄌᢙߢ޽ࠆ㧚ߔࠆߣ㧘 ᑼߦ߅޿ߡࠢ࡜ࠬC_jߦ㑐ߒߡߩㇱಽ㓸วF_jࠍ

Ⓧಽ▸࿐ߣߒߚⓍಽߪ㧘ᣂߒߊቯ⟵ߒߚㅪ⛯⏕₸ᄌ ᢙm_j㧘ߔߥࠊߜ⺋ಽ㘃᷹ᐲߩ㧘ᱜ㗔ၞࠍⓍಽ▸࿐ߣ ߒߚⓍಽߦᦠ߈឵߃ࠆߎߣ߇ߢ߈ࠆ㧚

0

[ ( ; ) 0 | ] ( | ) ( | )

j j j

j P d C

p m C dm p C d

F

f

!

³

^x ^x ^x ^ȁ

ߎࠇࠍ↪޿ߡ㧘ಽ㘃⺋ࠅᢙ࡝߽ࠬࠢએਅߩࠃ߁ߦቯ

⟵ߒߥ߅ߔߎߣ߇ߢ߈ࠆ㧚

( ) 0

( ) _j ( | )

J

j j j

R ^ȁ

¦

P C

³

^fp m C dm

ታ㓙ߩࡄ࠲࡯ࡦ⹺⼂⺖㗴ߩ߶ߣࠎߤߦ߅޿ߡ㧘ᮡ

ᧄࡄ࠲࡯ࡦⓨ㑆ߪ㜞ᰴరߢ޽ࠅ㧘ᑼ߹ߢߩಽ㘃࡝

ࠬࠢߦ߅ߌࠆⓍಽ߽ߘࠇߦᔕߓߡ㜞ᰴరߩⓍಽߦߥ ࠆ㧚ૐᰴరᮡᧄⓨ㑆ߦ߅޿ߡߪᮡᧄߩჇടߦ઻߁ఝ ࠇߚ෼᧤ᕈ⢻ࠍ␜ߔࡄ࡞࠷ࠚࡦផቯ߽㧘㜞ᰴరᮡᧄ

ⓨ㑆ߦ߅޿ߡߪߘߩ㒢ࠅߢߪߥ޿㧚৻ᣇ㧘ᑼߪ㧘 ᰴరᄌᢙߢ޽ࠆm_jߦ㑐ߔࠆⓍಽ⸘▚߆ࠄ᭴ᚑߐࠇ㧘

᧦ઙઃ߈⏕₸ኒᐲ㑐ᢙ߽න⚐ߥࠬࠞ࡜࡯ᄌᢙߩ㑐ᢙ ( _j| _j)

p m C ߣߥࠆ㧚ߎ߁ߒߚᏅ⇣ߦၮߠ߈㧘ಽ㘃ེ

ࡄ࡜ࡔ࡯࠲ߩ⸠✵ᴺߦᣂߚߥⷞὐࠍዉ౉ߢ߈ࠆࠃ߁ ߦᕁࠊࠇࠆ㧚

ࠢ࡜ࠬC_jߦ㑐ߔࠆ⏕₸ᄌᢙm_jߩ᧦ઙઃ߈⏕₸ኒ ᐲ㑐ᢙߪ㧘એਅߩࠃ߁ߥࡄ࡞࠷ࠚࡦផቯߢផቯߢ߈ ࠆ㧚

, 1

( ; )

1 1

( | )

j j

N

j j j k

N j j

j k

m d p m C

N hI§ h ·

¨ ¸

© ¹

¦

^x ^ȁ

ߎߎߢx_{j k}_, ߪࠢ࡜ࠬC_jߩk⇟⋡ߩ⸠✵ࠨࡦࡊ࡞㧘 ( , ; )

j j j k

m d

I§ h ·

¨ ¸

© ¹

ȁ

x ߪਈ߃ࠄࠇߚࠨࡦࡊ࡞ࡌࠢ࠻࡞

ࠍ1ᰴరⓨ㑆ߦ౮௝ߒߚ㧘d_j(x_{j k}_, ; )ȁ ࠍਛᔃߣߔࠆ

᏷߇hߩࠞ࡯ࡀ࡞㑐ᢙ㧘ߘߒߡN_jߪࠢ࡜ࠬC_jߩ⸠

✵ࠨࡦࡊ࡞ᢙߢ޽ࠆ㧚ߎߩ⏕₸ኒᐲ㑐ᢙࠍ૶߁ߣ㧘 ( )

R ȁ ߩផቯ୯ߪએਅߩࠃ߁ߦ⴫ߐࠇࠆ㧚

1 0

( )

( ) _j( | )

N j N

J

j j j

j

R ^ȁ

¦

P C

³

^fp m C dm

߹ߚ㧘⸠✵ࠨࡦࡊ࡞ߩ✚ᢙࠍ

1 J j j

N

¦

N ^ߣߒߡ

( _j)

P C ࠍN_j Nߢㄭૃߒ㧘ᑼߦᑼࠍઍ౉ߔ ࠆߣ㧘એਅߩࠃ߁ߦᦠ߈឵߃ࠆߎߣ߇ߢ߈ࠆ㧚

, 1 0

1

( ; )

1 1

( )

Nj

j j j k

N j

k J j

m d

N h h dm

R ^fI^§_¨ ^·_¸

© ¹

¦ ¦ ³

^ȁ

ȁ ^x

N( )

R ȁ ߪࡄ࠲࡯ࡦⓨ㑆ో૕ߦ㑐ߔࠆಽ㘃⺋ࠅᢙ࡝

ࠬࠢࠍ㧘ਈ߃ࠄࠇߚ⸠✵ࠨࡦࡊ࡞㓸วࠍ↪޿ߡផቯ ߒߚ߽ߩߢ޽ࠆ㧚ࠃߞߡᑼߩ⴫⃻ࠃࠅ㧘ฦ⸠✵

ࠨࡦࡊ࡞ߦኻߒߡ⺖ߐࠇࠆ៊ᄬࠍ㧘એਅߩࠃ߁ߦᣂ ߚߦቯ⟵ߒߥ߅ߔߎߣ߇ߢ߈ࠆ㧚

(5)

,

, 0

( ; )

( ( _{j k}; )) 1 m^j d^j ^{j k} _j

d dm

h h

O ^fI^§_¨ ^·_¸

© ¹

³

^ȁ

ȁ x

x

⥝๧ᷓ޿ߎߣߦ㧘න⚐ߥࠞ࡯ࡀ࡞㑐ᢙࠍㆡ↪ߔࠆߎ ߣߦࠃߞߡ㧘੹߹ߢߩ⼏⺰ߢ಴ߡ߈ߚ㊀ⷐߥ៊ᄬ㑐 ᢙࠍዉ಴ߢ߈ࠆ㧚଀߃߫ᣇᒻߩࠞ࡯ࡀ࡞㑐ᢙࠍ૶߁ ߣ㧘ᑼߩࠃ߁ߥ 㧙 ߩ୯ࠍขࠆ✢ᒻࡠࠫࠬ࠹ࠖ

࠶ࠢ៊ᄬ߇ᓧࠄࠇ㧘ࠟ࠙ࠪࠕࡦ㑐ᢙߦࠃߊૃߚᒻߩ

㊒㏹ဳߩࠞ࡯ࡀ࡞㑐ᢙࠍ૶߁ߣ㧘ᑼߩࠃ߁ߥᐔṖ ൻࠪࠣࡕࠗ࠼៊ᄬ߇ᓧࠄࠇࠆ㧚

෼᧤ᕈߣࡑ࡯ࠫࡦ

ࡄ࡞࠷ࠚࡦផቯߩᨒ⚵ߺߩዉ౉ߦࠃߞߡ㧘MCEߦ ߅ߌࠆ៊ᄬߩᐔṖᕈߩ೙ᓮߦᣂߚߥⷞὐࠍਈ߃ࠆߎ ߣ߇น⢻ߣߥࠆ㧚ࠞ࡯ࡀ࡞ߩ᏷hߩ୯ߩ೙ᓮߪ㧘៊

ᄬߩᐔṖᕈߩ೙ᓮߦߘߩ߹߹ኻᔕߒߡ޿ࠆ㧚৻ᣇߢ㧘

⸠✵ࠨࡦࡊ࡞㓸วߩⷐ⚛ᢙN߇Ⴧ߃ࠆߦᓥߞߡࠞ

࡯ࡀ࡞ߩ᏷hࠍ⁜߼ࠇ߫㧘ᑼߩಽ㘃࡝ࠬࠢߩផ ቯ୯ߪ⌀ߩಽ㘃࡝ࠬࠢR( )ȁ ߦ෼᧤ߔࠆߎߣ߇଻㓚 ߐࠇߡ޿ࠆ㧚ߟ߹ࠅ㧘៊ᄬߩᐔṖᕈߪ㧘⸠✵ࠨࡦࡊ

࡞㓸วߩⷐ⚛ᢙߦᔕߓߡ೙ᓮߔࠇ߫ࠃ޿ߣ޿߁ߎߣ ߦߥࠆ㧚ߎߩ෼᧤ᕈߩ⹦⚦ߥಽᨆߪෳ⠨ᢥ₂ ߢⴕ ࠊࠇߡ޿ࠆ㧚

਄ㅀߩࠃ߁ߦ㧘ࡄ࡞࠷ࠚࡦផቯߦࠃࠆࠕࡊࡠ࡯࠴

ᣇᴺߦࠃߞߡ㧘ẋㄭ෼᧤ߣ޿߁ᕈ⾰ߦⵣઃߌࠄࠇߚ ᐔṖᕈߩ೙ᓮᣇᴺ߇ᓧࠄࠇࠆ㧚ߒ߆ߒ⚿ዪታ↪⊛ߥ ႐㕙ߢߪ㧘᦭㒢ߩࡄ࠲࡯ࡦ㓸วߒ߆૶߁ߎߣߪߢ߈ ߕ㧘ẋㄭ෼᧤ᕈߩലᨐߪᦼᓙߢ߈ߥ޿㧚ߘࠇߢߪታ 㓙ߦߪ㧘ߤߩࠃ߁ߦߒߡᐔṖᕈࠍ೙ᓮߔࠇ߫ࠃ޿ߩ ߛࠈ߁߆㧫න⚐ߦ⠨߃ࠇ߫㧘ዋߥ޿⸠✵ࠨࡦࡊ࡞ࠍ

⵬߃ࠆࠃ߁ߦࠞ࡯ࡀ࡞ߩ᏷ࠍ⺞▵ߒ㧘ߘߩ୯ࠍ଻ߞ ߚ߹߹ߢቇ⠌ߔࠆߣ޿߁ᣇᴺ߇⠨߃ࠄࠇࠆ㧚ᱜߒ޿

ಽ㘃Ⴚ⇇ࠍ⸳⸘ߔࠆߩߦ㊀ⷐߥࠨࡦࡊ࡞ߪಽ㘃Ⴚ⇇

ㄭㄝߩࠨࡦࡊ࡞ߥߩߢ㧘ߘߩઃㄭߩࠨࡦࡊ࡞ࠍ⵬߁ ࠃ߁ߦࠞ࡯ࡀ࡞᏷ࠍ⸳ቯߔࠆߎߣࠍ⠨߃ࠆ㧚଀߃߫

ᣇᒻࠞ࡯ࡀ࡞㑐ᢙߥࠄ߫㧘߹ߕᱜߒߊ⹺⼂ߐࠇࠆ⸠

✵ࠨࡦࡊ࡞ߣ⺋ߞߡಽ㘃ߐࠇࠆ⸠✵ࠨࡦࡊ࡞ߩਛߢ㧘

ࠢ࡜ࠬႺ⇇ߦᦨ߽ㄭ޿ ߟߩ⸠✵ࠨࡦࡊ࡞ࠍㆬ߮ߛ ߔ㧚ߘࠇࠄߦኻᔕߔࠆࠞ࡯ࡀ࡞߇㊀ߥࠄߥ޿▸࿐ߢ

᏷ࠍߢ߈ࠆߛߌᄢ߈ߊߔࠇ߫㧘ਇ⿷ߒߡ޿ࠆࠨࡦࡊ

࡞ࠍ⵬߁ߎߣ߇ߢ߈ࠆߢ޽ࠈ߁Fig. 1㧚

Fig. 1. Schematic explanation of kernels that do not cross over to each other.

޽ࠆ⒟ᐲߩ᏷ࠍᜬߚߖߚࡄ࡞࠷ࠚࡦ࡮ࠞ࡯ࡀ࡞ࠍ

↪޿ࠆߎߣߦࠃࠅ㧘 ߩ୯ࠍขࠆࡠࠫࠬ࠹ࠖ࠶ࠢ

៊ᄬࠍᐔṖൻߒ㧘ಽ㘃Ⴚ⇇๟ㄝ㧘ߔߥࠊߜ⺋ಽ㘃᷹

ᐲߩ୯߇ ߣߥࠆ࿾ὐ๟ㄝߢ៊ᄬ㑐ᢙߦ൨㈩ࠍᜬߚ ߖࠆߎߣ߇ߢ߈ࠆ㧚ߎߎߦᦨㆡൻᚻᴺߩਛߢ߽ᮡḰ

⊛ߥ߽ߩߢ޽ࠆ൨㈩ᴺࠍ↪޿ࠇ߫㧘ಽ㘃⺋ࠅᢙ߇ᷫ

ࠆᣇะ㧘ߔߥࠊߜ៊ᄬߩ୯߇ࠃࠅዊߐߊߥࠆᣇะߦ ಽ㘃ེߩࡄ࡜ࡔ࡯࠲߇ᦝᣂߐࠇࠆ㧚ߎࠇߪ㧘⺋ಽ㘃

᷹ᐲߩ୯߽ࠃࠅዊߐߊߥࠆᣇะߦᦝᣂߐࠇࠆߣ޿߁ ߎߣ߽ᗧ๧ߔࠆ㧚ߎߎߢ㧘⺋ಽ㘃᷹ᐲߩ⛘ኻ୯ߪߘ ߩಽ㘃᳿ቯߩ⏕߆ࠄߒߐࠍ⴫ߔߣ޿߁੐ታߦ⌕⋡ߔ ࠆ㧚ߔࠆߣ⺋ಽ㘃᷹ᐲߩ୯߇⽶ߩߣ߈ߦߪ㧘ߘߩ⛘

ኻ୯߇ᄢ߈޿߶ߤᱜಽ㘃ߩᐲว޿߇ᒝ޿ߣ޿߁ߎߣ ߦߥࠆ㧚ߟ߹ࠅ⽶ߩ⺋ಽ㘃᷹ᐲߩ⛘ኻ୯ߪ㧘ਈ߃ࠄ ࠇߚࠨࡦࡊ࡞ߩಽ㘃⚿ᨐߣಽ㘃Ⴚ⇇߇ߤࠇ߶ߤ㔌ࠇ ߡ޿ࠆ߆㧘ߔߥࠊߜࡑ࡯ࠫࡦߩ৻⒳ߢ޽ࠆߣ⠨߃ࠆ ߎߣ߇ߢ߈ࠆߩߢ޽ࠆ㧚⺋ಽ㘃᷹ᐲߩ୯߇⸠✵ߦࠃ ߞߡߤߎ߹ߢዊߐߊᦝᣂߐࠇࠆ߆ߦߪ៊ᄬߩ൨㈩߇ 㑐ଥߒ㧘ߘߩ൨㈩ߦߪࡄ࡞࠷ࠚࡦ࡮ࠞ࡯ࡀ࡞ߩ᏷߇ 㑐ଥߔࠆߚ߼㧘਄⸥ߩ⼏⺰ߣ⠨߃วࠊߖࠆߣ㧘⚿ዪ ࡄ࡞࠷ࠚࡦ࡮ࠞ࡯ࡀ࡞ߩ᏷ߪ㧘ಽ㘃Ⴚ⇇߆ࠄߩࡑ࡯

ࠫࡦߣᷓߊ㑐ࠊߞߡ޿ࠆߣ⠨߃ࠄࠇࠆ㧚

⺋ಽ㘃᷹ᐲߩਇචಽᕈ

MCEቇ⠌ߪ㧘ಽ㘃⺋ࠅᢙߩᦨዊൻߣಽ㘃᳿ቯߩᱜ

⏕ߐࠍ⴫ߔࡑ࡯ࠫࡦߩჇടࠍ㧘หᤨߦ߁߹ߊታ⃻ߒ ߡ޿ࠆࠃ߁ߦ⷗ฃߌࠄࠇࠆ㧚ߎࠇࠄߪ⺋ಽ㘃᷹ᐲߩ ୯ࠍ㧘⽶ߩᣇะߦേ߆ߔࠃ߁ߦࡄ࡜ࡔ࡯࠲ࠍᦝᣂߔ

(6)

ࠆߎߣߢታ⃻ߐࠇߡ޿ࠆ㧚ߒ߆ߒߎߎߢቯ⟵ߐࠇߡ

޿ࠆࡑ࡯ࠫࡦ㧘ߔߥࠊߜ⽶ߩ⺋ಽ㘃᷹ᐲߩ⛘ኻ୯ࠍ ᄢ߈ߊߒߡ߽㧘ᔅߕߒ߽ߘࠇ߇ಽ㘃ེߩ⠴ᕈߩჇട ߦ❬߇ࠆߣߪ㒢ࠄߥ޿㧚଀߃߫㧘න⚐ߦోߡߩ್೎

㑐ᢙߦቯᢙࠍ߆ߌࠆߣߘࠇߦᔕߓߡ⺋ಽ㘃᷹ᐲߩ⛘

ኻ୯߽ᄢ߈ߊߥࠆ߇㧘ߘߩ႐วಽ㘃Ⴚ⇇ߦߪᄌൻ߇ ߥ޿㧚ࠃߞߡ⺋ಽ㘃᷹ᐲߪ㧘ߘࠇߛߌߢߪಽ㘃ེߩ

⠴ᕈࠍ⴫⃻ߔࠆߩߦਇචಽߢ޽ࠆߣ޿߁ߎߣߦߥࠆ㧚 ࠃࠅ⋥ធ⊛ߦ⠴ᕈࠍ⴫⃻ߢ߈ࠆࠃ߁ߥ⺋ಽ㘃᷹ᐲߩ ᡷ⦟߿㧘ᣂߒ޿⸠✵ᚻᴺࠍ MCE ቇ⠌ߦ⚵ߺㄟ߻ߎ ߣ߇᳞߼ࠄࠇࠆ㧚

ࠨࡐ࡯࠻࡮ࡌࠢ࠲࡯࡮ࡑࠪࡦ ⒳㘃ߩࡑ࡯ࠫࡦ

ࡑ࡯ࠫࡦߣ޿߁↪⺆ߪ㧘ฎߊ߆ࠄ✢ᒻ್೎㑐ᢙߩ

⎇ⓥߦ߅޿ߡ૶ࠊࠇߡ߈ߚ㧚ߒ߆ߒߎߩࡑ࡯ࠫ

ࡦߦߪ ⒳㘃ߩቯ⟵߇޽ࠆߎߣߦᵈᗧߒߥߌࠇ߫ߥ ࠄߥ޿㧚એਅߩ⼏⺰ߩߚ߼㧘 ⒳㘃ߩࡑ࡯ࠫࡦߩቯ

⟵ࠍਈ߃ࠆ㧚

න⚐ൻߩߚ߼㧘✢ᒻ್೎㑐ᢙߦࠃࠆಽ㘃ེߢ ࠢ

࡜ࠬߩࠨࡦࡊ࡞ࠍಽ㘃ߔࠆ໧㗴ࠍ⠨߃ࠆ㧚ਈ߃ࠄࠇ ߚࠨࡦࡊ࡞xࠍ㧘C߆Cߩߤߜࠄ߆ߩࠢ࡜ࠬߦಽ 㘃ߔࠆ႐วߩ್೎㑐ᢙࠍ㧘એਅߩࠃ߁ߦቯ⟵ߔࠆ㧚

g( )x ! w x b ߎߎߢwߪ㊀ߺࡄ࡜ࡔ࡯࠲ߦࠃࠆࡌࠢ࠻࡞ߢ㧘bߪ

ࠬࠞ࡜࡯ቯᢙߣߔࠆ㧚ᑼߢ⴫ߐࠇࠆᄙࠢ࡜ࠬߩ႐ วߩ᳿ቯೣߣߪ㆑޿㧘 ࠢ࡜ࠬߩ႐วߦߪߎߩ್೎

㑐ᢙ৻ߟߛߌߢ᳿ቯೣࠍ⴫⃻ߢ߈ࠆ㧚ߔߥࠊߜ㧘 ( ) 0

g x ! ߥࠄࠢ࡜ࠬCߦಽ㘃ߒ㧘g( )x 0ߥࠄࠢ࡜

ࠬCߦಽ㘃ߔࠆ㧘ߣ޿߁᳿ቯೣߢ޽ࠆ㧚

ߎߎߢᱜ⸃ࠢ࡜ࠬ߇Cߩߣ߈ߦߪ ߣ޿߁୯ࠍ ขࠅ㧘Cߩߣ߈ߦߪ ߣ޿߁୯ࠍขࠆࠢ࡜ࠬ࡮ࠗ

ࡦ࠺࠶ࠢࠬyࠍዉ౉ߔࠆ㧚ߔࠆߣyg( )x !0ߪಽ㘃⚿

ᨐ߇ᱜ⸃ߢ޽ࠆߣ޿߁ߎߣࠍ⴫ߒ㧘yg( )x 0ߪಽ㘃

⚿ᨐ߇ਇᱜ⸃ߢ޽ࠆߣ޿߁ߎߣࠍ⴫ߔ㧚

߹ߚ㧘yg x( ) ߪ㧘ಽ㘃᳿ቯߩ⏕߆ࠄߒߐࠍ␜ߒߡ

޿ࠆ߽ߩߣ⠨߃ࠆߎߣ߇ߢ߈ࠆ㧚හߜ㧘yg x( ) ߩ୯ ߇ዊߐߌࠇ߫ಽ㘃್ᢿߩ⏕߆ࠄߒߐ߇ᒙߊ㧘ߘߩ୯ ߇ᄢ߈ߌࠇ߫ಽ㘃್ᢿߩ⏕߆ࠄߒߐ߇ᒝ޿ߎߣࠍᗧ

๧ߔࠆߩߢ޽ࠆ㧚ࠃߞߡyg x( )ߪ㧘ಽ㘃Ⴚ⇇߆ࠄߩ

ࡑ࡯ࠫࡦߩ৻⒳ߢ޽ࠆߣ⠨߃ࠆߎߣ߇ߢ߈ࠆ㧚ߎߎ ߢ㧘yg x( )ߪ್೎㑐ᢙߩⓨ㑆਄ߢቯ⟵ߐࠇߡ޿ࠆࡑ

࡯ࠫࡦߢ޽ࠆ㧚ߎߩὐߦၮߠ߈㧘yg x( )ߪ㑐ᢙࡑ࡯

ࠫࡦߣ๭߫ࠇࠆ㧚᣿ࠄ߆ߦ㧘ߎߩ㑐ᢙࡑ࡯ࠫࡦߪ

ࠢ࡜ࠬߩ႐วߩ⺋ಽ㘃᷹ᐲߩ୯ߩ㧘ᱜ⽶ࠍ෻ォߐߖ ߚ߽ߩߦ╬ߒ޿㧚

⺋ಽ㘃᷹ᐲߩ႐วߣห᭽㧘್೎㑐ᢙߦ૗ࠄ߆ߩቯ ᢙࠍ߆ߌࠇ߫㑐ᢙࡑ࡯ࠫࡦߩ୯ࠍᄌ߃ࠆߎߣ߇ߢ߈ ࠆ㧚ߒ߆ߒ㧘ߎ߁ߒߡ㑐ᢙࡑ࡯ࠫࡦ߇ᄢ߈ߊߥߞߡ

߽㧘ኻᔕߔࠆಽ㘃Ⴚ⇇ߪᄌࠊࠄߥ޿㧚න⚐ߦ㑐ᢙࡑ

࡯ࠫࡦߩ୯ࠍᄢ߈ߊߔࠆߛߌߢߪ㧘ಽ㘃ེߩ⠴ᕈะ

਄ߦᓇ㗀ߒߥ޿ߎߣߦߥࠆ㧚

৻ᣇ㧘ಽ㘃ེߩ⠴ᕈࠍ⼏⺰ߔࠆߚ߼ߦ㧘⸠✵ࠨࡦ ࡊ࡞ߩਛߢಽ㘃Ⴚ⇇ߦ৻⇟ㄭ޿ࠨࡦࡊ࡞ߣಽ㘃Ⴚ⇇

ߣߩ㧔࡙࡯ࠢ࡝࠶࠼㧕〒㔌ߦࠃߞߡቯ⟵ߐࠇࠆᐞ૗

ࡑ࡯ࠫࡦ߇↪޿ࠄࠇߡ޿ࠆ㧚✢ᒻ್೎㑐ᢙߢ ࠢ࡜

ࠬߩࠨࡦࡊ࡞xࠍಽ㘃ߔࠆ႐วߩᐞ૗ࡑ࡯ࠫࡦrߪ㧘 xࠍಽ㘃Ⴚ⇇ߦ৻⇟ㄭ޿⸠✵ࠨࡦࡊ࡞ߣߒߡએਅߩ ᑼߢ⴫ߐࠇࠆ㧚

2

| |

|| ||

r ! w x b w

⸳⸘ߒߚಽ㘃ེߦࠃߞߡ㧘ోߡߩ⸠✵ࠨࡦࡊ࡞߇ᱜ ߒߊಽ㘃ߐࠇࠆߣ઒ቯߔࠆ㧚዁᧪౉ജߐࠇࠆᧂ⍮ߩ ࠨࡦࡊ࡞ߪ㧘⸠✵ࠨࡦࡊ࡞ߩ๟ㄝߦಽᏓߒߡ޿ࠆน

⢻ᕈ߇㜞޿ߩߢ㧘ᱜߒߊಽ㘃ߐࠇߡ޿ࠆࠨࡦࡊ࡞߽

ߢ߈ࠆߛߌಽ㘃Ⴚ⇇߆ࠄ㔌ߒߡ߅ߌ߫ಽ㘃⠴ᕈ߽㜞 ߊߥࠆߣ⠨߃ࠄࠇࠆ㧚ᐞ૗ࡑ࡯ࠫࡦߪಽ㘃Ⴚ⇇ߣߘ ࠇߦ৻⇟ㄭ޿ࠨࡦࡊ࡞ߣߩ〒㔌ࠍ⴫ߒߡ޿ࠆߩߢ㧘 ߘߩ୯ࠍᄢ߈ߊߔࠆߎߣߦࠃߞߡಽ㘃ེߩ⠴ᕈߦ⋥

ធ⽸₂ߔࠆߎߣ߇ߢ߈ࠆ㧚

ᐞ૗ࡑ࡯ࠫࡦᦨᄢൻߩߚ߼ߩ೙⚂

ᑼࠃࠅ㧘✢ᒻ್೎㑐ᢙߢ ࠢ࡜ࠬࠍಽ㘃ߔࠆ

໧㗴ߩ႐ว㧘ᐞ૗ࡑ࡯ࠫࡦߪ್೎㑐ᢙߩ⛘ኻ୯ߦᲧ

଀ߒ㧘㊀ߺࡌࠢ࠻࡞wߩL₂ࡁ࡞ࡓߦ෻Ყ଀ߔࠆߎߣ ߇ࠊ߆ࠆ㧚್೎㑐ᢙߩ⛘ኻ୯ߪ㑐ᢙࡑ࡯ࠫࡦߢ⟎߈

឵߃ࠆߎߣ߇น⢻ߥߩߢ㧘ᐞ૗ࡑ࡯ࠫࡦߪ㊀ߺࡌࠢ

࠻࡞wߩL₂ࡁ࡞ࡓߦࠃߞߡᱜⷙൻߐࠇߚ㑐ᢙࡑ࡯

ࠫࡦߢ޽ࠆߣ⠨߃ࠆߎߣ߇ߢ߈ࠆ㧚ࠃߞߡᐞ૗ࡑ࡯

ࠫࡦߩ୯ࠍჇ߿ߔ㧘ߔߥࠊߜಽ㘃ེߩ⠴ᕈࠍะ਄ߐ

(7)

ߖࠆߦߪ㧘ࡄ࡜ࡔ࡯࠲㧔㊀ߺࡌࠢ࠻࡞㧕ࠍ㑐ᢙࡑ࡯

ࠫࡦߩ⛘ኻ୯ࠍჇ߿ߔᣇะߦᦝᣂߔࠆࠕࡊࡠ࡯࠴ߣ㧘

㊀ߺࡌࠢ࠻࡞ߩL₂ࡁ࡞ࡓࠍᷫዋߐߖࠆᣇะߦᦝᣂ ߔࠆࠕࡊࡠ࡯࠴ߩ ⒳㘃߇⠨߃ࠄࠇࠆ㧚ߒ߆ߒߎࠇ ࠄ ߟߩ୯ߪ߅੕޿߇ଐሽ㑐ଥߦ޽ࠆߩߢ㧘ߤߜࠄ ߆ ᣇߩ୯߇ᄌൻߔࠆࠃ߁ߦᦝᣂࠍߔࠇ߫㧘߽߁ ᣇߩ୯߽ᄌࠊߞߡߒ߹߁㧚ߘߩᤨᐞ૗ࡑ࡯ࠫࡦߩ୯ ߇ᧄᒰߦჇ߃ࠆߩ߆ߪ଻㓚ߐࠇߥ޿㧚ߘߎߢ⏕ታߦ ᐞ૗ࡑ࡯ࠫࡦߩ୯ࠍჇ߿ߔߚ߼ߩࠕࡊࡠ࡯࠴ߣߒߡ㧘㑐ᢙࡑ࡯ࠫࡦߩ୯ࠍ৻ቯߩ୯ߦ࿕ቯߒߚ਄ߢ㧘㊀ߺ ࡌࠢ࠻࡞ߩL₂ࡁ࡞ࡓࠍᷫዋߐߖࠆࠃ߁ߦᦝᣂߐߖ ࠆ ᣇ ᴺ ߇ ⠨ ߃ ࠄ ࠇ ࠆ 㧚㑐 ᢙ ࡑ ࡯ ࠫ ࡦ ߩ ୯ ࠍ

| ! w x b| 1ߦ࿕ቯߒߚ႐วߪ㧘ᐞ૗ࡑ࡯ࠫࡦߪ એਅߩࠃ߁ߦߥࠆ㧚

2

1

|| ||

r

w SVMߢߪ㧘ߎߩࠃ߁ߥ೙⚂ࠍ⺖ߒߚᐞ૗ࡑ࡯ࠫࡦࠍ ᦨᄢൻߔࠆࠃ߁ߦಽ㘃ེࡄ࡜ࡔ࡯࠲ࠍ⸠✵ߔࠆ㧚

ᱜⷙൻ㗄ߣ៊ᄬߩᦨዊൻࠍ⋡ᜰߔ 58/

SVM ߪ㧘ၮᧄ⊛ߦ✢ᒻ್೎㑐ᢙߢోߡߩࠨࡦࡊ࡞

ࠍᱜߒߊಽ㘃ߔࠆߎߣ߇น⢻ߥ⁁ᴫ㧘ߔߥࠊߜ✢ᒻ ಽ㔌น⢻ߥ⁁ᴫࠍᗐቯߒߡ⼏⺰ߐࠇࠆ㧚ߒ߆ߒ㧘⸠

✵ࠨࡦࡊ࡞ߛߌߦ㒢ቯߒߚߣߒߡ߽㧘ోߡᱜߒߊಽ 㘃ߢ߈ࠆࠃ߁ߥ⁁ᴫߪ⃻ታߦߪ⒘ߢ޽ࠆߩߢ㧘ߎߎ ߢߪ✢ᒻಽ㔌ਇน⢻ߥ⁁ᴫߢ߽ㆡ↪ߢ߈ࠆࠃ߁ߦ᜛

ᒛߐࠇߚSVMߦߟ޿ߡ⺑᣿ߔࠆ㧚

SVM ߢߪ✢ᒻಽ㔌ਇน⢻ߥ⁁ᴫࠍ߁߹ߊᛒ߁ߚ

߼ߦ㧘㕖✢ᒻߥࠞ࡯ࡀ࡞㑐ᢙࠍ↪޿ߡฦࠨࡦࡊ࡞ࠍ

೎ߩⓨ㑆ߦ౮௝ߔࠆߣ޿߁ᣇᴺ߇↪޿ࠄࠇࠆ㧚ߒ߆ ߒ㧘ࠞ࡯ࡀ࡞㑐ᢙߣߒߡߤߩࠃ߁ߥ߽ߩ߇ᅷᒰߢ޽

ࠆߩ߆ࠍ․ቯߔࠆᣇᴺߪᧂߛ㐿⊒ߐࠇߡ޿ߥ޿㧚ਇ ㆡಾߥࠞ࡯ࡀ࡞ࠍㆬࠎߢߒ߹߃߫㧘౮௝ᓟߩⓨ㑆ߢ ࡌࠗ࠭࡝ࠬࠢߩ୯߇రޘߩࠨࡦࡊ࡞ⓨ㑆ߢߩ୯ߣߪ ᄌࠊߞߡߒ߹߁น⢻ᕈ߇޽ࠆ㧚߹ߚ㧘ᐞ૗ࡑ࡯ࠫࡦ ߩቯᑼൻ߽㧘౮௝ᓟߩⓨ㑆ߢߘߩ߹߹ㆡ↪ߢ߈ࠆߣ ߪ㒢ࠄߥ޿㧚

ߎߩࠃ߁ߦ㧘㕖✢ᒻߥ౮௝ߪ⼏⺰ࠍਇㅘ᣿ߦߒߡ ߒ߹߁㧚ࠃߞߡන⚐ൻߩߚ߼㧘ߎߎߢߪరߩⓨ㑆ߢ

✢ᒻ್೎㑐ᢙࠍ↪޿ߚ ࠢ࡜ࠬಽ㘃໧㗴ߢ㧘✢ᒻಽ

㔌ਇน⢻ߥ႐ว߽⠨ᘦߒߚ㧘SVMߩၮᧄ⊛ߥቯᑼൻ ࠍ ⠨ ߃ ࠆ 㧚 ⸠ ✵ ↪ ߩ ࠨ ࡦ ࡊ ࡞ 㓸 ว ߣ ߒ ߡ 㧘

1, 1 , , _n, _n , , _N, _N

S x y x y x y ߇ਈ߃ࠄࠇࠆ ߣ઒ቯߔࠆ㧚ߎߎߢx_nߪn⇟⋡ߩ⸠✵ࠨࡦࡊ࡞ߢ޽

ࠅ㧘y_nߪߘߩࠨࡦࡊ࡞ߦኻᔕߔࠆ ߆㧙 ߩ୯ࠍข ࠆࠢ࡜ࠬ࡮ࠗࡦ࠺࠶ࠢࠬߢ޽ࠆ㧚ߎߩߣ߈㧘એਅߩ ᦨㆡൻ໧㗴ࠍ⸃ߊߎߣߦࠃߞߡ ࠢ࡜ࠬߩࠨࡦࡊ࡞

ࠍಽ㘃ߔࠆ್೎㑐ᢙࠍ᭴ᚑߔࠆ⿥ᐔ㕙( , )w b ߇ᓧࠄ ࠇࠆ㧚

, , , , 1

1

minimize

subject to ( ) 0 ( 1, , )

N

b n n

N

n n n

n

C

y b

n N

P !

! t t

" "

¦

"

w

w w

w x

ߎߎߢ_"_nߪࠬ࡜࠶ࠢᄌᢙߣ๭߫ࠇࠆᄌᢙߢ޽ࠅ㧘

max(0, ( ))

n Pyn n! b

" w x ߣ޿߁᧦ઙࠍḩߚ

ߔ㧚߹ߚ㧘P( 0)! ߪ㑐ᢙࡑ࡯ࠫࡦߩ⋡ᮡ୯ߢ޽ࠅ㧘C ߪቯᢙߢ޽ࠆ㧚㑐ᢙࡑ࡯ࠫࡦߩ⋡ᮡ୯ߣታ㓙ߩ㑐ᢙ ࡑ࡯ࠫࡦߩ୯߇৻⥌ߔࠆࠨࡦࡊ࡞ࠍ㧘ࠨࡐ࡯࠻࡮ࡌ

ࠢ࠲࡯ߣ๭߱㧚ࠨࡦࡊ࡞x_nߦኻᔕߔࠆታ㓙ߩ㑐ᢙࡑ

࡯ࠫࡦߩ୯y_n( w x_n! b)߇࠲࡯ࠥ࠶࠻ߢ޽ࠆP ࠃࠅዊߐ޿႐วߪ㧘ࠬ࡜࠶ࠢᄌᢙ_"_n߇ߘߩᏅࠍ⴫ߔ㧚

৻ᣇߢ㧘㑐ᢙࡑ࡯ࠫࡦߩ୯߇Pࠃࠅᄢ߈޿႐วߪ㧘

ࠬ࡜࠶ࠢᄌᢙ_"_nߩ୯ߪ ߣߥࠆ㧚ⷐߔࠆߦࠬ࡜࠶ࠢ

ᄌᢙߩ୯ߪ㧘ኻ⽎ߣߥࠆࠨࡦࡊ࡞߇⺋ಽ㘃ߐࠇࠆ႐ ว߿㧘ᱜߒߊಽ㘃ߢ߈ߡ޿ߚߣߒߡ߽චಽߥ૛⵨߇

⏕଻ߐࠇߡ޿ߥ޿႐วߩ㧘⋡ᮡ㑐ᢙࡑ࡯ࠫࡦߣታ㓙 ߩ㑐ᢙࡑ࡯ࠫࡦߣߩᏅࠍ⴫ߒߡ޿ࠆߩߢ޽ࠆ㧚⸒޿

឵߃ࠆߣ㧘㑐ᢙࡑ࡯ࠫࡦߩ୯ࠍ޽ࠆ⋡ᮡ୯એ਄ߦߔ ࠆߎߣࠍቇ⠌ၮḰߣߒߚ႐วߩಽ㘃⺋ࠅᐲว޿㧘ߔ ߥࠊߜ៊ᄬߢ޽ࠆߣ⠨߃ࠆߎߣ߇ߢ߈ࠆ㧚

ಲ㑐ᢙߩ৻⒳ߢ޽ࠆࡅࡦࠫ៊ᄬࠍዉ౉ߔࠆߣ㧘ࠬ

࡜࠶ࠢᄌᢙࠍ៊ᄬߣߔࠆ᭎ᔨ߇ࠃࠅ᣿⏕ߦ⸃㉼ߢ߈ ࠆ㧚ߎߎߢࡅࡦࠫ៊ᄬߪ㧘㑐ᢙࡑ࡯ࠫࡦߩ୯߇Pࠃ ࠅᄢ߈޿႐วߪ ߦߥࠅ㧘ߘ߁ߢߥ޿႐วߦߪ

( )

n n

y b

P w x ! ߣ޿߁୯ࠍขࠆ߽ߩߣߔࠆ㧚ߔ ࠆߣᑼߢቯ⟵ߐࠇࠆSVMߩ⸠✵ᣇᴺߪ㧘 ߟߩ ၮḰࠍหᤨߦ↪޿ߚᦨㆡൻߢ޽ࠆߣ޿߁ߎߣ߇ࠊ߆ ࠆ㧚ߔߥࠊߜ㧘✢ᒻ್೎㑐ᢙߩ㊀ߺࡌࠢ࠻࡞ߩࡁ࡞

ࡓࠍᦨዊൻߔࠆߎߣߦࠃߞߡᐞ૗ࡑ࡯ࠫࡦߩᦨᄢൻ ࠍ⋡ᜰߔၮḰߣ㧘ࡅࡦࠫ៊ᄬߩᐔဋ୯ࠍᦨዊൻߔࠆ

(8)

ߎߣߦࠃߞߡ⸠✵ࠨࡦࡊ࡞㓸วߩ⺋ࠅᐲว޿ࠍᷫࠄ ߔߎߣࠍ⋡ᜰߔၮḰߢ޽ࠆ㧚ߒ߆ߒߎߩ⸠✵ၮḰߦ ߪ ߟߩ໧㗴߇޽ࠆ㧚߹ߕಽ㘃⺋ࠅᢙߩᦨዊൻߣ޿

߁ⷰὐ߆ࠄ⷗ࠆߣ㧘ࡅࡦࠫ៊ᄬߪ⺋ࠅᐲว޿ߦࠃߞ ߡᄢ߈ߊ୯߇ᄌࠊߞߡߒ߹߁ߩߢ㧘ಽ㘃⺋ࠅᢙࠍࠞ

࠙ࡦ࠻ߔࠆ៊ᄬߦߥࠄߥ޿ߣ޿߁໧㗴㧘ߘߒߡࠬ࡜

࠶ࠢᄌᢙߩଥᢙߣߒߡ૶ࠊࠇࠆቯᢙCߩ୯ࠍ㧘ࡅࡘ

࡯࡝ࠬ࠹ࠖ࠶ࠢߥᣇᴺߢߒ߆᳿ቯߢ߈ߥ޿ߣ޿߁໧

㗴ߢ޽ࠆ㧚Fig. 2ߪ ߟߩ៊ᄬ㑐ᢙࠍ࿑␜ߒߡ޿ࠆ㧚

Fig. 2. Schematic explanation of hinge, logistic, and smooth logistic losses.

/%' ߣ 58/ ߩᲧセ

ࠬ࡜࠶ࠢᄌᢙࠍ៊ᄬߣߒߡ⸃㉼ߔࠇ߫㧘MCE ߣ SVM ߪ౒ߦ៊ᄬࠍᦨዊൻߔࠆߎߣߦࠃߞߡಽ㘃᳿

ቯߦ᦭ലߥࠢ࡜ࠬࡕ࠺࡞ࠍ⸳⸘ߔࠆᚻᴺߢ޽ࠆߣ⸒

߁ߎߣ߇ߢ߈ࠆ㧚߹ߚ㧘⺋ಽ㘃᷹ᐲߣ㑐ᢙࡑ࡯ࠫࡦ ߩ╬ଔᕈࠃࠅ㧘 ߟߩᚻᴺߩ៊ᄬߪ㧘ߤߜࠄ߽㑐ᢙ ࡑ࡯ࠫࡦߩ㑐ᢙߢ޽ࠆߣ⠨߃ࠆߎߣ߇ߢ߈ࠆ㧚ߎߩ ࠃ߁ߥ㘃ૃᕈߩ৻ᣇߢ㧘MCEߣSVMߩ㑆ߦߪ㧘એ ਅߩࠃ߁ߥ ߟߩᄢ߈ߥ㆑޿߽ߺࠄࠇࠆ㧚

៊ᄬ㑐ᢙߩ㆑޿

ᦨㆡൻߦ߅޿ߡ೙⚂᧦ઙ߇ሽ࿷ߔࠆ߆ߤ߁߆ߩ

㆑޿

ᦨዊಽ㘃⺋ࠅ⏕₸⁁ᘒߣߩ৻⽾ᕈߣ޿߁ⷰὐ߆ࠄ⷗

ࠆߣ㧘SVMߢ૶ࠊࠇࠆࡅࡦࠫ៊ᄬࠃࠅ߽㧘MCE ߢ

૶ࠊࠇࠆᐔṖൻࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬ㧘ߔߥࠊߜಽ㘃

⺋ࠅᢙ៊ᄬߩᣇ߇᣿ࠄ߆ߦᦸ߹ߒ޿ߣ⠨߃ࠄࠇࠆ㧚

৻ᣇߢ㧘SVMߦߪᐞ૗ࡑ࡯ࠫࡦᦨᄢൻߩߚ߼ߦ᣿⏕

ߥ೙⚂߇⚵ߺㄟ߹ࠇߡ߅ࠅ㧘ᧂ⍮ࠨࡦࡊ࡞߳ߩ⠴ᕈ ࠍะ਄ߐߖࠆߎߣߦ⽸₂ߒߡ޿ࠆ㧚ߎࠇߪ㧘MCEߦ ߅޿ߡ៊ᄬߩᐔṖᕈࠍ೙ᓮߔࠆߎߣߦࠃࠅ㧘㑆ធ⊛

ߦಽ㘃⠴ᕈࠍะ਄ߐߖࠆߣ޿߁ᚻᴺߣߪኻᾖ⊛ߢ޽

ࠆ㧚ߒ߆ߒ޿ߕࠇߩᚻᴺ߽㧘዁᧪౉ജߐࠇࠆߢ޽ࠈ ߁ᧂ⍮ߩࠨࡦࡊ࡞߇㧘⸠✵ࠨࡦࡊ࡞ߩ๟ㄝߦሽ࿷ߔ ࠆߢ޽ࠈ߁ߣ޿߁⚛ᧉߥ೨ឭࠍᩮ᜚ߣߒߡ޿ࠆ㧚

ᐔṖᕈߩ೙ᓮߦട߃ߡ㧘ࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬߦߪ 㑐ᢙߩ㕖ಲᕈߦࠃߞߡᒁ߈⿠ߎߐࠇࠆᦨㆡൻߩ࿎㔍 ߐ߇ሽ࿷ߔࠆ㧚ࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬࠍ⿷ߒวࠊߖߚ

⚻㛎៊ᄬߦߪዪᚲ⊛ᦨዊ୯ࠍขࠆࡄ࡜ࡔ࡯࠲⁁ᘒ߇

޿ߊߟ߽ሽ࿷ߔࠆߩߢ㧘⌀ߩᦨㆡ⸃ࠍ⷗ߟߌߛߔߩ ߪᔅߕߒ߽ኈᤃߢߪߥ޿㧚ߘߩὐ㧘ࡅࡦࠫ៊ᄬߪಲ 㑐ᢙߢ޽ࠅ㧘ߘࠇࠍ⿷ߒวࠊߖߚ⚻㛎៊ᄬߩᦨዊὐ

߽ᮡḰ⊛ߥត⚝ࠕ࡞ࠧ࡝࠭ࡓߦࠃߞߡኈᤃߦ⷗ߟߌ ߛߔߎߣ߇ߢ߈ࠆ㧚ߎߩᦨㆡൻߩߒ߿ߔߐ߇㧘SVM ߇ᐢߊ᥉෸ߔࠆ৻࿃ߣߥߞߡ޿ࠆ㧚

ߒ߆ߒᦨㆡൻᣇᴺߦ૗ࠄ߆ߩᡷ⦟ߪᔅⷐߢߪ޽ࠆ

߽ߩߩ㧘㕖ಲߥ៊ᄬࠍ↪޿ߚ MCE ⸠✵ߩ೑ὐߪ⏕

߆ߦሽ࿷ߔࠆ㧚SVM ߩ⸠✵ᴺ߇࿕ቯᰴరߩ ࠢ࡜

ࠬ࡮ࠨࡦࡊ࡞ߦኻߔࠆಽ㘃ེߩ⸠✵ߦ೙㒢ߐࠇߡ޿

ࠆߩߦኻߒߡ㧘MCEߩቯᑼൻߢߪ㧘⿥ᄢⷙᮨߥᄙࠢ

࡜ࠬߩ㧘ߒ߆߽นᄌ㐳ࡄ࠲࡯ࡦߩಽ㘃ࠍⴕ߁ಽ㘃ེ

ߩ⸳⸘ࠍ฽߻㧘ታߦᐢ▸ߥಽ㘃ེߩ⸳⸘ࠍᛒ߁ߎߣ ߇ߢ߈ࠆὐߢ޽ࠆ㧚

એ਄ࠃࠅ㧘MCEߣSVMߦߪ㧘੕޿ߦ৻ᣇߛߌ߇ ᜬߟࠃ߁ߥ㐳ᚲߣ⍴ᚲ߇޿ߊߟ߆ሽ࿷ߔࠆ㧚ᄦޘߩ 㐳ᚲࠍ਄ᚻߊ⚵ߺวࠊߖߚቇ⠌ᚻᴺߩ㐿⊒߇ታ⃻ߐ ࠇࠇ߫ࡄ࠲࡯ࡦ⹺⼂ߩᛛⴚߩㅴዷߦᄢ޿ߦነਈߔࠆ ߦ㆑޿ߥ޿㧚

/%' ቇ⠌ߦ߅ߌࠆᐞ૗ࡑ࡯ࠫࡦᦨᄢൻ ߎߎߢߪMCEቇ⠌ߦSVMߩ೑ὐࠍขࠅ౉ࠇࠆᣇ ᴺࠍ⠨߃ࠆ㧚SVMߩ೑ὐࠍขࠅ౉ࠇߚMCEቇ⠌ߣ ߪ㧘ߘߩቇ⠌ᴺߩਛߦᐞ૗ࡑ࡯ࠫࡦᦨᄢൻߣ޿߁઀

⚵ߺࠍ⚵ߺㄟࠎߛ߽ߩߢ޽ࠆ㧚㑐ᢙߩᦨᄢൻಣℂߪ㧘 ൨㈩ត⚝ߩࠃ߁ߥᦨㆡൻᚻᴺߦࠃߞߡ◲නߦታⵝߢ ߈ࠆߩߢ㧘MCEߦ߅ߌࠆᐞ૗ࡑ࡯ࠫࡦᦨᄢൻߢ㊀ⷐ ߣߥࠆߩߪ㧘ᐞ૗ࡑ࡯ࠫࡦࠍ૗ࠄ߆ߩ㑐ᢙߩᒻߢቯ ᑼൻߒߡ㧘ಽ㘃ེࠍ⸠✵ߔࠆ㓙ߩ⋡ᮡ㑐ᢙࠍ᣿⏕ߦ ߔࠆߎߣߢ޽ࠆ㧚SVMߦ߅޿ߡ✢ᒻ್೎㑐ᢙߦኻߔ ࠆᐞ૗ࡑ࡯ࠫࡦ߇ዉ౉ߐࠇߡ޿ߚ߇㧘ታ↪⊛ߥࡄ࠲

࡯ࡦಽ㘃ེߩᄙߊߪ㧘㓝ࠇࡑ࡞ࠦࡈࡕ࠺࡞ߩࠃ߁ߥ

⏕₸᷹ᐲဳ್೎㑐ᢙ߿࠾ࡘ࡯࡜࡞ࡀ࠶࠻ߩࠃ߁ߥ㕖

✢ᒻ್೎㑐ᢙ㧘ⶄᢙࡊࡠ࠻࠲ࠗࡊࠍ↪޿ࠆ〒㔌᷹ᐲ

ဳߩ㕖✢ᒻ್೎㑐ᢙߥߤ㧘ᣢߦቯ⟵ߐࠇߡ޿ࠆᐞ૗

ࡑ࡯ࠫࡦߪㆡ↪ߢ߈ߥ޿್೎㑐ᢙࠍ↪޿ߡ޿ࠆ㧚ᓥ

(9)

ߞߡ㧘ߎࠇࠄߩታ↪⊛ߥ႐วߦ߽ㆡ↪น⢻ߥಽ㘃ེ

޽ࠆ޿ߪ್೎㑐ᢙࠍᗐቯߒߡᣂߚߦᐞ૗ࡑ࡯ࠫࡦࠍ ዉ౉ߔࠆᗧ⟵ߪᄢ߈޿㧚એਅߢߪ㧘ߎ߁ߒߚⷰὐߦ

┙ߞߡ㧘޿ߊߟ߆ߩઍ⴫⊛ߥಽ㘃ེߦ㑐ߒߡ㧘ᐞ૗

ࡑ࡯ࠫࡦߩቯᑼൻߩ଀ࠍ⚫੺ߔࠆ㧚

߹ߕ೨▵߹ߢߢ⼏⺰ߒߚࠃ߁ߦ㧘৻ጀࡄ࡯࠮ࡊ࠻

ࡠࡦ㧘޽ࠆ޿ߪ✢ᒻ್೎㑐ᢙߦࠃࠆಽ㘃ེߩ႐วߪ㧘 ᐞ૗ࡑ࡯ࠫࡦߩ㑐ᢙᒻߪ᣿ࠄ߆ߢ޽ࠅ㧘ᑼ㧘޽

ࠆ޿ߪߘࠇߦ೙⚂ࠍ౉ࠇߚᒻᑼߩᑼߢ޽ࠆ㧚೙

⚂ࠍ౉ࠇߚᑼߢߩ⸠✵ኻ⽎ߪ㊀ߺࡌࠢ࠻࡞ߩ L2ࡁ࡞ࡓ ||w||²ߢ޽ࠅ㧘ߎߩࡁ࡞ࡓࠍᦨዊൻߔࠆߎ ߣߦࠃߞߡᐞ૗ࡑ࡯ࠫࡦࠍᦨᄢൻߔࠆߎߣ߇ߢ߈ࠆ㧚 ߎߩᐞ૗ࡑ࡯ࠫࡦᦨᄢൻࠍMCEቇ⠌ߦ⚵ߺㄟ߻ߥ ࠄ߫㧘SVMߦ߅ߌࠆࡅࡦࠫ៊ᄬࠍ㧘ᐔṖൻࡠࠫࠬ࠹

ࠖ࠶ࠢ៊ᄬߦ⟎߈឵߃ࠇ߫⦟޿㧚ߎߩ႐ว㧘ಽ㘃⺋

ࠅᢙߩㄭૃߢ޽ࠆᐔṖൻࡠࠫࠬ࠹ࠖ࠶ࠢ៊ᄬߪ㕖ಲ ߥߩߢ㧘߿ߪࠅ⸠✵ߦ߅ߌࠆዪᚲ⊛ᦨㆡ⸃ߩ໧㗴ߪ ᱷࠆߎߣߦߥࠆ㧚

ᄙጀࡄ࡯࠮ࡊ࠻ࡠࡦߦ߅޿ߡ߽㧘਄⸥ߩ৻ጀࡄ࡯

࠮ࡊ࠻ࡠࡦߩࠕࡊࡠ࡯࠴ߪኈᤃߦㆡ↪ߢ߈ࠆ㧚ᦨ⚳

ጀ߳ߩ౉ജࡌࠢ࠻࡞ࠍ㧘ਈ߃ࠄࠇߚࠨࡦࡊ࡞ࡌࠢ࠻

࡞߇ᄌ឵ߐࠇߚ߽ߩߣ⠨߃ࠆߣ㧘ᄙጀࡄ࡯࠮ࡊ࠻ࡠ ࡦߩ⸠✵ߪ㧘ᄌ឵ᓟߩࠨࡦࡊ࡞ⓨ㑆ߦ߅ߌࠆ৻ጀࡄ

࡯࠮ࡊ࠻ࡠࡦ⸠✵ߣ⷗ߥߔߎߣ߇ߢ߈ࠆ㧚ᦨ⚳ጀߢ ߩ㊀ߺࠍᑼߢߩ㊀ߺࡌࠢ࠻࡞w㧘ߘߒߡᦨ⚳ጀ

߳ߩ౉ജࠍᑼߢߩ౉ജࡌࠢ࠻࡞xߣ⠨߃ࠆߣ㧘 ᄌ឵ᓟߩࠨࡦࡊ࡞ⓨ㑆ߢᣂߒߊᐞ૗ࡑ࡯ࠫࡦࠍቯ⟵

ߢ߈ࠆ㧚ߎߩߣ߈ಽ㘃ེߩ⸠✵ᚻᴺߣߒߡߪ㧘ᐞ૗

ࡑ࡯ࠫࡦߩᦨᄢൻߣಽ㘃⺋ࠅᢙ៊ᄬߩᦨዊൻࠍหᤨ

ߦ⋡ᜰߔࠃ߁ߥᣇᴺ߇⠨߃ࠄࠇࠆ㧚ߒ߆ߒ SVM ߦ ߅ߌࠆ㕖✢ᒻ౮௝ߣห᭽ߦ㧘రߩⓨ㑆ߣߪ⇣ߥࠆⓨ

㑆ߦ౮௝ߔࠆߎߣߦࠃߞߡ㧘ᐞ૗ࡑ࡯ࠫࡦߩᦨᄢൻ ߇ߤߩ⒟ᐲಽ㘃⠴ᕈะ਄ߦ⽸₂ߔࠆ߆ߪ߿߿ਇ᣿⍎

ߦߥࠆ߽ߩߣ⠨߃ࠄࠇࠆ㧚

ᰴߦ㧘ฦࠢ࡜ࠬࠍ⴫⃻ߔࠆࡕ࠺࡞ߣߒߡઍ⴫ߩࡊ ࡠ࠻࠲ࠗࡊࠍ૶޿㧘ߘࠇࠄߩࡊࡠ࠻࠲ࠗࡊߣߩ〒㔌 ࠍ್೎㑐ᢙߣߔࠆಽ㘃ེߢߩᐞ૗ࡑ࡯ࠫࡦߦߟ޿ߡ

⠨߃ࠆ㧚〒㔌ߣዕᐲ㧔⏕₸㧕ߪኒធߦ㑐ଥߒߡ޿ࠆ ߩߢ㧘ߎߩಽ㘃ེߦ਄ᚻߊᐞ૗ࡑ࡯ࠫࡦᦨᄢൻߩ઀

⚵ߺࠍขࠅ౉ࠇࠆߎߣ߇ߢ߈ࠇ߫㧘⃻࿷㖸ჿ⹺⼂ߥ

ߤߢᐢߊ૶ࠊࠇߡ޿ࠆ㓝ࠇࡑ࡞ࠦࡈࡕ࠺࡞߿㧘ᷙว

ࠟ࠙ࠬಽᏓߥߤߦ߽ᔕ↪ߔࠆߎߣ߇ߢ߈ࠆ㧚Fig. 3 ߪ㧘〒㔌ߦࠃࠆಽ㘃ེࠍ૶ߞߚ႐วߩ㧘 ࠢ࡜ࠬߩ Ⴚ⇇ߦᵈ⋡ߒߚᐞ૗ࡑ࡯ࠫࡦࠍ⴫ߒߡ޿ࠆ㧚pࠍࠢ

࡜ࠬCߩࡊࡠ࠻࠲ࠗࡊ㧘pࠍࠢ࡜ࠬCߩࡊࡠ࠻࠲

ࠗࡊߣߔࠆߣ㧘ಽ㘃Ⴚ⇇ߪߎࠇࠄߩࡊࡠ࠻࠲ࠗࡊࠍ

↪޿ߡ㧘ᰴߩ᧦ઙࠍḩߚߔࠃ߁ߥὐߩ㓸วߢ޽ࠆߣ

⠨߃ࠆߎߣ߇ߢ߈ࠆ㧚

ˆ ˆ

x p x p ߎߎߢ㧘xˆߪಽ㘃Ⴚ⇇਄ߩὐߣߔࠆ㧚ߔࠆߣಽ㘃Ⴚ

⇇ߣ㧘ߘࠇߦ৻⇟ㄭ޿ࠨࡦࡊ࡞ࡌࠢ࠻࡞xߣߩ〒㔌 ߢ޽ࠆᐞ૗ࡑ࡯ࠫࡦdߪ㧘એਅߩᑼߢ⴫ߐࠇࠆ㧚

2 2

d 2

x p x p

p p

Fig. 3. Schematic explanation of geometric margin for distance classifier.

ᑼߩಽሶߪ㧘㑐ᢙࡑ࡯ࠫࡦ߿⺋ಽ㘃᷹ᐲߣหߓ ᑼߦߥߞߡ޿ࠆ㧚ᐞ૗ࡑ࡯ࠫࡦߩ୯ࠍᦨᄢൻߔࠆߚ

߼ߦߪ㧘ಽሶߩ㑐ᢙࡑ࡯ࠫࡦߩ୯ࠍᄢ߈ߊߒ㧘ಽᲣ ߩ୯㧘ߔߥࠊߜࡊࡠ࠻࠲ࠗࡊห჻ߩ〒㔌ࠍዊߐߊߒ ߥߌࠇ߫ߥࠄߥ޿㧚ߒ߆ߒಽሶߩxߪ⸠✵ࠨࡦࡊ࡞

ࡌࠢ࠻࡞ߥߩߢ㧘േ߆ߔߎߣߪߢ߈ߥ޿㧚߹ߚ㧘✢

ᒻ್೎㑐ᢙߣห᭽ߦ㧘㑐ᢙࡑ࡯ࠫࡦߩ୯ߛߌࠍേ߆ ߒߡ߽ᔅߕߒ߽ಽ㘃Ⴚ⇇ߩ૏⟎߇ᄌൻߔࠆߣߪ㒢ࠄ ߥ޿㧚ࠃߞߡᐞ૗ࡑ࡯ࠫࡦࠍᦨᄢൻߔࠆߚ߼ߦߪ㧘 ಽሶߩ୯ࠍ৻ቯߦߔࠆࠃ߁ߥ૗ࠄ߆ߩ೙⚂ࠍ౉ࠇߚ

਄ߢ㧘ࡊࡠ࠻࠲ࠗࡊห჻ߩ〒㔌ࠍㄭߠߌࠇ߫⦟޿ߣ

⠨߃ࠄࠇࠆ㧚

ߎߎ߹ߢߩ⼏⺰߆ࠄ㧘ᐞ૗ࡑ࡯ࠫࡦࠍᦨᄢൻߔࠆ

(10)

MCEቇ⠌ࠍታ⃻ߔࠆߚ߼ߦߪ㧘✢ᒻ್೎㑐ᢙߦ߅޿

ߡߪࡄ࡜ࡔ࡯࠲ߩL₂ࡁ࡞ࡓ w ²㧘ࡊࡠ࠻࠲ࠗࡊࠍ↪

޿ߚಽ㘃ེߦ߅޿ߡߪࡊࡠ࠻࠲ࠗࡊห჻ߩ〒㔌ߢ޽

ࠆ pp ࠍᦨዊൻߔࠇ߫⦟޿ߣ޿߁ߎߣߦߥࠆ㧚 ߎࠇࠄ2⒳㘃ߩࡁ࡞ࡓߩᦨዊൻߪ㧘ߘࠇߙࠇߦኻᔕ ߔࠆ್೎㑐ᢙ߇୯ࠍขࠆ▸࿐ࠍ⁜߼ࠆߎߣߦ⽸₂ߔ ࠆ㧚ߟ߹ࠅᐞ૗ࡑ࡯ࠫࡦᦨᄢൻߪ㧘౉ജࡄ࠲࡯ࡦߩ

㆑޿ߦࠃࠆಽ㘃᳿ቯߩᄌൻᐲวࠍዊߐߊߔࠆߎߣߦ ࠃࠅ㧘⠴ᕈߩ㜞޿ಽ㘃ེࠍታ⃻ߒߡ޿ࠆߩߢ޽ࠆ㧚

߹ߣ߼

ࠃࠅ㜞޿⸠✵⠴ᕈࠍᜬߟᦨዊಽ㘃⺋ࠅቇ⠌ࠍ⋡ᜰ ߒ㧘MCEߣSVMߣ޿߁ ߟߩ⸠✵ᚻᴺߩቯᑼൻࠍ

⺑᣿ߒߚ㧚ߘߒߡ ߟߩᚻᴺߩ㐳ᚲߣ⍴ᚲࠍㅀߴߚ

਄ߢ㧘ಽ㘃⺋ࠅᢙߩᦨዊൻߣᐞ૗ࡑ࡯ࠫࡦߩᦨᄢൻ ࠍหᤨߦታ⃻ߔࠆ㧘ᣂߒ޿ MCE ቇ⠌ߦߟ޿ߡ⠨ኤ ߒߚ㧚

ߎߎߢ଀ߣߒߡขࠅ਄ߍߚࡊࡠ࠻࠲ࠗࡊߣߩ〒㔌 ߦࠃࠆಽ㘃ེߪ㧘ෳ⠨ᢥ₂ߩlarge margin HMMߣ

޿߁ᚻᴺߣૃߡ޿ࠆㇱಽ߇޽ࠆ㧚ߒ߆ߒߎߩᢥ₂ߢ ߪ㧘ឭ᩺ߐࠇߡ޿ࠆᚻᴺߣᐞ૗ࡑ࡯ࠫࡦᦨᄢൻߩ㑐 ଥ߇᣿⏕ߦ⸥ㅀߐࠇߡ޿ߥ޿㧚৻ᣇ㧘⼂೎ቇ⠌ߦ߅

޿ߡಽ㘃᳿ቯࠍࠃࠅ⠴ᕈߩ㜞޿߽ߩߦߔࠆߚ߼ߦߪ㧘

૗ࠄ߆ߩ೙⚂ࠍ౉ࠇࠆᔅⷐ߇޽ࠆߣ޿߁ᜰ៰߇ߥߐ ࠇߡ޿ࠆ㧚ߎߩᜰ៰ߪ㧘ᐞ૗ࡑ࡯ࠫࡦࠍᄢ߈ߊߔ ࠆߣ޿߁ᧄ᧪ߩ⸳⸘⠴ᕈะ਄╷ࠍ⋥ធ⊛ߦߪᛒࠊߥ

޿߽ߩߩ㧘޽ࠆ⒳ߩㄭૃ⊛ߥࠕࡊࡠ࡯࠴ߣ⷗஖ߔߎ ߣ߽ߢ߈ࠆ㧚߹ߚ㧘రޘߩ MCE ߩᨒ⚵ߺߢ⠴ᕈࠍ ะ਄ߐߖࠆ⹜ߺ߽ႎ๔ߐࠇߡ޿ࠆ߇㧘ᐞ૗ࡑ࡯ࠫࡦ ࠍᄢ߈ߊߔࠆⷞὐࠍᓧࠆߦ⥋ߞߡ޿ߥ޿^㧕㧚

ᧄ⺰ᢥߢߪ㧘⸠✵⠴ᕈࠍะ਄ߐߖࠆߚ߼ߦ⌀ߦ㊀ ⷐߥߎߣߪᐞ૗ࡑ࡯ࠫࡦߩᦨᄢൻߦ޽ࠆߎߣࠍ᣿⏕

ൻߒ㧘ߘࠇࠍታ⃻ߔࠆߚ߼ߩ MCE ቇ⠌ߩߚ߼ߦᣂ ߚߦታ↪⊛್೎㑐ᢙ↪ߩᐞ૗ࡑ࡯ࠫࡦߩቯ⟵ࠍዉ಴

ߒߚ㧚੹ᓟ㧘ឭ᩺ߒߚᦨᄢᐞ૗ࡑ࡯ࠫࡦ MCE ቇ⠌

ᴺߩౕ૕⊛ߥታⵝᴺߣߘߩ⹏ଔࠍⴕ߁੍ቯߢ޽ࠆ㧚

ᧄ⎇ⓥߩ৻ㇱߪ㧘ᣣᧄቇⴚᝄ⥝ળ⑼ቇ⎇ⓥ⾌⵬ഥ

㊄࡮ၮ⋚⎇ⓥB⺖㗴⇟ภ㧦ߩេഥߦࠃ ߞߡⴕࠊࠇߡ޿ࠆ㧚

ෳ⠨ᢥ₂

1) R.O.Duda and P.E. Hart, “Pattern Classification and Scene Analysis”, (Wiley Interscience Publishers, 1973).

2) S. Katagiri, B. Juang, and C. Lee, “Pattern Recognition Using a Family of Design Algorithms Based Upon the Generalized Probabilistic Descent Method”, Proc. IEEE., vol. 86, no. 11, pp. 2345-2373 (1998).

3) V. N. Vapnik, “The Nature of Statistical Learning Theory”, (Springer-Verlag, 1995).

4) J. Lafferty, A. McCallum, and F. Pereira, “Conditional Random Fields: Probabilistic Models for Segmental and Labeling Sequence Data”, Proc. ICML 2001, pp. 282-289 (2001).

5) J. Friedman, T. Hastie, and R. Tibshirani, “Additive Logistic Regression: A Statistical View of Boosting” The Annal of Statistics, vol. 28, no. 2, pp. 337-407 (2000).

6) S. Katagiri, “A Unified Approach to Pattern Recognition”, Proc. ISANN ’94, pp. 561-570 (1994).

7) E. McDermott and S. Katagiri, “A Derivation of Minimum Classification Error from the Theoretical Classification Risk Using Parzen Estimation”, Computer Speech and Language, vol. 18, pp. 107-122 (2004).

8) E. McDermott and S. Katagiri, “Discriminative Trainig via Minimization of Risk Estimates Based on Parzen Smoothing”, Appl. Intell., vol. 25, pp 35-57 (2006).

9) N. Cristianini and J. Shawe-Taylor, “An Introduction to Support Vector Machines and Other Kernel-based Learning Methods”, (Cambridge University Press, Cambridge, 2000).

10) H. Jiang, X. Li, and C. Liu, “Large Margin Hidden Markov Models for Speech Recognition”, IEEE Trans.

Audio, Speech, and Language Processing, vol. 4, No. 5, pp.1584-1595 (2006).

11)T. Poggio and F. Girosi, “Regularization Algorithms for Learning That Are Equivalent to Multi-Layer Networks”, Science, vol. 247, pp. 978-982 (1990).

12) D. Yu, L. Deng, X. He, and A. Acero, “Large-margin Minimum Classification Error Training: A Theoretical Risk Minimization Perspective“, Computer Speech and Language, vol. 22, pp. 415-429 (2008).