コールセンターの業務改善に向けた応答マニュアルの分析と検索手法の検討

ίʔϧηϯλʔͷۀ຿վળʹ޲͚ͨ

Ԡ౴ϚχϡΞϧͷ෼ੳͱݕࡧख๏ͷݕ౼

Analysis of FAQ and improvement of information retrieval for call center operation

ࢁԼ ྒྷਅ

∗1 Ryoma Yamashita

ݪ ݠհ

∗2 Kensuke Hara

ాଜ ఩࢚

∗1 Satoshi Tamura

଎ਫ ޛ

∗1 Satoru Hayamizu ∗1

_ذෞେֶ

Univ. of Gifu ∗2

_{גࣜձࣾ ηΠϊʔ৘ใαʔϏε}

Seino Information Service Co., Ltd.

At a call center, an operator searches for an appropriate manual based on an inquiry given by the customer. However, it is difficult to choose a correct manual from retrieval results. The purpose of this study is to analyze manuals used in an actual call center in order to build an effective retrieval method such as a two-question-and-two-answer scheme. We applied a clustering technique to the manuals and analyzed data by cluster visualization. We then investigated the potential to employ the approach. We finally compared the proposed method with the baseline method and showed its effectiveness.

1. ͸͡Ίʹ

೔ʑͷੜ׆΍ۀ຿ͳͲ༷ʑͳ৔໘Ͱසൟʹจॻݕࡧ͕ߦΘΕ ͍ͯΔɽͦ͜Ͱɼ๲େͳจॻ܈͔Βద੾ͳจॻͷݕࡧ͕ߦ͑Δ ͜ͱ͸ඇৗʹॏཁͱͳΔɽίʔϧηϯλʔۀ຿ʹ͓͍ͯ͸໰͍ ߹ΘͤΛड͚ͨࡍɼԠ౴ϚχϡΞϧͷݕࡧΛߦ͍ճ౴Λߦͬͯ ͍Δɽਝ଎͔ͭਖ਼֬ʹରԠΛߦ͏ඞཁ͕͋ΔͨΊɼԠ౴Ϛχϡ Ξϧͷݕࡧਫ਼౓޲্͸ಛʹظ଴͞ΕΔɽҰൠతͳݕࡧγεςϜ Ͱ͸ɼϢʔβ͕ೖྗͨ͠ΫΤϦʹର͠ճ౴ީิΛෳ਺݅ఏࣔ ͠ɼϢʔβ͕ద੾ͳީิΛબ୒͢Δɽ͔͠͠ɼݕࡧ݁Ռ͕ෳ਺ ఏࣔ͞Εͨͱ͖ɼͦͷ಺༰Λਝ଎ʹ൑அ͠બ୒͢Δ͜ͱ͕Ͱ͖ ͳ͔ͬͨܦݧΛ࣋ͭΦϖϨʔλ΋ଟ͍ɽ·ͨɼઐ໳ੑ͕ඞཁͳ ͜ͱ΋ؚΊɼΦϖϨʔλͷίʔϧηϯλʔۀ຿ͷܦݧ͕ઙ͍৔ ߹ʹ͸ద੾ͳΩʔϫʔυʹΑΔਖ਼֬ͳݕࡧ͕ߦ͑ͳ͍৔߹͕ ͋Δɽ Ҏ্ͷ՝୊͔Βɼίʔϧηϯλʔۀ຿ͷิॿΛ໨తʹɼैདྷ ͷݕࡧγεςϜΛର࿩ܗࣜʹஔ͖׵͑Δ͜ͱΛߟ͑Δɽର࿩ܗ ࣜʹ͸ɼର࿩ͷதͰճ౴ͷߜΓࠐΈΛߦ͑Δͱ͍͏ར఺͕͋ ΔɽຊݚڀͰ͸γεςϜଆ͔Β໰͍ฦ͠Λߦ͏2໰2౴ܗࣜ ʹ͢Δ͜ͱͰݕࡧਫ਼౓ͷ޲্Λ໨ࢦ͢ɽಛʹᐆດͳΫΤϦʹର ͯ͠௥Ճͷ࣭໰Λߦ͏͜ͱͰ࠷దͳճ౴ʹಋ͘͜ͱ͕ߟ͑Β ΕΔɽຊ࿮૊ΈͰ͸ద੾ͳճ౴Λಋͨ͘ΊʹԿΛγεςϜଆ ͔Β໰͍͔͚Δͷ͔͕՝୊ͱݴ͑Δɽ՝୊΁ͷղܾࡦͱͯ͠ɼ ϢʔβͷΫΤϦʹର͠ɼ௚઀Ԡ౴จΛఏࣔ͢ΔͷͰ͸ͳ͘ɼෳ ਺ͷԠ౴จΛؚΉάϧʔϓΛ਺݅ఏࣔ͢Δ͜ͱΛఏҊ͢Δɽݕ ࡧର৅ͷάϧʔϓ͕ఆ·Δ͜ͱͰճ౴ީิ͕ߜΓࠐΊΔͨΊɼ ݕࡧਫ਼౓ͷ޲্͕ظ଴͞ΕΔɽͦ͜Ͱɼ·ͣԠ౴ϚχϡΞϧͷ ΫϥελϦϯάʹΑΔάϧʔϓԽͱ෼ੳΛߦͬͨɽ࣍ʹɼϢʔ βͷΫϥελબ୒ޙʹݕࡧΛߦ͏͜ͱʹΑΔճ౴ͷߜΓࠐΈͷ ༗ޮੑʹ͍ͭͯݕূΛߦͬͨɽ࠷ޙʹɼҎ্ͷݕূ݁Ռ͔Β2 ໰2౴ݕࡧγεςϜͷ࣮ݱʹ޲͚ͯߟ࡯Λߦͬͨɽ ୈ2ষͰ͸ɼຊ࣮ݧͰ༻͍ͨσʔλͷ֓ཁΛࣔ͢ɽୈ3ষͰ ͸ɼԠ౴ϚχϡΞϧͷΫϥελϦϯάʹ͍ͭͯࣔ͢ɽୈ4ষͰ ͸ɼݕࡧਫ਼౓ʹؔ͢ΔධՁ࣮ݧͷํ๏ͱͦͷ݁Ռ͍ͭͯࣔ͢ɽ ୈ5ষʹ·ͱΊΛࣔ͠ɼ࠷ޙʹୈ̒ষͰࠓޙͷ՝୊Λࣔ͢ɽ ࿈བྷઌ:ࢁԼྒྷਅɼذෞେֶɼ[email protected] ද1: ࣭໰จ(ΫΤϦ) (ਖ਼ղจॻID͸ද2ͱରԠ) ࣭໰จ(ΫΤϦ) ਖ਼ղจॻID ిࢠாථʹࡌ͍ͬͯΔ΋ͷΑΓલͷ৘ ใ͸ݟΕ·͔͢ NT00026B92 excelॎॻ͖ ԣॻ͖ NT00025336 ిࢠாථͰ࣍ϖʔδͷόʔ͕ͳ͍ NT00029B12

2. ࣭໰Ԡ౴ίʔύε

ຊݚڀͰ࢖༻࣭ͨ͠໰Ԡ౴ίʔύεʹ͍ͭͯઆ໌͢Δɽ͜Ε ͸ɼίʔϧηϯλʔͷϔϧϓσεΫͰ࢖༻͞Ε࣭ͨ໰จ17,157 ݅ͱɼͦͷ࣭໰ʹର͢ΔԠ౴ϚχϡΞϧ5,171݅ͷσʔλ͔Β ߏ੒͞Ε͍ͯΔɽ ຊݚڀͰ͸ɼΫϥελϦϯάͷର৅ͱͯ͠Ԡ౴ϚχϡΞϧΛ ࢖༻͠ɼݕࡧਫ਼౓ͷධՁ࣮ݧʹ࣭໰จͱԠ౴ϚχϡΞϧΛ࢖༻ ͨ͠ɽ2.1અͰ͸࣭໰จɼ2.2અͰ͸Ԡ౴ϚχϡΞϧʹ͍ͭͯ આ໌͢Δɽ࠷ޙʹ2.3અͰ͸จॻͷલॲཧʹ͍ͭͯࣔ͢ɽ

2.1 ࣭໰จ (ΫΤϦ)

࣭໰จ͸ίʔϧηϯλʔͰ࣮ࡍʹߦΘΕ࣭ͨ໰ΛूΊͨίʔ ύεͰ͋Δɽ࣭໰จͷςΩετʹՃ͑ͯɼ࣭໰ʹରͯ͠ͲͷϚ χϡΞϧ͕ద͍ͯ͠Δ͔ͷϥϕϧ(ਖ਼ղจॻID)͕෇༩͞Εͯ ͍Δɽຊ࣮ݧͰ͸ɼ෇༩͞ΕͨϥϕϧΛΫΤϦʹର͢Δਖ਼ղจ ॻͱͯ͠ѻ͏ɽ࣭໰จͷྫΛද1ʹࣔ͢ɽ

2.2 Ԡ౴ϚχϡΞϧ (ճ౴จ)

Ԡ౴ϚχϡΞϧ͸ɼIDɼճ౴ຊจɼճ౴ͷλΠτϧɼେ෼ྨ (form)ɼத෼ྨ(system)ͷ5ͭͷ৘ใΛ࣋ͭɽຊ࣮ݧͰ͸ɼ ը૾ͷΈͷϚχϡΞϧͳͲͷจॻΛআ֎͠ɼܭ5,171݅Λѻͬ ͨɽྫͱͯ͠ɼ࣮ࡍͷճ౴ͷλΠτϧͱճ౴ຊจΛද2ʹࣔ ͢ɽճ౴λΠτϧ͸ΫΤϦͱྨࣅͨ͠จʹͳ͍ͬͯΔ͜ͱ͕ଟ ͍ɽ·ͨɼେ෼ྨɼத෼ྨͷৄࡉΛද3ʹࣔ͢ɽ͜ΕΒͷϥϕ ϧ͸ਓखͰ෇͚ΒΕͨԠ౴ϚχϡΞϧͷ෼ྨͰ͋Δɽ

2.3 จॻͷલॲཧ

࣭໰จ΍ճ౴จʹ͸ɼશ֯ͱ൒֯ͷҧ͍΍ɼΞϧϑΝϕοτ ͷେจࣈখจࣈͷҧ͍ʹΑΔදهͷ༳Ε͕ଘࡏ͢Δɽ͜ΕΒʹ ରॲ͢ΔͨΊɼΞϧϑΝϕοτͱ਺ࣈ͸൒֯ɼͦͷଞΛશ֯ʹ

1

The 34th Annual Conference of the Japanese Society for Artificial Intelligence, 2020

ද2: Ԡ౴ϚχϡΞϧจ(จॻID͸ද1ͱରԠ) จॻID Ԡ౴ϚχϡΞϧ NT00026B92 ճ౴λΠτϧ_: ిࢠாථʹࡌ͍ͬͯ Δՙ෺ࣄނۜߦৼࠐ໌ࡉͷաڈ෼Λ ݟ͍ͨ ճ౴ຊจ: ݱ࣌఺ͰӾཡՄೳͳ෼Α Γ΋աڈͷ෼͸ग़ͤͳ͍͜ͱΛҊ಺ ෼ྨ: ۀ຿֓ཁɼిࢠாථγεςϜ NT00025336 ճ౴λΠτϧ_{: excel}Ͱೖྗ߲໨͕ॎ ॻ͖ʹͳͬͯ͠·͏ ճ౴ຊจ_: ഑ஔϘλϯ͔Βॎॻ͖Λ ղআ ෼ྨ_: ۀ຿֓ཁɼpcؔ࿈ NT00029B12 ճ౴λΠτϧ_: ిࢠாථͷϖʔδૹ ΓͷϘλϯ͕ফ͍͑ͯΔ ిࢠாථը ໘ͰϖʔδૹΓͰ͖ͳ͍ ճ౴ຊจ_: දࣔ πʔϧόʔ ϖʔδ όʔΛνΣοΫͰදࣔ͠·͢ ෼ྨ_: ো֐ରԠɼిࢠாථγεςϜ ද3: ෼ྨ৘ใ ෼ྨཻ౓ ෼ྨ େ෼ྨ(form) ۀ຿֓ཁɼো֐ରԠ(2छ) த෼ྨ(system) pcؔ࿈ɼssxؔ࿈ɼΠϯϑϥؔ࿈ɼΧ ϯΨϧʔɼσʔλަ׵ɼσδλίɼϓϩ ϑΟοτɼϝʔϧɼԻ੠Ԡ౴γεςϜɼ ૹঢ়ՙࡳҹ࡮γεςϜɼిࢠாථγε ςϜɼి࿩faxؔ࿈ɹ(12छ) ͠ɼΞϧϑΝϕοτʹؔͯ͠͸શͯখจࣈʹ౷Ұͨ͠ɽ·ͨɼ ه߸ɼ۟ಡ఺ͷআڈΛߦͬͨɽจষΛ୯ޠ୯Ґʹ෼ׂ͢Δࡍʹ ͸ܗଶૉղੳثMeCabΛ༻͍ͨɽMeCabͷࣙॻʹ͸ݻ༗ද ݱʹڧ͍ͱ͞ΕΔmecab-ipadic-NEologdΛ࢖༻ͨ͠ɽ

3. Ԡ౴ϚχϡΞϧͷάϧʔϓԽͱ෼ੳ

Ԡ౴ϚχϡΞϧͷ෼ੳʹ͸จॻΫϥελϦϯάΛ༻͍ͨɽΫ ϥελϦϯάʹΑΔྨࣅจॻͷߜΓࠐΈͱɼจॻͷ·ͱ·Γ͝ ͱͷಛ௃Λଊ͑Δ͜ͱΛ໨తͱ͢ΔɽԠ౴ϚχϡΞϧͷάϧʔ ϓԽ͸ɼਓखͰ෇͚ΒΕͨେ෼ྨͱத෼ྨͷϥϕϧʹΑΓ෼ׂ ͨ͠ޙɼͦΕͧΕͷϥϕϧʹରͯ͠จॻΫϥελϦϯάΛߦ ͏ɽ͜ΕʹΑΓ107ͷΫϥελΛಘͨɽຊ࣮ݧͰ༻͍ͨจॻ ͷϕΫτϧԽʹ͍ͭͯ3.1અʹɼΫϥελϦϯάͷख๏ʹؔ͠ ͯ͸3.2અʹड़΂Δɽ3.3અͰ͸ΫϥελΛՄࢹԽͨ݁͠Ռͱ ߟ࡯Λࣔ͢ɽ

3.1 จॻͷϕΫτϧԽ

จॻͷϕΫτϧԽʹ͸TF-IDFͱword2vec[Mikolov 13]Λ ༻͍ͨɽword2vecʹΑΓಘΒΕͨ୯ޠ෼ࢄදݱͷIDFՃॏ ࿨Λ༻͍ͯΫϥελϦϯάΛߦͬͨɽ·ͨɼTF-IDFʹΑΓಘ ΒΕͨϕΫτϧΛจॻݕࡧʹ༻͍ͨɽҎԼʹTF-IDFͱ୯ޠ ෼ࢄදݱͷՃॏ࿨ʹ͍ͭͯͦΕͧΕࣔ͢ɽ 3.1.1 TF-IDF TF-IDFͱ͸ɼจॻதͷग़ݱස౓͕ߴ͘ɼଞͷจॻͰ͸ग़ݱ ස౓ͷ௿͍୯ޠ͕จॻΛಛ௃෇͚Δ୯ޠʹͳΔͱ͍͏Ծఆʹج ͖ͮॏΈ෇͚Λߦ͏ख๏Ͱ͋ΔɽTF-IDF͸ࣜ(1)ʹΑΓࢉग़ ͞ΕΔɽTF-IDFʹΑΓࢉग़ͨ͠஋Λ༻͍ͯޠኮ਺࣍ݩͷจॻ ϕΫτϧ͕֫ಘͰ͖Δɽ TFIDF(w, d) = TF(w, d)ɾ_IDF(w) (1) ͜͜ͰɼTF(w, d)͸จॻdͰͷ୯ޠwͷग़ݱස౓(Term Fre-quency; TF)Λද͠ɼIDF(w)͸୯ޠw͕ग़ݱͨ͠จॻͷස౓ ͷٯ਺(Inverse Document Frequency; IDF)Λද͢ɽC(w, d)

Λจॻdʹ͓͚Δ୯ޠwͷग़ݱස౓ͱ͠ɼ୯ޠwͷग़ݱ͢Δ จॻͷස౓(Document Frequency; DF)ΛDF(w)ɼจॻ௕Λ |d|ɼจॻ਺Λ|D|ͱ͢ΔͱɼͦΕͧΕҎԼͷࣜͰදݱ͞ΕΔɽ TF(w, d) = C(w, d)_|d| (2) IDF(w) = log( |D| DF(w)) + 1 (3) 3.1.2 ୯ޠ෼ࢄදݱͷՃॏ࿨ word2vecʹΑΓ୯ޠϕΫτϧΛ࡞੒͠ɼจʹग़ݱ͢Δ୯ޠ ͷϕΫτϧͷ૯࿨Λࢉग़͢Δɽ͜ͷͱ͖ɼ୯ޠͷIDF஋Λ܎ ਺ͱ֤ͯ͠୯ޠϕΫτϧʹ৐ࢉ͢Δ͜ͱʹΑΓɼॏཁͳ୯ޠ͕ ΑΓେ͖ͳӨڹΛ࣋ͭΑ͏ʹ͢Δɽࣜ(4)ʹ_i൪໨ͷจॻ_Di ͷϕΫτϧͷࢉग़ํ๏Λࣔ͢ɽ͜͜Ͱ_E(wj)͸୯ޠຒΊࠐΈ ʹΑΓ֫ಘͨ͠୯ޠ_wjͷϕΫτϧͱ͠ɼ_N͸จʹग़ݱ͢Δ୯ ޠ਺ͱ͢ΔɽຊݚڀͰ͸୯ޠͷ࣍ݩ਺Λ300ʹݻఆ͍ͯ͠Δɽ ͦͷͨΊɼจॻ͸300࣍ݩͷݻఆ௕ϕΫτϧͰද͞ΕΔɽ Di= N  j=1 (IDF(wj)ɾE(wj)) (4)

3.2 จॻΫϥελϦϯά

ຊ࣮ݧʹ༻͍ͨΫϥελϦϯάख๏ʹ͍ͭͯઆ໌͢Δɽ Stein-bachΒ[Steinbach 00]͸֊૚ܕΫϥελϦϯά΍k-means๏ ͳͲͷҰൠతͳจॻΫϥελϦϯάख๏ͷ࣮ݧ݁ՌΛࣔͯ͠ ͍ΔɽຊݚڀͰ͸ΫϥελϦϯάʹk-means๏Λ༻͍ͨɽਓ खͰ෼͚ΒΕͨάϧʔϓʹ͍ͭͯ1Ϋϥελʹ͖ͭ100จॻ ఔ౓ʹͳΔΑ͏ΫϥελϦϯάΛߦͬͨɽ͜ΕΒͷڭࢣͳ͠Ϋ ϥελϦϯά͸ɼϥϕϧ͕ͳ͘ͱ΋άϧʔϓԽ͕ՄೳͰ͋Δ͜ ͱ͕ར఺͕ͩɼҙਤͨ͠άϧʔϓ͕Ͱ͖͍ͯΔͱ͸ݶΒͳ͍ɽ Αͬͯɼਓ͕֤ؒΫϥελ͔ΒԿΒ͔ͷੑ࣭Λݟ͚ͭΔ͜ͱ͕ ॏཁʹͳΔɽ

3.3 ֤Ϋϥελͷಛ௃ޠͷՄࢹԽ

Ϋϥελͷಛ௃ޠΛTF-IDF஋ͷେ͖͞ʹج͖ͮϫʔυΫ ϥ΢υʹΑΔՄࢹԽΛߦͬͨɽຊݚڀͰ͸ෳ਺ͷಛ௃ޠͷՄࢹ ԽΛߦ͏͜ͱͰ֤Ϋϥελͷಛ௃Λ෼ੳͨ͠ɽਤ1ʹ֤άϧʔ ϓͷಛ௃ޠͷՄࢹԽΛߦͬͨ݁ՌͷҰ෦Λࣔ͢ɽΫϥελͷಛ ௃ޠ͕ਖ਼͘͠நग़͞Ε͍ͯΔ΋ͷ΋ݟΒΕ͕ͨɼҰݟ಺༰ʹࠩ ҟ͕ݟΒΕͳ͍Ϋϥελ΋ଘࡏͨ͠ɽ4.1અʹࣔ͢Ϋϥελਪ ఆ݁Ռ͔Βɼ֤Ϋϥελͷࠩҟʹؔ͢Δߟ࡯Λߦͬͨɽ

4. ݕࡧਫ਼౓ʹΑΔධՁ࣮ݧ

ຊݚڀͰఏҊ͢ΔݕࡧϞσϧͷ֓ཁΛਤ2ʹࣔ͢ɽຊϞσ ϧͰ͸ɼϢʔβ͕ΫϥελΛબ୒͢Δ͜ͱʹΑΓɼݕࡧൣғΛ ݶఆ͠จॻΛਖ਼͘͠ਪఆ͢Δ͜ͱ͕ՄೳʹͳΔͱߟ͑ΒΕΔɽ ͦ͜ͰɼϢʔβʹఏࣔ͢ΔΫϥελΛਖ਼͘͠ਪఆͰ͖Δ͔ɼΫ ϥελͷબ୒͕ݕࡧʹ༗ޮͰ͋Δ͔ͷ2఺Λݕূ͢Δඞཁ͕ ͋ΔɽΑͬͯɼຊষͰ͸ɼΫϥελͷਪఆʹؔ͢Δख๏ͱͦͷ

2

The 34th Annual Conference of the Japanese Society for Artificial Intelligence, 2020

ਤ 1: ֤ΫϥελͷϫʔυΫϥ΢υʹΑΔՄࢹԽ݁ՌͷҰ෦ (֤୯ޠͷେ͖͞͸TF-IDFʹجͮ͘ɽ·ͨɼਤதͷʮݪථʯ ͸ʮૹΓঢ়ʯͷ͜ͱΛࣔ͢ɽ) ਤ2: Ϋϥελͷਪఆ݁ՌΛ༻͍ͨݕࡧͷྲྀΕ(U͸Ԡ౴Ϛχϡ Ξϧͷจॻ܈ɼC͸ΫϥελɼD͸จॻɼQ͸ΫΤϦΛද͢ɽ D4Λਖ਼ղจॻͱͨ͠ͱ͖ɼΫΤϦ͔ΒྨࣅͷΫϥελΛఏࣔ ͠ɼબ୒͞ΕͨΫϥελͷத͔ΒจॻͷݕࡧΛߦ͏ɽ) ݁ՌΛࣔ͠ɼΫϥελ಺Ͱݕࡧ͢Δ͜ͱʹΑΔݕࡧਫ਼౓޲্΁ ͷ༗ޮੑΛ֬ೝ͢ΔධՁ࣮ݧʹ͍ͭͯड़΂Δɽ༗ޮੑΛ֬ೝ͢ Δʹ͋ͨΓɼҰൠతͳݕࡧγεςϜͱಉ͡શจ͔Βͷݕࡧͱɽ Ϋϥελͷਪఆ݁ՌΛ༻͍ͨݕࡧΛߦ͍ൺֱͨ͠ɽ 4.1અʹΫϥελͷਪఆख๏ʹ͍ͭͯઆ໌͠ɼ4.2અʹݕࡧ ख๏ɼ࠷ޙʹ4.3અͰ͸࣮ݧ݁ՌΛࣔ͠ɼͦͷߟ࡯Λߦ͏ɽ

4.1 Ϋϥελͷਪఆ

ຊ࣮ݧͰ͸ɼΫΤϦ͔Βਖ਼ղจॻ͕ଐ͢ΔΫϥελΛਪఆͰ ͖Δ͔Λݕূͨ͠ɽ୯ޠ෼ࢄදݱͷՃॏ࿨ʹΑΓ࡞੒͞ΕͨΫ ΤϦͷϕΫτϧΛೖྗͱ͠ɼ୯७ͳ3૚χϡʔϥϧωοτϫʔ ΫΛ༻͍ͯਫ਼౓ͷݕূΛߦͬͨɽͳ͓ɼग़ྗ͸֤ΫΤϦʹඥͮ ͘ਖ਼ղจॻͷΫϥελ൪߸ͱͨ͠ɽ·ͨɼͦͷࡍͷσʔλ਺Λ ද4ʹࣔ͢ɽΫϥελਪఆਫ਼౓ͷධՁ͸AccuracyΛ༻͍ͨɽ

4.2 จॻݕࡧख๏

จॻݕࡧ͸ϕΫτϧۭؒϞσϧʹج͍ͮͯߦͬͨɽΫΤϦͱ จॻΛͦΕͧΕϕΫτϧԽ͠ɼϕΫτϧಉ࢜ͷྨࣅ౓Λࢉग़͠ ͨɽWealΒ[Gomaa 13]͸ςΩετͷྨࣅੑʹؔ͢Δख๏Λ ·ͱΊ͍ͯΔɽྨࣅੑͷଌఆ͸จॻݕࡧʹ͓͍ͯ΋༻͍ΒΕɼ ຊݚڀͰ͸ɼྨࣅ౓ͷࢉग़ʹίαΠϯྨࣅ౓Λ༻͍ͨɽࣜ(5) ʹίαΠϯྨࣅ౓ͷࢉग़ํ๏Λࣔ͢ɽ͜͜Ͱ_x,y͸จॻϕΫ τϧͱ͢Δɽ ද4: ΫΤϦͷ෼ྨਪఆʹ༻͍ͨσʔλ਺ ༻్ σʔλ਺ ֶशσʔλ 12,009 ݕূσʔλ 1,715 ςετσʔλ 3,431 ද5: Ϋϥελਪఆͷਖ਼ղ཰ ਖ਼ղΫϥελͷग़ݱॱҐ ਖ਼ղ཰ ্Ґ1Ґ 66.2% ্Ґ3ҐҎ಺ 85.7% cos(x, y) = xɾy |x||y| = _n i=1xiyi _n i=1x2i _n i=1yi2 (5) จॻͷϕΫτϧԽ͸4.1અͱಉ༷ʹTF-IDFʹΑΓٻΊΔɽධ Ձʹ͸ද4ʹࣔͨ͠ςετ༻σʔλ3,431݅Λ༻͍ͨɽ 4.2.1 ධՁख๏ จॻݕࡧʹ͓͍ͯɼਖ਼ղจॻ্͕Ґʹग़ݱ͢Δ͔൱͔͸ॏཁ ͳࢦඪͱݴ͑ΔɽΑͬͯɼݕࡧ݁Ռ্Ґʹ͓͚Δద߹จॻ਺ͷ ૿ݮΛධՁ͢ΔͨΊʹɼ্Ґ10݅Λର৅ͱͨ͠࠶ݱ཰ͷฏۉ

(mean Recall; mRecall)Λ༻͍ͨɽ_Ci_{͸i൪໨ͷΫΤϦʹ͓} ͚Δਖ਼ղจॻ਺Λࣔ͠ɼ_Ri_{͸i൪໨ͷΫΤϦʹର͢Δ্Ґ10} ݅Ҏ಺ʹଘࡏ͢Δద߹จॻ਺Λࣔ͢ɽ mRecall = 1 |Q| |Q|  i=1 Recall(i) (6) Recall(i) = R i Ci (7) ·ͨɼਖ਼ղจॻ্͕Ґʹग़ݱ͢Δ΄Ͳݕࡧੑೳ͸ྑ͍ͱݴ͑Δ ͨΊɼฏۉద߹཰ͷฏۉ(Mean Average Precision; MAP)Λ ༻͍ͯධՁͨ͠ɽࣜ(8)͸ͦͷࣜΛࣔ͠ɼ͜Ε͸ࣜ(9)ʹΑͬ ͯٻΊͨ_i൪໨ͷΫΤϦͷAPͷ஋_AP(i)ΛશΫΤϦ਺_|Q| Ͱฏۉͨ͠΋ͷͱݴ͑Δɽͨͩ͠ɼࣜ(9)ͷ_O(rk)͸ਖ਼ղจ ॻ_rkͷॱҐΛද͢ɽ MAP = 1 |Q| |Q|  i=1 AP(i) (8) AP(i) = 1 Ci Ci  k=1 k O(rk) (9) ·ͨɼର࿩ܗࣜʹ͓͍ͯɼ্Ґ1Ґʹݕࡧ݁Ռ͕ݱΕΔ͜ͱ ͕ॏཁͰ͋ΔɽΑͬͯɼਖ਼ղจॻ্͕Ґ1Ґʹग़ݱ͢Δׂ߹Λ Precision@1ͱͯ͠ධՁࢦඪͱͨ͠ɽ

4.3 ݁Ռɾߟ࡯

4.3.1 Ϋϥελͷਪఆਫ਼౓ Ϋϥελͷਪఆʹؔ͢Δ࣮ݧ݁ՌΛ ද5ʹࣔ͢ɽϢʔβʹ ఏࣔ͢ΔΫϥελΛ3݅ఔ౓ͱԾఆ͠ɼ্Ґ3݅·Ͱʹਖ਼ղ ͷΫϥελͷਪఆ͕ߦ͑Δ͔Λ֬ೝͨ͠ɽ݁Ռ͔Β66.2%ͷ ΫΤϦ͸1Ґʹਖ਼͍͠ΫϥελΛਪఆͰ͖ɼ্Ґ3Ґ·Ͱʹ ͸85.7%ͷΫΤϦ͕ਖ਼͍͠ਪఆΛߦ͑Δ͜ͱΛ֬ೝͨ͠ɽ

3

The 34th Annual Conference of the Japanese Society for Artificial Intelligence, 2020

ਤ 3: ʮQ.ΩʔϘʔυͷࠨͱԼͷ໼ҹΩʔͷ൓Ԡ͕ѱ͍ʯʹ ͓͚ΔΫϥελͷਪఆ݁Ռ্Ґ3݅ͷϫʔυΫϥ΢υ(ׅހͷ த͸softmaxؔ਺ʹΑΔग़ྗͷ஋Λࣔ͢ɽ) ਤ4: ʮQ.PC͕উखʹىಈ͢Δʯʹ͓͚ΔΫϥελͷਪఆ݁ Ռ্Ґ3݅ͷϫʔυΫϥ΢υ(ׅހͷத͸softmaxؔ਺ʹΑ Δग़ྗͷ஋Λࣔ͢ɽࠨ୺ͷϫʔυΫϥ΢υ͕ΫΤϦʹ͓͚Δਖ਼ ղɼͦͷӈʹ͋Δ΋ͷ͕ਪఆ͞ΕͨΫϥελΛࣔ͢ɽ) 4.3.2 Ϋϥελͷਪఆʹؔ͢Δߟ࡯ ΫϥελਪఆΛߦͬͨ݁Ռɼ͍͔ͭ͘ͷΫΤϦ͸ਖ਼͘͠ਪఆ ͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨɽͦ͜ͰɼχϡʔϥϧωοτϫʔΫͷ ग़ྗͷ஋Λ֬ೝ͠ߟ࡯ΛߦͬͨɽχϡʔϥϧωοτϫʔΫͷग़ ྗ͸softmaxؔ਺Λ༻͍͓ͯΓɼͦͷ஋͸֤Ϋϥελʹॴଐ ͢Δ֬཰ͱݟͳ͢͜ͱ͕Ͱ͖Δɽਖ਼͘͠ਪఆ͕Ͱ͖͍ͯΔྫ Λਤ3ʹࣔ͢ɽ͜Ε͸ɼʮΩʔϘʔυͷࠨͱԼͷ໼ҹΩʔͷ൓ Ԡ͕ѱ͍ʯͱ͍͏ΫΤϦʹର͢ΔΫϥελͷਪఆ݁ՌͰ͋Δɽ ༗ࣝऀ͔Βɼग़ྗ݁Ռͷ1Ґͱ2ҐͷΫϥελ͸ɼPCؔ࿈ͷ ʮނো/อकʯͱʮૢ࡞ํ๏ʯΛࣔ͢ͱ͍͏ࠩҟ͕͋Δͱͷҙݟ ͕ಘΒΕͨɽ͜Ε͸ɼ1ҐͷΫϥελʹݟΒΕΔʮอकʯ΍ɼ 2ҐͷΫϥελʹݟΒΕΔʮ͔ͳೖྗʯͳͲͷ୯ޠ͔Β΋ਪଌ ͞ΕΔɽೖྗ͞ΕͨΫΤϦ͸ނোʹؔ͢Δ಺༰ͱͳ͓ͬͯΓɼ ਖ਼͘͠ਪఆͰ͖ͨͱߟ͑ΒΕΔɽͨͩ͠ɼҰݟྨࣅͨ͠Ωʔ ϫʔυ͕ฒΜͰ͓Γɼۀ຿ʹର͢Δཧղ͕ͳ͍৔߹ʹ͸Ϋϥε λͷ൑ผ͸ࠔ೉ʹ΋ݟ͑Δɽ Ϋϥελͷਪఆ͕Ͱ͖ͳ͔ͬͨྫΛਤ4ʹࣔ͢ɽਤ4ʹ͓͚ Δग़ྗ1Ґ͸ਤ3ʹ͓͚Δग़ྗ1Ґͱಉ༷ͱͳ͓ͬͯΓɼʮނ ো/อकʯʹؔ͢Δจॻ܈Λࣔ͢ɽΫΤϦͷ಺༰Λނোͱଊ͑ Δͱਪఆ݁Ռ͸ؒҧ͍ͱ͸ݴ͍੾Εͳ͍͕ɼਖ਼ղͷΫϥελ͸ ҟͳΔɽֶशσʔλͷภΓ͕͋ͬͨͱ΋ߟ͑ΒΕΔ͕ɼਖ਼ղͷ Ϋϥελͷਪఆ͕Ͱ͖ͳ͔ͬͨ͜ͱ͔ΒɼΫϥελϦϯάͷଥ ౰ੑʹ͸ٙ໰͕࢒Δɽ 4.3.3 ֤ݕࡧख๏ʹΑΔݕࡧਫ਼౓ ຊݚڀʹ͓͚Δ2໰2౴ͷ࿮૊ΈͰ͸ɼఏࣔͨ͠Ϋϥελ ΛϢʔβʹબ୒ͤ͞Δաఔ͕ଘࡏ͢Δɽຊ࣮ݧͰ͸ɼΫϥελ ͷਪఆ݁Ռ͔Β1Ґʹਪఆ͞ΕͨΫϥελ͕બ୒͞Εͨ৔߹ (Ծఆ1)ͱɼ3Ґ·ͰΛߟྀͦ͠ͷத͔Βਖ਼ղͷΫϥελ͕બ ୒͞Εͨ৔߹(Ծఆ2)ͷݕࡧਫ਼౓ΛධՁͨ͠ɽՃ͑ͯɼݕࡧ ൣғͷߜΓࠐΈΛߦΘͳ͍શจݕࡧͷਫ਼౓ͷൺֱΛߦͬͨɽ ݕࡧਫ਼౓ʹؔ͢Δ࣮ݧ݁ՌΛ ද6ʹࣔ͢ɽԾఆ2ͷ৔߹͕ ֤ࢦඪͰ࠷΋ߴ͍஋Λࣔͨ͠ɽΑͬͯɼϢʔβ͕ਖ਼͍͠Ϋϥε λͷબ୒Λߦ͏͜ͱ͕Ͱ͖Ε͹ɼݕࡧਫ਼౓ͷ޲্ʹܨ͕Δͱ ظ଴Ͱ͖Δɽ·ͨɼԾఆ1ͷmRecall͕࠷΋௿͍஋Λࣔͨ͠ɽ Ϋϥελਪఆͷ݁ՌΛ༻͍ͯݕࡧΛߦ͏࣌ɼΫϥελਪఆͷஈ ද6: ݕࡧਫ਼౓ʹؔ͢Δ࣮ݧ݁Ռ(Ϋϥελબ୒͸બ୒͞Εͨ Ϋϥελ಺ͰͷݕࡧΛࢦ͢ɽ·ͨɼ@Ҏ߱ͷ਺ࣈ͸ݕࡧ݁Ռͷ ্ҐԿ݅·ͰΛධՁର৅ͱ͢Δ͔Λࣔ͢ɽ)

ख๏ _{mRecall@10 MAP@10 Precision@1}

શจݕࡧ 0.630 0.366 0.257 Ϋϥελબ୒ (Ծఆ 1) 0.589 0.383 0.286 Ϋϥελબ୒ (Ծఆ 2) 0.769 0.510 0.387 ֊Ͱਖ਼ղ͕ಘΒΕͳ͍৔߹͸ɼਖ਼ղจॻ͕ݕࡧ݁Ռʹग़ݱ͢Δ ͜ͱ͸ͳ͘ɼ࠷ऴతͳݕࡧਫ਼౓ʹେ͖ͳӨڹΛٴ΅͢ͱߟ͑Β ΕΔɽMAP΍Precision@1ʹ͓͍ͯ͸ɼશจݕࡧΑΓΫϥε λબ୒ʹΑΔݕࡧ͕ߴ͍஋Λࣔͨ͠ɽΑͬͯɼΫϥελͷબ୒ ʹΑΔݕࡧൣғ͕ߜΓࠐΈ͕༗ޮʹಇ͘Մೳੑ͕͋Δɽ

5. ·ͱΊ

ຊݚڀͰ͸จॻݕࡧʹ͓͚Δ2໰2౴ͷ࿮૊Έʹର͢Δݕ ূΛߦͬͨɽ࠷ॳʹจॻݕࡧʹ͓͍࣮ͯࡍʹ࢖༻͞Ε͍ͯΔ ίʔϧηϯλʔͷϔϧϓσεΫʹ͓͚ΔԠ౴ϚχϡΞϧͷ෼ੳ Λߦ͍ɼͦͷޙ࣭໰Ԡ౴ίʔύεΛ༻͍ͨݕࡧਫ਼౓ͷݕূΛ ߦͬͨɽ·ͣɼԠ౴ϚχϡΞϧͷ෼ੳΛߦ͏ͨΊɼจॻΫϥε λϦϯάͷ݁ՌΛϫʔυΫϥ΢υʹΑΔಛ௃ޠͷՄࢹԽΛߦͬ ͨɽ࣍ʹɼΫϥελϦϯά݁ՌΛ༻͍ͨݕࡧਫ਼౓ͷݕূΛߦͬ ͨɽΫΤϦ͔ΒΫϥελΛਪఆ͢Δͱ͍͏աఔΛ͸͞Ή͜ͱʹ ΑΓɼ௨ৗͷશจॻݕࡧΑΓ΋ਖ਼ղจॻΛ্Ґ1Ґʹఏࣔ͢ Δ͜ͱ͕ظ଴Ͱ͖Δɽ

6. ࠓޙͷ՝୊

ຊ࣮ݧͰ͸٬؍ධՁʹΑͬͯ༗ޮੑͷݕূΛߦ͕ͬͨɼ͜Ε ͕࣮ࡍͷݱ৔Ͱ༗༻͔͸ݕূͰ͖͍ͯͳ͍ɽಛʹΫϥελͷબ ୒ʹ͓͍ͯ͸ਓ͕ؒ༰қʹద੾ͳΫϥελΛબ୒͕Ͱ͖Δͱ͸ ݶΒͣɼϢʔβ΁ͷΞϯέʔτͳͲΛ༻͍ͯओ؍ධՁΛߦ͏ඞ ཁ͕͋Δɽ·ͨɼ݁ՌΛݟΔͱશจݕࡧͱൺֱͯ͠ಛఆͷਫ਼౓ ͷ޲্͕ߦ͍͑ͯΔ΋ͷͷɼ࣮༻্े෼ͳݕࡧਫ਼౓ͱ͸ݴ͑ͳ ͍ɽຊ࣮ݧͰ͸TF-IDF΍word2vecͱ͍͏ݹయతͳख๏Λ ༻͍ͯจॻͷϕΫτϧԽΛߦ͕ͬͨɼۙ೥Ͱ͸χϡʔϥϧωο τϫʔΫΛ༻͍ͨϕΫτϧԽɼݕࡧ͕Α͘ߦΘΕ͓ͯΓߴ͍ਫ਼ ౓͕֬ೝ͞Ε͍ͯΔɽͦΕΒͷٕज़Λ༻͍ͯਫ਼౓ͷ޲্ΛਤΔ ͱͱ΋ʹɼຊ࿮૊Έ͕༗ޮͰ͋Δ͔ͷݕূΛߦ͏ඞཁ͕͋Δɽ

ࢀߟจݙ

[Mikolov 13] Tomas MikolovɼIlya SutskeverɼKai Chenɼ

Greg S CorradoɼJeff Dean : Distributed representa-tions of words and phrases and their compositionalityɼ

In Advances in Neural Information Processing Systems 26ɼpp.3111-3119ɼ2013

[Steinbach 00] Michael SteinbachɼGeorge KarypisɼVipin Kumar : A Comparison of Document Clustering Tech-niquesɼIn KDD-Workshop on Text Miningɼ2000 [Gomaa 13] Weal H. GomaaɼAly A. Fahmy : A survey

of text similarity approachesɼInternational Journal of Computer Applications, vol.68, noɽ13ɼpp.13-18ɼ

2013

4

The 34th Annual Conference of the Japanese Society for Artificial Intelligence, 2020