Hand Signal Recognition Aiming at Robot Control 利用統計を見る

(1)

4１

論文OImginalPapeNP

HandSignalRecognitionAimingatRobotControl

YukihiroMIYosHI

Abstrnct:Amethodtorecognizehandshape,i､e､,numbersoffingersstuckout;usingacolorimageispresent-

ed･Ameananddispersionofhuevaluesandameanofsaturationvalues,applyingtheHSLcolormodel,ina smalIsquareareaofanimagearerevealedtobeusefulfeaturestodistinguishskin-areaandcontourofthehand，

ThecontourofthehandistracedbytheborderfollowingmethodusingthesmaI1squarearea・Changesoftan-

gentialdirectionsofthecontourofthehandareusedtoidentifythenumberoffingersstuckout・Thewavelet shrinkageisusedunclosethefeaturepattemsofthetangentialdirectionsandmakeanautomatichand-recogni‐

tionpossible・Althoughonlyafewimagesevaluate｡，numbersofthelingerswererecognizedautomatically．

squarearea・ＣｈａｇｔｈｅｓｍａＩ１ｓ

ngersstuckout．

entifythe

Keywords:colorimagerecognition，ｍｅａｎｏｆｈｕｅ，dispersionofhue，skin-area，ｈａｎｄrecognition，wavelet shrinkage

惣「－１行

酢

１．IIltroductionn

１ｊｐＥ

Recognitionofhandsignalsusingdigitalimageprocess- ingisexpectedasamethodforhuman-computer interface1)andamethodtoinstructarobotonawork2〕.’、

BBBmL6～ロﾛ■

ﾛﾛ

■■

thispaperamethodtorecognizehandposture,especiaUy anumberoflingersstuckout，isdevelopedColorinfOr‐

mation,ｅ､9.,RGBvaluesandHSLvaluesofpixeIsinclude vari2tionandthedirectuseofthosevaluestendstoresult

paperamethodtoreco

mberoffingersstuck Ａこ■

一一

^’

inanunstablerecognition・Ａｍｅａｎａｎｄｓｏｍｅｋｉｎｄｏｆｄｉｓ‐

ゴ

persionareadequateasthecuetodetecthandareainan image・Inordertorealizeanaturalsignaling,neitherusage ofthegIovenorsimplecolorbackgroundsaresupposed forthemethodinthisDaDer・ＡｎｄａｉｍｉｎＲａｔｔｈｅｈｕｍａｎ‐paper・Ａｎｄ iｍｉｎｇａｔｔｈｅ Tobotcommunication，time-consumingandsophisticated

caIcuIationlikeanartificialneuralnetworkisnotsup- Fig.１lmageshowingahandandabackgroun。

posed．

2．Skilnareacognitioningmimage 250 skin

Ｆｉｇ．１ｉｓａ６４０ｘ４８０２４－ｂｉｔＲＧＢｃｏｌｏｒｉｍａｇeshowinga handandabackground・Thisimageiscapturedusinga colorvideocameraunderHuorescentceilinglightsina roomFig・ＺｓｈｏｗｓＲ，０，Bvaluesofpixelsintherectan‐

２

１ＤＥ

１

ｇｌｉ kiｎ

gularareaA，ｉ､ｅ，theskinarea，andintherectangular areaB,ｉ・ａ，thenon-skinarea,oftheimageshowninFig．

1．IntegernumbersbetweenOand255representsred，

greenandblueintensitiesofthepixels・Thepixelsinthe skinareaAandthoseinthenon-skinareaBoccupydiiTer- entRGBspace,asfarasfortheimageinFigl，soclassifi‐

cationintotwoclasses，ｉ,e､，askinareaandanon-skin xels・Ｔｈｅ

area,ispossibleusingsomesortofclassifier,ｅ,9.,aneural network3).Itisdesirable,however,thatfeaturestorecog‐

nizetheskinareaareintuitivelyunderstandable．ＨｕｅｉｓＦｉｇ,ZRGBintensityofpixelsintheskiｎａｒｅａＡａｎｄｔｈｅｎｏｎ－

ｓｋｉｎａｒｅａＢ oneofthosefeatures．

bｅｍａｐｐｅｄｔｏｈ

Ｒ，０，Ｂｖａｌｕｅｓｏｆｐｉｘｅｌｓｃａｎ ^{ue，ｓａｔｕｒａ－}

tionandIightnesscomponentsreferringtothedoublehex- coneHSLcolormodeI4)．Hue，saturationandlightness

(2)

TransactionsoftheKokushikanUniv・FacultyofEngineering・No.４０（2007）

4２

作|蝋琴二

＄|蝉二二二Ⅱ

small；（３）saturationvaluesforaskinareaarｅｎｏｔｓｏｓｍａｌＬＨｕｅａｌｏｎｅ，however，isinsufTicienttorecognizethe skinbecauseoftheoverlapintherangｅｂｅｔｗｅｅｎｌ２５ａｎｄ

ｌ５５・

Threefeatures(ameanofhuevalues,Ｈｈ,,dispersionof huevalues,Ｈａ,andameanofsaturationvalues,Ｓｍ,ina squarearea）canbeusedtorealizeabove-mentioned strategy・ＴｈｅＨｈ１ａｎｄｔｈｅＳｍａｒｅｅｓｔｉｍａｔｅｄａｓ：

Ｈｈ,＝ＬＺＨ『j，（４）

_〃ｘ,ｙ

ｓ,"＝-LzsxI，（５）

_〃ｘ,y

DispersionofhuevaluesiscalculatedapplyingtheMan- hattandistanceasfollows：

Ｈａ=上ｚ'Ｈ)"－Ｈｋｊ,’（６）

_〃ｘ,y

HXyandSⅢ｡,arethehuevalueandthesaturationvalueof

1４０＋

ｏＳｋｉｎａｒｅａ

＋Non-skinarea

＋

加川００００

１１８６４２

雫への①。｜⑩ンのゴー」。□○両」のＱの一□

＋＋

＋＋＋＋＋

+tFiﾆﾄ＋サキ＋

＋＋＋＋＋＋

＋＋鳶＋什土＋＋農

讃＋革鍼

＋＋

0

２０６０１００１４０１８０Ｍｅａｎｓｏｆｈｕｅｖａｌｕｅｓﾉｳﾙﾌ

Fig.４Meansanddispersionofthehuevaluｅｓｉｎｔｈｅｓｋｉｎａｒｅａａｎｄｔｈｅｎｏｎ－ｓｋｉｎａｒｅａ

500 500

400 400

の

Ｅ３００ｏｊ凸四

ji2OO

コ

のＥ３００ｏ

温_巴

ji2OO

。

1００ 1００

0 １００２００３０００１００２００

ＨｕｅＨＨｕｅＨ

（a)Ｓｋｉｎａｒｅａ（b)Non-skinarea

Fig、３HueandsaturationvaluesfortheskiｎａｒｅａＡａｎｄｔｈｅｎｏｎ－ｓｋｉｎａｒｅａＢ 300

(3)

HandSignalRecognitionAimingatRobotControl ^4３

thepixelofthecoordmatex,Ｊａｎ。〃isthenumberofpix- elsintherelevantsquarearea･Fig.４showsthemean,Ｈｈ,，

S,,,．Ｉｆｔｈｅａｒｅａａｔｔｈｅｃｅｎｔｅｒｉｓａｓｋｉｎａｒｅａ，then，a20x20 squareareaneighboringtoanon-skinbackgroundissear- chedwhilethesquareareaismovedfromthecentertoleft direction・Ｉｆｎｏｔ,arasterscanoperationisusedtoIinｄｔｈｅ anddispersion,Ｌｂ,ofthehuevaluescalculatedusingpix‐

ｅＩｓｉｎｌＯ×lOsquareareaswhicharedefinedbysectioning

cIosetheskinareaAanｄｔｈｅｎｏｎ－ｓｋｉｎａｒｅａＢＦｉＥ.４rev宮ａＡａｎｄｔｈｅｎｏｎ－ｓｋｉｎａｒｅａＢ、Fig.４ｒｅ boundarysquareskinarea・Ｆｉｇ．５illustratesthefirstcase：

acemerareaAisjudgedasaskinareaandaboundary squareareaBisfoundwhiletheareaismovedIeftfromA，

Acontourofthehandistracedbyusingsquareareasof lOx10pixelsstartingatthesquareareaBofFi9.5．Eight eaIsthataskinareacanbedistinguishedfromanon-skin

areabyusingbｏｔｈＨｈ,andHlﾉ:therelevantareaofaimage ｃａｎｂｅｒｅｇａｒｄｃｄａｓａｓｋｉｎａｒｅａｗｈｅｎＨｍｉｓｉｎａｎａｒｒｏｗｒａｎｇｅａｎｄ秘isverysmalLInwhatfolIows,ａｎａｒｅａｈａｖ－

ｉｎｇａＨ,Ivaluebetweenl20andl60,aHhvaluelessthan lOandaS,,ＪｖａＩｕｅｇｒｅａｔｅｒｔｈａｎ５０ｉｓｔａｋｅｎａｓａｓkinarea，

ie.，ａｎａｒｅａｏｎａｈａｎｄ．

lOxlOneighborsquareareas(horizontaLverticalanddi‐

agonaldirectneighborareasofthecurrentlOx10boun- darysquarearea)areinvestigatedbasedontheHh,,Hband S,,ltofindnextboundarysquarearea，Thenthecurrent boundaryareaisupdatcd、Theseoperationsareiterated

naIdirectn ,ｖａｌｕｅ

3．IIzmdcontourreCognitim

Ahandcontourisrecognizedfromacapturedimagein twosteps:（１）findasquareareaonahand;(2)tracethe contourmovingsquareskinareasthatareboundedby non-skinbackgroundarea・

Atthefirst，ａｓｑｕａｒｅａｒｅａｏｆ２０ｘ２０ｐｉｘｅｌｓａｔｔｈｅcenter

untilthecurrentboundaryareacomｅｓｂａｃｋｔｏｔｈｅｓｑｕａｒｅａｒｅａＢｏｆＦｉ９．５．Fig.６illustratestheresultantboundary lOx10squａｒｅａｒｅａｓｆｏｒｔｈｅｓａｍｅｉｍａｇｅａｓＦｉ９．５．Almost thecorrectboundaryisdetected．

4．HzmdsignrecognitioM ｏｆｔｈｅｉｍａｇｅｉｓｉｎｓｐｅｃｔｅｄｂｙｕｓｉｎｇｔｈeaforementioned

criteriabasedonthemeanofhuevaIues,Ｈ>,１，thedisper- sionofhuevalues,ＨＪ,andthemeanofsaturationvalues，

lnordertoidemifyshapeofthehand,especialIyanum- beroffingersstuckout，twoparametersareinvestigated・

Oneisatangentialdirection，Ｔ，ｏｆｔｈｅｃｏｎｔｏｕｒｏｆｔｈｅ

［一認

.■印を…ｂＬ

．厩１

，６－^可

］■

’

^－ ^－迅少

squarearea magecenter

■

F山１９

＿=－＝毎

￣￣

､〈

background

lOx10squareareas fOrmingacontourofthehand

Fig.５ SquaTeareaatcenterrecognized areaneighboringabackground

aｓｓｋｉｎａｎｄａｓｑｕａＴｃ

Fig.６ＢｏｕｎｄａｒｙｌＯｘ１０ｓｑｕａｒｅａｒｅａｓｏｆｔｈｅｈａｎｄ

（b）

Fig.７Examplesofhandimages (a）

頁

…聖 :=二三鑿蕊 _showinE

ontour

篝

￣？

二

^』^～

^黒

^白リヒ型

^富

^山－^－^、ご－^印 ^解司一

鱒

^■■

(4)

TransactionsoftheKokushikanUniv､FacultyofEngineering・No.４０（2007）

4４

300 ３００

０２００ト

１００

。』 200

1００

２０４０ 6０ Numbersofboundarysquares

（a)ＴｖａＩｕｅｓｏｆＦｉｇ６

２０４０ 0

Numbersofboundarysquares

（a)ＴｖａｌｕｅｓｏｆＦｉｇ６

6０

300 200 200 100

◎

ト１００

Ｏト

０２０４０６０８０

（b)ＴｖａｌｕｅｓｏｆＦｉｇ７(a）

1００１２０ 2０４０６０８０１００

（b)ＴｖａｌｕｅｓｏｆＦｉｇ７(a）

300 ３００

２００

○

ト１００

200

、

ト１００

０４０８０１２０１６０

Numbersofboundamsquares

（c)ＴｖａｌｕｅｓｏｆＦｉｇ７(b）

Fig.９Tangentialdirection7softhehandcontoursafterthe waveletshrinkage

０４０８０１２０１６０ Numbersofboundarysquares

（c)ＴｖａＩｕｅｓｏｆＦｉｇ７(b）

Fig.８Tangentialdirectionsofthehandcontours

TablelResultsoftheautomaticpatternrecognitionoftheimages NumbersoftheconvexesM

inthetangentialdirections Ｒａｔｉｏｒ：thecontouragainst theboundingboｘ

Ｏ７８

Recognizednumberofthefingers Images

Fig.６ Fig.７(a）

Fig.７(b）

１ Oｏｒｌ

２５

２ 1.03

５ 1.33

handThetangentialdirection,nisoneoftheeight(two horizontal,ｔｗｏverticalandfourdiagonaDdirectionsof thecurrentboundarysquare：Ｔ＝0.,90.,180．ａｎｄ２７０ｏｍｅａｎｘ，－ｙ，－xandydirectionofthecoordinateshown inFi９．６，respectively・Tisdecidedbythepositionalrela- tionamongtheformerboundarysquare，nextboundary squareandthenon-skinsquareintheeightneighbor squaresaroundthecurrentboundarysquare.Ａnotherisa ratiordehnedasfollows：

〃＝－〃ｃ（７）

〃Ｂ

where〃ｃａｎｄ〃ＢａｒｅｎｕｍｂｅｒｓｏｆｔｈｅｌＯ×１０squaresonthe contourofthehandandontheboundingboxrespectively・

Ｔｈｅｒａｔｉｏｒｉｓｌｅｓｓｔｈａｎｌｗｈｅｎｔｈｅｃｏｎｔｏｕｒｉｓａｎｅｌｌipse・

Ｆｉｇ．７ｓｈｏｗｓｔｗｏｅｘａｍｐｌｅｓｏｆｈａｎｄｉｍａｇｅｓ・Thecon- toursshown，onlyforillustrations，ｉｎＦｉｇ．７ｂｙｄｏｔｓａｒｅｄｅｔｅｒｍｉｎｅｄｂａｓｅｄｏｎｔｈｅｐositionalrelationsbetweenthe boundarysquareareasandthenon-skinbackground

ａｒｅａｓ・

Ｆｉｇ．８showsthetangentialdirections，７ｓ，ｏｆｔｈｅｃｏｎ‐

toursofthesehandsinFig6,Fig.７(a)ａｎｄFig.７(b).A changeoftangentialdirection,Tbover90゜meansaconvex oraconcave・Thechangesofthetangentialdirectionsvary bytheoutwardformofthehands，sousingthetangential directionsofthecontourscanrecognizetheshapesofthe hands：thenumberofthefingersstuckout，ＴｈｅｃｈａｎｇｅｓｓｈｏｗｎｉｎＦｉｇ８，however，exhibitvariationstoughto recognizeautomatically，Thewaveletshrinkageusingthe Haarwaveletisusedtounclosethefeaturepatterns・Fig9 showsthetangentialdirectionsafterthewaveletshrinkage byusingthescalingsequenceoflevel3・Tablelshowsthe resultsoftheautomaticpatternrecognition；ｔｈａｔｉｓ，the numberoftheconvexes,Ｍ,ｏｆＦｉｇ９ａｎｄｔｈｅｒａｔｉｏ,ｒ・One convexinFig9meanstwochangesofthetangentialdirec- tions：therecognizedhandisregardedashavingashape likeellipseincludingashapewithonefingerstuckout．

(5)

HandSignalRecognitionAimingatRobotControl ^4５

TwoconvexesinFig、９meansashapewithtwofingers、

５．Conclusion

ically．

References Amethodtorecognizehandshape,ｉ､e､，numbersoffin-

gers，ｕｓｉｎｇａｃｏｌｏｒｉｍａｇｅｉｓｐｒｅｓｅｎｔｅｄｉｎｔhispaper・Skin areasinanimagearedistinguishedfromanon-skinarea byvaluatingthreefeatures：ameananddispersionofhue valuesandameanofsaturationvalues，applyingtheHSL colormodel,iｎａｓｍａｌｌｓｑｕａｒｅａｒｅａｏｆａｎｉｍａｇｅａｒｅｃａｌcu- latedforskin-recognition、Movingthesmallsquarearea onacapturedimageandvaluatingthethreefeaturescoor- dinatesandtangentialdirectionsofacontourofahand areestimated、ChangesofthetangentialdirectioｎｓｏｆａｃｏｎｔｏｕｒｒｅＨｅｃｔａｓｈａｐｅｏｆｔｈｅｈand，thaｔｉｓ，anumberof fingersstuckout・Althoughonlyafewimagesevaluate｡，

numbersofthefingersstuckoutwererecognizedautomat‐

1）Ｊ・IsaacsandSFoo,OptimizedWaveletHandPoseEstima‐

tionforAmericanSignLanguageReCognition，Proceedings ofthe2004CongressonEvolutionaryComputationVol､１，

７９７－８０２．

２）Ｎ・KawarazakietaL,CooperativeWorkSystembetweenHu‐

manandManipulatorusingHandGestureInstructions(iｎＪapanese),JournaloftheRoboticsSocietyofJapan,ＶＯＬ23, No.６(2005)，121-126.

3）Ｍ､AAkbariandM、Nakajima,ANewSkinDetectorfor DifTerentColorSpaces,TechnicalreportoflEICE,ＩＥ２００４－

１４３，１１１－１１５．

４）Ｍ・ＴＡＫＡＧＬＨＳＨＩＭＯＤＡ（Eds.)，Handbookoflmage Analysis（ｉｎJapanese)，UniversityofTokyoPress，（2004)，

１１９１－１１９３．

Hand Signal Recognition Aiming at Robot Control 利用統計を見る

論文OImginalPapeNP