There is an enduring misconception about the r01e of nonverbal behavior in
communication attributing it with more than 90%ofthe message. Although it has
become an entrenched belief in popular culture, this idea has been debunked by
Beattie(2004)who pointed out that it is a misapplication of the results of some
classic research in psych010gy that investigated how facial expression afTects the
interpretation of attitudes (Mehrabian&Fe吊s,1967; Mehfabian &Wiener,1967)・
Still,it is important for native speakers*and language leamers alike to properly
interpret intention fbr speech acts suchas re恒saIs.ln the case ofJapanese refilsa1s,
for example, research has shown that leamers of Japanese as a second language
(JSL)perform refusaIs ofan offer and refusals ofan invitation somewhat differently
斤om Japanese native speakers (Jungheim,2004),and JSL leamers are not as good
at interpreting Japanese re血saIsasnative speakers when only the visual channel is
available(Jungheim,2006,2008)・
Previous research suggests that thereare not any obligatory contexts for
including gestures with refllsals (Jungheim,2009b),although anecdotal evidence
f10m this researcher s experience in Japan and 介om some participants in the present
study does suggest that gestures may serve to clarify the intention to refiIse.
Evidence from research on English as a fi)reign language (EFL)leamers shows
how gestures and fllcial eχpressions can aid listening comprehension (Sueyoshi&
ZI
The interaction of language and nonverbal behavior innuencing the perception of Japanese refusals
Nicholas O. Jungheim
こ二六
Hardison,2005).Finally,according to Calbris (1990), IFacial expression, positive
or negative, plays an even more positive role for foreign subjects whenever it lifts
theambiguityofanisolatedgesture,orwhenitcorroboratestheinformationcarried
by the gesture (p.36)・
ln light ofthe above, the purpose ofthis study is to examine how language and
nonverbal behavior interact to influence the perception of Japanese renlsals.
Combinationsof verbal negation, gesture, and facial eχpression will be used to
invelstigatel what is the most optimal way tol co lmunicate a sleriousl inltention to
refilse。
BaCkgrOund
Gestures have long been considered an integral component of language
leamers communicative competence (Canale,1983; Canale &Swain,1980; Celce−
Murcia,2008; Jungheim, 1995, 2001),Gestures are included in strategic and
sociolinguist competence (Canale,1983; Canale &Swain,1980).Celce−Murcia゛s
(2008)reeonsideration ofc01nnlunicative competence includes both the use of
speech acts and speech act sets along with gestures under interactionaI competence,
although it is not clear whethcr faciaI cχpressions are included in the category of
gestures as they are by some researchers who call them facial gestures. Finally,
Jungheim(1995,2001)provided a framework for evaluating leamers nonverbal
ability including gestures, head nods, and gaze direction under textual ability,
sociolinguistic ability,and strategic ability parallel to categories of Bachman s
(1990)Communicative Language Ability・
A number of problems related to nonverbal ability are also pertinent to the
performance of speech acts. For eχample, nonverbal behavior can afTbct the
evaluation of a lcamer゛s proficiency (Gullberg,1998; Jungheim, 2001; Nell, 1990;
Yamashita,1996).Neu(1990)found that a Japanese leamer of English received a
10wer rating for his oral proficiency than he deserved due to the nonverbal behavior
ア2
二二五
Theintera(ion oflanguage and nonvert)albehaviorinluencingtheperceplionofJapaneserefusals
accompanying his speech. Rat㎡s in Yamashita (1996)noted that JSL leamers
nonverbal behavior affbcted their ratings. ln an analysis of videos used in that
study, Jungheim (2004)fk)und that JSL learners used excessive bowing with their
re血sals of an invitation. This was related to the crosslinguistic influence of an
Englishre血sal formula that begins with an apology. lt is appropriate to bow when
saying jg朋加鉛印(I m sorry)in Japanese, but in Japanese refusaIs it usually
fbllows the reason for refusing, and it is optiona1. Jungheim(2004)also noted that
when JSL leamers refllsed an oar,they often used larger gestures with hand shapes
more indicative ofan English refilsal gestufe called the hand wag by Morris (1994)
that diflbrs fiom the so−called Hand Fan (Mo㎡s,1994)found only in Japan
(Jungheim,2006).
Although this study will fk)cus on only one gesture and two fllcial eχpressions
in relation to a re血sal of an oflbr, the above description of problems related to
language learners illustrates the importance ofthe appropriate use and performance
of gestures accompanying speech. The fbIIowing section is a review of issues
related to negation, gestures, and facial eχpression.
From Verbal to Nonverbal
Refusing in any language is not a simple act of saying no. ln face−to−face
interaction it includes a compleχ Γelationship among language, gesture, facial
expression,and even head movement such as bowing and head tilting as in the case
ofJapanese(Jungheim,2004),aII ofwhich are 助 ther mediated by the relationship
of the interlocutors.
The verbal side of refusals
This study will use only one allegedly simple Japanese refuse,kekkou desu.
When this researcher was a JSL student in loky0 40 years ago, his teacher made
only a cursory attempt to teach re血saIs.ln spite of this single mention in class, it
left a strong impression. ln Japanese the teacher explained that refilsa1s in Japanese
ア3
二二四
are very dimcult, sojustsaykekkou desu.W le this may work in some situations,
it would be somewhat rude to say if someone gives you an invitation, for eχample,
when it is more 叩propriate to give an eχcuse than to use a very direct re毎sallike
kekkou desu,even though the eχpression itsclfis p011te。
This eχpression is also quite ambiguous. Tiken alone, it can either be
interpreted as an acceptance or a refusal. A simple look at translations provided in
a Japanese−English diclionary can confirm this. Shogakukan s Japanese−English
dictionary(Kondo&Takan0,1986)translates it variously as ¨good;nice¨and゛NO,
涜α戒ya (p. 536).This ambiguity has been co㎡irmed thfough the researcher s
own experience. χVhen buying an θろa71θ(boχedlunch)at a convenience store, the
clerk asks ifyou would like some chopsticks. A number oftimes I said kekkou desu
and the clerk still put chopsticks in the bag. l neither gestured nor included the
verbal negation i泌.The clerk misinterpreted my intention. 0ne participant in the
current study further confirmed this ambiguity 丘om his own eχperience working in
a convenience store where he sometimes inadvertently put some chopsticks in a
bag because he misunderstood the customer s intention。
A110ther example of this kind of ambiguity is the use oldaりoubu,which can
a1so mean all right` (Kondo&T11kan0,1986,p. 997).Some participants in
Jungheim(2008)used the expression daUoubu when interpreting silent video clips
ofpeople re血sing given only visual clues to their meaning. M41en videos of these
interpretations were shown to participants in a subsequent study on obligatory
contexts for gestufes (Jungheim,2009b),some ofthose participants stated that they
were not sure ifthe person in the video meant to re恒se when they said jαびθ詞,z4。
1n the final analysis,sonle nleans are necessary to help disambiguate these
re恒saIs,Verbal negation such as j泌,gesture,and fllcial eχPression may contribute
to clari抄ing speakers intentions.
一
一一
一 一 一一 一
ア4
Theinteraction oflanguage and nonvert)albehaviorinluencing the percepljon of Japanese refusals
The gestural side of refusaIs
Gestures and speech may be two different communication channeIs, butthey
are part of the same psycho10gical process (McNeiII,1992,2005).Gesture and
speech unf01d simultaneously, bound to each other in time¨(McNeill,1992,p・
23).This linkage aIso includes semantic and pragmatic coeχpressiveness. ln other
words,co−occurring gestures and speech eχpress the same meaning and have the
same pragmatic fimction.
According to McNeill (1992),gestures can be iconics, metaphorics,beats,
cohesives,0r deictics. lconic gestures are cIosely related to the semantic content of
speech.Metaphoric gestures are similar to iconics, but rather than representing a
concrete image, they are more abstract. Beats are rhythmic gestures that indicate a
word is significant. Cohesive gestures serve to tie together related events that are
separated by time. Finally,deictic gestures are pointing gestures, which may be
pointing at a concrete or an imagined o句ect.A deictic gesture might even be
considered metaphoric when someone points backwards while refe哺ng to
something in the past, so that categories may not be mutually eχcIusive.
There are aIso gestures called emblems, 0r quotable gestures (Kendon,2004).
These are gestures that may occur without speech and are understood in the same
way by members of the same culture. The gesture for okay with the thumb and
index finger fk)rming a circle and the remaining three nngers spread vertically is one
example that can be very differently and even negatively understood in some
cultures.
Re血sal gestures are related to a specific speech act, the refilsa1. They a1so will
be a refilsaI of another speech act such as an offer, an invitation, 0r a suggestion.
XVhen used without speech, they may be considered emblems representing simple
negation.The head shake no could either be negation or a refusal, andthepalm
forward¨stop gesture could be interpreted as a refusal due to its hand shape, which
ア5
一 一 一 一一 一 一
is related to the vertical palm (VP)group ofthe Open H回d Prone (OHP)family of
gestufes(Kendon,2004)。
ln his classic work, de Jorio (1832/2000)described in detail some gestufes that
are used in Naples. He illustrated some of the more familiar gestures of negation
such as the headshake as head held vertica1, tuming it altemately 丘om right to len,
the neck remaining straight (p.290).There is the re血sal gesture of open hand
raised and oscillated fiom right t0 1eft (p.292),found among renlsa1s performed
by JSL leamers (Jungheim,2004).He a1so described a gesture in which the hands
lifted naturally towards the shoulders, with the palms opposed to whatever is denied
or rejected (p.294),which is similar to the 柘rm referred to by Kendon (2004)as
the VP/OHP, which resembles the above−mentioned gesture used to mean stop.
He noted that any gestu】″e that is used to express negation can aIso show the
intention to refuse。
The pragmatic 血nction of gestures and how they mark the particular function
of a speech act has a1so been emphasized in past research (Kendon 1995, 2004).
Among the pragmatic functions of gestures suggested by Kendon (2004)is the
performative function in which the gestuTe indicates the speech act a speaker is
performing. This is an especially marked feature of OHP family gestures that can
aIso fhnction as negative particles. Gestures fiom the VP group of gestures in the
OHP family can beseen asderived n・om actions of creating a barrier against the
advance of something, 0r pushing or sweeping something away¨(Kendon,2004,p.
283).These gestures can add meaning to the verbal message, lending valuable
support for re血sals that are ambiguous and use language that can be either
positively or negatively interpreted. Understanding these gestures could assist
language leamers whose conununicative skills are sti11 1ess than well developed。
ln four studies conceming the r01e of speech and gesture to aid pragmatic
understanding,Kel以Barr,Church,and Lynch (1999)revealed a number of
Z6
一 一 一 一一 一
The interaclion oflanguage and nonverbalbehaviorinluencingtheperceplionofJapaneserefusals
important ways that gesture and speech interacted to aid comprehension. First,they
found that people were better able to understand indirect requests if speech was
accompanied by an appropriate gesture, Second, they found that speech and gesture
were mutually disambiguating. Finally, they were able to apply these findings to
difkrent t)1)es of speech acts and gestures. These studies have important
implications for the present study ofthe interaction oflanguage, gesture, and facial
expression to realize a Japanese refusal.
Thereぷn are, however, very few studies of how Japanese use gestures with
re恒sals.ln their study ofJapanese learnersofEnglish as a foreign language (EFL),
Gass and Houck (1999)described what they referred to as a hand wave (p.125−
128)used as a refusal. They neglected, however,to note how this represents cross−
linguistic influenceof a gesture commonly used for negation in Japan. Mo哺s
(1994)called this the Hand Fan, which he attributed to Japan, describing the action
as, The open right hand, with palm to thc len, is moved back and forth in fyont of
the face, as if fanning a name….The hand takes over the role of the latera1 Head
Shake (p. 10).This gesture appears to be related to the VP gestures described by
Kendon(2004),although the orientation of the palm is to the left, rather than
fk)rward,suggesting that the forearrnis rotated to the neutraP position in the Hand
Fan gesture. Theacconlpanying illustration in Mo㎡s book is interesting in that the
gesture is pe咄)rmed directly in fiont ofthe face, unlike Gass and Houck s language
leamer who appears to be performing it slightly to the side ofthe face while shaking
herhead.See Figure l for an eχample ofa Japanese re血salusing a gesture n・om the
VPgroupoftheOHPfamilyandaneχampleofarefusalusingtheHandFanwith
the left hand.
As mentioned above,refusals are a response to another speech act. ln a
descriptivestudyofJSLleamerandJNSrefusaIsofanofTer of something to drink
and refusals of an invitation to attend a sale, Jungheim(2004)found that gestures
ZZ
二二〇
[ano kekkou degozaimasu] [iie iie kekkou desu]
uh aU right is no no a11 right is
Uhl,nol.thank yolu/,|li tlN0,n0,that s万all righlt,
Figure l. The figure on the ld is an example of a Japanese native speaker s gesture in the VP group of the OHP family of gestures used for refilsing an offer (Jungheim,
2006,p. 127),ThefigureontherightisanexampleofJapaneseHandFangesture usedforre鮨ing an offer(Jungheim,2006,p, 130),
varied not only according to the eliciting speech act but aIso according to whether
the refusal was performed by a JSL leamer or a JNS. For native speakers and
language leamers alike, re血sals of the offer of a drink were accompanied by
manual gestures. 0n the other hand, refilsaIs ofthe invitation were accompanied by
head movement, including bowing・
Gestures for refhsals of an oflbr1¥ereof the VP variety. Differences between
language leamers and native speakers aIso involved the style of the gesture. Both
groups included the VP gestures described here. However,three out of ten JNSs
used the Hand Fan, something JSL leamers did not d0. Language leamers used
whatMo㎡s(1994)calIs the Hand Wag, ‥.hand,with palm showing, is wagged
rapidly fiom side to side (p.131)and aIso the same gesture using both hands with
the nngers spread. JNSs did not use either a single−handed or double−handed Hand
Wag. As previously stated, JSLlearnersalso used more bowing with refusals ofan
invitation because many ofthem began their refusal with an apology, much like the
English formula of preceding a refusal with Fm sorry but ‥.¨bef1)re giving a
ア8
二二九
Theinteractionoflanguageandnonverbalbehaviorinluencin91heperception of Japanese refusals
reason in order to mitigate the effbct of refilsing・
The facial side of refusals
As mentioned in the introduction, there is a popular misunderstanding about
theroleofnonverbalbehaviorinc01nnlunication giving it a much larger percentage
of the message than it actually deserves (Beattie,2004).This is not to say that
gesture and facial eχpression do not bear an important share ofthe communication
load. CampbeII (1999)pointed out how facial expression is pragmatically important
in sign languages. ln other research that was ca函ed out to deveIop a model fi)rthe
construction of an artificial agent, the goal was to be able to express performatives
thfough facial expressions (Poggi&Pelachaud,1998).As Ekman (1999)noted,
The face is one of the primary sites of emotional eχpressions: the voice is the
other (p.51).
When there is ambiguity with the possibility of a number of meanings, facial
expression can play the role of eliminating inappropriate possibilities and focus on
the most likely one (Calbris,1990).Calbris also spoke of superfluous vs.
complimentary facial expression, stating that, A superfluous facial eχpression can
modify the meaning of a gesture (p. 17).Facial expression can not be overlooked
because it does carry a portion ofthe communication Ioad.
ln Japanese, it is p011te to refuse an offer,evenif you do not mean it. Even
though the person on the left in Figure l did not smile, greater politeness is indicated
bytheuseof面gθzα加心zj for the linking verb instead oldesu.lt has been suggested
that Japanesedo not like to display emotion either verbally ornonverbany,which
might cause them to avoid facial expressiveness (Bamlund,1989),although studies
ofre仙sals have had participants smile when they re血sed ofTers and invitations
(Jungheim,2004,2008,2009b).ln a talk given while deve10ping the test used in
this study, the researcher showed a video clip created for the test that had verbal
negation,a refusal gesture, anda neutral fllcial eχpression. 0ne immediate reaction
ア9
一 一一 一八
was/ He Iooks rude¨(McNeill,2007).ln French as we11 The relilsal can be
modulated:with a smile, the outward palm will be interpreted as a polite refusaL‥
(Calbris,1990,p. 102),This also seems to be the case for Japanese; there is the
expression wα,・αZZe go。1α尨皿。which is literally to try to deceive someone by
laughing or to smile it away (Kondo&Takan0,1986,p.632).
Listener gaze is another factor invoIving fllcial eχpression, as well as gestures・
The fllce is the default 10cation for fixation. Nobe,Hayamizu, Hasegawa, and
Takahashi(2000)pointed out that it is possible for a person to process a large
amount of information in the small area around a fiχation. 0n the other hand,
fiχation is not essential fbr comprehension (Jungheim,2008; Nobe, et al.,2000),
and peripheral gestures may be more salient (Gullberg&Holmqvist,1999).From
the culturaI standpoint, Watson(1970)claimed that Asians tend to have more
peripheral gaze rather than focusing on a person s f11ce or head. Participants in his
study thought that too much gaze was disrespect血l and insulting。
This Study
Thepurposeofthis study is to eχamine how language and nonverbal behavior
interact to infhlence a listener s perception of a refusal in Japanese. Focusing on
verbal and nonverbal featuresof a Japanese refusaI of an oflbr, the foIIowing
research questions are addressed:
1)To what eχtent do verbal negation, gesture, and fllcial eχpression affbct the
perception of someone s intention to refuse in Japanese?
2)To what eχtent are speakers of Japanese as a mother tongue conscious of the
efTect ofverbal negation, gesture,and facial eχpression on their perception of
someone s intention to refuse?
Abcttcrundcrstandingofthcr01cofgcsturcsandfaciaIcxprcssionsinrcfusals
may contribute to the teaching of JSL pragmatics.
二二七
80
二二六
white cup was placed. The background was also
dark blue. Performers were instructed to take their
glasses and jewelry off and wear a black t−shijlll
provided by the researcher (see Figure 2 for the・
setting).This was done to eliminate as much
context fiom the videos as possible and to remove
any supernuous distractions.
Theinteraction oflanguage and nonvert)al behavior influencing the perception of Japanese re粕sals
IVlethOd
Participants
Participants in this study werc 72 Japanese native speakcrs (41 females, 31
males)who were l8−66 years old (λf=31).They were recruited fiom among
undergraduate and graduate students, faculty members ofthe researcher s university,
and residents of the researcher s neighborhood, which gave the sample a greater
range of ages and backgrounds.
Nlaterials
Materials were a data c011ection video with 13 video clips ofpersons refilsing
an ofTbr of a drink (one example, eight target items, and four distractors)and a
rating sheet. The target items had eight combinations of士negation(治?),士gestufe
(VP/OHP gesture),and士facial expression(十=smile,−=neutral fllcial expression).
Performers in the videos were recruited fiom the local population in a m4jor
Ameriean city and paid S20.00 for their help. They were two fi,males and two males
in their 20s.
The videos f1)r the data collect instrument called the Refusal Perception Test
were created at a gesture lab attached to the psych010gy department of an American
university. AIl performers sat at a table covered with a dark blue c10th on which a
Each perfomler was shown a sample video of F桓a2.mustration of the seuing for the data
8 1 someone refi1sing an ofTer, They were then video conection videos.
十 一十 一十 一十 一十 一十 一
結構です(kekkou desu)
結構です(kekkou desu)
結構です(kekkou desu)
結構です(kekkou desu)
結構です(kekkou desu)
結構です(kekkou desu)
結構です(kekkou desu)
お構いなく(okamainaku)
いいです(iidesu)
いりません(irimasen)
Table l
Rゆ4s㎡Formulas Used in the Re和sal Perception Rst
L2 3 4 5 6 7 8 9 10Hp一 +十 一 一十十十 ︼
⊇一五
−
++十十十 一
recorded in the order shown in Table l with the researcher sitting by the camera and
saying b晶丿尨zjαzz辿θηθ朋jz?7心z4加(Would you like some cofTee or something
eIse to drink?)and actors responding with the designated refusal formula. They also
repeated the ofTeritself,and their voices were used to replace the researcher s voice
on the RefusaI Perception Test。
Four versions of the Re伍sal Perception Test were created by randomly
distributing the four sets of performances, making sure that no videos with similar
fk)rmulas were perfbrmed by the same person, which might make participants think
a performance was used twice. These were then arranged randomly in a PowerPoint
presentation.AIl instructions were included in the Refilsa1 Perception Test itselfl
The rating sheet and test instructed participants to rate the videos perfomlances
on whether or not they thought the person was serious about refusing the offbr of a
drink on a scale of l t0 5 from α&θ/zjZ6?加zlθZ∫erjθzgα&θ爾阿/沁加gtoα&θ/z4Zに砂
serious about 尺φ4sing・ The Japanese expression used for serious was /zθz7肩.A
space was providcd neχt to cach item for wTiting the reason for the rating, and the
back ofthe sheet was used for retrospective comments.
82
Theinleraction oflanguage and nonverbalbehaviorinluencingtheperoeptionofJapanese励Jsals
Procedure
Participants sat at a table with a notebook computer in fiont of them. The
researcher sat to the side and operated the Re恒saI Perception Test by remote
control.After reading the instructions, participants were aIIowed to ask for
clarification. They then viewed a sample video and rated it. The researcher gave
participants ample time to rate each video and write their explanations. After
completing the rating and comments, participants wrote retrospective comments on
the back of the rating sheet. The researcher read these immediately and asked
follow−up questions.W11en the data collection was completed, the purpose of the
study was explained and participants could ask questions or make further comments.
AIl participants were given chocolates or a tea bag set for their participation.
Ratings were input into a spreadsheet, and comments about each rating, as
well as retrospection, were coded by two graduate student JNS raters for mentions
of negation, gesture,and fllcial eχpression. This coding was also input into a
spreadsheet.
Analysis
ln this study, the independent variable is Japanese refilsal and the dependent
variable is native speaker perception. Native speaker perception was measured
using the RenlsaI Perception Test. Thealphalevelofsignificanceforthisstudywas
setat.05.
Descriptive statistics were calculated fbr the eight target items of the Refusal
Perception Test. A Multivariate Analysis of Variance (MANOVA)was carried out
comparing four items with a smile (十fllcial expression)with four items that had a
neutral face Gfacial expression)to determine ifthere was any significant difkrence
between participants judgments of seriousness to re拓se among the various
combinations of negation, gesture,and facial eχpression. The comparison of only
士facial eχpression pairs ofitems was made after it was found that facial eχpression
83