Implicit synesthetic perception in lexical processing(Summary of Awarded Presentation at the 29th Annual Meeting)

The Japanese Psychonomic Society

NII-Electronic Library Service

The JapanesePsychonomic Society

The

lapanese

lburual

or

Psychonomic

Sctertce

20Ll, VDL 3a,Ne. L,141- 142

Summary

ofAwarded

Presentation2-316

Implicit

synestheticperceptionin

lexical

_processing

Michiko

AsANo*'

*2:

#

and

Kazuhiko

YoKosAwA*

The

VhiversiCy

_of

Tokyo*

and

7kimagawa

Universitb,

Brain

Science

Institute*2

One

form

of synesthetic

_perception

involves

the

existence

of

non-arbitrary

mapping

between

linguistic

sounds

and visual shapes

{the

so-called

bouba/kiki

effecO.

The

mechanisms

underlying

such

synesthetic

percepts

are

unclear.

Here

we

report

that

this

synesthetic association

is

triggered

by

_phonological

activation

during

normal

on-line

lexical

_processing.

We

designed

an

irnplicit

interference

task

in

which

participants

made

lexical

decisions

about

Japanese

non-words

presented

in

shapes.

Consonant

sounds of

the

non-words and

the

visual shapes were either

synesthetically

matched or

mismatched,

The

non-words

were

written

in

either

Japanese

_phonetic

Hiragana

script

or

logographic

Kanji

script,

Generally,

_phonology

is

thought

to

mediate

lexical

access

to

Hiragana

words

but

not

Kanji

words,

Our

results

identified

synesthetic

associations

only

with

non-words

in

Hiragana

script,

suggesting

that

phonological

activation

during

lexical

processing

is

associated

with supra-modal

processing.

Key

words:

synesthetic

_perception,

boubalkiki

effect,

lexical

access

Synesthetic

perception

in

non-synesthetes refers

to

phenomena

in

which

a

stimulus

induces

not only

the

normal

percept

but

also

a

second

percept

associ-ated with a second sensory modality or along a

sec-ond

dimension.

One

exarnple

of

this

type

of

percep-tion

is

non-arbitrary

mapping

between

]inguistic

sounds

and visual

shapes.

For

instance,

when

indi-viduals who are

_presented

with a spiky shape and a

curvy

shape

are

asked

which

of

the

two

shapes

should

be

called "bouba"

or "kiki",

the

curvy shape

is

preferentially

associated with '`bouba,"

whereas

the

spiky

shape

is

more

often

referred

to

as

"kiki"

(bouba/kiki

effeet;

Ramachan

clran

&

Hubbard,

20O1).

The

rneehanisms underlying such synesthetic

per-cepts

are

not well understood.

In

this

study, we

investigated

whether

non-arbitrary

sound-shape

mapping

eccurs

automati-cally

during

lexical

processing.

In

a

related study

by

Westbury

_(2005),

lexical

decision

_(i.e.

_judging

be-tween

words

and

non-words)

on

a

non-word

was

facilitated

when

the

stimulus was

placed

in

a

shape

that

synesthetically

matched

the

consonant

sounds

*

_Dept.

_of

_Psychology,

_Grad.

_Sch.

_of

_Humanities

and

Sociology,

The

University

of

Tokyo,

7-3-1

Hongo,

Bunkyo-ku,

Tokyo

l13-O033,

Japan

#

_MA.

_is

now at

Faculty

of

Environment

and

Information

Studies,

Keio

University

(working

as a

JSPS

Research

Fellow)

and

Tamagawa

University

Brain

Science

Institute.

of

the

non-word

{e.g.,

stop

consonants

in

a

spiky

shape);

the

results suggested

that

sound-shape

map-ping

automatically

occurred

during

lexical

clecision.

To

confirm

that

synesthetic

_perception

is

triggered

by

_phonological

activation

during

lexical

_processing.

we

repeated

the

approach

used

by

Westbury

(2005),

but

with

two

Japanese

scripts:

Hiragana

{phonetic

characters) and

Kanji

(logographic

characters).

Gen-erally,

phonology

mediates

Iexical

access

for

words

written

in

Hiragana

characters,

but

net

for

those

written

in

Kanji

characters

(e.g.

Kimura,

1984),

We

hypothesized

that

the

effects

of

sound-shape

corre-spondence on

lexical

decisions

would

be

observed

with

Hlragana

non-words,

but

not

Kanji

non-words,

which should

induce

less

phonological

activation.

We

also

predicted

that

sound-shape consistency

would

increase

reaction

times,

because

lexical

deci-sions

are

usually

hindered

when

the

lexical

process-ing

of a non-word

is

facilitated.

Thus,

lexical

deci-sions

on

non-words

should

be

hampered

when

these

stirnuli are

_presented

with synesthetically matched

information;

the

previously

published

results

(West-bury,

2005)

conflict with

this

_prediction,

which

re-quires

further

inquiry.

Methods

Participants.

Forty-eight

adult

native

japanese

speakers

participated

in

this

experirnent.

Copyright2011,The

Japanese

PGychonomic

Society.

All

rights reserved.

The Japanese Psychonomic Society

NII-Electronic Library Service

The JapanesePsychonomic Society

142

The

_Japanese

_Journal

of

Psychonomic

S

Stimuli.

Thirty-six

Japanese

nen-words and

36

words

_(fi11ers)

were used as stimuli.

The

stimuli, which

all

consisted

of

four

syLlables,

were

presented

using

four

Hiragana

characters

for

the

Hiragana

con-djtion,

or

two

Kanji

characters

for

the

Kanji

condi-tion

(between-participant

factor).

The

first

and

third

syllables

of

each stimulus consisted

of

a

consonant

and

a

voweL

whereas

the

second

and

the

fourth

syllables

contained

only

a

vowel or

the

/n/

sound,

Half

of

the

non-words and words contained

stop

consonants

(/kL

!g!,

/t/,

and

/b/),

and

the

others

contained

resonant consonants

(!ml,

ln/,

lrL

and

/y!),

All

non-word and word stimuli

were

_presented

in

either

a

spiky

or

curvy

shape

(18

_possible

shapes

for

each shape

type>.

Consonant

sounds

of

non-word

or

word

stimuli and

the

associated

shapes

were

sy-nesthetically matched

in

half

of

the

trials

(matched

condition: e,g. stop consonants

in

a

spiky

shape>,

and

misrnatched

in

the

other

half

of

the

trials

(mis-matched condition: e.g. stop consonants

in

a curvy shape).

Examples

of

stimuli

are

shown

in

Figure

1.

Sound-shape

consistency was assessed as a

within-participant

factor.

Procedure.

Participants

were asked

to

rnake

_quick

lexical

decisions

on visually

_presented

non-word

or

word stimuli

that

were contained

in

shapes.

The

participants

responded

by

_pressing

a

key

and were

told

_to

ignore

the

surrounding

shape,

which

was

task-irrelevant,

Results

and

Discussion

Two-way

analysis

of

variance

(ANOVA)

with

con-sonant

type

(stop/resonant)

and

sound-shape

consis-tency

(match/mismatch)

as

factors

was conducted

separately

for

the

correct reaction

tirnes

with non-words

in

the

Hiragana

and

Kanji

conditions.

The

results revealed

that

only

the

main effect of seund-shape

consistency

under

the

Hiragana

condition

was

significant

[F(1,

23)=4.49,

_P<.05].

This

rneans

that

lexical

decisiens

were

hampered

when non-words

were

_presented

with a

tnsk-irrelevant

but

synestheti-cally

matched

visual

shapes

and

the

non-words

were

displayed

in

Hiragana

script,

The

mean accuracy

rates

were

89.8%

and

95.4%

in

the

Hjragana

and

Kanji

conditions, respectively.

ANOVA

on

the

accu-cience

Vol.

30,

No.

1

Figurel,

Exarnples

of

the

experimental

stimuli.

Both

images

are

from

mismatch

trials.

Left:

a

Hiragana

non-word with stop

consonants

(/ke-i-ki-nf)

in

a curvy

shape.

Right:

a

Kanji

non-word with resonant

consonants

(/no-u-yu-u/)

in

a

spiky

shape.

Table

1

Mean

correct

reaction

times

(in

milliseconds)

during

the

lexical

decision

task

with

non-words,

with

SEs

in

_parentheses.

Match

Mismatch

Hiragana

Kanji

922

(45)

672

<31)

888

₍₄₁₎

669

₍₃₂₎

racy

rates

identified

no

significant

effects

under

ei-ther

the

Hiragana

or

Kanji

condition.

Generally,

it

is

assumed

that

_phonology

rnediates

lexical

access

to

Hiragana

words

but

not

to

Kanji

words.

Our

results

showed

synesthetic

associations

only with non-words

in

Hiragana

script, suggesting

that

_phonological

activatien

during

lexicai

process-ing

is

Linked

automatically

to

supra-modal

process-ing,

This

result

is

consistent

with

our

study

on

Japanesc

grapheme-color

synesthesia

(Asano

&

Yo-kosawa,

in

_press),

in

which sound

_qualities

of written

characters elicited synesthetic

_perception.

References

Asano,

M.

&

Yokosawa,

K.

(in

press).

Synesthetic

colors

are

elicited

by

sound

_quality

in

Japanese

synesthetes,

Consciousness

and

Cognition.

Kimura.

Y.

(1984).

Concurrent

vocal

interference:

Its

effects on

Kana

and

Kanji,

Quarterly

journal

of

ExPerimental

llsychotog);

36A,

117-131.

Ramachandran,

V.

S.

&

Hubbard,

E.

M.

_<2001).

naesthesia-A

window

into

_perception,

thought

and

language.

fournat

of

Consciousness

Studies,

8,

3-34.Westbury,

C.

_(2005).

Implicit

sound symbolism

in

lexical

access:

Evidence

frorn

an

interference

task.