Modification of velocity perception by loud sounds(<Special Edition>Multisensory integration in perception, cognition and action)

The

_.lapanese

Jbumalofk)'ehenomthScience

2el1,Vol.30,No,1,19-24

Research

Note

Modification

of

velocity

_perception

Yasuhiro

TAKEsHIMA

and

Jiro

71ohohu U)iiversity*

by

loud

soundsi)

GyoBA

Introduction

Velocity

_perception

is

one of the

important

capaci-tiesforus indailylife.We need toperceive velocity

accurately inorder tododge or cateh moving objects

(e.g.,

arock or

ball).

In

addition,

this

ability

is

impor-tant for avoiding collisions intraffic. The mecha-nism of velocity _perception

has

not _yet

been

eluci-dated thoroughly. Previous studies have advocated

two

theories

of velocity _perception

(see

StrybeL

Span,

&

Witty,

1998

for

a review).

The

first

assumes

thatvelocity _perception isa

_primary

sensation and is

not inferred

frorn

distance and time estirnates. The

second holds that velocity

is

_perceived

indirectly

from estimations of travelingdistance and

duration

Auditory stimuli are

known

toalter visual _perception. However, theeffects of such stimuli on

velocity perception have not yet been examined.

A

well-known velocity

illusion

related

to

object

size

is

described

by

Brown's law. We can easily match object size with sound intensity.Therefore,

this study examined the potential modification of velocity _perception by auditory stimuli at

differentsound pressure levels

(SPLs).

The

results showed thatthe _perceived velocity, _particularly

when the object size was smalL diminished with a

high

SPL

auditory stimulus,

We

assume

two

interpretations

ofthisresult.

First,

high intensitysounds can modify theperceived object size and

alter the

perceived

velocity

by

replicating Brown's law since largeobjects tend tomatch well with

high

SPL

sounds, Second, _previous studies

indicated

thatstimuli with strong

intensities

seem to

have

been

presented

for

longer

durations.

Thus, stimulus

duration

may

be

_perceived as

longer

when

higher

SPL

sounds are presented simultaneously, which may cause the velocity to be

perceived as being slower.

Key

words: audio-visual

interaction,

velocity _perception, visual illusion,information reliabi]ity

hypothesis

of movement,

The

former

theory _predictsthat

veloc-* _{Graduate Schoel of Arts and Letters,Tohoku}

University,

27-1 Kawauchi,

Aoba-ku,

Sendai,

Miyagi

980'8576,

Japan

i}_{This work was supported by a}

_Japan

_Society

for the

PTomotion

of

Science

Grant-in-Aid

for

SpecificallyPromoted Research

(No.

19001004).

We thank Michiaki Shibata for his kind help

and advice, as well as theanonymous reviewers

for suggesting the alternative explanation

based

on the perceived

duration

and

for

cating appropriate references.

Copyright2011

ity discrimination should be more accurate than

what wou]d

be

predicted

from

measures of distance

and time discrimination.

Moreover,

velocity

adapta-tion _phenomena are said to support the primary

velocity view

(Lappin,

Bell,Harm,

&

Kottas,1975).In

fact,

velocity-tuned cells exist

in

monkeys

(Maunsell

&

Van

Essen,

1983}. However,

the

former theory

cannot explain velocity

illusions

(Strybel

et al. 1998).

If velocity

ts

perceived

directly,equal veLocities should always

be

_perceived as

being

equivalent,

but

they are not. However,

if

traveling distance and

duration of movement are related tevelocity

percep-tion,errors

in

these estirnates _produce cause velocity

i]lusions,

Therefore,we assume thatthe

latter

theory

ismDre valid with respect tovelocity illusions.

Many velocity illusionshave been examined in

previous studies

based

on the

latter

theory.

For

example, the apparent velocity is_perceived to be

faster

at

lower

luminance

(Hammett,

Champion,

Thompson, & Morland, 2e07; Vaziri-Pashkam & Cavanagh, 2008). Contrast isanother important

fac-torthatcauses visual velocity

illusions

(Thompson,

1982;Thompson, Brooks, & Hammett, 2006).

More-over, object size greatly affects velocity

perception.

The Japanese Psychonomic Society

NII-Electronic Library Service

The JapanesePsychonomic Society

20

TheJapanese

Journal

of

Psychonomic

Science

VoL30,

No.

1

Brown

₍₁₉₃₁₎

suggested

that

theapparent velocity

is

perceived tobe fasterwhen theobject size issmaller.

By

contrast, apparent velocity is _{perceived to} be slower whcn the object size is

Iarger,

This

velocity

iltusion

is

called

Brown's

law.

Until now, many studies on velocity illusion

have

examined only the visual modality.

However,

audi-tory stimuli are known to alter visual _perception

through audio-visual

interaction

(Driver

&

Spence,

2000;

Shimojo

&

Shams,

2001).

Typical _phenQmena

of audio-visual interaction are the

McGurk

effect

(McGurk

&

MacDonald,

1976)

and ventriloquisrn

cffect

{Jack

& Thurlow, 1973).These studies indicate

that yisual modality issuperior to other modalities

(visual

dominance),

However,

Sekuler,

Sekuler,

and

Lau

(1997)

and

Shams,

Kamitani, and

Shimojo

(2000)

showed that the auditory modality can also

domi-nate the visual modality.

For velocity _perception,Manabe and Riquimaroux

(2000)

reported that apparent motion

is

_perceived to befasterwhen shorter auditory stimuli are presented

in

the blank, Thus, visual velocitv _tends tobealtered

by

auditory stimuli.

However,

thisstudy

did

not

examine smooth motions, In real-life situations, it

may

be

quite rare

for

sounds to

be

emitted only

during apparent motion's blanks, which corresponds

tothe experimental conditions examined

by

them. Therefore, we tried to investigate the effects of sounds on the velocity _perception of real objects

moving with sound.

Another factornot examined in_previous studies is

the

intensity

ofauditory stimulL

Lipscomb

and

Kim

(2004)

showed thatwe can easily match sound

inten-sity with object size:that

is,

tomatch high intensity sounds with largeobjects and

low

intensity

sounds

with small objects. As described above, differences

in

object size alter _perceived velocity

(Brown's

law).

Therefore, it

is

highLy

likelythat adding auditory stimuli of

different

sound intensitiesdifferentiallv

modifies _perceived object size,

In

short, we

investigated

audio-visual interactions

invelocity _perception. We combined differentobject

sizes with

different

sound _pressure levels

(SPLs)

and

examined whether the perceived visual velocity of

theobjects could be modified. We hypothesized that

adding a

high

SPL auditory stimulus would make

perceived velocity slower, whereas adding a low SPL

auditorv stimulus would make

it

faster.

In

order to

J

examine amore realistic smooth motion, we used the motion of a _pendulum inthis study.

Method

Ptirticipants

A group of 8 observers

(4

females

and 4 males)

participated in

this

experirnent. All

had

normal

vi-sion and audition, and al] were naive as tothe pur-pose of thisexperiment.

Apparatus

A

pendulum stimulus was constructed and used

in

thisexperiment

(see

Figure

1).

The

_pendulum was

attached to a speaker

(HK206,

DELL) and 5-mm

light-emitting

diodes

(LEDs).

White

LEDs

(LD504W

3CD2B02P,

Linkman)

were used

for

circular visual

stimuli and ared LED

{LFTLED-R501,

Linkman) was

used as the

fixation

point.

The

luminance

of each

LED was 15 cd

(white)

or O,5cd

{red),

For the visual

stimulus, white

LEDs

were arranged intriplerings,

There

were

36

white

LEDs

in

total

(8,

12,

and

16

for

the small, medium, and large circles, respectively).

The

fixation

light

was attached at the

height

of the

participant'seye, and the visual stimuli were

at-tached

just

above the speaker. The

distance

from the fixationpoint to the center ef the circles was 4.37

deg. An audio interface

(ProFire

Lightbridge,

M-AUDIO),

signal synchronism

device

(Nanesynchs

HD, Rosendahl), AD/DA converter

(Ultragain

Pro-8

Digital

ADA8000,

Behringer}

and amplifier

(RSDA

202,Rasteme systerns

Co.

Ltd.)were used toturn on

thedevice

_{see

Figure 2>.Participants sat 200 cm in

front

of the pendu]um.

Head

movements were

re-strained

by

a chin rest device.

The

_generation

and

presentation of stimuli were controlled by a

custom-made program written using

Matlab

(The

Math-works, Inc.),a

Cogent

Graphics

and

2000

toolbox

(www,vislab.ucLac,uk/cogent.php),

and aPC

<PRECI-SION

T5400,

DELL;

WindowsXP, Microsoft>. The

visual and auditory stimuli were controlled toappear

when the _pendulum moved halfway along its

trajec-tory

<4.5

deg),

The

participants were

instructed

to

judge

the maximum velocity ef _pendulum motion.

a

2S7cm

Y.

TAKEsmMA

and

J.

GyoBA:

Modification

of velocity _perception by loud sounds

b small fixalsti medium .i '"5deg

l.iiiiliiil!.,,.

oB6""g

i

-"

6Scm l t.SSdeg e129 deg

lee

rmcaiivaww

---

"'"'

sidevievt

1.

(a}

Experimental apparatus used

in

present experiment and

(b)

visual

size. The apparatus was made from

pipes and

fasteners,

an attached

",as frentvievv Figure the stimuli's plastic

speaker, and two types of

LEDs.

Velocity

manipulated by changing the length between thefulcrum and speaker.

Figure 2.E]ectric diagramof the apparatus.

The experiment was conducted ina completely dark

room, and the

background

noise levelwas 43

dB

(A).

Stimuli

The visual stimuli were the _pendulum motions of

circular

LEDs,

and auditory stimuli were white

noises of two SPLs. The standard stLmulus was a

visual stimulus without white noise.

The

internal

circumference of thestandard stimulus was O.86deg,

21

and the external circumference was

1.15

deg.

The

comparison stimuli were of 6types

(2

object sizes ×3

auditery conditions). The 2object sizes xN'ere small

and

large.

The

internal

and external circumferences

of thesmall stimulus were O.43deg and O.72deg,and those of the largestimulus were 129 deg and 1.58

deg,

respectively,

The

3

auditory conditions werc

No-Sound, which indicatesthat the auditory

stimu-luswas not presented;50 dB,which indicatesthat 50

dB

CA)

white neise was _presented simultaneously

with visual stimuli; and

90

dB,

which

indicates

that

90dB

_(A)

white noise was presented simultaneously

with visual stimuli. According tothe study

con-ducted

by

Andersen,

Tiippana,

and

Sams

C2004),

we

controlled the inforrnation reliability of auditory

stimuli

by

maniputating the intensitiesof sounds.

All

stimu]us velocities were 1O degls.

In

catch trials, the visua] stimulus

_(size

was same as the standard

stimulus with a velocity of 8or 12 deg/s) was

pre-sented without white noise. There were two

move-ment direcLions,rightward and leftward.

Phrocedure

Each stimulus was _presented

by

swinging _the

pen-dulum.

Each

trialwas composed of one standard

stimulus and one comparison stimulus,

Participants

were

instructed

toperform a two-alternative

forced

choice task

(2AFC}.

Ifthe former stimulus was per-ceived to

be

faster,

they verbally responded "former,"

whereas

if

the latterstimulus was perceived to be

faster,they respQnded

L`latter:'

A 2×3factorial

de-sign was used with the ebject size and auditory condition as within-subject factors.24 trials

(20%

of

experimental

trials)

were added as eatch trialsof one of the 2 velocity levels. Intotal,each participant

performed

144

trials,

i.e.,

120 experimental trials

(20

repetitions per condition) and

24

catch tria]s

(12

repetitions

_per

condition). The tetal number of trials

was

divided

into two sections

depending

on

the

di-rection of movement, and the order of the sections

was counterbalancecl across participants.

The

order of the comparison stimulus

_(forrner

or latter)was

also counterbalanced across _{participants.}

The

fiow

of a single trial

is

depicted

in

Figure

3.

The

partici-pants

were not allowed to look at the apparatus

The Japanese Psychonomic Society

NII-Electronic Library Service

The JapanesePsychonomic Society

22

to!=E-v-Sev=:8:..:

za2.8

zagisg

£

.

The

Japanese

Journal

of Psychonomic Science Vol,30,No. 1

rval:ipuiated theveiocity

mulus

ing

inter-trialinterval/

Experimentermanipuleted thevelocity,

Figure 3. Schematic representation of the _procedure. Black rectangles

indicate

the _periocls

during

which _participantswore an eye-mask.

/

t

l80

'

160 t 40

-･i･

20 { o eeSmalVsLarge

I

No-Sound 50dB 90dB auditory condition

Figure4.

Horizontal axis indicates auditory

conditions, and vertical axis

indicates

the mean rate of the comparison stimulus

chosen as "faster." _The

error bar represents

the standard

deviation

(n=8),

interval

_(ITI).

Results

The

results of

the

catch

trials

were excluded frorn

theanalysis.

We

calculated the rate of the

compari-son stimulus chosen as

"faster,"

The results are de-picted

in

Figure 4,Furthermore, an analysis of

vari-ance

CANOVA)

with object size and auditory

condi-tion as within-subject

factors

was conducted after

angular transiormation of data, The main effect of

the object size was significant

F

(1,

7)=6.00,

_p<.05,

indicating that the perceived velocity varies as a

function of object size. Inaddition, the simple main

effect of size in the No-Sound condition was

sig-nificant F(1,21)=9,37,

P<,Ol,

indicating

_that

Brown's

law

was replicated

in

thisexperiment, ALso,

the main effect of the auditory condition was

mar-ginally significant F(2,

14)=3.59,p=.055.

Further-more, the

interaction

between the object size and

auditory condition was significant

F

(2,

14)=4.03,

P<.05.

Multiple

comparisons of the

interaction

be-tween object size and auditory condition

(Ryan's

method) revealed that the

differences

between

No-Sound/50

dB and

90

dB conditions were significant

when the object size was smalL indicating that the

velocity seemed slower when the

90dB

(A)

white noise was added when the object size was smalL In

contrast,

the

simple main effect was not significant when the object size was large.

Discussion

In

this study, we examined the effect of

audio-visual

interactions

on velocity _perception

by

com-bining

objects of

different

sizes with

different

SPLs.

The results indicated that a high SPL auditory

stirnulus tended tocause theperceived velocity tobe

slower. In _particular,thiseffect was found to be

much stronger when theobject size was small,

Fur-thermore, low SPL sounds did not affect the visual

perception of velocity,

In

a_previous study, the

veloc-ityof apparent motion of an object was perceivecl as

Y.TAKEsHIMA and

J,GyoBA:

Modification of velocity perception

by

loud

sounds

23

higher

when a shorter noise

burst

was _presented

in

the blank

_(Manabe

&

Riquimaroux, 2000). The

re-sults

differ

from

those of the present study, which

indicated

that the high

intensity

sound slowed the

perceived velocity of the object inreal motion. Inthe case of apparent motion, the inserted sound might

not

be

_perceptua]ly

bound

totheobject, whereas the presented sound might

be

onc of perceptual

attrib-utes ascribable tothe moving object inthe _present

study. Therefore,

it

can be assumed that the

dif-ferent

perceptua]

integration

processes might

be

in-volved inthe two cases,

There are two _plausibleexplanations forthe

pre-sent results. The firstassumes that high intensity

auditory stimulus may altcr the perceived visual obiect's size depending on the rnulti-modal

informa-tion

reliability.

According

to

Lipscomb

and

Kim

(2004),

high intensity sounds tend to match with

largeobjects. Inaddition, the information reliability

hypothesis states that the modality of high-reliability

information

is

superior to that of

low-reliability information

<Wada.

Kitagawa, & Noguchi,

2003). We manipulated theinforrnationreliability of an auditory stimulus by thesound intensitiesinthis

study.

Therefore,

it

is

plausible that an object might

be _perceived to be largerwhen accompanied by a

louder

SPL

sound, which

has

high-reliability

infor-mation relative tovisual stimulus, and thus the

per-ceived velocity isslowerdue toBrown's law. On the

other

hand,

the

information

reliability of

low

SPL

sounds islow reLative tovisual stimulus, and thus it does not affect

the

perceivcd visua] veloeity.

The

second explanation assumes thatthe duration

of visual stimuli may bemodulated by auditory stim-uli corresponding tothe stimuli's

intensity.

In

the

previous

studies, the duration of visual stimuli wa6

perceived as longer when these stimuli's

intensity

was stronger; that

is,

the visual size

is

larger

(c.g.,

Ono

&

Kawahara,

2007;

Thomas

&

Cantor,

1976;

Xuan, Zhang, He,

&

Chen,

2007). Ifthe same

phe-nomenon occurs

by

adding auditory stimuli, the

du-ration of visual stimuli isperceived as longer when

higher SPL sounds arc _presented simultaneously.

According

to the theory that velocity

is

_perceived

indirectlyfrom thc estimated traveling

distance

and

duration

of movcment, the velocity is_perceived as

slower when theperceived

duration

becomes

longer.

The factthat low SPL sounds did not alter the per-ceived visual velocity can

be

accounted for by the

shortage of theauditory stimuli's

intensity

for

modu-latingthe perccived duration.

However,

the effect of

the

modification of

per-ceived velocity was

found

to

be

much reduced when

largeobjects were accompanied

by

high

SPL

sounds.

The _perceived velocity was almost identicalamong

the auditory conditions. This rcsult might indicate

that the speed-down effect of

Brown's

law

ts

power-fulenough that

it

may produce a sort ef ceiiing effect

for the _perceivcd velocity, making thc influence of

sound stirnu]i relatively small or negligible.

These

arguments, including the validity testingof thc two types of explanations, must be clarified via further

cross-modal _{psychophysical}

investigation.

We obtained interesting knowledge on

audio-visual interactionwith velocity _perception from this

study. Inorder toevaluate thevalidity of the

discus-sion

based

on the two assumptions stated above,

further psychophysical experiments with an

in-creased number of stimulus _parameters that affect

the

information

rcliability and

(or)

stimuli

intensity

over a wider range are required.

References

Andersen, T.

S.

Tiippana, K,,& Sams, M.

(2004).

Factors

infiuencing audiovisual

fission

and fusion

illusions.CognitiveBrain Research, 21,301-308.

Brown,

J.

F.

(1931).

The visual _perception of velocity.

PsychotogischeJbrschung, 14,199-232.

Driver,

J.

&

Spence,

C.

(2000),

Multisensory

tion:Beyond modularity and convergence. Current

Biology,10,R731-R735,

Hammett, S.T. Champion, R.A. Thompson, P.G.

&

Morland,

A,

B.

(2007).

Perceptual

distortions of

speed at

low

luminance:

Evidence inconsistent

with aBayesian account of speed encoding. Vision

Research,47,564-568,

Jack,

C.E.

&

Thurlow, W. R.

_(1973).

Effcctsof degree

of visual associat[on and angle of

disp]acement

on

the "ventriloquism" effect.

fercoptuaZ

and

Motor

Sleitls,

37,

967-979,

Lappin,

J,

S.

BelL H. H,,Harm, O.

J.

& Kottas, B.

(1975).

On

the relation between

tirne

and space

in

the visual discrimination ef velocity.

Iburnal

of

ller-The Japanese Psychonomic Society

NII-Electronic Library Service

The JapanesePsychonomic Society

24 The

_Japanese

_Journal

of

Psychonomic

Science

VoL

3e,

No.

1

formance,

1,383-394.

Lipscomb,

S.

D.

&

Kim,

E.M.

(2004).

Perceived match

between visual _parameters and auditory

corre-lates:An experimental multimedia investigation.

Proceedings

of

the

8th

internationat

Conjbrence

on

Miisic

Rercoption

&

Cognition,

72-75.

Manabe, K.,& Riquimaroux, H.

_(2eOO),

Sound

con-trolsvelocity _perception of visual apparent

mo-tion,

lburnal

of

the

Acousticat

Societ))

ofJizPan

(E),

21,171-174.

Maunsell,

J,

H,R,,& Van Essen, D.C.

{1983),

Func-tionalproperties of neurons

in

middle temporal

visual area of the macaque monkey.

I.

Selectivity

for stimulus direciton,speed, and orientation.

fournal

of

AieurQphysiology,49,1127-1147.

McGurk,

H.

&

MacDonald,

J,

(1976).

Hearing

lips

and

seeing voices.

IVlature,

26C

746-748.

OnQ, F.,

&

Kawahara,

J.

(2007).

The subjective size of

visual stimuli affects the _perceived duration of

theirpresentation.

Pigrcoption

&

PsychQPhysics,

69,

952-957.

Sekuler, R.,Sekuler, A, B. & Lau, R,

(1997).

Sound

alters visua] motion _perception.IVdture,385, 308.

Shams,

L.

Kamitani,

Y.

&

Shimojo,

S.

(2000).

What

you see

is

what you hear,

Ndture,

408, 788, Shirnojo,

S.

&

Sharns,

L.

(2001).

Sensory

modalities

are not separate modalities: _plasticityand

interac-tions.

Cument

Cipinion

in

IVUerobiolQg],,

11,

505-509.

StrybeL T.Z.Span,

S.

A. & Witty, A.

M.

(1998).

The

effect oftiming and spatial separation on the

veloc-ity

of auditory apparent motion,

Percoption

&

Pspu-chophysics, 60,1441-1451.

Thomas, E.A.C.& Cantor, N.E.

_(1976).

Simultaneous

time and size perception,

Percoption

&

Rsychophys-ics,

19,

353-360.

Thompson, P.

(1982).

Perceived rate of movement

depends on contrast. VisionResearch, 22,377-380.

Thompson,

P.,

Brooks,

K.,

&

Hammett,

S.

T.

(2006).

Speed

can _go up as well as

down

at

low

contrast:

Implications formodels of motion _perception.

Vi-sion Research, 46,782-786.

Vaziri-Pashkam,

M.

&

Cavanagh,

P,

(2008).

Apparent

speed

increases

at

low

luminance.

Iburnal

of

Vision,

8(16),1-12.

Wada, Y.

Kitagawa,

N.

&

Noguchi,

K.

(2003),

Audio-visual

integration

in

temporal perception.

interna-tional

Journal

of

PsychQPhysiolcrgy,

50,

117-124. Xuan, B.Zhang, D. He, S.,& Chen, X.

(2007),

Larger

stimuli are

judged

to]ast

longer.

fournal

of

Vision,

7(10),

1-5.