Cognitive and higher-level contributions to illusory self-motion perception ("vection") : Does the possibility of actual motion affect vection?(Spatio-temporal integration of multimodal sensations,Symposium 2 at the 27th Annual Meeting)

The

_,1itPanase

JozamalofIlsy:honomicScience

2009,VoL 2S,No.1,l35-139

Lecture

Cognitiveand

higher-level

contributions

to

illusory

self-motion

'

perception

("vection")

Does

the

possibility

'

of

actual

motion

affect

vection?

Bernhard

E.

RIEcKE*

School

of

interactiveArts

&

7]2chnology

{SIA

T}

Simon

F},aser

Uimiversdy*

Large-fieldmoving visual stimuli have long been known to be capable of inducing compelting

illusions

ofself-motion

("vectien")

in

stationary observers,

Traditionally,

theorigin ofsuch visual]y

induced self-motion il]usionshas been attributed to low-level,

bottom-up

perceptual processes

without rnuch cognitive/higher-level contribution Inthe last_years,however, thisview has been

challenged, and an

jncreasing

number of studies

has

investigated

_potentialhigher-]eve]/cognitive

contributions.

This

paper aims at providing a concise review and

discusston

of one of theseaspects:

Does

the cognitive

framework

of whether or not actuat movement ispossible affect illusory self-motion?

Despite

a variety of

different

approaches, there

is

growing evidence that

both

cognitive and _perceptual information lnclicatingmovability can faciLitateself-motion perception, especially when cornbi,ned.

Thjs

has

important

implications

for

our understanding of cognitive/

perceptual cQntributions to self-motion perception as well as the _growing

field

of self-motion

simulations and virtual reality, where the need for _physical motion of the observer could be

reduced

by

intelligent

usage of cognitivelperceptual

frameworks

ef movabiiity. Key words: self-motion _perception,vection,

higher-level

infiuences

Introduction

There

is

a long tradition of

investigating

how

large-fieldmoving visual stimuli can

inducc

Musory

self-motions,

For

examp]e, when standing on a

bri-dge looking down on a fast-moving river, the initial

percept thatthe river

is

moving and one

is

stationary

can eventually

(after

a so-called vection onset

la-ten,cy} switch to a compelling _perception of illusory self-motion

in

the

direction

opposite of the moving

v･isualstimulus, The earliest accounts of vection _go

back

more

than

a century ago, when

Mach

(1875)

and

Helmholtz

(1896)

first

described

the phenome-non.

Since

then, vection has been extensive]y

stud-ied,and comprehensive reviews can bcfound in

CDic-hgans & Brandt, 1978;Howard, 1986;

Warren

&

Wer-theirn,

1990>.

More

recently, vection

has

also

been

djscussed

in

the context of self-motion simulation *

_School

_of

_Interact/ive

_Arts

_&

_Technology

_(SIAT},

Simon

Fraser

University,

250-l3450 102nd

nue, Surrey, BC V3T OA3, Canada

E-mai]:ber]@sfu.ca

and virtual reality, where the

illusory

sensatton of

self-motion might

be

able tocontribute te rnore

be-lievable,naturalistic, and effective simulations at

re-duced

cost and effort

<Hettinger,

2002;

Riecke,

Schulte-Pelkum,

Caniard,

&

BUIthoff,2005; Riecke,

VtistfjblL,Larssen,

&

Schulte-Pelkum, 2005;

Schulte-Pelkum,

2008).

Much of the _previous research focused on the

lnfl-uence of various physical stimulus _pararneters

like

the visua]

field

of view and spatial

frcquency

of the

stimulus and how these contribute to vection via

low-]evel,

bottom-up

perceptual processes

(e.g.

Dic-hgans & Brandt, 1978),During thelastdecades,

how-ever, the prevailing notign

that

vection

is

primarily

driven by ]ow-}evel perceptual processes has been

put

into

question,and an

increasing

number of

stud-ies

have proposed or investigated potential contribu-tionsof various higher level,cognitive processes

<An-dersen & Braunstein, 1985; Lepecq, GiannopuLu, &

Baudonniere, 1995; Mergner & Becker, 1990;Riecke

et al. 2005).Inthis_paper,Iwjll focus on the _question whether Musory seli-motion can befaellitatedi'i

cog-136 The

_Japanese

_Journal

of PsychonomlcScience Vo],28,No. 1

nitive and!or _perceptual information

indicates

that

actual motion

is,

in

fact,

_possible.

Does

the

possibility of actual self-motion

enhance vection?

One

of the

first

indications

of potential cognitiv･e

contributions tovection stems from Andersen and

Braunstein

_(1985),

who remarked that "several

sub-jects

tn

pilot studies and other observers had

previ-ously reported thatthe experience of self-motion was

inhibited

b}r

the

observation that they were

in

an environment inwhich they could not be physically moved"

(p.

124).

This

led

the

authors toseat

partici-pants

in

a moveable

booth

and

demonstrate

the

pos-sibility of motion _pr!or to the a6tual experiment,

Similar

procedures

have

been

used

in

earlier studies

fortranslationsusing a moveable cart

(e.g.,

Berthoz,

Pavard,

&

Young, 1975;

Pavard

&

Berthoz,

1977)

and rotations using a rotating chair

(e.g.

Lackner,

1977),

although none of these or earlier studies had

explic-itly

dernonstrated

that

knowledge

about the

poten-tialor _plausibilityof actual self-motion

does

indeed

affect vection. The earliest study that explicitly

ad-dressed

th

is

conjecture was tothe

best

ofour

knowl-edge the seminal study by Lepecq et aL

(1995},

in

which stationary observers

{children

aged

7

and 11 years) were seated either on a room-fixed chair or a moveable chair with rollers. Beiore the actual experi-ments,

half

of the participantswere shown and expe-rieneed themselves thatthechair was attached tothe

experimental apparatus and thus could not be

moved

("movement

impossible"

condition),

The

other

half

of theparticipants were shown and experienced

themselves that the chair could be moved

{`'rnove-ment _po$sible" condition).

When

subsequently

ex-posed to

backward

linear

vection stimuli,

partici-pants in the movement _possible condition

experi-enced vection earlier, although not more

frequently.

These results suggest thatcognitive factors

(the

kno-wledge and _prior experience that actual motion is

(im)possible)

can affect the onset ]atency of vection but not the occurrence of vection, at leastinchildren.

As

I

wM

discuss

below,

there

is

mixed evidence whether similar cognitive contributions to vection occur inadults.

Wright

DiZio.

and Lackner

{2006)

dernonstrated a

eognitive contribution tothe compellingness of

vec-tion

in

adults] Participants were _presented with a

movie ef a vertical oscillatien

Cat

O.2Hz with 1.7m

amplitude)

displayed

via a

head-mounted

display

subtending a fieldof view of 480 ×36C,Inthe

meve-ment _possiblecondition, _participants were seated in

the vertical oscillator that was used to create the movie and were _{given, prior} to the actual experi-ment, ademonstration ofthe osclllatory motion that

was used tocreate the stimuLus movie.

In

the

move-ment irnpossibleconditien, _participants were seated

on a

desk

chair

in

a separate room,

Although

there was never any actual motion throughout thevection

testing,_participantsinthe movement _possible

condi-tjon report ¢

d

morc compelling sensatjons or

up-down

_{"elevator")

vection than inthemovement

lm-possible condition,

Vection

amplitudes and on$et

la-tencieswere unaffected by the cognitive

manipula-tion,though. The authors _proposed

t'wo

dissociable

mechanisms

for

vection:

One

theone

hand,

aprocess

prirnarily

driven

by

the visual cues that

determines

thc vection onset

latency

and extent of the illusory

self-motion.

One

the other

hand,

aprocess susceptible

tocognitive factorsthat

determines

the

compelling-ness of vection,

Note

that these resu]ts differ

from

Lepecq

et aL's study where the cognitive

manipula-tionaffected vection onset times,

A

circular vection study thatalso used photorcalis-ticstimuli

faited

to

find

any cognit'Lve contributions

to vection inadults, though

_{(Schulte-Pelkum,}

Riecke,

&

BUIthoff,

2004, see also

Schulte-Pelkum,

2008, exp.

3}:

Partieipants

were presented with circular vection stimuli of a reat-world scene

Cthe

TUbingen market

place, which was familiar to all _{participants)}

dis-played on an

immer$ive

video projeetionsubtending

a

field

of view of 860 ×

63".

The whole setup was

mounted on a

6

degree-ef-frccdom

Stewart

motien

platform, and vision of the outside labwas excluded

through heavy eurtains, Inthe rnovement possible

condition, particjpants were shown prior to this

block how the _platform ceuld move. At the

begin-ning of

this

block

participants mounted

the

motion

platform,put on thesafety

belt,

and theplatform was

B.

E.

RIEcKE:

Cognitive

and

higher-level

contributions to i]lusoryself-rnotion perception

("vection")

137

further

physicalmotion.

In

themovement

impossible

condition, _participants were told that the platforrn

would not move, the platform remained switched off,

and _participants

did

not wear safety

belts.

Although

67% of _participants were

fooled

jnto

believing

that

they

physicallymoved

in

at

least

some of

the

move-rnent

_possible

trials,neither vection onset time nor

the intensity or cDnvincingness of vection were

affected

by

the cognitive manipulation.

This

lack

of any cognitive infiuence on any of the vection re-sponses might be related to a _possibleceiling effect,

as vectj,on

in

all conditions was quite strong and compelling,

Furthermore,

several

differences

in

the

experimental _procedure between the current study

and

(Wright,

DiZio,

&

Lackner, 2006) might account

ior

the

different

results,

including

difference

in

the

vection type

(circular

vection vs. osci]latory linear

(elevator)

vection>,

dispLay

device

(projection

screen

vs. head-mounted display),direct_priorexposure to

the physical motion

(seen

from

the

outside

in

Schulte-Pe]kum et al.

{2004)

vs. experienced

from

sitting on the rnevjng _platferm inWright, DiZio,&

Lackner

(2006)}

and

differences

in

the

match

be-tween the visually

_presented

scene and thc actual

surroundings: Whi!e

Schulte-Pelkum

et aL

<2004)

pre-sented a remote scene

in

both

conditions,

Wright,

DiZio,and Lackner

(2006)

presented participants in

the motion possible condition with a movie of the

actua] surrounding scene.

Pilot

studies conducted

by

J.

Schulte-Pelkum

and myself suggest that thislast

issue might indeed

be

critically affecting the

stren-gth of vection:

In

a pilot study

(unpublished),

we used

high-quality

_panoramic

images

of the actua]

testroom as the vection-inducing stimulus, such that the rotating visual stimulus

depicted

on the projec-tionscreen displayed what participants would have

seen

if

the

projectionscreen was a window onto the

real

lab,

While thisstimulus resuited instrong

circu-lar

vection,

it

also resulted in unexpectedly high

levels

of

dizziness

and

discornfort,

which lasted for several hours

for

one ]ab member. The unusually

strong vection and the

fact

thatiswas the firsttime that vection stimuli in our ]ab resulted inserious motion sickness suggests possible

higher-levcl!cog-nitive contributions to vection, jn the sense that

depicting

a rotating naturalistic view of the actual surrounding lab has

different

effects on observers

than displaying a sjmilarly naturalistic view of a

remote

location.

We

are _planning

further

studies to

further

investigate

the

infiuence

of consistency

be-tween

simulated and actual scene, although careful experimentation

is

needed

due

tothe

high

potential foradverse effects likedizziness.

Young

and colleagues

investigated

iftactilecues couLd affect visually

induced

roll

illusions

in

weight-Lessness

_(Young,

Crites,

&

Oman,

1983;

Young

&

Shelhamer,

1990),

Ina "free _fioating"

condfition,

par-ticipants'position

in

weightlessness was only

fixed

by a bite bar.In a

"tactite''

condition, _participants

werc additionally restrained by a shou!der harness

that

held

them on the

fioor

via elastic

bands.

This

addjtional restraint in the tactile condition reduced thestrength of roll vection and

increased

thevection

drop-out rate, and in some _participants even

in-creased vection onset times.

Note

that the restraint might

have

affected vection via both

cognitive/hig-her-leveLfactors

_(e.g.

knowledge that actual motion

was clearly

impossible)

and _{perceptual/lower-level}

factors

(e.g,,

sensation that one istied tothe

station-ary fioor).

Another

study

in

which

both

cognitizre and

percep-tual factors might have contributed investigated

auditory circular vection

(Riecke,

Feuereissen,

&

Rie-ser,

2008,

2009).

Priortothe vection experiment, auditory

vection-inducing stimuli were created

by

seating

partici-pants en a

hammock

chair rnounted above a circular

treadmill and _passively rotating them in the

lab

where two easily loca]izablesound seurces were

po-sitioned.

Participants

wore

in-ear

microphones that enabled individua]ized binaural recordtngs of what it

sounded

like

tophysica]ly rotate

in

the

lab.

During

the vection testing,partic'ipants were blindfolded

and seated on the same hammock chair while noise-cancelling headphones dLsplayed therotating sound-field.In a "movement

possible" condition,

partici-pants put theirfeet on a foot-restattached to the

hammock chair. In a "movement

impossible"

condi-tion,_{participants put} theirfeet on

the

stationary

138

The

Japanese

Journal

of

Psychonomic

Sciencc

Vol,28,

No.

1

ground

in

the "movement

possib]e condition",

vec-tionintensitywas increased,and therewas a

rnargin-al]y significant

(p

<

.1)

trend

towards

reduced vection onset times and higher rates of vection occurrence a$

well as increased realism of actually rotating inthe

lab.

While

participants

in

auditory vection studies are often seated on moveable chairs

{e.g.,

Lackner,

1977;Valjamae,

2007),

this

seems

to

be

the

first

study

thatactually

demonstrates

that thisprocedure does, in

fact,

facilitatevection. Simiiar tothe studies by

Young ctal,,both cognitive and _{perceptual processes}

might have contributed to the vection-facilitating effect: Having one's

feet

touch

the

stationary

floor

provides us on theone

hand

with

higher-Level,

cogni-tive "knowledge" _that

actual motion

is

impossible.

On

the other

hand,

it

provides us with sensory

infor-mation

_(e,g.

biomechanica],

tacti}e,and deep pressure cues)

indicating

the lack of _physical self-motion,

Further, careful experimentation

is

needed _to

disam-biguate

the potential

influence

from

cognitive and

perceptual cues, though,

Conclusions

In eoncluston, there isclear evidence that percep-tual and cognitive cues

indicating

thc potential

for

actual self-rnotion can enhance vection. This is

con-sistent with the often observed _practice of seating

participants

in

vection studies on moveable chairs or

platforms

(e.g.,

Berthoz et al. 1975; I.ackner, 1977;

Pavard

&

Berthoz,

1977;

Andersen

&

Braunstein,

1985; Valjamae, 2007),When only cognitive, but no

direct_perceptual information

indicates

the _petential

of actual se]f-motion, only some of the studies report

a clear vection-facilitating effecti Lepecq et al,

(1995)

observed reduced vection onset tirnes forbackward

linearvection in 7 and 1] _year old children when

they

knew

and

had

experienced

beforehand

t.hat

ac-tualmotion was

indeed

_possib]e.

Wr{ght

et aL

(2006)

showed asimilar vection-facilitating effect

for

verti-cal oscillatory vection

in

adults.

While

these results

are promising, further

investigations

are neeessary

for

a deeper understanding of why, how, and under

what conditions _priorknow]edge about the

possibil-ity of actual motion can affect self-motlon

percep-tion.Such deeper understanding how our

knowl-edge, expectations, and _priorassumptions can affect

eur self-motion _perception would not only

be

theo-retically

interesting,

but

could al$o

be

of consider-able applied interest:For example,

in

motion sjmula-tor-based

joy

rides, users are often

intentiona]ly

im-mersed intothecontext of theride theme and _primed

to believe that actual metion might be _possible,

Moreover,

they arc

hardly

allowed to see the

me-chanics and actual motion restrietions of the

simula-tors.

While

research and

development

results

in

such

commereial application$ are typical]ynet published

and openly accessible. thereseems reason to believe thatsuch measures might not on]y affect the users'

enjoyment and _pleasure,but also contribute to an

improved naturalism, convincingness, and

cffectiv-eness of thc simu]ated self-motions. Ipositthat onty

aresearch approach thatencompassed both a

percep-tual/lower-level _perspective and a cognitive/higher

level

perspective wM ultima'tely enable use to more

deeply understand and employ the fascinating

phe-nomenon of self-motion

Musions.

References

Andersen,

G.

J.,

&

Braunstein,

M.

L,

{1985).

Induced self-motion

in

central vision,

lburnal

of

taiRsychotogy-Human Perceptionand fleijbrmance,

ll(2),122-132,

Berthoz, A.,Pavard, B. & Young, L,

R.

O

975),

tion of

linear

horizontal

se]f-motion

induced

by

peripheral vision

(linearvection)-Basic

isticsand visual-vestibular interactions.

ExPeri-mental Brain Research,

,

23(5),471-･489.

Dichgans,

J.

&

Brandt,

T.

(1978}.

Visual-v･estibular

interactjon:

Effects

on se]f-motlon perception and

postural

control. InPtercePtion,ffdndbooh

_of

sensory

Physiotogy

(pp.

756-804}. Springer.

Hclmholtz, H.V.

_"896).

Adndbuch derPhysioSogischen

CV]lik.

Hamburg-Leipzig:

L,

Voss.

Hettinger,L.

J.

(2002).

Illusory

self-motion invirtual

environments. In K.M. Stanney

(Ed.),

Hdndboole

of

VirtualEnvironmenls

(pp.

471-492). Lawrence

baum,

Howard, I.P.

_<1986>.

The _perception of _posture, self

motion, and the vlsual verticaL In K. R. Boff,L.

Kaufman, &

J,

P.Thomas

(Eds.),

Sensor];

Processes

and

percoption,

f7dndbook

of

human

percaption

and

Pembrmance

CVoL

1, _pp. 18.!-18.62).New York:

Wiley.

Lackner,

J.

R.

(1977),

Induction

of

illusory

B.E.RIEcKE:

Cognttive

and

higher-level

contributions to

il]usory

self-motion perception

C"vection")139

Aviation

Space

and Environtnental

Mladicine,

48{2},

129-131.

Lepecq,

J,

C. Giannopulu, L,& Baudonniere, P.M.

(1995>.

Cognitive

effects on vjsually

induced

body

motion

in

children.

R?rcoption,

24(4),

435r449,

Mach, E.

_{1875).

Grandlinien derLehre von der

Bezve-gungsemptindung Leipzig,Germany: Enge]mann.

Mergner,

T.

&

Becker,

W.

(1990).

Perccption

of hori-zental se]f-rotation:

Mu!tisensory

and cognitive

pects.InR,

Warren,

& A. H.

Wertheim

(Eds.),

l]ler-coption & controi

of

seijZmotion

(pp.

219"263). New

Jersey,

Lond6n:

Erlbaum.

Pavard,

B.,

&

Berthoz,

A.

(1977).

Linear

acceleration

modifies _perceived velocity of a moving visual

scene.

.PercePtion,

6(5),529-540,

Riecke,

B.

Feuereissen,

D.,

&

Rieser,

J.

J,

{2008}.

tory self-motion i]lusions

{"circular

vectioni') can

be

facilitated

by

vibrations and the potential

for

actual motion.

In

A

CM

SIGGRAPU

Symposium

on

mpptied

PeT:coption

in

graphics and visualigation

e4PGV)

(pp.147-154).

doi:

http://doi,acm,org/

10,1145f1394281.1394309,

Riecke, B. E. Feuereissen, D.

&

Rieser,

J.

J.

(2009).

Auditory

self-motion simulation isfacilitated

by

haptic and vibrational cues $uggesting the

bility

of actual motion,

ACM

Tmns.

Appl.

Percept,

6.Riecke,

B. Schulte-Pelkum,

J.,

Caniard, F. & BUIthoff,

H.

H.

(2005),

Towards

lean

and e]egant self-motion

simulation

in

virtua] reality.

In

Proceedings

of

I[iEE

IiirtualReality

2005

(pp,

131-138),Bonn,

Germany.

Riecke, B,, Vastfjall,D., Larsson,.P,, &

Schu]te-Pelkum,

J.

(2005).

Top-down

and multi-Modal

infl-uences on self-motion perception

in

virtual reallty.

Inl]h'oceedings

of

HCI international2005

(pp.

1--10),

Las Vegas, NV, USA.

Schulte-Pelkum,

J.

(2008).

Perception

of

seijLmotioni

Vectionemperiments

in

multi-sensory virtual

environ-ments. PhD thesisRuhr-Universitat Boehum.

Schulte-Pelkum,

J.,

Riecke, B., & BUIthoff, H. H.

(2004).

Vibrational

cues enhance

believability

of ego-motion simulation,

In

interveational

Multisen-sory Research Fontnz

(IMRF),

Valjamae, A.

_(2007).

Sound

_for

muttisensotup motion

simulators.

PhD

thesis,

Chalmers

University

oi

Tec-hnology,

Warren, R. & Wertheim, A.H.

(1990).

Percoption&

Conmol

of

SeijimoSion.

New

Jersey,

London: Erl-baum.

Wright,

W.

G.

DiZio,

P.

&

Lackner,

J.

R.

(2006}.

Perceived

self-motion

in

two visual contexts:

ciable mechanisms underlie _perceptien.

Iburnal

of

Vbstib,

Research,

16(1-2),

23-28.

Young,

L.,

Crites,

T,,

&

Oman,

C.

(1983).

Brief

lessnessand tactilecues

infiuence

visually

induced

roll,Advances

in

Ot'e-Rhino-Laryngology,

30,

230an

234.Young,

L.

&

Shelhamer,

M.

(1990).

Weightlessness

enhances the relative contribution of

induced self-rnotion, In R. Warren, & A. H,

theim

(Eds.),

Percoplion

&

eontrel of selfmotion:

Re-sources

for

ecotogical

_Psychology.

Erlbaum,