The Japanese Psychonomic Society
NII-Electronic Library Service
The JapanesePsychonomicSociety
The
Japanese
Jettrnai
of
twchonomic
Science 198,2,Yol.9,No. 1,17-eoThe
as
Modality
Effect
a
Function
of
Machiko
Ndruto
Liniversity
on
Text
Processing
Presentation
Rate
SANNOMIYAi)
of
Teacher
Education
A
text was presentedin
one of three modalities(auditory,
visual, or audiovisual) atfast
Qr
slow rate.Modality
difference
of recall performance was observed onlyin
the
fast
ratecondition.
And
this modality effect was auditory superiority over visual and attdiovisualpresentation,
Visua!
inferiority
wasinterpreted
in
terms ofthe
necessity oftranslation.
It
is
supposedthat
the
translation
from
visualinput
into
an acousticforrn
consumes sometime and
it
leads
to
processing
insuficiency
when presentation rateis
fast.
Visual
inferior-ity
is
found
a]soin
theprocessing
of word-lists, while audiovisualinferiority
is
peculiar
to
text
processing.From
the
deterioration
of audiovisualperformance
atfast
presentation
rate,the visual component of audiovisual presentation also seems to
be
translated acousticallyduring
textprocessing.
Key
words: moclality effect,translation,
textprocessing.
An
effect
ofpresentation
modalityon
text
processing
has
been
reported(Sannomiya,
1982a,
1982b).
This
effect
indicates
the
supe-riority of auditorypresentation
over visual and audiovisualpresentation
in
free
recallperformance
oflearning
texts.
For
the
anal-ysis
of
recallprotocols,
propositions
or
idea
units were adopted as scoringgnits.
An
in-teresting
point
ofthis
phenomenon
is
that
the
modality effect
is
found
only
for
diMeult
texts.
That
is,
there
is
no significantdifference
in
recall amongthree
modalitiesin
the
case of easytexts
while auditorysuperiority
is
clearin
the
caseof
diMcult
texts,
whenpresenta-tion
rateis
fixed,
Text
dithculty
was deter-minedby
ratingson
five-point
scales (Sanno-miya,1982a)
erby
manipulatingthe
organiza-tion
level
ofthe
text
(Sannomiya,
1982b).
On
this
interaction
between
modality andtext
dithculty,
atentative
explanationhas
been
as-1)
Theauthorwishes
to
thank
Professor
Shigeru
Ono
andProfessor
Yoshiaki
Nakajima
ofOsaka
University
for
their
helpfut
comments on thisexperiment.
Thanks
are alsodue
toProfessor
Norihiko
Kitao
ofOsaka
Uniyersity
oiEducation
for
his
careful reading of.an
earlier version ofthe
manuscript,sumed
in
terms
oftranslation
process:
Visually
presented
texts
aretranslated
into
an acousticform,
Therefore
the
text
processing
during
visual
presentation
tends
to
be
imperfect
ow-ing
to
the
additionalprocess
oftranslation,
whenthe
given
text
is
adithcult
one which requires muchtime
for
processing.
This
in-suthciency oftext
processing
is
supposed
to
result
in
impaired
recall,'
From
the
above explanation,h6wever,
the
following
prediction
maybe
made:Even
aneasy
text
willgive
riseto
the
modality effectif
it
is
presented
ata
sudiciently
fast
rate,because
of
visual
disadvantage.
The
purpose
of
this
studyis
to
examinethe
abovepre-diction.
Another
puzzling
point
ofthe
modality
ef-fect
on
text
processing
is
audiovisualinferiority
to
auditorypresentation.
This
is
incompatible
with
the
findings
on word-Iist memory. Con-cerningthe
memoryfor
words, audiovisualpresentation
has
been
reportedto
be
aseffec-tive
as auditorypresentation
(Henmon,
1912).
The
audiovisual
inferiority
is
observed
in
the
processing
ofthe
dificult
texts.
Is
this
also causedby
processing
insuMciency
on
account
of
translation?
If
audiovisual
presentation
of
The Japanese Psychonomic Society
NII-Electronic Library Service
The JapanesePsychonomic Society
18
The
Japanese
Journal
oftexts
involves
the
translation
of
visualstimuli,
the
fast
presentation
of an easytext
will causedisadvantage
also
for
the
audiovisual
condition.
Method
Subjects
Fifty
four
Japanese
undergraduates at a university servedas
subjects,
The
ratiobe-tween
male andfemale
was made equalin
aliconditions.
Design
A
3
×2
factorial
design
with9
subjectsper
cell was used.
The
first
factor
waspresenta-tion
modality(auditory,
visual,
andaudio-visual).
The
secondfactor
waspresentation
rate
(6.5
syllableslsec
(slow)
and8,5
syllableslsec
(fast)).
Both
factors
were
between-sub-jects.fitkxterial
As
aneasy
text,
the
organizedtext
in
Japa-nese used
in
Sannomiya
(1982b)
was
employed(see
Appendix).
As
opposedto
the
disorganized
text,
the
organized
text
gave
riseto
nodif-ference
in
recall amongthree
modalities atthe
rate of6,5
syllableslsecin
the
previous
experiment.In
the
present
experiment,for
auditorypresentation
the
text
wasrecorded
on
tape
with afemale
voice attwo
kinds
of rates,.6.5 syllables/secand
8.5
syllablesXsee,Care
wastaken
to
readthe
text
withoutstrong
intonations
and
pauses
which could serve as remembering cues.For
visual
pre-sentation each sentence was
printed
on a6.5
×
18.0cm
white cards, which were made upinto
abooklet.
,thocedure
Subjects
wereinstructed
to
try
to
compre-hend
and rememberthe
text,
They
wereinformed
in
advancethat
afree
recalltest
wouldbe
administered.In
atraining
session subiectsgot
accustomedto
the
pace
of
pre-sentation.
The
visualgroup
learned
to
read
at
the
prescribed
pace
which was controlledby
the
sound
of
a
whistle
indicating
that
the
page
shouldbe
turned.
The
audiovi$ualgroup
learned
to
synchronize
reading andlistening.
0vert
vocalizationwas
notpermitted
in
allconditions.
After
the
training
session,the
text
to
be
learned
was
presented
three
times
Psychonomic
Science
Vol,
3,
No.
1
in
succession.
Immediately
afterthe
third
presentation,
subjects were askedto
writedown
on ablank
sheet ofpaper
whatthey
recalled of
the
text.
The
present
procedure
was
the
same asthat
in
Sannomiya
C1982b)
except
that
two
kinds
ofpresentation
ratewere used.
Resuits
Recall
prot6cols
were scoredfor
the
pre-sence oi30
propositions.
Scoring
wasbased
onthe
discussion
of
two
judges
including
the
author.
While
scoringthe
recallpretocols,
both
judges
were not aware ofthe
experi-mental condition
to
which aparticular
protocol
belonged,
One
point
was scoredif
the
gist
of a
proposition
was reproduced.Table
1
shows
the
mean number of recalledproposi-tions.
Hartley's
test
for
the
homogeneity
of variances showed anonsignificant
result(P"..x
=:6,85,P>.05).
A3
×2(modalityxpresentation
rate) analysis of variance
yielded
significant
main effects
of
modality
(F(2,
48)=9,02,
P<.Ol)
and
presentation
rate(F(1,48)=5,50,
P<.05),
And
what
is
moreimportant,
the
interaction was significant(F(2,48)=!19.41,
P<.Ol).
Mul-tiple
comparisonswere
performed
by
applyingSheff6's
test
to
the
recall scores obtained atthe
slowand
the
fast
rate respectively,No
significantdifference
amongthe
three
modali-tie$
wasfound
for
the
slow
rate, whereasthe
superiority
of auditory over visualand
audio-visual
presentation
was significant(P<,Ol;P
<.05)
for
the
fast
rate,"rhen
comparingthe
two
presentation
rates, no significantdifference
wasfound
for
auditorypresentation
(t(16)
=1.31),
whilethe
slow rategave
riseto
better
recall
than
the
fast
ratein
the
case ofvisual
(t(16)==2.88,
P<.05)
and audiovisualpresenta-tion
(t(16)=2.46,
P<.05).
Besides
the
above analysis,the
relatienbetween
modality and
Table
1.
Mean
number of recalledpropositions,
I
presentation
Modality
tion
RateAuditory
M
SD
VisualL
MSD
siow
l
Fast
114.33
1,25
i
13.89
1,20
11.33
2.21
15.89
3.14
Audiovisttal
MSD
114.67
1.41
12.22
2.44
The Japanese Psychonomic Society
NII-Electronic Library Service
The JapanesePsychonomic Society
M,
Sannomiva:
The
modalityserial
position
was examined aboutthe
medal-ity
differences
in
the
fast
rate condition.The
text
wasdivided
into
three
parts
(primacy,
middle,
and
recency),
which
respectively
con-sisted of
ten
propositions.
A
3
×3
(modality
xpart)
analysis
of
varianceyielded
nosigni-ficant
interaction
(F(4,
48)==1,98),
Discussion
The
modality effect was obtained atthe
fast
presentation
rate(8.5
syllables/sec), whichis
consistent withprediction.
The
slowpre-sentation of
the
text
(at
the
rate of6.5
syl-lables/sec)
did
not make anydifference
in
re-call among
three
modalities, andthis
resultis
a replication ofthe
previous
one(Sannomiya,
1982b).
The
introspection
of
subjects
in
the
visual
group
confirmedthat
they
finished
read-ing
all sentences evenin
the
fast
presentation.
This
fact
eliminatesthe
possibility
that
the
deteriorated
recallin
the
visual conditionis
owingto
unfinished reading.Visual
inferiority
in
the
fast
presentation
lends
support
to
the
assumption
that
the
translation
from
visual
input
into
an
acoustic
form
consumes sometime
andit
leads
to
in-suthciency oftext
processing
whenpresenta-tion
time
is
not
suthciently
provided.
In
the
case of auditorypresentation,
no
de-crement of
performance
was observedin
the
fast
rate
condition,
Frern
this
result,it
rnaybe
consideredthat
even
at
the
rateof
8.5
syl-lablesfsec,
the
given
text
couldbe
processed
as
suficiently as atthe
rate ef6.5
syllablesfsec whentranslation
is
not necessary.At
stillfaster
rate,however,
performance
willdecrease
evenin
the
auditory conditionbecause
of in-suthcienttime
for
processing.
The
assurnption ofprocessing
insuthciency
in
the
visual conditionis
applicableto
word-list
memory.Some
researchers explainedthe
modality
effect
on
word-list memoryin
terms
of visualdisadvantage
of
translation
(e.
g.
Levy,
1971).
But
the
problem
of
audiovisual inferi-orityis
not so simple.For
word-list memory, audiovisual superiority over visualpresenta-effect on
Text
Processing
19
tion
andthe
equivalence of auditory and audio-visualpresentation
have
been
reported(Henmon,1912).
These
phenomena
were attributedto
the
fact
that
the
subjectsin
the
audiovisual condition coulddepend
on
auditory
informa-tion.
In
the
present
experiment,
however,
subjects were
instructed
to
make use ofnot
only
auditory
but
also
visualinformation
dur-ing
text
processing.
And
it
was
demonstrated
that
audiovisual
presentation
is
not so eMcientas
auditory
presentation
for
text
memory,which
is
congruous withthe
previeus
results(Sannomiya,
1982a,
1982b).
It
seemsto
be
dif-ficult
to
explain
this
finding
without assumingthat
the
transration
occuredduring
text
pro-cessing also
in
the
audiovisual condition.But
if
the
translation
arisesin
the
audio-visual condition, whydoes
it
deteriorate
text
recall only anddoes
notdeteriorate
word-list recal}?Is
onlytext
processing
sensitiveto
the
cognitiveload
oftranslation
in
the
audio-visual condition?Or
does
the
translation
arise
during
text
processing
but
notduring
the
processing
of word-lists even whenboth
kinds
of materials arepresented
audiovisually?
The
answerto
these
questions
mustawait
further
studies onthe
modality effect.
References
Henrnon,
V.A.C,,
1912
The
relationbetween
mode of presentation and retention.
lagical
Review,
19,
79-96.
Levy,
B.A.,
1971
Role
of articulationin
auditoryand visual short-term memory.
fournal
of
Vlarbal
Learning
andVlarbal
Behavior,
10,
132.
Sannomiya,
M.,
1982a
The
effect ofpresentation
moclality on
text
memory as afunction
ofdificulty
Ievel.
The
fopanese
fournal
of
Ilsy-'
chonomicScience,
1,
85-90.
Sannomiya,
M.,
1982b
The
modality effect ontext
processing
as afunction
of organization.In
A.
Flarnmer
andW.
Kintsch
(Eds.),
Discourse
processing
(Advances
in
Psychology
8).
sterdam:
North-Holland
Publishing
Cempany,
263-268.
The Japanese Psychonomic Society
NII-Electronic Library Service
The JapanesePsychonomic Society
20
The
Japanese
Journal
ofPsychonomicScienceVol.
3,
No.
1
APPENDIX(Learningmaterial)
I
shall
describe
literature,
It
is
during
ad-olescence
that
we
first
encounter
literature.
In
adolescence, one ofthe
motivationsfor
studying
literature
is
te
examinethe
funda-mental
questions
oflife.
Since
literature
is
closely relatedto
life,
I
will
describe
in
the
following
myideas
aboutthe
stages oflife.
There
seemto
be
four
birthdays
during
life.
The
first
one
is
the
realbirthday,
whenwe
emerge
from
ourmother'swomb.The
secondbirthday
is
adolescence,when
we
become
awareof
eur
own selves andbegin
to
function
independently
as'a
human
being.
The
third
birthday
is
the
prime
of
life,
when weex-perience
the
pain
ofIife.
It
is
the
stage when wetry
to
live
once again with airesh
out-leok.
This
mightbe
called a religiousbirth-day
when we aim at spiritual rebirth,The
fourth
birthday
is
the
stage ofdeath,
when weare
reborn asGod
or
Buddha
Here
I
will
especially