Measuring Japanese EFL
Measuring Japanese EFL
Learners’ Implicit Knowledge
of
Semantic Constraints
A Case of English Prenominal Adjective Orders
My slide is available at
https://sites.google.com/site/kusanagikuni/home/cv
Japanese EFL learners showed …
1.
“moderate” sensitivity
to the
1.
“moderate” sensitivity
to the
violations of semantic constraints on
prenominal adjective orders in their
real-time reading comprehension
KUSANAGI Kunihiro
KUSANAGI Kunihiro
Background
• Differences between L1 & L2 sentence
processing
processing
– In general, L2 learners’ reading is …
• Slower
• Less accurate than that of L1 users
(Frank-mestre, 2002)
– L2 learners’ reading often shows …
• Different preferences
for syntactic ambiguity
• Different preferences
for syntactic ambiguity
resolution from that of L1 users
(e.g., Witzel, Witzel, Nicol, 2012)
• None of preferences
(e.g., Clahsen & Felser, 2006; Marinins.
Roberts, Felser, Clahsen, 2005; Felser, Roberts, Marinis, Gross, 2003)
• Serious online insensitivity
to agreement errors
Background
• Shallow Structure Hypothesis
(SSH) by
Clahsen and Felser (2006)
Clahsen and Felser (2006)
– L2 grammatical processing is a fundamentally
different mechanism from L1
• L2 learners can utilize
lexical-semantic
representations
during real-time comprehension
like L1 users
like L1 users
• L2 learners’
syntactic and morphological
Background
• Some of other claims
– L2 processing is
identical to that of L1
– L2 processing is
identical to that of L1
(e.g., Jackson & Dussias, 2009)– Rather,
more detailed
(Witzel, Witzel, Nicol, 2012)• Usual suspects
– A problem of
proficiency
(e.g., Hopp, 2006, 2010; Jackson, 2008)– L1
influence
(e.g., Jiang, et al. 2011; Sabourin & Stowe, 2008)– L1
influence
(e.g., Jiang, et al. 2011; Sabourin & Stowe, 2008)– Working memory
-based accounts
(e.g., Coughlin &Tremblay, 2012)
Background
Offline performance
(explicit knowledge)
Online performance
(implicit knowledge)
(explicit knowledge)
(implicit knowledge)
Syntactic
○
☓
Morphological
○
☓
Lexical-semantic
Background
Offline performance
(explicit knowledge)
Online performance
(implicit knowledge)
(explicit knowledge)
(implicit knowledge)
Syntactic
○
○
Morphological
○
○
Lexical-semantic
Background
Offline performance
(explicit knowledge)
Online performance
(implicit knowledge)
(explicit knowledge)
(implicit knowledge)
Syntactic
○
?
Morphological
○
?
Lexical-semantic
Background
Offline performance
(explicit knowledge)
Online performance
(implicit knowledge)
(explicit knowledge)
(implicit knowledge)
Syntactic
○
?
Morphological
○
?
Lexical-semantic
○
○
Background
• Relative reliance of semantic or pragmatic
cues
in L2 processing
cues
in L2 processing
– When information of various grammatical
domains competes in L2 learners’ real-time
comprehension, semantic and pragmatic ones
are dominant among others
(e.g., evidence from
attachment ambiguity or garden path studies)
→
No dispute!
attachment ambiguity or garden path studies)
→
No dispute!
– However,
it doesn’t necessarily imply L2
Background
• L2 learners’ online sensitivity to
purely
semantic
constraints
in the absence of
semantic
constraints
in the absence of
effects from other domains?
We need some linguistic phenomena
• Non-syntactic / non-structural
• Non-morphological
• Relatively context independent
• Relatively context independent
Prenominal Adjective Orders
• In English, when multiple prenominal
adjectives modify a noun, there are linear
adjectives modify a noun, there are linear
procedures of semantic classes to apply
(see,
Martin, 1969; Wulff, 2003 for detailed linguistic descriptions)
Semantic
Semantic
Class
[VALUE] [SIZE] [DIMMENTION] [PHYS.PROP] [COLOR]Example
Good
Big
Tall
Hot
Red
Prenominal Adjective Orders
a.
A nice
[VALUE]
small
[SIZE]
cup
b.
??
A small
[SIZE]
nice
[VALUE]
cup
b.
??
A small
[SIZE]
nice
[VALUE]
cup
Prenominal Adjective Orders
• Note that violations of those semantic
constraints on PAO strongly affect both of L1
constraints on PAO strongly affect both of L1
users’…
• Off-line acceptability ratings
(Danks & Glucksberg, 1971;
Danks & Schwenk, 1974)
• Time courses of reading comprehension
(Kenninson, 2010)
• The effect sizes was medium-level (
η
2
• The effect sizes was medium-level (
η
2
The Present Study
• RQs
– Do Japanese EFL learners show sensitivity to
– Do Japanese EFL learners show sensitivity to
violations of semantic constraints on PAO in
their
offline performance
(paper-based
task)?
– Then, what about their
online performance
(real-time reading)?
The Present Study
• Design
– Experiment 1 (Offline)
– Experiment 1 (Offline)
• Twenty-two highly proficient Japanese EFL
learners (group I) took part in the paper-based
order selection task
– Experment 2 (Online)
• Other twenty-four learners (group II) took part in
• Other twenty-four learners (group II) took part in
The Present Study
• Participants
– In Total 46 under-graduate/graduate students
– In Total 46 under-graduate/graduate students
in a university
TOEIC Score
Age
n
M
SD
M
SD
The Present Study
• Experiment 1
– Paper-based task
– Paper-based task
• Two versions (preferred and violation) of
decontextualized multiple adjective NPs
(Adj1+
Adj2+N) presented at the same time
• Participants were asked to
select the preferable
orders of adjectives
by the form of questionnaire
orders of adjectives
by the form of questionnaire
• 18 stimuli (9 preferred and 9 violated),
counter-balanced
The Present Study
• Experiment 2
– Self-paced reading on PCs
– Self-paced reading on PCs
• Presentation unit: one sentence
• Reading unit: word by word
• Method : Moving Windows
• TF questions (20%)
• Programmed by the author using HSP
• Programmed by the author using HSP
• Reading times were automatically recorded
The Present Study
• Stimuli
(K = 34)
– Two conditions
– Two conditions
• Preferred order
(k = 9)
• Violation
(k = 9)
– Filler (k = 16, almost 50%)
– Types (preferred vs violations)
– Types (preferred vs violations)
• VALUE-COLOR vs COLOR-VALUE
• VALUE-SIZE vs SIZE-VALUE
The Present Study
• Examples
– [VALUE]-[SIZE]
– [VALUE]-[SIZE]
•
○
The boy had a nice small cup in his house.
•
☓
The boy had a small nice cup in his house.
– [VALUE]-[COLOR]
•
○
The woman drank the nice red wine in the
restaurant.
•
☓
The woman drank the red nice wine in the
•
☓
The woman drank the red nice wine in the
restaurant.
– [VALUE]-[COLOR]
•
○
The girl brought a small red flower from the yard.
The Present Study
• Interest regions and structure
– The man
A-2
saw
A-1
a
A
little
B
blue
B+1
bird
B+2
in the park.
– Structures and Vocabulary levels (under 000 level, in
– Structures and Vocabulary levels (under 000 level, in
One-sample t-test
Hypothesis: true mean is
greater than 0.5 (the chance level)
t(21) = 11.20, p < .001, d = 2.38
M
0.81
81%
19%
0.60 0.70 0.80 0.90 1.00M
0.81
SD
0.13
95%CI
[0.75, 0.86]
81%
Preferred Violation 0.00 0.10 0.20 0.30 0.40 0.50 0.601 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Results
• Offline performance
– Quite high ratio for correct selection
– Quite high ratio for correct selection
– Most of the learners may have attained
native-like semantic representations and
400
500
600
0
100
200
300
R
e
a
d
in
g
t
im
e
(
m
s)
Preferred order
Violation
0
A-2
A-1
A
B
B+1
B+2
saw
a
little/blue
blue/little
bird
in
400
500
600
t
(22) = 0.37
0
100
200
300
R
e
a
d
in
g
t
im
e
(
m
s)
Preferred order
Violation
t
(22) = 0.37
p
= .74
d
= 0.06
0
A-2
A-1
A
B
B+1
B+2
saw
a
little/blue
blue/little
bird
in
400
500
600
t
(22) = 1.41
0
100
200
300
R
e
a
d
in
g
t
im
e
(
m
s)
Preferred order
Violation
t
(22) = 1.41
p
= .17
d
= 0.34
0
A-2
A-1
A
B
B+1
B+2
saw
a
little/blue
blue/little
bird
in
400
500
600
t
(22) = 0.16
0
100
200
300
R
e
a
d
in
g
t
im
e
(
m
s)
Preferred order
Violation
t
(22) = 0.16
p
= .87
d
= 0.04
0
A-2
A-1
A
B
B+1
B+2
saw
a
little/blue
blue/little
bird
in
400
500
600
0
100
200
300
R
e
a
d
in
g
t
im
e
(
m
s)
Preferred order
Violation
0
A-2
A-1
A
B
B+1
B+2
saw
a
little/blue
blue/little
bird
in
d in g t im e ( m s ) 0 0 5 0 0 6 0 0 7 0 0 R e a d in 2 0 0 3 0 0 4 0 0
A_V A_P B_V B_P Bplus1_V Bplus1_P
Preferred order Violation
Region
Preferred order Violation
Diff. t p d
M SD 95%CI M SD 95%CI
A Adj1 403 98 [362, 444] 397 93 [358, 436] 6 0.37 0.74 0.06
B Adj2 427 61 [401, 453] 456 110 [410, 502] 29 1.41 0.17 0.34
d in g t im e ( m s ) 0 0 5 0 0 6 0 0 7 0 0 R e a d in 2 0 0 3 0 0 4 0 0
A_V A_P B_V B_P Bplus1_V Bplus1_P
Preferred order Violation
Region
Preferred order Violation
Diff. t p d
M SD 95%CI M SD 95%CI
A Adj1 403 98 [362, 444] 397 93 [358, 436] 6 0.37 0.74 0.06
B Adj2 427 61
[401, 453]
456 110[410, 502]
29 1.41 0.170.34
d in g t im e ( m s ) 0 0 5 0 0 6 0 0 7 0 0 R e a d in 2 0 0 3 0 0 4 0 0
A_V A_P B_V B_P Bplus1_V Bplus1_P
Preferred order Violation
Region
Preferred order Violation
Diff. t p d
M SD 95%CI M SD 95%CI
A Adj1 403 98 [362, 444] 397 93 [358, 436] 6 0.37 0.74 0.06
B Adj2 427 61
[401, 453]
456 110[410, 502]
29 1.41 0.170.34
400
600
P
re
fe
rr
e
d
O
rd
e
r
A
600
d
O
rd
e
r
600
e
d
O
rd
e
r
200
200
400
600
P
re
fe
rr
Violation
B
B+1r
= .57
200
400
200
400
600
P
re
fe
rr
e
d
O
rd
e
r
Violation
200
400
200
400
600
P
re
fe
rr
e
d
O
rd
e
Violation
400
600
P
re
fe
rr
e
d
O
rd
e
r
A
Violations were
Preferred orders
were read
slower
600
d
O
rd
e
r
600
e
d
O
rd
e
r
200
200
400
600
P
re
fe
rr
Violation
B
B+1Violations were
read slower
r
= .57
200
400
200
400
600
P
re
fe
rr
e
d
O
rd
e
r
Violation
200
400
200
400
600
P
re
fe
rr
e
d
O
rd
e
Violation
400
600
P
re
fe
rr
e
d
O
rd
e
r
A
Violations were
Preferred orders
were read
slower
600
d
O
rd
e
r
600
e
d
O
rd
e
r
200
200
400
600
P
re
fe
rr
Violation
B
B+1Violations were
read slower
r
= .57
200
400
200
400
600
P
re
fe
rr
e
d
O
rd
e
r
Violation
200
400
200
400
600
P
re
fe
rr
e
d
O
rd
e
Violation
Results
• Online performance
– No statistically significant delay
in reading
– No statistically significant delay
in reading
times
– L2 learners are not always capable of utilizing
semantic information in their real-time reading
comprehension like native users
– Against their high proficiency,
their implicit
– Against their high proficiency,
their implicit
Discussion
• Proficiency matters?
– Very likely
, since the effect size was small, and
– Very likely
, since the effect size was small, and
also showed a certain tendency in their reading
times
(cf. Hopp, 2006)
Discussion
• L1 Influence?
– In Japanese, there is no strict rules for adjective
– In Japanese, there is no strict rules for adjective
ordering
– And unfortunately, not “purely semantic”
equivalent
• Akaku-te, okina hako / okikute, akai hako
Discussion
• Where difficulty lies?
– Semantic class deficit?
– Semantic class deficit?
• Semantic transfer
(Jiang, 2000)
– Null representation even in L1 (Japanese)?
– imperfect transfer?
• Cross-language priming experiment will be need in future
studies
– No Constraints?
– No Constraints?
• If the semantic classes were evident in L2 …
– Computational difficulty for ordering?
• WM
Conclusion
• Limitations and future studies
– Data with Higher proficiency
– Data with Higher proficiency
– Comparison with other L1s
– Cross-language priming
– Other tasks and methodology
• ERP
• ERP
Conclusion
Offline performance
(explicit knowledge)
Online performance
(implicit knowledge)
(explicit knowledge)
(implicit knowledge)
Syntactic
○
?
Morphological
○
?
Lexical-semantic
Conclusion
Offline performance
(explicit knowledge)
Online performance
(implicit knowledge)
(explicit knowledge)
(implicit knowledge)
Syntactic
○
?
Morphological
○
?
Lexical-semantic
References
• Frenck-Mestre, C. (2002). An on-line look at sentence processing in the second language. In R. R. Heredia & J. Altarriba (Eds.), Bilingual sentence processing (pp. 217–236). Amsterdam: Elsevier.
• Hopp, H. (2006). Syntactic features and reanalysis in near-native processing. Second • Hopp, H. (2006). Syntactic features and reanalysis in near-native processing. Second
Language Research, 22, 369–397.
• Hopp, H. (2010). Ultimate attainment in L2 inflexion: Performance similarities between non-native and non-native speakers. Lingua, 120, 901–931.
• Clahsen, H., & Felser, C. (2006). Grammatical processing of language learners. Applied
Psycholinguistics, 27, 3–42.
• Danks, J. H., & Glucksberg, S. (1971). Psychological scaling of adjective orders. Journal of
Verbal Learning and Verbal Behavior, 10, 63-67.
• Danks, J. H., & Schwenk, M. A. (1974). Comprehension of prenominal adjective orders.
Memory & Cognition, 2, 34-38.
• Jackson, C. (2008). Proficiency level and the interaction of lexical and morphosyntactic information during L2 sentence processing. Language Learning, 58, 875–909.
information during L2 sentence processing. Language Learning, 58, 875–909.
• Jackson, C., & Bobb, S. C. (2009). The processing and comprehension of wh-questions among second language speakers of German. Applied Psycholinguistics, 30, 603–636. • Jackson C and Dussias PE (2009) Cross-linguistic differences and their impact on L2
sentence processing. Bilingualism: Language and Cognition,12, 65–82.
• Jiang, N. (2004). Morphological insensitivity in second language processing. Applied
References
• Jiang, N, (2000). Lexical representation and development in a second language. Applied
linguistics, 21, 47-77.
• Jiang, N. (2007). Selective integration of L2 knowledge in adult second language learning.
Language Learning, 57, 1–31. Language Learning, 57, 1–31.
• Jiang, N., Novokshanova, E., Masuda, K., & Wang, X. (2011). Morphological congruency and the acquisition of L2 morphemes. Language Learning, 61, 940–967.
• Keating, G. (2009). Sensitivity to violations of gender agreement in native and nonnative Spanish: An eye-movement investigation. Language Learning, 59, 503–535.
• Kennison, S. (2010). Processing prenominal adjectives during sentence comprehension.
Perceptual and Motor Skills, 111, 141–157.
• Kemmerer, D. (2000). Selective impairment of knowledge underlying prenominal adjective order: Evidence for the autonomy of grammatical semantics. Journal of
Neurolinguistics, 13, 57-82.
• Kemmerer, D., Tranel, D., & Zdanczyk, C. (2009). Knowledge of the semantic constraints on adjective order can be selectively impaired. Journal of neurolinguistics, 22, 91-108.
adjective order can be selectively impaired. Journal of neurolinguistics, 22, 91-108.
• Kusanagi, K. (2013). Second language learners' online processing of asymmetric formulaic sequences: A preliminary study focusing on English N and N phrases. LET Journal of Central
Japan, 24, 15-24.
• Kusanagi, K., Leung, C. Y., Bando, T., Fukuta, J., & Sugiura, M. (2013). L2 learners’ online insensitivity to malformed collocations: A study using eye tracking and self-paced reading tasks. Kyklos: International communication, 10, 49-64.
• Martin, J. E. (1969). Semantic determinants of preferred adjective order. Journal of Verbal
References
•
Martin, J. E. (1969). Semantic determinants of preferred adjective order.
Journal of Verbal Learning and Verbal Behavior, 8, 697-704.
•
Marinis, T., Roberts, L., Felser, C., & Clahsen, H. (2005). Gaps in second
•
Marinis, T., Roberts, L., Felser, C., & Clahsen, H. (2005). Gaps in second
language sentence processing. Studies in Second Language
Acquisition, 27, 53–78.
•
Sato, M., & Felser, C. (2010). Sensitivity to morphosyntactic violations in
English as a second language. Second Language, 9, 101–118.
•
Sabourin, L., & Stowe, L. A. (2008). Second language processing: When are
first and second languages processed similarly? Second Language
Research, 24, 397–430.
•
Felser, C., Roberts, L., Marinis, T., & Gross, R. (2003). The processing of
ambiguous sentences by first and second language learners of English.
Applied Psycholinguistics, 24: 453–489.
Applied Psycholinguistics, 24: 453–489.
•
Wulff, S. (2003). A multifactorial corpus analysis of adjective order in English.
International Journal of Corpus Linguistics, 8, 245-282.
•
Tolentino, L. C., & Tokowicz, N. (2011). Across languages, space, and time: A
review of the role of cross-language similarity in L2 (morpho)syntactic
processing as revealed by fMRI and ERP methods. Studies in Second
KUSANAGI Kunihiro
KUSANAGI Kunihiro
Mean difference (ms)
d
95%CI for
95%CI for d
Mean difference (ms)
d
95%CI for
mean difference
95%CI for d
Region B
29
0.34
[-27, 87]
[(-).30, .98]
Effect size (d) Power (1-β) Required sample size for rejecting the null hypothesis at α = .05,
attained effect size, attained effect size,
power = .80
Region A (Adj1)
0.06
.059
3,612
Region B (Adj2)
0.34
.358
115
Effect size
d
Min
-0.66
1st Qu.
0.18
Median
0.35
M
0.35
3rd Qu.
0.51
Max
1.42
Bootstrapping 95CI for effect size
d
(10,000 trials, subset
n
= 24) at
region B (the main interest region)
0 .5 1 .0 1 .5 s c o re 0 .3 5 0 .4 0 0 .4 5 s c o re e q u e n c
y 10
0 0 1 5 0 0
95%CI
[-0.12, 0.81]
-0 .5 0 .0 s c
0 2000 6000 10000
0 .2 5 0 .3 0 s c score fr e q
-0.5 0.0 0.5 1.0 1.5
0
5
0