KUSANAGI JASELE2013 slides

Measuring Japanese EFL

Measuring Japanese EFL

Learners’ Implicit Knowledge

of

Semantic Constraints

A Case of English Prenominal Adjective Orders

My slide is available at

https://sites.google.com/site/kusanagikuni/home/cv

Japanese EFL learners showed …

1.

“moderate” sensitivity

to the

1.

“moderate” sensitivity

to the

violations of semantic constraints on

prenominal adjective orders in their

real-time reading comprehension

KUSANAGI Kunihiro

KUSANAGI Kunihiro

Background

• Differences between L1 & L2 sentence

processing

processing

– In general, L2 learners’ reading is …

• Slower

• Less accurate than that of L1 users

(Frank-mestre, 2002)

– L2 learners’ reading often shows …

• Different preferences

for syntactic ambiguity

• Different preferences

for syntactic ambiguity

resolution from that of L1 users

(e.g., Witzel, Witzel, Nicol, 2012)

• None of preferences

(e.g., Clahsen & Felser, 2006; Marinins.

Roberts, Felser, Clahsen, 2005; Felser, Roberts, Marinis, Gross, 2003)

• Serious online insensitivity

to agreement errors

Background

• Shallow Structure Hypothesis

(SSH) by

Clahsen and Felser (2006)

Clahsen and Felser (2006)

– L2 grammatical processing is a fundamentally

different mechanism from L1

• L2 learners can utilize

lexical-semantic

representations

during real-time comprehension

like L1 users

like L1 users

• L2 learners’

syntactic and morphological

Background

• Some of other claims

– L2 processing is

identical to that of L1

– L2 processing is

identical to that of L1

(e.g., Jackson & Dussias, 2009)

– Rather,

more detailed

(Witzel, Witzel, Nicol, 2012)

• Usual suspects

– A problem of

proficiency

(e.g., Hopp, 2006, 2010; Jackson, 2008)

– L1

influence

(e.g., Jiang, et al. 2011; Sabourin & Stowe, 2008)

– L1

influence

(e.g., Jiang, et al. 2011; Sabourin & Stowe, 2008)

– Working memory

-based accounts

(e.g., Coughlin &

Tremblay, 2012)

Background

Offline performance

(explicit knowledge)

Online performance

(implicit knowledge)

(explicit knowledge)

(implicit knowledge)

Syntactic

○

☓

Morphological

○

☓

Lexical-semantic

Background

Offline performance

(explicit knowledge)

Online performance

(implicit knowledge)

(explicit knowledge)

(implicit knowledge)

Syntactic

○

○

Morphological

○

○

Lexical-semantic

Background

Offline performance

(explicit knowledge)

Online performance

(implicit knowledge)

(explicit knowledge)

(implicit knowledge)

Syntactic

○

？

Morphological

○

？

Lexical-semantic

Background

Offline performance

(explicit knowledge)

Online performance

(implicit knowledge)

(explicit knowledge)

(implicit knowledge)

Syntactic

○

？

Morphological

○

？

Lexical-semantic

○

○

Background

• Relative reliance of semantic or pragmatic

cues

in L2 processing

cues

in L2 processing

– When information of various grammatical

domains competes in L2 learners’ real-time

comprehension, semantic and pragmatic ones

are dominant among others

(e.g., evidence from

attachment ambiguity or garden path studies)

→

No dispute!

attachment ambiguity or garden path studies)

→

No dispute!

– However,

it doesn’t necessarily imply L2

Background

• L2 learners’ online sensitivity to

purely

semantic

constraints

in the absence of

semantic

constraints

in the absence of

effects from other domains?

We need some linguistic phenomena

• Non-syntactic / non-structural

• Non-morphological

• Relatively context independent

• Relatively context independent

Prenominal Adjective Orders

• In English, when multiple prenominal

adjectives modify a noun, there are linear

adjectives modify a noun, there are linear

procedures of semantic classes to apply

(see,

Martin, 1969; Wulff, 2003 for detailed linguistic descriptions)

Semantic

Semantic

Class

[VALUE] [SIZE] [DIMMENTION] [PHYS.PROP] [COLOR]

Example

Good

Big

Tall

Hot

Red

Prenominal Adjective Orders

a.

A nice

_[VALUE]

small

_[SIZE]

cup

b.

??

A small

_[SIZE]

nice

_[VALUE]

cup

b.

??

A small

_[SIZE]

nice

_[VALUE]

cup

Prenominal Adjective Orders

• Note that violations of those semantic

constraints on PAO strongly affect both of L1

constraints on PAO strongly affect both of L1

users’…

• Off-line acceptability ratings

(Danks & Glucksberg, 1971;

Danks & Schwenk, 1974)

• Time courses of reading comprehension

(Kenninson, 2010)

• The effect sizes was medium-level (

η

2

• The effect sizes was medium-level (

η

2

The Present Study

• RQs

– Do Japanese EFL learners show sensitivity to

– Do Japanese EFL learners show sensitivity to

violations of semantic constraints on PAO in

their

offline performance

(paper-based

task)?

– Then, what about their

online performance

(real-time reading)?

The Present Study

• Design

– Experiment 1 (Offline)

– Experiment 1 (Offline)

• Twenty-two highly proficient Japanese EFL

learners (group I) took part in the paper-based

order selection task

– Experment 2 (Online)

• Other twenty-four learners (group II) took part in

• Other twenty-four learners (group II) took part in

The Present Study

• Participants

– In Total 46 under-graduate/graduate students

– In Total 46 under-graduate/graduate students

in a university

TOEIC Score

Age

n

M

SD

M

SD

The Present Study

• Experiment 1

– Paper-based task

– Paper-based task

• Two versions (preferred and violation) of

decontextualized multiple adjective NPs

(Adj1+

Adj2+N) presented at the same time

• Participants were asked to

select the preferable

orders of adjectives

by the form of questionnaire

orders of adjectives

by the form of questionnaire

• 18 stimuli (9 preferred and 9 violated),

counter-balanced

The Present Study

• Experiment 2

– Self-paced reading on PCs

– Self-paced reading on PCs

• Presentation unit: one sentence

• Reading unit: word by word

• Method : Moving Windows

• TF questions (20%)

• Programmed by the author using HSP

• Programmed by the author using HSP

• Reading times were automatically recorded

The Present Study

• Stimuli

(K = 34)

– Two conditions

– Two conditions

• Preferred order

(k = 9)

• Violation

(k = 9)

– Filler (k = 16, almost 50%)

– Types (preferred vs violations)

– Types (preferred vs violations)

• VALUE-COLOR vs COLOR-VALUE

• VALUE-SIZE vs SIZE-VALUE

The Present Study

• Examples

– [VALUE]-[SIZE]

– [VALUE]-[SIZE]

•

○

The boy had a nice small cup in his house.

•

☓

The boy had a small nice cup in his house.

– [VALUE]-[COLOR]

•

○

The woman drank the nice red wine in the

restaurant.

•

☓

The woman drank the red nice wine in the

•

☓

The woman drank the red nice wine in the

restaurant.

– [VALUE]-[COLOR]

•

○

The girl brought a small red flower from the yard.

The Present Study

• Interest regions and structure

– The man

_A-2

saw

_A-1

a

_A

little

_B

blue

_B+1

bird

_B+2

in the park.

– Structures and Vocabulary levels (under 000 level, in

– Structures and Vocabulary levels (under 000 level, in

One-sample t-test

Hypothesis: true mean is

greater than 0.5 (the chance level)

t(21) = 11.20, p < .001, d = 2.38

M

0.81

81%

19%

0.60 0.70 0.80 0.90 1.00

M

0.81

SD

0.13

95%CI

[0.75, 0.86]

81%

Preferred Violation 0.00 0.10 0.20 0.30 0.40 0.50 0.60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Results

• Offline performance

– Quite high ratio for correct selection

– Quite high ratio for correct selection

– Most of the learners may have attained

native-like semantic representations and

400

500

600

0

100

200

300

R

e

a

d

in

g

t

im

e

(

m

s)

Preferred order

Violation

0

A-2

A-1

A

B

B+1

B+2

saw

a

little/blue

blue/little

bird

in

400

500

600

t

(22) = 0.37

0

100

200

300

R

e

a

d

in

g

t

im

e

(

m

s)

Preferred order

Violation

t

(22) = 0.37

p

= .74

d

= 0.06

0

A-2

A-1

A

B

B+1

B+2

saw

a

little/blue

blue/little

bird

in

400

500

600

t

(22) = 1.41

0

100

200

300

R

e

a

d

in

g

t

im

e

(

m

s)

Preferred order

Violation

t

(22) = 1.41

p

= .17

d

= 0.34

0

A-2

A-1

A

B

B+1

B+2

saw

a

little/blue

blue/little

bird

in

400

500

600

t

(22) = 0.16

0

100

200

300

R

e

a

d

in

g

t

im

e

(

m

s)

Preferred order

Violation

t

(22) = 0.16

p

= .87

d

= 0.04

0

A-2

A-1

A

B

B+1

B+2

saw

a

little/blue

blue/little

bird

in

400

500

600

0

100

200

300

R

e

a

d

in

g

t

im

e

(

m

s)

Preferred order

Violation

0

A-2

A-1

A

B

B+1

B+2

saw

a

little/blue

blue/little

bird

in

d in g t im e ( m s ) 0 0 5 0 0 6 0 0 7 0 0 R e a d in 2 0 0 3 0 0 4 0 0

A_V A_P B_V B_P Bplus1_V Bplus1_P

Preferred order Violation

Region

Preferred order Violation

Diff. t p d

M SD 95%CI M SD 95%CI

A Adj1 403 98 [362, 444] 397 93 [358, 436] 6 0.37 0.74 0.06

B Adj2 427 61 [401, 453] 456 110 [410, 502] 29 1.41 0.17 0.34

d in g t im e ( m s ) 0 0 5 0 0 6 0 0 7 0 0 R e a d in 2 0 0 3 0 0 4 0 0

A_V A_P B_V B_P Bplus1_V Bplus1_P

Preferred order Violation

Region

Preferred order Violation

Diff. t p d

M SD 95%CI M SD 95%CI

A Adj1 403 98 [362, 444] 397 93 [358, 436] 6 0.37 0.74 0.06

B Adj2 427 61

[401, 453]

456 110

[410, 502]

29 1.41 0.17

0.34

d in g t im e ( m s ) 0 0 5 0 0 6 0 0 7 0 0 R e a d in 2 0 0 3 0 0 4 0 0

A_V A_P B_V B_P Bplus1_V Bplus1_P

Preferred order Violation

Region

Preferred order Violation

Diff. t p d

M SD 95%CI M SD 95%CI

A Adj1 403 98 [362, 444] 397 93 [358, 436] 6 0.37 0.74 0.06

B Adj2 427 61

[401, 453]

456 110

[410, 502]

29 1.41 0.17

0.34

400

600

P

re

fe

rr

e

d

O

rd

e

r

A

600

d

O

rd

e

r

600

e

d

O

rd

e

r

200

200

400

600

P

re

fe

rr

Violation

B

_B+1

r

= .57

200

400

200

400

600

P

re

fe

rr

e

d

O

rd

e

r

Violation

200

400

200

400

600

P

re

fe

rr

e

d

O

rd

e

Violation

400

600

P

re

fe

rr

e

d

O

rd

e

r

A

Violations were

Preferred orders

were read

slower

600

d

O

rd

e

r

600

e

d

O

rd

e

r

200

200

400

600

P

re

fe

rr

Violation

B

_B+1

Violations were

read slower

r

= .57

200

400

200

400

600

P

re

fe

rr

e

d

O

rd

e

r

Violation

200

400

200

400

600

P

re

fe

rr

e

d

O

rd

e

Violation

400

600

P

re

fe

rr

e

d

O

rd

e

r

A

Violations were

Preferred orders

were read

slower

600

d

O

rd

e

r

600

e

d

O

rd

e

r

200

200

400

600

P

re

fe

rr

Violation

B

_B+1

Violations were

read slower

r

= .57

200

400

200

400

600

P

re

fe

rr

e

d

O

rd

e

r

Violation

200

400

200

400

600

P

re

fe

rr

e

d

O

rd

e

Violation

Results

• Online performance

– No statistically significant delay

in reading

– No statistically significant delay

in reading

times

– L2 learners are not always capable of utilizing

semantic information in their real-time reading

comprehension like native users

– Against their high proficiency,

their implicit

– Against their high proficiency,

their implicit

Discussion

• Proficiency matters?

– Very likely

, since the effect size was small, and

– Very likely

, since the effect size was small, and

also showed a certain tendency in their reading

times

(cf. Hopp, 2006)

Discussion

• L1 Influence?

– In Japanese, there is no strict rules for adjective

– In Japanese, there is no strict rules for adjective

ordering

– And unfortunately, not “purely semantic”

equivalent

• Akaku-te, okina hako / okikute, akai hako

Discussion

• Where difficulty lies?

– Semantic class deficit?

– Semantic class deficit?

• Semantic transfer

(Jiang, 2000)

– Null representation even in L1 (Japanese)?

– imperfect transfer?

• Cross-language priming experiment will be need in future

studies

– No Constraints?

– No Constraints?

• If the semantic classes were evident in L2 …

– Computational difficulty for ordering?

• WM

Conclusion

• Limitations and future studies

– Data with Higher proficiency

– Data with Higher proficiency

– Comparison with other L1s

– Cross-language priming

– Other tasks and methodology

• ERP

• ERP

Conclusion

Offline performance

(explicit knowledge)

Online performance

(implicit knowledge)

(explicit knowledge)

(implicit knowledge)

Syntactic

○

？

Morphological

○

？

Lexical-semantic

Conclusion

Offline performance

(explicit knowledge)

Online performance

(implicit knowledge)

(explicit knowledge)

(implicit knowledge)

Syntactic

○

？

Morphological

○

？

Lexical-semantic

References

• Frenck-Mestre, C. (2002). An on-line look at sentence processing in the second language. In R. R. Heredia & J. Altarriba (Eds.), Bilingual sentence processing (pp. 217–236). Amsterdam: Elsevier.

• Hopp, H. (2006). Syntactic features and reanalysis in near-native processing. Second • Hopp, H. (2006). Syntactic features and reanalysis in near-native processing. Second

Language Research, 22, 369–397.

• Hopp, H. (2010). Ultimate attainment in L2 inflexion: Performance similarities between non-native and non-native speakers. Lingua, 120, 901–931.

• Clahsen, H., & Felser, C. (2006). Grammatical processing of language learners. Applied

Psycholinguistics, 27, 3–42.

• Danks, J. H., & Glucksberg, S. (1971). Psychological scaling of adjective orders. Journal of

Verbal Learning and Verbal Behavior, 10, 63-67.

• Danks, J. H., & Schwenk, M. A. (1974). Comprehension of prenominal adjective orders.

Memory & Cognition, 2, 34-38.

• Jackson, C. (2008). Proficiency level and the interaction of lexical and morphosyntactic information during L2 sentence processing. Language Learning, 58, 875–909.

information during L2 sentence processing. Language Learning, 58, 875–909.

• Jackson, C., & Bobb, S. C. (2009). The processing and comprehension of wh-questions among second language speakers of German. Applied Psycholinguistics, 30, 603–636. • Jackson C and Dussias PE (2009) Cross-linguistic differences and their impact on L2

sentence processing. Bilingualism: Language and Cognition,12, 65–82.

• Jiang, N. (2004). Morphological insensitivity in second language processing. Applied

References

• Jiang, N, (2000). Lexical representation and development in a second language. Applied

linguistics, 21, 47-77.

• Jiang, N. (2007). Selective integration of L2 knowledge in adult second language learning.

Language Learning, 57, 1–31. Language Learning, 57, 1–31.

• Jiang, N., Novokshanova, E., Masuda, K., & Wang, X. (2011). Morphological congruency and the acquisition of L2 morphemes. Language Learning, 61, 940–967.

• Keating, G. (2009). Sensitivity to violations of gender agreement in native and nonnative Spanish: An eye-movement investigation. Language Learning, 59, 503–535.

• Kennison, S. (2010). Processing prenominal adjectives during sentence comprehension.

Perceptual and Motor Skills, 111, 141–157.

• Kemmerer, D. (2000). Selective impairment of knowledge underlying prenominal adjective order: Evidence for the autonomy of grammatical semantics. Journal of

Neurolinguistics, 13, 57-82.

• Kemmerer, D., Tranel, D., & Zdanczyk, C. (2009). Knowledge of the semantic constraints on adjective order can be selectively impaired. Journal of neurolinguistics, 22, 91-108.

adjective order can be selectively impaired. Journal of neurolinguistics, 22, 91-108.

• Kusanagi, K. (2013). Second language learners' online processing of asymmetric formulaic sequences: A preliminary study focusing on English N and N phrases. LET Journal of Central

Japan, 24, 15-24.

• Kusanagi, K., Leung, C. Y., Bando, T., Fukuta, J., & Sugiura, M. (2013). L2 learners’ online insensitivity to malformed collocations: A study using eye tracking and self-paced reading tasks. Kyklos: International communication, 10, 49-64.

• Martin, J. E. (1969). Semantic determinants of preferred adjective order. Journal of Verbal

References

•

Martin, J. E. (1969). Semantic determinants of preferred adjective order.

Journal of Verbal Learning and Verbal Behavior, 8, 697-704.

•

Marinis, T., Roberts, L., Felser, C., & Clahsen, H. (2005). Gaps in second

•

Marinis, T., Roberts, L., Felser, C., & Clahsen, H. (2005). Gaps in second

language sentence processing. Studies in Second Language

Acquisition, 27, 53–78.

•

Sato, M., & Felser, C. (2010). Sensitivity to morphosyntactic violations in

English as a second language. Second Language, 9, 101–118.

•

Sabourin, L., & Stowe, L. A. (2008). Second language processing: When are

first and second languages processed similarly? Second Language

Research, 24, 397–430.

•

Felser, C., Roberts, L., Marinis, T., & Gross, R. (2003). The processing of

ambiguous sentences by first and second language learners of English.

Applied Psycholinguistics, 24: 453–489.

Applied Psycholinguistics, 24: 453–489.

•

Wulff, S. (2003). A multifactorial corpus analysis of adjective order in English.

International Journal of Corpus Linguistics, 8, 245-282.

•

Tolentino, L. C., & Tokowicz, N. (2011). Across languages, space, and time: A

review of the role of cross-language similarity in L2 (morpho)syntactic

processing as revealed by fMRI and ERP methods. Studies in Second

KUSANAGI Kunihiro

KUSANAGI Kunihiro

Mean difference (ms)

d

95%CI for

95%CI for d

Mean difference (ms)

d

95%CI for

mean difference

95%CI for d

Region B

29

0.34

[-27, 87]

[(-).30, .98]

Effect size (d) Power (1-β) Required sample size for rejecting the null hypothesis at α = .05,

attained effect size, attained effect size,

power = .80

Region A (Adj1)

0.06

.059

3,612

Region B (Adj2)

0.34

.358

115

Effect size

d

Min

-0.66

1st Qu.

0.18

Median

0.35

M

0.35

3rd Qu.

0.51

Max

1.42

Bootstrapping 95CI for effect size

d

(10,000 trials, subset

n

= 24) at

region B (the main interest region)

0 .5 1 .0 1 .5 s c o re 0 .3 5 0 .4 0 0 .4 5 s c o re e q u e n c

y 10

0 0 1 5 0 0

95%CI

[-0.12, 0.81]

-0 .5 0 .0 s c

0 2000 6000 10000

0 .2 5 0 .3 0 s c score fr e q

-0.5 0.0 0.5 1.0 1.5

0

5

0