Factors Affecting Cloze Test Performance

著者名(英) Daniel Jenks journal or

publication title

神田外語大学紀要

volume 23

page range 315‑333

year 2011‑03

URL http://id.nii.ac.jp/1092/00000596/

asKUIS 著作権ポリシーを参照のこと

315

TEST PERFORMANCE

Daniel Jenks

1. INTRODUCTION

!"

#

$

  The resulting data showed that the scoring methods used had little effect on

%

&

'

2. PREVIOUS CLOZE TEST RESEARCH

2.1 The cloze test

  "

#

#(

316

)

*)

*)'+,--,./0*2 )

!4"7*

)+!8"7*)9:;;/.<:*7 #

#)":;/;.,,-*"

)=,-->.:;:*

2.2 What do cloze tests measure?

  "? @A )"7:;;,.<B0*

)9,--,.>;*#

?##

C%A)E,--:.>;>*&

believed that these skills are the same as those that language users would utilise in )"7:;;,.<B0*&

second language learners highlight the correspondence between their knowledge of )"#:;//.BF*

  +

)":;;,.,,F*=

>:/

)*

)G :;;-./,* &

? A )"

7:;;,.<B;*

  %

H )"#

:;//.B<* ?7A ?A )G :;;-./>*(#

# # )I ,--,.F/,*=@

#)*

2.3 Methods of constructing cloze tests  

)"

7 :;;,.</<*

K?

# A)E:;B;.>BF*=

@ )E:;0F.FF-*

? )

*A)=,-->.:;-*=

>:0

)9:;/,.:F:*

  !

H?

A )I ,--,.F0:*" ? A)9

&:;/:.>:F*?

A)"

7:;;,.</<*

2.4 Methods of marking cloze tests

  8 !4"7 +!8"7 )+ # :;/<.,<:* L

@+!8"7?

A)E:;B;.>B-*

  ##

+!8"7

?A)E:;0-.>::*

#)"#:;//.B:*&

# M, # )9:;/,.:F:*&

!4"7+!8"7

)I,--,.F/B*H

>:;

+!8"7

#)E:;0,.BB*#

)

*

2.5 Reliability and validity of cloze tests

  " +!8"7 )9:;/,.:F:*+!8"7 )9:;;/.<,*&

%

% # )E :;;-.:,<*

# @

?A

  (

?

?)=,-->.:0;*

7 )":;/;.,,B*+!8"7 !4"7 )9 :;;/.<>*"@

#M,#)=

,-->.:0;*#?#

C#%#A)E:;;-.<0*

  The remainder of this article describes an experiment carried out using two

>,-

3. TEST METHOD

3.1 Cloze test text

  :B,

%

% # &

therefore represents a sample of the kind of writing that students at this level could L

&

# understanding the text and restoring its missing words using the limited knowledge )*

""

3.2 Cloze test construction

  &

)

*&

):,*

,F@)*

>,:

3.3 Test participants

  G L

!:::,9 %"

!!

3.4 Test procedure

  + :, ? A

!G

@)

*)

*

>,, 3.5 Marking procedure

  " # #

&#?A @ L )#

*N # ? A = # + L # L K # )*)

*

4. RESULTS

  :%

# :+!8"7

!4"7

Table 1 Mean scores for both groups in their recent term test and their cloze test Term test

(out of 50)

Cloze SEMAC (out of 25)

Cloze EXACT (out of 25) Test 1 30.67

SD = 13.71

8.63 SD = 6.44

6.37 SD = 4.82 Test 2 29.33

SD = 14.4

7.73 SD = 5.11

5.90

SD = 3.99

>,>

  & , !4"7

= answered than content items in either of the tests when using a single scoring

Table 2 Mean scores for both groups on function word items and content word items SEMAC scoring EXACT scoring

Test 1

Function items

(out of 10) 3.80 3.53

Content items

(out of 15) 4.83 2.83

Test 2

Function items

(out of 13) 3.87 3.57

Content items

(out of 12) 3.87 2.33

  7 % +!8"7 !4"7

>$

+%O#

( ,F :,=

$$8#)

*

>,<

Table 3 Correlations between SEMAC scores and EXACT scores Pearson product-moment correlation

Spearman’s All items Function

items

Content items Test 1

SEMAC : EXACT 0.98 0.99 0.91 0.98

Test 2

SEMAC : EXACT 0.98 0.98 0.89 0.95

  7 % <

= )

*

Table 4 Correlation between term test scores and cloze test scores Pearson product-moment correlation

Spearman’s All items Function

items

Content items Test 1

Term test : SEMAC 0.83 0.73 0.83 0.84

Test 1

Term test : EXACT 0.80 0.70 0.79 0.81

Test 2

Term test : SEMAC 0.66 0.55 0.68 0.58

Test 2

Term test : EXACT 0.65 0.56 0.68 0.59

>,F

  F

&) P% * +!8"7

!4"7

Table 5 Reliability estimates for the two cloze tests using both scoring methods Guttman’s split-half reliability estimate KR-20

reliability estimate Odd-numbered items :

Even-numbered items

Items 1-13 : Items 14-25

Test 1 SEMAC 0.92 0.94 0.92

EXACT 0.84 0.90 0.87

Test 2 SEMAC 0.89 0.69 0.87

EXACT 0.89 0.67 0.83

5. DISCUSSION

5.1 Performance on the two tests

= it is worthwhile considering the possible effects of the differences between the two

=%

:F-Q

# ),%@

*=:%

>,B

and the fact that the student who produced the text used in the test would have #

@

#

@

5.2 Validity of the two tests

  "

% ) <* : :

, )*

:%,)

@*

5.3 Reliability of the two tests

  F ) @ #* K P%

)E

>,/

:;;-.:/<* )9:;;/.<,*

P I',- @ )E:;;-.://*

I',-

5.4 Performance under the two scoring methods

  :% +!8"7 !4"7 ,

?A students’ answers to content items show greater variation; fewer of them will be

  7+!8"7!4"7

!4"7 )

*=#

+!8"7

>,0 5.5 Validity under the two scoring methods

  "+!8"7 !4"7 E $$8 +%

# "

+!8"7

%+

#+!8"7

5.6 Reliability under the two scoring methods

  )

*+!8"7)*

,#

P

6. CONCLUSIONS

  "

)*+!8"7 scoring method is the most appropriate should such tests be carried out with students

"#

?AL

>,;

)#

*

"

to a single group of students (at least without a considerable period of time between

%*

%

  " &

would be both level-appropriate for the test-takers (students at the same point in their

* @

@H#

@

#

REFERENCES

"'P77"):;;,*87 +."

&2$The Modern Language Journal/BV<<B0</;

"#IP):;//*K7 $9M$

TESOL Quarterly::V:F;B/

"G7):;/;*7 $$!H

>>-

MTESOL Quarterly:>V,,:;,,/

EMH):;0,*+7 +TESOL Quarterly :BV:B:/-

EMH):;0F*$7 H'' 2TESOL Quarterly:;V>F>FFFB

E M H ):;;-* Fundamental Considerations in Language Testing 9K$

E G ' ):;B;* H W 7 8 ' 7"Reading Research Quarterly<V>>F0>BF

E=2),---*Principles of Language Learning and Teaching)<

* M

EG2):;0-*'8H8+7 The Modern Language JournalB<V>>::>:/

=" ),-->* Testing for Language Teachers ),

* 7 K$

G G ):;;-*" 7.( 2 7 8X TESOL Quarterly,<V:B:0>

I8),--,*7 '.!&7 +"+8 The Modern Language Journal0BV<

F/:F0B

9 G ( ):;/,* + 8 2 M 7

$ ! + M The Modern Language JournalFBV>:F::F0

9 G( & N ):;/:*" 7 ! $ TESOL QuarterlyFV<>:F>,B

9 7 ' G ( + 7 $ ):;;/* Testing 7

331

!M+KE

' G + ' ),--,* Longman Dictionary of Language

>

M

+ G E # P ' ):;/<* 7 8 !

$The Modern Language JournalF0VF,>;,<:

APPENDIX A – Text used to produce cloze tests

  Hello, my name is Emiko. I come from Osaka, but now I live in Shizuoka with my family. They are my mother, my father, and my two sisters. My mother and father work in a shop. My sisters are elementary school students, but I am a junior high school student. My class is 1-6, and my teacher’s name is Mr.

Watanabe. He is an English teacher. Anna teaches English at my school too.

She is from Australia. I like English, but my favorite subject is P.E..

  I like sports very much. I play tennis and soccer at school and with my friends. I’m a member of the volleyball club, so I practice volleyball on Saturdays and Sundays. I don’t like swimming very much.

  I arrived in Shizuoka three years ago. I have lots of friends here now. I like Shizuoka because it has lots of nice mountains and beaches. It also has shops, schools and libraries, so it is an interesting place.

APPENDIX B – Cloze test 1

 Hello, my name is Emiko. I come from Osaka, but now I live in Shizuoka with my family. They are my mother, ( ) father, and my two ( ).

My mother and father ( ) in a shop. My ( ) are elementary school

>>,

students, ( ) I am a junior ( ) school student. My class ( ) 1-6, and my teacher’s ( ) is Mr. Watanabe. He ( ) an English teacher. Anna ( ) English at my school ( ). She is from Australia.

( ) like English, but my ( ) subject is P.E..

 I ( ) sports very much. I ( ) tennis and soccer at ( ) and with my friends. ( ) a member of the ( ) club, so I practice ( ) on Saturdays and Sundays. ( ) don’t like swimming very ( ).

  I arrived in Shizuoka ( ) years ago. I have ( ) of friends here now. ( ) like Shizuoka because it ( ) lots of nice mountains and beaches. It also has shops, schools and libraries, so it is an interesting place.

APPENDIX C – Cloze test 2

  Hello, my name is Emiko. I come from Osaka, but now I live in Shizuoka with my family. They ( ) my mother, my father, ( ) my two sisters.

My ( ) and father work in ( ) shop. My sisters are ( ) school students, but I ( ) a junior high school ( ). My class is 1-6, ( ) my teacher’s name is ( ) Watanabe. He is an ( ) teacher. Anna teaches English ( ) my school too. She ( ) from Australia. I like ( ), but my favorite subject ( ) P.E..

 I like sports ( ) much. I play tennis ( ) soccer at school and ( )

my friends. I’m a ( ) of the volleyball club, ( ) I practice volleyball

on ( ) and Sundays. I don’t ( ) swimming very much.

333

  I ( ) in Shizuoka three years ( ). I have lots of ( ) here now.

I like ( ) because it has lots of nice mountains and beaches. It also has

shops, schools and libraries, so it is an interesting place.