Kevin Mueller，”CEFR-J Self-assessment with Japanese First-year University Students,“ 言語学習と教育言語学2017年度版,pp.85-96,
CEFR-J Self-assessment with Japanese First-year University Students
Kevin Mueller †
†Tokyo International University 1-13-1 Matoba-kita, Kawagoe, Saitama, 350-1197 Japan E-mail: [email protected]†
Abstract
This study, undertaken with 257 Japanese first-year university students at a Japanese university, examines students’ self-assessment of their ability based on a portion of the CEFR-J can-do list. It was conducted at the outset of the year with students who are English Language Communication majors to examine their self-assessment of English ability prior to the beginning of university English courses. These students are enrolled in a program that has them take nine hours of compulsory English courses each week, primarily with non-Japanese nationality faculty members.
The findings reveal that, in general, more students stated they could do the framework proficiency skills at the lower end of the scale (in the A0 to A1 CEFR bands) while fewer self-assessed themselves as able to perform at the higher end of the scale. This study noted a few outlying outcomes, most notably, that more students responded they could perform at a higher level than at a lower level according to the CEFR-J rubric in two instances, and these apparent discrepancies are explained herein. Moreover, while the overall study has a high reliability coefficient as measured by the Cronbach alpha and the Mokken scale score, which measure the ranking order that a participant who answered positively to a more difficult question is assumed to answer an easier question appropriately, was rather low. These reliability factors are elucidated as well. The paper concludes with further considerations for future can-do self-assessment research employing the complete CEFR-J with Japanese university students.
Keywords CEFR，CEFR-J，self-assessment，self-evaluation
日本人大学 1 年生の CEFR-J 自己査定
ミューラー ケビン†
†東京国際大学埼玉県川越市的場北1-13-1〒350-1197 E-mail: †[email protected]
概要
日本で英語を学ぶ257名の大学1年生を対象に、本研究では、CEFR-Jのcan-doリストに基づき、学生の自分の能力に対する 自己査定に関して調査を行なった。この調査は、英語コミュニケーションを専攻する新入生が、学生が大学での英語の学びを始 める前の段階で、自分の英語能力に関する自己査定を調べるために入学時に実施された。これらの学生は主に日本人以外の教員 が担当する1週間あたり9時間の必修授業を受けるプログラムに登録していた。全体的に本調査を通して、多くの学生が 尺度 の下位となるCEFRの A0からA1の範囲で文章構造能力に関して出来たと述べる一方で、高い尺度で力を発揮できたと自己査 定した者は少なかった。さらに加えると、この研究では、とりわけCEFR-Jのルーブリックにおける2つの項目において、多く
の学生が下位のレベルよりも上位のレベルで力が発揮できるという、特異的な通常ではあり得ない結果も示しており、この明ら かなる矛盾点を本論文で明らかにしている。さらに本調査全体を通して、より難しい質問に正答できる対象者は、より易しい質 問には答えられるであろうとの想定のもとに順位を測定するCronbach alphaおよびMokken尺度法によって測定された高信頼係 数はかなり低かった。これらの信頼度因子も明瞭に説明した。本論文では、一貫したCEFR-J調査を日本人大学生に用い、今後
can-do自己査定研究により重点を置くことが肝要であるとの結論を得ることができた。
キーワード CEFR，CEFR-J，self-assessment，self-evaluation
1. Introduction
In 2001, after more than twenty years of research, the Council of Europe published The Common European Framework of Reference for Languages (CEFR) in both English and French. It is now published in forty languages, including Japanese and was “designed to provide a transparent, coherent and comprehensive basis for the elaboration of language syllabuses and curriculum guidelines, the design of teaching and learning materials, and the assessment of foreign language proficiency. It is used in Europe but also in other continents" [ 1 ] .
The CEFR outlines foreign language ability at six discrete levels: A1, A2, B1, B2, C1, and C2. In addition, it denotes three “plus” stages (A2+, B1+, B2+). With a foundation based on firsthand experimentation with second language learners and extensive input from linguists, educators and researchers, the framework, according to its writers, makes it possible: “t o e s t a b l i s h l e a r n i n g a n d te a c h i n g o b je c t i ve s , t o re v i e w c u r r i c u l a , t o de s i g n t e a c h i n g m a t e r i a l s a n d , to p r o vi d e a b a s i s f o r re c o g n i z i n g l a n g u a ge q u a l i f i c a t i o n s t h u s f a ci l i t a t i n g e d u ca t i o n a l a n d o c c u pa t i o n a l m o b i l i t y ” [2].
The CEFR document is far more comprehensive than merely containing a collection of “can-do” statements that outline discernable language skills that a learner is able to proficiently do. This paper will concentrate primarily on these “can-do” aspects of the CEFR in both it original form and the CEFR-J [3] (a framework conceived and organized solely for the English learning and teaching context in Japan) [4].
Within the original CEFR document (Council of Europe), two appendices focus on “can-do” statements.
The first, A p p e n d i x C, p r o vi d e s s e l f - a s s e s s m e n t s ta t e m e n t s a s we l l a s l a n g u a ge te s t s a n d f e e db a c k ; t h i s s y s te m i s i n p l a c e f o r l e a r n e rs , p r i m a ri l y t h o s e w h o a re i n d e pe n de n tl y s tu d y i n g o n e o f t h e f o l l o w i n g f o u r te e n E u ro p e a n l a n gu a g e s : D a n i s h , D u t c h , E n g l i s h , F i n n i s h , F r e n c h , Ge r m a n , G r e e k , I c e l a n d i c , I ri s h , I t a l i a n , N o r we g i a n , P o r t u g u e s e , S p a n i s h , a n d S w e d i s h [ 5 ] . T h e s e co n d, A p p e n di x D , s e e k s t o o u tl i n e “a s e t o f pe r f o rm a n ce- r e l a te d s ca l e s , de s c r i b i n g w h a t l e a r n e r s c a n a c t u a l l y do i n t h e f o re i g n l a n g u a g e ” [ 6 ]. T h e re a r e
“ c a n d o ” d e s c r i pto r s f o r e a c h o f t h e s i x l e ve l s . T h e s e “ ca n do ” de s c r i p to rs we r e c re a te d b y t h e A s s o ci a ti o n o f L a n g u a ge Te s te r s i n E u ro p e ( A LT E) .
W h i l e t h e C E F R w a s de s i g n e d f o r th e E u ro p e a n l a n g u a g e l e a rn i n g c o n t e x t , i t h a s e xp a n de d t h r o u g h o u t t h e wo r l d a s a n a l l - p u r p o s e , b ro a d co n s t r u c t f o r u n de r s t a n d i n g l a n g u a g e a b i l i t y f ro m t h e pe r s pe c t i v e s o f s tu d y i n g , i n s t r u c t i o n , a n d e v a l u a ti o n . To n o a n d N e gi s h i [ 7 ] a r g u e t h a t , b a s e d o n a n e i g h t - ye a r p e r i o d o f a n a l y s e s i n s i d e a n d o u t s i d e o f J a p a n , i m pl e m e n t i n g th e C E F R w i l l b e a k e y i n s t i g a to r i n th e t ra n s f o r m a t i o n o f E n gl i s h e d u c a ti o n i n J a pa n . N a g a i a n d O ’ D w y e r [ 8 ] co n c u r wi t h th e s e r e s e a r c h e r s a n d c l a i m t h a t t h e i m p l e m e n ta t i o n o f t h e C E F R r u b r i c i n J a pa n h a s b e e n g e n e ra l l y b e n e f i c i a l f o r l a n g u a g e e d u ca t i o n i n J a pa n . Three examples can be shown to illustrate how the CEFR has been applied in Japan at the governmental level.
First, Fenelly [9] points out that the December 13,
2013 “English Education Reform Plan
corresponding to Globalization” document explicitly mentions CEFR levels noting that junior high
school English education be at the A1-A2 levels and high school instruction be at the B1-B2 levels.
Second, t h e M i n i s t r y o f E du c a t i o n , C u l t u re , S po r t s , S c i e n ce , a n d Te c h n o l o g y ( M EX T ) r e l e a s e d a d o c u m e n t w i t h f i ve p ro po s i ti o n s a n d c l e a r g u i de l i n e s f o r i m p r o vi n g En g l i s h f o r w o rl d w i d e co m m u n i c a t i o n t h a t i n c o r po r a t e s m e a s u r i n g s t u de n ts ’ p r o f i ci e n c y w i t h a c a n - d o i n v e n t o r y, a s ci t e d b y To n o a n d N e gi s h i [ 1 0 ] . To n o a n d Ne g i s h i m a i n t a i n th a t th i s i n v e n to r y i s i n s pi r e d b y t h e C E F R . T h i r d , t h e a f o re m e n t i o n e d t w o r e s e a r c h e rs s ta t e th e N ih o n H o so K y o ka i ( N H K ) , t h e J a p a n B r o a d c a s t i n g C o r po r a t i o n , w h i c h i s , i n pa r t , f u n d e d b y t h e J a p a n e s e g o ve r n m e n t , h a s i m p l e m e n te d th e C E F R ru b r i c f o r t h e i r f o r e i g n l a n g u a g e t e x t b o o ks , a n d t e l e vi s i o n a n d r a di o p ro g r a m s .
A t t h e g o ve r n m e n ta l l e ve l , va r i o u s m e t h o d s o f i n c o r po r a t i n g C E F R a re o c c u r r i n g ; h o we v e r, Ne g i s h i ’s r e s e a r c h [ 11 ] r e ve a l e d e i gh t y pe r c e n t o f u n i ve r s i t y s tu d e n t s i n J a pa n a r e i n t h e A o r B b a n ds . T h u s , t h e o r i gi n a l C E F R A a n d B l e ve l s we r e a l t e re d to m o r e di s c r e te l y c l a s s i f y l e a r n e rs [ 1 2 ] ; A 1 , A 2 , B1 , B2 i n t h e o ri g i n a l w e re d i vi d e d u p i n t o n i n e g ro u p s ( A 1 . 1 , A 1 . 2 , A 1 . 3 , A 2 . 1 , A2 . 2 , B 1 . 1 , B 1 . 2 , B2 . 1 , B 2 . 2 ) . I n a dd i t i o n , a P re - A 1 s t a ge wa s i n co r p o ra t e d;
t h e r e f o re , t h e C E F R - J h a s t h e r e s u l t i n g t we l v e l e ve l s :
[ P re - A 1 ] , [ A 1 . 1 , A 1 . 2 , A 1 . 3 ] , [ A 2 . 1 , A 2 . 2 ] , [ B1 . 1 , B 1 . 2 ] , [ B 2 . 1 , B 2 . 2 ] , [ C 1 ] [ C 2 ]
2. Self-assessment Validity
F ro m a pe d a go g i c a l p e r s pe c t i ve , t h e r e a re a n u m b e r o f e x p l a n a t i o n s f o r e n c o u r a g i n g l a n g u a g e l e a rn e r s to c o n d u c t s e l f - a s s e s s m e n t . I n a l a n g u a ge l e a rn i n g c o n t e x t , Os k a r s s o n [ 1 3 ] e l u ci d a te s s i x re a s o n s w h y a l e a r n e r ’s s e l f - a s s e s s m e n t i s h e l p f u l . F i rs t , i t p r o vi d e s t h e l e a rn e r w i th e x p e ri e n c e i n a p p r a i s i n g p ro f i c i e n c y t h a t t h e n a i d s i n l e a r n i n g de v e l o pm e n t . Se c o n d , l e a r n e r s a n d i n s t r u c to r s g a i n a n i n c r e a s e d a p p re c i a ti o n o f di s t i n g u i s h i n g de g r e e s o f p ro f i ci e n c y. T h i r d , i t s ti m u l a te s a l e a r n e r t o f o c u s o n f u r t h e r l e a r n i n g o b j e c ti v e s . F o u r t h , a te a c h e r c a n
p re s e n t s t u de n t s w i th a v a r i e t y o f s e l f - m e a s u r e m e n t p r a c t i ce s . F i f t h , s t u de n t s h a v e o p po r t u n i t i e s to e n g a ge i n g a u g i n g th e i r o wn p r o f i ci e n c y. F i n a l l y, s t u de n t s c a n c a r r y t h e s e s e l f - a s s e s s m e n t s ki l l s w i t h t h e m a s t h e y c o n t i n u e s t u d yi n g o n ce a co u r s e i s co m pl e te d . A s a re s u l t o f b e i n g i n v o l ve d i n s e l f - a s s e s s m e n t , t h e l a n g u a ge l e a r n e r i s ca n b e a n a c ti v e p a r ti c i pa n t i n t h e l e a r n i n g p ro c e s s .
W h i l e b e i n g b e n e f i c i a l , s e l f - a s s e s s m e n t do e s h a v e a n u m b e r o f w e a k n e s s e s . A c c o r di n g to D u n n i n g, H e a t h a n d S u l s [ 1 4 ] , a f e w p s y c h o l o g i ca l f a c t o r s a r e a t wo r k t h a t re s u l t i n de f e c ti v e s e l f - a s s e s s m e n ts , a n d t h e y s o r t t h e m i n t o t wo m a i n t y p e s . F i rs t l y, i n a c c u r a t e s e l f - a s s e s s m e n t m a y o c c u r b e ca u s e pe o p l e u s u a l l y do n o t h a v e a l l t h e n e ce s s a r y i n f o r m a ti o n r e q u i re d to m a k e a co r r e c t a pp r a i s a l , a n d t h e y do n o t t a k e i n t o c o n s i d e ra t i o n w h a t i s u n k n o w n . S e co n dl y, i n a c c u r a te s e l f - a s s e s s m e n t o f t e n o c c u rs b e c a u s e p e o pl e d o n o t pa y c l o s e a t te n ti o n t o s i g n i f i c a n t a n d a d v a n t a ge o u s i n f o rm a t i o n w h i c h t h e y do h a v e . A n o t h e r r e l a te d d r a w b a c k o f s e l f - a s s e s s m e n t s i s t h a t s t u de n t s c a n de l i b e r a t e l y l i e w h e n a s s e s s i n g th e i r o w n s ki l l s [ 1 5 ] .
M o r e p e r t i n e n t t o th i s s tu d y, a n L 2 l e a r n i n g c o n t e x t , B l a n c h e a n d M e r i n o [ 1 6 ] , i n a m e t a - a n a l y s i s o f th e a c c u r a c y o f s e l f -a s s e s s m e n t , f o u n d t h a t a l e a r n e r ’s s e l f - a s s e s s m e n t s t r e t c h e d f r o m b e i n g r a t h e r a c c u ra t e t o ve r y a c c u ra t e . I n a n o t h e r m e t a - a n a l y s i s , R o s s [ 1 7 ] co r r o b o r a te d th e f i n d i n g s o f B l a n c h e a n d M e r i n o . H o w e v e r, t h e r e a r e a n u m b e r o f c a ve a ts t h a t re q u i r e e xp l a n a ti o n . K r u g e r a n d D u n n i n g [ 1 8 ] r e co g n i z e d t h a t pe o p l e w i th h i g h e r p ro f i ci e n c y te n de d to r a te th e m s e l v e s a s b e i n g l e s s p ro f i c i e n t w h i l e l e s s p ro f i ci e n t s t u de n t s te n de d t o r a t e t h e m s e l ve s m o re h i g h l y. I n r e s e a r c h do n e o n J a pa n e s e u n i v e rs i t y s tu d e n t s ’ E n gl i s h w r i ti n g a b i l i t y, M a t s u n o [ 1 9 ] e va l u a te d s tu d e n t s ’ s e l f- a s s e s s m e n t a n d i n s t r u c to r a s s e s s m e n t a n d f o u n d t h i s te n de n c y t h a t K r u g e r a n d D u n n i n g n o t e d a s w e l l : m o r e p ro f i c i e n t s tu d e n t s u n de r e s t i m a te d t h e i r a b i l i t y. M a t s u n o p o s i t s t h a t t h i s “ w a s
p ro b a b l y c a u s e d b y t h e t e n d e n c y o f m a n y J a p a n e s e t o di s pl a y a de g r e e o f m o d e s t y” [ 2 0 ] .
I n a d d i ti o n to t h e s e f a c t o r s , g e n d e r a pp e a r s t o a l s o po s s i b l y s k e w s e l f - a s s e s s m e n t r e s u l t s . I n a m e t a - a n a l y s i s s t u d y [ 2 1 ] de t e r m i n e d th a t ge n de r a f f e c t s s e l f - a s s e s s m e n t de p e n di n g o n th e s u b j e c t o r c o n t e n t b e i n g e x a m i n e d . I n t h e l a n g u a ge a r ts , f e m a l e s w e re s h o w n to h a v e h i g h e r s e l f - e f f i c a c y b e l i e f s ; h o we v e r, i n t h e f i e l d o f l a n g u a g e a r ts w h e n s t u de n ts o f c o m p a ra b l e p ro f i c i e n c y l e ve l s w e re a n a l y z e d , m a l e s o ve re s t i m a te d t h e m s e l v e s m o re t h a n f e m a l e s [ 2 2 ] . T h e r e a r e va r i o u s m e t h o d s f o r s t u d e n t s to a s s e s s t h e i r p ro f i ci e n c y t h e m s e l v e s i n t h e L 2 c o n t e x t , a n d o n e s t u d y b y B a c h m a n a n d P a l m e r [ 2 3 ] f o u n d t h a t t h e m o s t a c c u r a te q u e s t i o n t y p e a s k e d l e a r n e r s a b o u t t h e pe r c e i v e d d i f f i c u l t y w i th c e r t a i n f a c e ts o f E n gl i s h a s a n L 2 . T h e y n o t e d t h a t th e l e a s t e f f e c t i v e t y pe o f q u e s ti o n w a s t h e “ c a n-d o ” q u e s t i o n .
St u de n t s ’ s e l f- a s s e s s m e n t s c a n n o t g u a r a n te e o b je c t i v e m e a s u re m e n ts o f t r u e a b i l i t y.
H o we v e r, to c o m p e n s a te f o r s o m e o f t h e s e de f i ci e n c i e s R o s s n o t e s t h a t i n s t r u c t o r s “ c a n s t re n g t h e n r e l i a b i l i t y t h ro u g h s u c h s t r a te g i e s a s e n ga g i n g s t u d e n ts i n r u b r i c c o n s t r u c t i o n ” [ 2 4 ] a n d p ro v i di n g l e a r n e r s wi t h t r a i n i n g .
T h e p re s e n t r e po r t i s a s u m m a r y o f t h e f i n d i n g s o f a pa r t i c u l a r s e l f - a s s e s s m e n t o f s ki l l s o u t l i n e d i n t h e C E F R - J b y a f i r s t - y e a r J a p a n e s e u n i v e r s i t y s t u de n ts . I t n o te s n o t o n l y th e f i n di n gs b u t a l s o m e n ti o n s s h o r t co m i n g s a n d p o s s i b l e f u t u re r e s e a r c h a l o n g t h e s e l i n e s .
3. Method
Participants
257 students in first-year English as a foreign language courses of Tokyo International University voluntarily participated in this study.
Participants were from three different
departments: Language Communication,
International Relations, and Economics. The survey was administered at the beginning of April 2017.
Instrument
The survey was conducted in Japanese and administered online using Google Forms.
Participants were required to indicate “yes” or “no”
to indicate whether they can or cannot do a particular skill based on 40 CEFR-J can-do statements. The 40 can-do statements were taken from all five skills (understanding: listening, reading; speaking: spoken interaction, spoken production; and writing) from four of the CEFR-J levels: A1.2, A2.1, B1.1, and B2.1. The statements were randomized (see Appendices 1 and 2 for the original Japanese questions and translated English). The rationale for randomizing statements was to have participants be “blind” to the progression of difficulty inherent in the CEFR-J statements. These particular levels were selected as incoming freshmen students at Tokyo International University placed, to varying degrees, in these levels on an in-house placement paper-based test formulated on CEFR standards which was administered at the end of March 2017, prior to the commencement of 2017 spring term classes.
4. Results and Discussion
A s F i g u re 1 . a n d Ta b l e s 1 a n d 2 re v e a l t h a t t h e r e we r e m o re p o s i ti v e re s po n s e s a t t h e A 1 a n d A2 b a n ds . F e we r po s i t i v e re s po n s e s we r e f o u n d a t t h e B1 a n d B 2 b a n d s . On e n o t a b l e a n d s e e m i n gl y co n t r a d i c to r y e x c e p ti o n i s t h e s po k e n p r o du c t i o n s ki l l c a te g o r y i n t h e A1 . 2 a n d A 2 . 2 l e v e l s . H e re , m o re p o s i ti v e r e s po n s e s f o r t h e m o r e a d v a n ce d l e ve l ( A2 . 2 ) w e re g i ve n t h a n f o r t h e m o r e b a s i c l e ve l ( A 1 . 2 ) . T h e r e m a y b e t w o re a s o n s f o r t h i s . O n e , e a c h q u e s t i o n o n t h e q u e s t i o n n a i re a s k s p a r ti c i pa n t s to re s po n d i n a b i n a r y f a s h i o n ( e i t h e r “ ye s ” o r “ n o ” ) . S t u de n t s m a y h a v e m o r e di f f u s e o r s u b t l e r e s p o n s e s t o t h e q u e s t i o n s a s ke d a n d t h u s a L i k e r t s c a l e t y p e f o r m a t m a y h a ve b e e n m o r e a p p r o p r i a t e . I n a dd i t i o n , t h e t wo q u e s t i o n s a t t h e A 1 . 2 l e v e l ( Q u e s ti o n 25. I can express simple opinions related to limited, familiar topics, using simple words and
basic phrases in a restricted range of sentence structures, provided I can prepare my speech in advance. Question 29. I can give simple descriptions e.g. of everyday object, using simple words and basic phrases in a restricted range of sentence structures, provided I can prepare my speech in advance.) could be perceived as more challenging as the A2.1 questions (Question 26. I can introduce myself including my hobbies and abilities, using a series of simple phrases and sentences. Question 30. I can give a brief talk about familiar topics (e.g. my school and my neighborhood) supported by visual aids such as photos, pictures, and maps, using a series of simple phrases and sentences.). A possible rationale for the perceived level of difficulty could be that participants concentrated on the phrasing
“restricted range of sentence structures” in the A1.2 questions. What is meant by this expression is open to interpretation and each participant may be unclear as to how to gauge the meaning of
“restricted range”. Thus, these modifying statements in the questions at the A1.2 level may be perceived by participants as being more difficult.
Another apparent anomaly can be seen in the second spoken production question (questions 29 through 32. See Appendix 1 and 2). Here, the largest number of students (161) responded positively to the B1.1 level statement (154 participants stated “yes” for the A1 level statement and 149 stated “yes” to the A2 level statement).
Here, two possible explanations can be provided for this seeming contradiction. Firstly, in the A1 and A2 category questions, modifying phrases related to grammatical structure (“restricted range of sentence structures” in question 29 [A1] and “series of simple phrases and sentences” in question 30 [A2]) which do not exist in the B1 question may cause Japanese students who are particularly sensitive to grammatical points when speaking to not respond in the positive. A second reason for the apparent contradiction is the A1 and A2 level questions could be perceived as connoting that the respondent will give a public speech; the two questions explicitly state: “…provided I can prepare my speech in advance” (question 29) and “I can give a brief talk…”
(question 30). The B1 category question does not explicitly or implicitly connote a public speech The
B1 category question does not explicitly or implicitly connote a public speech as the language is vague (“I can talk about….”).
Ta bl e 1 : R a w N u m b e r o f P a r t i c i pa n t s ’ P o s i t i v e R e s p o n s e s to C E F R - J Q u e s t i o n s A c c o r di n g to S k i l l G r o u p
* N o te : T h e n u m b e r o f s t u d e n t s a t a g i ve n C E F R - J l e v e l o r a b o v e a n d t h o s e w h o r e s po n d e d “ y e s ” to C E F R- J c a n - d o q u e s t i o n s e x ce e d s t h e t o ta l n u m b e r o f p a r t i c i p a n ts . T h e n u m b e r a t s t u de n t s a t gi v e n C E F R - J l e ve l o r a b o ve i s m o r e t h a n th e n u m b e r o f pa r t i c i pa n t s b e c a u s e s t u de n t s a t t h e A 2 o r h i g h e r l e v e l s h o u l d , i n p r i n ci p l e , re s po n d “ y e s ” t o q u e s ti o n s
C E F R - J L e ve l L i s t e n i n g
A 1 . 2 4 7 0
A 2 . 1 4 0 5
B 1 . 1 2 3 8
B 2 . 1 1 3 0
C E F R - J L e ve l R e a d i n g
A 1 . 2 4 4 5
A 2 . 1 4 1 9
B 1 . 1 2 3 0
B 2 . 1 8 3
C E F R - J L e ve l
S p o k e n I n t e r a c ti o n
A 1 . 2 4 3 2
A 2 . 1 3 7 9
B 1 . 1 2 3 9
B 2 . 1 1 2 5
C E F R - J L e ve l
S p o k e n P r o d u c t i o n
A 1 . 2 3 2 3
A 2 . 1 3 7 0
B 1 . 1 2 5 6
B 2 . 1 6 2
C E F R - J L e ve l Wr i t i n g
A 1 . 2 4 4 9
A 2 . 1 3 5 8
B 1 . 1 1 8 1
B 2 . 1 8 0
a t t h e A 1 . 2 l e ve l ; t h i s w o u l d a l s o b e t r u e f o r t h e A 2 a n d B 1 l e v e l s f o r B1 a n d B2 s tu d e n ts r e s pe c t i v e l y.
F i g u r e 1 : R e l a ti o n s h i p a m o n g ra w n u m b e r o f s e l f - r a t i n g s i n f i v e s k i l l ca t e go r i e s
Williams and Andrade, in a study of 243 Japanese university students studying English in Japan, found that “ a n x i e t y w a s o f te n a s s o c i a t e d w i th t a s k s i n v o l v i n g s pe a k i n g i n f ro n t o f o t h e rs ” [ 2 5 ]. T h e re f o r e , a n xi e t y r e l a te d t o g i vi n g a m o re f o r m a l p u b l i c s pe e c h r a t h e r t h a n s i m pl y ta l k i n g a b o u t a to p i c c o u l d h a ve s k e we d r e s po n s e s .
The measure of internal consistency of participants’ self-ratings among the five skill groups in this study was statistically reliable when the raw responses of individual students was compared with each skill category as the Cronbach alpha was 0.9062. This scale is from 0 to 1.0 and the higher the number, the more statistical reliability. A reliability coefficient of 0.7000 or higher is deemed “acceptable” in the majority of social science studies. The coefficient in this present study mirrors the finding of internal consistency that Tokeshi and Gao [26] had. In their study that the C r o n b a c h ’s a l p h a v a l u e a m o n g t h e f i v e s k i l l c a t e go r i e s w a s 0 . 8 7 2 w h e n t h e y c a l c u l a te d “ t h e a ve r a ge s e l f- ra t i n g s o f
i n d i vi d u a l re s po n de n t s f o r e a c h s ki l l c a te g o r y ” [ 2 7 ] . A n o th e r s t u d y w i th s i m i l a r re l i a b i l i t y d a ta i s Runnels’ [28]. When calculating data on the reliability of the entire CEFR-J A level can-do statements with 590 first- and second-year Japanese university students in Runnels’ study, across all statements a strong reliability as measured by Cronbach’s alpha 0.944 scale was found.
Ta b l e 2 : Ra w N um b e r o f P a r ti c i p a n t s ’ P o s i t i v e Re s po n s e s t o Ea c h Q u e s ti o n A l o n g C E F RJ S k i l l
G r o u p s ( q u e s ti o n n u m b e r n o t e d i n p a re n t h e s e s )
Ta b l e 3 . C ro n b a c h Al p h a f o r C E F R - J L e ve l s A 1 . 2 A 2 . 1 B 1 . 1 B 2 . 1
L i s t e n i n g
2 5 0 ( 1 )
2 2 7 ( 2 )
1 7 3
( 3 ) 7 4 ( 4 ) 2 2 0
( 5 )
1 7 8
( 6 ) 6 5 ( 7 ) 5 6 ( 8 )
R e a d i n g
2 2 7 ( 9 )
2 2 2 ( 1 0 )
7 7 ( 11 )
3 0 ( 1 2 ) 2 1 8
( 1 3 )
1 9 7 ( 1 4 )
1 5 3 ( 1 5 )
5 3 ( 1 6 )
S p o k e n I n t e r a c ti o n
2 1 6 ( 1 7 )
1 8 8 ( 1 8 )
1 4 2 ( 1 9 )
6 3 ( 2 0 ) 2 1 6
( 2 1 )
1 9 1 ( 2 2 )
9 7 ( 2 3 )
6 2 ( 2 4 )
S p o k e n P r o d u c t i o n
1 6 9 ( 2 5 )
2 2 1 ( 2 6 )
9 5 ( 2 7 )
2 4 ( 2 8 ) 1 5 4
( 2 9 )
1 4 9 ( 3 0 )
1 6 1 ( 3 1 )
3 8 ( 3 2 )
Wr i t i n g
2 2 8 ( 3 3 )
1 7 0 ( 3 4 )
9 8 ( 3 5 )
4 6 ( 3 6 ) 2 2 1
( 3 7 )
1 8 8 ( 3 8 )
8 3 ( 3 9 )
3 4 ( 4 0 )
C E F R - J
L e v e l C r o n b a c h a l ph a
A 1 . 2 0 . 7 4 4
A 2 . 1 0 . 6 8 4
B 1 . 1 0 . 7 9 9
B 2 . 1 0 . 8 5 3
A s s h o w n i n Ta b l e 3 . , t h e A 2 . 1 l e ve l q u e s t i o n s h a d a f e w a n o m a l i e s a n d t h e y m a y e xp l a i n t h e r e l a t i ve l y l o we r C r o n b a ch a l p h a c o e f f i c i e n t i n co m pa r i s o n w i t h t h e o t h e r l e v e l s . A M o k k e n S ca l e a n a l y s i s f o r t h e A 1 . 2 , A2 . 1 , B 1 . 1 , a n d B2 . 1 s t a t e m e n t s a s g ro u p w a s c o n d u c t e d ( r e s u l ts d i s p l a y e d i n Ta b l e 4 . b e l o w ) . R u n n e l s ’ [ 2 9 ] r e s e a r c h u n de r t o o k s i m i l a r a n a l y s i s a n d s h e p o i n te d o u t t h a t “ M o k k e n s ca l i n g i s a s ta t i s ti c a l te c h n i q u e t h a t a s s u m e s t h e o r de r o f di f f i c u l t y o f i te m s i s n o t t h e s a m e a c ro s s a p o pu l a ti o n ( v a n S c h u u r, 2 0 0 3 ) a n d i t p ro v i de s a m e a s u re o f r e l i a b i l i t y b y i de n ti f y i n g i te m s f o r w h i c h G u t tm a n p a t te r n i n g i s o c cu r r i n g a t h i g h e r r a te s ( M o l e n a a r, 1 9 9 7 ; S i j ts m a & M o l e n a a r, 2 0 0 2 ) ” [ 3 0 ] . G u t t m a n s ca l i n g i s u t i l i ze d i n s u r v e y s , t e s t s o r q u e s t i o n n a i re s h a vi n g b i n a r y i te m s ( “ y e s ” o r
“ n o ” a s i n th e o n e e m pl o y e d i n t h e p r e s e n t s tu d y ) . I n a G u t t m a n s ca l e , i te m s a r e a r r a n g e d i n o r de r s o th a t i f a p a r ti c i p a n t a n s we r s “ y e s ” to a n B 2 . 1 l e ve l q u e s t i o n , t h e y s h o u l d a l s o re s po n d “ y e s ” to l o w e r p ro f i c i e n c y i te m s ( e . g . a t th e A 1 . 2 , A2 . 1 , a n d B 1 . 1 l e v e l s ) . T h u s , s t a t i n g “ y e s ” t o a n i t e m a t a h i g h e r l e ve l i m p l i e s th a t “ ye s ” s h o u l d b e n o te d a t a l l l o w e r l e ve l q u e s t i o n s . R u n n e l s [ 3 1 ] w r i te s t h a t M o k k e n s c a l i n g ge n e r a te s a r a t i o t h a t d i s p l a y s t h e pe r c e i ve d d i f f i c u l t y o f e a c h ca n - d o s ta t e m e n t a c c o r di n g to t h e s e l f - r a te d a b i l i t y o f e a c h p a r ti c i pa n t a n d t h e d e g re e to w h i c h a l a r g e r n u m b e r o f h i gh e r p r o f i c i e n c y p a r ti c i pa n t s p e r ce i ve d th e c a n - d o s t a te m e n t m o r e c h a l l e n g i n g . T h e s t a ti s t i c t h a t i s p ro d u c e d i s ca l l e d t h e c o e f f i c i e n t o f h o m o g e n e i t y ( H o r H - v a l u e ) , a n d i t re v e a l s t h e s t r u c tu r e o f a n s w e rs f o r e a c h s ta t e m e n t b y m e a n s o f i t e m l i m i t a ti o n s . T h u s , t h i s c o e f f i c i e n t n o te s t h e r e l i a b i l i t y s c a l e f o r e a c h c a n - d o s ta t e m e n t a n d e x po s e s t h e de g r e e t h a t a G u t t m a n m o d e l ca n b e s e e n f o r e a c h a n s w e r.
Co e f f i c i e n ts o f h o m o ge n e i t y r a n ge f ro m 0 t o 1 . 0 . A h i g h e r H - s co r e c o r r e l a t e s w i t h a n e l e m e n t t h a t m e a s u r e s m o r e i n s y n c w i t h G u t tm a n ’s p ro p o s i ti o n ( i . e . i t e m s a re a r r a n g e d i n o r de r s o t h a t i f a re s po n de n t a n s w e r s “ y e s ” to a n B 2 . 1 l e ve l q u e sti o n , t h e y s h o u l d a l s o r e s po n d “ ye s ” to l o we r p ro f i ci e n c y i t e m s ; s t a ti n g “ ye s ” t o a n
i te m a t a h i g h e r l e v e l i m pl i e s t h a t “ ye s ” s h o u l d b e n o te d a t a l l l o w e r l e v e l q u e s t i o n s . ) . H - v a l u e s b e n e a t h t h e 0 . 3 t h r e s h o l d a r e u n a c c e p ta b l e , a n d s c o re s o v e r 0 . 6 de n o te s s t ro n g re l i a b i l i t y [ 3 2 ] .
Table 4. Mokken Scales Coefficients for Each CEFR-J level Examined
C E F R - J L e ve l H - c o e f f i ci e n t
A 1 . 2 0 . 3 9 4
A 2 . 1 0 . 3 8 7
B 1 . 1 0 . 3 7 6
B 2 . 1 0 . 4 8 5
Two hypotheses for relatively lower h -coefficient results (all are on the lower cusp of acceptable results of 0.3 as aforementioned). Firstly, in contrast to Runnels’
study [33] where participants responded to the five A levels in the CEFR-J (A1.1, A1.2, A1.3, A2.1, A2.2), the present study took a larger view and examined a range from of levels from A1.2 to B2.1. As previously noted, conspicuous incongruities in participants’ answers to two A2.1 can-do statements relative to A1.2 and B1.2 queries may have skewed reliability. Secondly, the present study only had respondents select from binary options (“yes” or
“no”). Having a more restricted field of options, while producing more discrete data, constrains and compels participants in an either-or situation which may not truly reflect participants’ ability. In a similar study, Runnels’
[34] utilized a five category Likert-scale response form (strongly disagree to strong agree). Providing more breadth of responses possibly reduces the error co-efficient and diffuses the data.
A number of weaknesses with this study must be detailed. Firstly, the entire 110 CEFR-J can-do list was not employed and thus while an overview of the participants’ self-ratings can be recognized, a
complete picture of the group’s CEFR-J
self-assessment cannot be seen. This would be a step for a further study. The rationale for this limited survey was twofold; first, at the start of an academic year appropriating time for freshmen students to participate in a voluntary-based questionnaire is limited and therefore the researcher decided to use an abbreviated version, and second, a t t h e o u ts e t o f th e a ca d e m i c y e a r, t h e s t u de n ts i n q u e s t i o n s a t f o r a n i n - h o u s e
pl a c e m e n t te s t th a t wa s d e ve l o pe d a l o n g C E F R l i n e s . T h e p u r po s e o f t h i s t e s t w a s to s t r e a m s tu d e n t s i n to s e c ti o n s s o t h a t s t u de n ts w o u l d b e wi t h s t u de n t s a t s i m i l a r p ro f i ci e n c y l e v e l s . T h i s i n - h o u s e p l a c e m e n t te s t w a s de s i g n e d b a s e d o n th e Ca m b r i d ge K E T e x a m i n a ti o n ( ge a r e d f o r A1 a n d A 2 l e a r n e rs ) . F o l l o w i n g n u m e r o u s p i l o t t ri a l s a n d f o l l o w - u p a n a l y s i s a n d m o di f i c a t i o n s , t h e i n - h o u s e te s t r e ve a l e d m e a s u re a b l e r e l i a b i l i t y t o t h e K E T te s t ( 0 . 7 5 3 C r o n b a c h a l p h a ) . T h u s , a dm i n i s te r i n g a q u e s t i o n n a i re w i th q u e r i e s f r o m t h e A1 , A2 , B 1 , a n d B 2 l e ve l s w a s d e e m e d s u i ta b l e . A s e co n d d r a w b a c k o f t h i s s tu d y w a s t h e b i n a r y r e s po n s e o p t i o n s ( y e s / n o ) . Wi t h m o r e n u a n c e d o p ti o n s i n a L ik e r t s ca l e ( e . g . “ s t ro n g l y a g r e e , / s o m e w h a t a g re e / s o m e w h a t di s a g re e / s t r o n g l y di s ag re e ” ) m a y h a v e p r o vi de d m o r e te l l i n g a n d i n f o r m a ti v e da t a . T h i r dl y, ce r t a i n t e rm s i n t h e C E F R - J ca n - do l i s t m a y b e u n f a m i l i a r t o s tu d e n t s a n d t h u s ca u s e c o n f u s i o n a n d t h e r e f o re r e s u l t i n m i s l e a d i n g d a t a . A s t h e m a j o ri t y o f pa r t i c i pa n t s i n t h i s s t u d y w e r e f i r s t - y e a r J a pa n e s e u n i v e r s i t y s t u de n ts w h o e n t e re d th e u n i v e rs i t y i n A p r i l , q u e s t i o n s pe r t a i n i n g to a wo r k e n v i ro n m e n t w o u l d n o t n e c e s s a r i l y pe r t a i n to r e s po n de n ts . F o r e xa m pl e , q u e s t i o n 16 states “I can understand in detail specifications, instruction manuals, or reports written for my own field of work, provided I can reread difficult sections.” Students in their first year of studies at university in Japan most likely would not be exposed to interacting with English in such a fashion and therefore may be at a loss as to how to respond. A fourth weakness of the study revolves around the term “native speakers” which arises a number of times throughout the questionnaire. The entire notion of
“native speaker” and its relevance to English proficiency is a different topic (e.g. English as lingua franca); however, germane to this study is that numerous participants may not have had opportunities to interact with “native speakers”.
Thus, a question such as number 24, “I can discuss abstract topics, provided they are within my terms of knowledge, my interests, and my experience, although I sometimes cannot contribute to discussions between native speakers” may have
confounded certain respondents.
6. Conclusion
T h i s s t u d y o f f e rs i n t r o d u c t o r y a n d n a r ro w r e s u l t s o n th e C E F R - J c a n - do s ta t e m e n ts a n d m e a s u re m e n t m e t h o ds . W h i l e t h e consistency of participants’ self-ratings among the five skill groups was statistically reliable as measured by the Cronbach alpha when the raw responses of individual students was compared with each skill category, there were a number of areas where the CEFR-J scales appeared to require additional investigation as to appropriateness and reliability.
The findings reveal, as measured by the Mokken scale h-coefficient, that internal reliability was rather weak; possible explanations for this weakness were commented on and further research on CEFR-J can-do statements can take measured steps to possibly decrease self-rating reliability concerns. Further studies investigating the internal reliability of the CEFR-J statements should be undertaken, noting one particular weakness of this present study of using binary option responses; employing a Likert-scale format may reduce low Mokken scale coefficient results.
In addition, the limitations of self-rating, notably with Japanese university students’ perceptions of their English skill, and the possible gap with actual proficiency as measured on the CEFR-J requires more examination. One possible method to link participants’ self-assessments with more objective proficiency could be analyzing these self-rating data points alongside KET scores (synced with the original CEFR, not the CEFR-J).
Acknowledgement This work was supported by Tokyo International University Special Educational Research Grant.
文 献
[1] European Framework of Reference for Languages, Common European Framework of Reference:
Learning, teaching, assessment (CEFR).
Retrieved August 2, 2017 from
https://www.coe.int/en/web/common-european-fr amework-reference-languages/
[2]Council of Europe, Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg: Council of Europe / Cambridge University Press, 2011.
[3]N. Nagai,and F. O’Dwyer, The actual and potential impacts of the CEFR on language
education in Japan, Synergies Europe 6, pp.141-152, 2011.
[4]Y.Tono,and M. Negishi, M., The CEFR-J:
Adapting the CEFR for English Language Teaching in Japan. Framework & Language Portfolio SIG Newsletter, 8, pp.5-12, 2012.
[5]Council of Europe, Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg: Council of Europe / Cambridge University Press, p. 256, 2011.
[6]Council of Europe, Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg: Council of Europe / Cambridge University Press, p. 244, 2011.
[7]Y.Tono and M. Negishi, The CEFR-J:
Adapting the CEFR for English Language Teaching in Japan. Framework & Language Portfolio SIG Newsletter, 8, pp.5-12, 2012.
[8] N. Nagai,and F. O’Dwyer, The actual and potential impacts of the CEFR on language education in Japan, Synergies Europe 6, pp.141-152, 2011.
[ 9 ] M . G . F e n n e l l y, T h e I n f l u e n ce o f C E F R o n E n gl i s h E d u c a ti o n i n J a p a n , Bu l l e ti n o f S h i k o k u U n i ve r s i t y , 4 6 , p p . 1 0 9–1 2 2 , 2 0 1 6 . [10] Y.Tono,and M. Negishi, The CEFR-J:
Adapting the CEFR for English Language Teaching in Japan. Framework & Language Portfolio SIG Newsletter, 8, pp.5-12, 2012.
[11] M. Negishi, The Development of the CEFR-J:
Where we are, where we are going, in New perspectives for foreign language teaching in higher education: Exploring the possibilities of application of CEFR, ed. N. Tomimori, M.
Furihatat, K. Haida, N. Kurosawa, & M.
Negishi, pp. 105-116, Tokyo WOLSEC, Tokyo University of Foreign Studies, 2012.
[12] M. Negishi, M,. CEFR-J Kaihatsu no Keii [The Development Process of the CEFR-J], ARCLE Review, vol. 5, issue 3, 37-52, 2011.
[13] Oskarsson, M., Approaches to self-assessment
in foreign language
learning, London: Pergamon, 1980.
[14] D.Dunning, C. Heat and J.M. Suls, Flawed Self-Assessment. Psychological Science in the Public Interest,5 (3), pp. 69-106.
doi:10.1111/j.1529-1006.2004.00018.x, 2004.
[ 1 5 ] D . G a r d n e r a n d L . M i l l e r, L . , E s ta b l i s h i n g S e l f - A c ce s s : F ro m T h e o r y t o P r a c ti c e . C a m b ri d g e U n i ve r s i t y P r e s s , C a m b r i d ge , 1 9 9 9 .
[16] P. Blanche and B.J. Merino, Self-assessment of foreign- language skills: Implications for
teachers and researchers. Language
Learning,39 (3), pp. 313-338.
doi:10.1111/j.1467-1770.1989.tb00595.x, 1989.
[17] S.Ross, Self-assessment in second language testing: a meta-analysis and analysis of experiential factors. Language Testing,15(1), pp.1-20. doi:10.1177/026553229801500101, 1998.
[18] J. Kruger and D.Dunning, Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology,77(6), pp. 1121-1134.
doi:10.1037//0022-3514.77.6.1121, 1999.
[19] S. Matsuno, Self-, peer-, and
teacher-assessments in Japanese university
EFL writing classrooms. Language
Testing,26(1), pp. 75-100.
doi:10.1177/0265532208097337, 1999.
[20] S. Matsuno, Self-, peer-, and
teacher-assessments in Japanese university
EFL writing classrooms. Language
Testing,26(1), pp. 75-100.
doi:10.1177/0265532208097337, 1999.
[21] K.Denies and R. Janssen, Country and gender differences in the functioning of CEFR-based can-do statements as a tool for self-assessing English proficiency. Language Assessment
Quarterly,13 (3), pp. 251-276.
doi:10.1080/15434303.2016.1212055, 2016.
[22] F. Parajes, Gender and perceived self-efficacy in self-regulated learning. Theory Into Practice,
41(2), pp. 116-125.
h t t p: / / d x . do i . o r g / 1 0 . 1 2 0 7 / s 1 5 4 3 0 4 2 1 t i p4 1 0 2 _ 8 , 2 0 0 2 .
[ 2 3 ] L.F. Bachman and A.S.Palmer,The construct validation of self-ratings of communicative language ability. Language Testing,6 (1), pp.14-29. doi:10.1177/026553228900600104, 1999.
[ 2 4 ] J . A . Ro s s , T h e r e l i a b i l i t y, v a l i di t y, a n d u ti l i t y o f s e l f - a s s e s s m e n t . P ra c t i c a l A s s e s s m e n t , R e s e a r c h , a n d E v a l u a ti o n , 11 ( 1 0 ) , p p . 1 - 1 3 , 2 0 0 6 .
[25] K.Williams and M.Andrade,Foreign Language Learning Anxiety in Japanese EFL University Classes: Causes, Coping, and Locus of Control, Electronic Journal of Foreign Language Teaching, 5(2), pp.181–191, 2008.
[26] M. Tokeshi and L. Gao,CEFR-J Based Survey for Japanese University Students, The European Conference on Education 2015 Official Conference Proceedings, pp. 433-447, 2015.
[27] M. Tokeshi and L. Gao,CEFR-J Based Survey for Japanese University Students, The European Conference on Education 2015 Official Conference Proceedings, pp. 433-447, 2015.
[28] J. Runnels, An exploratory reliability and content analysis of the CEFR-Japan’s A-Level can-do statements, JALT Journal 36(1), pp.
69-89, 2014.
Appendix 1
CEFR-J Can-do questionnaire (Japanese version)
(Retrieved from
http://www.tufs.ac.jp/ts/personal/tonolab/cefr-j/engl ish/download.html)
1. 趣 味 や ス ポ ー ツ 、 部 活 動 な ど の 身 近 な ト ピ ッ ク に す る 短 い 話 を 、 ゆ っ く り は っ き り と 話 さ れ れ ば 、 理 解 す る こ と が で き る 。
2. ゆ っ く り は っ き り と 放 送 さ れ れ ば 、公 共 の 乗 り 物 や 駅 や 空 港 の 短 い 簡 潔 な ア ナ ウ ン ス を 理 解 す る こ と が で き る 。
3. 外 国 の 行 事 や 習 慣 な ど に 関 す る 説 明 の 概 要 を 、ゆ っ く り は っ き り と 話 さ れ れ ば 、 理 解 す る こ と が で き る 。 4. 自 然 な 速 さ の 標 準 的 な 英 語 で 話 さ れ て い れ ば 、テ レ ビ 番 組 や 映 画 の 母 語 話 者 同 士 の 会 話 の 要 点 を 理 解 で き る 。
5. 日 常 生 活 の 身 近 な ト ピ ッ ク に つ い て の 話 を 、ゆ っ く り は っ き り と 話 さ れ れ ば 、 場 所 や 時 間 等 の 具 体 的 な 情 報 を 聞 き と る こ と が で き る 。
6. 学 校 の 宿 題 、旅 行 の 日 程 な ど の 明 確 で 具 体 的 な 事 実 を 、 は っ き り と な じ み の あ る 発 音 で 指 示 さ れ れ ば 、 要 点 を 理 解 す る こ と が で き る 。
7. 自 分 の 周 り で 話 さ れ て い る 少 し 長 め の 議 論 で も 、は っ き り と な じ み の あ る 発 音 で あ れ ば 、 そ の 要 点 を 理 解 す る こ と が で き る 。
8. ト ピ ッ ク が 身 近 で あ れ ば 、長 い 話 や 複 雑 な 議 論 の 流 れ を 理 解 す る こ と が で き る 。
9. 簡 単 な ポ ス タ ー や 招 待 状 等 の 日 常 生 活 で 使 わ れ る 非 常 に 短 い 簡 単 な 文 章 を 読 み 、理 解 す る こ と が で き る 。 10. 簡 単 な 語 を 用 い て 書 か れ た 人 物 描 写 、場 所 の 説 明 、 日 常 生 活 や 文 化 の 紹 介 な ど の 、 説 明 文 を 理 解 す る こ と が で き る 。
11. 学 習 を 目 的 と し て 書 か れ た 新 聞 や 雑 誌 の 記 事 の 要 点 を 理 解 す る こ と が で き る 。
12. 現 代 の 問 題 な ど 一 般 的 関 心 の 高 い ト ピ ッ ク を 扱 っ た 文 章 を 、 辞 書 を 使 わ ず に 読 み 、 複 数 の 視 点 の 相 違 点 や 共 通 点 を 比 較 し な が ら 読 む こ と が で き る 。
13. 身 近 な 人 か ら の 携 帯 メ ー ル な ど に よ る 、 旅 の 思 い 出 な ど が 書 か れ た 非 常 に 短 い 簡 単 な 近 況 報 告 を 理 解 す る こ と が で き る 。
14. 簡 単 な 語 を 用 い て 書 か れ た 短 い 物 語 や 伝 記 な ど を 理 解 す る こ と が で き る 。
15. ゲ ー ム の や り 方 、 申 込 書 の 記 入 の し か た 、 も の の 組 み 立 て 方 な ど 、 簡 潔 に 書 か れ た 手 順 を 理 解 す る こ と が で き る 。
16. 難 し い 部 分 を 読 み 返 す こ と が で き れ ば 、 自 分 の 専 門 分 野 の 報 告 書 ・ 仕 様 書 ・ 操 作 マ ニ ュ ア ル な ど を 、 詳 細 に 理 解 す る こ と が で き る 。
17. 基 本 的 な 語 や 言 い 回 し を 使 っ て 日 常 の や り と り
（ 何 が で き る か で き な い か や 色 に つ い て の や り と り な ど ） に お い て 単 純 に 応 答 す る こ と が で き る 。
18. 順 序 を 表 す 表 現 で あ る first, then, next な ど の つ な ぎ 言 葉 や 「 右 に 曲 が っ て 」 や 「 ま っ す ぐ 行 っ て 」 な ど の 基 本 的 な 表 現 を 使 っ て 、 単 純 な 道 案 内 を す る こ と が で き る 。
19. 身 近 な ト ピ ッ ク （ 学 校 ・ 趣 味 ・ 将 来 の 希 望 ） に つ い て 、 簡 単 な 英 語 を 幅 広 く 使 っ て 意 見 を 表 明 し 、 情 報 を 交 換 す る こ と が で き る 。
20. あ る 程 度 な じ み の あ る ト ピ ッ ク な ら ば 、 新 聞 ・ イ ン タ ー ネ ッ ト で 読 ん だ り 、 テ レ ビ で 見 た ニ ュ ー ス の 要 点 に つ い て 議 論 す る こ と が で き る 。
21. ス ポ ー ツ や 食 べ 物 な ど の 好 き 嫌 い な ど の と て も な じ み の あ る ト ピ ッ ク に 関 し て 、は っ き り 話 さ れ れ ば 限 ら れ た レ パ ー ト リ ー を 使 っ て 、 簡 単 な 意 見 交 換 を す る こ と が で き る 。
22. 補 助 と な る 絵 や も の を 用 い て 、 基 本 的 な 情 報 を 伝 え 、 ま た 、 簡 単 な 意 見 交 換 を す る こ と が で き る 。 23. 個 人 的 に 関 心 の あ る 具 体 的 な ト ピ ッ ク に つ い て 、 簡 単 な 英 語 を 多 様 に 用 い て 、 社 交 的 な 会 話 を 続 け る こ と が で き る 。
24. 母 語 話 者 同 士 の 議 論 に 加 わ れ な い こ と も あ る が 、 自 分 が 学 ん だ ト ピ ッ ク や 自 分 の 興 味 や 経 験 の 範 囲 内 の ト ピ ッ ク な ら 、 抽 象 的 な ト ピ ッ ク で あ っ て も 、 議 論 で き る 。
25. 前 も っ て 発 話 す る こ と を 用 意 し た 上 で 、 限 ら れ た
身 近 な ト ピ ッ ク に つ い て 、 簡 単 な 語 や 基 礎 的 な 句 を 限 ら れ た 構 文 を 用 い 、 簡 単 な 意 見 を 言 う こ と が で き る 。 26. 一 連 の 簡 単 な 語 句 や 文 を 使 っ て 、 自 分 の 趣 味 や 特 技 に 触 れ な が ら 自 己 紹 介 を す る こ と が で き る 。 27. 使 え る 語 句 や 表 現 を 繋 い で 、 自 分 の 経 験 や 夢 、 希 望 を 順 序 だ て 、 話 し を 広 げ な が ら 、 あ る 程 度 詳 し く 語 る こ と が で き る 。
28. あ る 視 点 に 賛 成 ま た は 反 対 の 理 由 や 代 替 案 な ど を あ げ て 、 事 前 に 用 意 さ れ た プ レ ゼ ン テ ー シ ョ ン を 聴 衆 の 前 で 流 暢 に 行 う こ と が で き 、一 連 の 質 問 に も あ る 程 度 流 暢 に 対 応 が で き る 。
29. 前 も っ て 発 話 す る こ と を 用 意 し た 上 で 、 日 常 生 活 の 物 事 を 、 簡 単 な 語 や 基 礎 的 な 句 を 限 ら れ た 構 文 を 用 い 、 簡 単 に 描 写 す る こ と が で き る 。
30. 写 真 や 絵 、地 図 な ど の 視 覚 的 補 助 を 利 用 し な が ら 、 一 連 の 簡 単 な 句 や 文 を 使 っ て 、 身 近 な ト ピ ッ ク （ 学 校 や 地 域 な ど ） に つ い て 短 い 話 を す る こ と が で き る 。 31. 自 分 の 考 え を 事 前 に 準 備 し て 、 メ モ の 助 け が あ れ ば 、 聞 き 手 を 混 乱 さ せ な い よ う に 、 馴 染 み の あ る ト ピ ッ ク や 自 分 に 関 心 の あ る 事 柄 に つ い て 語 る こ と が で き る 。
32. デ ィ ベ ー ト な ど で 、 そ の ト ピ ッ ク が 関 心 の あ る 分 野 の も の で あ れ ば 、 論 拠 を 並 べ 自 分 の 主 張 を 明 確 に 述 べ る こ と が で き る 。
33. 簡 単 な 語 や 基 礎 的 な 表 現 を 用 い て 、身 近 な こ と（ 好 き 嫌 い 、 家 族 、 学 校 生 活 な ど ） に つ い て 短 い 文 章 を 書 く こ と が で き る 。
34. 日 常 的 ・ 個 人 的 な 内 容 で あ れ ば 、 招 待 状 、 私 的 な 手 紙 、 メ モ 、 メ ッ セ ー ジ な ど を 簡 単 な 英 語 で 書 く こ と が で き る 。
35. 自 分 に 直 接 関 わ り の あ る 環 境 （ 学 校 、 職 場 、 地 域 な ど ） で の 出 来 事 を 、 身 近 な 状 況 で 使 わ れ る 語 彙 ・ 文 法 を 用 い て 、 あ る 程 度 ま と ま り の あ る か た ち で 、 描 写 す る こ と が で き る 。
36. 自 分 の 専 門 分 野 で あ れ ば 、 メ ー ル や フ ァ ッ ク ス 、 ビ ジ ネ ス ・ レ タ ー な ど の ビ ジ ネ ス 文 書 を 、 感 情 の 度 合 い を あ る 程 度 含 め 、 か つ 用 途 に 合 っ た 適 切 な 文 体 で 、 書 く こ と が で き る 。
37. 簡 単 な 語 や 基 礎 的 な 表 現 を 用 い て 、 メ ッ セ ー ジ カ ー ド （ 誕 生 日 カ ー ド な ど ） や 身 近 な 事 柄 に つ い て の 短 い メ モ な ど を 書 け る 。
38. 文 と 文 を and, but, because な ど の 簡 単 な 接 続 詞 で つ な げ る よ う な 書 き 方 で あ れ ば 、 基 礎 的 ・ 具 体 的 な 語 彙 、 簡 単 な 句 や 文 を 使 っ た 簡 単 な 英 語 で 、 日 記 や 写 真 、 事 物 の 説 明 文 な ど の ま と ま り の あ る 文 章 を 書 く こ と が で き る 。
39. 身 近 な 状 況 で 使 わ れ る 語 彙 ・ 文 法 を 用 い れ ば 、 筋 道 を 立 て て 、 作 業 の 手 順 な ど を 示 す 説 明 文 を 書 く こ と が で き る 。
40. 旅 行 記 や 自 分 史 、 身 近 な エ ピ ソ ー ド な ど の 物 語 文 に つ い て 何 か 自 分 が 知 っ て い れ ば 、 多 く の 情 報 源 か ら 統 合 し て 情 報 や 議 論 を 整 理 し な が ら 、 そ れ に 対 す る 自 分 の 考 え の 根 拠 を 示 し つ つ 、 あ る 程 度 の 結 束 性 の あ る エ ッ セ イ や レ ポ ー ト な ど を 、 幅 広 い 語 彙 や 複 雑 な 文 構 造 を あ る 程 度 使 っ て 、 書 く こ と が で き る 。
Appendix 2 CEFR-J Can-do questionnaire
(English version) (Retrieved from
http://www.tufs.ac.jp/ts/personal/tonolab/cefr-j/engl ish/download.html)
1. I can understand short conversations about familiar topics (e.g. hobbies, sports, club activities), provided they are delivered in slow and clear speech.
2. I can understand short, simple announcements e.g. on public transport or in stations or airports, provided they are delivered slowly and clearly.
3. I can understand the gist of explanations of cultural practices and customs that are unfamiliar to me, provided they are delivered in slow and clear speech involving rephrasing and repetition.
4. I can understand the main points of a conversation between native speakers in television programmes and in films, provided they are delivered at normal speed and in standard English.
5. I can catch concrete information (e.g. places and times) on familiar topics encountered in everyday life, provided it is delivered in slow and clear speech.
6. I can understand the main points of straightforward factual messages (e.g. a school assignment, a travel itinerary), provided speech is clearly articulated in a familiar accent.
7. I can understand the main points of extended discussions around me, provided speech is clearly articulated and in a familiar accent.
8. I can follow extended speech and complex lines of argument provided the topic is reasonably familiar.
9. I can understand very short, simple, everyday texts, such as simple posters and invitation cards.
10. I can understand explanatory texts describing people, places, everyday life, and culture, etc., written in simple words.
11. I can understand the main points of English newspaper and magazine articles adapted for educational purposes.
12. I can read texts dealing with topics of general interest, such as current affairs, without consulting a dictionary, and can compare differences and similarities between multiple points of view.
13. I can understand very short reports of recent events such as text messages from friends' or relatives', describing travel memories, etc.
14. I can understand short narratives and biographies written in simple words.
15. I can understand clearly written instructions (e.g. for playing games, for filling in a form, for assembling things).
16. I can understand in detail specifications, instruction manuals, or reports written for my own field of work, provided I can reread difficult sections.
17. I can respond simply in basic, everyday interactions such as talking about what I can/cannot do or describing colour, using a limited repertoire of expressions.
18. I can give simple directions from place to place, using basic expressions such as "turn right" and "go straight" along with sequencers such as first, then, and next.
19. I can express opinions and exchange information about familiar topics (e.g. school, hobbies, hopes for the future), using a wide range of simple English.
20. I can discuss the main points of news stories I have read about in the newspapers/ on the internet or watched on TV, provided the topic is reasonably familiar to me.
21. I can exchange simple opinions about very familiar topics such as likes and for sports, foods, etc., using a limited repertoire of expressions, provided that people speak clearly.
22. I can get across basic information and exchange simple opinions, using pictures or objects to help me.
23. I can maintain a social conversation about concrete topics of personal interest, using a wide range of simple English.
24. I can discuss abstract topics, provided they are within my terms of knowledge, interests, and my experience, although I sometimes cannot contribute to discussions between native speakers.
25. I can express simple opinions related to limited, familiar topics, using simple words and basic phrases in a restricted range of sentence structures, provided I can prepare my speech in advance.
26. I can introduce myself including my hobbies and abilities, using a series of simple phrases and sentences.
27. I can talk in some detail about my experiences, hopes and dreams, expanding on what I say by joining together words, phrases and expressions I can readily use to make longer contributions.
28. I can give a prepared presentation with reasonable fluency, stating reasons for agreement or disagreement or alternative proposals, and can answer a series of questions.
29. I can give simple descriptions e.g. of everyday object, using simple words and basic phrases in a restricted range of sentence structures, provided I can prepare my speech in advance.
30. I can give a brief talk about familiar topics (e.g.
my school and my neighborhood) by visual aids such as photos, pictures, and maps, using a series of simple phrases and sentences.
31. I can talk about familiar topics and other topics of personal interest, without causing confusion to the listeners, provided I can prepare my ideas in advance and use brief notes to help me.
32. I can develop an argument clearly in a debate by
providing evidence, provided the topic is of personal interest.
33. I can write short texts about matters of personal relevance (e.g. likes and dislikes, , and school life), using simple words and basic expressions.
34. I can write invitations, personal letters, memos, and messages, in simple English, provided they are about routine, personal matters.
35. I can write a description of substantial length about events taking place in my immediate environment (e.g. school, workplace, local area), using familiar vocabulary and grammar.
36. I can write business documents (e.g. e - mail, fax, business letters), conveying degrees of emotion, in a style appropriate to the purpose, provided they are in my professional field.
37. I can write message cards (e.g. birthday cards) and short memos about events of personal relevance, using simple words and basic expressions.
38. I can write texts of some length (e.g. diary entries, explanations of photos and events) in simple English, using basic, concrete vocabulary and simple phrases and sentences, linking sentences with simple connectives like and, but , and because . 39. I can write coherent instructions telling people how to do things, with vocabulary and grammar of immediate relevance.
40. I can write reasonably coherent essays and reports using a wide range of vocabulary and complex sentence structures, synthesising information and arguments from a number of sources, provided I know something about the topics.
