The Development of a Classroom Observation Instrument for English Lessons in Japanese Junior High Schools

全文

(1)The Development of a Classroom Observation lnstrument for English Lessons in Japanese Junior High Schools. 教科・領域教育学専攻言語系コース. MO91311 濟木美沙.

(2) The Development ofa Classroom Observation lnstrument for English Lessons in Japanese Junior High Schools. A Thesis. Presented To. The Faculty ofthe Graduate Course at. Hyogo University ofTeacher Education. In Partial Fulfillment. Of the Requirements for the Degree of Master of School Education. by. Misa Saiki. （Student Number： MO913 11）. December 2010.

(3) Acknowledgement. This thesis owes much to the thoughtfu1 guidance and support of a number of. people and organizations， and 1 would like to thank all those who have helped me accomplish the study． 1 would first like to express my heartfelt thanks， admiration， and appreciation to. Associate Professor Tatsnhiro Yoshida， my seminar supervisor， for his support， encouragement， enthusiasm， and patience during the prolonged process of completing my first research． He always offered invaluable advice and insightfu1 suggestions，. which stimulated and sustained my interest in the study． He providing me the opportunity for conferences and classroom observations in j unior high school English classes that necessary for my research and these opportunities are based on my research．. 1 am also gratefu1 to the other teaching staff in the Department of English. Language at Hyogo University of Teacher Education． They are providing me with valuable oppo血mity of studying language education in their maj ors， not only fbr education． Every lesson is interesting and important for me to have widened view． 1 am also gratefu1 to English teachers， Mr． Yoshitaka Ohyama， Ms． Makiko Okada，. Ms． Kumiko Koike， and Ms． Yumi Kido in Attached Junior High School of Hyogo University of Teacher Education． We had many conferences to discuss the ideas about. teacher assessment and they gave me a lot of suggestions and great insight about English lessons in Japanese classrooms． Iam also grate血l to English teachers who are the cooperator fbr web survey to. answering the questionnaire． Without their cooperatives， this study would not have been possible． 1 would also like to thank Mr． Mark Taylor， University of Hyogo， who provided.

(4) の一. ●1. me great support and suggestions about my research． Without their positive efforts， suggestions， and encouragement， this study would not have been possible． 1 am also gratefu1 to my seminar colleagues who encouraged me， discuss ideas，. and provided insights that have contributed to the study． 1 wish to express my gratitude for the help and support given by Mr． Zeng Gang， Mr． Katsunori Kanmbara，. Ms． Noriko Kawakami， Ms． Kana Nakabayashi， Ms． Keiko Kawaoka， Ms． Keiko Yamamoto， Mr． Jerry Huang， Ms． Keiko Uemura， and Ms． Yumi Osugi． 1 also express my heartfelt thanks go to my peers from the Department of English. Language at Hyogo University of Teacher Education and the entire students of Hyogo University of Teacher Education， for the sustained， sincere， and loving support and. encouragement they gave me throughout the academic years．. Finally， 1 would express my heartfelt thanks to my family． They always. encouraged and help me and provided me for the opportunity to study language education at Hyogo University of Teacher Education for two years． Without their help， 1 would not spend this great time in this graduate school．. Misa Saiki. Yashiro， Hyogo December 20 1 O.

(5) iii. Abstract. The present study explores a theoretical framework for observing lessons of. English as a Foreign Language （EFL） and aims to develop a lesson observation instrument， which will be utilized to evaluate and improve EFL lessons in j unior high. schools in Japan． Under the recent educational reform mandate， teacher evaluation is conducted in various local school districts fbr the pu叩ose of quality contro1． However， teachers tend to be evaluated in a top−down way and it seems that the evaluation system. does not necessarily provide teachers with opportunities to reflect upon their teaching and participate in their own professional development． We strongly believe that lesson. observation and evaluation should be utilized to promote teachers’ own professional. development and encourage their own understanding of the classroom practice． Although， the discussion will be restricted to English language teaching， we would like to design a lesson observation instrument， which can be used by teachers for their own. professional development． We wi11 base the development of the lesson observation instrument on the one. called the Sheltered lnstruction Observation Protocol （SIOP）（Echevarria， Vogt and. Short， 2009）． The SIOP was developed and used in United States of America. developed by the Center for Research on Education， Diversity and Excellence California University at Berkley （CREDE） and Center for Applied Linguistics （CAL）． It has been developed in order to evaluate a Sheltered Instruction （SI）， where English language learners （ELLs）， whose L l s are other than English， learn the content area. ㎞owledge as well as the lImguage． Success血l SI are supposed to lead the ELLs to the. main stream classes of the school． The SIOP consists of thirty assessment features，. which are grouped into eight main features； they are Lesson Preparation， Building.

(6) iv. Background， Comprehensible lnput， Strategies， lnteraction， Practice／Application， Lesson Delivery， and Review／Assessment． The SIOP （a） presupposes high quality SI lessons， which can be validated by modern theories of teaching and learning，（b） involves teachers as well as administrators， principals， colleagues， in the process of. evaluation and further planning of lessons， and （c） provides a reliable and valid observational sheet． In the present study we would like to incorporate the framework of the S IOP into. teachers’ lesson evaluation， which can be utilized in the Japanese EFL classroom by. considering the contextual differences between the schools in the US and those in Japan．. To modify the SIOP and develop a lesson assessment instrument， which suits Japanese j unior high school contexts， the following steps were taken． First， the thirty items in the SIOP were translated into Japanese by the author， and the list was discussed. and examined by several people． As a result， we obtained 62 assessment items under the following eight main features：（1） Lesson Preparation，（2） Building Background，（3） Comprehensible lnput，（4） Strategies，（5） lnteraction，（6） Practice／Application，（7）. Lesson Delivery， and （8） Review／Assessment． “Team−teaching” was added because it is a maj or characteristic of Japanese Engiish lessons． To examine the usefulness of the sixty−two items for assessment， we asked nineteen English language teachers to j oin the. survey and rate the items． They were all Japanese English language teachers， all of whom had taught at junior high schools for more than 10 years． After the survey， we calculated the mean scores of usefulness and standard deviations （SD） of the rating for. each assessment item． To diminish overlap among the items and to increase the feasibility of the assessment tool， the initial 61 items were reduced to thirty−two items．. Among the nine features， “scaffolding” included in “Strategies” was decided to be.

(7) v. separated because scaffolding by the teacher is considered very different丘om strategies．. As a consequence， our assessment instrument comes to thirty−two features in ten main. headings． To test the reliability of our observation instrument， we put it to use in the. Japanese j unior high school context and calculated the interrater reliability， following. the procedure described in Guarino， et al．（2001）． Three video−taped lessons were observed and rated by two raters independently． After collecting the evaluation sheets， Pearson correlations coefficient was calculated for all three lessons． The mean of the coefficients of the three lessons was ．907， which is considered an appropriate estimate of interrater reliability． However， individual coefficients of the rating of the three. lessons varied from ．818 for Lesson A to ．943 for Lesson C． A detailed analysis suggested that the inconsistency was due to the varied j udgment for interaction and scaffolding， which tended to require subj ective j udgment by the raters． Although some. inconsistencies of scores among the raters were observed， we believe our lesson observation instrument has been sufficiently modified to the Japanese context to evaluate lessons in a relatively reliable way．. We believe that lesson observation incorporating teacher intention will enrich the. classroom community and provide a means for teacher development． In conclusion， we suggested the refinement of the lesson evaluation instrument be continued and teachers need to be involved in the development process．.

(8) vi. Contents. Acknowledgement．．．．．．．．．．．．．．．．．．．．．．．，．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． i. Abstract．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．iii. Contents．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．，．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．vi. Introduction．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．，．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．1. Chapterl Classroom Research and the SIOP．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 6. 1．1 Classroom interactional analysis ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 6. 1．2 The Sheltered lnstruction Observation Protocol ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 9. 1．2．1 Background of the development of the SI model instruction and the SIOP．．．． 9 12．2 The SIOP ”．””．””m”．””．”．”“”””．．，．””．”．．．””””．””．”””””．”””．．”．”．”．mm“”． 10. 1．2．3 The scoring by the SIOP．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 13. 1．2．4 The validity and reliability ofthe SIOP．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 15. 1．3 How can the SIOP be utilized in the Japanese context？．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 15. C血叩ter 2 The Development of a Lesson Observation lnstrument。．．＿．．．．．＿．＿．．．．．．．17 2．1 Purpose of the research．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 17. 2．2 Development of an observation instrument．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 17. 2．2．1 Reconsideration of the SIOP sheet ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 17. 2．2．2 A survey on the Web site ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 18. 2．2．3 Data analysis．．．．．．．．．．．．．．．．”．”．”．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 20. 2．2．4 Defining the assessment items．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 22. 2．2．5 The product of observation instrument．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 24. Ch叩ter 3 Reliability of the L，esson Observation Instrument＿＿＿＿．＿．．．．＿．＿＿＿．30 3．1 Aprevious study testing validity and reliability of the S IOP ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 30.

(9) vii. 3．2 lnterrater reliability of our observation instmment ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 31. 3．2．1 Raters ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．，．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 31. 3．2．2 The lessons observed ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 31. 3．2．3 Procedure ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 32. 3．3 Summary．．m．””．“．”．””””m．””m””．”m．．”””．．m．．．．””．．．”．．．．”．m．．．．．”m．”．．m．．．．“，．“． 34. Chapter 4 Discussion．．．．．．．．．．．．．．．．．，．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．35. 4．1 Revision ofthe provisional version ofthe observation instrument．．．．．．．．．．．．．．．．．．．．．． 35. 4．2 The necessity ofrater training as professional development．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 35. References ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 37. Appendix A： Deployment of new school staff evaluation system in the Miyazaki prefecture．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．・・…．．．．．・・・・・…e39. 回転pendix B： The SIOP evaluation sheet ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．，．．．．．．．．．．．．．．．．．．．． 41. 回転pendix C：Items of web−based survey＿．．．．．，＿＿．．．．．．．．．．．＿．．．．＿．．＿．．．．。．＿．＿．．．．．．．．．．＿．．．．．43.

(10) viii. List ef Tables. Table 1． The mean scores ofthe main features．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 21. Table 2． Pearson correlations coefficients between the two raters ．．．．．．．．．．．．．．．．．．．．．．．，．．．．．．．．． 33. Table 3． The congruence of the two raters．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 33. List of Figures. Figure 1． ltems in the teacher evaluation in Miyazaki ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 3. Figure 2． Schematic representation of design used in the “comparative method” studies （Long， 1983： 4）．”””““””””””””．．””“”．．”．．．．．．．．．．．”．．．”．．”．．．””．．．”“”．“””．”．．””．．．“．”．． 7. Figure 3． A sumrnary of eight features in the SIOP．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 13. Figure 4． A sample descriptor of the SIOP ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 14. Figure 5． A sample descriptor which includes NA ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 14. Figure 6． A sample image of the web−based survey （the face sheet）．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 19. Figure 7．Asample image of the web−based survey ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 20. Figure 8． The mean scores of the usefulness of assessment items ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 21. Figure 9． Assessment features of the lesson evaluation instrument ．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 24. Figure 10． A sample rating scale．，．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．． 25. Figure 11． A provisional version of the lesson observation instrument．．．．．．．．．．．．．．．．．．．．．．．．．． 25.

(11) 1. Introduction. The present study explores a theoretical framework for observing lessons of. English as a Foreign Language （EFL） and aims to develop a lesson observation instrument， which wi11 be utilized to evaluate and improve EFL lessons in j unior high schools in Japan． The observation instrument to be described in the following chapters aims to promote teachers’ better lesson planning and deepen the understanding of their teaching and students’ learning， and eventually facilitate their professional development．. The motivation lying behind the present study comes from our worry about the recent educational reform mandate， in which teacher evaluation is conducted in various local school districts for the purpose of quality control． ln most of cases， teachers tend to be. evaluated in a top−down way and it seems that the evaluation system does not necessarily provide teachers with opportunities to reflect upon their teaching and participate in their own professional development． Rather， it has caused a lot of. resistance against the evaluation among teachers． We strongly believe that lesson observation and evaluation should be utilized to promote teachers’ own professional. development and encourage their own understanding of their classroom practice． Although， the discussion will be restricted to English language teaching， we would like. to design a lesson observation instrument which can be used by teachers for their own. professional development． How can we develop such an instrument？ Before answering the question， we would like to look at the current situation of educational reforms in Japan and their impact on teaching．. Recent educational reform and teacher evaluation in Japan Under the recent mandate educational reform in Japan， the. enhancement of.

(12) 2. teachers’ professional development and the improvement of teacher quality has been an urgent issue and discussed both at the national level and at local district or school levels．. For example， the Ministry of Education， Culture， Sports， Science and Technology （hereafter MEXT） initiated a program called the Action Plan to Cultivate Japanese with English A bilities （MEXT， 2003）， which aimed to improve students’ language abilities. and enhance English language teachers’ professional development． ln the program， it. was mandatory that English teachers teaching at the secondary public schools participate in professional development programs which were offered by local boards of. education． The programs typically consisted of lectures and workshops amounting to. 20 hours． Further， the action plan recommended the use of external language proficiency tests such as the STEP test， TOEFL or TOEIC as obj ective measures to. indicate English language teachers’ preferable proficiency i． The action plan was implemented丘om 2003 to 2008， but the effectiveness and impact of the program on teachers’ professional development was not fully measured or surveyed by MEXT．. The reform of professional development is not limited to English language teaching． The Central Council for Education， an advisory board to the Minister of. Education， has been discussing the reform of teacher education and professional. development， and has published a report titled Redesigning Compulsory Education （MEXT， 2005）． The report suggests several strategies for improving primary and lower secondary education； in particular， it suggests that more autonomy be given to local authorities and schools letting them handle matters of personnel， budget allocation，. and class composition． Teacher evaluation， including lesson evaluation， is conducted and used as evidence ofthese administrative matters． 1. Butler and Iino（2003）argue against 1血e use of such tests to evaluate teachers’professional quality because the purposes of the tests are to measure English language abilities required for. purposes other than teaching．.

(13) 3. For example， the board of education in Miyazaki prefecture introduced a teacher. evaluation system in 2006． Since the content of the evaluation system is fully described in Kariya， et al．（2010） and thus easy to access， we would like to examine it. here． The system aims to evaluate teachers’ professional performance and achievement of their teaching practices at school． To improve the transparency and validity of the evaluation， the system utilizes evaluation sheets which are designed for different positions within the school （i．e． the principal， the vice principals， senior teachers， teachers， etc．）（For details of the evaluation sheets， see Appendix A）． The. evaluation sheet for teachers’ professional performance， for example， consists of three. evaluation categories； ‘instructional performance’， ‘professional attitude’， and ‘panicipation in and contribution to school management’．. Instructional performance Classroom teaching −planning . 一practlce. −evaluation Understanding and guiding students 一 communication ability with students 一 understanding of students 一 guiding and counseling students’．. Fundamental attitudes as a teacher Mission and ethics as a teacher. Self−management Paptici−tatilon in and contribution to school management. Planning Con−tn． ’bution to school organization jS2gulggyg1gp！pgpL−lfdlt. Negotiation skills Figure ！． ltems in the teacher evaluation in Miyazaki.

(14) 4. Teacher’s performance is basically self−evaluated by the teachers themselves， and. the evaluation sheet is submitted to the principal of the school， who examines it and. adds comments in the space provided in the sheet， As is evident， however， lesson evaluation is only a part of the whole evaluation． Since we assume teachers benefit. most丘om knoWing of how their own lessons are evaluated， we think this category should be described and assessed in more detail．. The Present Study Although teacher evaluation， which has been encouraged and introduced by the. local boards of education， aims to foster teachers’ professional development and improvement of school quality， they are utilized for personnel purposes or， in most cases， school accountability． However， we believe teacher evaluation， whether it is self−assessment or other−assessment， should contribute to the professional development. and innovation of classroom teaching． lt is from this key point that we began to. develop our own lesson eval耐ion instrument． In the following chapters， we would. like to describe how we developed such an evaluation instrument． First， the development of our instrument was based on the Sheltered lnstructional Observation Protocol （SIOP， hereafter）， which was created and used in the United States to improve. English language lessons for English Language Learners （ELLs）． ELLs， which used to be and are still called English as second language （ESL） learners， do not share a first language and cultural background and， thus， it is often the case that they are taught in so−called ‘pull−out’ classes， where the aim is to master subj ect concepts （i．e． science，. social studies or mathematics） along with the English language in order to integrate. them into the main stream classes． Therefore， teachers teaching ELLs need to be equipped with teaching principles and teaching techniques specially tailored to this.

(15) 5. context． This set of principles and techniques is referred to collectively as Sheltered. Instruction （SI）， and the SIOP has been developed to evaluate SI and to facilitate professional development． ln Chapter 1， we will look at SI and the SIOP in detail and. discuss how we can create our own lesson evaluation based on the SIOP． Chapter 2 describes the procedure of how our evaluation instrument was developed． We discuss. the issue with junior high English teachers and conducted a web survey with 19 experienced language teachers， asking them to score the assessment items． We then identified 32 items， which we believe are particularly effective for evaluating English. Ianguage lessons conducted in Japanese junior high schools． ln Chapter 3， the 32 items are put to use to evaluate video−taped lessons． The lessons were rated by different. raters and the interater reliability is confimied． We discuss some of the findings of the. lesson evaluation and its application to the actual evaluation in Chapter 4． We conclude our study by discussing some remaining issues regarding lesson evaluation in English language teaching．.

(16) 6. Chapterl Classroom Research and the SIOP. In our introduction， we briefly discussed how the teacher evaluation system has. been introduced in the Japanese schools under the current educational reform． lt was found out that these systems were implemented in a top−down fashion as part of the quality control of schools and， as they were connected to the personnel affairs ofteacher，. a resistant auitude toward the system is observed on the teachers’ side． Thus， the. introduction of the new evaluation system may not lead to sound professional development for teachers． To enhance teachers’ professional development， it is necessary to involve teachers in the evaluation process and to enable them to look at. how they actually teach in the classroom． This is the motive丘om which we believe a. classroom observation instrument should be developed． ln order to develop a good lesson observation system for teachers， we would like to review some previous studies. of classroom observation or interaction analysis in language teaching． However， we. must admit that in these studies attention is paid to the teacher’s behavior or teacher−student interaction， by outside observers， i．e． researchers． ln the following. section， we will critically review some previous studies on interaction analysis and argue that the studies did not necessarily contribute to the teachers’ professional. development．. 1．1 Classroom interactional analysis Long （1983） and Chaudron （1988）， although dated， are both seminal papers on. classroom research and provide us with informative summaries of research conducted in the past． Long （1983） did a meta−analysis of twenty−two classroom research inquires. conducted during the 1960s and 1970s． Classroom research conducted in those.

(17) 7. periods is defined in the following way；. Research on second language learning and teaching all or part of whose data are. derived from the observation or measurement of the classroom performance of teachers and students （p．4）．. He illustrated the basic assumption lying behind the interaction analysis studies at the period in the diagram below．. Control this． Measure this iititilill［1．一．［1［lput V＞ rg1ieEugggLiassroom 一ttput＞ i L／giliApntq’ X” students’. students，. teachers， methods， etc．. 2. second language. evnare. Deduce what. went on. Figure 2． Schematic representation of design used in the “comparative method” studies （Long， 1983： 4）. As the title of his paper suggests， the classroom was considered a ‘black box’ and. what happened inside the box was either deduced from the data obtained or worse， simply ignored． The researchers paid attention to the correlation between teachers’ behavior in the classroom （input） and the outcome of students’ learning （output）．. Those studies typically dealt with the data by coding the teacher’s and learners’ utterances or their behaviors based on predetermined category systems and rated the quality or counted the frequency of particular behaviors， ranging from “high” to “low”. or “very frequent” to “never”． Moskowitz （1976）， for example， followed Flanders’ pioneering work called FIAC system （Flanders， 1970）， which investigated L l classroom interaction， using a coding system in which observations were tallied in 34 categories．.

(18) 8. In the system， called the FLint system， classroom interaction was observed and recorded. every three seconds and later quantitatively analyzed． ln a similar vein， Fanselow. （1977） developed a classroom observation system called FOCUS， which used four maj or categories， “to structure”， “to solicit”， “to respond”， and “to react”．. As another seminal work on classroom analysis， Chaudron （1988） reviewed 26. studies oonducted from the 1960s to the early 1980s and categorized them into particular types of interaction analysis based on the purpose， method， and specific focus. of the study． She further categorized traditions of classroom interaction research into. four： psychometric， interaction analysis， discourse ana！ysis， and ethnographic． Moskowitz （1976） and Fanselow （1977）， mentioned above， were included in interaction analysis． Characteristics of each different research methodology will not be described in detail． However， we would like to discuss the significance of the issues raised by. Chaudron （1988） and their relevance to professional development． As fundamental issues to be discussed in classroom research， Chaudron raises the following four issues；（1） the relationship between quantitative and qualitative methods of the approaches，（2）. the instiumentation and basic categories of analysis used in classroom observation，（3）. the different dimensions used in multidimensional instruments for observation and analysis， and （4） the reliability and validity of these instruments and other observations. for analysis of classroom behaviors （p．15）． These issues are certainly significant for. the researchers， whose interest is to explain how and why classroom interaction takes. place． However， it is not clear how investigation into these four issues and the research findings contributed to the improvement of classroom teaching and teachers’ professional development． This is the point， where we feel the need to develop a. classroom observation instrument designed from the practitioner’s perspective． lt was not until the turn of止e cen鱒that we were able to see a systematic lesson.

(19) 9. observation instrument actually used in止e classroom；that is， The She〃ered lnstruction Observation Protocol （SIOP， hereafter）（Echevarria， Vogt， and Short， 2009）． ln the following section， we will describe the theoretical premises it is based on and discuss how it is actually used in the classroom．. 1．2 The Sheltered lnstruetion Observation Protoco1 1．2．1 Background of the development ofthe SI model instruction and the SIOP. The Sheltered lnstruction Observation Protocol （Echevania， Vogt， and Short，. 2009） is a framework that was developed and used in the United States of America. developed by the Center for Research on Education， Diversity and Excellence Califomia University at Berkley （CREDE） and the Center for Applied Linguistics （CAL）． The development of the SIOP began in the late 80s when an increasing number of students from different cultural and language backgrounds immigrated to the U．S．， and the number of limited English proficient （LEP） students， or simply called. English Language Learners （ELLs）， dramatically increased in the classroom by the mid 1990s． In order to be integrated into the main stream classes， where native speaker students and proficient language learners learn， ELLs typically learn English as well as content areas（e．9． science， math， social studies and etc．）in the so−called pull−out classes．. Teachers are thus required to master instructional skills to improve their content area. knowledge as well as language skills． The demand on language teachers was further. stepped up when the No Child Left Behind Act was proposed and passed under President George W． Bush in 2001． Since then， each state has set the academic standards to be achieved by Grade K−12 in public schools and standardized tests were. administered in the schools to provide evidence of effectiveness． NCLB Act had a significant impact on the schools located in the areas where LEP students resided.

(20) 10. because they were at a disadvantage with regard to language proficiency as well as. academic competence． Thus， it was mandatory that teachers demonstrate that their. instructional skills were effective enough to improve LEP students’ language and academic perforrnance to satisfy the standards． ln order to respond to the language. teachers’ needs to evaluate their own lessons， CREDE and the CAL collaboratively began a proj ect the aim of which was to （1） develop an explicit model of SI，（2） use the. SI model to train teachers in effective sheltered strategies and （3） conduct field experiments and collect data to evaluate teacher change and the effects of S I on LEP. students’ English language development and content knowledge （Echevarria， et al．， 2008）．. 1．2．2 The SIOP. The SIOP is止e instrument used to observe， rate and provide feedback on SI. based lessons． lt “refers to both the observation instrument for researchers， administrators， and teachers to match the implementation of a lesson to the model of. instruction and the instmctional model for teachers to plan and deliver lessons” （Echevarria， et al， 2008： 16）． The SIOP evaluates lessons based on empirical data generated by the research of successfu1 teachers’ instructional behavior． Specifically， the SIOP has thirty assessment items， which are categorized into eight maj or features；. they are Lesson Preparation， Building Background， Comprehensible lnput， Strategies， Interaction， Practice／Application， Lesson Delivery， and Review／Assessment． Lesson Preparation consists of the lesson planning process， including language. and content obj ectives， the use of supplementary materials， and meaningfu1 activities．. Building Background has the teacher make co皿ections between students’leaming background or experiences and the lesson． lt aims to provide meaningfu1 learning.

(21) 11. contexts for the learners． Comprehensible lnput， the notion claimed by Krashen （1985），. has a（加st the rate of speech or use multimodal tec㎞iques to enhance students’. comprehension of the content areas as well as the language itself． To achieve Comprehensible lnput， teachers need to pay close attention to students’ language use and the challenges students have in their learning processes． Strategies aim to help. students with their cognitive processing， such as monitoring， thinking and assessing their own learning． Strategies are based on findings from the research done in the field of learning strategies （e．g． O’malley ＆ Chamot， 1990）． Under Strategies is included scaffolding technique which refers to the teacher’s intervention into students’ learning．. The notion of scaffolding is associated with Vygotsky’s Zone of Proximal Development （ZPD）， where children can achieve more than they can do by themselves when support and guidance from experts， or teachers in our case， are available （Vygotsky， 1978）． For. ELL learners， strategy use is considered integral in the development of academic and higher thinking skills． Since the purpose of the S I is to enable ELP students to. improve their academic achievement so that they can be included in the main stream classes， lnteraction is a very important feature． Teachers need to provide structured. opportunities by configuring effective pair or group work and inviting students into active interaction in which students use not only social language but also academic language． To develop sound concepts in the content areas， their first languages should. also be used in effective ways． ln Practice／Application， students are provided with. oppo】血nities to al）ply the㎞owledge Imd skills they Ieamed in previous activities、 Teachers thus need to design tasks and activities in which students apply content area knowledge to the activities while using a variety of language skills （i．e． speaking，. listening， reading and writing）． Lesson Delivery assesses how well a lesson and activities are organized to meet the obj ectives and to support students’ learning． lt is.

(22) 12. necessary for teachers to deliver the lesson in a way that maximizes student engagement in the activities． Finally， Review／Assessment not only addresses “what teachers do at. the end of a lesson to see if students have learned what was intended． Rather，. reviewing and assessing occur throughout each lesson and then again as the lesson concludes” （p．168）． ln the SIOP， no hierarchy is assumed among the eight features；. they are interrelated and integrated into every iesson， although some emphases may be given to particular features within a lesson． Thus， the authors of the SIOP strongly. recornmend to observe consecutive lessons with one unit rather than to focus on one single lesson． Otherwise， you may fail to understand the intention of a particular teacher behavior or activities and end up with a shallow observation of them． The eight main features in the SIOP are sumrriarized in the table below．. Objectives. Main Features. Lesson preparatien. Content obj ectives， Language obj ectives， Appropriate content. concepts， Supplementary materials， Adaptation of content， Meaningfu1 activities. Building backgrollnd. Concepts linked to students’ background， Links between past. learning and new leaming， Developing key vocabulary：. academic language. Comprehensible. Appropriate speech， Clear explanation of academic task， A. Input. variety of techniques used. Strategies. Learning strategies， Scaffolding teclmiques， Higher−order. questtoning. Figure 3． A summary of eight features in the SIOP．（continued）.

(23) 13. Interaction. Frequent opportunities for interaction， Grouping configurations， Sufficient wait time， Clarify concepts in L l. Practice／Application. Hands−on practice with new knowledge， Application of content and language knowledge in new ways， lntegration of all language skills. Lesson delivery. Support content and language obj ectives during lessons， Promote student engagement， Pace lesson appropriately. Review／Assessment. Key vocabulary， Key content concepts， regular feedback on student output， Assess student comprehension of obj ectives. Figure 3． A summary of eight features in the SIOP．. The SIOP further specifies 30 assessment items for the eight features， which are. contained in the SIOP evaluation sheet （See Appendix B）． The evaluation can be conducted during the lesson or after the lesson． Evaluation of a watching a video−taped lesson is also possible． Teachers， principals， supervisors， and researchers can evaluate lessons by the SIOP， and it is important for teachers to be involved in the post−lesson. observation and scoring and discuss the lesson with other observers or colleagues．. 1．2．3 The scoring by the SIOP Raters score items on the SIOP evaluation sheet using a five−point scale （from. 4 to O）． A descriptor allocated to each assessment item helps the rater determine levels of attainment for each item （See Figure 4）．. As we will discuss later， however， people who use the SIOP need to be trained. to understand the purpose of each item and to rate the items consistently throughout the. evaluation． Also， a space for comments from the evaluators i s provided on the sheet.

(24) 14. which the teacher of the lesson and the evaluators share in the post−lesson discussions．. Echevarria and Short （2004） recommend that raters participate in training sessions， in which “they can raise questions about different interpretations， seek clarifications， and make decisions about how to score” （p．39）．. 6 . Meaningful activities that. Meaning血l activities廿1at. No meaningful. activities. integrate lesson concepts. integrate lesson concepts. （e．g． interviews， letter. but provide few language. that integrate lesson concepts with language. writing， simulations， models. practice opportunities for. practice. with language practice. reading， writing， listening，. opportunities for reading，. andror speaking. writing， listening， and／or. speaking. Comments： Figure 4． A sample descriptor of the SIOP. One thing to be noted about rating is that， since it is not necessary that all the. features are present in one lesson， some assessment features have scoring options． For. example， Figure 5 shows SIOP Feature 19 “Ample Opportunity for Students to Clarify Key Concept in L l”． This feature includes “not applicable （NA）” in its scoring area because not all SI lessons need to use students’ L l s in clarifying key concepts． This is. more evident in the advanced learners’ classes．. 19． Ample oppo血mltles. Some oPPo血nities fbr. No oPPo血mities fbr. for students to clarify. students to clarify key. students to clarify. key concepts in Ll as. coneepts in Ll. key concepts in Ll. needed with aide， peer or L 1 text. Comments： Figure 5． A sample descriptor which includes NA.

(25) 15. 1．2．4 The validity and reliability of the SIOP. The SIOP has been developed based on the observation and empirical studies of. high quality SI model lessons， where teachers interact with students， integrating language learning into content area leaming， scaffolding students’ learning． They adj ust their speech （i．e． its speed， complexity） to foster students’ understanding． They. also adjust the tasks and activities to maximize students learning opportunities． Based on these observations， the validity and reliability ofthe SIOP has been confirmed by the researchers during the process of its development （e．g． Guarino， at el．， 2001）． We will. describe the procedure of establishing validity and reliability ofthe S IOP in Chapter 3．. 1．3 How can the SIOP be utiliZed in t血e J叩anese context？. The SIOP seems to have some advantages as an observational instrument because （a） it presupposes high quality SI lessons， which can be validated by modern theories of teaching and learning，（b） it involves teachers as well as administrators， principals， colleagues， in the process of evaluation and further planning of lessons， and. （c） an observation sheet has already been made available． ln the present study we would like to incorporate the framework of the SIOP into teachers’ lesson evaluation in the Japanese EFL classroom． However， our ovvn contexts are clearly different from the US context， where the SIOP is utilized， in the following points． First， English is taught as a foreign language not as a second language． ln the SI lesson， teachers aim to teach. not only language but also content area knowledge so that the students are later integrated into the main stream classes． This is a significant difference between the two educational contexts and it is fair to say that mastering language， i．e． vocabulary and. grammar， is the main focus in the Japanese English language lessons． However， the textbooks recently published include various topics such as environmental issues， peace，.