A Preliminary Report on the Montana Study Program
Giles PARKER
Study abroad programs form an important part of tertiary education. A sojourn in a foreign country may lead to benefits to language skills, an increased perspective of students' position in the world, the chance to earn important credits and a wealth of experiences that may not be available at home (Batchelder 1993).
Benefits to host and sponsoring institutions include financial profits and a growth III prestige (Milleret 1990).
The process to accreditation usually involves getting data on the success, effectiveness and efficiency (Brown 1995) of a program. What follows is a preliminary report on the Montana Study Program (MSP) which aims to show how the MSP is worthy of accreditation at Nagasaki University. The first half out lines problems in conducting traditional quantitative program evaluation. The second half describes the MSP based on qualitative data that was collected over three weeks III August 1997. It is hoped that this paper will aid the process to accreditation.
What is a study abroad program?
A typical program involves a stay of between two to SiX weeks at an educational institute in a foreign country. Accommodation may be in dormitories with other participants, or with homestay families. Weekday mornings are usually spent in the classroom and afternoons often involve free-time or less academic activities.
Programs also include excursions to sites of special interest such as museums and art galleries, and chances to go shopping. Anecdotal evidence suggests that many programs tend not to emphasize the quality of education, and indeed, many Japanese universities may concur with this attitude. It may be that financial concerns are more important to both the host and sponsoring institution, while participants are more interested in the quality of the experience. Such programs are important because they provide Japanese students a chance to 'touch' the foreign culture in a safe and regulated format without needlessly overburdening them with study.
Other programs are more academically oriented and provide a chance for participants to experience and benefit from a foreign education. These courses are often pre-sessional and designed to aid students who intend to study in the host country for an extended period. They may be accredited within the host country i.e., graduates from such a program can receive credits which are transferable to other institutions. Furthermore, credits from such programs are also often transferable to the home institution. This is an ideal and increasingly popular situation. However, it forces us to ask whether foreign programs are suitable for our students and
worthy of accreditation.
Problems with traditional program evaluation
Sponsors require hard data so as to decide whether a program IS suitable or not.
However, they often do not recognize the appropriacy of the method used to collect data (Alderson 1992). Often we search for a measurable change in participants, usually through collecting empirical data gained from tests. As Brown (1995) points out; "A classic example of what many people think an evaluation study ought to be is a quantitative, statistics-based experimental study designed to investigate the effectiveness of a given program. However, as we shall see, this raises many concerns.
A major problem is the definition of what we are to evaluate. A study abroad program usually aims to: 1) fulfill its objectives, 2) change participants' behavior e.g. increase their language skills, 3) change their attitudes e.g. teach them to be more multicultural, 4) help them master the course content. Evaluators need to be certain of what it is they aim to evaluate. For example, some objectives may not be observable or measurable, or may be too vague. Furthermore, showing change in behavior is not always easy, as Geis (1997) and Yamamoto (1992) conclude.
Kumagai (1992) and Iwakiri (1993) also conclude that measuring change in attitudes is difficult. There seems to be a lack of construct validation that is necessary III defining what exactly is to be evaluated.
Methodology is also a major concern. Standardized language tests are initially attractive ways of measuring change in language behavior. They are statistically reliable, easy to administer, and the results may lead to easy generalizations.
However, there are also numerous problems involved. Most importantly, as Bachman (1989) reminds us, should evaluators accept the definition of language abilities that informs the test? Such tests do not reflect the objectives or content of a program, unless they explicitly teach towards the test. Alderson (1992) points out that often standardized tests do not reveal much improvement, which may conflict with impressionistic evidence. Improvement may be the result of test-taking experience rather than the effect of the program. Furthermore, such tests do not explain why there is a change, or allow for unexpected outcomes (Bachman 1992).
Qualitative data
While statistical data is preferable in showing a program's prowess, it is not the only form of data necessary. A balanced evaluation also includes qualitative information. Brown (1995) presents a list of procedures and instruments that maybe useful for collecting qualitative data. This includes records analysis, observations, interviews and questionnaires. The next section describes an initial, qualitative investigation into the MSP based on these procedures.
Six aspects of the MSP were investigated. These are: 1) objectives, 2) curriculum, 3) syllabus and lessons, 4) criteria for hiring teachers, 5) criteria for choosing homestay families, and 6) initial outcomes. These were felt to represent a detailed description of the program. Procedures for getting data were 1) interviews, 2) observations, 3) questionnaires, and 4) on-going correspondence. The aspects and procedures are presented in table 1. Observations, questionnaires, and interviews with the director and vice director of the MSP and the vice director of the FVCC, and ex- and current participants were conducted during a three week period in August 1997. The vice director has also been very generous in continually supplying information in correspondence via e-mail.
Aspects to be investigated Procedure for investigation
Objectives Correspondence
Interview Documents
Curriculum Correspondence
Interview Documents
Syllabus and lessons Correspondence
Interview Observation Documents Criteria for hiring instructors Correspondence
Interview Criteria for choosing homestay Correspondence
families Interview
Initial outcomes Questionnaire
Observation
Table 1: Aspects of MSP and procedures for getting data.
A description of the Montana Study Program
Based amid the splendor of Western Montana, the MSP has handled more than 600 Japanese students since 1990. The MSP is organized under the Montana Non- Profit Corporation Act. This raises many interesting implications which will be discussed below. The MSP provides a six-week homestay program. Participants are taught a range of classes which are validated and accredited by The Flathead Valley Community College (FVCC). The credits are transferable to any university in America. Credits are based on over 200 hours of classes with qualified and experienced teachers during the SIX weeks. The MSP also provides excursions to local
areas of interest and vanous types of organized physical activities.
Objectives
When we evaluate a program, it is important to understand the initial and on- going objectives that motivate it. The top priority for the MSP is educational, i.e., to provide high quality education at the lowest cost. The MSP aims to improve participants' competence in English and to provide participants with a wider view of the problems and issues that affect the environment. It also aims to foster closer ties between Montanan communities and Japan, and to dispel pervasive and inaccurate stereotypes each nation holds about the other. Another objective is to introduce and increase autonomy and independence whereby students can begin to question the world around them and take greater responsibility for their learning.
The MSP is legally prevented from earning a profit. The commitment to quality is shown by the very favorable student-teacher ratio of 4: 1 which is the keystone to the program. In this way, the MSP claims that it can offer very high quality classes that will directly effect the students. Flathead Valley Community College is linked with the MSP because it is committed to creating a high quality, multicultural campus.
Curriculum
It is also essential to understand the curriculum of a program. Tpis is usually a document that sets out how a program will achieve its objectives. The MSP in 1997 offered six areas of study: 1) Computer Basics = 1 credit, 2) English Immersion
=3 credits, 3) Contemporary Environmental Issues =2 credits, 4) Physical Activities
& Recreation = 1 credit, 5) American Occupations & Education = 1 credit, and 6)
Regional American Culture & Customs =2 credits. These courses are validated by the Flathead Valley Community College (FVCC). The FVCC reviews and approves course syllabi and instructor credentials, evaluates courses and instructors, provides certification of completion and maintains transcripts in case a participant wishes to transfer her credits to another university. The FVCC also provides facilities for computer courses, library research and more academic oriented classes. (See Appendix A for an example of the agreement between the Flathead Valley Community College and the Montana Study Program.)
Syllabi
The syllabi IS the way in which the curriculum is realized and specifies what takes place during class. Daily lessons provide the focus of the MSP. Students are divided into groups of four per instructor based on assessment of a pre-program composition, a paper test and an interview. The favorable student-instructor ratio is an obvious advantage and may greatly benefit language competence.
Lessons are only in English. Students and their parents sign an agreement that says they will accept responsibility for being sent back to Japan if they disrupt classes by speaking Japanese, or if they act inappropriately. This strict code of conduct reinforces the MSP's commitment to quality. Lessons start when the instructor for the group for that day picks each student up from their home-stay family between 8.00 a.m. and 8.30 a.m. and drives them to the site for the day's lessons. This may be a timber yard, a coffee shop, a public gym, a golf course etc.
During the drive the instructor leads a guided discussion that is aimed to generate the concepts and vocabulary necessary for the next lesson. Lessons are usually 90 minutes in duration and finish at 4.00 p.m. This implies that students are getting content-focused language eight hours a day. (See Appendix B for an example of the activities and lessons for one day,) Students have very little free-time. MSP's policy is that students should be accompanied by a native at all times.
Instructors are allowed to create lessons that include different parts of different courses. For example, a lesson where students gain an understanding of the early American homesteader life style will go towards meeting the credits for English Immersion and Regional American Culture and Customs. Another lesson designed to show students a working daycare facility goes towards meeting the credits for American Occupations & Education. Each instructor must provide a lesson plan for each lesson of the day a few weeks in advance. These are then approved by the directors and the FVCC. (See Appendix C for example lesson plans,) Lesson content can be adapted depending on the needs of a particular group. This implies that the MSP may be responSlve to needs and suggestions expressed by sponsormg institutions. Indeed, the director and vice-director both agree that the MSP is able to provide 'specialist' courses if there are more than 12 students. This would provide an opportunity for the sponsoring institution to help create a course more reflective of its own students' needs.
Lessons are generally 'hands-on' in that it is felt that students learn better by going somewhere and actually using language to do something. The MSP tries to provide experiences whereby students can gain the ability and autonomy to take responsibility for their own learning. Instructors favor a style in which students are encouraged to ask questions. For many participants from Japan this may be a new but not unprofitable experience.
An obvious example is when students learn to use a computer by making their own address books. Apart from using English to learn a useful skill, they are also producing something meaningful.
Another example is when students go to a sports shop and decide what equipment they need for a two day hunting trip. Students have to ask shop assistants for information and make criteria for their decisions.
A last example illustrates an experiential treatment of literature. Whereas III
Japan literature is taught by translation, in the MSP it is presented as something with which the reader interacts. Thus, in studying A River Runs Through It, students learn how to fly-fish, take lessons by a river during which they discuss the novel in its original setting, and create posters to illustrate their impressions.
Instructors
The quality of the instructors is of great concern to sponsoring organizations.
It is also beneficial to know the criteria under which they are hired. MSP instructors are hired for their high energy and motivation. This is because they are teaching eight hours a day, five days a week, during their summer holiday. Instructors must also be sensitive to sudden cultural immersion on the part of the students, flexible enough to teach multilevel classes, and have experience in different kinds of classrooms. Generally most instructors work full-time at elementary schools or junior and senior high schools. All have post-graduate qualifications with more than a few qualified at Ph.D. level. Instructors must also see themselves as learners.
Indeed, American grade- and high school teachers are constantly under pressure to continue their education and improve their qualification or face losing their teaching certificate. Lastly, they need to be able to follow plans, and adapt from them if necessary. These qualities are ascertained by a rigorous interview process.
Homestay families
Homestay families form a very important part of the participants' expenence.
It is through interaction with their family that students realize the real use of language, that is, to interact with others in an open-ended context. Homestay families provide an opportunity to become deeply involved with American culture.
Participants are encouraged to spend as much time as possible with their family during the evenings and weekends. It is pointed out that this time also provides a chance for the families to learn about Japan.
Homestay families are chosen through a rigorous process of interviewing based on their responses to various criteria. They do not receive remuneration equivalent to their expenditure in allowing a student to stay with them. Therefore they are not motivated by profit, thought they do receive an honorarium. Families understand the philosophy of the MSP and must show devotion to education. They must show an attitude of willingness to learn from their guest and to aid their experience in Montana. Families range from lower middle class to upper middle class, from ranchers and farmers to lawyers and accountants. Students are matched with compatible families. If there are any problems, both the students and the families are encouraged to consult the directors, one of whom is Japanese which should allay fears of discussing a potentially embarrassing problem in a foreign language.
Initial outcomes
The MSP is not overly concerned with quantifiable data e.g., standardized norm- referenced test results to justify itself. It is recognized that collecting such data is questionable and difficult. However, in order to gain accreditation from sponsoring institutions it may be necessary to demonstrate its prowess.
Out of 600 students since 1990, approximately 50 or 12% have returned to America for further study at either graduate or undergraduate level. Currently 17 ex-participants are attending graduate school after finishing university in Japan and working for three or four years. Two ex-participants currently studying Masters courses at the University of Montana were interviewed. One is graduating in Forest Management while the other is studying Business. Both claimed that they would never have thought about taking post-graduate qualifications abroad were it not for the MSP. They also maintained that the skills and confidence they gained during their sojourn with the MSP made their entrance into American academic life smooth and successful. This implies that the MSP fulfills its own criteria and objectives.
Nine members of the English department at Nagasaki university took part in the MSP during summer 1997. Impressionistic evidence based on subsequent classroom performance back in Japan suggests a great improvement in language competence, supported by an increased confidence and assertiveness in all nine.
Another way to view the success of the program is to investigate participants' opinions about the MSP. A very simple post-program questionnaire consisting of two instructions, 1) In a paragraph, explain what you liked most about the Montana Study Program, and 2) In a paragraph, explain what you liked least about the Montana Study Program, revealed no obvious trend or theme for complaint in that everyone had something different to complain about. On the other hand, small class size, constant pressure to use English, and the warmth and generosity of the homestay families were often cited as positive aspects. (See Appendix D for an example post-program questionnaire.)
Areas that might benefit from improvement
This section briefly outlines some issues in the teaching style that might benefit from improvement. These comments are based on three weeks of observing and video-taping classes, interviews with the students, and comments in the post-program questionnaire.
1) Students might benefit from a more explicit outline of the classes' relationship to each other. It may also be useful to explain to the students the goals of each class.
2) Teachers might try to make classes more productive. This means students should have the opportunity to use their new skills and language more.
3) Students may appreciate the opportunity to teach each other. After all, they
are learning independence and self-confidence, why not let them take more control in some of the classes? This could be in the form of small research projects and presentations.
4) There is a need for some kind of formal assessment of the content at the end of the program. Brinton et.al. (1989) provide hints on evaluation of content-based instruction that maybe of some help.
5) A form of participatory evaluation (Alderson & Scott 1992) where evaluation is conducted by insiders and outsiders to their mutual benefit may improve the program. Despite the initial qualitative data, there may still be a need for statistical data to show the effect of the program. Participatory evaluation would help in solving this problem.
Conclusion
Evaluation IS an important part of accrediting a program. This paper provides an initial report on the Montana Study Program using data gained through different procedures during a three week sojourn in 1997. It is recognized that statistical data would increase the information we have about the MSP. The next stage is to conduct a participatory evaluation to investigate the empirical effects. It is hoped that this report will further the discussion.
Acknowledgment
This paper owes a lot to the prompt and detailed explanations given by the program vice director, Gary Cook. I would also like to thank Kathy Hughes at the FVCC for sparing time for an interview, and all the participants, teachers and homestay families who helped by answering my questions. Any mistakes are my own.
Bibliography
Alderson, J .C. 1992 Guidelines for the evaluation of language education. In Alderson, J .C. and Beretta, A. (Eds,) 1992.
Alderson, J .C. & Scott, M. 1992 Insiders, outsiders and participatory evaluation. In Alderson, J.C. & Beretta, A. (Eds.) 1992.
Alderson, J.C. & Beretta, A. 1992 Evaluating Second Language Education. Cambridge:
Cambridge University Press.
Bachman, L.F. 1989 The development and use of criterion referenced tests of language ability tin language program evaluation. In Johnson, R. (Ed.) 1989 The Second Language Curriculum. Cambridge: Cambridge University Press.
Batchelder, D. (1993) The green banana. In Theodore Gochenour (Ed.), Beyond Experience: The experiential approach to cross-cultural education. Maine: Intercultural Press.
Brinton, D., Snow, M. & Wesche, M. 1989 Content-Based Second Language Instruction.
New York: Newbury House.
Brown, J.D. 1995 The Elements of Language Curriculum. Boston: Heinle & Heinle
consideration of the meaning of the summer program in the United In The Kyushu Academic Society of English Language Education Geis, K&E Fukushima, C. 1997 Overview or a shady Abroad Program. In The Language Teacher 21. 11.
Iwakiri, M. 1993 Effects of a study abroad program on the development of Japanese college students. In The Japan Association of College English Teachers Bulletin. No.
24.
Kumagai, M. 1992 A States of America.
Bulletin No. 20.
Milleret, M. 1990 Evaluation and the summer language program abroad: A review essay.
The Modern Language Journal 74/4
Yamamoto, H. 1992 Effects of an overseas short term intensive English program, on the development of English proficiency among participating students. In Bulletin of Seinan Jogakuin Junior College. No. 39.
Appendix A
MEMORANDUM OF AGREEMENT
THIS AGREEMENT is entered into this 1st day of May 1997 between
Flathead Valley Community College. hereinafter referred to as the "College" and the following named organization hereinafter referred to as the "Program".
Name:
Address:
The Montana Study Program 4366 Rocky Point Road Polson. Mf59860
Contact Person: Mac Swan, President Telephone: 883-5479
WHEREAS. the Program desires to engage the College to provide academic credit courses:
THEREFORE THE PARTIES AGREE AS FOLLOWS:
1. In accordance with College policy. the College will prOvide the following courses(s):
CMPA 170 ENGL 170 ENVL 170 HPER 170 IDS 170 SSCI 170
Computer Basics English Immersion
Contemporary Environmental Issues Physical Activities and Recreation Amertcan Occupations and Education Regional American Culture and Customs
1credit 3 credits 2 credits 1credit 1 credit 2 credits
2. The Programwill recommend currtculum. and Program representatives will participate in the screening and interviewing of instructors.
3, The College will award academic credit. and adm1n1ster and maintain enrollment and grade records. In addition. the Collegewill: Review and approve course syllabi and instructor credentials: hire instructors; evaluate courses and instructors; provide certificates of completion.
4. The Program will recruit Japanese college students. provide a homestay program for students.
arrange for student transportation and social activities.
5. The College will review Program produced printed matertals and advertisements regarding College courses prior to release.
6. The Program will be properly licensed or registered as reqUired by the laws of the State of Montana.
7, Liablllty insurance coverage specifically for the program will be the responsibility of the Program.
8. The College's liability insurance will selVe as a second insurer to the instructor's personal insurance when transporting students In instructor vehicles.
Appendix B
July 17
9:00--10:30 10:30-1:00 1:00--2:30 2:30--4:00
Sue groups 5, 6, 7, 1
Computer Lab
group 1 group 1 group 1 group 7
Morris Eagle Bend Golf Creston Fish Tri-Gty Lumber
Course hatchery 41 Reserve Drive,
837-7300 758-6868 Kalispell
752-9663
group 2 group 2 group 2
Dawn
270 Pierce Lane, BigFork Kootenai Gallery Montana Athletic
837-3760 Oub
group 3 group 3
Lee
Conrad Mansion Lone Pine State Park, Kalispell
group 4 group 4 group4 group 4
Mary
Woodland Park FVCC Rcpm 140
group 6 group 5 group 5
Trish Mapping your way Montana Veterans
through downtown Home Creston Fish Hatchery/Echo Lake Kalispell . Columbia Falls
group 7 group 6
John
Mike Manning Farm Sykes GroceE}'
298 Fairmont Road, Kal. 202 2nd Ave W., Kal.
Appendix C
Montana Study Program Lesson Plan Summer 1997
Roxanne McHenry
Course Name: Home, Work and Family
Course Objective: To give students an overview of topics important to families. To compare family life in the Flathead with other parts of the USA. The course
incorporates research in libraries, on computer, and hands-on activities, as well as some tours.
Lesson 1 Introduction and Course Overview
Lesson Objective: To introduce students to what we will be seeing and how it relates to American home-and family. We will talk about course requirements of library/computer research, asking questions and interviewing, as well as preparing a short written synopsis highlighting what the student finds interesting during this course.
Curriculum Area: ENGL 170 English Immersion - 1 hour
Activities: Question students about what they already know regarding American home and family. Show students an example page of written synopsis.
Vocabulary: recycling, day care, parenting, police, decision making, nutrition, charity, research, recipe
Location: 1235 4th Avenue West, Kalispell
Contact person/phone: 756-9636
Car Lesson: Introduce instructor, have students tell about themselves.
Travel Time: 15 minutes.
Lesson 2 Research Theory and Interviewing Technique
Lesson Objective: To teach the students how to ask questions. Show them what kinds of questions are appropriate according to circumstances. Show the students how to write down quoted statements. To talk about footnotes and documenting sources of information.
Curriculum Area: ENGL170 English Immersion - 1 hour
Appendix D
PROGRAM EVALUATION
1. In a paragraph, explain what you like most about the Montana Study Program.
Tirst of all, I truly feel happy to meeting all of the people I met in Montana. The time I spent with my host family, the teachers and the students who participated in this was the most precious time for me. I liked the policy not to speak Japanese even when we talk with Japanese people. Though it was very difficult not to speak Japanese, it was really important to think in English when we try to speak English. I truly enjoyed not only the classes I'had during this program but also the chatting time we had in the ear all way from my house to FVCC and back to my house every day. My pick-up teacher was really great!!
We telked many things in the car. She always shared her opinion and thoughts with us.
I learned a lot from those chatting. I really enjoyed my stay in Montana. I will never forget about my. precious time in Montana.
1. In a paragraph, explain what you liked least about the Montana Study Program.
I wanted to have the mixed group on the feld days. We always had the same members every day, so I wanted to talk more with other students.
I think to know how the other groups or students doing makes it better to improve our English.
