Assessing Students’ Vocabulary Profiles for Better Selection of Course Materials: A Case Study

Better Selection of Course Materials: A Case Study

著者 ISHII Tomoko

journal or

publication title

明治学院大学教養教育センター紀要 : カルチュー ル = The MGU journal of liberal arts studies : Karuchuru 

volume 12

number 1

page range 113‑121

year 2018‑03‑25

URL http://hdl.handle.net/10723/2402

Introduction

　　Since the publication of “Vocabulary: A Neglected Aspect of Second Language Acquisition”

(Meara, 1980), attention to vocabulary research has continued to grow. Decades of research since then have established the importance of vocabulary in second language learning and use. A strong correlation has been suggested between vocabulary knowledge and other skills, such as reading and writing; consequently, there is no debate over the importance of vocabulary. Given the significance of vocabulary, this paper presents a case study in which a selection of English language course materials was evaluated through assessment of the vocabulary profiles of first-year, non-English-major university students.

　　Although the importance of vocabulary is widely recognized, Japanese university students seem not to pay sufficient attention to expanding their vocabulary. Many high school students study English intensely in order to pass the university entrance exam. However, after entering the university, although some interested students continue studying English, quite a few do not. For many of them, compulsory English courses are often their sole contact with English in their first year in college.

With the pressure of job hunting, they resume studying English with the aim of attaining a higher score on the TOEIC

, only to find in many cases that their vocabulary has diminished. Okamoto (2007), for instance, tracked the vocabulary size of Japanese students during their first semester at college and found that they had lost about 25% of their vocabulary after only three months.

　　Because compulsory English courses are often students’ sole contact with English, it is important to ensure their exposure to vocabulary appropriate to their level to give them a better chance to retain it. This paper presents a case study that examined materials provided in a compulsory English course from the perspective of students’ vocabulary profiles and the kinds of vocabulary the materials provide.

　　The next section, first, provides a brief review of some key issues covered in past research on vocabulary, focusing on those that are particularly relevant to the current case study. The subsequent sections then describe the vocabulary knowledge of the students when the course started, the

Assessing Students’ Vocabulary Profiles for Better Selection of Course Materials: A Case Study

Tomoko Ishii

materials used in the course, the methods used to conduct the study, and the results, which reveal how their vocabulary profiles had changed by the end of the semester. The conclusion discusses implications of the study and offers suggestions for further research.

Literature Review

　　Among the many vocabulary-related research topics, two particularly well investigated ones are which vocabulary items should be learned and how they are to be learned. On the first aspect, corpus linguistics has shown that heavily used vocabulary items are quite limited in number, covering a considerable percentage of the words used in texts. According to Nation (2013), for instance, the most frequent 1000 word families in the British National Corpus cover more than 80% of the corpus he examined, with the next 1000 words adding about 8%. With 4000 word families, the coverage is as high as 95%. Therefore, although acquiring a vocabulary comparable to that of native speakers is a great, almost impossible challenge, those who learn English as a second language can expect to gain a reasonable command of English with a much smaller vocabulary. Many attempts have been made to identify the vocabulary items that constitute the basic vocabulary for many decades (e.g., West, 1953) and continuing into the present (e.g., Browne, 2014).

　　As for how vocabulary items should be learned, a distinction often made in the field is whether the items are learned deliberately or incidentally. Deliberate learning, on the one hand, refers to the type of learning where the main focus is on vocabulary. Typical activities for this type include trying to memorize vocabulary items by means of wordbooks or flashcards. On the other hand, the incidental type refers to learning whose focus is something else, such as reading and speaking, and vocabulary is learned as a secondary outcome. These approaches should be regarded as complementary rather than competing, as both have strengths and weaknesses.

　　In incidental learning, the learners meet new words in meaningful contexts, and the available information about the words tends to be much richer than in deliberate learning. For instance, learners can see the grammatical patterns in which the words are used and with which other words they collocate. In deliberate vocabulary learning, on the other hand, information of this sort tends to be limited.

　　The advantage of deliberate learning is that it is often more efficient in terms of time. The learning target is clearly specified, and learners can focus on what they are supposed to learn. The abundant literature on incidental vocabulary learning through reading (e.g., Nation & Wang, 1999;

Waring & Takaki, 2003) points to two things. First, learners need to know a very large proportion,

such as 95%, of the words in the passage they read in order to understand the story and guess the

meaning of unknown words from the context. Second, learners need to meet new words repeatedly in

order to learn them. Meeting a word once or twice is often not enough to retain it in memory. Thus, before starting to learn vocabulary through reading, learners need to have a reasonable amount of vocabulary already, and they need to spend a large amount of time reading if they expect to learn a significant number of vocabulary items.

　　In addition, when learners encounter a new word in incidental learning, they must figure out its meaning, often by guessing from the context. Two concerns arise here: the risk of not being able to guess the right meaning and uncertainty as to whether information obtained this way is committed to memory. If learners can make sense of the material, they may not pay any extra attention to the vocabulary items. Thus, although reading and listening are important skills to develop, they are likely not the most efficient when considered as vocabulary learning activities.

　　Given the strengths and weaknesses of both approaches, it is best if the two are combined for learning English. Learners need to focus on selected vocabulary items to master basic vocabulary, especially in the early stages of learning. Basic vocabulary helps with accomplishing meaningful language activities, in which students may learn further vocabulary items and which can also be a chance to cultivate a deeper understanding of the words they have learned.

　　These research findings are often reflected in textbook designs. A very large proportion, if not all, of the wordbooks currently published are based on some sort of corpus analysis. Many wordbooks available on the Japanese market are based on a particular test (e.g., TOEFL

, TOEIC

, university entrance exams). Typically, past exams are tallied to make a corpus of a particular kind of test, and the wordbooks list the vocabulary items according to their frequency of appearance.

　　Some books lean toward a more deliberate learning style, with limited information for each entry so that learners can cover a large number of words in a short time. Others embed the target items in passages so that students can encounter the words with richer information about how they are used.

Learning words through such wordbooks is different from the incidental learning described above, as the primary focus is on vocabulary. However, the idea behind the design of such books is certainly connected to the benefit of learning words in context.

　　When selecting and examining course materials from the perspective of vocabulary learning, these two aspects should be taken into consideration. Namely, the vocabulary items students are expected to learn deliberately need to be checked, as well as what input is provided in non- vocabulary-focused activities to ensure repeated encounters with the words the learners already know. This paper reports a classroom study conducted in a mandatory English course at a university in Japan in order to examine whether the vocabulary items presented to the students matched their vocabulary needs. The following research questions guided the study:

　1)　What were the students’ vocabulary profiles like at the outset of the semester?

Assessing Students’ Vocabulary Profiles for Better Selection of Course Materials: A Case Study

　2)　Did the vocabulary presented in the materials match the students’ vocabulary profile?

　3)　Did the students’ vocabulary profiles change over the course of a three-month semester?

　　The study assessed the vocabulary profiles of the students at the outset of the semester, right after they had entered the university, and compared their profiles against the vocabulary presented in the materials used in the course. The students’ vocabulary knowledge was re-assessed three months later at the end of the semester to track the changes in their vocabulary.

Class Settings Students

　　This study took place in a first-year university EFL course. The class met once a week for 14 weeks. All 26 students in the class, 15 females and 11 males, were majoring in politics, and this course was mandatory for them. According to the instructor’s observations, they were a friendly group of students who took the assignments given in the course very seriously. However, as they were non- English majors, not all of the students were particularly keen on developing their English ability.

Materials

　　The course focused on a) vocabulary development and b) pronunciation practice. About 40 of the 90 minutes of each lesson were devoted to vocabulary exercises using DIALOGUE 1800 (Akiba

& Mori, 2012) as the text. This book targets learners aiming at a score of 700 on the TOEIC

. The target expressions consist of either a single word or a phrase and are embedded in a dialogue of 200‒250 words. A total of 1800 expressions are covered in the book across 108 units, meaning that 16‒17 expressions are introduced in one unit on average. With the limited class time, the course covered only 12 units out of 108. In those 12 units the learners studied 190 target expressions and read dialogues with a total of 2710 words.

　　According to the preface of DIALOGUE 1800, the target words are embedded in dialogues to provide readers information about how those expressions are used. In addition, although the authors do not mention this, some extra learning of non-target expressions can be expected to occur. As research on incidental vocabulary learning suggests, meeting the words repeatedly is essential to keeping them active in the learners’ vocabulary. By seeing relatively easy words, most of which are known and more frequent, alongside the target words, students can expect to solidify their knowledge of such words.

　　In addition to this, about 40 minutes were spent on pronunciation practice. The students were

presented passages written in simple English for this purpose. These passages were collected from

various sources, such as free online articles written for English learners. The total number of words in these passages was 1355.

Results

Initial Vocabulary Profiles

　　At the outset of the semester, data were collected on the students’ vocabulary profiles. For this purpose, vocabulary tests based on the New General Service List (Browne, Culligan, & Phillips, 2013a) and the New Academic Word List (Browne, Culligan, & Phillips, 2013b) were used. The New General Service List (NGSL) contains 2800 head words and the New Academic Word List (NAWL) has 963 head words. These two lists together cover 92% of the academic corpus (Browne et al., 2013a). The mastery of these two lists is therefore an optimal goal for those in academic settings. Multiple forms of vocabulary tests based on these two lists have been developed (Bennett, Stoeckel, & Ishii, 2016;

Stoeckel, Ishii, & Bennett, in press), and one of these versions was used in this study. A sample item from the test is shown below.

　justify: We cannot justify this.

　　　　　a. 打ち砕く 　　　　　b. 追求する 　　　　　c. 変更する 　　　　　d. 正当化する

　　The target word justify is embedded in a sentence just brief enough to indicate that it is a verb.

This follows the format of the Vocabulary Size Test (VST) (Nation & Beglar, 2007), which is widely used in the field. The original VST has brief English explanations as the options. However, in order to exclude the possibility that some learners miss the items because they cannot comprehend the options rather than because they do not know the target items, this study used a bilingual versions developed for Japanese learners.

　　The version used for the current study comprised 120 items: 70 items sampled from the 2800 words in the NGSL and 50 items from the 928 words in the NAWL. For testing purposes, the NGSL was broken into five bands of 560 words and the NAWL into two bands, and items were sampled from each level band. Band 1 of the NGSL test contained 10 items, whereas Bands 2‒5 contained 15 each because Band 1 list includes many function words such as the, it, and can and a smaller number of content words than the other bands. A smaller sample from this band would therefore ensure a fairer representation of the target words and slightly shorten the test-taking time. The students

Assessing Students’ Vocabulary Profiles for Better Selection of Course Materials: A Case Study

participating in this study spent 12 to 18 minutes completing the test.

　　The students’ average vocabulary profile at the beginning of the semester is shown in Table 1.

Only the data from the 23 students who took tests at both the beginning and the end of the semester are presented in Table 1.

Table 1　The Initial Vocabulary Profile of the Students (% of Correctly Answered Items; N = 23)

NGSL NAWL

Band 1 Band 2 Band 3 Band 4 Band 5 TOTAL Band 1 Band 2 TOTAL

Average 95.22 94.14 90.70 83.17 80.26 88.23 57.74 66.96 62.35

Minimum 70.00 86.00 80.00 60.00 53.33 32.00 40.00

Maximum 100.00 100.00 100.00 100.00 100.00 　 76.00 80.00 　

　　From the data above, we can see that the students in this case study almost hit the ceiling of the NGSL with very high average scores, especially for the first two bands. We see some decline from Band 3, however, indicating that they need to learn more of the words in these bands for mastery of the entire list, as the NGSL is a collection of words that support basic performance in English as a second language.

　　On the other hand, the low average scores for the NAWL sections clearly indicate the need for some assistance in academic vocabulary. The students showed a higher average score for Band 2 of the NAWL test, even though Band 1 is supposedly higher frequency and therefore more likely to be learned first. This could be a result of the sampling of the items from the two bands, or might reflect the fact that the actual frequencies of occurrences of words in these two bands do not greatly differ (Brown, 2012) and therefore do not properly predict the order in which students meet and learn the words.

In-Class Activity and Vocabulary Profiles of the Materials

　　About 40 minutes of the 90-minute class time were spent on activities related to DIALOGUE 1800. One dialogue was assigned for each class time, and the students first read the dialogue with blanks in place of the target expressions and discussed what the dialogue was about in small groups of three to four. They then filled in the blanks as they listened to the dialogue. They shared their answers in the same small groups and repeated the cycle two to three times, depending on how well students were doing each time. When 80‒90% of the blanks seemed to be filled in, the instructor asked the students to check their answers with the wordbook and took a few minutes to go over the target expressions of the unit. They listened to the dialogue once again as a wrap-up of this activity.

The following week, a vocabulary quiz was given, and students were encouraged to review the words

as a homework assignment. For the quiz, the students received the same worksheet as they had used

the previous week and were asked to fill in the blanks by listening to a Japanese translation of the

dialogue read aloud by the instructor. They were therefore encouraged to master the form-meaning mapping of the target expressions. The average percentage of correct answers on the 12 quizzes administered over the semester was 80.82, varying from a lowest of 61.40 to a highest of 100, and students generally seemed to have taken the homework and quizzes seriously.

　　The rest of the class time was devoted to pronunciation activities, wherein students participated in various read-aloud activities in addition to brief instructions on how to pronounce particular sounds in English. In each lesson, two to three sounds were featured as “today’s sound” and the students engaged in various activities, paying particular attention to those sounds. The activities included reading a short story aloud and performing a short skit in small groups.

　　Table 2 presents the vocabulary profiles of the materials used for these vocabulary and pronunciation activities. AntWordProfiler (Anthony, 2014) was used for this analysis. Proper nouns and colloquial fillers such as “oh,” as well as acronyms such as ATM and ICU, have been manually excluded from the word count.

Table 2　Vocabulary Distribution of the Materials Used in the Course

DIALOGUE 1800 Read-aloud materials

Number of

tokens % of tokens Cumulative

% of tokens Number of

tokens % of tokens Cumulative

% of tokens

NGSL Band 1 2015 78.22 78.22 1069 80.86 80.86

Band 2 195 7.57 85.79 74 5.60 86.46

Band 3 115 4.46 90.26 50 3.78 90.24

Band 4 50 1.94 92.20 20 1.51 91.75

Band 5 29 1.13 93.32 17 1.29 93.04

NAWL Band 1 7 0.27 93.59 1 0.08 93.12

Band 2 16 0.62 94.22 4 0.30 93.42

Off Lists 149 5.78 100.00 87 6.58 100.00

Total 2576 1322

Judging from these profiles, the materials do not appear to fully match the vocabulary needs of the students in this course. The students already had a reasonable mastery of NGSL at the beginning of class, particularly in Bands 1 and 2, and their weaknesses appear in the NAWL vocabulary. However, neither the vocabulary materials nor the read-aloud materials provided vocabulary input at an academic level.

Vocabulary Profiles at the End of the Semester

　　Table 3 shows the vocabulary profiles of the students at the end of the semester. Overall, students answered 91.80% of the items on the NGSL test correctly and 66.60% of those on the NAWL test, compared to 88.23% and 62.35%, respectively, at the beginning of the semester. Because of the limited number of participants, as well as the skewedness of the data in the case of the NGSL test,

Assessing Students’ Vocabulary Profiles for Better Selection of Course Materials: A Case Study

Wilcoxon signed-rank tests were conducted to compare the means of pre- and post-test scores. The results showed that the score gain on the NGSL test was statistically significant (Z = -3.40, p < .01), whereas that on the NAWL scores was not (Z = -1.72, p = .09).

Table 3　The Vocabulary Profile at the End of the Semester (% of Correctly Answered Items; N = 23)

NGSL NAWL

Band 1 Band 2 Band 3 Band 4 Band 5 TOTAL Band 1 Band 2 TOTAL

Average 98.70 95.65 93.62 85.51 87.83 91.80 61.04 70.96 66.60

Minimum 90.00 86.67 80.00 60.00 66.67 32.00 48.00

Maximum 100.00 100.00 100.00 100.00 100.00 　 　 76.00 80.00 　

Discussion and Conclusions

　　First, it is worth mentioning that the observed score for NGSL vocabulary increased, albeit marginally, to reach an average score of over 90% by the end of the semester. At the time of the pre-test, just 10 students scored over 90%. This number increased to 18 at the time of the post- test, which is over 78% of the students whose scores were analyzed. This increase in the proportion of students who scored very high led to a difference in the medians of the data from the two tests (88.57 and 92.86), which a Wilcoxon signed-rank test confirmed as significant. We can thus assert that the students in this course now have a very good mastery of NGSL and are ready to handle various tasks in English. This improvement was probably partly supported by repeated exposures to the vocabulary items at this level through different materials used for the course. As shown in some past research (Okamoto, 2007), a dramatic drop in vocabulary is sometimes seen among Japanese university students; thus, re-enforcing the knowledge of the words they already know is certainly worthwhile. The selection of the course materials seems successful in this sense.

　　On the other hand, the students did not show any significant gain in academic vocabulary, which is understandable because the materials presented in the course did not ensure exposure to such vocabulary items. The instructor of this class should perhaps fill this gap in the input by either introducing the students to a variety of academic word lists or using introductory academic passages for pronunciation practice.

　　This case study was conducted in one sample class from a university-wide curriculum taught to nearly 2000 students at both higher and lower levels. As vocabulary expansion is one focus of the curriculum, in order to improve the selection of materials, it would be desirable to conduct an investigation similar to that presented in this paper. Even this small-scale case study showed a mismatch between the input and the students’ actual vocabulary profiles. Conducting a similar study on a wider scale might reveal very useful information for the entire curriculum.

References

Akiba, T., & Mori, H. (2012). Dialogue 1800 (3rd ed.). Tokyo, Japan: Obunsha.

Anthony, L. (2014). AntWordProfiler (Version 1.4.1) [Computer Software]. Tokyo, Japan: Waseda University.

Available from http://www.laurenceanthony.net/

Bennett, P., Stoeckel, T., & Ishii, T. (2016). Japanese-English bilingual versions of the New General Service List Test and the New Academic Word List Test. Paper presented at the 20th Japan Language Testing Association Anniversary Conference, Kanagawa, Japan.

Brown, D. (2012). The frequency model of vocabulary learning and Japanese learners. Vocabulary Learning and Instruction, 1 (1), 20‒28.

Browne, C. (2014). The New General Service List version 1.01: Getting better all the time. Korea TESOL Journal, 11 (1), 35‒50.

Browne, C., Culligan, B., & Phillips, J. (2013a). The New General Service List. Retrieved from http://www.

newgeneralservicelist.org.

Browne, C., Culligan, B., & Phillips, J. (2013b). The New Academic Word List. Retrieved from http://www.

newacademicwordlist.org.

Meara, P. (1980). Vocabulary acquisition: A neglected aspect of language learning. Language Teaching and Linguistics: Abstracts, 13 (4), 221‒246.

Nation, I. S. P. (2013). Learning vocabulary in another language. Cambridge, England: Cambridge University Press.

Nation, I. S. P., & Beglar, D. (2007). A vocabulary size test. The Language Teacher, 31 (7), 9‒13.

Nation, P., & Wang, M. (1999). Graded readers and vocabulary. Reading in a Foreign Language, 12, 355‒380.

Okamoto, M. (2007). Lexical attrition in Japanese university students: A case study. JACET Journal, 44, 71‒84.

Stoeckel, T., Ishii, T., & Bennett, P. (in press). A Japanese-English bilingual version of the New General Service List Test. JALT Journal.

Waring, R., & Takaki, M. (2003). At what rate do learners learn and retain new vocabulary from reading a graded reader? Reading in a Foreign Language. 15 (2), 130‒163.

West, M. (1953). A general service list of English words. London: Longmans.

Assessing Students’ Vocabulary Profiles for Better Selection of Course Materials: A Case Study