Recent Reflections on Applying Statistical
Research Methods to Second Language
Acquisition Research
著者
Phillip Rowles
雑誌名
dialogos
号
10
ページ
73-79
発行年
2010-03
URL
http://id.nii.ac.jp/1060/00004965/
Creative Commons : 表示 - 非営利 - 改変禁止
http://creativecommons.org/licenses/by-nc-nd/3.0/deed.ja
73
Recent Reflections on Applying Statistical Research
Methods to Second Language Acquisition Research
Phillip Rowles
The aim of this paper is to address two goats． First、 the goaいs to ref］ect on lny reading and sludyillg of staristical literature related to second language acquis ition（SLA）so as to clarify the complicaled topics that I have considered recentlv． The second goahs to colnmunicate this message to readers． The ン L L retlections illclude my thoughts on statistics． clualltitat▲ve research methods． altematives． and SLA in general．RetfTection O〃ε』Negatiγe Impact ofPo∫tmodernis’η∫o〃SLA Re∫earch
SLA research suftξrs fl’on．）adilemmti in the forIll of the prolifel＾atioll of theories． This glut of’ theoiies is a real obstacle to progress ill SLA resea1’ch（Long、 2007）．In tlle l990’s． postlnoderllisn〕s／poststructuralisms apPeare〔いn SLA and applied lillguistic》1嚇esearch．“As with a王i forms of epistemological rejativism、 claims fbr the existence ofトトcorrecゴ’ways of knowing（eg．， sclemific ways）， or for⑱’correcピ’knowledge systems（e．g．、accepted research findings）， are rega1’ded as spurious「「（Lon g，．20r）7」47）． Accol“dUlg to relativism， theories are above critique， it is only about how they are interpreted by hldividuals， Therefore， there has been a proliferation of theories beyond scrutiny．’」AIld there’s the rub・Contrary to what the postmodernistg． claim、 colnparing and evaluahllg theories is not all optiollJlot a luxury、 not all ivory low’er activity．110t a game， 110t impossible． but‘・∬e〃t～（11 for progre∬in sciellcピ｛Lo11g，2007，150）． This ‘L≠獅凾狽?奄獅X90es句「attitude espoused by postmodernists has had a llegative e廿ect74
Philhp Rowles
oll SLA reseal℃h． This IE．tissez－faire appr〔｝ach to unqueslioned acceptance of allything as a Iegitin〕atc theory is pl℃ving to be a large problelll il］our neld． L℃［aims that the（）1’y prolife1’tltiol）is of no collcelll because the・i’e is lw）objectjve Ileahty． no facts of thc IuatteI’、 thal peoPle al’e their ov）「11 tlleorists、 that all knowledge is socitili＞「collstructed and llo1］LJellel’alizab］e， are quickly forg（）tlen i川he face of a rapidly approaching Mack tl’uck． Every filne relativists cross a sn’ect． rhey Inake use of the saIne l’apid mental calculations based oll distance and the ve］ocity of－ollcolning tra†’t’ic as everyolle else‥（Long．2007、156）． It is time that theor｛es took the teSt of scrし｝lillY． Do they work？If so． they are accepted、 If nOt、 they are rejected． Learlling a fOreign lallgUage inVOIVeS SCienCe、 not shamani／ m．ReL17ection Two’Change∫
Lazaraton’s（2〔｝05）chapter oll quantitative research methods indicatedthat a fundamental change in outlook is underway ill applied linguistic
research． away fronl all eniphasis in the 1980s on quasi－experimental studies ush19 parametric statistics．to a wider t’ocus in the 199（）t to the present ill multidisciplinary research methodology． Thus． changes are underway． with redefinitions of research goals， methods， and worldviews、 acceptance of qua】itative methods． questiolling of research sigllificance、 and exploration of reSearCh altematiVeS， 111this context． Tabachnick and Fidel］’s（2007）Using Multivariate Statistics4 is very conlprehensive． Not only does it cover the typical parametric 9． tatistic！、but it also covers statistical techniques that are se｜dom used in our applied lhlguistlcs field、1ike Time－Series Analysis． Tabachnick and Fide1Fs （2007）book is also extrelmely hetpful as it opened nly eyes to poteIlt▲al optiOIIS that are oftell llot used in our field． Possibly mally people feel that some of theRecellt Reflecnons oll Applyillg Stalistica｜Rescarcll Methods to Secolld L〔mguagc Ac（luisitkoii Research
75
more obscure oPtiOIIs are out of their league、 too complicated、 or they may lleve｜’ have heard ot’them bcfore． Original l’esearch、 which could involve rarely－used stati： tic’al tecllniques、 is olle of the goalsto exPand the SLA research fiel〔i．Rqf7ection Three：Doctoral Coarse Statistical CoJ，erage
Lazaratol1、 Riggenbach and Ediger5s日987． c“ed in Lazaraton、20｛〕5｝ stlldy examined 121 aPphed｜il19uistics professiollals wllo comPleted sul’veys on g． tatistics and empirical research altitudes． Most pal’ticipants had taken only one or two courses ill research desigll and statislics、 which they feh was inadequafe for their job purPoses． Parτicipants ability to llse alld interPret 23 statistical terms（for example、 mean o1’mediam三md statistical procedures（for examPle cor1’elation or r－tlest）were recorded on a 4－point Likert scale， The results ind｛cated t王lat pa1’tlcipants responses showed a mean of｜e∬than 2 for terminology hke Scheffe tesL Rasch mode1， and power，When I compare Lazaraton、 Riggellbach and Ediger「s（1987）study
participallts tally of quantitative courses lo the TUJ Ed．D prog］’am， I was vel’y lucky at Temple University Japal1（TUJ）． Conlpared to the professionals olle or two courses h川’esearcll design and statis｛ics， ill the TUJ Ed．D． program l had 4courses ill statistics． Tlユe only problem at TUJ is that l want nlore sUltistics cla∬es because I am ex．tremely imerested ill this line of stしldy． Having Dr． Jim Sick in class to in〔roduce the Rasch family of Models to most of the class was greaL Evell though r had taken Dr， Trevor Bond「s TUJ Distinguished Lecture｜’ Seminal’on Rasch Model AI．1alysis at the end of 2006． Dr， Sick’s presentation Was very lnterest】ng．76
Ph川ip Rowles
Reflection Four：euantitatiWe Research＿What，∫Gone」B¢fore 17 An．y
Gaps2
しazaratol1（2005）examined’】regular…articles fl”Olll four apPhed linguistics ＿i O tl I．1．｝ a l ＿S ＿ L（ノノ1／c’～’ζ～．9）〈） L 〈）〈”．ノ～ i’～9 、 M（ノ（∫〔ノ1，i～ 乙〔」 ノ～＿9， ～↓（ノ9〈） ＿1（ ノ～rノーノ？（ ’／ ， ＿∫～～’（～～〈ノ＿S』 ～1～ 5．‘）〔 ．〔）’～（／ Lω～9～‘α9ピAc9～’～、～’itio〃．‘〃～（／τ芭∫OLρ〃〈〃’te〃ぎ． represenhng an I l－year period from 199｜to 200 L From the 524 empirical articles allalyzed．45（．）（86（差）were quallntaUve，67（13921were qualitative． and 7（1．’のwere a m▲xture of botl1． As pointed out by Lazaraton（20〔〕5．1． these tligures are nlisleading as except for TE∫OL（？1．t（〃’tel’～V、 the other three journals had over 86 9，． clu［111titative rePresentaliol1． More specifically、 qualltitative research represelltation was： La〃，g・1．｛a，ge Le〈〃・1～～ノ7、g（96〔7e）． Mθ‘／〈ソ・ノ～L（〃1，g tta，9． （・ノθ〃〃～（〃（87プ／／・．）、 and．E；tiidie，s 〃1∫e（’Oiid LCtns，ucl，gc’A（’qui．s’itiθfl（95ワ乞）． In contrast． TESOLρ〃‘〃・te〃｝・had quantitative research represelltation of ollly about 60％．． Lazaraton（2005）also ex ．1 mi rl ed statistical procedures used ill the four applied lillguistics joumals． The 1110st coll］TTIOn procedures were called ’‘ р?唐モ窒奄垂狽奄魔?fstatistics． represellted by frequencies、 pcrcelltages． n〕ealls， alld standard deviations． Sul’prisillgiy，76C／r，（L（〃’9”α8ピLea〃～～η、9）to 90％．（TE∫OL Qu（〃’tei’～y）oF the articles ill all four jounlals showed descriptive s【afistics． including sQme of the“qualitative”studies． Analysis《）f Variance（ANOVA）represented over 40 9ic of tlle statistical analyses in the four joumals． More precisely、 ANOVA was used ill aln⊥ost 509f． of the articles in Lc〃iSJU（79（・Le〈〃◆ノ1 il～ g）and in over 50％of∫tu〈「～‘・∫ii’1 Se（’oη‘～ L〈〃7g～’〈7gρA（・（luisiti〔〃～． Despite this high use of’ANOVAs、 ANOVAs have strict assunlptions which IIIay nol have always beell met， thus、 it w’as e9－ tinlated that al least some of the published studie，．；violated some of the a∬uMPtiollSL． Mally of the ANOVA studies had large sample sizes， however， olle ANOVA study il1 ∫rttd～es i」～∫e（．’‘〃～‘～L‘〃1：．71t‘∫9εAピ〔／1．｛isiti（／〃～／～‘7（1θ〃！．、’ノ2 participallts， Similarly．Recent Reflect▲ons ol．1 Applying Statistical Research Methods to Second Language Acquisjtion Rescarch
77
one factor analytic study illルfo‘／e’・1～L（J〃，qltage．1〈π〃・ノ～‘ll h‘ノ〈～力’ぱv 4／participallts （Lazaraton、2005）． Pearson correlation was the next most commoll statisr▲cal procedure with 28％．usage in the statistical analvses． The next most comlllon was t－tests with ～ il usage of 23％．、 regression analysis with l3％． usこlge， and ChLsquare with l I c／cusage、 Those statistical procedures with under lO％usage were MANOVA．
factol－analysis、 and ANCOVA． Thus， as pointed out by Lazaraton（2005）， if applied lingu▲sts have｛o iearn to use and interpl’et only Qne statistical procedu1’e， it should be ANOVA、 otherwise they will not be able to read and assess tlle results in about hali’of the empirical studies ill these four journals． On the other side of the coin、 the advanced multivariate statistical procedures I studied ill Tabaclmick and Fidell （2007）were a good opportunily me to break away from the pack andしlse fresh quantitative procedures that are rarely used in our field．Reflection F加e： The l〃zportance of Checking A∬1〃nptions
Illitially checking assumptions before apPlyillg a statistical procedure is one of the points I have focused on， and this is something backed up by the literature．“While the findings suggest thar parametric statistical procedures still‘rei即supreme．’Iwould llope that more care would be takell in applyillg α〃statistical procedures apPI’opriately as per their underlying assumptions” Lazaraton（2005．219）．’‘And this can only occur if researcllers alld graduate students seek out educat｛onal opPortunities for training ill researcll design and statistics”Lazaraton（2005，219）． Seekillg out advanced statistical education is indeed what I have been endeavoring to accomplisll imny readillgs、78
Phillil）Rowles ReLf7ection Six．’The Next Frontier is the Rasck Fa〃tily of Models －Perhaps lhe ncxt frollnel’in applicd lingui s． tics I’esearch should be developmg alterllatives lo parametric statistics for snコall－scale reseal’ch studies that involve limited amounts of dependent data「’Lazararon（2005．219）、 A way of following these suggestiolls is Rasch Model analysis， The Rasch Model meets Lazaraton’s（20（：15）suggestions． as it does Ilot assume a norinally－di．stributed da亡a set． alld ct｛ll be carr｛ed out on relanvely snlalldata sets． RefTectiO’1∫｛アψ〈～〃．’The“Sig〃力fica〃ce，，（りf Effect sizes a’ld Confide〃ce 1η’ervals Lazaraton（2005）reported that L〔〃’9～tct，g，cvム〈・‘〃’ノ～～〃9 journal as of Septen］ber， 2000．began requiring authors to include measures of eft’ect sizes． ar least for the major statistics repoited． LLI applaud this effort to steer writers（and readers） away from putting too much stock in sigllificallt IJ values、 which are highly dependent oll sample sizes「’Lazaratol1（2005．2．12）． This is complemelltary to the views ot’ Kline（2005）expressed in 8∩・（）〃‘／∫「gη1／『c〈〃～（・〔・7ヒ・stinsg， Klille（2005） puts fol’ward the view that there are limitations to statistical tests and suggests aiternative data analysis methodology like effect size and confidence interval estilnation． Null hypothesis． sigllificance Ies白ng（NHST）is still very dominant in our field， but we also need to be open to alte1’natives f（）r the field to stay healthy alld aljve， Reflec’ion Eight： Significan’輌〃What・Sense 2 The use of the word bヤsignifical1ピ亀can be collfusing（Newton＆Rudestam． 1999）．There is also a difference between stc～tist～c‘〃signific田1ce and sLtわ∫r〔itl〃v〈・ ＞ significance．Therefore、 a fhlding may h乏we statitstical siglliflcance、 but may not be a substantively importallt fillding、 It could be argued that’bst飢istical significance is a necessarv but llot a suf’t’icient condiUoll for stibstantive 」 ンRecellt Reflections on Applying Stahstica｝Research Methods loSecolld Language Acquisition Resca！’ch
