Kobe Shoin Women’s University Repository
Title
Testing the Spoken Language
Author(s)
Alan E Jackson
Citation
Shoin Literary Review，No.20：15-37
Issue Date
1986
Resource Type
Bulletin Paper / 紀要論文
Resource Version
URL
Right
Testing
the Spoken
Language
AlanE.Jackson
1.lntroduction Inmanycountriesintheworld,andnotleastinJapan,theEnglish Languageteacherwhoseprimeconcernlieswiththeteachingofthe spokenlanguagefacesaproblem.Namely,thattheexternalexamina-tions,whichinmanycasesformanimportantpartoftheframework withinwhichheoperates,failtoreflectinsufficientmeasurethein- creasingimportanceofthespokenwordwithinthelanguagecur-riculum.Furthermore,evenwhereatestincludesasubstantialoral component,theitemtypesselectedforinclusionareofteninadequate tosatisfyteacherswhomoreandmoreconceiveoftheirjobincom一 サ コ municativeterms. Whythisshouldbesohasoftenbeendiscussedintheliterature. Morrow,forexample,paintstothefactthatwhereasthestudyof literature,thewrittenword,hasprestige,theabilitytouselanguage foreverydaycommunication,especiallyinspeaking,islessacceptable asproofofacademicorinteUectualworth∫1)Harrisonpointstothecon-siderablepracticalproblemsinvolvedintheadministrationoforaltests andparticularlytothedifficultyinachievingacceptablelevelsof reliability.(2)Fabian,ontheotherhand,seemsconvincedthattheprac-(1}K.Morrow,"TestingSpokenLanguage,°inJ.B.Heaton(ed),LanguageTesting, (Oxford:ModernEnglishPublications,1982)p.56. (2}A.Harrison,"CommunicativeTesting:JamTomorrow?"inA.HughesandD. Porter(eds),CurrentDevelopmentsinLanguageTesting,(Oxford:Academic Press,1983),p.84. -15一ticalproblemscanbeovercomeandseesthedearthofgoodoraltests asduetoanexcessofconservatismonthepartofexaminationboards.(3) Whateverthetruthofthematter,thecontentionhereisthat,given thegrowinginterestintheteachingandlearningofthespoken language,andtheimportanceoftestinginthelanguagelearningpro-gramme,thereisastrongcasetobemadefortheinvestmentof resourcesinthedevelopmentofaseriesoforaltestsappropriatetothe teachingofgeneralEnglishwithinacommunicativeframeworkand responsivetothespecialneedsofstudentsinJapan.Suchaseriesof testswouldprovidestudentswithafocusfortheirstudies,a recognizeablemilestoneontheirwaytowardsfluencyinEnglish,and, ifnecessary,formalcertificationoftheirincreasingability. Amodel,ormoreaccurately,anumberofmodelsforanap- propriatetestalreadyexist.Inabriefsurveyofwhataregenerallycall-ed"communicativeperformance'tests(CPtests),CarrollandHalllist thefollowing,allofwhichhaveasubstantialoralcomponentandare aimedatstudentsofgeneralEnglish:{4) *TheAssociationofRecognisedEnglishLanguageSchoolsQral Test. *TheRoyalSocietyofArtsCommunicativeUseofEnglishEx-amination. *ThePergamon(Oxford)EnglishTests. Whilethesetestsmaynotthemselvesbeentirelysuitableinthe Japanesecontext,thetheoryoftestingonwhichtheyareallbased,the commongroundtheyallshare,maywellprovideanadequatebasisfor (3)P.A.Fabian,"Examinations‐WhyTolerateTheirPaternalism,°inHeaton,pp. 25-26. (4)B.J.CarrollandP.J.Hall,MakeYorcrOwnLanguageTests,(Oxford:Pergamon Press,1985),pp.162‐165. -16一
aseriesoftestsspecifictotheneedsofgeneralstudentsinJapan. Asafirststeptowardsdecidingwhetherthisisinfactso,thenext sectionofthispaperattemptsasketchofwhatCPtestsseekto achieve. 2.AnApproachtoTesting Thetestslistedabove,thoughemployingsomewhatdifferentitem typesandadministrationprocedures,areconstructedinaccordance withanapproachtotestingwhich,withsomesimplification,canbe seenascomprisingthefollowingelements: a)Communication Itwasonceassumedthatalearner'slanguageproficiencycould bestbeestimatedbyadministeringagrammarbasedtestwhich,as withgrammarbasedapproachestoteaching,focussedonthelearner's abilitytoformgrammaticalsentencesbymakinguseofknowledgeof phonology,lexicalitems,rulesofmorphology,rulesofsyntax,and sentencegrammarsemantics.Whilefewwoulddisputethatlanguage testsshouldassess,inpartatleast,possessionofsuchknowledge,pro-ponentsofabroadercommunicativeapproachtotestingwouldargue withMorrowthatknowledgeoflanguageasasymbolicsystemshould notbeseenasanendinitselfbutasameanstoanend‐theabilityto uselanguageforeffectivecommunicationintherealworld.(5) Trim,indiscussingthetypesofknowledgeofinteresttothecom-municativetester,givesthefollowingdescriptionoftheresources broughtintoplayduringaninteraction: "Forcommunicationtobeeffective ,bothpartiesdrawontheir understandingofthelanguageconcernedasasymbolicsystem (5)K.Morrow,°SomeCommentson'Issues',"inHughesandPorter,p.115. -17一
ii (linguisticcompetence)andtheconventionsgoverningitsuse(com- municativecompetence)aswellasoftheirunderstandingoftheextra- linguisticparametersofthesituationinwhichthemessageisembedd-edandtheprinciplesofsocialinteraction(socialcompetence)."⑥ Inaccordancewiththisdescription,communicativetestsattempt totakesystematicaccountofallaspectsofalanguageuserscom-petence.Inotherwords,theyrequirethattesteesdemonstratebothan abilitytorecognizesignificantaspectsofthecontextoflanguageuse, andanappreciationofhowthephonological,lexical,syntacticand semanticsystemsofthelanguagecanbemadeuseoftotransfermean-inginthatcontext. b)∫ セ吻 物nce Testswhichattempttotaptheknowledgeunderlyingsuccessful languageuse,competencetests,arenotinthemselvesadequateto guidethedecisionsofconsumersoftestinformationwhoarein-terestedinwhatacandidatecandowithlanguage.Theyalsorequire evidenceofanabilitytomobilizethatcompetenceinperformance.As CanaleandSwainputit: "Teachingmethodolo騨d assessment血strumentsmustbe designedsoastoaddressnotonlycommunicativecompetencebutalso communicativeperformancei.e.theactualdemonstrationofthis knowledgeinrealsecondlanguagesituationsandforauthenticcom-municativepurposes."(7) Realworldperformanceabilitymaybeaffectedbyanumberoffac- (6}J.L.M.Trim,"ThePlaceofTesting,AssessmentandEvaluationinCom-municativeLanguageLearningSystemsDevelopment,"inJ.A.VanEkandJ. L.M.Trim(eds),AcrosstheThreshold,(Oxford:PergamonPress,1984),p.162. (7)M.CanaleandM.Swain,"TheoreticalBasesofCommunicativeApproachesto SecondLanguageTeachingandTesting,"inAppliedLinguistics1.1{1980),p.6. -18一
tors(psychologicalconstraints,volition,motivationandpathology,ac-cordingtoCanaleandSwain(8))and,inasmuchastheseperformance factorsaffectindividualsdifferentially,competencetestsmaynotbe adequatepredictorsofperformanceability.Indeedthereal‐time natureoforalinteractionandtherapidrole‐switchingthat characterisesitclearlymakegreatdemandsonaparticipant'smental andphysicalskills.Totakeaccountofthese,testsshouldassessperfor-mancedirectlya皿dso"avoidrelianceonatheoreticalunderstandhlgof theperformance-competencerelationshipfortheirvahdity."⑨ Thatatestofspeakingabilityshouldseektoelicitsamplesof spokenperformancemayseemsoobviousasnottobeworthstating. Attemptshavebeenmade,however,toconstructpaperandpencil testsofspokenEnglish(Oralclozetests,forexample{10帆11))andinargu・ ingforperformancetestsitisthistypeoftestthatisbeingrejected. c)lntegration WhileitisconceivablethataCPtestcouldtaketheformofa discretepointtesti.e.onecomprisingitemsthatfocusononeandonly oneelementofthelanguagesystematatime,thereareanumberof theoreticalandpracticalreasonswhyanintegratedtest,inwhichthe testeeandinterlocutorareinvolvedinthejointproductionofanextend-eddiscourse,istobepreferred. (8)CanaleandSwain,pp.6-8. (9)J.C.Alderson,"ReportontheDiscussiononCommunicativeLanguage Testing,"inJ.C.AldersonandA.Hughes(eds),ELTDocuments111‐Issuesin LringuageTesting,(London:BritishCouncil,1981),p.56. (10)A.Hughes,"ConversationalClozeasaMeasureofOralAbility,"inEnglish Lσ π卿g召Teaching/ournal35.2(1981). (11)D.Brown,"ConversationalClozeTestsandConversationalAbility,"inEnglish L8π 卿8召Teaching/IJ'37.2(1983), 一・19一
Tobeginwith,theassumptionunderlyingdiscretepointtests,that atestee'sabilitytouselanguagecanbeinferredfromhisabilitytohan-dlediscreteelements,hasbeenquestionedbyCarrollandTrim amongstothers.Theypointoutthatinauthenticcommunicationthe wholeisfargreaterthanthesumoftheparts,involvingasitdoesboth individualskillsandthecomplexrelationshipsbetweenthem.(12x13)In addition,andperhapsofevengreaterconcern,isthefactthatmanyof thepartsorsub‐skillsassessedindiscretepointtests,includinggram-maticality,maynotbecrucialtoeffectivecommunication.Notonlydo languageusersemploycommunicationstrategiestomakebestuseof thelanguageattheirdisposalbutasPickettsays"languageishierar-chicalandcontextualizedsuchthatdefectscanbecompensatedforby clueshigherinthehierarchyorinthecontext."(14) Amongstpracticalreasonsforrejectingthediscretepointapproach istherequirement,notedin2aabove,thatacommunicativetestmust servetoassessatestee'sresponsivenesstothecontextoflanguage use.Ifdiscretepointtestswereused,thequantityofrubricand/or metalanguageusedtoestablishcontextineachitemwouldrenderthe testasawholeimpractical.Integratedtests,ontheotherhand,are relativelyefficientinthisrespect. d)Directness AfourthelementofthetheoreticalbasisofCPtestsisthenotionof directnessi.e.thattesttasksshouldresembleascloselyaspossible reallifelanguagebehaviour.Iftheidealofdirectnessisachieved ,then (12)J.B.Carroll,"ThePsychologyofLanguageTesting,"inA.Davies(ed), LanguageTestingSymposium,(London:OxfordUniversityPress,1968) ,p.56. (13)Trim,p.164. (14)D.G.Pickett,"OralExaminingRevisited,"inCOMLON,(London:London ChamberofCommerceandIndustry,Summer1984),p.5. -20一
atestcanclaimwhatWeircalls"firstordervalidity"inthatthe behaviourelicitedintheexaminationroomissimilartothetypesof behaviourtowardswhichleamingisultimatelydirected.(15}Thisim-mediaterelationshipbetweentestandlearningobjectivesmakestests attractivetobothtesteesandusersoftestinformationalike,anden-suresthattheeffectoftestingonteachingisbenign.AsCarrollsays: "Thedynamicrelationshipbetweenlanguagelearningprog ram- mesandlanguagetestsobligesustoapplywidereducationalconsidera- tionstotheevaluationoftestssuchthattheirnatureisnottobedecid- edentirelybyscientificconsiderations.Testsmustofthemselvesap-pealtothetesteesasbothhumanandworthwhileandwillbethebetter iftheyincorporatethecharacteristicsoflanguageasitisusedfrom daytoday."(is) Thisappealforauthenticityintesttaskswouldseemtoimplythat examinationroomactivitiesshouldbelargelyinteractive(rulingout speech‐makingexceptwheretheabilitytospeakatlengthiscon- sideredanimportantcommunicativeneed),contextualisedandpur-poseful.Inaddition,thetestaemustbeallowedameasureoffreedom tobothinitiateandcontrolthecourseofinteractionssuchthatconver-sationsgeneratedwillbetosomeextentunpredictable. Indirecttests,whichelicitlessauthenticlanguagebehaviourandre- lyfortheirvalidityontherebeingastablerelationshipbetweentheat-tributeactuallymeasuredandtheabilityofultimateconcern,may havetheadvantageofeconomyandmayevenbeshowntoactasgood (15)C.J.Weir,"ReactiontotheMorrowPaper{1},"inAldersonandHughes,p.29. (16)B.J.Carroll,"lssuesintheTestingofLanguageforSpecificPurposes,"in HughesandPorter,p.111. -21一
predictorsoforalperformanceability.Theyareunlikely,however,to encouragetherightsortoflanguagelearningbehaviorandmaywell su丘erfromadeclineovertimeintheirabilitytopredictasleamerscon・ centratetheira#fortsonlearningtopassthetestratherthanlearning thelanguage. e}Criterion‐Referenced TheconcernofCommunicativeLanguageTeachingwiththeI satisfactionofreal-worldcommunicativeneeds(asexpressedin Ib ehaviouralterms)andthedemandsofteachersforteststhatare responsivetoteachingimpliesastrongrelationshipbetweenthethree elementsofthelanguagelearningprogrammei.e.Objectives{derived fromsomespecificationoflearnerneeds},Pedogogy(syllabusand methodology}andEvaluation(testing).Addtothistheconcernfor directness(see2dabove},anditbecomesclearthatdesignersofCP testsmustnotonlypublishthesyllabusspecificationonwhichtheir testisbased(preferablyonetowhichteacherscanagree)butalso endeavourtoconstructatestthatishighincontentvalidityi.e.that containsasampleoftesttaskshighlyrepresentativeofthatsyllabus. Theresultingtestisofnecessitycriterion‐referencedtopredetermin- edbehavioralcriteria(asetofcom血unicativetasksandassociatedper-formancestandards)andconsequently`transparent'toallconcernedin itsadministration.Bothlearnersandteachers,inotherwords,know preciselywhatmustbeaccomplishedinpreparationforthetest,and i usersoftestinformation(oftennotthemselvestestingspecialists)are presentedwitheasilyinterpretabletestresultsintheformofastate-mentexpressedinbehavioraltermsofwhateachcandidatecando. 一一22一
3.SomeProblems Thetheoryoftestingoutlinedaboverepresentsanideal.Inprac-tice,problemsattendonboththeconstructionandadministrationof CPtestscallingintoquestiontheirfeasibility.Themoreseriousof theseproblemsarediscussedbelow. a)Sampling Theconcernofthispaperiswithexternalexaminationsi.e.withpro-ficiencytestsofrelativelylongtermobjectives,andnotwithinformal progresstestsadministeredintheclassroomtoassessmasteryof isolatedunitsofwork.The`behaviourdomain'1beingassessedisconse-quentlylargeandiftestconstructionistoresultinatestthatis economicallypracticable,aschemeforsamplingtesttasksmustbe used.Furthermore,sincepedogogicalconsiderationsdictatetheadop- tionofacriterion‐referencedassessmentprocedurecapableofachiev-ingahighlevelofcontentvalidity,itisimperativethatthesampleof tasksincludedinanyonetestberepresentativeoftheuniverseoftest taskscomprisingthetestsyllabus. Withtraditionaldiscretepointtestsoflinguisticcompetence, representativenessoftestsampleisbothlessimportant‐suchtests relyrelativelylittleoncontentvalidity‐andmoreeasilyguaranteed ‐thedomainofinterestbeingrestrictedtotheformalelementsofthe language.Asaresult,atestbatteryofnomorethanahundreditems andcapableofcompletioninacomparativelyshorttimecanclaimade-quatecoverage. ForCPtests,ontheotherhand,thedomainassessed,evenatbasic levels,ismuchmorecomplex,includingasitdoestheabilitytocon- structdiscoursesetinandresponsivetoavarietyofdefinedsocio-linguisticcontexts.Moreover,itisnotatallclearthatperformanceon -23一
anyonetaskorinanyonecontextcanbegeneralizedtopredictperfor- manceinothersituations.Asaresult,samplerepresentativenesscan-noteasilybeguaranteed.AsBrownsays,"forcomplexbehaviourthat ismorethanjustthesumofsimplebehaviours...alimitednumberof testtasksmaynotbesufficienttofullydescribethecomplexobjec-tives."(17) Againstthis,however,onecanpointtofeaturesofCPtestswhich wor1【infavourofsamplerepresentativeness.Notonlyaretesttasksin- tegratedandcapableofelicitingarelativelylargesampleoftesteeper-formancebuttheskillsunderlyingperformanceonanyoneitemare commontoawiderangeofsuperficiallydissimilarcommunicative tasks.Inaddition,theadoptionofapolicyofpublishingthetestcon-tentspecificationandsystematicallysamplingacrossallpossibletest tasksoveraperiodofyearsislikelytopromotecoverageofthe syllabusbylearnersandproducewhatmightbetermeda`second order'generali2eability‐not`hecandotaskAandisthereforelikely tobecapableoftaskBbecausetheoneisrepresentativeoftheother' but`hecandotaskAandisthereforelikelytobecapableoftaskB becauselearninghasbeendirectedequallytowardsaccomplishmentof both.' Theconclusionseemstobethat,whilethehighlevelofcontent validityideallyrequiredofcriterion‐referencedtestsisprobablynotat-tainable,aleveldeemedadequatebythestandardsofpracticaltesting canbeachieved. (17)S.Brown,WhatdoTheyKnow?AReviewofCriterion‐ReferencedAssessment,i (Edinburgh:Sc°ttishEducati°nDept.,H.M.5.°.,198°),p.37. -24-i
b)肋 〃励'名`吻 Inmakingacasefordirectasagainstindirecttestsin2dabove,it.. waspointedoutthatanidealtesttaskshouldaimtoelicitauthentic languagebehaviouri.e.thejointproductionincontextofapurposive interactionwithaminimumofconstraintplacedonthetestee's freedomtoaffectcourseandoutcome.Butisthispossiblegiventhat testingtakesplaceinanenvironmentfarremovedfromtherealworld? AsLowsays,"itisonlypassibletosimulateormodelthecontextsin whichtesteesoperatelinguisticallyandthetaskstheyarelikelyto havetoperforminthosecontexts."(18)Atbest,then,onlyaseriesof pseudrealisticinteractionscanbegeneratedandforsomestudents eventhismaynotbepossibleduetofactorsnotdirectlyrelatedtooral proficiency.Therequirement,forexample,thatdetailsofrole‐relation- ship,purposeandsettingbeincludedinitemrubricmaymakeex-cessivedemandsonatestee'sreceptiveskills,particularlyatbasic levels.Whethertheseproblemssocompromisetheprincipleofdirect-nessastodetractexcessivelyfromthevalueofperformancetestsisa matterforjudgementandcanprobablyonlybeknownaftersuchtests havebeenadministeredandevaluated. c)BackgroundSkills Itisimportantwhenconstructingteststomakesurethatsuccessful performanceisnotdependentonpossessionofbackground knowledge,includingknowledgeofthetargetculture,beyondwhat canreasonablybeexpectedofalltestees.Indeed,testconstructors routinelymonitoritemsforsuitabilityinthisregardandtheproblemis {18)G.D.Low,"ValidityandtheProblemofDirectLangugeTests,"inJ.C.Alder-son(ed),」LancasterPractical」gapersin.English・ 肱 錐8翅9召Educatiart,Zノ'od.6' Evaluation,(Oxford:PergamonPress,1985),p.159. -25一
notgenerallydifficulttoremedy.Amuchthornierproblem ,however, isthatofbackgroundskill,particularlyforcommunicativeperfor-mansetests,inthatthesuccessfulcompletionofacommunicativetask oftenrequiresthedeploymentofskillsotherthanthosegenerallycon- sideredpartoflanguage.Agooddealofcommunicationisnon- linguisticandthedegreeofsuccessenjoyedbyacandidateonacom-municativeperformancetestmaywelldependonpersonalityand cognitivestyle.Thisraisesthequestionasto"whethercommunicative languagetestsareactuallymeasuringlanguageperformanceassub-sumableunderlanguageinuseorwhethertheyaremeasuring variablesthatmightbesaidtobeextraneous,non-language related."(is)Thequestionisafundamentalonewhichconcernsnoton-lytheconstructorsoftestsbutalsothoseresponsibleforthesettingof learningobjectives.Ifthepropergoaloflanguageteaching/learning isconsideredtobethepossessionofasetoflinguistic,communicative andsocialcompetencies,thenonemustviewtestswhichpenalizeacan-didateforshyness,lackofassertivenessoraninabilitytodealwith,for example,visualmeansofpresentinginformationasunreasonable.If , ontheotherhand,theobjectiveisseentobethepossessionofa real-worldperformanceability,thenitbecomeshardnottoacceptthe viewputforwardbyFabianthatinvadingextra‐linguisticterritoryis notonlyinevitablebutdesirable.(20) Acceptingthislatterpointofview,onecanseethatmanyofAlder-son's`extraneous,non-languagerelatedvariables'areinfactcrucialto performanceandshouldbeassessedassuch.Onebackgroundskill, (19)Alderson,pp.61‐62. (2d)P.A.Fabian,"ARELSOrals:TheModernLinguist'sDrivingLicence,°inR. R.Jordan(ed),CaseStudiesinELT,(Landon:Collins,1983),p.73. -26一
however,theabilitytosuspenddisbeliefandengageinroleplay,may stillcauseproblems.Thoughnoempiricalevidenceisavailabletosup-partthisassertion,itmaybetruethatsometestcandidatesareless adeptthanothersatroleplaying(aninevitabletestingtechnique)such thattheirtestperformancemayfailtoreflectadequatelytheirtrue communicativeability.Whetherandtowhatextentthisistruestands inneedoffutherinvestigation. d)PerformanceAssessment Thelengthysampleofoutputelicitedfromatesteeinaperfor-mancetestmustbeassessedbytheexaminerinaccordancewithan establishedmarkingprotocol.Ideally,thisassessmentshouldbeobjec-tivebutinasmuchaslanguageperformanceishighlycomplexand, moreover,unpredictable,itisclearthatnocosteffectiveobjective markingschemecouldpossiblydescribeitfully.Rathersomefromof subjectiveassessmentmustbeemployed. Onemethod,usingasingleassessortoawardmarksbasedonan overallimpressionofthetestee"sspeech,hasthevirtueofsimplicity butseemsunlikelytoachieveacceptablelevelsofreliability(precision andconsistencyofmarking).Greaterrefinementiscalledforandanim-provedproceduremightincludeoneormoreofthefollowingelements: *Useofmultiplemarkers. *Assessmentnotoftheoverallperformancebutofimportantper-formanceattributese.g.accuracy,appropriacyetc. *Useofperformancescaleswhichdistinguishanumberoflevels ofperformanceoneachattribute. *Useofstandardizationmeetingsandassessormonito血gtoen-sureconsistencybetweenmarkers. *Useofanexaminerwhoawardsmarkswhileobservingthe -27一
testeeinconversationwithaninterlocutor. Whatevertheprecisedetailsoftheprocedureeventuallyadopted, andtherearemanypossibilities(seeParkinson〔21),Pickett〔221,welr(23) andCarro皿`24)),thetrade-offisclear.Acceptablelevelsofreliability canonlybehadatapriceand,giventhatoraltestingisalreadyanpx-pensivebusiness,itseemslikelythatperformancetestersmustsettle foralowerlevelofreliabilitythanhithertoconsideredacceptableby measurementspecialists. e)y吻 」伽 掘0π Assumingthatsatisfactorycompromisesolutionscanbefoundto theaboveproblems,thereremainsthequestionofhowtoestablish thatthetestisvalidi.e.thatittestswhatitpurportstotest . Whileavarietyofvalidationproceduresareavailable,eachserving todefineandestimateadifferentaspectofatest'svalidity,whichis notitselfdirectlyobservable,onlystatisticalproceduresinvolvingcor- relationoftestresultswithsomeexternalreferentarereallyconvinc-ing.Unfortunately,avalidandreliableexternalreferentagainstwhich tovalidatecommunicativeperformancetestsisnoteasilyfound.Cor- relationoftestresultswiththeresultsofsomeexisting,noncom-municativetestis,asMorrowsays,oflittlevaluesincesuchtestsdo notconcernthemselvesdirectlywithoralperformanceabilityandmay (21)B.Parkinson,"AssessmentofPupilAchievementintheIndependentEvalua-tionoftheTourdeFrance,"inHughesandPorter ,p.94. (22)Pickett,p.6. (23)Weir,p.35. (24)B.J.Carroll,°LanguageTesting:IsThereAnotherWay?°inHeaton,p.7. (25)K.Morrow,"CommunicativeLanguageTesting:RevolutionorEvolution?"in C.J.BrumfitandK.Johnson(eds),TheCommunicativeApproachtoLanguage Teaching,(London:OxfordUniversityPress,1979),p.150. -28一
notthereforebevalidfromacommunicativepointofview.〔25) Analternativeapproach,correlationoftestresultswithsubjective assessmentsofspeakingabilitymadebyteachers,seemsmorepromis-ingbutevenherethereisconsiderabledoubtastowhetherteachers, untrainedinassessmenttechniques,canmakevalidandreliable judgements. Itmayofcoursebethatcommunicativeperformancetestswilleven-tuallysucceedi皿establishingtheirownexternalvalidity,asothertest typeshavedoneinthepast,butforthetimebeingtheymustrelyfor theirjustificationoninternalnon-statisticalprocedureswhichattempt tomeasuretheextenttowhichtesttasksarerelevanttoandrepresen-tativeoftestobjectives. 4.Conclusion Intheintroductiontothispaperthequestionwasposedasto whethercommunicativeperformancetestsmightrepresentanaccep-tablesolutiontotheproblemsoforaltesting,and,byextension,some oftheproblemsoforalEnglishteaching,inJapan.Asyetnoconfident answercanbegiven.InasmuchasCPtestsmakeaseriousattemptto testsystematicallywhatshouldbetestedi.e.theabilitytouselanguage formeaningfulcommunication,andnotmerelywhatcanbetestedeasi-ly,theyseemtoofferawayofraisingthestatusoforaltestingtothe samelevelofthoroughexaminingenjoyedbytheotherskills.They alsoaddressdirectlythedemandsofteachers(andlearners)fortests thatseemrelevanttoandhaveapositiveeffectontheteaching/learn-ingprocess,therebycontributingtoastronggoalorientedlanguage learningprogramme.Theyinevitably,however,fallsomewhatshortof theobjectivestheysetthemselvesand,whileitisearlydaysandmuch canbeexpectedoftheworkcurrentlybeingdonetosolveproblemsof -29一
testconstruction,administrationandvalidation,itseemsunlikelythat theidealofatrulyvalid,reliableandcosteffectivedirecttestoforal performancecanbeachieved.Attheveryleast,adegreeofreliability willhavetobesacrificed. Still,thereisnothingverysurprisinginthis.Notestiseverperfect (performancetestswerethemselvesfirstconstructedinanattemptto rectifytheproblemsofearliertests)andcommunicativeperformance tests,withalltheirfaults,mayverywellrepresentanimprovement overexistingoralassessmentprocedures.Howoneevaluatesthem dependsonwhereonestands.Thosewhodemandthattestsjustify themselvesstatisticallymayhavetowaitsometimebeforesatisfactory resultscanbeachieved.Thoseintheclassroom,ontheotherhand,will surelybemorepositive,andifoneaccordstheirconcernsagreater priority,itishardtoavoidconcludingthatthereismuchtorecommend thecontractionandtrialadministrationofacommunicativetestoforal performanceinJapan. ・4ppendixl・4nExa吻leCo〃amu漁ativeTestofOral」Peformance. Thefollowingcommunicativetestoforalperformancewascon-struttedbythiswriterforusewithyoungadultlearnersofgeneral EnglishinJapanatapproximatelytheleveloftheCouncilofEurope's `Threshold .'itspresentationhereisintendedtoserveasanillustration ofaCPtest. 1)SamplingMatrix Amatrixwasusedtoguideitemselectionandconstruction.Slots werechosenfromeachofthehorizontalrowsandavariedselectionof theverticalcolumnssoastocoverapproximately10%ofthematrix. -30一
11'eather 寸ggq -N寸 ForeignRestaurallt LanguageM卜,一 寸 卜 目N寸ur}o PlacesWorkplace NOOPつo F司Nマu自)ゆ SereiceslIIIIflBank 一 の ゆNLO -Ncつ ゆo FoodandpIIIIIIITheatre/pIIIISocializing Drinko寸oo目Cinema寸 くρ 9一くNM1」 の ⑩ 卜 ShoppiugSchool oつ トoM ①NM山 巾 ⑩ EducationClinic/Asking Nqコ ①HealthNPermissionゆ ooNm寸Centre￠ 卜 HealthandTravelRequesting WelfareHnooBureau一 寸 卜NCり 寸 Φ 卜噛 可r→M寸 山∩ 卜9 1derltificaしior1頃 σ}uつq⊃ 一 一N乱 Ω Y ① 口
継 蕪
ξ1
=050xヨ の 曽qov ・口pox 』o . 邸 』 口 .≡ 芝 呂 詰 ≡ _旬 ①CO'NΣ ⇔ 碧 ≡ ヨ:亜 署COCO 雌 額uNy._AQuU塁 一31一ll 2)TestItems Thetestconsistedofthefollowingfouritems.(ll=inmothertongue onactualtestpaper) 1.flYoumusttalkwiththeexaminerforfiveorsixminutes.He/she willaskyousomequestionsaboutyourself,youreducationand whereyoulive,Du血gtheconversationtheexaminerwillask youtotalkatlength(aboutoneminute)onsomepointthat arises. 皿.ImagineyouareamemberoftheEnglishnewspaperclubat yourschoolorcollege.TheeditorhasaskedyoutointerviewaI newEnglishteacher(playedbytheexaminer)inpreparationfor ashortarticleabouthim/hertobeincludedinthenextissueof thenewspaper.Beforeconductingtheinterview,youlookata similararticle(reproducedbelow)whichappearedinthe newspapersomemonthsbeforeanddecidetobaseyourques- tionsonitscontents.Readthearticleandtheninterviewtheex-aminer. Introducing...PhilipClarke Asyoumayknow,anewteacherhasrecentlyjoinedthestaffhereat theStanfordSchoolandwetakethisoppoi加nitytowelcomehimto Japanandintroducehimtoourreaders. PhilipClarkeisathirty‐twoyearoldEnglishmanwhocomesfrom Southport,aseasideresortintheNorth‐WestofEngland.Thoughheis newtoJapan,withovertenyears'experienceasateacherbothinBritain andabroad,hecertainlyisn'tnewtoEnglishteaching. Aftergraduatingfromuniversity{B.A.History‐Birmingham University),hespenttwoyearsworkinginanelementaryschoolin NigeriabeforereturningtoBritaintotakeupateachingpostatalarge -32一
privatelanguageschoolinLondon.Hewentabroadagainin1981and spentfourveryhappyyearswiththeBritishCouncilinHongKong.
1騨
Hegotmarriedin1980(hiswife,Andrea,isalsoanEnglishteacher) andtheyhavetwosmallchildren,Robert3andLisagmonths. Amongsthishobbiesandinterests,heliststravelling,playingthe guitarandcollectingpottery.Healsoenjoysagameofte皿isbutsayshe is'moreenthusiasticthanskilful'whenitcomestosport. WhileinJapan,hehopestotravelwidelyandisespeciallylookingfor-wardtoseeingKyotoandNara.HealsowantstovisitMashikoandIinari toseethepotteriesandperhapsaddtohisgrowingcollection. Asforthefuture?Well,hedoesn'thaveanyclearplansasyet.Hehas onlyjustarrivedinJapanandiskeentoenjoytheJapaneseexperienceto thefull. 皿.Youhaveenrolledintheprivatelessonprogrammeofa languageschoolinTokyo(3hourstuitionperweek}.Youmust talkwithyourteacher(playedbytheexaminer)aboutyourpre-sentEnglishabilitiesandyourreasonsforstudyingsothatyou canjointlydecideonastudyprogramme. 】V.Youwillbeaskedtocontinuethreeofthefollowingfiveconver-sationsinaccordancewiththeinstructionsgiven.Anexampleis providedforyoutopractiseon.Itisnotofcourseassessed. Example: Youwanttobookaroominahotel(singlewithshower)forthreenights, Thereceptionistisplayedbytheexaminer. Receptionist:Goodafternoon.CanIhelpyou? You:Yes.1'dlikeasingleroom,please. : : : : -33一杷
i(a)
Youseeyourboss(playedbytheexaminer)andaskpermissiontotake
nextTuesdayoff.Giveareason.
Yourboss:Comein!
You:GoodmorningMr.Palmer.Iwantedtoseeyouabout
something.Aboutnextweekactually.
Yourboss:Yes?
:
:
:
{b) Inthestreet.Yougivedirectionstoastranger(playedbytheexaminer) whowantstogototheBritishConsulate.YouarestandingoutsideKo-jimachistation. 一」L_」
Stranger:Excuseme.DoyouspeakEnglish? You:Yes.CanIhelpyou? Stranger:Please.1'mtryingtofindtheBritishConsulate .You don'tknowwhereitis,doyou? 1 : : : -34一{c) Yousuggestsomeplacesforyourfriend(playedbytheexaminer)totake hisrelatives. Friend:MybrotherandhisfamilyarearrivingfromEnglandnext Tuesday. You:Foraholiday? Friend:Yes.Justtendays.1'llbetakingsometimeoffworkto showthemaround.1'mnotsurewhattodowiththem, though.Anyideas? : 二 : : (d) InthetravelinformationofficeatEustonStationinLondon.Youwantto traveltoLancasteronWednesdaynextweek,returningonSaturday. Findouttraintimesandticketprices. Clerk:Goodmorning. You:Oh,goodmorning.Couldyougivemesomeinformation abouttrainstoLancaster,please? Clerk:Lancaster? : : : -35一
ず (e) YouareinabookshopinLondonlookingforthisbook; Title:AguidetotheLakeDistrict. Author:JamesStevens. Publisher:Methuen. Youcan'tfinditontheshelvessoyouasktheassistanttoorderitfor you. You:Excuseme? Assistant:Yes.CanIhelpyou? : : : : 3)Pe伽nce/assessment Markswereawardedbytheexaminerinaccordancewiththefollow-ingsetofperformancedescriptors.Band5{3marks)wasconsidered minimallysatisfactory. -36-一 一
