kusanagi et al 2015 最近の更新履歴草薙邦広のページ kusanagi et al 2015

(1)

NII-Electronic Library Service

Foreign

Language

Grammatical

CarefulnessScale:

Scale

Development

and

Its

Initial

Validation

Kunihiro

KUSANAGI

Junya

FUKUTA

Graduate

SbhooL

AidgqyaUitiversity

JopanSbcietyfortheP)"omotion

ofStrience

Yusakti

KAWAGUCHI

Yu

TAMURA

Aki

GOTO

AI(ari

KURITA

Daisuke

MUROTA

GraduateSbhooL

Migaya

Uhiver:sity

Abstract

Thisstudy aimed to

develop

and validate a scale tomeasure the GrarnmaticalCarefulness

(GC)

of fbreignlanguagelearners.GC, by its

definition,

refersto

psychological,

behavioral,and meta-cognitive traitsof a

learner,

and itentails highlycontrolled, cautious, analytical, and

time-consuming

laiiguage

use. By conducting a sct of

questionnairesurveys

targeting _Japanese

jum'or

highschool, highschool, and university students

_(N

=

2,288),a ForeignLaiiguage GramrnaticalCarefulnessScale

(FLGCS)

with 14items,written inJapanese,was developedand

tested

for

itsfactorialstmcture, reliability,convergent, content, and criterion validity. Theresults

demonstratedthatFLGCS

yields

threefactors:

_(a)

phonological,

(b)

lexical-syntactic,and

(c)

pragmaticcarefulness, with ahighreliability fbreach. The factorialvaliditywas also supported

by

using bothexploratory and confirmatory factoranalyses. Further,a set of analyses confirmed

various typesofvalidity. [[heevidence fbrthe validity isas foliows:

_(a)

the1inguisticexperts

(n

=

10)consistently

judged

thatalltheiterns

properlyreferred to each factorinan appropriate linguisticsense,

(b)

FLGCS showed correlations with learner

beliefs,

consisteni with

theoretical

expectations, and

(c)

FLGCS correlated tothescores of a

C-test,

and with

the

time tofinishthe C-test.Theapplical)ility ofFLGCS

in

EFL teachngand research wi11also bediscussed,

1.Background

largeUndoubtedly,

grammaticalperfbrrnanceof a secondlforeigri

language

shows a relatively

variance among learnersincomparison with thatof theirfirstlanguage.Researchersin

(2)

Japan Society of English Language Education

JapanSociety ofEnglish Language Education

secondrforeign

language

Iearning

and teachinghavethusattempted fora longtimetodiscover

which

factors

explain this

large

variance in

grammaticalperforrnanceamong individnals.Second languageacqpisition

(SLA)

theories,fbrinstance,haveoffered various frameworksrelating tothe

developmentof

grammatical perfbrmance

(e.g.,

Segalowitz& Segalowitz,1993).Task-related factorsserve as another importantcontributor to the variance

(e.g,,

Tarone,1985).Also,

behavioraland

psychologicaltraitswithin individuals,such as aptitudes, attitudes, motivation,

beliefs,and anxiety are

yetmore importantfactors,It

is

selievident that

such factorsinteractina complex manner to

predictone's grammaticalperfbrmance, and theymay

_jointly

affect the

acquisition ofa second!fbreign lariguage.

Itisa consideral)le challenge tocover allof

these

factors

using

_just

a single framewotk, However,a concept which hasbeencommonly adopted inthefieldofcognitive

psychologyand

psychologicalmeasurement is possiblyone which captures the

inter-learner

variance of

gramrnatical

perfbrmance;

thatis,

Speed-Accuracy

Tradeoff

(SAT),

SAT generallyproposesthat task_perfbrmance,with regard tovarious aspects, shows a very similar

pattern,whereby faster

actions result inloweraccuracy, while slower actions

have

a higheraccuracy

(e.g.,

Dennis&

Evans, 1996;Goldhammer & Kroehne,2014;van derLinden,2007).See_plot

_(a)

in

Figure

1,

which

graphicallyrepresents thisconcept. Taking an example in

the case of

gramrnatical perfbrmance,itcan besaid thatifataskisspeeded up ortimed,or

thetesttakerisinahurry,they

may exhibit reduced accuracy. On theother hand,ifthe

persondoing

the task can take enough

timeto accomplish thetask,helshecan

planand monitor histherlanguageuse deliberately.

'Ihis

basicallyleadstohigheraccuracy, Thistendencyof

SAT

may be common inmany aspects of

languageuse and languageassessment.

Theoretically,thc SAT patterncxhibits functions

as in

_(a)

of Figure 1.Howeyer, itis hypothesizedthat,in

paralleltoski11development,thefunetionsfbrSAT

will also change, as in

(b)

inFigure1.[[hechanges ofthe functionsmay correspond tosome ofthe SLA theories,such as

ski11-acquisition theoryorautomatization

_(e,g,,

Segalowitz

&

Segalowitz,1993).

b98e

(a) {b)

'h,,'N,,,'1,`,,,...

bg8e

(c)

XL"-":::::.-::J

S-L

1 -LLu.--Ls--'--::

it9ge

hL;IL-hlRTittLlllxtLs-s

hsit;x:""----E)..

hxXL-"ei--

s--Speed Speed Speed

FVgurei,Schematic

plotsof theconcepts inSAT. Plot

(a)

shows thebasictradeoff

pattem,Plot

(b)

explains thechanges of thefUnctionscaused by development.Plot

_(c)

shows the

inter-learner

variance ofthe compromise

points.

(3)

Thereisyetanother importantviewpointinthisframework,Irrespectiveof development, irrter-learnervariance stillrernains, Thiscan becaptured

by

considering thecompromise

pointof

SAT functionsof individuals.Assume thatone

personat leastina specific situation tends to

prioritize

accuracy, and another doesnot choose fbraccuracy, butspeed, A

partof such a variance

of compromise

pointsamong individnalscan be determinedby histherpsychologicaland

behaviora1traits.Thismay beresponsible fortherest ofthe variance,asin

_(c)

inFigure1,

Thepresentstudy calls

this

hypothetical

traitGrammaticalCarefulness

(GC),

which we will

viewas a constmct ina

psychometricsense.i The next section will introducetheconcqpts of ([}C

and review some of therelevarrt studies intheliterature.Thisarticle wi11report the

development

and

initial

validation ofa scaletomeasure thisnew constmct, GC,inthelattersections.

2.

Grammatical

Carefulness

in

ForeignLanguage

2.1Definitionof GrammaticalCarefulness

GC inforeignlaiiguagerefet;s toa behavioral,psychological,and rneta-cognitive traitof individualswhich

is

characterized

by

the

fbllowing:

_(a)

it

entails

highly

cautious, carefu1,

deliberate,

intentional,

and analytical

language

use,

fo)

it promotes relatively slow, time-consuming,and cogriitively demandinglanguageuse and leaming,which leadstoa higher

accuracy oflearners with some

grammaticaltasks,and

(c)

itcomplexly linkstoother inter-learner

varial)les, such as aptitudes, attitudes,motiyation,

beliefs,

and anxiety.

The SAT

framewotk

regards GC as amoderator of thecompromise

points,Inotherwerds,

itishypothesizedthatsomeone with a

higher

GC tendstoachieve higheraccuracy attheexpense of speed, and another

personwitha lowerGC tendstoperfbrrnspeedily and lessaccurately. 2.2GrammaticalPerformanceand Inter-LearnerVariablesintheLiterature

A

couple of

previousstudies attempted toreveal therelationships

between

inter-learner

variables and

grammaticalper[Ebrmance.For instance,Krashen

(1978),

inhisearly theoretical

wotk, suggested thatthereare two

types

of second laiiguage

learnersi

monitoFunder-usens and

monitor-over-useng

(See

also Seliger,1980).Kormos

(1999)

extended thisidea,and empirically

investigatedtheeffects of thetwo

different

spealdng stylesof

individuals

_{(aecuracly-centered}

and

fluenay-centerec()

on

their

selfcorrection behaviorsbyobserving Ll-HungarianEnglishlearners'

speech

productionand

questionnaireanswers. Kormos lookedat the interplaysamong the

speaking styles and thefrequencyof selfcorrection behaJvior;theaceuracy-centered

participants

showed higherfrequenciesof selfcorrection behaviorthanthosewith afluency-centeredstyle.

One

other case isa recent classroom-based study condncted by Kartchavaand Ammar

(2014),

which investigatedthe effect of learnerbeliefsahoutcorrective

feedl)ack

on noticing

behaviorsand leamingoutcomes. They reported that some beliefsmediated thefrequenciesof

noticing behaviors,butnot thelearningoutcomes.

(4)

These studies focusedonly on very specific behaviorsand situations of learners'

perfbrmance;Kormos

(1999)

isconcerned with speaking, especially selfcorrection, while

Kartchavaand Ammar

(2014)

are concerned with noticing,fbrinstance.The studies

pickedup

very 1irnitedlearners'traits

(speakmg

styles and beliefsahoutcorrective feedbacki.More critically, some studies

did

not consider themeasurement as constmcts. For instance,Kormos' study only

used five_{questionnaire}itemsfordeterminingthelearnersJspeaking styles,of which reliability and validity remained unclear, Kartchavaand Arnmar'sstudy, on theother hand,reported

the

reliabilityand

factorial

stmctures,

but

ofcourse

further

validation would bedesirable,

Thepresentstudy takesa

broader

view,

wnh some methodological sophistication regarding

therelationships between

grammatical perfbrrrianceand inter-leainervariables. GC isa traitof

individualswhich

directly

linksto

grammaticalperformanceingeneral,unlike beliefsregarding

some specific

behaviors,

GC

also

has

value with regard to

its

relationships

with

other typesof

inter-learnervariables, such as beliefs.lnthelattersection, fbrinitialvalidation ofthe

developed

scale, we will report thatthe scales of GC are actually correlated to a

partof grammatical

perfbrmance,and we show thetheoreticallyplausiblerelationships with certajn

types

ofbeliefs.

2.3Signhicanceof GC inTheoryand Practice

Estal)lishingGC as a

psychologicalconstmct and developingitsreliablemeasures would

be

a

promisingway to shed light

on various fieldsof study in

the

future.

For

instance,

in

psycholinguisticexperiments, itcan

be

considered thatcontrolling and establishing thevariahles

ofSAT-relatedinter-learnervariance such asGC will beboththeoreticallyand methodologically

lmportant,

Inclassroom-based studies,GC can beapplied tomeasure moderators of theoutcome of

students' leaming directly.Also,theimpactson GC of certain teaching metheds or treatments

would be

interesting

research topics.More specificallyin

practice,understating students' traits,

such asGC, can

providemuch informationtoteachersinthecontext ofcurriculum

design,

choice ofteaching materials,and everyday teaching_practice.A reliable,validated,and also easy-to-use

psychologicalscale ishencestrongly desired.

3.ScaleDevelopment

3.1The PreliminarySurvey

inorder to

develop

a

psychologicalscaleto measure GC, we condncted two

questionnaire

surveys. The main

purposeof thefirst

_{tpreliminary)}

survey was the initialitemselection forthe

scale. Intotal,169students intwo

privateuniversities participated,All

of the

participants

were

first

yearstudents who took English classes. Theiracademic majors includedeconomics

and education. The survey was canied out at thebeginningof Apri1,2014,The

participants

answered the

questionnaireduringtheirEnglishclasses.

The

questionnaireconsisted of

<a)

a facesheet,

_(b)

(5)

GC itemswritten inJapanese

(k

=

40),as detailedbelow,and

(c)

anchor scales

(k

=

14,Tanaka & Ellis,2003).Thesurveys were conducted inthestyle ofa Likertscale, fromone toseven

points.

The initialitempool

(k

=

40) was created

by

the authors. By referring to learners'

retrospective

data

about

lar)guage

use intheliterature,theauthors composed theitemsin

consultation with each other. Alltheitemsintheinitialitempoolare availahle online atthefirst

author's website

(see

Appendix),

All the

data

were typed and were verified twice by theauthors. The response of one

participantwas excluded becauseofsome missing values;

the

number ofvalid responses was l68. Beforetheinitialitemselection byfactoranalysis, we excluded 18itemswhich obviously violated

the

normal

distribution,

Since

the

goodnessof

fit

indicesoftheinitialexploratory factoranalysis

(22

itemswere submitted), which extracted threefactors,were unfavorahle, 7itemswhich caused

a misfitwere excluded, using a step-wise exploratory factoranalysis

(SEFA).

Hence,15

items

out of40 were selected forthesecondary study. These15itemscan beseen intheAppendix,

3.2The SecondarySurvey

The secondary

survey was undertaken fromMay toJune,2014,using theselected items

_(k

=

15).Intotal2,288

participants

took

partin

the survey, and 2,098 answers, with no missing values or extraordinary responses, were analyzed, Theparticipantsconsisted ofjunior

high

school

students sampled fromtwo

publicschools

(n

=-216),highschool students

(n

-1,078)fromtwo

normal

publicschools, and university students from11national, public,and privateuniversities

(n

=

804).Almostallof theuniversity students were first

yearstudents and hadvarious academic

majors. Juniorhighand highschool students,on theother hand,were sampled ina well-balanccd way intermsoftheir academic

years.

As inthepreliminarysurvey, the

participants

answered the

questionnaireintheirEnglish

classes. The

questionnaireconsisted of the facesheet and the 15 itemsrelated toGC. The

secondary survey used acomputer-readable

questionnaire,Ihe

data

were automatically

processed

using scanners and computers. Then,theauthors yalidated theresponses twice _byhand, Firstly,descriptivestatistics of all thevalid answers

(n

=

2,098)were calculated. Befbre

condncting factoranalyses, we confirrned thedistributionsof alltheresponses

(k

=

15),ItemNo. 7showed astrongly

biased

distribution,

which may negatively affect

the

factorial

stmcture.

Hence,

theitemwas excluded. Then,we conducted an exploratory

factor

analysis to

determine

the

constmcts ofGC. Thisstudy also

perfbrmedconfirrnatory factoranalyses forthemodel inorder to

confirm itsfactorialvalidity.

The distributionsof theresponses are graphicallyrepresented inthemultiple histogramsin Figure2.Tal)le1summarizes correlation coeMcients and the variance!covariance matrix of the

itemresponses

(k

=

14,excludmg item7).

(6)

JapanSociety ofEnglishLanguage Education

[teml ltem2 1tem3 1tem4 1tem5

)h 7lt )h )h h

.O'g･liMlb.

:g!ma

S!IMk

S:IM]}

SEinlh..,

IS57 1357 1357 1357 1357

Rate Ftare Rate Hate Rate

ltem6 lteme 1tem9 ltemlO

h h )h }.

iElg:n..

Slma

iy'llMln-

lk.!l:[!b..

1357 135T 1357 1357

nste Mte date tete

1temM ltem12 ltem13 ltemd4 ltemrl5

S X )h pt X

SEI[!11)i

[1!lma

S':lua

Ig･!ima

i,-!in#I)

1357 la57 1357 1357 1357

fete Flate Rate Rate Rate

]Fligttre 2.Histogramsrepresenting thedistributionsof theresponses.

Table

1,

CbrrelationCbEz(i7cientsand P'Ziriancel(]ovarianceMatrix

_ofthe

ftemResponses,

ItemNo.1 2 345689101112131415

123･4568910111213l4152.01 .49 ,55 .50 .47 .43 .40 ･op.44 .36 .36 .40 .40 .35 1.062.35 .41 51 .32 .37 ,37 .33 .32 .39 .36 .39 .40 ,37 1,Ol,811,66,59 ,53 55 .43 .63 .63 .29 ,35 .44 .42 ,32 ,991,091.061,93 ,39 .40 .37 .48 .44 .26 ,34 .45 .41 .31 ,97 .731.00 .792.14 .37 .3S･" .48 .29 .36 .39 .37 .30 ,79.74.92.72.701.72.45.57.58.36.30.32.3528.83 .83 .81 .75 .75 .g62.13 .55 ,56 ,36 ,35 .38 ,42 .39 .82.671.07.87.85.991,051.73,71,33.34.40.40.36.82 .641.06 .80 ,93 .991,081221.72 .30 .37 .38 .39 ,31 .81 .95 ,59 .57 ,66 .74 ,82 ,68 .612.45 .63 .54 ,57 ,66 .79 .84 ,69 ,72 .81 .60 .78 .69 .741.502.32 .67 .68 .66 .Sl .87 ,81 ,90 .82 .61 .80 .76 .721211.462.06 .72 .61 .84 ,92 .81 .86 ,80 ,69 ,92 .79 .771.331,561,552,25 .61 .76 ,86 .62 .65 .67 .55 ,86 .72 ,631.591,541,341.402.32

7Vbte.Valueson theleftsiderepresent correlation coeencierrts,right

for

covariance.

Table2

presents

the

descriptive

statistics,the

surnmary of thefhctoranalysis, and the

reliability foreach factor.The factoranalysis extracted three factors.The

goodnessof fitindices

were demonstratedtobenot favorable,

btrt

at an acceptahle level,

x2(52),==285.44,p

< .Ol,TLI

=

,93,RMSEA =

.08,and with a 90 % confidence interval

_(CI)

_[.07,

.08].ItemsNo. 11to15

82

(7)

showed

higher

loadings

fbrFactor1,and theseitemswere allrelated tothecarefulness towards

the

_phonologicalaspects of

grammaticalperfbrmance.ItemsNo,6to1OloadedFactor2heavily,

and theseoorresponded tothelexical-syntacticaspects, Therest of theitems

(from

ItemNo. 1to

5)

showed relevance

fbr

the_pragrnaticaspects.

Hence,

thisstudy narned thefactors

phonolQgieal

carEzti(lnesty,

lexical-syntactic

carefuiness, andprtigmatic carE:tiiiness respectively,

Table2.

Descriptive

Statistics

and theRegults

_ofthe

Ex/ploratoiy jFkectorAnalysis Descriptivestatistics Pattern!Structmematrix

Item

M saSkewness KurtosisFactor 1Factor 2Factor 3Communality

131412151110968421353.573,823.844.083.103.263263.303,963.464,093.553203211.441,501,521,521,561.311.311,311,461.391,531.421.291,46O.24O,15O.18o.oo-O.02O.30O.30O,24O.06022-O.06O.27O.25O.40.O.40-O,53-O,59-O,63-O,78.O.09-O.11-O,16-O,45-O,31-O,65-O.32.O.08-O,28 .87/.80.86/.81.801.80.751.83.70/.75

-.OIL44.OIL49.04L41-.02f,44-.08L40-.05!.49-,10!.58-,11L44.12L49.22f,42 ,70.65.57,67,66

-.03f.44-.Ol!.46-,OIL40,18!.48 .92L86.82/.83.601.6857L65

-.071.59.02f.62,11!.55-,04L50 .74.69.47.44

-,07L44,16L48.04L47-.121.45,06L43-.04/.64-.131,42.09L55.421.75.29L56 .841.77.621.62.591.77.54L77.32L57 .60Al.48.68.37

FactorCorrelationsFactor

2Factor

3

,55,63

,73 Reliahilitya

coesucients

Averagecorrelationcoeencients

.90,64 .84,57 .82.48

Sums ofsquares ofloadings

Proportionofvariance Cumulative

proportionofvariance

3,28 .23 ,23

2.68 .19 .43

2,16

J5

.58

?Vbte.The

factor

analysis was conducted using maximum

1ikelihood

estimation method, and

Promaxrotation,with thenumber of factors,three,as suggested bytlie

parallelanalysis,and we

judged

thatthismodel was alsotheoreticallythemost

plausible.

83

(8)

The reliability of each

factor

was calculated

with

Cronbach'sa coethcients and average

inter-correlationcoecacients, The reliabilitycoecacients of allthe

factors

were sucacient, as can

beseen inTable2.

Themodel was thensubmitted teconfirrnatory

factor

analyses. Allthe

pathstotheobserved

variables were statisticallysignificantatp < .O1

,

and

the

goodnessof

fit

indicesshowed acceptable

levels,lnorder toobserve thedifferencesinthisfactorialstmcture among thethree

groupsof

participants,confirmatory factoranalyses, using thesame model, were condncted by dividingthe

three

groups

(see

Figure

3

fbrits

path

diagi;am).

Table3 sumrnarizes thecomparison of the

goodness offit indicesamong the

groups.Theindicesshowed almost equal

goodness offitamong the

groups.Also,multiple sample stmcture equation modeling was used todetectthedifferences

arnong the_groups.We testedfourmodels:

(a)

"configural",

ofwhich the

pathsare equal among

the_groups,

_fo)

"weak

measurement invariance",of which loadingsare invariant,

_(c)

a

"strong

measurement inyariance",of which loadingsand interceptsare invariant,and

(d)

another typeof

theformer,ofwhich loadings,intercepts,and means are invariant,As theresults demoilstrate,all of themodels showed a

goodnessof fit,as inTable 4.The fburthmodel, which was under the

strongest constraints,was the

best

model. Hence,itcan

be

safely statedthatat leastthefactorial

structureand itsloadingswere not invariantamong the_groups.Thissuggests thatthescalewhich

the

presentstudy developedcan measure theGC ofvarious levelsoflearners. Table3.

Cbmparison

ofGoociness

of]FVt

indicesamong the Groups

(lroup n

_f

_of

p CFITLI RMSEA SRMR

A!1Junior high HighUniversity

2,098 1,121.27 216 206.98 1,078 567.61 804 537.29

74747474<.Ol<.

Ol<. Ol<. Ol

.94.93.94.92 .92.92.92.91 .08

[.07,

,08]

.09

[.08,

.11]

.08

[.07,

,09]

.09

[.08,

,11]

.05.05.05.06

Table4.

Sle(mmar;I?

_{ofMeasurement}

invarianceamong theMuttipieSbnrples

Model

_f

ctf"pCFI RMSEA BIC

Configuralmodel

Weak measurement invariancemodel

(equal

loadings)

Strongmeasurement invariancemodel

(equal

loadings+ intercepts)

St!!Qng-gigqsuig!ugn!-igya!iaggg-!ugdg!

tr t dl

(equal

loadings+ intercqpts+ means)

1,311.87

1,333.l8

1,345.09

-138045

222

2an

266

<.Ol .93

<,Ol ,93

Z2tZ2

<,Ol

..9.3

.05

.04

=04

90,127.45

89,980.58

89,824.28

.8.N...

981385

(9)

.3s/.s4l.s41(･si}-)[iiEiiilll'N

.so!.6s1.6s1(.7o)

.631.671.67/(.66}-!l!Ellll21-.61/.ss/.s7

.391.32!.31/(.33}p

m

.78/.82/.83

･4i1･4g1.s4/{.so}-.Il!Eliilll.

･77/･7i/･6s

,63!,6S/,60

.6oi.s7/.64/(･6o)

-b[iiEiiil5]'

,451,52f,S4/(.51)p

a

.74/.S9/.68

.391,561.61!{.55)p

a

.781.661.63

,231,32f.31/{.3o}-[l!EiiiSl.

.88f.82/.83!

,91/.80/.84l

.161,3s/.3o!C3o) ltemlO

,4Sl,46f,48!C47) ltemll

,74/J3/.

.34!.33/.28/(.31) ltem12

.821.82/.

,331,311,37/(,33} ltem13 .s21.831,

.281,301.3s!C31) ltem14

･85/.84/.

.81/.77/.801(.78)

.351,411.36/{.39) ltemlS

.F7gure 3.Pathdiagramrepreseming themodel in

!

university

1 _(all)".

N=

2,098.

The results so farprovidedsufficierrt psychologicalscale,foreignLanguage

which

yields thethreefactors:phonological pragmaticcarefulness, Forreference, thedescriptive

Allthescores exhibited anormal distribution.

.65/.S61.53/(.S7)

(.64)

question,with standardized estimates. The

standardized estimates foreach

groupwere shown intheformof

`tiunior

highschool /highschool

empirical evidence forestablishing the new

Gvammatical

Ckerefulness

Sbate

(FLGCS,

hereafter),

carefulness, Iexical-syntacticcarefulness, and

statistics of thesummated scale scores are

sumniarized inTal)le5.Phonologicalcarefulness exhibited relatively higherscores thantheothers.

Table5,

DescriptiveSlatistias

ofthe

ShrmmatedSbaleSZroregkM

sw Skewness Kurtosis Phonologicalcarefulness

_(item

No.1

1

to

15)

Lexical-Syntacticcarefulness

(item

No.6to1O)

Pragrnaticcarefulness

(item

No.1to5)

All

54514

3.883.443.503.62

1.271.111.llO.98

O.14O.22O.14O.13

-O.35O.06-0.35-O.13

Nbte,Thesummated scale scores herewere themean scores fortheresponses ofthe itemsfbr

each. Thefactorscores were notused here.n

=

2,098.

(10)

4.InitialValidatien

For themitialvalidation

procedure,

thepresentstudy furtherexamined thecontent

and criterion validity of FLGCS by condncting multiple fbllow-upanalyses. We testedthethree hypothesesbelow

(hypothesis

ItoIII),

H)/pothesis

I:Thecontents ofall theitemsinFLGCS match thetheoreticalconcepts ofeach

factor.Forinstance,itwas hypothesizedthattheitemsfbr

phonologicalcarefulness actually refeT

tothe_phonologicalaspects of

grarnmaticalperfbrmance inlinguisticterms.

thpothesis

IZIiEachtypeof

grammaticalcarefulness iscorrelated tolearnerbeliefswith a

medium levelof strength. More specifically,

GCs

are correlated to analytic

beliefs

_(Tanaka

& Ellis,2003)more strongly thantoexperientialbeliefs.

Hilpothesis

I[l:GCs are correlated totheaccuracy ofa C-test,which issupposed tomeasure

generallariguageperfbrmance, and thetime which test-takerstaketocomplete thetest.As discussedintheBackgroundsection of thisstudy, GC was considered as a typeof moderator of compromise

pointsinthe

SAT

framework;thus,itisassumed thatsemeone with a higher

GC

should exhibithigheraccuracy and lowerspeed inthetask.

4.1

Content

Validity

Ten 1inguisticsexperts voluntarily

participatedinthispartof thestudy, Usingan online

version of

the

questionnaire,we asked theparticipantstoread thequestionnaireitems

(k

=

14)

carefu11y,thentoselectwhich typeof

grammaticalperformancetheitemrefers to,inlinguistic

temis,by choosing from one of fouralternatives:

(aj

phonological,

th)

lexical-syntactic,

(c)

pragmatic,and

(d)

none ofthem. Itwas not allowedtoskip an item.

The result was thatallthe

participantsanswered thatthe itemsNo. 1to5 referred to the

pragrnaticaspects of

grammaticalperfbrrnance,6to1Othe

lexical

and syTitacticaspects,and 11to

15the_phonologicalaspects. This

providesus with empirical support forthecontent validity of

FLGCS on a certain

point.

4.2CriterionValidity

4.2.1RelationshipwithLearnerBeliefs

Inorder toconfirrn a

partof

the

criterionvalidity

(especially

convergent and

discriminate

validity) of FLGCS, thisstudy investigatedthecorrelation

patternsbetween FLGCS and two types,oflearner

beliefs,

analyticand experientialbeliefs

_(H/mpothesis

_M.

Analyticand experiential

beliefs

_(AB

and EB foreach) were established byTanakaand Ellis

_(2003).

Theft)rmertyperefers

tolearners'beliefswhich support analytical

types

of learningmethods and theirbenefits,and

consisted of 7 _{questionnaire}

items

_(e.g.,

I can

learn

well

_by

writing clown evetything inno? notebooe, while theother typesupports experiential ones, with 7

questionnaire

items

(e.g.,

Ican

(11)

learnwell Lly

spealdngwith otheKs inEnglish).Allofthe

items

in

the

Japanese-translatedversion

are alsoavailable online

_(see

Appendix).Theoretically,itcan

be

expected thatGC and beliefs

show some correlations, and GCs are related toanalytic beliefsmorc strongly thanexperiential ones

in

termsoftheirconceptual relevance.

The

data

ef thissection was cempared with that of the

preliminarystudy, which included boththeGC questionnaireitemsand thelearnerbelief

items.

[[hus,all

the

participants

(168

answers were used) were

first

_yearuniversity students.

Firstly,the

descriptive

statisticsand

the

reliabilitycoethcients

for

each of the'summated

scale scores were calculated

(see

Tahle6),The sarnple showed relatively higherexperiential

beliefs,and a lowerlevelof GCs than theresults ofthe secondary study. Itis

possibletoinferthai

thesubsample hadatendencytosupport experientialbeliefs_preferahlyand

be

less

grammatically

carefu1. We

_judged

thattherelial)ilityofeach score was sufficient

(.73

to.91).

Then, acorrelation analysis among thefivesummated scale scores was conducted. Figure4 graphicallysumrnarizes

the

correlation pattem and the

disuibutions

of

the

scores, We also used

classical mutti-dimensional scating

_(Crvfl)S,

also known as

principlecooTdinate ana4ysis;

see

Coxon,1982).

CMDS

is

a statisticalmethod tovisualizethesimilarityofvariables. Basedon the

correlation coefficients matrix, CMDS can

placeeach variable on a two-dimensionalscale.Thus, itcan be interpretedthata

pairof closer variables intheplotmeans thattheyhavea higher

correlation, and more distantvarial)les show lowercorrelations. Figure5 shows theresults of

CMDS.

Table6.

DescriptiveSinttstiasand Reliability

ofthe

Sle{mmatedSbaleSloores

ofE[LGC:S

andLearner Beli(:tS

k

M

saSkewness

Kurtosis ct

Phonologicalcarefulness

(PH)

_(LS)

Pragmaticcarefulness

(P)

Analyticbeliefs

_(AB)

Experiemialbeliefs

_(EB)

545 3.492.983,301.211.331.29O.19 O.54-O,57

O.04O.16O.67

.87,91,89

77 4.154.59O.95O,97O,13-O,33 O.46O.52 ,73,74

IVlote.n ==

168.

Theresults ofthe correlationanalysis clearlysupported IlypothesisU;allofthe GSs showed

low tomiddlelevelsof correlation coecacients, butmore specifically theywere more strongly

relatedtoanalytical

beliefs,

PH: r= ,63,

with

its

95% CI

being

_[,53,

.71], LS:r

==

.51

_[.39,

.61], P:

r

= .61

[.51,

,70],

than

toexperiential ones, PH: r

=

,33,with

its

959'6CI

being

[.19,

,46],LS:r

= ,22

[.07,

.36],P:r= .39

[.25,

.51].Also,as Figure5presents,allofthe GCs were locatedcloser

toanalytic beliefsthantoexperiential ones. [rhislinks

perfectlytotheconceptual relevance among them.

(12)

JapanSociety ofEnglishLanguage Education

1 3 57 1 3 57

tn{]

ma

Ei]

tw

[iSl

l･

ieniilistl[IEI][l5i][l21]

'

ff

pm

kifim

[IIEi]

[EIIEI

l'

:-pa

MM

rm

EII

r

papas[iiijllimal

1 3 S 7 I S 5 7 1 3 5 7 jF igure4.Scatter

plotmatrix representing the

correlation coeracients on theupper side,the

histograrns

in

the

middle

columns, and the

scatter

pletswith 1inearregressions on the

lowerside.

9-:

g

8 :

R

P,(i]ilg

-1

,O-O,5o,oO.51,O

Figure 5. Plotrepresenting the

distances

between the variables, based on their

correlationcoeMcients matrix.

4.2.2RelationshipwiththePerformanceofa C-test

This section wi11report the results of theexperiment which investigatedtherelationship

betweenGC and language_perfbrmance.We assumed

that

GC as amoderator

in

SAT

will show a

correlation with boththeaccuracy

_(score)

and thespeed

(time

tocomplete) ofa

language

test.The

presentstudy

focused

on the_{perfbrrriance}ofa

C-test.

Thenumber ofpanicipants was 77.Allof

the

panicipantswere

first

yearuniversitystudents,

The

participantsoverlapped withthesecondary study. Afterthesecondary study,

they

participated ina C-test

_(detailed

below)as a

partofthe learningactivity of theirEnglishclasses,

in

June,2014,

Allthe

participantswere women. We also used thedataabout theirGC, as determined

in

the

secondary study.

TIheC-testwas created

by

theauthors

(also

avai1ahle on theauthors' web

page).The text

typewas narrative

(a

letter

toawriter's

friend),

The lengthofthe textwas 249words, including

some blankwords. The number of items

_folanks)

was 17,which isequal toalmost 7% of the

whole text.The readability scores ofthe text,ignoringthe

blanks,

were 91atFleschReadmg Ease, 2.6atFlesch-KincaidGrade Level;theselevelsare usually regarded tobe easy in･fbreigri languagereading studies.The examiner asked

the

participantsto

fi11

in

the

blanks

intheuntimed

condition

(there

was no time1imt),butalso asked the_participantstoreportthe time when they

had

completed answering. A digitalcount-up timer was displayedon the monitor of the

classrooms, and the_participantscould note thetimewhen theyfinishedanswering, using this.

Table7presentsthedescriptive

statisticsand thereliabilityfbrthesmmated scale scores of

GCs,

thescores of theC-test,and thetimetocomplete thetest.Thissubsample may harreshown

(13)

lowerGCs incornparison tothewhole dataof thesecondary study, The reliahilitycoethcients were acceptable, Figures6and 7summarize thecorrelation

patterns,as intheprevioussection.

Table7.

Description

_of

theStimmatedSbaleSbores

of

GCs,

C-7layt

the

Slrows,

and the7TmetoCbmpletethe

k M swSkewness Kurtosisa

Phonologicalcarefulness

_(PH)

(LS)

Pragmaticcarefulness

(P)

ScoreTime

tocomplete

_(sec)

545 3.182.622.79O,97O.91O.87O.03O,20-O.08O.03-O.14-O.74.81.79.79

17n.a. 5,45491,342.35138.36O.58O.54 O.421.45,63n,a.

IVbte,n =

77.

135 04Slt

[ImuEi61ESIEillEIZI:

i

ilRili

rm

[il2i]

[!!l]

[III6]

F

tw

Eiiill]

[ii[iiN

[illiE]

[ilill

r

iew

ge

[illl

[ilii]

e

!llllll

[kiii]

[iiiiE]

[liillll

IZilill

:

1SS lt34

ZIOEco jFVgure 6.Scatter

plotmatrix representing the distributionsand thecorrelations

between

the

variables fbr

H)/pothesis

ILIL

g･

g

g-8g8

dy

S re

wy

-O.6-O.4-O.2

O.O O,2 O.4 O,6

jFVgure 7.Plotgraphicallyrepresenting the

results of CmoS.

The results supported I]5/pothesisllI.GCs are correlated toboththescores, PH: r

= .35

[,21,

.48],LS: r= ,41

_[.28,

.53],P:r

-.38

_[.24,

.50],and thetmes,PH: r

-.27

[,12,

,41],LS:r

= .36

[,22,

,49],P:r

=

.31

_[.17,

.44],with lowtomiddle

levels

for

the

coedicients. Theresults of

CMDS alsosuggest thatGCs havea correlation with thescore and thetime,with almost thesame magnitudes. Thismeans thatGC linksto

both

theaccuracy and speed of language

perforrnance,

exactly as theframewotkof

SAT

expected,

4.3Summary of theInitialVafidation

Our initialvalidation

providedinfbrmatienregarding

both

thecontent and criterion validity of thescale. The sumrriary of

the

results of our mitialvalidation, using a hypothesistesting

procedure,

is

shown

in

Table8.

(14)

Table8.

71JieSIimmary

pfthe

Results

ofthe

initialVbliciation

Hypotheses Content Results Evidence

I ThecontentsofalltheitemsinFLGCSmatch Supported Properly

judged

by 10

thetheoreticalconcepts of each factor. Iinguisticexperts

II GCs are correlated toanalytical beliefsmore SupportedShowed the correlation strongly thantoexperientialbeliefs. patternexactly asexpected

III GCs are correlated to boththescore ofa Supported Showed the correlation

C-testand thetimetofinishthetest, patternexactly asexpected

5.GeneralDiscussion

The results presentedal)oveleadus toconclude

thatthenew

psychologicalscale, FLGCS,

with

its

threefactors

(k

=

14),isa statistically reliahle measure. Itsstmctural, content,and criterion validity were also supported by conducting multiple analyses. Funhermore,multiple sample stmctural equation modeling

demonstrated

thatFLGCS showed measurement mvariance among

the

groups.However,FLGCS has

acouple ofpotential

limitations,

as noted below. Most importantly,regardless of itshigh

reliability,FLGCS covers only a small area of GC

as aconstmet. As inter-correlationand inter-factorcorrelation coefficientssigriified

quitestrongly, the_{questionnaire}itemsmay measure very close behaviorsand characteristicsof individuals.This phenomenon iscalled bandwidlrh:fidelity

dilemma.

However, since GC isa new concept, our

preliminaryaim was toestablishareliablescaleattheexpense ofits coverage, inorder to_provide

abasisforfurtherresearch. As intheBackground section and theliteraturereview of the

present

study, therationale forGC underlies

the

concepts of SAT, and thescale was mainly

designed

to

be

applied in

psycholinguistic

studies,classroom-based studies, and teaching

practice.Needless

to say, lessreliable measures leadtoattenuation

problems,statistically,Hence,we

presumed to

judge

that

amore reliable scale was _preferableinthiscase.

Obviously,validation isnot a dichotomous

judgment

and fUrthervalidation isalways strongly

desired,

A

partof theevidence whichtheinitialvalidation _providedmay cover only a very small range of validity.Futurcstudies should confirm thelinksbetweenGC and othertypes of individualdiffbrences,the developmentof ([}C,and relationships with other types of

grammatical perfbrrnance

(e.g.,

grammaticality

judgment,

sentence verification,and

imprornptu

speech), Additionally,whereas

the

_presentstudy was a 1arge-scalesurvey,

it

never

denies

the

existence of sampling errors, Datawithmore varied and 1argersamples wi11also beneeded.

Itshould

be

noted thatthe

presentstudy failedto

assess the feasibilityand theconsequential aspects of validity.Itwi11be importarrtto analyze

the

washback efft)cts on leaminglteaching behaviorsoflearnersfteachers

in

practice.

(15)

6. Conclusion

Thisstudy

developed

and validated a

psychologicalscale tomeasure GC, which isrelated to

irrter-learner

variance on SAT. SAT, a sophisticated frameworkconceming human behaviors,

may explain a large

partof languageperfbrmance,and

GC,

as an

individual's

trait,will

be

key

to

capturing

the

dynamics

of numerous varial)lesconcemed withlanguage_{perforrnance.}However,

the

importance

ofthis

is

not

limited

to

theories

ofsecondlfbreign languageacquisition and use.

Inteachingpractice,FLGCS can also

provideteachersmnch infbrmationabout theirstudents.

FLGCS will enable ustounderstand students' traits.It

wi11

also

helpteachersspecify what kmd of

grammaticalcarefulness

(phonological,

lexical-syntactic,and

pragrnatic)of a panicularstudent is

(in)suthcient.

The infbrmationwi11contribnte totheeveryday teaching_practiceof Englishby

playingvarious roles inthework ofteachers. Likewise,FLGCS willbeusefu1,even fbrlearnersto

understand

their

own traits,Thismay

promote

leamers'selfregulated learning. Notes

LThe

terrn

GC

has

numerous simi1ar terTnssuch as meta-lingtiisticawarenexy, language

awareness, languagesensitivity, and

grammatieal

sensitivity, However, these

generallyrefer

toone's

knowledge,

orcertaintypesoflanguage-related ski11s,which are mainly measured by languagetests.We

intended

torefer toGC only as a

psychologicaland behavioraltrait,which

we consider tobefUndamentallyseparate fromlanguage

knowledge

or ski11s,

However,

we alsoassume

that

theymay

be

correlatedtoeach other tosome extent.

References

Coxon,A, P.M,

(1982).

77)euserls

guidetomultidimensional scaling: Mith

speeialrc!XZirence

to theMDS. London: Heinemann EducationalBooks.

Dennis,I.,& Evans,J.St.B,T.

(1996),

The speed-error trade-offproblemin

psychometric

testing.

British.loumal

ofRsychology,

87,105-129.

Goldhammer,F.,& Kroehne,U.

(2014).

Controllingindividuals'time spent on task inspeeded

perfic}rrnancemeasures: Experimenta1tirne

limits,

posterior

time limits,and response time modeling.

Al?plied

Rsycholqgi'calA4easurement,38,255-267,

Kartehava,E.,& Ammar, A.

(2014).

Learners'beliefsas mediators ofwhat isnoticed and learned

inthelanguageclassroom. 1:ES()L

euarterly,

48,86-109.

Krashen,S,D.

(1978),

Individnalvariation intheuse ofthe monitor. InW. Ritchie

_(Ed.),

Sticond

languageacquisition research

(pp.

175-183).New Yotk,NY: AcademicPress,

Kormos, J,

_(1999).

The effect of speaker variables on theselfcorrection

behaviour

of

L2

learners.

Srgtem,

27,207-221,

(16)

Japar ユ　 Society 　of 　 English 　 Language 　 Education

Segalowiセ

，　N ，

，

＆ Segalowitz

，　S．」．

（1993），　Ski11ed　

pe曲 mance

，　

practice，　and

the

differentiation

　　of 　speed −up 丘om 　automa 重ization　effbcts ：Evidence倉om 　second 　language　word 　recognition ．　　ノ

勿 1ガθ

4

勾 6乃o〜inguistics，／4，369−

385

，

Seliger

，　H．

（1980），

　Utterance　

planning

　and　correction　behavior：Its　ft ction 　l皿 the　

grammar 　　construction　

process

　ibr　second 　language　leamers． H．W ．　Dechert

，

　M ．　Raupach （Eds．），　　Tovvarcts

a　

cro∬ 伽

g istic　a∬essment （

抑 θε

吻 roぬo伽

（pp、

87

−

99

）．　Frank顛，　Germany：　　Lang．

Tanaka

，　K．

，

＆ Ellis ，

　R ．

（2003）．　Study　abroad

，

　language　

proficiency，

　and 　leamer　

beliefs

　about

　　language

　lean血

g．　L尻乙τ

ソ

：

ournal

，

25

，

63−85・ Tarone

，　E．

（1985）．

　Variability　in　interlanguage　use ：Astudy　of　style

−

shift血

g　in　morphology 　and

　　syntax．　Language　Leaming，

35

，

373

−₄₀₄

，

van 　der　Linden

，

　W ，　J．

（2007）．　A　hierarchical　frarnework　for　modeling 　speed　and　accuracy 　on　test 　　items．　Psychometrika

， 73

，

287−308．

Appen 〔hx

Foreign

Language

（］ran 〃natical 　Carefalness　Scale　

（FLGcs ）

Item　1

（P）

ltem　2

（P）

ltem　 3

（P）

ltem　4 （P｝ Item　5

（P）

Item　 6

（LS） ltem　8

（LS）

外国語を使うとき，会話の流れの不自然さに

ついてよく考える

外国語を使うとき，表現が文脈にあわないと

考えこんでしまう外国語を使うとき，

一

貫してない表現や曖昧

な表現にはよく気がつく

外国語を使うとき，一

貫していない表現があ

ると考えこんでしまう

外国語を使うとき，失礼な表現や丁寧過ぎる

表現がよく気になる

外国語を使うとき、語の形の変化の誤りには

よく気がつく方だ

外国語を使うとき，単語のつづりが間違って

いるとよく気になる

1tem　 9

（LS）

ltem　10

（LS）

Item　

11

（PH）

Item　 t　

2

（PH）

Item　13 （PH） item　14

（PH）

ltem　15

（PH）

外国語を使うとき，文章の中で間違った単語があるとよく気がっく

外国語を使うとき，単語の間違いにはよく気

づく方だ

外国語を使うとき，発音が正確が考えることが多い

外国語を使うとき，いつも発音が正しいかどうか気になる

外国語を使うとき，発音が正確でないと考え

こんでしまう

外国語を使うとき，発音が誤っていると気に

なってしまうことが多い

外国語を使うとき，発音が本当に正確か確認

することがある Note．　_Item　₇　was 　

deleted

（see　

the

　section　concerning 　scale 　development

）．　Note

，

just

　for　reference ，　　Item

　₇ ：

“

外国語を使うとき，文法規則に合わない表現によく気がつく

” ．　

P

＝

pragmatic

　carefUlness，　LS ＝

lexical

−

syntactic　carefUlless

，　PH ＝

phonological　carefUlness．　Supplementary　data　including

　　（

a

）the

　mitial　

item

pool，（

b

）the　

C

−test，　and

（c

）the　final

version 　

of　

the　questio皿 aire　

used 　

_in

　　the

present

　stUdy

，　are　

available 　

at　_the　_lilst　author

’

swebsite ：

hゆs：1／sites．

google．com

／site！kUsanagikt　ni！home！

pr（｝

jects

／

gc

92

N工工

一

kusanagi et al 2015 最近の更新履歴 草薙邦広のページ kusanagi et al 2015

Foreign

Language

Grammatical

CarefulnessScale:

Scale

Development

and

Its

Initial

Validation

Kunihiro

KUSANAGI

Junya

FUKUTA

SbhooL

ofStrience

Yusakti

KAWAGUCHI

Yu

TAMURA

Aki

GOTO

AI(ari

KURITA

Daisuke

MUROTA

Migaya

develop

(GC)

definition,

psychological,

learner,

laiiguage

jum'or

(N

(FLGCS)

for

yields

(a)

(b)

(c)

by

(a)

(n

judged

(b)

beliefs,

theoretical

(c)

C-test,

the

in

language

language

Iearning

factors

large

(SLA)

(e.g.,

(e.g,,

is

jointly

these

factors

just

inter-learner

gramrnatical

perfbrmance;

Speed-Accuracy

(SAT),

have

(e.g.,

(a)

in

1,

SAT

(a)

(b)

(e,g,,

kusanagi et al 2015 最近の更新履歴草薙邦広のページ kusanagi et al 2015

_(N

_(a)

_(a)

_jointly

_just

_(a)

_(a)

_(e,g,,

_(c)

_(c)

_(a)

_{(aecuracly-centered}