Volume 2010, Article ID 956907,30pages doi:10.1155/2010/956907

*Research Article*

**QML Estimators in Linear Regression Models with** **Functional Coefficient Autoregressive Processes**

**Hongchang Hu**

*School of Mathematics and Statistics, Hubei Normal University, Huangshi 435002, China*

Correspondence should be addressed to Hongchang Hu,retutome@163.com Received 30 December 2009; Revised 19 March 2010; Accepted 6 April 2010 Academic Editor: Massimo Scalia

Copyrightq2010 Hongchang Hu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

This paper studies a linear regression model, whose errors are functional coeﬃcient autoregressive processes. Firstly, the quasi-maximum likelihoodQMLestimators of some unknown parameters are given. Secondly, under general conditions, the asymptotic propertiesexistence, consistency, and asymptotic distributionsof the QML estimators are investigated. These results extend those of Maller2003, White1959, Brockwell and Davis1987, and so on. Lastly, the validity and feasibility of the method are illuminated by a simulation example and a real example.

**1. Introduction**

Consider the following linear regression model:

*y**t**x*^{T}_{t}*βε**t**,* *t*1,2, . . . , n, 1.1

where*y** _{t}*’s are scalar response variables,

*x*

*’s are explanatory variables,*

_{t}*β*is a

*d-dimensional*unknown parameter, and the

*ε*

*t*’s are functional coeﬃcient autoregressive processes given as

*ε*1

*η*1

*,*

*ε*

*t*

*f*

*t*θε

*t−1*

*η*

*t*

*,*

*t*2,3, . . . , n, 1.2

where*η** _{t}*’s are independent and identically distributed random errors with zero mean and
finite variance

*σ*

^{2},

*θ*is a one-dimensional unknown parameter and

*f*

*θ is a real valued function defined on a compact setΘwhich contains the true value*

_{t}*θ*0as an inner point and is a subset of

*R*

^{1}. The values of

*θ*

_{0}and

*σ*

^{2}are unknown.

Model1.1includes many special cases, such as an ordinary linear regression model
when*f** _{t}*θ≡0; see1–11 . In the sequel, we always assume that

*f*

*θ*

_{t}*/*0,for some

*θ*∈Θ, is a linear regression model with constant coeﬃcient autoregressive processeswhen

*f*

*θ*

_{t}*θ, see Maller*12 , Pere 13 , and Fuller 14 , time-dependent and functional coeﬃcient autoregressive processes when

*β*0, see Kwoun and Yajima 15 , constant coeﬃcient autoregressive processes when

*f*

*θ*

_{t}*θ*and

*β*0, see White 16, 17 , Hamilton 18 , Brockwell and Davis 19 , and Abadir and Lucas 20 , time-dependent or time-varying autoregressive processeswhen

*f*

*t*θ

*a*

*t*and

*β*0, see Carsoule and Franses21 , Azrak and M´elard22 , and Dahlhaus23 , and so forth.

Regression analysis is one of the most mature and widely applied branches of statistics. Linear regression analysis is one of the most widely used statistical techniques.

Its applications occur in almost every field, including engineering, economics, the physical sciences, management, life and biological sciences, and the social sciences. Linear regression model is the most important and popular model in the statistical literature, which attracts many statisticians to estimate the coeﬃcients of the regression model. For the ordinary linear regression modelwhen the errors are independent and identically distributed random variables, Bai and Guo 1 , Chen 2 , Anderson and Taylor 3 , Drygas 4 , Gonz´alez- Rodr´ıguez et al.5 , Hampel et al.6 , He7 , Cui8 , Durbin9 , Hoerl and Kennard10 , Li and Yang11 , and Zhang et al.24 used various estimation methodsLeast squares estimate method, robust estimation, biased estimation, and Bayes estimationto obtain estimators of the unknown parameters in1.1and discussed some large or small sample properties of these estimators.

However, the independence assumption for the errors is not always appropriate in applications, especially for sequentially collected economic and physical data, which often exhibit evident dependence on the errors. Recently, linear regression with serially correlated errors has attracted increasing attention from statisticians. One case of considerable interest is that the errors are autoregressive processes and the asymptotic theory of this estimator was developed by Hannan and Kavalieris 25 . Fox and Taqqu26 established its asymptotic normality in the case of long-memory stationary Gaussian observations errors. Giraitis and Surgailis 27 extended this result to non-Gaussian linear sequences. The asymptotic distribution of the maximum likelihood estimator was studied by Giraitis and Koul in 28 and Koul in29 when the errors are nonlinear instantaneous functions of a Gaussian long-memory sequence. Koul and Surgailis 30 established the asymptotic normality of the Whittle estimator in linear regression models with non-Gaussian long-memory moving average errors. When the errors are Gaussian, or a function of Gaussian random variables that are strictly stationary and long range dependent, Koul and Mukherjee31 investigated the linear model. Shiohama and Taniguchi32 estimated the regression parameters in a linear regression model with autoregressive process.

In addition toconstant or functional or random coeﬃcientautoregressive model, it has gained much attention and has been applied to many fields, such as economics, physics, geography, geology, biology, and agriculture. Fan and Yao 33 , Berk 34 , Hannan and Kavalieris35 , Goldenshluger and Zeevi36 , Liebscher37 , An et al.38 , Elsebach39 , Carsoule and Franses 21 , Baran et al.40 , Distaso 41 , and Harvill and Ray 42 used various estimation methodsthe least squares method, the Yule-Walker method, the method of stochastic approximation, and robust estimation methodto obtain some estimators and discussed their asymptotic properties, or investigated hypotheses testing.

This paper discusses the model 1.1-1.2 including stationary and explosive
processes. The organization of the paper is as follows. InSection 2some estimators of*β, θ,*

and *σ*^{2} are given by the quasi-maximum likelihood method. Under general conditions,
the existence and consistency the quasi-maximum likelihood estimators are investigated,
and asymptotic normality as well, inSection 3. Some preliminary lemmas are presented in
Section 4. The main proofs are presented inSection 5, with some examples inSection 6.

**2. Estimation Method**

Write the “true” model as

*y**t**x*^{T}_{t}*β*0*e**t**,* *t*1,2, . . . , n, 2.1
*e*1 *η*1*,* *e**t**f**t*θ0e*t−1**η**t**,* *t*2,3, . . . , n, 2.2

where*f*_{t}^{}θ0 df*t*θ/dθ|_{θθ}_{0}*/*0, and*η** _{t}*’s are i.i.d errors with zero mean and finite variance

*σ*

_{0}

^{2}. Define

_{−1}

*i0**f** _{t−i}*θ0 1, and by2.2we have

*e**t*^{t−1}

*j0*

_{j−1}

*i0*

*f** _{t−i}*θ0

*η*_{t−j}*.* 2.3

Thus*e** _{t}*is measurable with respect to the

*σ-fieldH*generated by

*η*

_{1}

*, η*

_{2}

*, . . . , η*

*, and*

_{t}*Ee** _{t}*0, Vare

*t*

*σ*

_{0}

^{2}

*t−1*

*j0*

_{j−1}

*i0*

*f*_{t−i}^{2} θ0

*.* 2.4

Assume at first that the*η** _{t}*’s are i.i.d.

*N0, σ*

^{2}. Using similar arguments to those of Fuller14 or Maller12 , we get the log-likelihood of

*y*

_{2}

*, y*

_{3}

*, . . . , y*

*conditional on*

_{n}*y*

_{1}:

Ψ*n*

*β, θ, σ*^{2}

log*L** _{n}*−1

2n−1log*σ*^{2}− 1
2σ^{2}

*n*
*t2*

*ε** _{t}*−

*f*

*θε*

_{t}*t−1*

_{2}

− 1

2n−1log2π. 2.5

At this stage we drop the normality assumption, but still maximize 2.5 to obtain QML
estimators, denoted by*σ*_{n}^{2}*,* *β*_{n}*,* *θ** _{n}*when they exist:

*∂Ψ**n*

*∂σ*^{2} −*n*−1
2σ^{2} 1

2σ^{4}
*n*

*t2*

*ε** _{t}*−

*f*

*θε*

_{t}*t−1*2

*,* 2.6

*∂Ψ**n*

*∂θ* 1
*σ*^{2}

*n*
*t2*

*f*_{t}^{}θ *ε** _{t}*−

*f*

*θε*

_{t}*t−1*

*ε*_{t−1}*,* 2.7

*∂Ψ**n*

*∂β* 1
*σ*^{2}

*n*
*t2*

*ε** _{t}*−

*f*

*θε*

_{t}*t−1*

*x*

*−*

_{t}*f*

*θx*

_{t}*t−1*

*.* 2.8

Thus*σ*_{n}^{2}*,* *β**n**,* *θ**n*satisfy the following estimation equations:

*σ*_{n}^{2} 1

*n*−1
*n*

*t2*

*ε**t*−*f**t*

*θ**n*

*ε*_{t−1}_{2}

*,* 2.9

*n*
*t2*

*ε**t*−*f**t*

*θ**n*

*ε*_{t−1}*f*_{t}^{}

*θ**n*

*ε** _{t−1}*0, 2.10

*n*
*t2*

*ε**t*−*f**t*

*θ**n*

*ε*_{t−1}*x**t*−*f**t*

*θ**n*

*x*_{t−1}

0, 2.11

where

*ε*_{t}*y** _{t}*−

*x*

^{T}

_{t}*β*

_{n}*.*2.12

*Remark 2.1. If* *f** _{t}*θ

*θ,*then the above equations become the same as Maller’s 12 . Therefore, we extend the QML estimators of Maller12 .

To calculate the values of the QML estimators, we may use the grid search method, steepest ascent method, Newton-Raphson method, and modified Newton-Raphson method.

In order to calculate inSection 6, we introduce the most popular modified Newton-Raphson method proposed by Davidon-Fletcher-Powellsee Hamilton18 .

Letd2×1 vector→−

*θ*^{m} σ^{m2}*, β*^{m}*, θ*^{m}denote an estimator of→−

*θ* σ^{2}*, β, θ*that
has been calculated at the*mth iteration, and letA*^{m} denote an estimation of H→−

*θ*^{m} ^{−1}.
The new estimator→−

*θ*^{m1}is given by

−

→*θ*^{m1}→−

*θ*^{m}*sA*^{m}*g*
→−

*θ*^{m}

2.13

for*s*the positive scalar that maximizesΨ*n*{→−

*θ*^{m}*sA*^{m}*g*→−

*θ*^{m}},whered2×1 vector

*g*
→−

*θ*^{m}

*∂Ψ**n*

→−
*θ*

*∂*→−
*θ* |→−

*θ*→−
*θ*^{m}

⎛

⎜⎜

⎜⎜

⎜⎜

⎝

*∂Ψ**n*

*∂σ*^{2}|_{σ}^{2}_{σ}^{m2}

*∂Ψ**n*

*∂β* |* _{ββ}*m

*∂Ψ**n*

*∂θ* |* _{θθ}*m

⎞

⎟⎟

⎟⎟

⎟⎟

⎠

2.14

andd2×d2symmetric matrix

**H**
→−

*θ*^{m}

−*∂*^{2}Ψ*n*

→−
*θ*

*∂*→−
*θ ∂*→−

*θ*^{T}

|→−
*θ*→−

*θ*^{m}

⎛

⎜⎜

⎜⎜

⎜⎜

⎜⎝

*∂*^{2}Ψ*n*

*∂σ*^{2}^{2}

*∂*^{2}Ψ*n*

*∂σ*^{2}*∂β*

*∂*^{2}Ψ*n*

*∂σ*^{2}*∂θ*

∗ *∂*^{2}Ψ*n*

*∂β∂β*^{T}

*∂*^{2}Ψ*n*

*∂β∂θ*

∗ ∗ *∂*^{2}Ψ*n*

*∂θ*^{2}

⎞

⎟⎟

⎟⎟

⎟⎟

⎟⎠

|→−
*θ*→−

*θ*^{m} *,* 2.15

where

*∂*^{2}Ψ*n*

*∂σ*^{2}^{2} *n*−1
2σ^{4} − 1

*σ*^{6}
*n*

*t2*

*ε**t*−*f**t*θε_{t−1}_{2}
*,*

*∂*^{2}Ψ*n*

*∂σ*^{2}*∂β* − 1
*σ*^{4}

*n*
*t2*

*ε**t*−*f**t*θε_{t−1}*x**t*−*f**t*θx_{t−1}_{T}*,*

*∂*^{2}Ψ*n*

*∂σ*^{2}*∂θ* − 1
*σ*^{4}

*n*
*t2*

*ε**t*−*f**t*θε_{t−1}*f*_{t}^{}θ,

2.16

*∂*^{2}Ψ*n*

*∂β∂β** ^{T}* − 1

*σ*

^{2}

*n*
*t2*

*x** _{t}*−

*f*

*θx*

_{t}

_{t−1}*x*

*−*

_{t}*f*

*θx*

_{t}

_{t−1}

_{T}*,* 2.17

*∂*^{2}Ψ*n*

*∂β∂θ* − 1
*σ*^{2}

*n*
*t2*

*f*_{t}^{}θε_{t−1}*x*_{t}*f*_{t}^{}θε*t**x** _{t−1}*−2f

*θf*

_{t}

_{t}^{}θx

_{t−1}*ε*

_{t−1}*,*

*∂*^{2}Ψ*n*

*∂θ*^{2} − 1
*σ*^{2}

*n*
*t2*

*f*_{t}^{}^{2}θ *f** _{t}*θf

_{t}^{}θ

*ε*^{2}* _{t−1}*−

*f*

_{t}^{}θε

*t*

*ε*

_{t−1}*.*

2.18

Once→−

*θ*^{m1}and the gradient at→−

*θ*^{m1}have been calculated, a new estimation*A*^{m1}
is found from

*A*^{m1}*A*^{m}−*A*^{m} Δg^{m1} Δg^{m1}*T*

*A*^{m}
Δg^{m1}*T*

*A*^{m} Δg^{m1} −

Δ→−
*θ*^{m1}

Δ→−

*θ*^{m1}
_{T}

Δg^{m1}*T*
Δ→−

*θ*^{m1}

*,* 2.19

where

Δ→−

*θ*^{m1}→−

*θ*^{m1}−→−

*θ*^{m}*,* Δg^{m1}*g*
→−

*θ*^{m1}

−*g*
→−

*θ*^{m}

*.* 2.20

It is well known that least squares estimators in ordinary linear regression model
are very good estimators, so a recursive algorithms procedure is to start the iteration with
*β*^{0}*, σ*^{02} which are least squares estimators of*β* and *σ*^{2}, respectively. Take *θ*^{0} such that
*f** _{t}*θ

^{0}0. Iterations are stopped if some termination criterion is reached, for example, if

→−

*θ*^{m1}−→−
*θ*^{m}

→−
*θ*^{m}

*< δ,* 2.21

for some prechosen small number*δ >*0.

Up to this point, we obtain the values of QML estimators when the function*f**t*θ
*ft, θ*is known. However, the function*f** _{t}*θis never the case in practice; we have to estimate
it. By2.12and1.2, we obtain

*f*
*t,θ*_{n}

*ε*_{t}

*ε*_{t−1}*,* *t*2,3, . . . , n. 2.22

Based on the dataset{*ft,* *θ** _{n}*, t2,3, . . . , n}, we may obtain the estimation function

*f*t,

*θ*

*of*

_{n}*ft, θ*by some smoothing methods see Simonﬀ 43 , Fan and Yao 33 , Green and Silverman44 , Fan and Gijbels45 , etc.

To obtain our results, the following conditions are suﬃcient.

A1*X*_{n}_{n}

*t2**x*_{t}*x*^{T}* _{t}* is positive definite for suﬃciently large

*n*and

*n→ ∞*lim max

1≤t≤n*x*^{T}_{t}*X*_{n}^{−1}*x**t*0, 2.23

lim sup

*n*→ ∞ |λ|_{max}

*X*_{n}^{−1/2}*Z**n**X*_{n}^{−T/2}

*<*1, 2.24

where*Z**n* 1/2_{n}

*t2*x*t**x*^{T}_{t−1}*x*_{t−1}*x*^{T}* _{t}*and|λ|

_{max}·denotes the maximum in absolute value of the eigenvalues of a symmetric matrix.

A2There is a constant*α >*0 such that

*t*
*j1*

_{j−1}

*i0*

*f*_{t−i}^{2} θ

≤*α* 2.25

for any*t*∈ {1,2, . . . , n}and*θ*∈Θ.

A3The derivatives*f*_{t}^{}θ *df** _{t}*θ/dθ, f

_{t}^{}θ

*df*

_{t}^{}θ/dθexist and are bounded for any

*t*and

*θ*∈Θ.

*Remark 2.2. Maller* 12 applied the condition A1, and Kwoun and Yajima15 used the
conditionsA2andA3. Thus our conditions are general.A1delineates the class of*x** _{t}*for
which our results hold in the sense required. It is further discussed by Maller in12 . Kwoun
and Yajima15 call{e

*t*}stable if Vare

*t*is bounded. ThusA2implies that{e

*t*}is stable.

However,{e*t*}is not stationary. In fact, by2.3, we obtain that

Cove*t**, e*_{tk}*σ*_{0}^{2}
_{k−1}

*i0*

*f** _{tk−i}*θ0

*f*

*θ0*

_{t}

^{k}*i0*

*f** _{tk−i}*θ0 · · ·

^{t−2}*l0*

*f** _{t−l}*θ0

^{tk−2}*i0*

*f** _{tk−i}*θ0

*,*
2.26

which is dependent of*t.*

For ease of exposition, we will introduce the following notations which will be used later in the paper.

Defined1-vector*ϕ* β, θ, and

*S**n* *ϕ*

*σ*^{2}*∂Ψ**n*

*∂ϕ* *σ*^{2}
*∂Ψ**n*

*∂β* *,∂Ψ**n*

*∂θ*

*,* *F**n* *ϕ*

−σ^{2} *∂*^{2}Ψ*n*

*∂ϕ∂ϕ*^{T}*.* 2.27

By2.7and2.8, we get

F_{n}*ϕ*

⎛

⎜⎜

⎜⎝

*X** _{n}*θ

^{n}*t2*

*f*_{t}^{}θε*t−1**x*_{t}*f*_{t}^{}θε*t**x** _{t−1}*−2f

*θf*

_{t}

_{t}^{}θx

*t−1*

*ε*

_{t−1}∗ ^{n}

*t2*

*f*_{t}^{}^{2}θ *f**t*θf_{t}^{}θ

*ε*_{t−1}^{2} −*f*_{t}^{}θε*t**ε*_{t−1}

⎞

⎟⎟

⎟⎠*,* 2.28

where*X** _{n}*θ −σ

^{2}∂

^{2}Ψ

*n*

*/∂β∂β*

*and the∗indicates that the element is filled in by symmetry.*

^{T}Thus,

*D**n**E* *F**n* *ϕ*0

⎛

⎜⎜

⎝

*X**n*θ0 0

∗ ^{n}

*t2*

*f*_{t}^{}^{2}θ0 *f**t*θ0f_{t}^{}θ0

*Ee*_{t−1}^{2} −*f*_{t}^{}θ0Ee*t**e*_{t−1}

⎞

⎟⎟

⎠

⎛

⎜⎜

⎝

*X**n*θ0 0

∗ ^{n}

*t2*

*f*_{t}^{}^{2}θ0Ee_{t−1}^{2}

⎞

⎟⎟

⎠

*X**n*θ0 0

∗ Δ*n*θ0*, σ*0

*,*

2.29

where

Δ*n*θ0*, σ*0

*n*
*t2*

*f*_{t}^{}^{2}θ0Ee^{2}_{t−1}*σ*_{0}^{2}
*n*

*t2*

*f*_{t}^{}^{2}θ0^{t−2}

*j0*

_{j−1}

*i0*

*f*_{t−i}^{2} θ

*On.* 2.30

**3. Statement of Main Results**

**Theorem 3.1. Suppose that conditions (A1)–(A3) hold. Then there is a sequence**A* _{n}* ↓

*0 such that,*

*for eachA >0, asn*→ ∞, the probability

*P*

*there are estimatorsϕ*_{n}*,σ*_{n}^{2} *withS*_{n}*ϕ*_{n}

0, *and*
*ϕ*_{n}*,σ*_{n}^{2}

∈*N*_{n}^{}A

−→1. 3.1

*Furthermore,*

*ϕ**n**,σ*_{n}^{2}

−→*p*

*ϕ*0*, σ*_{0}^{2}

*,* *n*−→ ∞, 3.2

*where, for eachn*1,2, . . . , A >*0 andA** _{n}*∈0, σ

_{0}

^{2}; define neighborhoods

*N*

*n*A

*ϕ*∈*R** ^{d1}*:

*ϕ*−

*ϕ*0

_{T}

*D**n* *ϕ*−*ϕ*0

≤*A*^{2}
*,*
*N*_{n}^{}A *N**n*A∩

*σ*^{2}∈

*σ*_{0}^{2}−*A**n**, σ*^{2}_{0}*A**n*

*.*

3.3

* Theorem 3.2. Suppose that conditions (A1)–(A3) hold. Then*
1

*σ*_{n}*F*_{n}^{T/2}*ϕ*_{n}*ϕ** _{n}*−

*ϕ*

_{0}

−→*D**N0, I**d1*, *n*−→ ∞. 3.4

*Remark 3.3. Forθ*∈*R*^{m}*, m*∈*N, our results still hold.*

In the following, we will investigate some special cases in the model 1.1-1.2.

Although the following results are directly obtained from Theorems3.1and3.2, we discuss these results in order to compare with the corresponding results.

**Corollary 3.4. Let**f*t*θ *θ. If condition (A1) holds, then, for*|θ|*/1,*3.1,3.2, and3.4*hold.*

*Remark 3.5. These results are the same as the corresponding results of Maller*12 .
**Corollary 3.6. If**β0 andf* _{t}*θ

*θ, then, for*|

*θ*|

*/1,*

_{n}

*t2**ε*^{2}_{t−1}*σ*_{n}

*θ** _{n}*−

*θ*

_{0}

−→*D**N0,*1, *n*−→ ∞, 3.5

*where*

*σ*^{2}* _{n}* 1

*n*−1
*n*

*t2*

*ε**t*−*θ**n**ε** _{t−1}*2

*,* *θ**n*
_{n}

*t2*ε*t**ε*_{t−1}_{n}

*t2**ε*^{2}_{t−1}*.* 3.6

*Remark 3.7. These estimators are the same as the least squares estimators*see White16 .
For|θ|*>*1,{ε*t*}are explosive processes. In the case, the corollary is the same as the results of
White17 . While|θ|*<*1, notice that*σ*_{n}^{2}→*p**σ*_{0}^{2}and1/n−1_{n}

*t2**ε*^{2}* _{t−1}*→

*p*

*Eε*

_{t}^{2}

*σ*

_{0}

^{2}

*/1*−

*θ*

^{2}

_{0}, and byCorollary 3.6we obtain

√*n*

*θ** _{n}*−

*θ*

_{0}

−→*D**N*

0,1−*θ*_{0}^{2}

*.* 3.7

The result was discussed by many authors, such as Fujikoshi and Ochi46 and Brockwell and Davis19 .

**Corollary 3.8. Let**β0. If conditions (A2) and (A3) hold, then*F*^{1/2}_{n}

*θ**n*

*σ**n*

*θ**n*−*θ*0

−→*D**N0,*1, *n*−→ ∞, 3.8

*where*

*F*_{n}*θ*_{n}

^{n}

*t2*

*f*_{t}^{}^{2}

*θ*_{n}*f*_{t}

*θ*_{n}*f*_{t}^{}

*θ*_{n}

*ε*^{2}* _{t−1}*−

*f*

_{t}^{}

*θ*

_{n}*ε*_{t}*ε*_{t−1}*,*

*σ*_{n}^{2} 1

*n*−1
*n*

*t2*

*ε** _{t}*−

*f*

_{t}*θ*_{n}*ε*_{t−1}_{2}

*.*

3.9

**Corollary 3.9. Let**f* _{t}*θ

*a*

_{t}*. If condition (A1) holds, then*

1
*σ**n*

_{n}

*t2*

x*t*−*a*_{t}*x** _{t−1}*x

*t*−

*a*

_{t}*x*

_{t−1}

^{T}

_{T/2}*β** _{n}*−

*β*

_{0}

−→*D**N0, I**d*, *n*−→ ∞. 3.10

*Remark 3.10. Let* *a**t* 0. Note that _{n}

*t2**x**t**x*_{t}^{T}*O*√*n* and *σ*_{n}^{2}→*p**σ*_{0}^{2}; we easily obtain
asymptotic normality of the quasi-maximum likelihood or least squares estimator in
ordinary linear regression models from the corollary.

**4. Some Lemmas**

To prove Theorems3.1and3.2, we first introduce the following lemmas.

**Lemma 4.1. The matrix***D**n* *is positive definite for large enough* *n* *with* *ES**n*ϕ0 *0 and*
VarS*n*ϕ0 *σ*_{0}^{2}*D**n**.*

*Proof. It is easy to show that the matrixD** _{n}*is positive definite for large enough

*n. By*2.8, we have

*σ*_{0}^{2}*E*
*∂Ψ**n*

*∂β* |_{ββ}_{0}

^{n}

*t2*

*E* *e** _{t}*−

*f*

*θ0e*

_{t}

_{t−1}*x*

*−*

_{t}*f*

*θ0x*

_{t}

_{t−1}^{n}

*t2*

*x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}*Eη** _{t}*0.

4.1

Note that*e** _{t−1}*and

*η*

*t*are independent of each other; thus by2.7and

*Eη*

*t*0, we have

*σ*_{0}^{2}*E*
*∂Ψ**n*

*∂θ* |_{θθ}_{0}

^{n}

*t2*

*E*

*e** _{t}*−

*f*

*θ0e*

_{t}*t−1*

*f*_{t}^{}θ0e*t−1*

^{n}

*t2*

*f*_{t}^{}θ0E *η*_{t}*e** _{t−1}*
0.

4.2

Hence, from4.1and4.2,

*E* *S**n* *ϕ*0

*σ*_{0}^{2}*E*
*∂Ψ**n*

*∂β* |_{ββ}_{0}*,∂Ψ**n*

*∂θ* |_{θθ}_{0}

0. 4.3

By2.8and2.17, we have

Var

*σ*_{0}^{2}*∂Ψ**n*

*∂β* |_{ββ}_{0}

Var
_{n}

*t2*

*e**t*−*f**t*θ0e*t−1* *x**t*−*f**t*θ0x*t−1*

Var
_{n}

*t2*

*η**t* *x**t*−*f**t*θ0x*t−1*

*σ*_{0}^{2}*X**n*θ0.

4.4

Note that{f_{t}^{}θ0η*t**e*_{t−1}*, H**t*}is a martingale diﬀerence sequence with
Var *f*_{t}^{}θ0η*t**e*_{t−1}

*f*_{t}^{}^{2}θ0Eη^{2}_{t}*Ee*^{2}_{t−1}*σ*_{0}^{2}*f*_{t}^{}^{2}θ0Ee^{2}_{t−1}*,* 4.5

so

Var

*σ*_{0}^{2}*∂Ψ**n*

*∂θ* |_{θθ}_{0}

Var
_{n}

*t2*

*η*_{t}*f*_{t}^{}θ0e*t−1*

^{n}

*t2*

*f*_{t}^{}^{2}θ0Ee^{2}_{t−1}*σ*_{0}^{2}Δ*n*θ0*, σ*_{0}.

4.6

By2.7and2.8and noting that*e** _{t−1}*and

*η*

*t*are independent of each other, we have

Cov

*σ*_{0}^{2}*∂Ψ**n*

*∂β* |_{ββ}_{0}*, σ*_{0}^{2}*∂Ψ**n*

*∂θ* |_{θθ}_{0}

*E*

*σ*_{0}^{2}*∂Ψ**n*

*∂β* |_{ββ}_{0}*, σ*_{0}^{2}*∂Ψ**n*

*∂θ* |_{θθ}_{0}

*E*
_{n}

*t2*

*η*_{t}^{2} *x**t*−*f**t*θ0x*t−1*

*f*_{t}^{}θ0e*t−1*

*E*
_{n}

*t3*

*η*_{t}*x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}

^{t−1}*s2*

*η*_{s}*f*_{s}^{}θ0e_{s−1}

*E*
_{n}

*s3*

*η**s**f*_{s}^{}θ0e*s−1*

*s−1*
*t2*

*η**t* *x**t*−*f**t*θ0x*t−1*
0.

4.7

From4.4–4.7, it follows that VarS*n*ϕ0 *σ*_{0}^{2}*D** _{n}*.

* Lemma 4.2. If condition (A1) holds, then, for anyθ* ∈ Θ, the matrix

*X*

*n*θ

*is positive definite for*

*large enoughn, and*

*n*lim→ ∞max

1≤t≤n*x*^{T}_{t}*X*^{−1}* _{n}* θx

*t*0. 4.8

*Proof. Letλ*_{1}and*λ** _{d}*be the smallest and largest roots of|Z

*n*−

*λX*

*|0. Then from the study of Rao in47, Ex 22.1 ,*

_{n}*λ*1 ≤ *u*^{T}*Z**n**u*

*u*^{T}*X*_{n}*u* ≤*λ**d* 4.9

for unit vectors *u. Thus by* 2.24, there are some *δ* ∈ max{0,1 − 1 min_{2≤t≤n}|
*f*_{t}^{2}θ|/max2≤t≤n|f*t*θ|},1and*n*0δsuch that*n*≥*N*0implies that

u^{T}*Z*_{n}*u*≤1−*δu*^{T}*X*_{n}*u.* 4.10

By4.10, we have

*u*^{T}*X*_{n}*u*^{n}

*t2*

*u*^{T}*x** _{t}*−

*f*

*θx*

_{t}

_{t−1}^{2}

^{n}

*t2*

*u*^{T}*x** _{t}*2

*f*_{t}^{2}θ

*u*^{T}*x** _{t−1}*2

−*f** _{t}*θu

^{T}*x*

_{t−1}*x*

^{T}

_{t}*u*−

*f*

*θu*

_{t}

^{T}*x*

_{t}*x*

^{T}

_{t−1}*u*

≥^{n}

*t2*

*u*^{T}*x** _{t}*2

min

2≤t≤n

*f*_{t}^{2}θ^{n}

*t2*

*u*^{T}*x** _{t−1}*2

−max

2≤t≤n*f** _{t}*θ

*u*

^{T}*Z*

_{n}*u*

≥*u*^{T}*X**n**u*min

2≤t≤n

f_{t}^{2}θu^{T}*X**n**u*−max

2≤t≤n*f**t*θ*u*^{T}*Z**n**u*

≥

1min

2≤t≤n

f_{t}^{2}θ−max

2≤t≤n*f**t*θ1−*δ*

*u*^{T}*X**n**u*
*Cθ, δu*^{T}*X**n**u.*

4.11

By the study of Rao in47, page 60 and2.23, we have

*u*^{T}*x**t*

2

*u*^{T}*X*_{n}*u* −→0. 4.12

From4.12and*Cθ, δ>*0,

*x*^{T}_{t}*X*_{n}^{−1}θ sup

*u*

*u*^{T}*x**t*

2

*u*^{T}*X**n*θu

≤sup

*u*

*u*^{T}*x**t*

2

*Cθ, δu*^{T}*X**n**u*

−→0. 4.13

**Lemma 4.3**see48 . Let*W*_{n}*be a symmetric random matrix with eigenvaluesλ** _{j}*n,1 ≤

*j*≤

*d.*

*Then*

*W**n*−→*p**I*⇐⇒*λ**j*n−→*p*1, *n*−→ ∞. 4.14

**Lemma 4.4. For each**A >0,

sup

*ϕ∈N**n*A

*D*^{−1/2}_{n}*F*_{n}*ϕ*

*D*_{n}^{−T/2}−Φ*n*−→*p*0, *n*−→ ∞, 4.15

*and also*

Φ*n*−→*D*Φ, 4.16

*c→*lim0lim sup

*A*→ ∞ lim sup

*n*→ ∞ *P*

*ϕ∈N*inf*n*A*λ*min

*D*^{−1/2}_{n}*F**n* *ϕ*
*D*_{n}^{−T/2}

≤*c* 0, 4.17

*where*

Φ*n*

⎛

⎜⎝

*I** _{d}* 0

0
_{n}

*t2**f*_{t}^{}^{2}θ0e^{2}* _{t−1}*
Δ

*n*θ0

*, σ*

_{0}

⎞

⎟⎠*,* Φ *I*_{d1}*.* 4.18

*Proof. LetX** _{n}*θ0

*X*

^{1/2}

*θ0X*

_{n}*n*

*θ0be a square root decomposition of*

^{T/2}*X*

*θ0. Then*

_{n}*D*_{n}

*X*_{n}^{1/2}θ0 0

∗ !

Δ*n*θ0*, σ*_{0}

*X*_{n}* ^{T/2}*θ0 0

∗ !

Δ*n*θ0*, σ*_{0}

*D*^{1/2}_{n}*D*^{T/2}_{n}*.* 4.19

Let*ϕ*∈*N**n*A. Then

*ϕ*−*ϕ*0

_{T}

*D**n* *ϕ*−*ϕ*0

*β*−*β*0

_{T}

*X**n*θ0 *β*−*β*0

*θ*−*θ*0^{2}Δ*n*θ0*, σ*0≤*A*^{2}*.* 4.20

From2.28,2.29, and4.18,

*D*_{n}^{−1/2}*F**n* *ϕ*

*D*^{−T/2}* _{n}* −Φ

*n*

⎛

⎜⎜

⎜⎜

⎜⎝

*X*^{−1/2}_{n}*θ*_{0}*X*_{n}*θX*_{n}^{−T/2}*θ*_{0}−*I*_{d}*X*_{n}^{−1/2}θ0_{n}

*t2* *f*_{t}^{}θε*t−1**x**t**f*_{t}^{}θε*t**x**t−1*−2f*t*θf_{t}^{}θε*t−1**x**t−1*

!Δ*n*θ0*, σ*0

∗

*n*
*t2*

*f*_{t}^{}^{2}θ *f**t*θf_{t}^{}θ

*ε*^{2}* _{t−1}*−

*f*

_{t}^{}θε

*t*

*ε*

*t−1*

−*n*

*t2* *f*_{t}^{}^{2}θ0e^{2}_{t−1}

!Δ*n**θ*0*, σ*0

⎞

⎟⎟

⎟⎟

⎟⎠*.*
4.21

Let

*N*^{β}* _{n}*A

*β*: *β*−*β*0

_{T}

*X*^{1/2}* _{n}* θ0

^{2}≤

*A*

^{2}

*,*4.22

*N*

_{n}*A*

^{θ}

*θ*:|θ−*θ*0| ≤ *A*

!Δ*n*θ0*, σ*_{0}

*.* 4.23

In the first step, we will show that, for each*A >*0,

sup

*θ∈N*^{θ}*n*A

X^{−1/2}* _{n}* θ0X

*n*θX

_{n}^{−T/2}θ0−

*I*

*d*−→0,

*n*−→ ∞. 4.24

In fact, note that

*X*^{−1/2}* _{n}* θ0X

*n*θX

_{n}^{−T/2}θ0−

*I*

_{d}*X*

_{n}^{−1/2}θ0X

*n*θ−

*X*

*θ0X*

_{n}

_{n}^{−T/2}θ0

*X*_{n}^{−1/2}θ0T1*T*_{2}−*T*_{3}X_{n}^{−T/2}θ0, 4.25

where

*T*_{1}^{n}

*t2*

*f** _{t}*θ0−

*f*

*θ*

_{t}*x*_{t−1}*x** _{t}*−

*f*

*θ0x*

_{t}*t−1*

*T*

*,*

*T*_{2}^{n}

*t2*

*x** _{t}*−

*f*

*θ0x*

_{t}*t−1*

*x*

^{T}

_{t−1}*,*

*T*_{3}^{n}

*t2*

*f** _{t}*θ0−

*f*

*θ*

_{t}_{2}

*x*_{t−1}*x*^{T}_{t−1}*.*

4.26

Let*u, v* ∈ *R*^{d}*,* |u| |v| 1, and let*u*^{T}_{n}*u*^{T}*X*_{n}^{−1/2}θ0, v_{n}^{T}*X*_{n}^{−T/2}θ0v. By Cauchy-
Schwartz inequality,Lemma 4.2, conditionA3, and noting that*θ*∈*N*_{n}* ^{θ}*A, we have that

*u*^{T}_{n}*T*_{1}*v*_{n}

*n*
*t2*

*f** _{t}*θ0−

*f*

*θ*

_{t}*u*^{T}_{n}*x*_{t−1}*x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}*T*

*v*_{n}

≤max

2≤t≤n*f** _{t}*θ0−

*f*

*θ*

_{t}*n*
*t2*

*u*^{T}_{n}*x*_{t−1}*x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}*T*

*v*_{n}

≤max

2≤t≤n*f**t*θ0−*f**t*θ_{n}

*t2*

*u*^{T}_{n}*x*_{t−1}*x*^{T}_{t−1}*u**n*

1/2

·
_{n}

*t2*

*v*_{n}^{T}*x**t*−*f** _{t}*θ0x

_{t−1}*x*

*t*−

*f*

*θ0x*

_{t}

_{t−1}

_{T}*v*

_{n}_{1/2}

≤max

2≤t≤n*f** _{t}*θ0−

*f*

*θ*

_{t}

_{n}*t2*

*u*^{T}_{n}*x*_{t}*x*^{T}_{t}*u** _{n}*
1/2

≤max

2≤t≤n

f_{t}^{}

*θ*|θ0−*θ| ·*√
*n*max

1≤t≤n

*x*_{t}^{T}*X*^{−1}* _{n}* θ0x

*t*

≤*C*

"

*n*

Δ*n*θ0*, σ*0*o1*−→0.

4.27

Here*θaθ* 1−*aθ*0for some 0≤*a*≤1. Similar to the proof of*T*_{1}, we easily obtain that
*u*^{T}_{n}*T*_{2}*v** _{n}*−→0. 4.28

By Cauchy-Schwartz inequality,Lemma 4.2, conditionA3, and noting that*N*_{n}* ^{θ}*A, we have
that

u^{T}_{n}*T*3*v**n*
*u*^{T}_{n}

*n*
*t2*

*f**t*θ0−*f**t*θ_{2}

*x*_{t−1}*x*_{t−1}^{T}*v**n*

≤max

2≤t≤n*f** _{t}*θ0−

*f*

*θ*

_{t}^{2}

_{n}*t2*

*u*^{T}_{n}*x*_{t}*x*^{T}_{t}*u*_{n}*n*

*t2*

*v*^{T}_{n}*x*_{t}*x*^{T}_{t}*v*_{n}_{1/2}

≤*n*max

2≤t≤n

*f*_{t}^{}

*θ*^{2}|θ0−*θ|*^{2}max

1≤t≤n

*x*^{T}_{t}*X*_{n}^{−1}θ0x*t*

≤ *nA*^{2}

Δ*n*θ0*, σ*_{0}*o1*−→0.

4.29

Hence,4.24follows from4.25–4.29.

In the second step, we will show that
*X*_{n}^{−1/2}θ0_{n}

*t2* *f*_{t}^{}θε_{t−1}*x*_{t}*f*_{t}^{}θε*t**x** _{t−1}*−2f

*θf*

_{t}

_{t}^{}θε

_{t−1}*x*

_{t−1}!Δ*n*θ0*, σ*0 −→*p*0. 4.30

Note that

*ε**t**y**t*−*x*_{t}^{T}*βx*_{t}^{T}*β*0−*β*
*e**t**,*
*ε** _{t}*−

*f*

*θ0ε*

_{t}

_{t−1}*x*

*−*

_{t}*f*

*θ0x*

_{t}

_{t−1}*T*

*β*_{0}−*β*
*η*_{t}*.*

4.31

Consider

*J* ^{n}

*t2*

*f*_{t}^{}θε_{t−1}*x*_{t}*f*_{t}^{}θε*t**x** _{t−1}*−2f

*θf*

_{t}

_{t}^{}θε

_{t−1}*x*

_{t−1}^{n}

*t2*

*ε*_{t−1}*f*_{t}^{}θ *x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}*f*_{t}^{}θ *ε** _{t}*−

*f*

*θ0ε*

_{t}

_{t−1}*x*

*2f*

_{t−1}*θ*

_{t}*f*

*θ0−*

_{t}*f*

*θ*

_{t}*ε*_{t−1}*x*_{t−1}*T*_{1}*T*_{2}*T*_{3}*T*_{4}2T_{5}2T_{6}*,*

4.32

where

*T*_{1}^{n}

*t2*

*x*^{T}_{t−1}*f*_{t}^{}θ *β*_{0}−*β* *x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}*,* *T*_{2}^{n}

*t2*

*f*_{t}^{}θe_{t−1}*x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}*,*

*T*_{3}^{n}

*t2*

*f*_{t}^{}θ *x** _{t}*−

*f*

*θ0x*

_{t}

_{t−1}*T*

*β*_{0}−*β*

*x*_{t−1}*,* *T*_{4}^{n}

*t2*

*f*_{t}^{}θη*t**x*_{t−1}*,*

*T*_{5}^{n}

*t2*

*f*_{t}^{}θ *f** _{t}*θ0−

*f*

*θ*

_{t}*x*^{T}_{t−1}*β*_{0}−*β*

*x*_{t−1}*,* *T*_{6}^{n}

*t2*

*f*_{t}^{}θ *f** _{t}*θ0−

*f*

*θ*

_{t}*e*_{t−1}*x*_{t−1}*.*
4.33

For*β*∈*N*_{n}* ^{β}*Aand each

*A >*0, we have

*β*

_{0}−

*β*

_{T}*x*_{t}^{2} *β*_{0}−*β*_{T}

*X*^{1/2}* _{n}* θ0X

_{n}^{−1/2}θ0x

*t*

*x*

_{t}

^{T}*X*

^{−T/2}

*θ0X*

_{n}

_{n}*θ0*

^{T/2}*β*

_{0}−

*β*

≤max

1≤t≤n

*x*^{T}_{t}*X*_{n}^{−1}θ0x*t* *β*_{0}−*β**T*

*X** _{n}*θ0

*β*

_{0}−

*β*

≤*A*^{2}max

1≤t≤n

*x*^{T}_{t}*X*_{n}^{−1}θ0x*t*

*.*

4.34