Shrinkage GMM estimation in conditional moment restriction
models,
Supplementary material: Technical appendix
Ryo Okui∗ Kyoto University November 16, 2009
Abstract
This paper is a supplementary Material for “Shrinkage GMM estimation in conditional mo-ment restriction” and collects the proofs of the theorems in the paper.
∗Insitute of Economic Research, Kyoto University, Yoshida-Hommachi, Sakyo, Kyoto, Kyoto, 606-8501, Japan,
1
Proofs
This appendix contains the proofs of the theorems. λmax(A) denotes the maximum eigenvalue of a matrix A. “CS” and “ME” are the abbreviations of the Cauchy-Schwartz inequality and the maximum eigenvalue inequality, respectively.1
1.1 The proof of Theorem 1
Define R(β) ≡ E(E(ρi(β)|xi)2/σi2), ˆR(β) ≡ gˆ(β)′Υ( ˜ˆ β)− 1
ˆ
g(β), ˆRs(β) ≡ gˆ1(β)′Υ11( ˜ˆ β)−1gˆ1(β) +
s˜g2(β)′Υ22( ˜˜ β)−1˜g2(β), and ˆβ ≡argmaxβRˆ(β). Note that ˆβs = argminβRˆs(β). In the view of the proof of Theorem 5.3 in Donald, Imbens and Newey (2003), we have:
sup β |
ˆ
R(β)−R(β)| →p 0, Rˆ( ˆβ)−R(β0)→p0,
asR(β) is uniquely minimized atβ0 ∈Θ, Θ is compact by Assumption 1, andR(β) is continuous by Assumptions 1 and 2. Since ˆRs(β)−Rˆ(β) =Op(K2/N)→p 0 and ˆRs( ˆβ)−Rˆ( ˆβ) =Op(K2/N)→p 0 by 1−s=Op(K2/N), we get
sup β |
ˆ
Rs(β)−R(β)| →p 0, Rˆs( ˆβs)−R(β0)→p 0.
Therefore, ˆβ→pβ0 by Lemma A.1 in Donald, Imbens and Newey (2003). Next, we show asymptotic normality. By the mean value theorem,
∂2ˆ Rs( ¯β)
∂β∂β′ ( ˆβ−β0) +
∂Rˆs(β0)
∂β = 0, where ¯β is a mean value. ∂2ˆ
Rs( ¯β)/∂β∂β′ →p E(σi−2did′i) by Lemmas 1 and 2. √
N ∂Rˆs(β0)/∂β→d N(0, E(σi−2did′i)). Therefore,
√
N( ˆβs−β0)→dN(0, E(σi−2did′i)− 1
).
1.2 The proof of Theorem 2
In various steps of our proofs, we use the approach developed by Donald, Imbens and Newey (2009). The objective function of the shrinkage GMM can be written as:
ˆ
g(β)′Ψˆ−1
( ˜β)ˆg(β),
where
ˆ
Ψ−1≡Fˆ
((
I 0
0 s−1/2I
)
ˆ
F′Υ( ˜ˆ β) ˆF
(
I 0
0 s−1/2I
))−1
ˆ F′,
and
ˆ F =
(
I −Υ11( ˜ˆ β)−1Υ12( ˜ˆ β)
0 I
)
.
Tedious but straightforward algebra shows that
ˆ Ψ =
(ˆ
Υ11 Υ12ˆ
ˆ
Υ21 s−1
( ˆΥ22−Υ21ˆ Υˆ−111Υ12) + ˆˆ Υ21Υˆ− 1 11Υ12ˆ
)
.
1
The shrinkage GMM estimator together with the auxiliary vector parameter λsolves 1 N ∑ i ∂ ∂βρi(β)q
′
iλ= 0, 1 N
∑
i
qiρi(β) + ˆΨλ= 0.
For simplicity, assume the true value of parameter is 0 without loss of generality. Also assume the dimension ofβ is 1. The multidimensional case is analyzed in a very similar fashion.
Let θ= (β, λ′)′ and ˆθ is an estimator ofθ obtained by solving the following equation
ˆ
m(θ) = 1 N
∑
i
mi(θ) = 0.
Then, ˆθ has an expansion:
ˆ
θ = −M−1m−M−1( ˆM−M)ˆθ−1 2
∑
j ˆ
θjM−1Ajθˆ+R,
R = −1
2
∑
j ˆ
θjM−1(Aj−Aˆj)ˆθ− 1 6 ∑ j ∑ k ˆ
θjθˆkM−1Bjk∗ θ,ˆ
where
m≡mˆ(0), Mˆ ≡ 1 N
∑
i
∂ ∂θmi(0)
′ = 1
N
∑
i
Mi(0), Aˆj ≡ 1 N ∑ i ∂ ∂θj
Mi(0),
M and Aj are population values (conditional on xis) of ˆM and ˆAj, respectively, and the rows of
B∗
jk are the second derivatives of the corresponding rows of Mi evaluated at θ∗, which is between ˆ
θ and 0. Note that each row ofB∗
jk has a different mean valueθ∗. Then, for the SGMM estimator,
m= ( 0 ¯ g )
, Mˆ =
(
0 Γ¯′
0 ¯
Γ0 Ψ( ˜ˆ β)
)
, M =
(
0 Γ′
0 Γ0 Ψ
)
,
ˆ Aj =
(
0 Γ¯′
1 ¯ Γ1 0
)
ifj= 1,
0 otherwise,
Aj =
(
0 Γ′
1 Γ1 0
)
ifj= 1,
0 otherwise,
B∗jk =
(
0 Γ¯∗′
2 ¯ Γ∗
2 0
)
ifj= 1, k= 1
0 otherwise, where
¯ g≡ 1
N
∑
i
qiρi(0), Υ≡
1 N
∑
i
σi2qiqi′, Γ0¯ ≡ 1 N ∑ i qi ∂ ∂βρi(0),
Γ1≡ 1 N
∑
i
qiE
(
∂2
∂β2ρi(0)|xi
)
, Γ1¯ ≡ 1 N
∑
i
qi
∂2
∂β2ρi(0), Γ¯
∗ 2 ≡ 1 N ∑ i qi ∂3 ∂β3ρi(β
∗).
Noting that
M−1 =
(
−Ω−1
s Ω−s1Γ′0Ψ−1 Ψ−1
Γ0Ω−1
s Σ
)
where Ωs≡Γ′0Ψ− 1
Γ0 and Σ≡Ψ−1 −Ψ−1
Γ0Ω−1 s Γ′0Ψ−
1
, we have the following expansion. ˆ
β = −Ω−s1Γ′0Ψ− 1
¯
g+Tβ +Rβ1,
Tβ ≡ Ω−s1(¯Γ0−Γ0)′λˆ−Ω−s1Γ′0Ψ−1( ˆΨ( ˜β)−Ψ)ˆλ−Ωs−1Γ′0Ψ−1(¯Γ0−Γ0) ˆβ +1
2Ω
−1
s Γ′1βˆλˆ− 1 2Ω
−1 s Γ′0Ψ−
1 Γ1βˆ2,
and
ˆ
λ = −Σ¯g+Tλ+Rλ1,
Tλ ≡ −Ψ−1Γ0Ω−s1(¯Γ0−Γ0)′λˆ−Σ(¯Γ0−Γ0) ˆβ−Σ( ˆΨ( ˜β)−Ψ)ˆλ,
where Rβ1 = op(K 2
/N2/3
) by Lemma 4 and Rλ
1 = Op(1/N) by Lemma 6. Also, we define R β 2 ≡ Tβ+Rβ1 so that ˆβ=−Ω−
1
s Γ′0Ψ−1g¯+Rβ2 andR β
2 =Op(K/N) by Lemmas 3 and 4 andRλ2 ≡Tλ+Rλ1 so that ˆλ=−Σ¯g+Rλ2 and Rλ2 =Op(ζ(K)
√
K/N) by Lemmas 5 and 6. Furthermore, Lemma 7 gives
ˆ
Ψ( ˜β)−Ψ = ˆΥv1+ 2 ˜βΥρη+RΨ,
where ˆ Υv1=
1 N
∑
i
(ρ2i −σi)qiqi′, Υρη = 1 N
∑
i
σρη,iqiqi′, and ||R Ψ
||=Op(ζ(K) √
K/N +K/N).
We rewrite ˆλby using ˆβ =−Ω−1
s Γ′0Ψ−1¯g+Rβ2, ˆλ=−Σ¯g+Rλ2 and ˆΨ( ˜β)−Ψ = ˆΥv1+2 ˜βΥρη+RΨ so that
ˆ
λ = −Σ¯g+ Ψ−1Γ0Ω−s1(¯Γ0−Γ0)′Σ¯g+ Σ(¯Γ0−Γ0)Ω−s1Γ′0Ψ−1¯g +Σ ˆΥv1Σ¯g+ 2 ˜βΣΥρηΣ¯g+Rλ3,
whereRλ
3 =op(K 2
/N3/2
) by Lemma 8.
Using these results, we can write each term ofTβ as
Ω−s1(¯Γ0−Γ0)′λˆ = −Ω−s1(¯Γ0−Γ0)′Σ¯g+ Ω−s1(¯Γ0−Γ0)′Σ ˆΥv1Σ¯g+Rβ3, Ωs−1Γ′0Ψ−1( ˆΨ( ˜β)−Ψ)ˆλ = −Ωs−1Γ′0Ψ−1Υˆv1Σ¯g+ Ω−s1Γ′0Ψ−
1ˆ
Υv1Σ ˆΥv1Σ¯g −2 ˜βΩ−s1Γ′0Ψ−1ΥρηΣ¯g+Rβ4,
Ω−s1Γ′0Ψ−1(¯Γ0−Γ0) ˆβ = −Ω−s1Γ0Ψ′ −1(¯Γ0−Γ0)Ω−s1Γ′0Ψ−1g¯+Rβ5, Ω−s1Γ′1βˆλˆ = Ω−
1
s Γ′1Σ¯gΩ− 1 s Γ′0Ψ−
1 ¯ g+Rβ6, Ωs−1Γ′0Ψ−1Γ1βˆ2 = Ω−s1Γ′0Ψ−1Γ1(Ω−s1Γ′0Ψ−1¯g)2+Rβ7,
where Rβ3 =op(K2/N3/2) by Lemma 9, R4β =op(K2/N3/2) by Lemma 10, Rβ5 =op(K2/N3/2) by Lemma 11, Rβ6 =op(K
2 /N2/3
) by Lemma 12, andRβ7 =op(K 2
/N2/3
Summing up, we have the following expansion of β
√
Nβˆ = Ω−s1
h+
4
∑
j=1
Tjh+Zh
,
h = −√NΓ′0Ψ−1¯g, T1h = −
√
N(¯Γ0−Γ0)′Σ¯g, T2h = −
√
NΓ′0Ψ−1Υˆv1Σ¯g,
T3h = 2 √
Nβ˜Γ′0Ψ−1ΥρηΣ¯g+ √
NΓ′0Ψ−1(¯Γ0−Γ0)Ωs−1Γ′0Ψ−1¯g +1
2 √
NΓ′1Σ¯gΩ−s1Γ′0Ψ−1¯g−1 2
√
NΓ′0Ψ−1Γ1(Ω−s1Γ′0Ψ−1g¯)2, T4h =
√
N(¯Γ0−Γ0)′Σ ˆΥv1Σ¯g− √
NΓ′0Ψ−1Υˆv1Σ ˆΥv1Σ¯g,
Zh = √NΩs(Rβ1 +R β 3 −R
β 4 −R
β
5 + 1/2R β
6 −1/2R β 7), andh=Op(1),T1h=Op(K/
√
N) by Lemma 14,Th 2 =Op(
√
K/N) by Lemma 15,Th
3 =Op(1/ √
N) by Lemma 16, T4h =Op(ζ(K)K/
√
N) by Lemma 17 and Zh =op(K2/N). This means that
(√Nβˆ)2
= Ω−1 s hh′Ω−
1 s + Ω−
1 s
T1hT1h′+ 2 4
∑
j=1 Tjhh′
Ω
−1
s +op(γK,N).
E(Tjhh′|X) =op(γK,N) for j= 1, . . . ,4 by Lemma 18, 19, 20, and 21.
Now Ωs−1E(hh′)Ωs−1 = Ω∗−1+ Ω∗−1(Ω∗−Ω + (1−s)2Ω2)Ω∗−1+op(γK,N). To show this, first it is easy to show that
E(hh′|X) = Ω1+s2Ω2= Ω∗−2(Ω∗−Ωs) + Ω∗−Ω + (1−s)2Ω2. Second, a simple matrix algebra shows that
Ω−s1= (Ω∗−(Ω∗−Ωs))−1 = Ω∗−1−Ω∗−1(Ω∗−Ωs)Ω∗−1+op(γK,N), since||(Ω∗−Ω
s)||2 =op(γK,N). Then, we have
Ω−s1E(hh′|X)Ω−s1 = (Ω∗−1−Ω∗−1(Ω∗−Ωs)Ω∗−1+op(γK,N)) ×(Ω∗−2(Ω∗−Ωs) + Ω∗−Ω + (1−s)2Ω2) ×(Ω∗−1−Ω∗−1(Ω∗−Ωs)Ω∗−1+op(γK,N))
= Ω∗−1−2Ω∗−1(Ω∗−Ωs)Ω∗−1+ Ω∗−1(Ω∗−Ω + (1−s)2Ω2)Ω∗−1 +Ω∗−1(Ω∗−Ωs)Ω∗−1+ Ω∗−1(Ω∗−Ωs)Ω∗−1+op(γK,N)
= Ω∗−1+ Ω∗−1(Ω∗−Ω + (1−s)2Ω2)Ω∗−1+op(γK,N). Then, by Lemma 22, the decomposition (3) in the main paper holds with
S(s) = s 2
Π22
N + Ω
∗−1
(Ω∗−Ω + (1−s)2Ω2)Ω∗−1.
2
Lemmas
Lemma 1. ||ΓiΨ−1Γj||=Op(1) and ||ΓiΣΓj||=Op(1) for i, j= 0,1,2. Proof.
ΓiΨ−1Γj = ΓiΥ−1Γj+ (1−s)Γi,2Υ˜−221Γj,2.
||ΓiΥ−1Γj|| = Op(1) and ||Γi,2Υ−221Γj,2|| = Op(1) by Lemma A.3 in Donald, Imbens and Newey (2003). (1−s) =Op(1) by the assumption. A similar argument shows that||ΓiΣΓj||=Op(1).
Lemma 2. ||ΓiΨ−1(¯Γj−Γj)||=Op(1/√N)and ||ΓiΣ(¯Γj−Γj)||=Op(1/√N). Also, ||Γ′0Ψ−1g¯||=
Op(1/√N) and ||Γ′0Σ¯g||=Op(1/ √
N) for i, j= 0,1,2. Proof.
ΓiΨ−1(¯Γj−Γj) = ΓiΥ−1(¯Γj−Γj) + (1−s)Γi,2Υ˜22−1(¯Γj,2−Γj,2). ||ΓiΥ−1(¯Γj−Γj)||=Op(1/
√
N) and||Γi,2Υ˜−221(¯Γj,2−Γj,2)||=Op(1/ √
N) by Lemma A.4 in Donald, Imbens and Newey (2003). 1−s=Op(1) by the assumption. Others can be shown similarly.
Lemma 3. Tβ =Op(K/N).
Proof. Recall that Tβ has the form:
Tβ = Ω−s1(¯Γ0−Γ0)′λˆ−Ω−s1Γ′0Ψ−1( ˆΨ( ˜β)−Ψ)ˆλ−Ωs−1Γ′0Ψ−1(¯Γ0−Γ0) ˆβ +1
2Ω
−1
s Γ′1βˆλˆ− 1 2Ω
−1 s Γ′0Ψ−
1 Γ1βˆ2.
First, since ||(¯Γ0−Γ0)′λˆ|| = O
p(K/N) by CS, ||Ω−s1(¯Γ0 −Γ0)′ˆλ|| = Op(K/N) = Op(K/N). For the second term, using ||Γ′0Ψ−1
( ˆΨ( ˜β) −Ψ)|| = Op(
√
K/N) by Lemma A.4 in Donald, Imbens and Newey (2003) and Lemma 7, we get ||Ω−1
s Γ′0Ψ− 1
( ˆΨ( ˜β)−Ψ)ˆλ|| = Op(K/N) by CS. Next, observe that ||Γ′0Ψ−1
(¯Γ0 −Γ0)|| = Op(1/ √
N) by Lemma 2. Hence, ||Ω−1
s Γ′0Ψ−1(¯Γ0 −Γ0) ˆβ|| =
Op(1/ √
N)Op(1/ √
N) = Op(K/N). For the fourth term, ||Ω−s1Γ′1βˆλˆ|| = ||Ω− 1
s || · ||Γ′1λˆ|| · ||βˆ|| = Op(1/
√
N)Op(1/ √
N) =Op(K/N). For the last term,||Ω−s1Γ′0Ψ−1Γ1βˆ2|| =Op(1/N) =Op(K/N), as||Γ′
0Ψ− 1
Γ1||=Op(1) by Lemma 1.
Lemma 4. Rβ1 =op(K2/N3/2), where
Rβ1 = 1 2Ω
−1
s (¯Γ1−Γ1)′βˆλˆ− 1 2Ω
−1 s Γ′0Ψ−
1
(¯Γ1−Γ1) ˆβ2−1 6Ω
−1 s Γ′0Ψ−
1 Γ∗2βˆ
3 +1
6Ω
−1 s Γ∗′2βˆ
2ˆ λ.
Proof. First, since||(¯Γ1−Γ1)′ˆλ||=Op(K/N) by CS,||Ω−s1(¯Γ1−Γ1)′βˆλˆ||=Op(K/N)Op(1/√N) =
op(K2/N3/2). For the second term,||Ω−s1Γ′0Ψ− 1
(¯Γ1−Γ1) ˆβ2
||=Op(1/√N)Op(1/N) =op(K2/N3/2), where||Γ′0Ψ−1
(¯Γ1−Γ1)||=Op(1/ √
N) by Lemma 2. Next,||Γ′0Ψ−1 Γ∗
2||=Op(1) by Lemma 1, which implies that ||Ω−1
s Γ′0Ψ−1Γ∗2βˆ 3
|| =Op(1/N3/2) = op(K2/N3/2). For the last term,||Ω−s1Γ∗′2βˆ 2ˆ
λ||= ||Ω−1
s || · ||Γ∗′2ˆλ|| · ||βˆ 2
||=Op(1/√N)Op(1/N) =op(K2/N3/2).
Lemma 5. Tλ=Op(ζ(K)√K/N).
Proof. Recall that Tλ has the form:
Tλ =−Ψ−1Γ0Ω−s1(¯Γ0−Γ0)′λˆ−Σ(¯Γ0−Γ0) ˆβ−Σ( ˆΨ( ˜β)−Ψ)ˆλ. First, note that||Ψ−1
Γ0||=Op(1) by ME and||(¯Γ0−Γ0)′λˆ||=Op(K/N) by CS. Then||Ψ−1Γ0Ω−s1(¯Γ0− Γ0)′λˆ|| = O
p(K/N). For the second term, ||Σ(¯Γ0 −Γ0) ˆβ|| =Op(
√
K/N)Op(1/ √
where ||Σ(¯Γ0 −Γ0)|| = Op(
√
K/N) by ME and ||Γ0¯ −Γ0)|| = Op(
√
K/N). For the third term, first note that||Σ( ˆΨ( ˜β)−Ψ)ˆλ|| ≤ ||Σ ˆΥv1λˆ||+||2 ˜βΣΥρηλˆ||+||ΣRΨλˆ||by Lemma 7. Now,||Σ ˆΥv1λˆ||=
Op(ζ(K)√K/N) by CS and Lemma 7,||2 ˜βΣΥρηλˆ||=Op(√K/N) by M, and||ΣRΨλˆ||=Op(ζ(K)K/N3/2+
K3/2/N3/2) by CS and Lemma 7. Hence,||Σ( ˆΨ( ˜β)−Ψ)ˆλ||=Op(ζ(K)√K/N)
Lemma 6. Rλ1 =Op(1/N), where
Rλ1 = − 1 2Ψ
−1
Γ0Ω−s1Γ′1βˆλˆ− 1 2ΣΓ1βˆ
2
−12Ψ−1Γ0Ω−s1(¯Γ1−Γ1)′βˆλˆ−1
2Σ(¯Γ1−Γ1) ˆβ 2
−16ΣΓ∗2βˆ 3
−16Ψ−1Γ0Ω−s1Γ∗′2βˆ 2ˆ
λ.
Proof. First, note that||Ψ−1
Γ0||=Op(1) by ME. Hence,||Ψ−1Γ0Ωs−1Γ′1βˆλˆ||=Op(1/ √
N)Op(1/ √
N) = Op(1/N). For the second term, ||ΣΓ1βˆ2||=Op(1/N), where ||ΣΓ1||=Op(1) by ME. Third, since ||(¯Γ1−Γ1)′λˆ||=O
p(K/N) by CS, it follows that||Ψ−1Γ0Ω−s1(¯Γ1−Γ1)′βˆλˆ||=Op(K/N)Op(1/ √
N) = Op(1/N). For the fourth term, noting that||Σ(¯Γ1−Γ1)||=Op(
√
K/N) by ME, we have||Σ(¯Γ1− Γ1) ˆβ2|| = Op(
√
K/N)Op(1/N) = Op(1/N). Fifth, applying ME gives ||ΣΓ∗2|| = Op(1). Hence, ||ΣΓ∗
2βˆ 3
||=Op(1/N3/2) =Op(1/N). Lastly,||Ψ−1Γ0Ω−s1Γ∗′2βˆ 2ˆ
λ||=Op(1/ √
N)Op(1/N) =Op(1/N).
Lemma 7. Ψ( ˜ˆ β)−Ψ = ˆΥv1+ 2 ˜βΥρη+RΨ, where
ˆ Υv1 =
1 N
∑
i
(ρ2i −σ2i)qiqi′, Υρη= 1 N
∑
i
σρη,iqiqi′, and ||R Ψ
||=Op(ζ(K) √
K/N +K/N)
Proof. Ψ( ˜ˆ β)−Ψ can be written as: ˆ
Ψ( ˜β)−Ψ
=
(ˆ
Υ11 Υ12ˆ
ˆ
Υ21 s−1
( ˆΥ22−Υ21ˆ Υˆ−111Υ12) + ˆˆ Υ21Υˆ
−1 11Υ12ˆ
)
−
(
Υ11 Υ12
Υ21 s−1
(Υ22−Υ21Υ−111Υ12) + Υ21Υ
−1 11Υ12
)
= ˆΥ( ˜β)−Υ +
(
0 0
0 (s−1
−1)[( ˆΥ22−Υ21ˆ Υˆ11−1Υ12)ˆ −(Υ22−Υ21Υ
−1 11Υ12)]
)
.
As in the proof of Proposition 1 in Donald, Imbens, and Newey (2009), the matrix ˆΥ( ˜β)−Υ has a decomposition: ˆΥ( ˜β)−Υ = ˆΥv1+ 2 ˜βΥρη +RΥ, where RΨ = Op(ζ(K)√K/N +K/N). Since 1−s=Op(K2/N) and ( ˆΥ22−Υ21ˆ Υˆ−111Υ12)ˆ −(Υ22−Υ21Υ−
1
11Υ12) =op(1), ˆ
Ψ( ˜β)−Ψ = ˆΥ( ˜β)−Υ +op( (sK)2
N ) = ˆΥv1+ 2 ˜βΥρη+R Ψ
,
whereRΨ
=Op(ζ(K) √
K/N +K/N).
Lemma 8. Rλ3 =op(K2/N3/2), where
Rλ3 =−Ψ− 1
Γ0Ω−1
s (¯Γ0−Γ0)′Rλ2 −Σ(¯Γ0−Γ0)Rβ2 + ΣR Ψ
Σ¯g−Σ( ˆΨ( ˜β)−Ψ)Rλ2 +Rλ1.
Proof. First, since ||(¯Γ0−Γ0)′Rλ
2|| = Op(
√
K/N)Op(ζ(K)√K/N3/2) by CS and Lemmas 5 and 6, ||Ψ−1
Γ0Ω−1
s (¯Γ0−Γ0)′R2λ|| = op(K2/N3/2). For the second term, noting that ||Σ(¯Γ0−Γ0)|| =
Op(
√
K/N) and ||Rβ2|| = Op(K/N) by Lemmas 3 and 4, applying CS gives ||Σ(¯Γ0−Γ0)R β 2|| = Op(
√
K/N)Op(K/N) = op(K2/N3/2). For the third term, ||ΣRΨΣ¯g|| = op(K2/N3/2) by M and Lemma 7. For the fourth term, ||Σ( ˆΨ( ˜β)−Ψ)R2λ|| = Op(ζ(K)
√
K/N)Op(ζ(K)√K/N) =
Lemma 9. Rβ3 =op(K2/N3/2), where
Rβ3 = Ω
−1
s (¯Γ0−Γ0)′Ψ− 1
Γ0Ω−s1(¯Γ0−Γ0)′Σ¯g+ Ω−s1(¯Γ0−Γ0)′Σ(¯Γ0−Γ0)Γ′0Ψ−1¯g +2 ˜βΩ−s1(¯Γ0−Γ0)′ΣΥρηΣ¯g+ Ω−s1(¯Γ0−Γ0)′Rλ3.
Proof. Since||(¯Γ0−Γ0)′Ψ−1
Γ0||=Op(1/√N) by Lemma 2 and||(¯Γ0−Γ0)′Σ¯g|| ≤λmax(Σ)||Γ0¯ −Γ0||· ||g¯||=Op(K/N), ||Ω−s1(¯Γ0−Γ0)′Ψ−1Γ0Ω−s1(¯Γ0−Γ0)′Σ¯g||=Op(1/
√
N)Op(K/N) =op(K2/N3/2). For the second term, note that ||(¯Γ0 −Γ0)′Σ(¯Γ0 −Γ0)|| ≤ λmax(Σ)||Γ0¯ −Γ0||2
= Op(K/N) and that ||Γ′0Ψ−1
¯
g|| = Op(1/√N) by Lemma 2. Hence, ||Ω−s1(¯Γ0 − Γ0)′Σ(¯Γ0 − Γ0)||Γ′0Ψ−1¯g|| =
Op(K/N)Op(1/ √
N) =op(K2/N3/2). For the third term,||β˜Ωs−1(¯Γ0−Γ0)′ΣΥρηΣ¯g||=Op(1/ √
N)× Op(K/N3/2) =op(K2/N3/2), because||(¯Γ0−Γ0)′ΣΥρηΣ¯g|| ≤λmax(ΣΥρηΣ)||(¯Γ0−Γ0)||·||¯g||. Lastly, ||(¯Γ0−Γ0)′Rλ3|| =Op((K/N)
3/2
) by CS and Lemma 8, which implies that ||Ω−1
s (¯Γ0−Γ0)′Rλ3||= op(K2/N3/2)
Lemma 10. Rβ4 =op(K2/N3/2), where
Rβ4 = Ω− 1 s Γ′0Ψ−
1ˆ
Υv1Ψ−1Γ0Ω−s1(¯Γ0−Γ0)′Σ¯g+ Ω− 1 s Γ′0Ψ−
1ˆ
Υv1Σ(¯Γ0−Γ0)Ω−s1Γ′0Ψ− 1
¯ g +2 ˜βΩ−s1Γ′0Ψ−1Υˆv1ΣΥρηΣ¯g+ Ω−s1Γ′0Ψ−
1ˆ Υv1Rλ3 +2 ˜βΩ−s1Γ′0Ψ−1ΥρηRλ2 + Ω−
1 s Γ′0Ψ−
1 RΨλ.ˆ
Proof. First, noting that||Γ′
0Ψ− 1ˆ
Υv1Ψ−1Γ0||=Op(1/√N) and||(¯Γ0−Γ0)′Σ¯g||=Op(K/N) by M, we get||Ω−1
s Γ′0Ψ−1Υˆv1Ψ−1Γ0Ω−s1(¯Γ0−Γ0)′Σ¯g||=Op(1/√N)Op(K/N) =op((sK)2/N3/2). Second, since||Γ′0Ψ−1ˆ
Υv1Σ(¯Γ0−Γ0)||=Op(K/N) by M and||Γ′0Ψ−1¯g||=Op(1/ √
N) by Lemma 2, we get ||Ω−1
s Γ′0Ψ− 1ˆ
Υv1Σ(¯Γ0−Γ0)Ω−s1Γ′0Ψ− 1
¯
g||=Op(K/N)Op(1/√N) =op(K2/N3/2). The third term is:
||β˜Ω−s1Γ′0Ψ−1Υˆv1ΣΥρηΣ¯g||=||β˜|| · ||Ωs−1|| · ||Γ′0Ψ− 1ˆ
Υv1ΣΥρηΣ¯g||=Op(1/ √
N)Op(1)Op(K/N), where||Γ′
0Ψ− 1ˆ
Υv1ΣΥρηΣ¯g||=Op(K/N) by CS. Fourth, we have||Ω−s1Γ′0Ψ− 1ˆ
Υv1Rλ3||=op(K 2
/N3/2 ) by Lemma 8. Fifth,||β˜Ω−1
s Γ′0Ψ−1ΥρηRλ2||=Op(1/ √
N)Op(ζ(K)K/N) =op(K2/N3/2) by Lemmas 5 and 6. Lastly, applying CS gives||Γ′0Ψ−1
RΨˆ
λ||=Op(ζ(K)K/N3/2+ (K/N)3/2). Hence, we have ||Ω−1
s Γ′0Ψ−1RΨλˆ||=op(K2/N3/2).
Lemma 11. Rβ5 =op(K2/N3/2), where Rβ5 = Ω−1
s Γ′0Ψ− 1
(¯Γ0−Γ0)Rβ2.
Proof. ||Γ′0Ψ−1(¯Γ0−Γ0)||= Op(1/√N) by Lemma 2 and ||Rβ2|| =Op(K/N) by Lemma 3 and 4. ThereforeRβ5 =Op(1/
√
N)Op(K/N) =op(K2/N3/2).
Lemma 12. Rβ6 =op(K2/N3/2), where Rβ6 =−Ω− 1
s Γ′1Σ¯gR2β+ Ω− 1
s Γ′1Rλ2βˆ.
Proof. ||Γ′1Σ¯g|| = O
p(1/ √
N) and ||Rβ2|| = Op(K/N) by Lemma 2 and Lemma 6 respectively. Hence,||Ω−1
s Γ′1Σ¯gR β
2||=Op(1/ √
N)Op(K/N) =op(K2/N3/2). For the second term, since||Γ′1Rλ2||= Op(ζ(K)K/N) by CS and Lemma 5 and 6,||Ω−s1Γ′1Rλ2βˆ||=Op(ζ(K)K/N)Op(1/
√
N) =op(K2/N3/2).
Lemma 13. Rβ7 =op(K2/N3/2), whereRβ7 =−2Ω−1
s Γ′0Ψ− 1
Γ1Ω−1 s Γ′0Ψ−
1 ¯
gRβ2+Ω− 1 s Γ′0Ψ−
1
Γ1(Rβ2) 2
.
Proof. Since||Γ′
0Ψ− 1
Γ1||=Op(1) by Lemma 1,||Γ′0Ψ− 1
¯
g||=Op(1/ √
N) by Lemma 2, and||Rβ2||= Op(K/N) by Lemmas 3 and 4, It follows that||Ω−s1Γ′0Ψ−
1 Γ1Ω−1
s Γ′0Ψ−1gR¯ β2||=Op(1/ √
N)Op(K/N) =
op(K2/N3/2) and||Ω−s1Γ′0Ψ− 1
Γ1(Rβ2) 2
||=Op(K2/N2) =op(K2/N3/2).
Proof. Applying M gives ||(¯Γ0−Γ0)′Σ¯g||=O
p(K/N).
Lemma 15. T2h =Op(√K/N). Proof. ||Γ′0Ψ−1ˆ
Υv1Σ¯g||=Op(1/√N)Op(
√
K/N) =Op(√K/N) by Lemma A.4 in Donald, Imbens and Newey (2003) and CS.
Lemma 16. T3h =Op(1/√N).
Proof. For the first term, noting that ||Γ′0Ψ−1ΥρηΣ¯g|| = Op(1/√N) by by Lemma A.4 in Don-ald, Imbens and Newey (2003), we have ||β˜Γ′0Ψ−1
ΥρηΣ¯g|| = Op(1/N). For the second term, as Lemma 2 implies that||Γ′0Ψ−1
(¯Γ0−Γ0)||=Op(1/ √
N) and that||Γ′0Ψ−1 ¯
g||=Op(1/ √
N), we have ||Γ′
0Ψ− 1
(¯Γ0−Γ0)Ω−1 s Γ′0Ψ−
1 ¯
g||=Op(1/N). Similarly, for the third term, as ||Γ′0Σ¯g||=Op(1/ √
N) and ||Γ′0Ψ−1
¯
g|| =Op(1/ √
N) by Lemma 2, we have ||Γ′1Σ¯gΩ−1
s Γ′0Ψ−1g¯|| =Op(1/N). For the last term, noting that ||Γ′0Ψ−1
Γ1|| = Op(1) by Lemma 1 and ||Γ′0Ψ−1g¯|| = Op(1/ √
N) by Lemma 2, ||Γ′
0Ψ− 1
Γ1(Ω−1 s Γ′0Ψ−
1 ¯ g)2
||=Op(1/N).
Lemma 17. T4h =Op(ζ(K)K/√N).
Proof. First, note that||Σ ˆΥv1Σ¯g||=Op(ζ(K) √
K/N). Then, CS gives that||(¯Γ0−Γ0)′Σ ˆΥ
v1Σ¯g||=
Op(
√
K/N)Op(ζ(K) √
K/N) =Op(ζ(K)K/N3/2), and that||Γ′0Ψ− 1ˆ
Υv1Σ ˆΥv1Σ¯g||=Op(
√
K/N)× Op(ζ(K)√K/N) =Op(ζ(K)K/N3/2).
Lemma 18. E(T1hh′|X) =op(γK,N).
Proof. Define B =QΣQ′/N andA=QΨ−1
Γ0. Then,
T1hh′ =N(¯Γ0−Γ0)′Σ¯g¯g′Ψ− 1
Γ0 =N 1 Nη
′QΣ1
NQ
′ρ1
Nρ
′QΨ−1
Γ0 =η′Bρρ′A/N.
Therefore, N E(Th
1h′|X) is the sum of the terms: E(ηiBijρjρkAk|X) for i, j, k = 1, . . . , N. All of these terms are 0 by the independence and the third moment condition, which implies that E(T1hh′|X) = 0 =op(γK,N).
Lemma 19. E(T2hh′|X) =op(γK,N).
Proof. Define B =QΣQ′/N andA=QΨ−1
Γ0. Then,
T2hh′ =NΓ′0Ψ− 1ˆ
Υv1Σ¯gg¯′Ψ−1Γ0 =NΓ′0Ψ−1 1 NQ
′V QΣ1
NQ
′ρ1
Nρ
′QΨ−1
Γ0 =A′V Bρρ′A/N.
Therefore,Th
2h′ is the sum of the terms: AiViiBijρjρkAkfori, j, k= 1, . . . N. By the independence and the third moment condition, we need to consider only the cases withi=j =l. Therefore,
E(T2hh′|X) =
∑
i
E(Viiρ2i|xi)A2iBii=
∑
i
(κi−1)A2iBii=Op(K/N) =op(γK,N).
Proof.
T3hh′ = −2Nβ˜Γ′0Ψ− 1
ΥρηΣ¯gg¯′Ψ−1Γ0−NΓ′0Ψ−1(¯Γ0−Γ0)Ω−s1Γ′0Ψ− 1
¯
gg¯′Ψ−1Γ0 −12NΓ′1Σ¯gΩ−s1Γ0Ψ′ −1¯gg¯′Ψ−1Γ0+ 1
2NΓ
′
0Ψ−1Γ1(Ω−s1Γ′0Ψ−1g¯)2g¯′Ψ−1Γ0.
Now, by the independence and the third moment condition, the expectations of the second, third and fourth terms on the right-hand side of the equation are 0. It is therefore enough to consider only the first term. Now, ˜β = Φ′ρ/N +o
p(N−1/2). Letting B = QΣQ′/N and A = QΨ−1Γ0, we can write
Nβ˜Γ′0Ψ−1ΥρηΣ¯gg¯′Ψ−1Γ0 = N 1 Nρ
′ΦΓ′0Ψ−1 1 NQ
′V
ρηQΣ 1 NQ
′ρ1
Nρ
′QΨ−1
Γ0+op(γK,N) = ρ′ΦA′VρηBρρ′A/N2+op(γK,N).
Now, ρ′ΦA′V
ρηBρρ′Aconsists of the terms ρiΦiAjVρη,jkBklρlρmAm fori, j, k, l, m= 1, . . . , N. The expectations of all of these are zeros by the independence and the third moment condition.
Lemma 21. E(T4hh′|X) =op(γK,N).
Proof. Define B =QΣQ′/N andA=QΨ−1
Γ0. Then,
T4hh′ = −N(¯Γ0−Γ0)′Σ ˆΥv1Σ¯g¯g′Ψ−1Γ0+NΓ′0Ψ−1Υˆv1Σ ˆΥv1Σ¯gg¯′Ψ−1Γ0 = −N 1
Nη
′QΣ1
NQ
′V QΣ1
NQρ 1 Nρ
′QΨ−1
Γ0+NΓ′0Ψ−1 1 NQ
′V QΣ1
NQ
′V QΣ1
NQ
′ρ 1
Nρ
′QΨ−1 Γ0 = −η′BV Bρρ′A/N+A′V BV Bρρ′A/N.
E(η′BV Bρρ′A) = 0 by the independence and the third moment condition. ByA′V BV Bρρ′A/N =
A′V Bρρ′A/N+o
p(γK,N) and the proof of Lemma 19,E(A′V BV Bρρ′A/N|X) =op(γK,N).
Lemma 22. E(T1hT1h′|X) = (s∑
iE(ηiρi|xi)ξii,2)2+op(γK,N).
Proof.
T1hT1h′ =N(¯Γ0−Γ)′Σ¯gg¯′Σ(¯Γ0−Γ) =N 1 Nη
′QΣ1
NQ
′ρ 1
Nρ
′QΣ1
NQ
′η =η′Bρρ′Bη/N,
whereB =QΣQ′/N. By the independence assumption,
E(T1hT1h′|X) =
∑
i
E(η2iρ2i|xi)Bii2/N +
∑
i6=j
E(ηi2|xi)E(ρ2j|xj)B2ij/N
+∑
i6=j
(E(ηiρj|xi))2BijBji/N +
∑
i6=j
E(ηiρi|xi)E(ηjρj|xj)BiiBjj
=
( ∑
i
E(ηiρi|xi)Bii
)2
+op(γK,N).
Now,
( ∑
i
E(ηiρi|xi)Bii
)2
=s2
( ∑
i
E(ηiρi|xi)ξii,2
)2