TA5 最近の更新履歴 Econometrics Ⅰ 2016 TA session

TAsession note #5

Shouto Yonekura

May 18, 2016

Contents

1 chi-squre distribution 2

2 t-distribution 5

3 F-distribution 6

4 Distribution of ^(n−k)s_σ2 ² 7

5 Distribution of q^βˆi−βi V [ ˆβ]_ii

7

6 Distribution of ^{( ˆβ−β)}

′(X^′X)( ˆβ−β)

s^{2 8}

1 chi-squre distribution

def 5.1

Let Xi ∼iid N(0, 1) i = 1, 2, · · · n. Then

Z:= X₁²+ X₂²_{+ · · · Xn}²

is called chi-squre distribution with degrees of freedom n and denoted Z ∼ χ²(n). Its p.d.f is given by below: f(x) :=ⁿ_Γ(n/2)2¹ n/2x^n/2−1e^−x/2 ,0 < x < ∞

where Γ(s) :=^´₀^∞x^s−1e^−xdx.

prop 5.2

Let X ∼ χ^{2(n). Then}

MX(t) = (1 − 2t)^−n/2, E[X] = n, V[X] = 2n. Proof

By the definition, MX(t) can be calculated as follows:

MX(t) =^´ _Γ(n/2)2¹ n/2x^n/2−1e^−x/2e^txdx

= constant´ x^n/2−1e^{−x2(1−2t)}dx

= constant^{´ }_1−2t^{r n/2−1}e^−r/2_1−2t¹ dr (r := x(1 − 2t))

= (_1−2t¹ )^n/2´ _Γ(n/2)2¹ n/2r^n/2−1e^−r/2dr

= (1 − 2t)^−n/2. Next we can derive E[X] and V [X] as follws:

M_X^′ _{(t) = −}ⁿ_{2(1 − 2t)}^−n/2−1(−2) = n(1 − 2t)^−n/2−1. M_X^′ _{(t) |}t=0= E[X] = n(1 − 0)^{−n/2−1= n.}

M_X^′′(t) = −2n(−ⁿ₂− 1)(1 − 2t)^−n/2−2. M_X^′′_{(t) |}t=0= n(n − 2).

V[X] = (M_X^′ _{(t) |}t=0)²_{− MX}^′ _{(t) |}t=0

= n²_{− n}²+ 2n

= 2n Q.E.D.

prop 5.3

Let x ∼ Nk(µ, Σ). Then

(x − µ)^′Σ−1(x − µ) ∼ χ^2(k).

Proof

Let z := Σ^−1/2(x − µ). Then z ∼ N^{k(0 Ik}). Thus ,by the definition of chi-squre distribution, z^′z_{∼ χ(k)} and we can rewrite this as follows:

z^′z= (Σ^−1/2_{(x − µ))}^′(Σ^−1/2_{(x − µ))}

= (x − µ)^{′(Σ−1/2)′Σ−1/2}(x − µ)

= (x − µ)^{′Σ−1/2Σ−1/2}(x − µ) , (Σ^{′ = Σ)}

= (x − µ)^′Σ−1(x − µ). Q.E.D

prop 5.4

Let A be a symmetric and idempotent n×n matrix and its rank is k. Let x ∼ Nn(µ, In). Then x^′Ax_{∼ χ}²(k).

Proof

Since A is a symmetric and idempotent, there exisits n×n matrix P such that (see appendix.): P^′P = P P^′ = In

P^′AP =



















 1

. .. 1

0 . ..

0



















 .

Let y = P x. Then we get

y_{∼ N}n(0, P IP^′)

= Nn(0, In). Furthermore, we can rewrite it as follows:

y= P^′x

⇐⇒ P y = P P^′x

⇐⇒ P y = x. Using these results, we can obtain

x^′Ax= (P y)^′A(P y)

= y^′P^′AP y

=^Pk_i=1y_i². Since each yi∼iid N(0, 1), thus^Pk_i y_i²_{∼ χ}²(k) Q.E.D.

prop 5.5

Let A be a symmertric and idempotent n×n matrix and its rank is k. Let x ∼ Nn(0, σ²In). Then

1 σ^2x

′Ax_{∼ χ}²(k). Proof

Define y := ¹_σx. Then E[y] = 0 and

V[y] = E[(_σ^x) ^x_σ
^′]

=_σ¹2E[xx^′]

=_σ¹2σ²In

= In.

Thus y ∼ Nn(0, In) and y^′Ay∼ χ(k) (prop5.4). Finally we obtain y^′Ay = (^x_σ)^′A(^x_σ)

= _σ¹2x^′Ax_{∼ χ}²(k) Q.E.D

2 t-distribution

def 5.6

Let x ∼ N(0, 1) and y ∼ χ²(n). Then if x and y are independent, z:= √^x

y/n

is called t-distribution with degrees of freedom n and denoted z ∼ t(n).

prop 5.7

Let x ∼ t(n). If n → ∞, then x →dN(0, 1) where →d means convergence in probability(we will study later calass.) .

Although we provide proof of this proposision, you do not need to understand it at present. Proof

Let z ∼ N(0, 1) and y ∼ χ^{2(n). If vi} ∼^{iid χ2}(1) , i = 1, 2, · · · n, then ^Pni=1^vi ∼ χ²(n).Since E[vi] = 1 (prop5.2), we can show that E[_n^1Pn_i vi_{] →}p1 by using the law of large number. Thus^py_{n →}p1 (Continuous mapping theorem).From these results, we can finally obtain that √^z

y/n ^{→d N}(0, 1) (Slutsky’s theorem) as required. Q.E.D.

Remark

It you need to show that some r.v. is distributed from t-distribution, you have to show following 3 things: (1) a numerator is distributed N (0, 1), (2) a denominator is distributed χ²(n), and (3) they are independent.

3 F-distribution

def 5.8

Let u ∼ χ²(n) and v ∼ χ²(m). If they are independent, then z:=_v/m^u/n

is called F-distribution with degrees of freedom (n,m) and denoted z ∼ F (n, m).

prop 5.9

If x ∼ t(n), then x²∼ F (1, n). Proof

Let y ∼ N(0, 1) and z ∼ χ²(n). Suppose that they are independent. Then √^y

z/n ∼ t(n). Thus the distribution of x² is as same as the distribution of _z/n^y2 . Since y²and z are independent and y²_{∼ χ}¹_{(1), z ∼} χ²(n), we can get x²=_z/n^y2 ∼ F (1, n). Q.E.D.

Remark

If you want to show that some r.v. has F (n, m), you you have to show following 3 things: (1) a numerator is distributed from χ²(n), (2) a denominator is distributed from χ²(m), and (3) they are independent.

4 Distribution of

prop 5.10

Suppose that assumptions A1 ∼ A6 hold. Let ˆ^β is the OLSE of β and e := y − X ˆ^{β. Then}

(n−k)s² σ^{2 ∼ χ}

2_{(n − k),}

where s²:= _n−k^e′e. Proof

First we rewrite ^(n−k)s_σ2 ² as ^e_σ^′2^e. Let MX:= In− X(X^{′X)−1X′. Since e′e= u′M}Xu, we can get:

e^′e σ^{2 =}

u^′MXu σ²

=^u_σ^′MX^u_σ.

Since u ∼ Nn(0, σ²In), ^u_{σ ∼ N}n(0, In). Recall that MXis symmetric and idempotent matrix. Thus ^u_σ^′MX^u_σ ∼ χ²(rankMX) (prop5.5). As we studied last class, if a matrix A is symmetric and idempotent, then rankA = trace(A) and rankMX is could be calculated as follows:

rnakMX= traceMX

= trace(In_{− X(X}

′X)⁻¹X^′)

= n − k. Therefore we obtain ^u_σ^′MX^u_σ ∼ χ²(n − k) as required. Q.E.D.

5 Distribution of _√

V[ ˆβ]_ii prop 5.11

Suppose that assumptions A1 ∼ A6 hold. Let ˆ^β is the OLSE of β. Then

βˆi⁻βi

q V [ ˆβ]_ii ⁼

βˆi⁻βi

√(s²(X^′X)⁻¹)ii ∼ t(n − k),

where s²:= _n−k^e′e and V [ ˆβ]_ii means (i, i) component of V [ ˆβ]. Proof

As mentined above, we have to show that (1) a numerator is distributed from N (0, 1), (2) a denominator is distributed from χ²(n − k), and (3) they are independent.

(1)

First we rewrite √ ^βˆi−βi

(s²(X^′X)⁻¹)ii

as follows:

βˆi⁻βi

√(s²(X^′X)⁻¹)ii

=√ ^βˆi−βi

(σ²X^′X)⁻¹)ii

qσ² s²

=√^zi

s²/σ^{2 ,(zi:=}

βˆi⁻βi

√(σ²X^′X)⁻¹)ii

)

= r ^zi

e′ e/(n−k) σ2

= q^zi

l n−k

(l := ^e_σ^′2^e).

Since ˆβi_{∼ N(β}i, σ²((X^′X)⁻_ii¹), ˆβi_{− β}i _{∼ N(0, σ}²((X^′X)⁻_ii¹).Therefore zi=√ ^βˆi−βi

(σ²X^′X)⁻¹)ii ^{∼ N(0, 1).}

(2)

l_{∼ χ}²(n − k) is already shown (prop5.10). (3)

Since both e and ˆβ are linear functions of u which has standard normal distribution, we only need to show that Cov(e, ˆβ) = 0n.

Cov(e, ˆβ) = E[MXu((X^′X)⁻¹X^′u)^′] , e = MXu

= E[MXuu^′X(X^′X)⁻¹]

= E[(In_{− X(X}

′X)⁻¹X^′)uu^′X(X^′X)⁻¹]

= E[uu^′X(X^′X)⁻¹_{− X(X}^′X)⁻¹X^′uu^′X(X^′X)⁻¹]

= E[uu^′]X(X^′X)⁻¹_{− X(X}^′X)⁻¹X^′E[uu^′]X(X^′X)⁻¹

= σ²X(X^′X)⁻¹_{− X(X}^′X)⁻¹X^′σ²X(X^′X)⁻¹ ,(E[uu^′] = σ²In)

= σ²X(X^′X)⁻¹_{− σ}²X(X^′X)⁻¹X^′X(X^′X)⁻¹

= σ²X(X^′X)⁻¹_{− σ}²X(X^′X)⁻¹= 0n, thus ei and ˆβi are independent.Therefore q^βˆi−βi

V [ ˆβ]_ii ∼ t(n − k) Q.E.D.

6 Distribution of

′(X^′X)( ˆβ_−β) s²

prop 5.12

Suppose that assumptions A1 ∼ A6 hold. Let ˆ^β is the OLSE of β.Then

( ˆβ−β)^′(X^′X)( ˆβ−β)/k

s² ∼ F (k, n − k),

where s²:= _n−k^e′e . Proof

First, we rewrite ^{( ˆβ−β)′(X}_s^′2^{X)( ˆβ−β)} as follows:

( ˆβ−β)^′(X^′X)( ˆβ−β)/k

s^{2 =}

( ˆβ−β)^′(X^′X)( ˆβ−β)/k e^′e/n−k

= ^{( ˆβ−β)}

′(X^′X)( ˆβ−β)/k

σ² × _{e′ e/(n−k)}¹

σ2

=_l/n−k^u/k , u:== ^{( ˆβ−β)}

′(X^′X)( ˆβ−β) σ^{2 , l:=}

e^′e σ^2.

As mentioned above, we have to show that (1) u is distributed from χ²(k), (2) l is distributed from χ²_{(n − k),} and (3) they are independent.

(1)

Since ˆβ_{∼ N}k(β, σ²(X^′X)⁻¹), we get u ∼ χ²(k). (prop5.3, ^(X_σ^′2^X) corresponds to Σ⁻¹and µ = β.) (2)

l_{∼ χ}²(n − k) is already shown. (prop5.10) (3)

Since u is a linear fuction of ˆβ and l is a linear function of e, we only need to show that Cov(e, ˆβ) = 0n

and this is already proved in proof of prop5.11.Therefore ^{( ˆβ−β)}

′(X^′X)⁻¹( ˆβ−β)/k

s² ∼ F (k, n − k) Q.E.D.

Appendix

lem 5.13

Let A be a n×n symmetric matrix. Then there exisits n×n orthogonal matrix P such that

P^′AP =













λ1 0 _{· · ·} 0 0 λ2 0 ^... ... 0 . .. 0 0 _{· · ·} 0 λn











 ,

where λi are eigenvalues of A. Proof

Let pi be a eigenvectors which correspond to λi and k pik= 1, ∀i. That is, Ap1= λ1p1, Ap2= λ2p2,_{· · · Ap}n= λnpn. (#)

Let P := (p1, p2,_{· · · p}n). Since pi and pj are orthogonal for any i 6= j, P is a n×n orthogonal matrix. Let Λ be a matrix whose diagonal components are λi. Then we can rewrite (#) as AP = P Λ. Thus we obtain P^′AP = Λ. Q.E.D.

prop 5.14

Let A be a n×n symmetric and idempotent matrix. Then there exisits n×n orthogonal matrix P such that

P^′AP =



















 1

. .. 1

0 . ..

0



















 .

Proof

Since A is a n×n symmetric and idempotent matrix, its eigenvalues are 1 or 0 (see note #3). Thus combining this and prop5.13 yields prop5.14 Q.E.D.