Econometrics I: Solutions of Homework 3

Econometrics I: Solutions of Homework 3

Hiroki Kato ^* April 27, 2020

Contents

1 Solutions 1

1.1 Question 1 (1) . . . . 1 1.2 Question 1 (2) . . . . 3

2 Review: Distributions 4

2.1 Normal Distribution . . . . 4 2.2 Chi­squared Distribution . . . . 5 2.3 t Distribution . . . . 5

1 Solutions

1.1 Question 1 (1)

The OLS estimator of β is

β ˆ =

∑

t

(y

− y)(X ¯

− X) ¯

∑

t

(X

− X) ¯

(1)

*

e­mail: [email protected]. Room 503. All materials I made are published in github:

https://github.com/KatoPachi/2020EconometricsTA.git. If you have any errors in handouts and materials,

please contact me via e­mail or make an issue in github.

Then, substituting y

= α + βX

+ u

into this equation yields β ˆ =

∑

t

(α + βX

+ u

− y)(X ¯

− X) ¯

∑

t

(X

− X) ¯

= β ∑

t

(X

− X)X ¯

+ ∑

t

(X

− X)u ¯

∑

t

(X

− X) ¯

= β +

∑

t

(X

− X)u ¯

∑

t

(X

− X) ¯

= β + ∑

t

ω

u

The moment­generating function of β ˆ is defined by M(θ) ≡ E[exp(θ(β + ∑

t

ω

u

))] (2)

Then, we have

M (θ) = E[exp(θβ + θ ∑

t

ω

u

)]

= E[exp(θβ) exp(θ ∑

t

ω

u

))]

= exp(θβ)E[exp(θ ∑

t

ω

u

))]

= exp(θβ)

∏

t=1

E[exp(θω

u

)] (3)

Since u

∼ N (0, σ

),

E[exp(θω

u

)] = M (θω

) = exp(σ

(θω

)

/2).

Substituting this into (3) yields

M (θ) = exp(θβ) ∏

t

exp(σ

(θω

)

/2)

= exp(θβ + 1

2 σ

θ

∑

t

ω

)

This implies that the exact distribution of β ˆ is

β ˆ ∼ N(β, σ

∑

t

ω

) (4)

We will derive the exact distribution more formally, using the following properties:

M

(0) = E(X

) (5)

where M

(θ) = ∂

M(θ)/∂θ

. E(X

) is called k­th moment of random variable X. First, we will calculate first and second­order derivative of MGF as follows:

M

(θ) = (β + σ

θ ∑

t

ω

) exp( · ), M

(θ) = (σ

∑

t

ω

) exp( · ) + (β + σ

θ ∑

t

ω

)

exp( · ).

By evaluating two equations at θ = 0, we obtain first and second moment:

E( ˆ β) = β E( ˆ β

) = σ

∑

t

ω

+ β

.

Finally, variance of β ˆ is

V ( ˆ β) = E[( ˆ β − E( ˆ β))

] = E( ˆ β

) − E( ˆ β)

= σ

∑

t

ω

.

1.2 Question 1 (2)

Let Z ≡ ( ˆ β − β)/σ √∑

t

ω

. Then, Z ∼ N (0, 1) since β ˆ is normally distributed, and E(Z) = E( ˆ β) − β

σ √∑

t

ω

= 0 V (Z ) = E( ˆ β

) − β

σ

∑

t

ω

= 1

Using a property that ks

/σ

∼ χ

(k) where k is a degree of freedom and s

is unbiased and consis­

tent estimator of σ

, we obtain (T − 2) s

σ

= (T − 2) s

∑

t

ω

σ

∑

t

ω

∼ χ

(T − 2).

Let V ≡ (T − 2)s

∑

t

ω

/σ

∑

t

ω

. Then,

√ Z

V /(T − 2) =

β ˆ − β σ √∑

ω

/√ s

∑

t

ω

σ

∑

ω

=

β ˆ − β s √∑

ω

∼ t(T − 2).

Thus, ( ˆ β − β)/s √∑

t

ω

is a t­distribution with T − 2 degrees of freedom.

2 Review: Distributions

2.1 Normal Distribution

Let X ∼ N(µ, σ

) and Y ∼ N (µ

, σ

).

• Property 1. aX + b ∼ N (aµ + b, a

σ

)

• Property 2 (standarization). Z = (X − µ)/σ ∼ N (0, 1)

• Property 3 (Reproduction). Suppose that X and Y are mutually independent. Then, X+Y ∼ N (µ + µ

, σ

+ σ

).

Proof. First, prove Property 1.

E[exp(θ(aX + b))] =E[exp(θaX )] exp(θb)

= exp(µθa + σ

(θa)

/2) exp(θb)

= exp(θ(aµ + b) + θ

(σa)

/2)

Note that second equality comes from X ∼ N (µ, σ

) and its MGF M (θ) = exp(µθ + σ

θ

/2). This implies that aX + b ∼ N (aµ + b, a

σ

).

Second, prove Property 2.

E[exp(θ(X − µ)/σ)] =E [exp((θ/σ)X)]/ exp(µ(θ/σ))

= exp(µ(θ/σ) + σ

(θ/σ)

/2)/ exp(µ(θ/σ))

= exp(θ

/2)

This is equivalent the moment­generating function whose random variable is a standard normal dis­

tribution.

Third, prove Property 3. To prove it, we first show that the moment­generating function of X + Y is M

(t)M

(t).

M

(t) = E[exp(t(X + Y ))] = E[exp(tX ) exp(tY )] = E[exp(tX)]E[exp(tY )] = M

(t)M

(t).

Third equality holds since E(XY ) = E(X)E(Y ) only if X and Y are independent. Finally, we

obtain

M

(θ) = exp(µθ + θ

σ

/2) exp(µ

θ + θ

σ

/2)

= exp(θ(µ + µ

) + θ

(σ

+ σ

)/2).

This implies that X + Y ∼ N (µ + µ

, σ

+ σ

).

2.2 Chi­squared Distribution

Let X ∼ N(µ, σ

). Consider Z = (X − µ)/σ ∼ N (0, 1). Then,

V =

∑

i

z

∼ χ

(n).

If µ are unknown parameter, and we use sample mean x ¯ = ∑

i

x

/n instead of µ, then V =

∑

i

ˆ

z

∼ χ

(n − 1),

where z ˆ

≡ (x

− x)/σ. ¯

Recall that unbiased estimator of σ

is s

= ∑

i

(x

− x) ¯

/(n − 1). Then, we obtain σ

V = (n − 1)s

V = (n − 1) s

σ

2.3 t Distribution

Let X ∼ N (µ, σ

). The sample means is x ¯ = ∑

i

x

/n, and the unbiased sample variance is s

=

∑

i

(x

− x) ¯

/(n − 1). Then,

T = x ¯ − µ s/ √

n ∼ t(n − 1) Moreover,

T = x ¯ − µ σ/ √

n

/ s/ √ n σ/ √

n = x ¯ − µ σ/ √

n / s

σ = Z

√ V /(n − 1) ∼ t(n − 1),

where Z

∼ N (0, 1) by CLT.

Generally, Z ∼ N (0, 1), V ∼ χ

(k), and Z is independent of V . Then, Z/ √

V /k ∼ χ

(k).