Econometrics I: Solutions of Homework 1

Econometrics I: Solutions of Homework 1

Hiroki Kato ^* April 19, 2020

Contents

1 Solutions 1

1.1 Question 1 . . . . 1 1.2 Question 2 . . . . 4 1.3 Question 3 . . . . 5

2 Review 7

2.1 Properties of Expectaion and Variance . . . . 7 2.2 Optimization . . . . 9

1 Solutions

1.1 Question 1

Let S(α, β) be the sum of squares residuals:

S(α, β ) = X

t=1

u

= X

t=1

(y

− α − βX

)

(1)

*e­mail: [email protected]. Room 503. All materials I made are published in github:

https://github.com/KatoPachi/2020EconometricsTA.git. If you have any errors in handouts and materials, please contact me via e­mail or make an issue in github.

The Ordinary Least Squares estimators (hereafter, OLS estimators) can be derived by minimizing (1):

( ˆ α, β) ˆ ∈ arg min

α,β

S(α, β) (2)

The first­order conditions of this problem are

∂S(α, β )

∂α = − 2 X

t=1

(y

− α ˆ − βX ˆ

) = 0 (3)

∂S(α, β )

∂β = − 2 X

t=1

X

(y

− α ˆ − βX ˆ

) = 0 (4)

Note that the second­order condition is hold since the Hessian matrix is positive defenite

H =



 



∂S(α,β)

∂α∂α

∂S(α,β)

∂α∂β

∂S(α,β)

∂β∂α

∂S(α,β)

∂β∂β



 

 =



 



2T 2 P

t

X

2 P

t

X

2 P

t

X



 



⇒| H | =

2T · 2 X

t

X

−

2 X

t

X

· 2 X

t

X

= 4T 1

T X

t

X

− ( X

t

X

/T )

⇒| H | = 4T · V (X

) > 0 where V (X

) is the variance of X

By equation (3), we have

ˆ α = 1

T ( X

t

y

− β ˆ X

t

X

)

= ¯ y − β ˆ X ¯ (5)

where y ¯ = P

t

y

/T and X ¯ = P

t

X

/T (sample mean). Substituting equation(5) into equation (4) yields

β ˆ X

t

X

− X ¯ X

t

X

= X

t

y

X

− y ¯ X

t

X

β ˆ = P

t

y

X

− y ¯ P

t

X

P

t

X

− X ¯ P

t

X

β ˆ =

P

t

X

(y

− y) ¯ P

t

X

(X

− X) ¯ (6)

Thus, OLS estimators are

( ˆ α, β) = ˆ

¯

y − β ˆ X, ¯ P

t

X

(y

− y) ¯ P

t

X

(X

− X) ¯

(7)

Note that the estimator of β, β, can be rewritten as follows: ˆ β ˆ = Cov(y

, X

)

V (X

) (8)

where Cov(y

, X

) is covariance (

) between y

and X

, and V (X

) is variance (

) of X

. To prove it, we need to recall the defenition of covariance and variance. First, the defenition of covairance is

Cov(y

, X

) = E[(y

− E(y

))(X

− E(X

))].

Then, sample covariance is S

= 1

T − 1 X

t

(y

− y)(X ¯

− X) ¯

= 1

T − 1 X

t

(y

X

− y

X ¯ − yX ¯

+ ¯ X y) ¯

= 1

T − 1 X

t

y

X

− X ¯ X

t

y

− y ¯ X

t

X

+ T X ¯ y ¯

= 1

T − 1 X

t

y

X

− y ¯ X

t

X

= 1

T − 1 X

t

X

(y

− y) ¯

Second, the defenition of variance is

V (X

) = E[(X

− E(X

))

].

Then, sample variance is

S

= 1 T − 1

X

t

(X

− X) ¯

= 1

T − 1 X

t

X

− 2 ¯ X T − 1

X

t

X

+ 1

T − 1 T X ¯

= 1

T − 1 X

t

X

− X ¯ X

t

X

= 1

T − 1 X

t

X

(X

− X) ¯

Finally, we have

β ˆ = S

S

t

= P

t

X

(y

− y) ¯ P

t

X

(X

− X) ¯ . Thus, the OLS estimator of β is β ˆ = Cov(y

, X

)/V (X

), or

β ˆ = P

t

(y

− y)(X ¯

− X) ¯ P

t

(X

− X) ¯

(9)

1.2 Question 2

From (9), we have

β ˆ = P

t

y

(X

− X) ¯ − y ¯ P

t

(X

− X) ¯ P

t

(X

− X) ¯

= P

t

y

(X

− X) ¯ P

t

(X

− X) ¯

since P

t

(X

− X) = ¯ P

t

X

− T X ¯ = P

t

X

− P

t

X

= 0. Substituting y

= α + βX

+ u

into this equation yields

β ˆ = P

t

(X

− X)(α ¯ + βX

+ u

) P

t

(X

− X) ¯

= β P

t

(X

− X)X ¯

+ P

t

(X

− X)u ¯

P

t

(X

− X) ¯

= β + P

t

(X

− X)u ¯

P

t

(X

− X) ¯

= β + X

t

ω

u

where ω

= (X

− X)/ ¯ P

t

(X

− X) ¯

.

Because β and X

are not random variables, E( ˆ β) = β + X

t

ω

E(u

) = β.

The variance of β ˆ is

V ( ˆ β) = E [( ˆ β − E( ˆ β))

] = E( ˆ β − β)

= E( X

t

ω

u

)

= E [ X

t

ω

u

+ 2 X

t

X

t^′̸=t

ω

ω

u

u

]

= X

t

ω

E(u

) + 2 X

t

X

t^′̸=t

ω

ω

E(u

u

)

= X

t

ω

E[(u

− E(u

))

] + 2 X

t

X

t^′̸=t

ω

ω

E(u

)E(u

)

= σ

P

t

(X

− X) ¯

( P

t

(X

− X) ¯

)

= σ

P

t

(X

− X) ¯

. To derive it, we use following properties:

• mutual independece assumption implies E(u

u

) = E(u

)E(u

).

• By E(u

) = 0, E(u

) = E[(u

− E(u

))

] = V (u

).

1.3 Question 3

Recall that the estimator of α is

ˆ

α = ¯ y − β ˆ X. ¯

Substituting y ¯ = α + β X ¯ + ¯ u (average both sides over t ∈ { 1, . . . T } ) into this equation implies ˆ

α = α − ( ˆ β − β) ¯ X + ¯ u.

Since E( ˆ β) = β and E(¯ u) = E( P

t

u

)/T = P

t

E(u

)/T = 0, we obtain

E( ˆ α) = α.

The variance of α ˆ is

V ( ˆ α) = E[( ˆ α − E( ˆ α))

] = E( ˆ α − α)

= E( − ( ˆ β − β) ¯ X + ¯ u)

= E[( ˆ β − β)

X ¯

− 2( ˆ β − β) ¯ X u ¯ + ¯ u

]

= ¯ X

E( ˆ β − β)

− 2 ¯ XE( ˆ β − β)¯ u + E(¯ u

). (10) The first term of equation (10) is

X ¯

E( ˆ β − β)

= X ¯

σ

P

t

(X

− X) ¯

. The third term of equation (10) is

E(¯ u

) = E( P

t

u

)

T

= E[ P

t

u

+ 2 P

t

P

t^′̸=t

u

u

] T

= P

t

E(u

) + 2 P

t

P

t^′

E(u

u

) T

= P

t

E[(u

− E(u

))

] + 2 P

t

P

t^′

E(u

)E(u

) T

= σ

T

The second term of equation (10) is rewritten as follows:

2 ¯ XE( ˆ β − β)¯ u

=2 ¯ XE P

t

(X

− X)u ¯

P

t

(X

− X) ¯

P

t

u

T

= 2 ¯ X T P

t

(X

− X) ¯

E X

t

(X

− X)u ¯

X

t

u

= 2 ¯ X T P

t

(X

− X) ¯

E X

t

(X

− X) ¯

u

+ X

t^′̸=t

u

u

= 2 ¯ X T P

t

(X

− X) ¯

X

t

(X

− X)E(u ¯

) + X

t

X

t^′

(X

− X)E(u ¯

u

)

= 2 ¯ X T P

t

(X

− X) ¯

X

t

(X

− X)E[(u ¯

− E(u

))

] + X

t

X

t^′

(X

− X)E(u ¯

)E(u

)

= 2 ¯ X T P

t

(X

− X) ¯

σ

X

t

(X

− X) = 0. ¯

Hence, we have the variance of α ˆ as follows:

V ( ˆ α) =

X ¯

σ

P

t

(X

− X) ¯

+ σ

T

= σ

P

t

X

T P

t

(X

− X) ¯

2 Review

2.1 Properties of Expectaion and Variance

In this section, we will review some properties of expctation and variance that are used to solve homework.

Mutual Independence

Let X and Y be random variables, which are mutually and independentlly distributed. Then, following properties of expectation and variance must be hold:

1. E(XY ) = E(X)E(Y );

2. Cov(X, Y ) = 0

These two properties means that there is no correlation between X and Y .

Caveate: Independece is sufficient condition for uncorrelatedness (exceptional cases: multivari­

ate normal distribution).

Proof. Without loss of generarity, we assume X and Y are continuous random variables. Let f (x, y) be joint distribution of X and Y . By defenition, mutual independce leads to f (x, y) = f

(x)f

(y).

1. By defenition,

E(XY ) = Z Z

xyf (x, y)dydx

= Z Z

xyf

(x)f

(y)dydx

= Z

xf

(x)dx Z

yf

(y)dy

= E(X)E(Y )

2. By defenition,

Cov(X, Y ) = E[(X − E(X))(Y − E(Y ))]

= E[XY − XE(Y ) − E(X)Y + E(X)E(Y )]

= E(XY ) − E(X)E(Y )

= E(X)E(Y ) − E(X)E(Y ) = 0.

Additivity of Expectaion and Variance

Let X

, . . . , X

be random variables, and let a

, . . . , a

be constants. Then, the following prop­

erties is hold:

1. E( P

i

a

X

+ b) = P

i

a

E(X

) + b;

2. V ( P

i

a

X

+ b) = P

i

a

V (X

) + 2 P

i

P

j̸=i

a

a

Cov(X

, X

)

Proof. Without loss of generarity, we assume X

are continuous random variables. Let f (x

, . . . x

) be joint distribution of X

, . . . , X

.

1. By defenition, E( X

i

a

X

+ b)

= Z

· · · Z

(a

X

+ · · · + a

X

+ b)f(x

, . . . x

)dx

· · · dx

=a

Z

· · · Z

X

f (x

, . . . x

)dx

· · · dx

+ · · · + b Z

· · · Z

f (x

, . . . x

)dx

· · · dx

=a

Z X

Z

X2

· · · Z

Xn

f (x

, . . . x

)dx

· · · dx

dx

+ · · · + b

=a

Z

X

f (x

)dx

+ · · · + b = X

i

a

E(X

) + b

2. By defenition, V ( X

i

a

X

+ b)

=E

( X

i

a

X

+ b) − E( X

i

a

X

+ b)

=E

( X

i

a

X

+ b) − ( X

i

a

E(X

) + b)

=E X

i

a

(X

− E(X

))

=E X

i

a

(X

− E(X

))

+ X

i

X

j̸=i

a

a

(X

− E(X

))(X

− E(X

))

= X

i

a

E(X

− E(X

))

+ X

i

X

j̸=i

a

a

E(X

− E(X

))(X

− E(X

))

= X

i

a

V (X

) + X

i

X

j̸=i

a

a

Cov(X

, X

)

2.2 Optimization

Consider a case that we aim to obtain a point x ∈ R which maximizes or minimizes a function y = f (x). In this case, if x

∈ R attains the maximum or minimum, we have the following first order condition at the beginning:

df (x) dx

x=x⁰

= 0. (11)

In addition, when we consider whether the optimum is a maximum or a minimum, the sufficient condition for the optimum becomes as follows:

d

f (x) dx

x=x⁰

< 0 for a maximum; (12)

d

f (x) dx

x=x⁰

> 0 for a minimum. (13)

Here consider a function y = g(x)( ∈ R ) where x = (x

, . . . , x

)

∈ R

, denoted as g : R

→ R .

If an x

= (x

, . . . , x

)

∈ R

maximizes or minimizes g(x), we apply the following theorem.

If a function g : R

→ R is maximized (minimized) at the point x

= (x

, . . . , x

), then the following equation holds:

∂g(x)

∂x

x=x⁰

=



 

 

 



∂g(x⁰)

∂x1

.. .

∂g(x⁰)

∂xn



 

 

 



= 0. (14)

Moreover, we use the following Hessian matrix to discern a maximum and a minimum.

A Hessian matrix of a function g : R

→ R is defined as follows:

H = ∂g(x)

∂xx

=



 

 

 



∂²g(x)

∂x1∂x1

· · ·

.. . . .. .. .

∂²g(x)

∂xn∂x1

· · ·



 

 

 



Assume that g

(x

) = g

(x

) = · · · = g

(x

) = 0 holds, where g

(x) for i ∈ { 1, . . . , n } denotes the partial derivative of g(x) with respect to x

. The following theorem is a way to distinguish whether x attains a muximum and a minimum.

Suppose that a smooth function g : R

→ R satisfies g

(x

) = · · · = g

(x

) = 0. Then, we can confirm that if:

1. H is a negative definite matrix, x

is a maximum point.

2. H is a positive definite matrix, x

is a minimum point.

As for the positiveness or negativeness of a matrix, we have the following theorem.

A necessary and sufficient condition for a symmetric matrix A ∈ R

to be positive (negative) definite is that eigenvalues λ such that det(A − λI) = 0 are protive (negative), where I is identity matrix:

I =



 

 

 



1 · · · 0 .. . 1 .. . 0 · · · 1



 

 

 



For example, in the case of f (x, y) = x

+ 4xy + 5y

− 2x − 8y + 5, we have f

= 2x + 4y − 2 and f

= 4x + 10y − 8. By solving f

= f

= 0, we obtain an optimum point (x, y) = ( − 3, 2).

Also, the Hessian matrix is given as follows:

H

=



 

 2 4 4 10



 

 .

To obtain eigenvalues, we subtract the diagonal matrix with eigenvalues from the Hessian matrix:

H

− λI =



 



2 − λ 4 4 10 − λ



 



Then, the determinant of this matrix is

f (λ) = (2 − λ)(10 − λ) − 16.

Since f (λ) is convex, all eigenvalues are positives if f (0) > 0 (Write rough graph by yourself).

Then, f(0) = 2 · 10 − 16 > 0. Note that f (0) is correspond to the determinant of Hessian matrix.

Thus, (x, y) = ( − 3, 2) is a minimum point. We can analyze an optimum of a multivariable function

for more variables in the same manner.