平成 26 年度計量経済学特別講義時系列分析入門

平成 26 ^年度

計量経済学特別講義 時系列分析入門

2

6

13:00 – 14:30

14:45 – 16:15 VAR (vector Auto Regression)

2

13

14:45 – 16:15

16:30 – 18:00 Volatility Models (

)

場所： 経済研究所 第一共同研究室 （経済研究所 本館 ４

F

代表的テキスト：

・

J.D. Hamilton (1994) Time Series Analysis

(2006)

(

)

・

A.C. Harvey (1981) Time Series Models

(1985)

・沖本竜義

(2010)

1 Time Series Analysis (

)

1.1 Introduction

1. Stationarity (

) :

Let y

, y

, · · · , y

be time series data.

(a) Weak Stationarity (

) : E(y

) = µ,

E((y

− µ )(y

− µ )) = γ ( τ ) , τ = 0 , 1 , 2 , · · · The first moment does not depend on time.

The second moment depends only on time di ff erence.

(b) Strong Stationarity (

) :

Let f (y

, y

, · · · , y

) be the joint distribution of y

, y

, · · · , y

. f (y

, y

, · · · , y

) = f (y

, y

, · · · , y

) All the moments are same for all τ .

2. Auto-covariance Function (

) :

E((y

− µ )(y

− µ )) = γ ( τ ) , τ = 0 , 1 , 2 , · · · γ ( τ ) = γ ( −τ )

3. Auto-correlation Function (

) : ρ ( τ ) = E((y

− µ )(y

− µ ))

√ Var(y

) √

Var(y

) = γ ( τ ) γ (0)

Note that Var(y

) = Var(y

) = γ (0) in the case of stationary process.

4. Partial Autocorrelation Coe ffi cient (

), φ

:

The partial autocorrelation coe ffi cient between y

and y

, denoted by φ

, is a measure of strength of the relationship between y

and y

, after removing influence of y

, · · · , y

.

φ

= ρ (1)

( 1 ρ (1)

ρ (1) 1

) ( φ

φ

)

= ( ρ (1)

ρ (2) )

 





1 ρ (1) ρ (2) ρ (1) 1 ρ (1) ρ (2) ρ (1) 1

 





 





φ

φ

φ

 



 =

 





ρ (1) ρ (2) ρ (3)

 





...

 







1 ρ (1) · · · ρ (k − 2) ρ (k − 1) ρ (1) 1 ρ (k − 3) ρ (k − 2)

... ... ... ...

ρ (k − 1) ρ (k − 2) · · · ρ (1) 1

 







 







φ

φ

...

φ

 





 =

 







ρ (1) ρ (2)

...

ρ (k)

 







Use Cramer’s rule (

) to obtain φ

.

φ

=

1 ρ (1) · · · ρ (k − 2) ρ (1) ρ (1) 1 ρ (k − 3) ρ (2)

... ... ... ...

ρ (k − 1) ρ (k − 2) · · · ρ (1) ρ (k)

1 ρ (1) · · · ρ (k − 2) ρ (k − 1) ρ (1) 1 ρ (k − 3) ρ (k − 2)

... ... ... ...

ρ (k − 1) ρ (k − 2) · · · ρ (1) 1

5. Sample Mean (

) :

µ ˆ = 1 T

∑

t=1

y

6. Sample Auto-covariance (

) : γ ˆ ( τ ) = 1

T

∑

t=τ+1

(y

− µ ˆ )(y

− µ ˆ )

7. Correlogram (

, or

) : ρ ˆ ( τ ) = γ ˆ ( τ )

γ ˆ (0) 8. Lag Operator (

) :

L

y

= y

, τ = 1 , 2 , · · ·

9. Likelihood Function (

) — Innovation Form : The joint distribution of y

, y

, · · · , y

is written as:

f (y

, , y

, · · · , y

) = f (y

| y

, · · · , y

) f (y

, · · · , y

)

= f (y

| y

, · · · , y

) f (y

| y

, · · · , y

) f (y

, · · · , y

) ...

= f (y

| y

, · · · , y

) f (y

| y

, · · · , y

) · · · f (y

| y

) f (y

)

= f (y

)

∏

t=2

f (y

| y

, · · · , y

) . Therefore, the log-likelihood function is given by:

log f (y

, y

, · · · , y

) = log f (y

) +

∑

t=2

log f (y

| y

, · · · , y

) .

Under linear model with normality assumption, f (y

| y

, · · · , y

) is given by the

normal distribution with mean E(y

| y

, · · · , y

) and variance Var(y

| y

, · · · , y

).

1.2 Time Series Models (

)

Autoregressive Model (

or AR

): AR(p) y

= φ

y

+ φ

y

+ · · · + φ

y

+

Moving Average Model (

or MA

): MA(q) y

=

+ θ

+ θ

+ · · · + θ

ARMA Model: ARMA(p , q)

y

= φ

y

+ φ

y

+ · · · + φ

y

+

+ θ

+ θ

+ · · · + θ

ARIMA Model: ARIMA(p , d , q)

∆ y

= y

− y

= (1 − L)y

,

∆

y

= ∆ y

− ∆ y

= (1 − L)

y

, ...

∆

y

= (1 − L)

y

.

∆

y

∼ ARMA(p , q) ⇐⇒ y

∼ ARIMA(p , d , q)

∆

y

= φ

∆

y

+ φ

∆

y

+ · · · + φ

∆

y

+

+ θ

+ θ

+ · · · + θ

SARIMA Model: SARIMA(p , d , q)

∆

y

= y

− y

, s = 4 for quarterly data, and s = 12 for monthly data

∆

∆

y

∼ ARMA(p , q) ⇐⇒ ∆

y

∼ ARIMA(p , d , q)

∆

∆

y

= φ

∆

∆

y

+ φ

∆

∆

y

+ · · · + φ

∆

∆

y

+

+ θ

+θ

+ · · · +θ

1.3 Autoregressive Model (

or AR

)

1. AR( p) Model :

y

= φ

y

+ φ

y

+ · · · + φ

y

+

, which is rewritten as:

φ (L)y

=

, where

φ (L) = 1 − φ

L − φ

L

− · · · − φ

L

. 2. Stationarity (

) :

Suppose that all the p solutions of x from φ (x) = 0 are real numbers

When the p solutions are greater than one in absolute value, y

is stationary.

Suppose that the p solutions include imaginary numbers.

When the p solutions are outside unit circle, y

is stationary.

Example: AR(1) Model: y

= φ

y

+

for

∼ iid N(0 , σ

)

1. The stationarity condition is: the solution of φ (x) = 1 − φ

x = 0, i.e., x = 1 /φ

, is greater than one in absolute value, or equivalently, |φ

| < 1.

2. Rewriting the AR(1) model, y

= φ

y

+

= φ

y

+

+ φ

= φ

y

+

+ φ

+ φ

...

= φ

y

+

+ φ

+ · · · + φ

.

As τ goes to infinity, φ

approaches zero. = ⇒ Stationarity condition 3. For stationarity, y

= φ

y

+

is rewritten as:

y

=

+ φ

+ φ

+ · · · MA representation of AR model, i.e., AR(1) = MA( ∞ ) 4. Mean of AR(1) process, µ

µ = E(y

) = E(

+ φ

+ φ

+ · · · )

= E(

) + φ

E(

) + φ

E(

) + · · · = 0 5. Variance of AR(1) process, γ (0)

γ (0) = V(y

) = V(

+ φ

+ φ

+ · · · )

= + φ + φ + · · ·

= V(

) + φ

V(

) + φ

V(

) + · · ·

= σ

(1 + φ

+ φ

+ · · · ) = σ

1 − φ

Note that V(aX + b) = a

V(X) and V(X + Y) = V(X) + V(Y) when two random variables X and Y are independent.

6. Autocovariance and autocorrelation functions of the AR(1) process:

Rewriting the AR(1) process, we have:

y

= φ

y

+

+ φ

+ · · · + φ

.

Therefore, for τ = 1 , 2 , · · · , the autocovariance function of AR(1) process is:

γ ( τ ) = E((y

− µ )(y

− µ )) = E(y

y

)

= E (

( φ

y

+

+ φ

+ · · · + φ

)y

)

= φ

E(y

y

) + E(

y

) + φ

E(

y

) + · · · + φ

E(

y

)

= φ

γ (0) .

The autocorrelation function of AR(1) process is:

ρ ( τ ) = γ ( τ ) γ (0) = φ

.

Or, multiply y

on both sides of the AR(1) process and take the expectation:

E(y

y

) = φ

E(y

y

) + E(

y

) γ ( τ ) =  

 φ

γ ( τ − 1) , for τ , 0,

φ

γ ( τ − 1) + σ

, for τ = 0.

Using γ ( τ ) = γ ( −τ ), γ ( τ ) for τ = 0 is given by:

γ = φ γ + σ = φ γ + σ .

Note that γ (1) = φ

γ (0). Therefore, γ (0) is given by: γ (0) = σ

1 − φ

7. Partial autocorrelation function of AR(1) process:

φ

= ρ (1) = φ

φ

=

1 ρ (1)

ρ (1) ρ (2)

1 ρ (1)

ρ (1) 1

= ρ (2) − ρ (1)

1 − ρ (1)

= 0

8. Estimation of AR(1) model:

(a) Likelihood function

log f (y

, · · · , y

) = log f (y

) +

∑

t=1

log f (y

| y

, · · · , y

)

= − 1

2 log(2 π ) − 1 2 log

( σ

1 − φ

)

− 1

σ

/ (1 − φ

) y

− T − 1

2 log(2 π ) − T − 1

2 log( σ

) − 1 σ

∑

t=2

(y

− φ

y

)

= − T

2 log(2 π ) − T

2 log( σ

) − 1 2 log

( 1

1 − φ

)

− 1

2 σ

/ (1 − φ

) y

− 1 2 σ

∑

t=2

(y

− φ

y

)

Note as follows:

f (y

) = 1

√

2 πσ

/ (1 − φ

) exp

(

− 1

2 σ

/ (1 − φ

) y

)

f (y

| y

, · · · , y

) = 1

√ 2 πσ

exp (

− 1

2 σ

(y

− φ

y

)

)

∂ log f (y

, · · · , y

)

∂σ

= − T 2

1

σ

+ 1

2 σ

/ (1 − φ

) y

+ 1 2 σ

∑

t=2

(y

− φ

y

)

= 0

∂ log f (y

, · · · , y

)

∂φ

= − φ

1 − φ

+ φ

σ

y

+ 1 σ

∑

t=2

(y

− φ

y

)y

= 0 The MLE of φ

and σ

satisfies the above two equation.

σ ˜

= 1 T

 

 (1 − φ ˜

)y

+

∑

t=2

(y

− φ ˜

y

)

 

 φ ˜

=

∑

t=2

y

y

∑

t=2

y

+ (

φ ˜

y

− σ ˜

φ ˜

1 − φ ˜

) / ∑

t=2

y

(b) Ordinary Least Squares (OLS) Method S ( φ

) =

∑

t=2

(y

− φ

y

)

is minimized with respect to φ

.

φ ˆ

=

∑

t=2

y

y

∑

t=2

y

= φ

+

∑

t=2

y

∑

t=2

y

= φ

+ (1 / T ) ∑

t=2

y

(1 / T ) ∑

y

−→ φ

+ E(y

) E(y

) = φ

OLSE of φ

is a consistent estimator.

The following equations are utilized.

E(y

) = 0

E(y

) = Var(y

) = γ (0)

9. Some formulas:

(a) Central Limit Theorem

Random variables x

, x

, · · · , x

are mutually independently distributed with mean µ and variance σ

. Define x = (1 / T ) ∑

t=1

x

. Then,

x − E(x)

√ V(x) = x − µ

σ/ √

T −→ N(0 , 1) (b) Central Limit Theorem II

Random variables x

, x

, · · · , x

are distributed with mean µ and variance σ

. Define x = (1 / T ) ∑

t=1

x

. Then,

x − E(x)

√ V(x) −→ N(0 , 1)

(c) Let x and y be random variables.

y converges in distribution to a distribution, and x converges in probability to a fixed value. Then, xy converges in distribution.

For example, consider: y −→ N( µ, σ

) and x −→ c.

Then, we obtain: xy −→ N(c µ, c

σ

).

10. Asymptotic distribution of OLSE ˆ φ

=

∑

t=2

y

y

∑

t=2

y

= φ

+ (1 / T ) ∑

t=2

y

(1 / T ) ∑

t=2

y

:

√ T ( ˆ φ

− φ

) −→ N(0 , 1 − φ

)

Proof:

y

is distributed with mean E(y

) = 0 and variance V(y

) = V(y

)V(

) = σ

γ (0) = σ

1 − φ

.

y

, t = 1 , 2 , · · · , T , are iid, because Cov(y

, y

) = E(y

y

) = 0 for t > s.

From the central limit theorem, (1 / T ) ∑

t=1

y

− E((1 / T ) ∑

t=1

y

)

√

V((1 / T ) ∑

t=1

y

)

= (1 / T ) ∑

t=1

y

√ σ

/ (1 − φ

) / √ T

−→ N(0 , 1) .

Rewriting,

√ 1 T

∑

t=1

y

−→ N(0 , σ

1 − φ

) . Next,

1 T

∑

t=1

y

−→ E(y

) = γ (0) = σ

1 − φ

yields:

√ T ( ˆ φ

− φ

) = (1 / √ T ) ∑

t=1

y

(1 / T ) ∑

t=1

y

−→ N(0 , 1 − φ

) .

11. AR(1) + drift: y

= µ + φ

y

+

Mean:

Using the lag operator,

φ (L)y

= µ +

where φ (L) = 1 − φ

L.

Multiply φ (L)

on both sides. Then, when |φ

| < 1, we have:

y

= φ (L)

µ + φ (L)

. Taking the expectation on both sides,

E(y

) = φ (L)

µ + φ (L)

E(

)

= φ (1)

µ = µ

1 − φ

Example: AR(2) Model: Consider y

= φ

y

+ φ

y

+

.

1. The stationarity condition is: two solutions of x from φ (x) = 1 − φ

x − φ

x

= 0 are outside the unit circle.

2. Rewriting the AR(2) model,

(1 − φ

L − φ

L

)y

=

. Let 1 /α

and 1 /α

be the solutions of φ (x) = 0.

Then, the AR(2) model is written as:

(1 − α

L)(1 − α

L)y

=

, which is rewritten as:

y

= 1

(1 − α

L)(1 − α

L)

=

( α

/ ( α

− α

)

1 − α

L + −α

/ ( α

− α

) 1 − α

L

)

3. Mean of AR(2) Model:

When y

is stationary, i.e., α

and α

are outside the unit circle, µ = E(y

) = E( φ (L)

) = 0

4. Autocovariance Function of AR(2) Model:

γ ( τ ) = E((y

− µ )(y

− µ )) = E(y

y

)

= E (

( φ

y

+ φ

y

+

)y

)

= φ

E(y

y

) + φ

E(y

y

) + E(

y

)

=  

 φ

γ ( τ − 1) + φ

γ ( τ − 2) , for τ , 0,

φ γ τ − + φ γ τ − + σ , τ =

The initial condition is obtained by solving the following three equations:

γ (0) = φ

γ (1) + φ

γ (2) + σ

, γ (1) = φ

γ (0) + φ

γ (1) , γ (2) = φ

γ (1) + φ

γ (0) . Therefore, the initial conditions are given by:

γ (0) =

( 1 − φ

1 + φ

) σ

(1 − φ

)

− φ

, γ (1) = φ

1 − φ

γ (0) =

( φ

1 − φ

) ( 1 − φ

1 + φ

) σ

(1 − φ

)

− φ

. Given γ (0) and γ (1), we obtain γ ( τ ) as follows:

γ ( τ ) = φ

γ ( τ − 1) + φ

γ ( τ − 2) , for τ = 2 , 3 , · · · .

5. Another solution for γ (0):

From γ (0) = φ

γ (1) + φ

γ (2) + σ

,

γ (0) = σ

1 − φ

ρ (1) − φ

ρ (2) where

ρ (1) = φ

1 − φ

, ρ (2) = φ

ρ (1) + φ

= φ

+ (1 − φ

) φ

1 − φ

.

6. Autocorrelation Function of AR(2) Model:

Given ρ (1) and ρ (2),

ρ ( τ ) = φ

ρ ( τ − 1) + φ

ρ ( τ − 2) , for τ = 3 , 4 , · · · ,

7. φ

= Partial Autocorrelation Coe ffi cient of AR(2) Process:

 







1 ρ (1) · · · ρ (k − 2) ρ (k − 1) ρ (1) 1 ρ (k − 3) ρ (k − 2)

... ... ... ...

ρ (k − 1) ρ (k − 2) · · · ρ (1) 1

 







 











φ

φ

...

φ

φ

 











=

 







ρ (1) ρ (2)

...

ρ (k)

 





 ,

for k = 1 , 2 , · · · .

φ

=

1 ρ (1) · · · ρ (k − 2) ρ (1) ρ (1) 1 ρ (k − 3) ρ (2)

... ... ... ...

ρ (k − 1) ρ (k − 2) · · · ρ (1) ρ (k)

1 ρ (1) · · · ρ (k − 2) ρ (k − 1) ρ (1) 1 ρ (k − 3) ρ (k − 2)

... ... ... ...

ρ (k − 1) ρ (k − 2) · · · ρ (1) 1

Autocovariance Functions:

γ (1) = φ

γ (0) + φ

γ (1) , γ (2) = φ

γ (1) + φ

γ (0) ,

γ ( τ ) = φ

γ ( τ − 1) + φ

γ ( τ − 2) , for τ = 3 , 4 , · · · . Autocorrelation Functions:

ρ (1) = φ

+ φ

ρ (1) = φ

1 − φ

, ρ (2) = φ

ρ (1) + φ

= φ

1 − φ

+ φ

,

ρ ( τ ) = φ

ρ ( τ − 1) + φ

ρ ( τ − 2) , for τ = 3 , 4 , · · · .

φ

= ρ (1) = φ

1 − φ

φ

=

1 ρ (1)

ρ (1) ρ (2)

1 ρ (1)

ρ (1) 1

= ρ (2) − ρ (1)

1 − ρ (1)

= φ

φ

=

1 ρ (1) ρ (1) ρ (1) 1 ρ (2) ρ (2) ρ (1) ρ (3)

1 ρ (1) ρ (2) ρ (1) 1 ρ (1) ρ (2) ρ (1) 1

= ( ρ (3) − ρ (1) ρ (2)) − ρ (1)

( ρ (3) − ρ (1)) + ρ (2) ρ (1)( ρ (2) − 1) (1 − ρ (1)

) − ρ (1)

(1 − ρ (2)) + ρ (2)( ρ (1)

− ρ (2)) = 0 . 8. Log-Likelihood Function — Innovation Form:

log f (y

, · · · , y

) = log f (y

, y

) +

∑

t=3

log f (y

| y

, · · · , y

) where

f (y

, y

) = 1 2 π

γ (0) γ (1)

γ (1) γ (0)

exp

 

 − 1 2 (y

y

)

( γ (0) γ (1)

γ (1) γ (0)

)

( y

y

)  , f (y

| y

, · · · , y

) = 1

√ 2 πσ

exp (

− 1

2 σ

(y

− φ

y

− φ

y

)

)

. Note as follows:

( γ (0) γ (1)

γ (1) γ (0) )

= γ (0)

( 1 ρ (1)

ρ (1) 1 )

= γ (0)

( 1 φ

/ (1 − φ

)

φ

/ (1 − φ

) 1 )

.

9. AR(2) + drift: y

= µ + φ

y

+ φ

y

+

Mean:

Rewriting the AR(2) + drift model,

φ (L)y

= µ +

where φ (L) = 1 − φ

L − φ

L

.

Under the stationarity assumption, we can rewrite the AR(2) + drift model as follows:

y

= φ (L)

µ + φ (L)

. Therefore,

E(y

) = φ (L)

µ + φ (L)

E(

) = φ (1)

µ = µ

1 − φ

− φ

Example: AR(p) model: Consider y

= φ

y

+ φ

y

+ · · · + φ

y

+

. 1. Variance of AR(p) Process:

Under the stationarity condition (i.e., the p solutions of x from φ (x) = 0 are outside the unit circle),

γ (0) = σ

1 − φ

ρ (1) − · · · − φ

ρ (p) . Note that γ ( τ ) = ρ ( τ ) γ (0).

Solve the following simultaneous equations for τ = 0 , 1 , · · · , p:

γ ( τ ) = E((y

− µ )(y

− µ )) = E(y

y

)

=  

 φ

γ ( τ − 1) + φ

γ ( τ − 2) + · · · + φ

γ ( τ − p) , for τ , 0,

φ

γ ( τ − 1) + φ

γ ( τ − 2) + · · · + φ

γ ( τ − p) + σ

, for τ = 0.

2. Estimation of AR(p) Model:

1. OLS:

min

φ1,· · ·, φp

∑

t=p+1

(y

− φ

y

− φ

y

− · · · − φ

y

)

2. MLE:

max

φ1,· · ·, φp

log f (y

, · · · , y

) where

log f (y

, · · · , y

) = log f (y

, · · · , y

, y

) +

∑

t=p+1

log f (y

| y

, · · · , y

) ,

f (y

, · · · , y

, y

) = (2 π )

| V |

exp

 





 − 1

2 (y

y

· · · y

)V

 







y

y

...

 







 







V = γ (0)

 







1 ρ (1) · · · ρ (p − 2) ρ (p − 1) ρ (1) 1 ρ (p − 3) ρ (p − 2)

... ... ... ...

ρ (p − 1) ρ (p − 2) · · · ρ (1) 1

 







f (y

| y

, · · · , y

) = 1

√ 2 πσ

exp (

− 1

2 σ

(y

− φ

y

− φ

y

− · · · − φ

y

)

)

3. Yule = Walker (

) Equation:

Multiply y

, y

, · · · , y

on both sides of y

= φ

y

+ φ

y

+ · · · + φ

y

+

= y

, take expectations for each case, and divide by variance γ (0).

Moreover, replace the autocorrelation function ρ ( τ ) by the correlogram ˆ ρ ( τ ).

 







1 ρ ˆ (1) · · · ρ ˆ (p − 2) ρ ˆ (p − 1) ρ ˆ (1) 1 ρ ˆ (p − 3) ρ ˆ (p − 2)

... ... ... ...

ρ ˆ (p − 1) ρ ˆ (p − 2) · · · ρ ˆ (1) 1

 







 











φ

φ

...

φ

φ

 











=

 







ρ ˆ (1) ρ ˆ (2)

...

ρ ˆ (p)

 







where

γ ˆ ( τ ) = 1 T

∑

t=τ+1

(y

− µ ˆ )(y

− µ ˆ ) , µ ˆ = 1 T

∑

t=1

y

, ρ ˆ ( τ ) = γ ˆ ( τ )

γ ˆ (0) .

3. AR(p) + drift: y

= µ + φ

y

+ φ

y

+ · · · φ

y

+

Mean:

φ (L)y

= µ +

where φ (L) = 1 − φ

L − φ

L

− · · · − φ

L

. y

= φ (L)

µ + φ (L)

Taking the expectation on both sides,

E(y

) = φ (L)

µ + φ (L)

E(

) = φ (1)

µ

= µ

1 − φ

− φ

− · · · − φ

4. Partial Autocorrelation of AR( p) Process:

φ

= 0 for k = p + 1 , p + 2 , · · · .