Least Squares Regression
Theorem 4.8 MSFE
4.27 Exercises
Exercise 4.1 For some integerk, setµk=E[Yk].
(a) Construct an estimatorµbkforµk. (b) Show thatµbkis unbiased forµk.
(c) Calculate the variance ofµbk, say var£ µbk
¤. What assumption is needed for var£ µbk
¤to be finite?
(d) Propose an estimator of var£ µbk¤
.
Exercise 4.2 CalculateE
·³
Y−µ´3¸
, the skewness ofY. Under what condition is it zero?
Exercise 4.3 Explain the difference betweenY andµ. Explain the difference betweenn−1Pn
i=1XiXi0and E£
XiXi0¤ .
Exercise 4.4 True or False. IfY =X0β+e,X∈R,E[e|X]=0, andebiis the OLS residual from the regres- sion ofYi onXi, thenPn
i=1Xi2ebi=0.
Exercise 4.5 Prove (4.15) and (4.16) Exercise 4.6 Prove Theorem 4.6.
Exercise 4.7 Let βebe the GLS estimator (4.17) under the assumptions (4.13) and (4.14). Assume that Ω=c2ΣwithΣknown andc2unknown. Define the residual vectoree=Y−Xβe, and an estimator forc2
ce2= 1
n−kee0Σ−1ee.
(a) Show (4.18).
(b) Show (4.19).
(c) Prove thatee=M1e, whereM1=I−X¡
X0Σ−1X¢−1
X0Σ−1. (d) Prove thatM01Σ−1M1=Σ−1−Σ−1X¡
X0Σ−1X¢−1
X0Σ−1. (e) FindE£
ce2|X¤ .
(f ) Isce2a reasonable estimator forc2?
Exercise 4.8 Let (Yi,Xi) be a random sample withE[Y |X]=X0β. Consider theWeighted Least Squares (WLS) estimatorβewls=¡
X0W X¢−1¡
X0W Y¢
whereW =diag (w1, ...,wn) andwi=X−2j i , whereXj i is one of theXi.
(a) In which contexts wouldβewlsbe a good estimator?
(b) Using your intuition, in which situations do you expectβewlsto perform better than OLS?
Exercise 4.9 Show (4.27) in the homoskedastic regression model.
Exercise 4.10 Prove (4.35).
Exercise 4.11 Show (4.36) in the homoskedastic regression model.
Exercise 4.12 Letµ=E[Y] ,σ2=Eh
¡Y −µ¢2i
andµ3=Eh
¡Y −µ¢3i
and consider the sample meanY =
1 n
Pn
i=1Yi. FindE
·³
Y−µ´3¸
as a function ofµ,σ2,µ3andn.
Exercise 4.13 Take the simple regression modelY =Xβ+e,X ∈R,E[e|X]=0. Defineσ2i =E£ e2i |Xi¤ andµ3i=E£
ei3|Xi¤
and consider the OLS coefficientβb. FindEh
¡ βb−β¢3
|Xi .
Exercise 4.14 Take a regression modelY =Xβ+ewithE[e|X]=0 and i.i.d. observations (Yi,Xi) and scalarX. The parameter of interest isθ=β2. Consider the OLS estimatorsβbandθb=βb2.
(a) FindE£ θb|X¤
using our knowledge ofE£ βb|X¤
andVβb=var£ βb|X¤
. Isθbbiased forθ? (b) Suggest an (approximate) biased-corrected estimatorθb∗using an estimatorVb
βbforV
βb. (c) Forθb∗to be potentially unbiased, which estimator ofV
βbis most appropriate?
Under which conditions isθb∗unbiased?
Exercise 4.15 Consider an i.i.d. sample {Yi,Xi}i=1, ...,n whereXisk×1. Assume the linear conditional expectation modelY =X0β+e withE[e|X]=0. Assume thatn−1X0X =Ik (orthonormal regressors).
Consider the OLS estimatorβb. (a) FindVβb=var£
βb¤
(b) In general, areβbjandβb`for j6=`correlated or uncorrelated?
(c) Find a sufficient condition so thatβbjandβb`forj6=`are uncorrelated.
Exercise 4.16 Take the linear homoskedastic CEF
Y∗=X0β+e (4.61)
E[e|X]=0 E£
e2|X¤
=σ2
and suppose thatY∗is measured with error. Instead ofY∗, we observeY =Y∗+uwhereuis measure- ment error. Suppose thateanduare independent and
E[u|X]=0 E£
u2|X¤
=σ2u(X)
(a) Derive an equation forY as a function ofX. Be explicit to write the error term as a function of the structural errorseandu. What is the effect of this measurement error on the model (4.61)?
(b) Describe the effect of this measurement error on OLS estimation ofβin the feasible regression of the observedY onX.
(c) Describe the effect (if any) of this measurement error on standard error calculation forβb. Exercise 4.17 Suppose that for the random variables (Y,X) withX>0 an economic model implies
E[Y |X]=¡
γ+θX¢1/2
. (4.62)
A friend suggests that you estimateγandθby the linear regression ofY2onX, that is, to estimate the equation
Y2=α+βX+e. (4.63)
(a) Investigate your friend’s suggestion. Defineu=Y −¡
γ+θX¢1/2
. Show thatE[u|X]=0 is implied by (4.62).
(b) UseY =¡
γ+θX¢1/2
+uto calculateE£ Y2|X¤
. What does this tell you about the implied equation (4.63)?
(c) Can you recover eitherγand/orθfrom estimation of (4.63)? Are additional assumptions required?
(d) Is this a reasonable suggestion?
Exercise 4.18 Take the model
Y =X10β1+X20β2+e E[e|X]=0
E£ e2|X¤
=σ2
whereX=(X1,X2), withX1k1×1 andX2k2×1. Consider the short regressionYi=X1i0 βb1+ebiand define the error variance estimators2=(n−k1)−1Pn
i=1eb2i. FindE£ s2|X¤
.
Exercise 4.19 LetY ben×1,X ben×k, andX∗=XCwhereC isk×kand full-rank. Letβbbe the least squares estimator from the regression ofY onX, and letVb be the estimate of its asymptotic covariance matrix. Letβb∗andVb∗be those from the regression ofY onX∗. Derive an expression forVb∗as a function ofVb.
Exercise 4.20 Take the model in vector notation
Y =Xβ+e E[e|X]=0 E£
ee0|X¤
=Ω.
Assume for simplicity thatΩis known. Consider the OLS and GLS estimatorsβb=¡
X0X¢−1¡ X0Y¢
and βe=¡
X0Ω−1X¢−1¡
X0Ω−1Y¢
. Compute the (conditional) covariance betweenβbandβe: Eh
¡ βb−β¢ ¡
βe−β¢0
|Xi Find the (conditional) covariance matrix forβb−βe:
Eh¡ β−b βe¢ ¡
βb−β¢0
|Xi . Exercise 4.21 The model is
Yi=Xi0β+ei
E[ei|Xi]=0 E£
ei2|Xi¤
=σ2i
Ω=diag{σ21, ...,σ2n}.
The parameterβis estimated by OLSβb=¡ X0X¢−1
X0Y and GLSβe=¡
X0Ω−1X¢−1
X0Ω−1Y. Letbe=Y−Xβb andee =Y −Xβedenote the residuals. LetRb2 =1−be0be/(Y∗0Y∗) andRe2 =1−ee0ee/(Y∗0Y∗) denote the equationR2whereY∗=Y−Y. If the erroreiis truly heteroskedastic willRb2orRe2be smaller?
Exercise 4.22 An economist friend tells you that the assumption that the observations (Yi,Xi) are i.i.d.
implies that the regressionY =X0β+eis homoskedastic. Do you agree with your friend? How would you explain your position?
Exercise 4.23 Take the linear regression model withE[Y |X]=Xβ. Define theridge regressionestimator βb=¡
X0X+Ikλ¢−1
X0Y
whereλ>0 is a fixed constant. FindE£ βb|X¤
. Isβbbiased forβ? Exercise 4.24 Continue the empirical analysis in Exercise 3.24.
(a) Calculate standard errors using the homoskedasticity formula and using the four covariance ma- trices from Section 4.16.
(b) Repeat in your second programming language. Are they identical?
Exercise 4.25 Continue the empirical analysis in Exercise 3.26. Calculate standard errors using the HC3 method. Repeat in your second programming language. Are they identical?
Exercise 4.26 Extend the empirical analysis reported in Section 4.23 using theDDK2011dataset on the textbook website.. Do a regression of standardized test score (totalscorenormalized to have zero mean and variance 1) on tracking, age, gender, being assigned to the contract teacher, and student’s percentile in the initial distribution. (The sample size will be smaller as some observations have missing vari- ables.) Calculate standard errors using both the conventional robust formula, and clustering based on the school.
(a) Compare the two sets of standard errors. Which standard error changes the most by clustering?
Which changes the least?
(b) How does the coefficient ontrackingchange by inclusion of the individual controls (in compari- son to the results from (4.55))?