Exercise 1

(a light theoretical 'warming-up')

Question 1.

1. In a two-group randomized experimental design, a response y is measured, and the model
   y_i=μ_i+ε_i
   is to be fitted, where

   μ_i=β₀+β₁ x_i,
   ε_i are i.i.d. with zero means and

   x_i=0 in Group 1, x_i=1 in Group 2.

   Express the parameters β₀ and β₁ in terms of the group means μ₁ and μ₂.

   The same model is fitted, but now x_i is defined by

   x_i=-1/2 in Group 1, x_i=1/2 in Group 2

   What do parameters β₀ and β₁ represent now?

   A third definition of x_i is

   x_i=a in Group 1, x_i=b in Group 2

   where a and b are arbitrary constants. What do parameters β₀ and β₁ represent in this general case?

Question 2.

For the regression model

y_i=g(x_i)+ε_i

with a continuous explanatory variable x₁ and a factor x₂: x_2i=0, i=1,...,n₁; x_2i=1, i=n₁+1,...,n₁+n₂, give plots that represent the following linear models, where x₃=x₁²

1. g(x)=β₀+β₁ x₁+β₂ x₂
2. g(x)=β₀+β₁ x₁+β₃ x₃
3. g(x)=β₀+β₁ x₁+β₂ x₂+β₃ x₃
4. g(x)=β₀+β₁ x₁+β₃ x₃+β₄ x₁ x₂
5. g(x)=β₀+β₁ x₁+β₃ x₃+β₅ x₃ x₂

Question 3.

For a simple linear regression model y_i=β₀+β₁ x_i+ε_i, i=1,...,n show that the diagonal entries of the projection (hat) matrix H are

$h_{ii}=\frac{1}{n}+\frac{(x_i-\bar{x})^2}{\sum_{j=1}^n(x_j-\bar{x})^2}$

Question 4.

Consider a standard linear model but with correlated errors: y=Xβ+ε, where ε_i's have zero means and Var(ε)=V, i.e. Var(ε_i)=V_ii, Cov(ε_i,ε_j)=V_ij

1. Find a vector of Weighted Least Squares Estimators for β that minimizes (y-Xβ)^t V^-1 (y-Xβ)
2. Is it unbiased estimator of β?
3. Find a variance-covariance matrix for the vector of weighted least squares estimators.
4. What is the hat-matrix H for weighted least squares linear regression?
5. Show that if, in addition, ε_i are normal, then weighted least squares estimators are also maximum likelihood estimators.