Exercise 1

Question 1.

Using the factorization theorem, find the sufficient statistic for a sample from the Poisson distribution. What is the distribution of the sufficient statistic? Is it minimal? Show its completeness using the definition of a complete statistic.

Question 2.

Suppose we have a sample of n independent observations $Y_1,\ldots,Y_n$, where each $Y_i$ belongs to one of $K$ categories: P($Y$ belongs to the j-th category)=pj, j=1,...,K; p1+...+pk=1 (multinomial distribution). Show that this distribution belongs to the exponential family with k-1 parameters. Find the natural parameters and sufficient statistic. Is it complete? Calculate the means and the variance-covariance matrix of the sufficient statistic.

Question 3.

Let $Y_1,\ldots,Y_n$ be a sample of independent observations, where $Y_i \sim f_{\theta_i}(y)$ is from the one parameter exponential family (i.e. each $Y_i$ belongs to the same class of distributions $f_\theta(y)$ from the exponential family but not necessarily with the same parameter), and $\eta_i=c(\theta_i)$ are the corresponding natural parameteres. The following (generalized linear regression) model describes $\eta_i$ as a function of $p$ explanatory variables $x_1,\ldots,x_p$: $$\eta_i=\beta_0+\beta_1 x_{i1}+\ldots+\beta_p x_{ip}$$

Show that the joint distribution of the data belongs to the exponential family with $p+1$ parameters, find the natural parameters and sufficient statistic. Write down the likelihood function as a function of β for the particular case of Poisson distribution.

Question 4.

Let $Y_1,\ldots,Y_n$ be a sample from an exponential distribution $\exp(\theta)$ with $\mu=EY=1/\theta$.
  1. Find UMVUEs for $\theta$ and $\mu$, and check whether they achieve Cramer-Rao lower bound.
    (hint: recall that $Y_1+\cdots Y_n \sim \frac{1}{2\theta}\chi^2_{2n}=Gamma(\alpha=n,\beta=\theta)$)
  2. The survival function of exponential distribution $S(t)$ is defined as $S(t)=P(Y \geq t)=e^{-\theta t}$. Verify that for a given $t$, a trivial unbiased estimator for $S(t)$ is $1$, if $Y_1 \geq t$ and $0$ otherwise. Using it and applying the Rao-Blackwellization, find the UMVUE for $S(t)$.
    (hint: derive first the conditional density of $Y_1$ given $Y_1+\ldots+Y_n$ via the joint density of $Y_1$ and $Y_1+\ldots+Y_n$)