Exercise 7

Question 1.

Assume that a distribution of Y belongs to a parametric family of distributions f(y|θ), where a priori θ ~ π(θ). After getting the first observation y1 we update the distribution of θ by calculating its posterior distribution p(θ|y1) and use it as a prior distribution for θ before we observe y2. Having y2, we update the distribution of θ again by deriving its posterior distribution. Show that one will get the same posterior distribution of θ if instead of caclulating it sequentially, he/she would use the whole data (i.e. y1 and y2) and the original prior π(θ).

Question 2.

Suppose Y1,...,Yn ~ exp(θ), where EY=1/θ.
  1. Find a noninformative prior for θ (according to the Jefrreys' rule) and the corresponding posterior distribution.
  2. Estimate θ w.r.t. to the quadratic error and compare the resulting Bayesian estimator with the MLE.
  3. Repeat the previous paragraph for estimating μ=EY and p=P(Y>a).
  4. In addition to the sample of Y's, we have another independent sample X1,...,Xm ~ exp(φ), EX=1/φ. Again, we use the noninformative prior for φ. We are interested in the ratio θ/φ. Find its posterior distribution, the posterior mean and compare it with the MLE for this ratio.
(hint: recall some of basic distributions you know like χ2, F, etc.)

Question 3.

Three friends, Optimist, Realist and Pessimist, are going to participate in a certain kind of gambling game in casino when they do not know the probability of win p. Motivated by an exciting course in Bayesian statistics, they decided to apply Bayesian analysis. Optimist chose a prior on p from the conjugate family of distributions. In addition, he believes that the chances are 50%-50% (i.e. E(p)=1/2) with Var(p)=1/36. Realist chose a uniform prior U[0,1].
  1. Show that both priors belong to the family of Beta distributions and find their parameters for the priors of Optimist and Realist.
    (hint: E(Beta(α,β))=α/(α+β); Var(Beta(α,β))=αβ/(α+β)2/ (α+β+1))
  2. Pessimist does not have any prior beliefs and decides to choose the noninformative prior according to the Jeffreys' rule. What is his prior? Does it also belong to the Beta family?
  3. Being poor students in Statistics, Optimist, Realist and Pessimist do not have enough money to gamble separately, so they decided to play together. They played the game 25 times and won 12 times. What is the posterior distribution of each one of them?
  4. Find the corresponding posterior means and calculate the corresponding 95% Bayesian credible intervals for p.
    (hint: use the fact that if p~Beta(α,β) and ρ=p/(1-p) is the odds ratio, then (β/α)ρ ~ F2α,2β - those who do not believe it, can (should!) easily verify it!)
  5. Three friends told their classmate Skeptic about the exciting Bayesian analysis each one of them has done. However, Skeptic is, naturally, very skeptical about Bayesian approach and does not belive in any priors - his credo is "In G-d we trust... All others bring data". So he decided to perform a "classical" (non-Bayesian) analysis of the same data. What is his estimate and the 95%-confidence interval for p? Compare the results and comment them briefly.

Question 4.

The waiting time for a bus at a given corner at a certain time of day is known to have a uniform distribution U[0,θ]. From other similar routes it is known that θ has a Pareto distribution Pa(7,4), where the density of Pareto distribution Pa(α,β) is πα,β(θ)=(α/β)(β/θ)α+1 for θ ≥ β and 0, otherwise. Waiting times of 10, 3, 2, 5 and 14 minutes have been observed at the given corner at the last 5 days.
  1. Show that the Pareto distribution provides a conjugate prior for uniform data and find the posterior distribution of θ.
  2. Estimate θ w.r.t. to the quadratic error (recall that E(Pa(α,β))=αβ/(α-1), α>1).
  3. Find a 95% HPD for θ.
  4. Test the hypothesis H0: 0 ≤ θ ≤ 15 vs. H1: θ > 15 by choosing the most likely (a posteriori) hypothesis.