The goals of this course are:
To provide graduate students in Statistics and related fields with
an introduction to estimation under Incomplete Data, the EM algorithm,
Hidden Markov models and Probabilistic Expert Systems.
Course Pre-requisites: Theory of Statistics, Introduction to
- Likelihood, score function, Fisher information. Maximum Likelihood
estimation (MLE) - its consistency and asymptotic normality.
The exponential family.
- The incomplete data model. Complete and incomplete data. Examples:
censoring and interval data, missing data, mixtures.
- Theoretical analysis of the incomplete data model. The EM algorithm.
Dempster et al techniques.
- Alternatives to the EM algorithm and estimation of the asymptotic
variance of the MLE under the EM algorithm.
- Applications of and to Reliability and censoring.
- Applications of and to mixture models.
- The Hidden Markov model (HMM). Baum et al techniques - the forward
and backward construction.
- Implementation of the EM algorithm on HMM.
- Applications of EM and HMM to some Pattern Recognition problems.
- Applications of HMM to the detection of ``mental states'' from
- Applications of HMM to financial problems and Insurance.
- An introduction to Probabilistic Expert Systems (or Influence
Diagrams), as an extension of HMM to finite Markov fields. The
ideas of Lauritzen, Spiegelhalter, Dawid and others. Analysis of the
ASIA standard example.
Partial list of helpful references
- Baum, L. E., Petri, T., Soules, G. and Weiss, N (1970). A maximization
technique occuring in the statistical analysis of probabilistic
functions of Markov chains. Ann. Math. Statist., 41, 164-171.
- Dempster, A. P., Laird. N. M. and Rubin, D. B. (1977) Maximum
Likelihood from Incomplete Data via the EM algorithm (with Discussion).
JRSS(B), 39, 1-38.
- Lauritzen, S. L. and Spiegelhalter, D. J. (1988). Local computations
with probabilities on graphical structures and their application to
expert systems (with discussion). JRSS(B), 50, 157-224.
- Little, R. J. A. and Rubin, D. B. (1987). Statistical Analysis with
Missing Data. New York: John Wiley.
- Meilijson, I. (1989). A Fast Improvement to the EM algorithm on its
Own Terms. JRSS(B), 51, 127-138.
Last update: October 16, 2002