Assignments: PATTERN RECOGNITION

Pattern Recognition (fall 2007)

The course is for 3rd year undergraduates and graduate students of CS. We will study general principles of Statistical Learning Theory as well as different frameworks for solving Pattern Recognition (PR) problem on the base of Training Sample Set (TSS). Special emphasis is given to a new promising PR structure, called Support Vector Machine. Different methods of Cluster Analysis and Features Selection are also considered in the course.

Course plan:

Historical background: PR and its connection with different fields of science. Different fields of application of PR methods.
Review of some basic results in Multivariate Probability and Statistics.
Basic notions of Statistical Pattern Recognition. Error rate estimates. Cross-validation estimate and its properties.
Bayessian approach to PR. Classifiers and Discriminant Functions.
Influemce of dimensionality on quality of solution.
Linear Regression and Robust Statistics. Application to PR.
Non-Parametric technique: k-NN rule, Perceptron.
Artificial Neural Nets and statistical PR.
Tree methodology in PR.
Features selection and extraction. Principal Components, Karhunen-Loeve system of functions. Bootstrap method of estimation.
Cluster Analysis and PR.
Statistical Learning Theory. V-C Dimension and basic results.
Support Vector based approach to PR. Principles, Numerical procedures, Comparison with other methods.
Syntactic approach to PR.
Review of some optimization methods (Sumulating Annealing, Genetic Algorithms, Evolutionary Programming) and their application to PR.
Statistical models in PR: Bayessian Probabilistic Networks, Hidden Markov Models and others.

Assignements: 6 homeworks with exercises. At the end of the course the students should pass exams in the form of a special homework (during three days). The final mark is mostly based on the result of this final homework. The results of mentioned 6 homeworks also have an influence on the final mark.

Requirements: Standard courses on Mathematics (Probability and Statistics, Linear Algebra, Infi).

Literature:

R. Duda, P. Hart, D. Strok Pattern Classification, Wiley, 2001.

S. Theodoridis, K. Koutroumbas, Pattern Recognition, Third Edition, Academic Press, 2006.

P.J. Rousseeuv and A.M. Leroy, Robust Regression and Outlier Detection' Wiley, 1987.

L. Breiman et others, Classification And Regression Trees, Wadsworth Int. Group, 1984.

A.K. Jain, R.C. Dubes, Algorithms for Clustering Data, Prentice Hall, 1988.

V.N. Vapnik, The Nature of Statistical Learning Theory, Springer, 1995.