Topics in Modern Statistics

Topics:

• Nonparametric Regression
  • Univariate Nonparametric Regression
    • kernel estimation
    • local polynomial regression
    • smoothing splines
    • generalized Fourier series
    • polynomial series
    • Fourier series
    • wavelets
  • Multiple Nonparametric Regression
    • extensions of univariate methods, curse of dimensionality
    • structural constraints: additive models, backfitting algorithm, single index models, projection pursuit
    • neural networks, deep neural networks
• Classification
  • misclassification error, Bayes classifier
  • empirical risk minimization, PAC-learnability, VC-dimension, fundamental theorem of statistical learning
  • plug-in classifiers: discriminant analysis, logistic regression, multinomial logistic regression
  • support vector classifiers, support vector machines (SVM)
  • k-nearest neighbour
  • neural networks, deep neural networks
  • classification trees (CART)
  • ensemble classifiers: bagging, boosting, random forests
• Clustering (unsupervised learning)
  • combinatorial algorithms, K-means, K-medoids
  • convex clustering
  • hierarchical clustering: agglomerative and divisive methods
  • model-based clustering

Literature

• Fan, J., Li, R., Zhang, C-H. and Zou, H. Statistical Foundations of Data Science
• Friedman, F., Hastie, T. and Tibshirani, R. The Elements of Statistical Learning
• Wasserman, L. All of Nonparametric Statistics
• others

Example files:

• Kernel estimation
• Motorcycle example
• Generalized Fourier series estimation
• A short wavelets tour
• VC-dimension
• Classification
• CART and ensemble methods (boosting, bagging, random forests)
• Clustering

Homework

I strongly encourage you to submit homeworks using R Markdown. R Markdown produces .pdf, HTML, Word files and slides that may include text, R code, the corresponding code output, mathematical formulas etc. When you click the Knit button in RStudio a document will be generated that includes both content (text, formulas, etc.) as well as the output of any embedded R code (plots, tables etc.) within the document. For more details on using R Markdown see, for example, R Markdown reference guide or R Markdown: The Definitive Guide. Using RStudio install R packages markdown and knitr. Also, in order to include mathematics in your documents you need to install LaTex/MikTex on your computer.

• Exercise 1 (5 December)
• Exercise 2 (12 December)
• Exercise 3 (9 January)
• Exercise 4 (6 February)