Neural Computation: Statistical Perspective

למידה סטטיסטית בחישוב עיצבי

0368-4149-01  Autumn 2011-12

Tuesday 16-19 Shenkar 222

Prof. Nathan Intrator


Course Mailing List         Yedion


Brief Abstract                                                                      


The first part of the course will discuss statistical learning theory and model estimation as well as issues related to the "Curse of Dimensionality" and over-fitting. The second part will discuss different methods for supervised and unsupervised (neural) learning. The third part will be devoted to model validation methods: model complexity, the variance/bias dilemma and model ensembles.  Several of the latest modeling methods such as Deep Networks will be presented. Classical methods such as BackProp or Support Vector Machines will also be presented. Different real-world applications will be discussed throughout the course.


Prerequisites:  First Year Calculus, Algebra and Probability courses.

The Course is open for third year undergraduate students (sometimes not at the first bidding).


The course has little overlap with courses given by Prof. Brielovski, Yeshurun and Wolf and none of these courses is a pre-requisite for the current course.


   Instructor:  Prof. Nathan Intrator, Schreiber 221, x7598, Office hours: Wednesday 4-5. Contact via email


Partial list of web pointers used in creation of the lectures

Geoff Hinton    Wlodzislaw Duch     Brian Blais    Daniel Silver   Michael Jordan   IAPR Repository  


Other Useful Pointers                                                         

David Horn,  Eshel  Ben Jacob,  Hezy Yeshurun,  Eytan Ruppin,  Talma Hendler (Brain Imaging),  NCSP Lab


Brief Outline

·         Unsupervised Learning

o   Short bio motivation

o   Unsupervised Neuronal Model

o   Connection with Projection Pursuit and advanced feature extraction

·         Supervised Learning Schemes

o   Perceptron and Multi Layer Perceptron

o   RBF, SVM, Trees

o   Training and optimization

·         Model Selection and Validation (advanced training methods)

o   Cross Validation

o   Regularization

o   Noise injection

o   Ensembles

·         Brain Machine Interface

o   EEG, fMRI modalities

o   Brain state interpretation based on machine learning model

o   Recent Research in BMI


Tentative Lectures


Exercise 3   Data for Ex3


Exam is on Sep 14

   Some reading material


Past relevant course


Neuroscience and Cognitive Science

Theory of Cortical Plasticity Cooper et al.

Theoretical Neuroscience Dayan & Abbott

How are 3D objects represented Bulthoff et al

Brain Tutorial

Unsupervised Learning H. Barlow

Tutorial about the Brain

Online Vision Books

UCI Machine Learning Data

SVM Light

Machine learning and Statistics

Quick Overview of Books on the topic

Pattern recognition and neural networks B. Ripley

Neural networks for pattern recognition C. Bishop

Online Book based on lectures of Tali Tishby

Information Theory T. Cover

Introduction to Radial Basis Functions M. Orr

Pattern Classification (2nd Edition) Duda, Hart, Stork

The Elements of Statistical Learning Hastie, Tibshirani, Friedman

Statistics and Neural Networks  Kay, Titterington

The Bias Variance Dilemma  Geman, Bienenstock,  Doursat

Complex Network Slides