Welcome to Our Course Page!

0366.4520.01 Mathematical Methods for Digital Image Processing

Fall Semester 2011-2012
Graduate Course

Time and Place:
thursday 14-17, room - Schreiber 007



BOOKS

Digital Image Processing, second/third edition by R.C. Gonzalez & R.E. Woods
See also their web site
They also have a second book - Digital Image Processing using MATLAB first/second edition

Fundamentals of Digital Imaging by H.J. Trussel and M.J. Vrhel

Image Processing The Fundamentals by Maria Petrou and Panagiota Bosdogianni

Digital Image Processing by William Pratt (3rd edition)

Digital Image Processing, 4e by B. Jahne

Digital Image Processing by K.R. Castleman

Fundamentals of Digital Image Processing by A.K. Jain

Algorithms for Image Processing and Computer Vision by J.K. Parker

The Image Processing Handbook by John C. Russ

Computer Vision and Image Processing by Scott E. Umbaugh

A Simplified Approach to Image Processing by R. Crane

Image Processing, Analysis and Machine Vision
by Milan Sonka, Vaclac Hlavic and Roger Boyle

The Computer Image by Alan Watt and Fabio Policarpo

Handbook of Image and Video Processing, editor Al Bovik

Anisotropic Diffusion in Image Processing by J. Weickert

Image Proessing and Analysis
Variational, PDE, Wavelet and Stochastic Methods by T. Chan and J. Shen

The course relies on the use of MATLAB. The main site for MATLAB information is at mathworks
and mathworks academia
One book on MATLAB is Mastering MATLAB 7 by Duane Hanselman & Bruce Littlefield

Some introductory material is
introduction to matlab by Nir Gavish
crash introduction to matlab
A tutorial for matlab is available at Matlab tutorial

A list of MATLAB image processing functions is given in functions

Some free alternatives to MATLAB are OCTAVE and PYTHON
There is also an Euler Math toolbox
MATLAB clones for Android are Addi and Mathmatiz
Some web sites of interest are
Vision Bibliography
DSP and Fourier filtering
ImageMagick - software to create, edit, and compose bitmap images in many formats
Tucows software

Course outline - Syllabus

Additional notes are available in notes
See also notes in Hebrew by Michael Elad at MVP Laboratory

This list will be updated as the course progresses.
Books listed below are examples where the material can be found. It is not exhaustive.

  1. lecture #1
    • Digital Image Fundamentals
    • Introduction to probability & Matlab
  2. lecture #2 Color
    • Gonzalez and Woods - chapter 6
    • Watt and Policarpo - chapter 25
  3. lecture #3 Histograms
    • Gonzalez and Woods - chapter 3.1-3.4
    • Watt and Policarpo - chapter 9
    • Bovik - chapter 2.1
  4. Fourier Transform - FFT
    • Gonzalez and Woods - chapter 4.1-4.2
    • Sonka, Hlavac and Boyle - chapter 12
    • Bovik - chapter 2.3
      More general books are
    • Harding, Kammler
  5. lectures #4,5 Noise - linear and nonlinear filters
    • Gonzalez and Woods - chapter 3.5-3.8, 4.3-4.6, 5.1-5.4
    • Castleman - chapters 9-11
    • Jahne - chapters 4,11
    • Bovik - chapters 3.1, 3.2
  6. lectures #6,7 Noise - Edge Detection
    • Gonzalez and Woods - chapter 10.1-10.3
    • Sonka,Hlavac&Boyle - chapter 4.3
    • Jahne - chapter 12
    • Watt and Policarpo - chapter 10
    • Bovik - chapter 4.11, 4.12
    • Parker - chapter 1
    • Umbaugh - chapter 2, 3,4
  7. lecture #8 Thresholding, Hough Transform and Watershed
    • Davies - chapters 8-13
    • Gonzalez and Woods - chapter 10.4-10.6
    • Sonka,Hlavac&Boyle - chapter 5
    • Watt and Policarpo - chapter 12
  8. lectures #9,10 Morphology lecture
    • Gonzalez and Woods - chapter 9
    • Sonka,Hlavac&Boyle - chapter 11
    • Watt and Policarpo - chapter 13
    • Bovik - chapter 2.2, 3.3
    • Parker - chapter 2
  9. lectures #11,12 de-Blurring (Wiener filter)
    • Parker - chapter 6
    • General Inverse Problems - Regularization
  10. lecture #13 PDEs in image processing - Bayesian statistics
    • Heat Equation
    • Anisotropic equations
  11. lecture #14 Image Compression
    • Gonzalez and Woods - chapter 8
    • Sonka,Hlavac&Boyle - chapter 13
    • Watt and Policarpo - chapter 26
    • Umbaugh - chapter 5
  12. Motion
    • Sonka,Hlavac&Boyle - chapter 15
    • Jahne - chapter 14
    • Watt and Policarpo - chapter 17
    • Bovik - chapter 3.10
  13. Texture
    • Sonka,Hlavac & Boyle - chapter 14
    • Jahne - chapter 15
    • Watt and Policarpo - chapter 4
    • Parker - chapter 4
  14. Computer Graphics
    • Foley, van Dam, Feiner & Hughes - chapter 11

Requirements

The course is open to all graduate students in applied mathematics or computer science.
Knowledge of Fourier series is required.
There will be a only a brief review of the DFT and MATLAB in class.
Attendance is not required but HIGHLY recommended.

Grading Policy

There will be homework assignments during the semester.
These consists of manipulations to the pictures provided below.
All programs are to be in MATLAB.
Homeworks can be submitted in PRINT only - complete with all descriptions, programs and figures
submit either in class or in mailbox 034
CDs are in ADDITION not instead of printed version!
It is not necessary to supply the homework on a disk or CD
Homework and project can be in Hebrew or English
For reasons of anonymity homework and projects need include only a teudat zehut number or a name.
If one wishes a response it would be preferable to include an email address.
Grading of homeworks

  • algorithm - 50%
  • description of code and results both text (printed, ASCII, html, pdf or MS Word)
    and pictures (printed or jpg or tif format) - 40%
    Describe in detail what you learned from the assignment.
    A 5 line summary is NOT sufficient (several pages is too long)
    Typica; assignments are 10-20 pages including pictures and MATLAB code
  • Grades above 90% depend on originality
Points will be reduced for assignments submitted late.

The total of the homeworks will be about 25% of the total grade.

The final project will be worth about 75% of the total grade.
Grading of final project

  • basic algorithm
  • description of code and results both text (printed, ASCII, html,pdf or MS Word)
    and pictures (printed or jpg format)
  • Grades above 90% depend on ingenuity/originality
The final project should be submitted in print
description should be sufficient to understand both what was done, how it was done and what are the results
The actual code, description and pictures MUST be included either as a disc/CD (preferable)
MUST include some driver routine (please use self evident name such as main, driver, runit, etc.)
with a readme file explaining how to run it
or sent by email as an attachment (last resort - don't send many small files, zip them)
If it is submitted by email please verify that the project has been received
by sending a short message in a separate email
Leaving the final project on a web site is NOT acceptable
If the file is compressed use some zip format (or unix compress and tar)
Final project is due after Pesach, May 1, 2011. Points will be reduced for late submission.

Homework Assignments

No need to hand in diskettes or CDs
Each assignment counts for 5% of the final grade.
If assignment is sent electronically it MUST BE in MS WORD or PDF format (not zipped). I do not read other formats !!
Send all assignments to eliturkel@gmail.com ONLY (the post account does not have enough storage)
  1. Write program to perform Gamma function with several examples
    Apply it to color pictures in both RBG, (YIQ) and HSI/HSV (on all or part of the fields)
    Describe the advantages/disadvantages of each.

    Write program to calculate histogram of picture and then perform histogram equalization and matching.
    For more credit use randomization to improve the results.
    Also try color in RGB and HSI/HSV

    Consider histogram matching to a
    1.parabolic or V shaped profile and inverse V
    2. p(s) = alpha*exp(-alpha*s) alpha=0.5
    3. logarithmic or exponential function
    4. match histogram of one picture to histogram of a different picture
    Some possible pictures are "dark image", copter, dolphins, salon
    Picture jar_eq or F22 is a good example of separating centerpiece from background
    see also Bryce Canyon
    Show the pictures and histogram before and after equalization/matching.
    Do NOT use built in MATLAB functions, for example, imhist, cumsum, histeq
    - except for comparison.
    One CAN use the conversion MATLAB routines to/from color to gray
    and between color types e.g. rgb2hsi
    extra-extra credit try K-L transform on color picture and compare to
    standard histogram equalization on each channel
    Due December 1

  2. Filtering
    Use IMNOISE to add various levels of noise both Gausssian and salt&pepper
    Linear Filters
    (a) Use several weights for filtering in "image space" e.g. the filter 1/4 (1 2 1) in 2D
    (c) Use filters in Fourier space such as ideal low pass filter, Butterworth and/or Gaussian
    (don't forget to center)
    Create your own masks do NOT use "fspecial". You can use imfilter or conv if needed.
    What is the effect of the free parameters of the Butterworth or Gausssian filters?
    Compare all these for the various noise models
    Possible pictures to see the effect of the averaging are "pattern" or "shapes" (bmp)
    or "chessboard" together with Gaussian noise
    For extra credit try color pictures and do averaging on RGB
    and HSI/HSV or YIQ or CIELAB with averaging on only some of the components
    Nonlinear filters:
    First try median and alpha trim (T) nonlinear filters
    Apply filters 10 times consecutively
    For additional credit do other nonlinear or else adaptive filters.
    eg weighted median, hybrid median, Kuwahara, adaptive, bilateral
    Apply the filters you constructed to color pictures in RGB and/or HSI to some or all the fields
    Compare pictures before and after filters.
    Describe the advantages and disadvantages of various filters.
    Due December 15

  3. Compare various edge detectors
    Roberts, Sobel, compass, Frei-Chen, Laplacian, LOG, DOG
    Compare both "easy" and "hard" pictures - see "edge" pictures below
    As usual add types of noise to pictures before trying an edge detector
    Use of second derivatives without zero-crossing automatically gives a low grade!!
    See effect of noise on results - the "best" ones may dependent on the amount of noise present
    See effect of thresholding on results - How sensitive is the optimal threshold to the picture?
    You can use "edge" command in MATLAB ONLY with the parameter 'zerocross'
    to find zero crossings or with 'canny'
    extra: find optimal thresholding for Canny (picture with noise)
    and see how it varies between different pictures and different amounts of noise
    You can use imcontour to find edges
    You can use MATLAB command to do the convolution
    extra: use nonlinear weighted median (negative central weight) to find edges
    extra: try other adaptive nonlinear filters to find edges
    extra: color pictures in various spaces e.g. RGB and HSI,YIQ,CIELAB,YCBCR etc
    extra: use quantitative measure in comparison of different filters
    Due December 29
  4. Choice:
  5. Convert figure to binary figure using thresholding
    Use opening and closing to improve segmentation
    hit&miss or more advanced thresholding
    Consider gray scale and/or color and your own structuring elements
    You can use the MATLAB commands imerode and imdilate
    The stress is on constructing your own structuring elements and/or
    combinations of erode/dilate to achieve certain effects

    or

    Use MATLAB commands for watershed and/or other segmentation techniques for picture with noise
    See effect of type and amplitude of noise on oversegmentation
    Base watershed on gray level and/or gradient and/or distance
    explain the difference between these approaches
    For extra credit try various definitions of the distance function and see its effect on the watershed
    Alternatively use other commands to prevent oversegmentation
    What happens to Color Images?

    Due January 12
  6. One can use the MATLAB commands for the following assignments
    Add noise to all the images to see the effect of various levels and kinds of noise
    Add blurring to picture
    PSF = fspecial('motion',50,45);
    or

    I = checkerboard(8);
    noise = 0.1*randn(size(I));
    PSF = fspecial('motion',21,11);
    Blurred = imfilter(I,PSF,'circular');
    BlurredNoisy = im2uint8(Blurred + noise);

    Use PSF with DIFFERENT parameters in the deconvolution to see the effect of lack of knowledge
    Blurred=imfilter(picture,PSF1,'replicate','conv')
    Deblur using inverse and Wiener filters
    Compare with 'wiener2' and 'deconvwnr'
    extra credit: use Laplacian regularization, 'deconvreg' and/or color
    deconvreg allows the choice of other operators rather than the Laplacian (squared). Experiment with other possibilities
    Try deconvlucy and compare to others
    See help or doc for further information on these commands
    Due January 29

Final Project

Those working in pairs must also submit translation/explanation of journal article in addition to project.
Some journals of interest are
IEEE Transactions on Image Processing
IEEE Transactions on Pattern Recognition
Journal of Mathematical Imaging and Vision
Computer Vision Graphics and Image Processing
Journal of Electronic Imaging
Pattern Recognition
Pattern Recognition Letters
vision bibliography

Any published article on image processing is acceptable.
Due April 22, 2012 Sample topics

    Thresholding
    Watersheds/region growing
    Advanced edge detection
    Bilateral and nonlinear filters
    Anisotropic PDEs
    Hough transform
    Morphology
    optical flow
    inpainting
    Monte-Carlo methods
    textures
    Multiscale techniques
    Level sets
    fractals
    OCR
Do NOT use pre-existing routines from the internet !! (except for secondary tools)

Final must be submitted in written (paper) version
In addition there should be a second copy either on a disc/CD or else sent as an attachment
If you are sending attachments by email and there are more than 2 or 3 files zip them into one package
Please keep a copy of the final until you receive verification as the mailer occasionally destroys mail
Code must contain driver and be self contained so I can run it without further programming
Please include an email address with the final project
Some of the harder pictures are fighter plane, plane and chinatown, blackbird, briefcase, car, coffeemaker,
golfcart, machine (very difficult), mona (medium), shoppingcart, silicon (very difficult), street (difficult),
Leonarda di-Vinci (very hard), trashcan (medium), typewriter (medium)
One can supply one's own pictures
One MUST provide
  • Written desription of algorithm and discussion of results
  • MATLAB program and computer image of the pictures if not on my web site
    or standard MATLAB
  • Pictures before and after procedures
  • All material should be downloadable to my PC.

One can use all MATLAB commands.
The scope should be wider than a homework assignment and less than a thesis
Part of the work has to be beyond calling MATLAB commands
Part of the grade depends on ingenuity of the project
All choices involve noise. The noise can be either natural (e.g. from scanner) or artificial (IMNOISE in MATLAB)
Try different levels and different types of noise
The options given below should be combined with other techniques given in class
examples include, gamma correction, histogram equalization etc.
Where applicable use quantitative measurement to compare filters,
e.g. Pratts's figure of merit, NMSE,SNR
This is in addition to subjective criteria

To get grade over 90 requires harder pictures and/or advanced algorithms and originality e.g. :
extra credit: use adaptive, local features or nonlinear operators
extra credit: use color pictures in RGB & HSI - see effect of using only some of the HSI basis
for better resolution submit color pictures on CD/disc
MUST include detailed discussion of results. Discuss each set of graphs presented.
Do not give many pictures with 5 lines of short summary of results.

Listed below are several options to be chosen by student.
Individual suggestions beyond these by the student is preferable.
All decisions for the topic must be okayed.

MOST students choose topics of their own choice (subject to approval).
For others some suggested projects are:

  1. Take picture(s) with edges and add noise (Salt & Pepper and Gaussian).
    Experiment with various advanced nonlinear filters coupled with
    advanced and adaptive edge detectors. Use various window sizes
    to get a pre-filter that removes noise without affecting edges.
    What is the effect of the weights (positive & negative) for weighted mean
    of the various nonlinear filters and linear FIR filters.
    Extensions to color images - what is effect of basis RGB etc.
    Use quantitative measurement in comparison of different merhods
  2. De-blurring with various filters and regularizations
    Compare with 'deconwnr' and 'deconvwnr'
  3. Thresholding for multi-body pictures and/or watersheds
    Markers to improve problems with over-segmentation
  4. Discuss various options for distance functions and fast algorithms
    Effect of thresholding, applications to watersheds
    extension to gray level and color pictures
  5. Solution of nonlinear anisotropic PDEs
  6. (include email address if you want grade by email)


IMAGES

see database of images

Barbara Picture: jpg

Varda Picture: jpg Noise-Picture jpg

Lenna Picture: jpg Noise-Picture jpg

Blonde Woman Picture: jpg

Building Picture: jpg Noise-Picture jpg

------------------------------------- Good for checking histogram equalization/matching

Dark Picture Picture: jpg:

Dolphins Picture: jpg

Helicopter Picture: jpg:

Color Dark Picture Picture: jpg:

Underexposed Picture Picture: jpg:

Frog Picture: jpg

F22 Picture: jpg

Jars Picture: jpg

Merkava Tank Picture: jpg

Plane & Chinatown Picture: jpg

Seascape Picture: jpg

Skiing Picture: jpg

------------------------------------- Good for checking edge detection and segmentation

Beads Picture: jpg

Bicycle Handle Picture: jpg

Boat Picture: jpg

Chessboard Picture: jpg

Chinese Picture: jpg

Chinese Dragon Picture: jpg

Coins (Quarters) Picture: jpg Noise-Picture jpg Salt&Pepper-Picture jpg

Corridor Picture: jpg

Kanizsa Triangle Picture: jpg

Letter edges Picture: jpg

5 Agora Coin Picture: jpg

Israeli coins Picture: jpg

Diamonds Picture: jpg

Fingerprint - clear Picture: jpg

Fingerprint - fuzzier Picture: jpg

Fingerprint - fuzziest Picture: jpg

Machine Picture: jpg

Nuts&Bolts Picture: jpg

Pattern Picture: jpg

Peppers Picture: jpg

Shapes Picture: bmp

Shapes1 Picture: gif

Circles Picture: bmp

Blocks Picture: bmp

Silicon Picture: jpg

Simple Shield Picture: jpg

Shield Picture: jpg

Street Picture: jpg

Tools Picture: jpg

Tools2 Picture: jpg

Typewriter Picture: jpg

X Picture: jpg

Wall Picture: jpg

Wheel Picture: jpg

Windmill Picture: jpg

Zebra Picture: jpg

----------------------------------------------------------------------

Blurred pictures for Inverse and Wiener Filter, created using
H = fspecial('motion',20,45); (or other parameters)
MotionBlur = imfilter(I,H,'replicate');
Blurred cameraman Picture: jpg

Blurred Chessboard Picture: jpg

----------------------------------------------------------------------

hard pictures - uneven illumination, shadows, textures

Bryce Canyon Picture: jpg

Chess game Picture: jpg:

Dalmation in snow Picture: jpg:

Dogs Picture: jpg

Glacier Picture: jpg:

Lightning Picture: jpg

London in the Snow Picture: jpg

Old Buildings Picture: jpg

Ostracon Picture: jpg

Pinecone Picture: jpg

Salon - use color histogram equalization Picture: jpg

Shadows Picture: jpg

Shoppingcart Picture: jpg

Skeleton Picture: jpg

Sphinx Picture: jpg

Street-map Picture: jpg

Textures Picture: jpg

Textures2 Picture: bmp

----------------------------------------------------------------------

Astronomy & Sky pictures

Andromeda Galaxy Picture: jpg

Ant Nebula Picture: jpg

Bird Migration Picture: jpg

Blackhole Picture: jpg

Crab Nebula Picture: jpg

Eye in universe Picture: jpg

Face in Clouds Picture: jpg

Galaxy Picture: jpg

Hubble Picture: jpg

Hubble - zoom Picture: jpg

Perfect Storm Picture: jpg

Rocket Launch Picture: jpg

Starry Night Picture: jpg

----------------------------------------------------------------------

Plane pictures

B2 Picture: jpg

B2 #2 Picture: jpg

Blackbird Picture: jpg

F117A Picture: jpg

F15 Picture: jpg

F15 #2 Picture: jpg

F15 NASA Picture: jpg

F18 Picture: jpg

Fighter Picture: jpg

Fighter Plane Picture: jpg

Fighter Formation Picture: jpg

Jet Plane Picture: jpg

Helicopter Picture: jpg

Helicopter & Whale Picture: jpg

747 Picture: jpg

MIG29 Picture: jpg

Plane Picture: jpg

Plane in water Picture: jpg

Plane landing Picture: jpg

Sonic Boom Picture: jpg

Sonic Boom-2 Picture: jpg

Stealth Picture: jpg

----------------------------------------------------------------------

General pictures

Briefcase Picture: jpg

Brussels - Carpet Picture: jpg

Car Picture: jpg

Chocaletes Picture: jpg

Cut Ups Picture: jpg

Eagles Picture: jpg

Eggs Russian Picture: jpg

Elephant Picture: jpg

GolfCart Picture: jpg

Grand Canyon Picture: jpg

Ibex Picture: jpg

Jungle Picture: jpg

Kotel Picture: jpg

Molecule Picture: jpg

Mona Lisa Picture: jpg

Mountain Lion Picture: jpg

Rocks Picture: jpg

Sketch - Leonarda da Vinci Picture: jpg

Submarine&Sailors Picture: jpg

Submarine Picture: jpg

Submarine - grid Picture: gif

Submarine - Yellow Picture: jpg

Surfboat Picture: gif

Tablecloth Picture: jpg

Tents Picture: jpg

Text Picture: jpg

Waves Picture: jpg

You are visitor
Since December 2000

Return to homepage