The Mathematical Visual Perception (MVP) Laboratory
Department of Applied Mathematics

School of Mathematical Sciences

Faculty of Exact Sciences

Tel-Aviv University

 

Waterfall by M.C Escher

The Mathematical Visual Perception (MVP) Laboratory, at the applied mathematics department in Tel Aviv University, deals with computational vision and image processing experiments. 
The main research activity of the lab includes application of Differential Geometry and PDE's in image processing and computer vision. The main topics are denoising, segmentation, color and texture analysis, stereo vision, motion analysis, structure from motion and applications to medical imaging. The activity includes theoretical and mathematical aspects as well as numerical and computational considerations. The robotics activity of the lab is based on LEGO robots designed for computer vision applications, and it includes projects and experiments for graduate research and undergraduate students.
 

Staff:


Academic Supervisor: Dr. Nir Sochen 
email: 
sochen@math.tau.ac.il

Lab Manager: Micha Feigin
email: 
michf@post.tau.ac.il

M.C Escher

Research Students in MVP Lab:

 

Chen Sagiv

Lorina Daskal

Ofer Pasternak

Rami Ben-Ari

Yaron Gvili

Micha Feigin

 

Workshop in Vision & Robotics

 

Program:

The workshop consists of lectures and a final project. The syllabus includes an introduction to computer vision and offers tools to deal with various problems such as segmentation, image enhancement, 3D vision, etc. The projects are designed for one or two students. The grade will be based on the achievements in the project.

 

Who can attend?

Undergraduate students on the last year of their studies (pending permission) and graduate students.

 

Computational Tools:

Matlab is recommended.

 

Proposed Projects:
 

Image Processing

 

 

1)       Segmentation


Projects on this issue will be proposed at the beginning of the semester.

 

Number of students required for each project: One

 

2)       Super Resolution


In this project a simple web camera is used in order to capture images of a scene. 
The goal is to use 2 or more images of the scene with low resolution and reconstruct an image with higher resolution.

 

Number of students required: One

Vision & Image Processing

 

 

3)       Stereo Vision

In stereo vision, two images of a scene are used in order to evaluate the depth of each visible point.

Pictures of a mannequin’s head  will be taken from two different locations. Based on the depth map the 3D representation of the scene should be constructed (by tools such as VRML).

Number of students required: One / Two

 

4)       Camera Calibration

In this project a camera calibration process is planned and carried out. An algorithm is developed for obtaining the calibration matrix of a given camera.

Number of students required: One

 

5)       Stereo Head Calibration

The stereo head is a device including 2 (or more) synchronized cameras connected by a rig. This system is often used for depth (3D coordinate) estimation of the points in a scene. The calibration process will include beyond the standard camera calibration also a rectification process and correction for lens distortions.

Number of students required: One / Two

 

6)       Motion Estimation

 

A sequence of images of a rolling ball or a pendulum will be captured. The motion parameters (speed, acceleration, etc.) will be evaluated by optical flow calculations and compared with those calculated using simple rules of mechanics.

 

Number of students required: One / Two

 

 

7)       Structure from motion

The task in this project is construction of 3D representation of a scene from a sequence of images captured by a moving camera. A single digital video cam will be mounted on a LEGO robot that will move in a known constant velocity. Based on the stream of the images captured by the camera, the 3D representation of the scene should be constructed.

Number of students required: One

Vision & Robotics

 

 

8)       Pursuit

In this project, two (Lego) robots are involved. One robot will be used as a target (moving on a straight path) and the second as the pursuer. The images of the scene and the robots are captured by a digital video camera located statically above the scene.  The images then are transferred to the host computer, where the image processing is carried out.  The host outputs are steering commands to the pursuer robot guiding the pursuer to capture the target.

Number of students required: Two

 

 

 

9)       Piano movers Problem

Given a robot in the scene including obstacles, the optimal way for passing the obstacles while avoiding collision is known as the “piano movers” problem.

Here we use a LEGO robot in a scene which includes obstacles. A digital video camera will be mounted statically above the scene. The first task is calculating an optimal path for the robot based on the visual data captured by the camera. The second task is guiding the robot through the scene by a control loop based again on the visual data.

Number of students required: Two

Completed Projects

Robot

The robots at the MVP lab are constructed from the LEGO-Mindstorms Kits. The LEGO Mindstorms Kit is a robotics kit from the LEGO Group including LEGO pieces, a large brick containing a CPU with an LCD known as the RCX, and a couple of motors, and sensors such as light and touch sensors.
The Mindstorms kit operates normally with MS Windows. Lego provides a MS Windows software tool that lets you graphically assemble programs for the Mindstorms, using a building-block metaphor to create code. Once the program has been "assembled" in this way, the software compiles your program into a byte-code. This byte-code is then downloaded to the robot, by an IR port (tower), where the firmware of the RCX processes the byte-code and controls the machine based on the instructions in the byte-code. The problem with the LEGO’s software is that it is not possible to have on-line automatic communication with the RCX. But besides parsing byte-code, the RCX firmware has many OS-like functions: it controls the hardware, threading, and in particular, controls the IR port that is used to communicate with the robot. It also has the ability to accept specific commands (as opposed to complete programs) from the IR port or a special remote control, and move the robot based on those commands

              Robot and the IR tower                                    The RCX

Robots Constructed in the MVP lab


Communication between the robot and the host computer:

Projects involving a robot at the MVP lab utilizes usually a closed loop. This loop can begin with capturing an image or sequence of images. The next step will be the image processing. Due to the RCX very limited computational power, the images are up linked to the host (PC) where the image processing is carried out. The final result of the image processing is a command or a set of commands to the robot. These commands are down-linked from the host to the RCX. Thus, there are two kinds of linkages, one between the camera and the host, and the second between the host and the RCX. At this stage we use a cabled camera in order to up-link the images to the host. The problem that usually appears is down-linking the command to the RCX. This issue occupied students and Lego fans and there are several ways to deal with it. One way is by LEJOS (Lego Java Operation System). You can find more information about LEJOS and download the software in here. The important feature of the LEJOS in my opinion is, the option of sending on-line commands to the RCX directly from the Matlab environment (see for example this project). Matlab is an efficient environment for many mathematical calculations including image processing.

There are also other ways. NQC (Not Quite C) for example is a simple language with a C-like syntax that can be used to program the RCX. The NQC enables on-line communication between a C program and the RCX through the IR tower. NQC is free software released under the Mozilla Public License (MPL).

 

 

Useful Stuff

Books in Computer vision:
Here You can find collection of 9 chapters dealing with several issues in image processing and computer vision (in Hebrew). These chapters were written by Dr. Michael Elad.

Chap-1
Chap-2
Chap-3
Chap-4
Chap-5
Chap-6
Chap-7
Chap-7a
Chap-8
Chap-9

Links:
MVP Seminar: http://www.tau.ac.il/~chensagi/mvp_seminar.html

-----------------------------------------------------------------------------------------------------------------------------------------

Remark:

This site was constructed by Rami Ben-Ari. For any comments please contact me at ramiben@post.tau.ac.il

Last updated: 29.3.08