The Mathematical
Visual Perception (MVP) Laboratory
Department of Applied Mathematics
School of Mathematical Sciences
Waterfall
by M.C Escher
The main research activity of the lab includes application of Differential
Geometry and PDE's in image processing and computer vision. The main topics are
denoising, segmentation, color and texture analysis, stereo vision, motion
analysis, structure from motion and applications to medical imaging. The activity
includes theoretical and mathematical aspects as well as numerical and
computational considerations. The robotics activity of the lab is based on LEGO
robots designed for computer vision applications, and it includes projects and
experiments for graduate research and undergraduate students.
|
Staff:
Lab Manager: Micha Feigin |
M.C Escher |
|
|
Research
Students in MVP Lab: |
|
|
Chen Sagiv
Lorina DaskalOfer Pasternak |
Rami Ben-AriYaron Gvili Micha Feigin |
|
Workshop in Vision & Robotics
Program:
The workshop consists of
lectures and a final project. The syllabus includes an introduction to computer
vision and offers tools to deal with various problems such as segmentation,
image enhancement, 3D vision, etc. The projects are designed for one or two
students. The grade will be based on the achievements in the project.
Who can
attend?
Undergraduate students on
the last year of their studies (pending permission) and graduate students.
Computational
Tools:
Matlab is recommended.
Proposed Projects:
|
Image
Processing |
|
|
1) Segmentation
Number of students required for each project: One |
|
2) Super
Resolution
Number of students required: One |
|
Vision
& Image Processing |
|
|
3) Stereo Vision In stereo
vision, two images of a scene are used in order to evaluate the depth of each
visible point. Pictures of a
mannequin’s head will be taken
from two different locations. Based on the depth map the 3D representation of
the scene should be constructed (by tools such as VRML). Number of students required: One / Two |
|
4) Camera
Calibration In this project
a camera calibration process is planned and carried out. An algorithm is
developed for obtaining the calibration matrix of a given camera. Number of students required: One |
|
5) Stereo Head
Calibration The stereo head
is a device including 2 (or more) synchronized cameras connected by a rig. This
system is often used for depth (3D coordinate) estimation of the points in a
scene. The calibration process will include beyond the standard camera
calibration also a rectification process and correction for lens distortions. Number of students
required: One / Two |
|
6) Motion Estimation A sequence of images of a rolling ball or a pendulum
will be captured. The motion parameters (speed, acceleration, etc.) will be
evaluated by optical flow calculations and compared with those calculated
using simple rules of mechanics. Number of students required: One / Two |
|
|
7) Structure
from motion The task in this
project is construction of 3D representation of a scene from a sequence of
images captured by a moving camera. A single digital video cam will be
mounted on a LEGO robot that will move in a known constant velocity.
Based on the stream of the images captured by the camera, the 3D
representation of the scene should be constructed. Number of students
required: One |
|
|
|
|
8) Pursuit In this project,
two (Lego) robots are involved. One robot will be used as a target (moving on
a straight path) and the second as the pursuer. The images of the scene and
the robots are captured by a digital video camera located statically above
the scene. The images then are
transferred to the host computer, where the image processing is carried
out. The host outputs are
steering commands to the pursuer robot guiding the pursuer to capture the
target. Number of students required: Two |
|
|
|
9) Piano movers
Problem Given a robot in the scene including
obstacles, the optimal way for passing the obstacles while avoiding collision
is known as the “piano movers” problem. Here we use a LEGO robot in a scene which
includes obstacles. A digital video camera will be mounted statically above
the scene. The first task is calculating an optimal path for the robot based
on the visual data captured by the camera. The second task is guiding the
robot through the scene by a control loop based again on the visual data. Number of students required: Two |
Robot
The robots at the
MVP lab are constructed from the LEGO-Mindstorms
Kits. The LEGO Mindstorms Kit is a robotics kit from the LEGO Group including
LEGO pieces, a large brick containing a CPU with an LCD known as the RCX, and a couple
of motors, and sensors such as light and touch sensors.
The Mindstorms kit operates normally with MS Windows. Lego provides a MS
Windows software tool that lets you graphically assemble programs for the
Mindstorms, using a building-block metaphor to create code. Once the program
has been "assembled" in this way, the software compiles your program
into a byte-code. This byte-code is then downloaded to the robot, by an IR
port (tower), where the firmware of the RCX processes the byte-code and
controls the machine based on the instructions in the byte-code. The problem
with the LEGO’s software is that it is not possible to have on-line automatic
communication with the RCX. But besides parsing byte-code, the RCX firmware has
many OS-like functions: it controls the hardware, threading, and in particular,
controls the IR port that is used to communicate with the robot. It also has
the ability to accept specific commands (as opposed to complete programs) from
the IR port or a special remote control, and move the robot based on those
commands.
Robot and the IR tower
The RCX
Robots Constructed in the MVP lab
Projects involving a robot at the MVP lab
utilizes usually a closed loop. This loop can begin with capturing an image or
sequence of images. The next step will be the image processing. Due to the RCX
very limited computational power, the images are up linked to the host (PC)
where the image processing is carried out. The final result of the image
processing is a command or a set of commands to the robot. These commands are
down-linked from the host to the RCX. Thus, there are two kinds of linkages,
one between the camera and the host, and the second between the host and the
RCX. At this stage we use a cabled camera in order to up-link the images to the
host. The problem that usually appears is down-linking the command to the RCX.
This issue occupied students and Lego fans and there are several ways to deal
with it. One way is by LEJOS (Lego Java Operation System). You can find more
information about LEJOS and download the software in here. The important
feature of the LEJOS in my opinion is, the option of sending on-line commands
to the RCX directly from the Matlab environment (see for example this project). Matlab is an efficient
environment for many mathematical calculations including image processing.
There are also other ways. NQC (Not Quite C) for example is a
simple language with a C-like syntax that can be used to program the RCX. The
NQC enables on-line communication between a C program and the RCX through the
IR tower. NQC is free software released under the Mozilla Public License (MPL).
Useful Stuff
Books in Computer
vision:
Here You can
find collection of 9 chapters dealing with several issues in image processing
and computer vision (in Hebrew). These chapters were written by Dr. Michael
Elad.
Chap-1
Chap-2
Chap-3
Chap-4
Chap-5
Chap-6
Chap-7
Chap-7a
Chap-8
Chap-9
Links:
MVP Seminar: http://www.tau.ac.il/~chensagi/mvp_seminar.html
-----------------------------------------------------------------------------------------------------------------------------------------
Remark:
This site was
constructed by Rami Ben-Ari. For any comments please contact me at ramiben@post.tau.ac.il
Last updated: 29.3.08