CSE 527: Introduction to Computer Vision

(Graduate Course)

Fall 2024

Basic Information:

·         Lecture info: Mon/Wed 6:30pm – 7:50pm, LGT ENGR LAB 102

·         Instructor: Haibin Ling (haibin.ling AT stonybrook.edu), NCS 147

·         Office hour: Wed 9am-11am or by appointment

·         TAs:

o   Md Shahrar Fatemi (mfatemi AT cs.stonybrook.edu), office hour: Thu 4-6pm

o   Snehal Tomar (stomar AT cs.stonybrook.edu), office hour: Fri 12-2pm

o   Yufeng Wang (yufengwang AT cs.stonybrook.edu), office hour: Thu 3:30-5:30pm

·         Syllabus: PDF

Useful Links:

·         Main textbook: Computer Vision: Algorithms and Applications, Richard Szeliski, 2nd ed., 2022.

·         Computer Vision: A Modern Approach, David Forsyth and Jean Ponce, 2nd ed., 2012.

·         Multiple View Geometry in Computer Vision, Richard Hartley and Andrew Zisserman, 2nd ed., 2004.

·         Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006.

·         Monocular Model-Based 3D Tracking of Rigid Objects: A Survey, Vincent Lepetit, Pascal Fua, 2005.

Course Schedule (tentative, will be actively updated during semester)

Date

Topics

Materials

Week 1

8/26

8/28

·         Introduction

o   Topics, history, applications, related fields

o   Course logistics

·         Image Formation

o   Geometric primitives and transformations

·         Szeliski 1.1, 1.2

·         Szeliski 2.1, 2.2, 2.2, 2.3

 

Week 2

9/4

·         Review of math and coding (by Yufeng Wang)

o   Basic math concept review

o   Tutorial to Google Colab  

Week 3

9/9

9/11

·         Image Formation

o   Photometric image formation

·         Guest lecture (9/11) by Prof. Chenyu You

o   Robust Machine Learning for Biomedical Data: Efficiency, Reliability, and Generalizability

·         Szeliski 2.2, 2.3

·          

Week 4

9/16

9/18

·         Image Processing

o   Linear filtering: separable filtering

o   Non-linear filtering, bilateral filtering

o   Fourier transforms

o   Image pyramid

o   Geometric transform

·         Model Fitting and Optimization

o   Line fitting, robust fitting

o   Hough transform

o   RANSAC

·         Szeliski 3.2, 3.3, 3.4, 3.5, 3.6

·         Szeliski 4.1

·         Szeliski 7.4, 8.1

 

9/22

·         Homework 1 Due (11:59pm EST)

 

Week 5

9/23

9/25

·         Machine Learning Overview

o   Supervised learning

o   Unsupervised learning

·         Recognition

o   Instance recognition

o   Image classification (traditional methods)

·         Szeliski 5.1, 5.2

·         Szeliski 6.1, 6.2

Week 6

9/30

10/2

·         Deep Learning

o   Deep neural networks

o   Convolutional neural networks

o   Architecture

o   Applications

·         Szeliski 5.3, 5.4, 5.5, 6.2

·         Deep Learning, Goodfellow et al., 2016.

·         Deep Learning Tutorial, Stanford

Week 7

10/7

10/9

·         Deep Learning Practice (by Tao Sun)

o   Training

o   Data augmentation

·         Transformer & Vision Transformer (by Tao Sun)

o   Transformer

o   Vision Transformer

·         Szeliski 5.5, 6.3

Week 8

10/16

·         Detection

o   Traditional solutions

o   Deep learning solutions

 

10/20

Homework 2 Due (11:59pm EST)

 

Week 9

10/21

10/23

·         Midterm (10/21)

·         Detection and Segmentation

o   Detection with Transformers

o   Segmentation

·         Szeliski 6.3, 6.4

·         End-to-End Object Detection with Transformers, Carion, et al., ECCV 2020

 

Week 10

10/28

10/30

·         Semantic Segmentation

o   Segmentation - conventional

o   Semantic segmentation – deep learning solutions

·         Szeliski 6.3, 6.4

·         Segment Anything, Kirillov et al., 2023

Week 11

11/4

11/6

·         Feature detection and matching

o   Points and patches

o   Edges and contours

·         Motion estimation and tracking

o   Review of alignment

o   Optical flow

·         Szeliski 7.1, 7.2, 7.5

11/10

Homework 3 Due (11:59pm EST)

 

Week 12

11/11

11/13

 

·         Depth estimation

o   Epipolar geometry

·         Guest lecture (11/13)

o   6D pose estimation by Ruyi Lian

·         Szeliski 8.1, 8.2, 8.3

·         Szeliski 9.1, 9.3, 9.4

 

Week 13

11/18

11/20

·         Depth estimation

o   Correspondences

o   Local and global methods

o   Deep learning solution

·         Motion estimation and tracking

o   Object tracking

·         Szeliski 9.1, 9.3, 9.4

Week 14

11/25

·         Structure from motion and SLAM

o   Two-frame structure from motion

o   Multi-frame structure from motion

o   Simultaneous localization & mapping (SLAM)

·         Szeliski 12.1, 12.2, 12.3, 12.4, 12.5, 12.6

 

12/3

Homework 4 Due (11:59pm EST)

 

Week 15

12/2

12/4

·         Generative models

o   Autoencoder

o   Generative adversarial network

o   Diffusion models

·         Final review (12/2)

·         Szeliski 11.1, 11.2, 11.3, 11.4, 11.5

 

Week 16

 

 

12/9

Final Exam, 6:30-9:00pm LGT ENGR LAB 102