CSE 527: Introduction to Computer Vision

Fall 2023

Basic Information:

        Lecture info: Mon/Wed 7:00pm C 8:20pm, Frey Hall 102 (Map)

        Instructor: Haibin Ling (haibin.ling AT stonybrook.edu), NCS 147

        Office hour: Wed 3pm-5pm or by appointment

        TAs:

o   Zihan Ding (Zihan.Ding AT stonybrook.edu), 3:00-5:00pm Thursday or by appointment

o   Kalyan Garigapati (Kalyan.Garigapati AT stonybrook.edu), 7:00-9:00pm Thursday or by appointment

o   Ruoyu Xue (ruoxue AT cs.stonybrook.edu), 1:15-3:15pm Wednesday or by appointment

        Syllabus: PDF

Useful Links:

        Main textbook: Computer Vision: Algorithms and Applications, Richard Szeliski, 2nd ed., 2022.

        Computer Vision: A Modern Approach, David Forsyth and Jean Ponce, 2nd ed., 2012.

        Multiple View Geometry in Computer Vision, Richard Hartley and Andrew Zisserman, 2nd ed., 2004.

        Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006.

        Monocular Model-Based 3D Tracking of Rigid Objects: A Survey, Vincent Lepetit, Pascal Fua, 2005.

 

Course Schedule (tentative, will be actively updated during semester)

Date

Topics

Materials

Week 1

8/28

8/30

        Introduction

o   Topics, history, applications, related fields

o   Course logistics

        Image Formation

o   Geometric primitives and transformations

        Szeliski 1.1, 1.2

        Szeliski 2.1, 2.2

 

Week 2

9/6

        Image Formation

o   Photometric image formation

o   Anatomy of cameras

        Tutorial to Google Colab (By Kalyan Garigapati)

        Quiz 1

        Szeliski 2.2, 2.3

Week 3

9/11

9/13

        Image Formation (cont)

o   Photometric image formation

o   Anatomy of cameras

        Image Processing

o   Linear filtering: separable filtering

o   Non-linear filtering, bilateral filtering

        Szeliski 2.2, 2.3

        Szeliski 3.2, 3.3

 

Week 4

9/18

9/20

        Image Processing

o   Fourier transforms

o   Image pyramid

o   Geometric transform

        Model Fitting and Optimization

o   Line fitting, robust fitting

o   Hough transform

o   RANSAC

        Szeliski 3.4, 3.5, 3.6

        Szeliski 4.1

        Szeliski 7.4, 8.1

 

Week 5

9/25

9/27

        Machine Learning

o   Supervised learning

o   Unsupervised learning

        Recognition

o   Instance recognition

o   Image classification (traditional methods)

        Szeliski 5.1, 5.2

        Szeliski 6.1, 6.2

10/1

        Homework 1 Due (11:59pm EST)

 

Week 6

10/2

10/4

        Deep Learning

o   Deep neural networks

o   Convolutional neural networks

o   Architecture

o   Applications

        Deep Learning Practice

o   Training

o   Data augmentation

        Szeliski 5.3, 5.4, 5.5, 6.2

        Deep Learning, Goodfellow et al., 2016.

        Deep Learning Tutorial, Stanford

Week 7

10/11

Midterm

 

Week 8

10/16

10/18

        Image classification

o   Vision Transformer

        Detection

o   Traditional solutions

        Szeliski 5.5, 6.3

        End-to-End Object Detection with Transformers, Carion, et al., ECCV 2020

Week 9

10/23

10/25

        Detection

o   Deep learning solutions

        Semantic Segmentation

o   Segmentation

        Szeliski 6.3, 6.4

        Segment Anything, Kirillov et al., 2023

 

10/29

Homework 2 Due (11:59pm EST)

 

Week 10

10/30

11/1

        Semantic Segmentation

o   Segmentation

o   Semantic segmentation

        Szeliski 6.3, 6.4

        Segment Anything, Kirillov et al., 2023

Week 11

11/6

11/8

        Feature detection and matching

o   Points and patches

o   Edges and contours

        Szeliski 7.1, 7.2, 7.5

Week 12

11/13

11/15

 

        Image alignment and stitching

o   Pairwise alignment

o   Image stitching

o   Global alignment

        Motion estimation and tracking

o   Translational alignment

o   Optical flow

o   Object tracking

        Szeliski 8.1, 8.2, 8.3

        Szeliski 9.1, 9.3, 9.4

 

Week 13

11/20

        Motion estimation and tracking

o   Object tracking

        Szeliski 9.1, 9.3, 9.4

11/26

Homework 3 Due (11:59pm EST)

 

Week 14

11/27

11/29

        Depth estimation

o   Epipolar geometry

o   Correspondences

o   Local and global methods

o   Deep learning solution

 

        Quiz 2 (11/29)

        Szeliski 12.1, 12.2, 12.3, 12.4, 12.5, 12.6

 

Week 15

12/4

12/6

        Structure from motion and SLAM

o   Camera pose estimation

o   Two-frame structure from motion

o   Multi-frame structure from motion

o   Simultaneous localization & mapping (SLAM)

        Diffusion model for semantic segmentation (by Neelesh Verma, 12/6)

        Final review (12/6)

        Szeliski 11.1, 11.2, 11.3, 11.4, 11.5

 

Week 16

12/11

        Advanced topic

o   Object pose estimation and application (by Ruyi Lian)

 

12/17

Homework 4 Due (11:59pm EST)

 

12/20

Final Exam, 5:30-8:00pm, Frey Hall 102