CSE 527: Introduction to Computer Vision

Fall 2023

Basic Information:

· Lecture info: Mon/Wed 7:00pm – 8:20pm, Frey Hall 102 (Map)

· Instructor: Haibin Ling (haibin.ling AT stonybrook.edu), NCS 147

· Office hour: Wed 3pm-5pm or by appointment

· TAs:

o Zihan Ding (Zihan.Ding AT stonybrook.edu), 3:00-5:00pm Thursday or by appointment

o Kalyan Garigapati (Kalyan.Garigapati AT stonybrook.edu), 7:00-9:00pm Thursday or by appointment

o Ruoyu Xue (ruoxue AT cs.stonybrook.edu), 1:15-3:15pm Wednesday or by appointment

· Syllabus: PDF

Useful Links:

· Main textbook: Computer Vision: Algorithms and Applications, Richard Szeliski, 2nd ed., 2022.

· Computer Vision: A Modern Approach, David Forsyth and Jean Ponce, 2nd ed., 2012.

· Multiple View Geometry in Computer Vision, Richard Hartley and Andrew Zisserman, 2nd ed., 2004.

· Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006.

· Monocular Model-Based 3D Tracking of Rigid Objects: A Survey, Vincent Lepetit, Pascal Fua, 2005.

Course Schedule (tentative, will be actively updated during semester)

Date	Topics	Materials
Week 1 8/28 8/30	· Introduction o Topics, history, applications, related fields o Course logistics · Image Formation o Geometric primitives and transformations	· Szeliski 1.1, 1.2 · Szeliski 2.1, 2.2
Week 2 9/6	· Image Formation o Photometric image formation o Anatomy of cameras · Tutorial to Google Colab (By Kalyan Garigapati) · Quiz 1	· Szeliski 2.2, 2.3
Week 3 9/11 9/13	· Image Formation (con’t) o Photometric image formation o Anatomy of cameras · Image Processing o Linear filtering: separable filtering o Non-linear filtering, bilateral filtering	· Szeliski 2.2, 2.3 · Szeliski 3.2, 3.3
Week 4 9/18 9/20	· Image Processing o Fourier transforms o Image pyramid o Geometric transform · Model Fitting and Optimization o Line fitting, robust fitting o Hough transform o RANSAC	· Szeliski 3.4, 3.5, 3.6 · Szeliski 4.1 · Szeliski 7.4, 8.1
Week 5 9/25 9/27	· Machine Learning o Supervised learning o Unsupervised learning · Recognition o Instance recognition o Image classification (traditional methods)	· Szeliski 5.1, 5.2 · Szeliski 6.1, 6.2
10/1	· Homework 1 Due (11:59pm EST)
Week 6 10/2 10/4	· Deep Learning o Deep neural networks o Convolutional neural networks o Architecture o Applications · Deep Learning Practice o Training o Data augmentation	· Szeliski 5.3, 5.4, 5.5, 6.2 · Deep Learning, Goodfellow et al., 2016. · Deep Learning Tutorial, Stanford
Week 7 10/11	Midterm
Week 8 10/16 10/18	· Image classification o Vision Transformer · Detection o Traditional solutions	· Szeliski 5.5, 6.3 · End-to-End Object Detection with Transformers, Carion, et al., ECCV 2020
Week 9 10/23 10/25	· Detection o Deep learning solutions · Semantic Segmentation o Segmentation	· Szeliski 6.3, 6.4 · Segment Anything, Kirillov et al., 2023
10/29	Homework 2 Due (11:59pm EST)
Week 10 10/30 11/1	· Semantic Segmentation o Segmentation o Semantic segmentation	· Szeliski 6.3, 6.4 · Segment Anything, Kirillov et al., 2023
Week 11 11/6 11/8	· Feature detection and matching o Points and patches o Edges and contours	· Szeliski 7.1, 7.2, 7.5
Week 12 11/13 11/15	· Image alignment and stitching o Pairwise alignment o Image stitching o Global alignment · Motion estimation and tracking o Translational alignment o Optical flow o Object tracking	· Szeliski 8.1, 8.2, 8.3 · Szeliski 9.1, 9.3, 9.4
Week 13 11/20	· Motion estimation and tracking o Object tracking	· Szeliski 9.1, 9.3, 9.4
11/26	Homework 3 Due (11:59pm EST)
Week 14 11/27 11/29	· Depth estimation o Epipolar geometry o Correspondences o Local and global methods o Deep learning solution · Quiz 2 (11/29)	· Szeliski 12.1, 12.2, 12.3, 12.4, 12.5, 12.6
Week 15 12/4 12/6	· Structure from motion and SLAM o Camera pose estimation o Two-frame structure from motion o Multi-frame structure from motion o Simultaneous localization & mapping (SLAM) · Diffusion model for semantic segmentation (by Neelesh Verma, 12/6) · Final review (12/6)	· Szeliski 11.1, 11.2, 11.3, 11.4, 11.5
Week 16 12/11	· Advanced topic o Object pose estimation and application (by Ruyi Lian)
12/17	Homework 4 Due (11:59pm EST)
12/20	Final Exam, 5:30-8:00pm, Frey Hall 102