CSE 527: Introduction to Computer Vision

(Graduate Course)

Fall 2024

Basic Information:

· Lecture info: Mon/Wed 6:30pm – 7:50pm, LGT ENGR LAB 102

· Instructor: Haibin Ling (haibin.ling AT stonybrook.edu), NCS 147

· Office hour: Wed 9am-11am or by appointment

· TAs:

o Md Shahrar Fatemi (mfatemi AT cs.stonybrook.edu), office hour: Thu 4-6pm

o Snehal Tomar (stomar AT cs.stonybrook.edu), office hour: Fri 12-2pm

o Yufeng Wang (yufengwang AT cs.stonybrook.edu), office hour: Thu 3:30-5:30pm

· Syllabus: PDF

Useful Links:

· Main textbook: Computer Vision: Algorithms and Applications, Richard Szeliski, 2nd ed., 2022.

· Computer Vision: A Modern Approach, David Forsyth and Jean Ponce, 2nd ed., 2012.

· Multiple View Geometry in Computer Vision, Richard Hartley and Andrew Zisserman, 2nd ed., 2004.

· Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006.

· Monocular Model-Based 3D Tracking of Rigid Objects: A Survey, Vincent Lepetit, Pascal Fua, 2005.

Course Schedule (tentative, will be actively updated during semester)

Date	Topics	Materials
Week 1 8/26 8/28	· Introduction o Topics, history, applications, related fields o Course logistics · Image Formation o Geometric primitives and transformations	· Szeliski 1.1, 1.2 · Szeliski 2.1, 2.2, 2.2, 2.3
Week 2 9/4	· Review of math and coding (by Yufeng Wang) o Basic math concept review o Tutorial to Google Colab
Week 3 9/9 9/11	· Image Formation o Photometric image formation · Guest lecture (9/11) by Prof. Chenyu You o Robust Machine Learning for Biomedical Data: Efficiency, Reliability, and Generalizability	· Szeliski 2.2, 2.3 ·
Week 4 9/16 9/18	· Image Processing o Linear filtering: separable filtering o Non-linear filtering, bilateral filtering o Fourier transforms o Image pyramid o Geometric transform · Model Fitting and Optimization o Line fitting, robust fitting o Hough transform o RANSAC	· Szeliski 3.2, 3.3, 3.4, 3.5, 3.6 · Szeliski 4.1 · Szeliski 7.4, 8.1
9/22	· Homework 1 Due (11:59pm EST)
Week 5 9/23 9/25	· Machine Learning Overview o Supervised learning o Unsupervised learning · Recognition o Instance recognition o Image classification (traditional methods)	· Szeliski 5.1, 5.2 · Szeliski 6.1, 6.2
Week 6 9/30 10/2	· Deep Learning o Deep neural networks o Convolutional neural networks o Architecture o Applications	· Szeliski 5.3, 5.4, 5.5, 6.2 · Deep Learning, Goodfellow et al., 2016. · Deep Learning Tutorial, Stanford
Week 7 10/7 10/9	· Deep Learning Practice (by Tao Sun) o Training o Data augmentation · Transformer & Vision Transformer (by Tao Sun) o Transformer o Vision Transformer	· Szeliski 5.5, 6.3
Week 8 10/16	· Detection o Traditional solutions o Deep learning solutions
10/20	Homework 2 Due (11:59pm EST)
Week 9 10/21 10/23	· Midterm (10/21) · Detection and Segmentation o Detection with Transformers o Segmentation	· Szeliski 6.3, 6.4 · End-to-End Object Detection with Transformers, Carion, et al., ECCV 2020
Week 10 10/28 10/30	· Semantic Segmentation o Segmentation - conventional o Semantic segmentation – deep learning solutions	· Szeliski 6.3, 6.4 · Segment Anything, Kirillov et al., 2023
Week 11 11/4 11/6	· Feature detection and matching o Points and patches o Edges and contours · Motion estimation and tracking o Review of alignment o Optical flow	· Szeliski 7.1, 7.2, 7.5
11/10	Homework 3 Due (11:59pm EST)
Week 12 11/11 11/13	· Depth estimation o Epipolar geometry · Guest lecture (11/13) o 6D pose estimation by Ruyi Lian	· Szeliski 8.1, 8.2, 8.3 · Szeliski 9.1, 9.3, 9.4
Week 13 11/18 11/20	· Depth estimation o Correspondences o Local and global methods o Deep learning solution · Motion estimation and tracking o Object tracking	· Szeliski 9.1, 9.3, 9.4
Week 14 11/25	· Structure from motion and SLAM o Two-frame structure from motion o Multi-frame structure from motion o Simultaneous localization & mapping (SLAM)	· Szeliski 12.1, 12.2, 12.3, 12.4, 12.5, 12.6
12/3	Homework 4 Due (11:59pm EST)
Week 15 12/2 12/4	· Generative models o Autoencoder o Generative adversarial network o Diffusion models · Final review (12/2)	· Szeliski 11.1, 11.2, 11.3, 11.4, 11.5
Week 16
12/9	Final Exam, 6:30-9:00pm LGT ENGR LAB 102