CSE 519 - Data Science

Fall 2018

Data Science is a rapidly emerging discipline at the intersection of statistics, machine learning, data visualization, and mathematical modeling. This course is designed to provide a hands-on introduction to Data Science by challenging student groups to build predictive models for upcoming events, and validating their models against the actual outcomes.

• Course Time: 8:30-9:50AM Tuesday and Thursday
  Place: 102 Frey Hall
• Steven Skiena's office hours are 2:30AM-4PM Tuesday-Thursday, in 251 New Computer Science, and by appointment.
• The course teaching assistants will be:
  • Allen Kim. His email address is allen.kim@stonybrook.edu. He will have office hours Monday and Wednesday 2:30PM-4PM in room 340 New Computer Science.
  • Harsh Agarwal. His email address is hagarwal@cs.stonybrook.edu. He will have office hours Thursday 2-4PM in the TA room 2217 in Old Computer Science.
  • Shouvik Roy. His email address is shroy@cs.stonybrook.edu. He will have office hours Tuesday 3-4:30PM and Wednesday 4-5:30PM in the TA room 2206 Old Computer Science.
  • Raveendra Soori. raveendra.soori@stonybrook.edu. He will have office hours Tuesday and Thursday form 1:30-3PM in the TA rooms 2203 and 2217 in Old Computer Science.
  • Sahil Sobti. ssobti@cs.stonybrook.edu. He will have office hours Thursday 2-4PM in the TA rooms 2203 Old Computer Science.
• Videos and slides from my Fall 2016 lectures is available here. The video from Fall 2017 appears here , but the quality is not good. The best stuff should always be available at www.data-manual.com.
• Sign up for the Piazza class discussion board at https://piazza.com/stonybrook/fall2018/cse519.
• Syllabus
• Lecture Schedule

Textbook

We will use my new book The Data Science Design Manual, Springer-Verlag, 2017.The associated website www.data-manual.com points to many resources, including lecture notes/videos, errata, a problem solution Wiki, and sample Python notebooks for generating figures from the book.

I will welcome feedback on the book. Please keep track of errata in the book send them to me, ideally in one batch at the end of the semester.

Homework Assignments

• Homework 1 DUE 9/6/18 -- The course videos are available from www.quant-shop.com and the Quant Shop Vimeo channel.
• Homework 2 DUE 9/25/18 -- This assignment revolves around the Kaggle New York City Taxi Fare Prediction challenge.
• Project test run Drawings are DUE 10/4/18 with peer grading DUE 10/11/18.
• Homework 3: Due 10/16/18 -- Data integration and modeling, this time around the Kaggle Google Analytics Customer Revenue Prediction challenge.
• Course projects : proposals due 10/23/18, progress report due 11/15/18, and final report/poster due 12/6/18.

  Here is an example of a project proposal from last year which can serve as a model. For many projects, however, having some prelimnary data explorations would be better than such a detailed description of future approach.

Lecture Notes

I will give about 25 formal lectures this semester. All classes will be filmed by Echo360 and made available on Blackboard.

Ritika Nevatia is graciously making the lecture notes she takes in class available to all interested students. Check them out.

• Lecture 0: Course Administration
• Lecture 1: Introduction to Data Science
• Lecture 2: Mathematical Preliminaries
• Lecture 3: Python for Data Science I Lecture 4: Python for Data Science II
• Lecture 5: Correlation
• Lecture 6: Assembling Data Sets
• Lecture 7: Data Cleaning
• Lecture 8: Scores and Rankings
• Lecture 9: Statistical Distributions
• Bonus: HW Figures, Good and Bad
• Lecture 10: Statistical Significance
• Lecture 11: Principles of Visualizing Data
• Lecture 12: Practice of Data Visualization
• Lecture 13: Building Models
• Lecture 14: Validating Models
• Bonus: Ranking Drawings by Peer Reviews
• Lecture 15: Linear Algebra Review
• Bonus: Project Proposal Comments
• Lecture 16: Linear Regression
• Lecture 17: Gradient Descent Search and Regularization
• Lecture 18: Logistic Regression and Classification
• Lecture 19: Nearest Neighbor Methods
• Lecture 20: Clustering
• Bonus: Comments about Progress Reports
• Lecture 21: Introduction to Machine Learning
• Lecture 22: Topics in Machine Learning
• Lecture 23: Achieving Scale
• Lecture 24: Human-centric Data Science

Old lecture notes are available from the previous offering in Fall 2014.

Semester Projects

Roughly half of the course grade will come from a course project. Students will typically work in small groups (2-3 people) on independent research projects. I will distribute a list of possible projects about six weeks into the semester. You will be encouraged to develop your own project ideas, although I must approve.

Recommended Readings

The field of data science is still emerging, but there are several books which it will be useful to read and consult:

• Python for Data Analysis, by Wes McKinney, O'Reilly Media, 2013 -- This book is a nuts and bolt's guide to data wrangling with Python, including such tools/libraries as Pandas, NumPy, and IPython. You will be expected to use these tools in doing your course project.
• The Signal and the Noise: Why so many predictions fail but some don't, by Nate Silver, Penguin Press, 2012 -- This popular, easy-to-read book focuses on how effectively data can be used to make predictions in domains like sports, science, economics, and politics. This is exactly what we are trying to do in this course, and Silver's book is an excellent model to build on.

Videos: The Quant Shop

The Quant Shop is a series of eight 30 minute programs on Data Science, which are a product of the Fall 2014 offering of this course. Watch them for inspiration at the Quant Shop Vimeo channel.

Related Links

• Google makes it easier to search for datasets: https://toolbox.google.com/datasetsearch
• My algorithms textbook is the way to get a job at Google
• Bing Predicts uses search queries and other modeling to predict the outcome of a variety of events.
• Results from Zillow Challenge.
• CS109 Data Science, Harvard University, Fall 2015 -- This course stresses statistical modeling and Python programming. Very interesting, well thought-out assignments

Professor

Steven S. Skiena

251 New Computer Science Building

Department of Computer Science

Stony Brook University

Stony Brook, NY 11794-2424, USA

skiena@cs.stonybrook.edu

631-632-9026