Foundations of Probabilistic Modeling — Spring 2023

Instructor: Professor Adji Bousso Dieng (adji@princeton.edu)
Course Assistant: Gianluca Bencomo (gb5435@princeton.edu)
Day and Time: Mondays and Wednesdays 3:00 – 4:20 PM EST
Location: Friend Center 004
Website: https://cos513.princeton.edu

Office Hours:

• Instructor: Fridays 9:00 – 10:00 AM EST (Room CS 406, 35 Olden Street)
• TA: Wednesdays 1:00 PM – 3:00 PM EST (Friend Center Graduate Student Lounge)

Programming Help Sessions:

• TA: Wednesdays 1:00 PM – 3:00 PM EST (Friend Center Graduate Student Lounge, 1 hour during TA office hours)

In this course, we will study foundational topics of probabilistic machine learning.

This course covers fundamental topics in probabilistic modeling and allows you to contribute to this important area of machine learning and apply it to your work. We learn how to model data arising from different fields and devise algorithms to learn the structure underlying these data for the purpose of prediction and decision making. We cover several model classes–including deep generative models–and several inference algorithms, including variational inference and Hamiltonian Monte Carlo.

We’ll cover the following topics in the course

• A review of probability, basics of Bayesian statistics, and maximum likelihood
• Simplest probabilistic model, conjugate priors
• Conditional models: linear regression, logistic regression
• Bayesian mixture models, Gibbs sampling, Metropolis Hastings
• Mixed membership models, variational inference
• Matrix factorization
• The exponential family, generalized linear models
• Neural networks, Bayesian neural networks
• Deep latent variable models, amortized inference, reparameterization
• Variational Auto-Encoders (VAEs)
• Hamiltonian Monte Carlo, application to VAEs

Useful References There is no required reading for the course. Useful optional readings will be posted on the website as the semester progresses.

This is a graduate-level course open to PhD students. Master’s students with the right preparation can enroll after receiving permission from the course instructor. This course is not open to undergraduate students.

The course assumes you have had an introduction to probability, statistics, optimization, linear algebra, and machine learning. These topics are covered in COS 302 / SML 305 and COS 324 at Princeton.

Some of the assignments and the final project will require coding skills. Familiarity with Python, Git/Github, and automatic differentiation tools such as Jax or PyTorch will be useful. Please reach out to the instructor if you have concerns about your fit for the course before making your decision to enroll.

There will be two lectures every week, lasting 80 minutes each. The lecture slides will be posted on canvas before or right after every lecture. The course lectures are complemented with weekly programming help sessions during TA office hours. These help sessions are meant to introduce you to tools for probabilistic modeling and help you with coding-related issues for your final project. Attendance to these help sessions is optional but strongly recommended.

There will be four assignments and one final project. The final project can be about a theoretical contribution, a new model for a dataset that is of interest to you, a new methodology to learn from a certain type of data, a solution to an existing problem in the probabilistic modeling literature, a purely empirical study about existing probabilistic modeling tools, new ways to evaluate probabilistic models, new methodology for uncertainty estimation, etc. All reports related to the final project will use the NeurIPS format. The template is posted on canvas. There will be no in-class final exam and no in-class midterm exam.

You will be able to ask questions to your fellow classmates and course instructors in Ed and Canvas. You will use Gradescope to submit homework assignments and project reports.

Grading policy: The assignments will each be worth 10% of your final grade while the final project will account for 55% of your final grade. Course attendance and participation will carry 5% of your final grade.

The final project will be graded in three phases: a project proposal, a project presentation during a course poster session, and a final project report.

Your final project will be evaluated for its relevance to the course, technical correctness, and quality of writing.

Regrading policy: If you judge that we made a mistake grading your work, please submit a regrading request through Gradescope. Your grade may be updated or not depending on your request.

Late work policy: You are required to submit assignments and project reports on time. However, you are given four free late days to use during the semester, with at most two late days per assignment. Work submitted more than two days after the original deadline will not be graded. No late day can be applied to the submission of your final report; the final report should be submitted on Dean’s Date without exception. A final report submitted after Dean’s date will not be graded.

Collaboration policy: Assignments and final projects should be completed individually. You are not allowed to copy-paste solutions from your classmates or from an external source (e.g. ChatGPT). However, you are allowed to consult each other to discuss assignments and projects and learn from each other.

Auditing: Auditing is not allowed for this course.

Communication: You are required to communicate respectfully with your fellow classmates. Use their preferred names and pronouns if and when provided.

Honor Code: This course abides by Princeton University’s honor code, in addition to all the policies mentioned above.