Depth First Learning Fellowship on Normalizing Flows with Variational Inference

Depth First Learning ( http://depthfirstlearning.com/ ) is an initiative aiming to develop curricula leading to mastery of specific important papers in machine learning.  These curricula are developed by running online study groups via videoconferencing on the following topics.

– Variational Inference with Normalizing Flows
– Spherical CNNs
– Stein Variational Gradient Descent
– Resurrecting the sigmoid in deep learning through dynamical isometry

About the Fellow

Dr Steve Kroon obtained MCom (Computer Science) and PhD (Mathematical Statistics) degrees while studying at Stellenbosch University. He joined the Stellenbosch University Computer Science department in 2008. His PhD thesis considered aspects of statistical learning theory, and his subsequent research has focused on decision making in artificial intelligence, including machine learning, reinforcement learning, and search techniques. He has supervised and co-supervised 5 graduated and 3 current master’s students, and has published 3 journal articles and 8 peer-reviewed conference and conference workshop articles. He has served as a reviewer for the journals Algorithmica, the Journal of Universal Computer Science, and the South African Computer Journal, as well as on the programme committee for 2 conferences. He holds a Diploma in Actuarial Techniques, and is a member of the Centre for Artificial Intelligence Research, the Institute of Electrical and Electronics Engineers (IEEE) and the IEEE Computational Intelligence Society, the International Computer Games Association, the South African Statistical Association, and the South African Institute for Computer Scientists and Information Technologists.

Programme

I’ll be running a  curriculum (for 6 weeks, starting later this month) on normalizing flows with variational inference.  We would like to include people from Africa (especially beyond South Africa) in the group, so please sign up (by clicking the register button) if you’re interested.

Time

There will be weekly videoconferencing sessions of 1.5 hours, so
you’ll need a fairly good internet connection.  Possible times, not
yet finalized (please let me know in your email which ones you are
available for):
(a) Mon 12:30-14:00 South Africa Time
(b) Wed 12:30-14:00 South Africa Time
(c) Wed 15:00-16:30 South Africa Time

You would need to commit around 6 hours per week for prescribed
reading and exercises.

Rough prerequisites for the course:

• Programming in Python and NumPy.  (Experience in a framework such as
  PyTorch or TensorFlow recommended – you will likely need to edit or
  write code in these.)
• Basic computer science: order complexity, DAGs
• Vector calculus: gradient, Jacobian, convexity, Jensen’s inequality
• Linear algebra: rank, determinant
• Basic statistics: understand probability densities, joint and
  conditional distributions, change of variable formula, some common
  distributions (Bernoulli, Categorical/multinoulli, binomial,
  (multivariate) Gaussian, Beta), maximum likelihood estimation
  Basics of neural networks, e.g. multi-layer perceptron, activation
  functions (ReLU/tanh), loss functions (squared error/cross-entropy),
  stochastic gradient descent, learning rate
  Familiarity with LaTeX (for scribing notes from sessions)