Deep Learning – Spring 2021
Class HoursTue: 5:30 pm — 7:00 pm, Thur: 7:15 pm — 8:45 pm Office Hours and Contact Info.Instructor: Mohsen Ali Teaching Assistant 1: Obaid Ullah Ahmad Teaching Assistant 2: Hafiz Muhammad Abdullah Zia Teaching Assistant 3: Muhammad Hamad Akram 
Course BasicsCore Course PrerequisiteEnthusiasm, Energy, and Imagination 
Course Overview
We are going to take the “get your hands dirty” approach, you will be given assignments and projects to implement ideas discussed in the class. Projects and assignments will contain miniature versions of reallife existing applications and problems (e.g can you train your computer to generate dialogues in Shakespeare style or convert your image into painting as done by Monet, sentiment analysis, etc.. ).
The course will concentrate on developing both mathematical knowledge and implementation capabilities. We will start from training a single perceptron, move to training a deep neural network, study why training large networks is a problem and what could be its possible solutions. After dipping our toes in deep belief networks and recurrent neural network we will start looking into applications of deep learning in three different areas, textanalysis, speech processing, and computer vision. The objective of this approach is to make you comfortable enough that you can understand various research problems and, if interested, can implement deep learningbased applications.
Course Objectives
In the last few years, machine learning has matured from science fiction to reality. We are living in a world where we have already seen industry bringing to reality selfdriving cars, facerecognizers that work on a massive scale (Facebook), speech translation systems that can translate from one language to many other simultaneously and in realtime, and more interestingly we have machines that can learn to play atari games in a similar fashion as we do.
A lot of these victories have come from the exciting field of Deep Learning; a learning methodology based on the concept that the human mind captures details at multiple levels or at multiple abstract levels. One property of deep learning is removing the responsibility of humans to design features, instead, Deep Learning is given a task to find the appropriate representation.
Grading Policy
 45% Assignments
 5% Class participation and Creating Notes
 20% Final Project
 10% Quizzes
 10% Midterm Exam
 10% Final Exam
Honor Code
All cases of academic misconduct will be forwarded to the disciplinary committee. All assignments are groupbased unless explicitly specified by the instructor. In the words of Efros, let’s not embarrass ourselves.
Tentative and Rough Course Outline
Weeks  Topics  Evaluations 
1  Introduction to Deep Learning
Difference between Machine Learning and Deep Learning Basic Machine Learning: Linear & Logistic Regression, 

2  Supervised Learning with Neural Networks
Deep Learning, Single and MultiLayer Neural Networks, Perceptron Rule, Gradient Descent, Backpropagation, Loss Functions Tutorial 1: Python/Numpy Tutorial/Anaconda 

3  Hyperparameters tuning, Regularization and Optimization
Parameters vs Hyperparameters, Why regularization reduces overfitting? Data Augmentation, Vanishing/Exploding gradients, Weight Initialization Methods, Optimizers Tutorial 2: Building a Linear Classifier 

4  Convolutional Neural Networks
Convolutional Filters, Pooling Layers, Classic CNNs: AlexNet, VGG, GoogleNet, ResNet, DenseNet. Transfer Learning Tutorial 4: CNN Visualization 

5  Deep Learning for Vision Problems
Object Localization & Detection, Bounding box predictions, Anchor boxes, Region Proposal Networks, Detection Algorithms: RCNN, Faster RCNN, Yolo, SSD. Tutorial 5: Caffe & Object Detection 

6  Sequence Models
Recurrent Neural Networks (RNN), Gate Recurrent Unit (GRU), Long Short Term Memory (LSTM), Bidirectional RNN, Backpropagation through time. Image Caption Generation, Machine Translation, Text Generation & Summarization and Transformers Tutorial 6: Image Captioning & Text Generation 

7  AutoEncoders & Generative Models
Variational AutoEncoders, Stacked AutoEncoders, Denoising AutoEncoders, Concept of Generative Adversarial Networks (GANs) 

8  Miscellaneous
Capsule Networks, Convolutional LSTM, Attention Networks, Restricted Boltzmann Machine, OneShot Learning, Siamese Networks, Triplet Loss, Graph CNN, Approximate and EnergyEfficient Design for Deep CNN (Dr. Rehan Hafiz) 
Course Notes
Topics  Notes / Reading Material / Comments  News  
09th Mar 2021  Introduction  45% Assignments
20% Final Project 5% Class participation and Creating Notes 10% Quizzes 10% Midterm Exam 10% Final Exam 

Recommended Resources 


11th Mar 2021  UnSupervised and Supervised Learning 
Assigned Readings:
Recommended Readings:
Refresh: Concepts of local minima, local maxima, convex functions, concave functions, critical points, chain rule, saddle point 

16th Mar 2021  No Class  Makeup class will be announced soon  
18th Mar 2021  No Class  Makeup class(es) will take place on Saturday 27th March, 2021  
23rd Mar 2021  No Class on account of Pakistan Resolution Day  Makeup class will be announced soon  
25th Mar 2021  Linear Regression 

Assignment 1 
27th Mar 2021  Optimization, Gradient Descent and Logistic Regression 


30th Mar 2021  Logistic Regression, Classification, Loss Functions 
Take home task
Assigned Readings:
Recommended Reading: Softmax and Cross Entropy Assigned Readings (Linear Algebra): 

1st Apr 2021  Multi Class Classification, Optimization and Gradient Descent 
Recommended Readings: Recommended Readings: Gradient Descent: Video Lecture from Coursera, Andrew NG 

2nd Apr 2021  Neural Networks 
Assigned Readings:
Upto complete section 6.3. Recommended Readings: Perceptron Rule 

3rd Apr 2021  Backpropagation 
Assigned Readings:
Recommended Readings
Optional: How to do backpropagation in a brain by Hinton Video Lecture: Lecture 4: Backpropagation; Dhruv Batra 

6th Apr 2021  Neural Network Training 
Assigned Readings:

Assignment 2 
8th Apr 2021  Weight Initialization and Batch Normalization 
Assigned Readings Recommended Reading: 

13th Apr 2021  Regularization and Dropout 
Home Work:
Assigned Readings: Video Lectures
Recommended Readings: 

15th Apr 2021  Texture and Convolution filters 


20th Apr 2021  Filters and Convolutional Neural Networks 
NOTES Assigned Readings Recommended Readings 

22th Apr 2021  CNNs 
Assigned Readings 

27th Apr 2021  Back Propagation in CNNs 
Reference
Video Lecture: 
Assignment 3 Deliverable 2 
29th Apr 2021  Transfer Learning 

Assignment 3 Deliverable 1 
4th May 2021  Transfer Learning and CNN Architectures 
Assigned Reading
Recommended Readings


6th May 2021  Semantic Segmentation 
Assigned Reading Recommended Readings 

18th May 2021  Localization and Object Detection 
Recommended Readings 

20th May 2021  Object Detection, Recurrent Neural Networks 
Assigned Reading Sequence Modeling: Recurrent and Recursive Nets, Chapter 10 from textbook. Recommended Readings 

Text Book
 Text Book: Deep Learning by Ian Goodfellow Link
 Reference Book: Dive into Deep Learning by Aston Zhang and co Link
Recommended Readings
Following are recommended for reading.
Toolkits
PyTorch 
Top Conferences to Follow
 International Conference on Machine Learning (ICML)
 Conference on Neural Information Processing Systems (NIPS)
 International Joint Conference on Artificial Intelligence (ICAI)
 Conference on Computer Vision and Pattern Recognition (CVPR)
 International Conference on Computer Vision (ICCV)
 British Machine Vision Conference (BMVC)
Assignments

Assignment 1: Classification of MNIST Digits Using Pytorch:

Assignment 2: Implementation of Neural Network:

Assignment 3: Implementation of Convolutional Neural Network:
 Assignment 3 Deliverable 1: Dry run by hand
 Assignment 3 Deliverable 2: Implementation with python language
Some Interesting Links
 Linear algebra review/primer by Martial Hebert
 Some of the research groups working with commercial entities
 Machine Learning Group – Geoffrey Hinton
 New York University – Yann Lecun
 Stanford University – Andrew Ng, FeiFei Li‘s groups
 Microsoft Research
 Google DeepMind – Alex Graves
 Amazon Research
Projects
The project list of the students will be shared here.