We are going to take “get your hands dirty” approach, you will be given assignments and projects to implement ideas discussed in the class. Projects and assignments will contain miniature versions of real life existing applications and problems (e.g can you train your computer to generate dialogues in Shakespeare style or convert your image into painting as done by Monet, sentiment analysis etc.. ).
Course will concentrate in developing both mathematical knowledge and implementation capabilities. We will start from training a single perceptron, move to training a deep neural network, study why training large networks is a problem and what could be its possible solutions. After dipping our toes in deep belief networks and recurrent neural network we will start looking into applications of deep learning in three different areas, textanalysis, speech processing and computer vision. Objective of this approach is to make you comfortable enough that you can understand various research problems and, if interested, can implement deep learning based applications.
In last few years machine learning has matured from the science fiction to reality. We are living in a world where we have already seen industry bringing to reality selfdriving cars, facerecognizers that work on massive scale (facebook), speech translation systems that can translate from one language to many other simultaneously and in realtime, and more interestingly we have machines that can learn to play atari games in a similar fashion like we do.
A lot of these victories have come from the exciting field of Deep Learning; a learning methodology based on the concept that human mind captures details at multiple levels or at multiple abstract levels. One property of deep learning is removing the responsibility of humans to design features, instead Deep Learning is given a task to find the appropriate representation.
Instructor: Dr. Mohsen Ali
2:00 pm to 4:00 pm Tuesday & Thursday (or by appointment)
Email: mohsen.ali@itu.edu.pk
Teaching Assistant: : Ali Hassan (TA Hours: Fridays 9:30 am — 11:30am)
Email: mscs15049@itu.edu.pk
Teaching Assistant: : Afsheen Rafaqat Ali (volunteer)
Email: afsheen.ali@itu.edu.pk
Probability, linear algebra, programming ability and desire to read & implement.
All cases of academic misconduct will be forwarded to the disciplinary committee. All assignments are individual unless explicitly specified by the instructor. In the words of Efros, let’s not embarrass ourselves.
Lecture  Topic  Assignment 

Feb 2, 2017  Introduction to Deep Learning  
Feb 3, 2017  Linear Regression Least Squares Estimation 

Feb 7, 2017  Linear Classifier Perceptron Rule Sigmoid Perceptron Reading: 

Feb 9, 2017  Logistic Regression Concepts of local minima, local maxima, convex functions, concave functions, critical points Error Function of Linear Regression and convexity A simple code for the Perceptron 

Feb 14, 2017  Gradient descent: 

Feb 16, 2017  Back Propagation1 Reading:


Feb 21, 2017  Back Propagation2  
Feb 28, 2017  How to Train Neural Network Reading: 

Mar 2, 2017  Representational or Expressive Power of Deep Neural Network (Section 6.4 Deep Learing, by Good Fellow, Bengio & Courville) Universal Approximation Properties of Single Layer Neural Network Problems in training, generalization and efficiency Does increasing depth always results in much better generalization? 

Mar 7, 2017  Convolutional Neural Networks Video Lecture : Reading : 

Mar 9, 2017  CNN, Regularizations and Updation Rules Reading: Things to Take Care of While implementing DNN 

Mar 14, 2017  Network Visualization  
Apr 4, 2017  Object Localization and Detection Regression for the localization Concept of object proposals RCNN/FastRCNN/FasterRCNN YOLO 

Apr 6, 2017  Object Localization Reading & Resources 

Apr 11, 2017  RNN LSTM Image Caption Generation using LSTM 

Apr 13, 2017  Image Segmentation Architectures Concepts of Dropout, connect it to visualization 

Apr 18, 2017  Presentation on LipNet: EndtoEnd Sentencelevel Lipreading Presented by Sanullah Manzoor PhdCS16003 

Apr 20, 2017  Slides Hugo Larochelle’s lectures are treat 

Apr 25, 2017  Generative Adversarial Networks Paper An introduction to Generative Adversarial Networks (with code in TensorFlow) Introductory guide to Generative Adversarial Networks (GANs) and their promise! 

Apr 27, 2017  Understanding different architectures used for the Video Analysis 