Week 2:

  1. Paper A: Foundations & Recent Trends in Multimodal Machine Learning Definitions, Challenges, & Open Questions - Section 1, Section 2, Section 3

  2. Paper B: Foundations & Recent Trends in Multimodal Machine Learning Definitions, Challenges, & Open Questions - Section 1, Section 2, Section 4

  3. Paper C: Foundations & Recent Trends in Multimodal Machine Learning Definitions, Challenges, & Open Questions - Section 1, Section 2, Section 5

  4. Paper D: Foundations & Recent Trends in Multimodal Machine Learning Definitions, Challenges, & Open Questions - Section 1, Section 2, Section 6

  5. Paper E: Foundations & Recent Trends in Multimodal Machine Learning Definitions, Challenges, & Open Questions - Section 1, Section 2, Section 7
  6. Paper F: Foundations & Recent Trends in Multimodal Machine Learning Definitions, Challenges, & Open Questions - Section 1, Section 2, Section 8

Week 3:

  1. Zeiler and Fergus, Visualizing and Understanding Convolutional Networks. ECCV 2014
  2. Selvaraju et al., Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. ICCV 2017
  3. Karpathy et al., Visualizing and Understanding Recurrent Networks. arXiv 2015
  4. Khandelwal et al., Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context. ACL 2018
  5. (optional reading) Learning Translation Invariance in CNNs

Week 5: For this week, you are expected to read one of the following papers:

  1. Paper A: Multimodal Learning using Optimal Transport for Sarcasm and Humor Detection
  2. Paper B: Deep Multimodal Clustering for Unsupervised Audiovisual Learning
  3. Paper C: On the Benefits of Early Fusion in Multimodal Representation Learning
  4. Paper D: Improving Multimodal fusion via Mutual Dependency Maximisation 

(Optional Readings)

  1. Multiplicative Interactions and Where to Find Them
  2. Self-Supervised Learning from a Multi-View Perspective
  3. Learning Factorized Multimodal Representations