Readings

Week 2

• Paper 1: Foundations & Recent Trends in Multimodal Machine Learning Definitions, Challenges, & Open Questions - Sections 2 and 3 in particular
• Paper 2: Representation Learning: A Review and New Perspectives - Sections 1-3, 6-8, 11
• Paper 3: Experience Grounds Language
• Paper 4: Pragmatics in Language Grounding

Week 3

Unimodal representations

• Paper 1: SimCSE: Simple Contrastive Learning of Sentence Embeddings
• Paper 2: LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
• Paper 3: Visualizing and Understanding Convolutional Networks
• Paper 4: Masked Autoencoders As Spatiotemporal Learners

Multimodal Representations

• Paper 5: Learning Transferable Visual Models From Natural Language Supervision
• Paper 6: On Deep Multi-View Representation Learning: Objectives and Optimization
• Paper 7: Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• Paper 8: Sigmoid Loss for Language Image Pre-Training