Latest version (in pdf) available on Piazza.

Course Format

The course is expected to be in-person given its discussion-based approach. Students are expected to read papers ahead of each course session, and discussions will be moderated to ensure deep understanding of all papers and critical thinking of the paper’s strengths and possible avenues for future directions and improvements. The course is planned for 6 credit units. Optionally, students can register for 12 credit units, with the expectation to do a comprehensive research project as part of the semester. These course projects are expected to be done in teams, with the research topic to be in the realm of multimodal machine learning and pre-approved by the course instructors.

Website: The main course website is available on online:

https://cmu-multicomp-lab.github.io/adv-mmml-course/spring2022/

Piazza: We will be using Piazza for class communication and announcement. The system is highly catered to getting you help fast and efficiently from classmates and the instructors. Rather than emailing questions to the teaching staff, you are encouraged to post your questions on Piazza. You can post privately to the instructor and TAs through Piazza website.

https://piazza.com/cmu/spring2022/11877/home

Canvas: Students are asked to submit their project assignments through the website Canvas. This platform will be used for grading and to handle any request for re-grading.

https://canvas.cmu.edu/courses/28476

Course Material

Required:

  • Reading material will be based on published technical papers available via the ACM/IEEE/Springer digital libraries or freely available online (e.g., arxiv.org). All CMU students have already free access to these digital archives.
  • For project assignments, previous experience in Python and deep learning (e.g., Pytorch) programming is expected

Course Project Timeline

(This section applies only to 12-unit version of the course. Exact timeline subject to change.)

  • Project preferences (Due Monday 1/24 at 8m ET) – Online form to share your interests about research projects and help with team matching.
  • Pre-proposal (Due Wednesday 1/31 at 8pm ET) – You should have selected your teammates, dataset, and task. Submit a 1-page pre-proposal plan.
  • Proposal and Literature Review (Due Wednesday 2/16 at 8pm ET) -
  • Midterm report (Due Wednesday 3/16 at 8pm ET) – Intermediate report documenting the initial results exploring new research ideas.
  • Final report (Due Monday 5/2 at 8pm ET) – Final report describing explored research ideas, with experimental results and discussion.

Grades

Grading breakdown for 6-unit version (no course project)

  • Reading assignments 40%
  • Participation and discussions 32%

  • Discussion synopsis leads 28%

  • Reading assignments
    • There are a total of 11 reading assignments planned this semester. Each reading assignment will consist of 2 main parts:
    • Assigned reading paper: Reading the assigned papers and summarizing the main take-away points of each paper
    • Research question probes: Reflect on the question probes related to the reading papers and prepare discussion points.
    • The grading for each reading assignment is planned as:
      • 2 points for the take-away points of all assigned reading papers
      • 2 points for the discussion points related to the question probes
      • 1 point for scouting relevant papers, blog posts or other resources
    • Reading assignments will also contain a section for students to share their clarification questions about the reading papers. This section is optional and is not directly graded. This section will be used to help instructors and discussion leads prepare for the main discussion during lecture time.
    • The final score will be computed by taking the top 8 scores, out of 11 reading assignments
      • Students are expected to submit all reading assignments. If a reading assignment is not submitted (see late submission section below for details), then 2 points will be removed from the final score for each missing submission.
  • Participation and discussions
    • A core component of this course is centered around live discussions during the course lecture times. Students are expected to be active participants of these discussions. Discussions will be usually performed in smaller groups (8-10 students per group).
    • Small group discussions will be performed in a round table setting, where all students are given opportunity to share their observations and discussion points.
      • A first part of the discussion will focus on clarifying any questions or misunderstandings related to the two research papers.
      • The main part of the discussion will focus on the research question probes. Each student is expected to actively participate in this discussion.
    • The grading for each discussion session is planned as follows:
      • 2 points for the insight and quality of the shared discussion points
      • 2 points for interactivity and participation as follow-up to other’s questions and suggestions.
    • The final score will be computed by taking the top 8 scores, out of 11 discussion sessions
      • Given the live nature of the discussions, students are expected to attend all discussion sessions.
      • Although the final grades are computed with the top 8 scores, an absences unproperly justified (see Attendance section below for more details about absences) will remove 2 points from the final grade.
  • Discussion synopsis leads
    • 2, 3 or 4 times during the semester (depending on the number of registered students), each student will be scheduled to be leading the effort of summarizing the discussion and create a synopsis.
      • We plan to have 2 discussion synopsis leads per lecture, one lead for each small group.
    • The main tasks of the discussion synopsis leads are
      • Reading support: The discussion synopsis leads are also expected to read the assigned papers with extra details, to assist other students with follow-up questions, when possible.
      • Note-taking: during the discussion sessions, the lead will be in charge to take detailed notes from the discussions. These notes will be shared internally with discussion members, but not shared outside the course.
      • Synopsis: both leads will be tasked to meet and create a coherent synopsis from both discussions. These synopses are planned to be made public on the course website.
    • The grading of each lead assignment is planned as follows:
      • 3 points for interaction and support for other students who have questions regarding the assigned papers
      • 4 points for notes summarizing the observations and points made during small group discussions
      • 7 points for the synopsis created by the two leads to summarize the main take-home messages of these discussions
    • The final score will be computers by taking the top 2 scores, if the student was discussion synopsis lead more than twice during the semester.

Grading breakdown for 12-unit version (with course project)

  • Grading breakdown of the 6-unit version will be scaled to 50%. The second 50% comes from the course project:
    • Pre-proposal and project preferences 5%
    • Proposal and literature review 15%
    • Weekly updates 30%
    • Mid-term report 20%
    • Final report 30%
  • Project preference form
    • This form is designed to help students with the team matching process, for the course research project.
  • Pre-proposal
    • This short 1-page pre-proposal is designed to confirm teams and share initial thoughts about the research project, including which dataset is planned to be used as part of the project
  • Proposal and literature review
    • The proposal report should present the initial research ideas for the course project. Students are expected to explore new research ideas as part of the course research project. The proposal should summarize these research ideas. It should also give an overview of the dataset and main research tasks that will be addressed.
    • An important part of this proposal report will be a detailed literature review, including recent papers and models related to the dataset, research tasks and the new research ideas.
  • Weekly updates
    • Project teams are expected to meet with instructors on a regular basis.
      • Project meetings will be about 20-30 minutes long.
      • Each meeting will be usually with one instructor.
      • Each team should plan to have one project meeting per week.
    • To help streamline the project meetings, team members should prepare some update document before each meeting
      • Each team can decide to use either an online Google Docs or Google Slides for these updates.
      • The goal is to keep these updates informal, with only the main points highlighted in the updates.
        • For example, a bullet list with 3-4 items may be sufficient.
      • The same online document should be used for all weekly meetings, so that instructors can easily review previous updates.
    • Weekly updates will be graded by instructors, 3 points per update meeting.
    • The final score will be computed by taking the top 10 scores for the whole semester.
  • Midterm report
    • The goal of the midterm report is to summarize the current research progress. Students should have started already exploring new research ideas. The midterm report should summarize these initial results and discuss them. This report should also present the updated list of research ideas that the team plan to explore.
  • Final report
    • The final report should follow similar structure of a research paper. It should motivate the problem and research ideas. It should present the novel approaches, describe the experiment and discuss the results.

Notes about absences and late submissions

In general, submitting assignments on time lets the instructional team provide feedback in a more timely and efficient manner. Timely submissions are particularly important for assignments with discussions and peer feedback. Also, it is expected that students will attend the lectures in person (or via Zoom when the course is required to be performed remotely). Live attendance is an essential component of this course given it is centered around live discussions. Given the live nature of the discussions, course lectures will not be recorded.

Medical-related absences: If for medical reason you require some extra time for an assignment or may not be able to attend the lecture in person, please contact instructors as soon as possible (the best option is usually via Piazza) and we will help define a new plan that aligns with your constraints.

  • Absence requests
    • Students should contact instructors as promptly as possible regarding course absences, with a preference before the course lecture itself.
    • If you plan to be absent for more than one course lecture, it will be important to contact instructors as promptly as possible.

Late submission wildcards: We offer students and project teams some late submission wildcards to help deal with potential overlaps with other courses or research deadlines. The details are expressed below:

  • Reading assignment wildcards (3 per students)
    • Each wildcard gives the student a 24-hour extension for the reading assignment deadline.
    • Maximum of 1 wildcard per week
      • This constraint is to ensure that sufficient time is made available to prepare for the Friday discussion.
  • Project assignment wildcards (2 per team)
    • This gives the project team a 24-hour extension for their project assignment deadline.
    • These can be used for proposal, midterm and/or final deadlines
    • It is possible to use 2 wildcards for the same deadline, giving the team 48 hours extension.
    • Teams are required to message instructors via Piazza before the deadline, to inform that they will be using 1 or 2 wildcards.

Statement on Student Wellness

This semester is unlike any other. We are all under a lot of stress and uncertainty at this time. Attending Zoom classes all day can take its toll on our mental health. Make sure to move regularly, eat well, and reach out to your support system or me (morency@cs.cmu.edu) if you need to. We can all benefit from support in times of stress, and this semester is no exception.

As a student, you may experience a range of challenges that can interfere with learning, such as strained relationships, increased anxiety, substance use, feeling down, difficulty concentrating and/or lack of motivation. These mental health concerns or stressful events may diminish your academic performance and/or reduce your ability to participate in daily activities. CMU services are available, and treatment does work. You can learn more about confidential mental health services available on campus at: http://www.cmu.edu/counseling/. Support is always available (24/7) from Counseling and Psychological Services: 412-268-2922.

Diversity statement

Every individual must be treated with respect. The ways we are diverse are many and are fundamental to building and maintaining an equitable and an inclusive campus community. These include but are not limited to: race, color, national origin, sex, disability, age, sexual orientation, gender identity, religion, creed, ancestry, belief, veteran status, or genetic information. We at CMU, will work to promote diversity, equity and inclusion not only because it is necessary for excellence and innovation, but because it is just. Therefore, while we are imperfect, we all need to fully commit to work, both inside and outside of our classrooms to increase our commitment to build and sustain a campus community that embraces these core values.

It is the responsibility of each of us to create a safer and more inclusive environment. Incidents of bias or discrimination, whether intentional or unintentional in their occurrence, contribute to creating an unwelcoming environment for individuals and groups at the university. If you experience or observe unfair or hostile treatment on the basis of identity, we encourage you to speak out for justice and support in the moment and/or share your experience using the following resources:

All reports will be acknowledged, documented, and a determination will be made regarding a course of action. All experiences shared will be used to transform the campus climate to be more equitable and just.