Spring 2025

CSCI 699 · Trustworthy Large Foundation Models

Logistics

Time:
Mondays 3:30–6:50 PM PST
Location:
GFS 213

Teaching Team

Jieyu Zhao

Instructor: Jieyu Zhao

Office Hour: directly after the class

Teaching Assistants

  • Taiwei Shi — TBD

Introduction

Although there have been impressive advancements in large foundation models (e.g., LLMs, VLMs), several studies report that NLP models contain social biases. Even worse, the models run the risk of further amplifying stereotypes and causing harms to people. As LFMs continue to advance and be integrated into various domains such as healthcare, finance, marketing, and social media, important ethical concerns need to be addressed. In this course, students will critically examine the ethical implications of NLP, including issues related to bias/fairness, privacy, safety, and social impact. Through discussions, case studies, and guest lectures, students will explore the ethical challenges associated with AI models and develop a deep understanding of the ethical considerations that arise when designing, implementing, and deploying large foundation models.

Students will get a broad understanding about possible issues in current large foundation models and how current research has tried to alleviate those issues. This class will equip students with the ability to read and write critical reviews about research papers, and learn how to conduct research related to AI fairness, interpretability and robustness.

News

Syllabus

Calendar subject to change. All work (except final report) is due 11:59 PM PT on the listed date.

Wk Date Topic Readings Due
1 01/13 Introduction, Logistics and Biases in NLP
2 01/20 MLK Holiday — No class
3 01/27 LLMs and VLMs
  • Check the signup sheet
01/27 Guest Lecture: Beyond Autoregressive LMs — insertion-based, diffusion, parallel decoding (Sidi Lu)
Find project team members
4 02/03 VLMs & Biases in LFMs
02/03 Guest Lecture (Zhecan Wang)
5 02/10 Safety in LLMs
02/10 Guest Lecture: Beyond Monolithic LMs (Weijia Shi)
6 02/17 President's Day — No class
Project Proposal
7 02/24 Harms in downstream tasks
02/24 Guest Lecture: Socially Impactful & Trustworthy Generative FMs (Yue Huang)
8 03/03 Human–AI Alignment
03/03 Guest Lecture: AI Alignment (Xiaoyuan Yi)
9 03/10 Human–AI Alignment (cont'd) + Privacy
Midterm Report
03/10 Guest Lecture (Yiren Feng)
10 03/17 Spring Recess — No class
11 03/24 Project Midterm Presentation Workshop — I
03/24 Guest Lecture: Mechanistic Interpretability for AI Safety (Ninghao Liu)
12 03/31 Project Midterm Presentation Workshop — II
03/31 Guest Lecture (Zining Zhu)
13 04/07 AI Agent
04/07 Guest Lecture (Xuezhe Ma)
14 04/14 LFMs + X
04/14 Guest Lecture: Auditable decision making under uncertainty (Oliver Liu)
15 04/21 Final Project Presentation
04/21 Guest Lecture: AI + medicine (Zhenyuan Qin)
16 04/28 Final Project Presentation
Final Report
04/28 Guest Lecture: Understanding LLMs from Pretraining Data Distribution (Xinyi Wang)

Logistics

  • Office hours: Check above. For special requests, contact the TA individually (please put [CSCI 699] in the subject line).
  • Assignments: Submit through the course Google Drive folder shared via Slack. Use your USC account.
  • Discussions: We use Slack for general course-related questions and announcements.
  • In-person policy: The class is conducted in person. There will be no Zoom recordings. Let the instructor know in advance if you have an emergency and cannot join.
  • Paper sign-up: Sign up for paper presentation & project team members in the course spreadsheet shared via Slack.

Prerequisites

  • Familiarity with NLP and/or ML. Ideal pre/co-requisites are CSCI 544 (Applied NLP) or CSCI 567 (Machine Learning).
  • We will not teach any NLP/ML/CV concepts/techniques in this class!
  • Programming skills. We mainly use Python with PyTorch, but you may use other libraries for your final project.

Grading

Grades are based on attendance (10%), paper presentation (30%), and a course project (60%).

Attendance and Discussion (10%):

  • In-person attendance

Paper Reading and Discussion (30%):

  • Present paper, lead class discussion
  • Sign up as reviewers, peer-review others’ presentations

Course Project (60%):

  • Project Proposal (10%)
  • Midterm Report (10%)
  • Final Presentation (20%)
  • Final Report (20%)

Late Days

You have 4 late days for any assignment excluding the final report. Each late day allows submission 24 hours later than the deadline. Maximum 2 late days per assignment. If working in a group, one late day means each member spends a late day.

Paper Presentation

Paper presentation helps students develop the skills to give research talks. Each student presents 2 papers, prepares slides, and leads discussion. Each week another student signs up as the feedback provider (reviewer). The reviewer provides feedback to the instructor or TAs.

Grading rubric: correctness of content (40%), clarity (20%), discussion (20%), slides & presentation skills (20%).

Final Project

The final project can be done individually or in groups of up to 3. This is a chance to freely explore ML methods and how they can be applied to a task of your choice. You’ll also learn best practices for developing ML methods — inspecting data, establishing baselines, and analyzing errors.

Each group should finish one research project related to the class topics. A “deliverable” result is expected — the project should be self-contained and reproducible. Typical successful projects: (1) a novel and sound solution to an interesting research problem, (2) correct and meaningful comparisons among baselines and existing approaches, or (3) applying existing techniques to a new application. Negative results are not penalized as long as the proposed approach is thoroughly explored and justified.

Use the standard *ACL paper submission template for project reports.

  • Project proposal (10%) — Due Week 4, ~2 pages. Articulate the research question, justify significance, evidence knowledge of the literature. Include a timeline.
  • Midterm progress report (10%) — Due Week 10, ~4 pages. Clear research goal statement, overview of completed work, challenges and solutions, initial results.
  • Final presentation (20%) — In the last two weeks, 30-min presentation with 5-min Q&A. Cover goal, motivation, related work, methodology, results.
  • Final project report (20%) — Follow NLP conference paper format (abstract, intro, related work, results, discussion). 8 pages total (excluding references).

Resources

Related courses: