Fall 2023

CSCI 699 · Ethics in NLP

Logistics

Time:
Tuesdays & Thursdays 4:00–5:50 PM
Location:
DMC 150

Teaching Team

Jieyu Zhao

Instructor: Jieyu Zhao

Office Hour: Thu 12–1 PM @ PHE 332

Teaching Assistants

  • Pei Zhou — Mon 3–4 PM @ RTH 313
  • Peifeng Wang — Thu 2:30–3:30 PM @ SAL 213

Introduction

Although there have been impressive advancements in natural language processing (NLP), several studies reported that NLP models contain social biases. Even worse, the models run the risk of further amplifying the stereotypes and causing harms to people. As NLP technology continues to advance and be integrated into various domains such as healthcare, finance, marketing, and social media, it raises important ethical concerns that need to be addressed. In this course, students will critically examine the ethical implications of NLP, including issues related to bias, fairness, privacy, transparency, accountability, and social impact. Through discussions, case studies, and guest lectures, students will explore the ethical challenges associated with NLP and develop a deep understanding of the ethical considerations that arise when designing, implementing, and deploying NLP applications.

Students will get a broad understanding about possible issues in current NLP models and how current research has tried to alleviate those issues. This class will equip students with the ability to read and write critical reviews about research papers, and learn how to conduct research related to NLP fairness, interpretability and robustness.

News

Syllabus

Calendar subject to change. All work (except final report) is due 11:59 PM PT on the listed date.

Wk Date Topic Readings Due
1 Tue Aug 22 Introduction
Thu Aug 24 Introduction (cont'd)
2 Tue Aug 29 Project Examples / Human Subjects Research
Thu Aug 31 Social Biases in NLP
3 Tue Sep 5 Bias Evaluation
Thu Sep 7 Models vs morality
4 Tue Sep 12 Harms of LLMs in downstream tasks
Thu Sep 14 Bias Mitigation
Project Proposal
5 Tue Sep 19 Guest Lecture: What happened in Industry Research (Sunipa Dev)
Thu Sep 21 Harms of LLMs: Bias and Stereotype
6 Tue Sep 26 Hate Speech / Bias-Stereotype-Harm summary
Thu Sep 28 Midterm Presentation Workshop
7 Tue Oct 3 Midterm Presentation Workshop
Thu Oct 5 Guest Lecture: LLM memorization (Eric Wallace)
8 Tue Oct 10 Privacy
Thu Oct 12 Fall Recess — No class
9 Tue Oct 17 Human-Centered AI
Thu Oct 19 Guest Lecture: Human & AI Interaction (Weiyan Shi) / Bias in Dialogue
10 Tue Oct 24 Guest Lecture: Human-Centered AI (Sherry Wu) / Microaggression
Thu Oct 26 Multilingual Biases
Midterm Report
11 Tue Oct 31 Multimodal biases
Thu Nov 2 Reduce bias in language models
12 Tue Nov 7 Guest Lecture: Interpretation (Hanjie Chen)
Thu Nov 9 Alignment with humans
13 Tue Nov 14 Trade-offs between different metrics
Thu Nov 16 Summary and Final Project Presentations
14 Tue Nov 21 Final Project Presentations
Thu Nov 23 Thanksgiving — No class
15 Tue Nov 28 Final Project Presentations
Thu Nov 30 Final Project Presentations
16 Tue Dec 5 No class
Thu Dec 7 No class
Project Report

Logistics

  • Office hours: Check above. For special requests, contact the course staff individually.
  • Assignments: Submit through the course Google Drive folder shared by the course staff.
  • Discussions: We use Slack for general course-related questions and announcements. For individual matters, email directly (please put [CSCI 699] in the subject line) or come to office hours.

Prerequisites

  • Familiarity with NLP and/or ML. Ideal pre/co-requisites are CSCI 544 (Applied NLP) or CSCI 567 (Machine Learning).
  • We will not teach any NLP concepts/techniques in this class!
  • Programming skills. We mainly use Python with PyTorch, but you may use other libraries for your final project.

Grading

Grades are based on attendance (10%), paper presentation (30%), and a course project (60%).

Attendance and Discussion (10%):

  • In-person attendance

Paper Reading and Discussion (30%):

  • Present paper, lead class discussion
  • Sign up as reviewers, peer-review others’ presentations

Course Project (60%):

  • Project Proposal (10%)
  • Midterm Report (10%)
  • Final Presentation (20%)
  • Final Report (20%)

Late Days

You have 4 late days for any assignment. Each late day allows submission 24 hours later than the deadline. If working in a group, one late day means each member spends a late day.

Paper Presentation

Paper presentation helps students develop the skills to give research talks. Each student presents 2 papers, prepares slides, and leads discussion. Each week another student signs up as the feedback provider (reviewer).

Grading rubric: correctness of content (40%), clarity (20%), discussion (20%), slides & presentation skills (20%).

Final Project

The final project can be done individually or in groups of up to 3. Each group should finish one research project related to the class topics. A “deliverable” result is expected — the project should be self-contained and reproducible. Typical successful projects: (1) a novel and sound solution to an interesting research problem, (2) correct and meaningful comparisons among baselines and existing approaches, or (3) applying existing techniques to a new application. Negative results are not penalized as long as the proposed approach is thoroughly explored and justified.

Use the standard *ACL paper submission template for project reports.

  • Project proposal (10%) — Due Week 4, ~2 pages.
  • Midterm progress report (10%) — Due Week 10, ~4 pages.
  • Final presentation (20%) — Last two weeks, 30-min presentation with 5-min Q&A.
  • Final project report (20%) — 8 pages total (excluding references), NLP conference format.

Resources

Related courses: