Syllabus

Time and place: Tu/Th 12:30–1:45pm · CSI 2117
Instructor: Prof. Furong Huang
Teaching Assistants: Minghui Liu, Ho Sy Tuyen

Course objectives

Understand key architectures and algorithms for generative AI agents.
Critically analyze and reproduce state-of-the-art research in LLMs, RL, and alignment.
Design, implement, and evaluate novel AI agent systems.
Gain research skills through projects: reading papers, reproducing baselines, implementing new ideas, running experiments.
Develop presentation, teamwork, and scientific communication skills.

Prerequisites

This course assumes a solid foundation in machine learning, mathematics, and programming. Students should have completed prior coursework in machine learning or deep learning, and be comfortable implementing algorithms in Python. Specific prerequisites are outlined below:

Machine Learning Foundations
- Supervised, unsupervised, and reinforcement learning
- Core concepts: classification, regression, cross-validation, overfitting, generalization
- Neural networks: basic architectures and training methods
Mathematics for ML
- Calculus & Linear Algebra: gradients of multivariate functions, dot products, projections, solving multivariate equations, matrix inversion, matrix factorization. See math4ml, linear algebra review, and advanced linear algebra review.
- Optimization: constrained optimization with Lagrange multipliers, convexity, and convex optimization basics. See convex analysis review and optimization review.
- Probability & Statistics: random variables, expectations, variance, Bayes’ rule, conditional independence, maximum likelihood estimation for common distributions. See probability review.
Programming Skills
- Proficiency in Python
- Experience with numerical and ML libraries (e.g., NumPy, PyTorch, or TensorFlow)
- Familiarity with version control, debugging, and modular coding

Background in natural language processing or computer vision is helpful but not required; necessary concepts will be introduced as needed during the course.

Course Format & Workload

This course is structured around a mix of lectures, paper discussions, and a semester-long research project. Lectures will introduce concepts and recent research on generative AI agents, often accompanied by assigned readings to be discussed in class. Students will engage in breakout discussions, short impromptu quizzes, and TA-led Q&A sessions to deepen understanding.

A major emphasis of the course is the team-based project, where students will design, implement, and evaluate their own AI agents. Progress will be documented through interim reports and presentations, culminating in a final written report and showcase at the end of the semester.

Students should expect to spend approximately 8–12 hours per week outside of class on readings, coding, experiments, and collaboration with their teammates. All assessment details (including grading weights and deadlines) are provided on the Assessments and Schedule pages.

Readings

Assigned readings will primarily be recent research papers and technical reports relevant to each lecture topic. Additional materials may include blog posts, slides, and tutorials to support student self-learning. Students are expected to read assigned papers before class discussions.

Assessments

In-class quizzes (10%): Impromptu short quizzes during lectures. Sep 2 - Nov 27.
Homework (20%): Due Sep 30, Oct 30, Nov 30.
Midterm Report (30%): Due Oct 15. Focuses on project progress and baseline results.
Final Project (40%):
- Final Presentation (20%): Dec 2, Dec 4, Dec 9, Dec 11.
- Final Report (20%): Due Dec 12.

More information on assessments here. Late work is managed through a 72-hour late bank, applicable to all assignments except in-person exams/presentations.

Communication and Discussion

We will be using Slack for class-related discussion and communication. The system is highly catered to getting you help fast and efficiently from classmates, the TA, and myself. Rather than emailing questions to the teaching staff, please post your questions on Slack (either as public discussions or as private posts to instructors). All messages sent to the instructors email addresses will be redirected to Slack.
Find our Slack server here. Use this invitation link to join (expires Sep 29).

What is expected of you

Attend lectures, participate in discussions, and complete assigned readings.
Start your project early, be proactive in finding partners, and contribute responsibly to teamwork.
Be critical and creative when analyzing papers and designing solutions.
Be resilient—expect to revise or restart approaches as part of the research process.
Be supportive, communicative, and respectful toward peers.

What to expect from your instructor

Guidance through interactive lectures, Q&A sessions, and project discussions.
Feedback on reports and presentations.
Support in finding and refining project ideas relevant to the course.
Encouragement to think critically, ask questions, and engage with the research frontier.