CMSC848N Generative AI Agents - Fall 2025
Course Description
This course covers the foundations and frontiers of Generative AI Agents, blending theory and practice with state-of-the-art research in LLMs, RL/RLHF, alignment, reasoning models, self-improvement, and agent safety. Generative AI agents are autonomous systems powered by large language and vision models that can reason, plan, act, and adapt in complex environments. Students will study core methods and then apply them in research-oriented projects where they design, implement, and evaluate novel agent systems.
- LLMs & Reasoning Models: chain-of-thought, Monte Carlo Tree Search, and efficient reasoning methods.
- RL & RLHF Foundations: DPO, GRPO, bilevel optimization (PARL), and alignment challenges (MaxMin-RLHF, test-time alignment, Transfer-Q, GenARM, Collab).
- Self-Improvement & Safety: ensemble strategies (e.g., EnsemW2S), robustness, jailbreak/poisoning defenses, AI-generated content detection & watermarking.
- Agentive Workflows: design patterns, communication graphs, role optimization.
- Web/Code/Tool-Use Agents: architectures, capabilities, vulnerabilities, and orchestration frameworks.
- World Models: agents for web, robotics, and simulation environments.
Learning Outcomes
- Understand key architectures and algorithms for generative AI agents.
- Critically analyze and reproduce SoTA methods from research papers.
- Design, implement, and evaluate novel AI agent systems.
- Conduct research suitable for submission to top-tier AI/ML venues.
Logistics
When & Where
Section 0101
Lecture: Tuesday and Thursday 12:30pm - 1:45pm ยท CSI 2117
Instructors
Furong Huang
Brendan Iribe Center for Computer Science and Engineering, Room 4124
furongh@umd.edu
https://furong-huang.com
Teaching Assistants
Minghui Liu (Ming)
minghui@umd.edu
Office hours: Asynchronous on Slack
Sy Tuyen Ho (Tuyen)
stho@umd.edu
Office hours: Asynchronous on Slack
Contact Us
If you're a registered student, send a message to instructors on Slack. If not, send an email including "848N" in the title (not recommended).
Back to top | © Furong Huang at UMD | View template on Github