Schedule
This schedule is tentative and subject to change. Readings will be assigned research papers and supplementary material. Key deadlines are included below.
| Week | Class | Assessments and tasks |
|---|---|---|
| 1 (Sep 2, Sep 4) |
Topic: Course introduction, Introduction to AI Agents & RL Slides: W1-L1-P1-Intro.pdf, W1-L1-P2-RL.pdf, W1-L2-RL.pdf Reading: Prerequisites, Sutton and Barto - Reinforcement Learning |
Join the slack server, find potential project partners |
| 2 (Sep 9, Sep 11) |
Topic: Foundations of RLHF – DPO, GRPO Slides: W2-L1-RL.pdf, W2-L1-dpo.pdf, W2-L2-grpo.pdf Reading: DPO, GRPO |
Sign up for projects |
| 3 (Sep 16, Sep 18) |
Topic: Bilevel optimization (PARL), Alignment Challenges – MaxMin-RLHF, Test-time Alignment (Transfer-Q, GenARM, Collab) Slides: W3-L1&2.pdf Reading: PARL, MaxMin-RLHF, Transfer-Q, GenARM, Collab |
TBA |
| 4 (Sep 23, Sep 25) |
Topic: Reasoning Models – Chain-of-thought, MCTS, ThinkLite-VL Slides: W4-L1&2.pdf Reading: MCTS, ThinkLite-VL |
Review Slides for September Assignment |
| 5 (Sep 30, Oct 2) |
Topic: TA-led Q&A; Self-Improvement – EnsemW2S, SoTA with Less, MORSE-500 Reading: EnsemW2S, SoTA with Less, MORSE-500 |
Complete September Assignment (Due Sep 30 11:59pm) |
| 6 (Oct 7, Oct 9) |
Topic: Agentive Workflows – Design patterns, communication graphs, role optimization Slides: W5-L1&2.pdf Reading: Web-Agent Vulnerability Analysis, AegisLLM, AdvBDGen, RLHFPoisoning |
TBA |
| 7 (Oct 14, Oct 16) |
Topic: Overflow / Project check-in Slides: W6-L2.pdf Reading: Language Agents Tutorial: Reasoning, Memory, and Planning, WebDreamer |
TBA |
| 8 (Oct 21, Oct 23) |
Topic: Web Agents – Architectures, capabilities, vulnerabilities Slides: W7-L1.pdf, W7-L2.pdf Reading: WebArena, VisualWebAgent, WorkArena++, WebLINX, AgentOccam, AgentSandbox, Language Agents Tutorial Application & Data Evaluation, Black Hat EU 2023, Black Hat USA 2025, AWS re:Invent 2024, Scale/BrowserART |
Complete Midterm Report (Due Oct 15 11:59pm) |
| 9 (Oct 28, Oct 30) |
Topic: Code Agents – Code generation, debugging, security Slides: W8-CodeAgents_Lecture1.pdf Reading: SWE-bench, SWE-bench Verified, SWE-agent, OpenHands, AutoCodeRover, Self-Debug, Reflexion, DebugBench, DebugEval |
Complete October Assignment (Due Oct 31 11:59pm) |
| 10 (Nov 4, Nov 6) |
Topic: Code Agents & Tool-Use Agents – Tool integration, orchestration frameworks Slides: W9-CodeAgents_Lecture2.pdf Reading: OWASP Top 10, CSET 2024, Do Users Write More Insecure Code with AI Assistants?, Veracode 2025, Apiiro 2025 risk study, |
TBA |
| 11 (Nov 11, Nov 13) |
Topic: World Models – for web, robotics, and simulation-based agents Reading: TBA |
TBA |
| 12 (Nov 18, Nov 20) |
Topic: Safety and Robustness – Jailbreak, poisoning, agentic defenses Reading: TBA |
TBA |
| 13 (Nov 25, Nov 27) |
Topic: AI-Generated Content Detection – watermarking, detectors Reading: TBA |
TBA |
| 14 (Dec 2, Dec 4) |
Topic: Final Presentations Reading: TBA |
TBA |
| 15 (Dec 9, Dec 11, Dec 12) |
Topic: Final Presentations Reading: TBA |
TBA |
Back to top | © Furong Huang at UMD | View template on Github