CS 6300 - Spring 2023

Artificial Intelligence
CS 6300
Spring 2024

Schedule:	Mon/Wed 3-4:20pm
Location:	JTB 140
Instructor:	Daniel Brown
Office Hours:	MEB 2172; TBD
TAs:	Atharv Belsare and Chia Tsai
Office Hours:	MEB 3115, M: 11:30 - 12:30, Tu: 3-4pm, W 11:30-12:30pm, Th: 11-12pm

Links: [Canvas/Piazza/Gradescope] [Course information] [Homework] [Projects] [Exams] [Practice]

In the readings, RN refers to Russell and Norvig; SB refers to Sutton and Barto.

See Homeworks link above for LaTex files to complete homework. The links below just have the pdf

Date Topics Readings Due Notes

8 Jan Introduction to AI opt: RN 1.1,2

Search

10 Jan Search
Depth, breadth, uniform, heuristics, A* search 3.1-3.6
opt: RN 3.2,4.1-4.2

15 Jan Martin Luther King Day
No Class P0 (16 Jan)

17 Jan Game Playing I
Minimax search RN 5-5.3 HW1 (Jan 19)

22 Jan Game Playing II
Expectimax search and Utility RN 5.4-5.5

24 Jan Probability
Everything you need to know! RN 13-13.5
opt: RN 13.6 HW2 (Jan 25)

Sequential Decision Making

29 Jan Markov Decision Processes I
Value iteration RN 17.1-2
SB 3, 4.4 P1 (30 Jan)

31 Jan Markov Decision Processes II
Policy iteration RN 17.3
SB 4.1-3 HW3 (Feb 1)

5 Feb Reinforcement Learning I
TD-Learning and Q-Learning RN 21.1-3
SB 6.1-4, 8.1-8.2

7 Feb Reinforcement Learning II
Functional Approximation and Deep Q-Learning RN 21.4-5
SB 8.1,8.2 HW4 (Feb 8)

12 Feb Reinforcement Learning III
Policy representations and policy gradients Spinning up in Deep RL: intro Part 1-3 HW5 (Feb 15)

14 Feb No Class

19 Feb President's Day
No Class P2 (20 Feb)

21 Feb Reinforcement Learning IV
Monte-Carlo Tree Search, AlphaGo AlphaGo Nature Paper

26 Feb Midterm Review HW6 (Feb 26)

28 Feb Midterm

3-10 Mar Spring break No Class

Reasoning Under Uncertainty

11 Mar No Class

13 Mar Bayes' Nets I
Representation RN 14-14.4 P3 (22 Mar)

18 Mar Bayes' Nets II
Independence, D-Separation RN 14.4-5

20 Mar Bayes' Nets III
Factors and Variable Elimination

25 Mar Bayes' Nets IV
Sampling

27 Mar Decision Diagrams, Markov Chains
VPI, Mini-Forward Algorithm RN 16.5-6, 15.1-3 HW7 (Mar 27)

1 Apr Hidden Markov Models I
HMMs and Particle Filters RN 15.2

3 Apr Hidden Markov Models II
Viterbi Algorithm, Dynamic Bayes Nets RN 15.5 HW8 (Apr 3)

8 Apr POMDPs
Intro to partial observability in MDPs HW9 (Apr 10)

10 Apr Imitation Learning
Behavioral cloning and interactive imitation learning

15 Apr Reward Learning
Inverse reinforcement learning and preference learning

17 Apr LLMs and ChatGPT
Transformers, finetuning, and RLHF

22 Apr Final Exam Review P4 (22 Apr)

25 Apr Final Exam 3:30pm-5:30pm, In Class (JTB 140)

Date	Topics	Readings	Due	Notes
8 Jan	Introduction to AI	opt: RN 1.1,2
Search
10 Jan	Search Depth, breadth, uniform, heuristics, A* search	3.1-3.6 opt: RN 3.2,4.1-4.2
15 Jan	Martin Luther King Day No Class		P0 (16 Jan)
17 Jan	Game Playing I Minimax search	RN 5-5.3	HW1 (Jan 19)
22 Jan	Game Playing II Expectimax search and Utility	RN 5.4-5.5
24 Jan	Probability Everything you need to know!	RN 13-13.5 opt: RN 13.6	HW2 (Jan 25)
Sequential Decision Making
29 Jan	Markov Decision Processes I Value iteration	RN 17.1-2 SB 3, 4.4	P1 (30 Jan)
31 Jan	Markov Decision Processes II Policy iteration	RN 17.3 SB 4.1-3	HW3 (Feb 1)
5 Feb	Reinforcement Learning I TD-Learning and Q-Learning	RN 21.1-3 SB 6.1-4, 8.1-8.2
7 Feb	Reinforcement Learning II Functional Approximation and Deep Q-Learning	RN 21.4-5 SB 8.1,8.2	HW4 (Feb 8)
12 Feb	Reinforcement Learning III Policy representations and policy gradients	Spinning up in Deep RL: intro Part 1-3	HW5 (Feb 15)
14 Feb	No Class
19 Feb	President's Day No Class		P2 (20 Feb)
21 Feb	Reinforcement Learning IV Monte-Carlo Tree Search, AlphaGo	AlphaGo Nature Paper
26 Feb	Midterm Review		HW6 (Feb 26)
28 Feb	Midterm
3-10 Mar	Spring break No Class
Reasoning Under Uncertainty
11 Mar	No Class
13 Mar	Bayes' Nets I Representation	RN 14-14.4	P3 (22 Mar)
18 Mar	Bayes' Nets II Independence, D-Separation	RN 14.4-5
20 Mar	Bayes' Nets III Factors and Variable Elimination
25 Mar	Bayes' Nets IV Sampling
27 Mar	Decision Diagrams, Markov Chains VPI, Mini-Forward Algorithm	RN 16.5-6, 15.1-3	HW7 (Mar 27)
1 Apr	Hidden Markov Models I HMMs and Particle Filters	RN 15.2
3 Apr	Hidden Markov Models II Viterbi Algorithm, Dynamic Bayes Nets	RN 15.5	HW8 (Apr 3)
8 Apr	POMDPs Intro to partial observability in MDPs		HW9 (Apr 10)
10 Apr	Imitation Learning Behavioral cloning and interactive imitation learning
15 Apr	Reward Learning Inverse reinforcement learning and preference learning
17 Apr	LLMs and ChatGPT Transformers, finetuning, and RLHF
22 Apr	Final Exam Review		P4 (22 Apr)
25 Apr	Final Exam 3:30pm-5:30pm, In Class (JTB 140)