Dynamic Programming and Optimal Control (ECE 271C)

Term Specific Information

Lectures and Homeworks

Term Specific Information

This course provides an overview of game theory with a special emphasis on its application to engineering applications and multiagent systems. Game theory focuses on the study of systems that are comprised of a collection of interacting and possibly competing decision making entities. Examples will be drawn from engineered, economics, and social models, including multivehicle robotics, data networks, sensor networks, and electronic commerce.

Term: Fall Quarter, 2020

Lecture: Monday and Wednesday, 12:30-1:45, Online

Office Hours: Online, Times: TBD or by email appointment

Course Ad

Syllabus

Teaching Assistant: Bryce Ferguson (blf (at) umail.uscb.edu)

Office Hours: Online, Times: TBD or by email appointment

Lecture #1: Introduction to Staged Optimization [slides]

Lecture #2: Deterministic Dynamic Programming [slides]

Lecture #3: Probability Review - Part I [slides]

Lecture #4: Probability Review - Part II [slides]

Lecture #5: Viterbi Algorithm [slides]

Lecture #6: Worst-Case Dynamic Programming [slides]

Lecture #7: Stochastic Dynamic Programming [slides]

Lecture #8: Controlled Markov Chain [slides]

Lecture #9: Inventory Control Problems [slides]

Lecture #10: Termination Problems [slides]

Lecture #11: Imperfect Information [slides]

Lecture #12: Imperfect Information Examples [slides]

Lecture #13: Infinite Horizion Dynamic Programming [slides]

Lecture #14: Stochastic Shortest Path, Policy Iteration [slides]

Lecture #15: Discounted Infinite Horizon Problems [slides]

Lecture #16: Temporal Difference Learning [slides]

Lecture #17: Q-Learning [slides]

Simulators and Projects

Understanding the principle of optimality is essential for designing online control policies for larger scale stage-optimization problems where value iteration is no longer a suitable design approach.  The following Tetris simulaors allows you to explore various design methodologies to construct efficient policies.  

3x3 Tetris Simulator (with documentation): [MATLAB]

General Tetris Simulator (with documentation): [MATLAB]