Course Details

Subject {L-T-P / C} : EE6334 : Stochastic Control Theory {3-0-0 / 3}
Subject Nature : Theory
Coordinator : Prof. Asim Kumar Naskar

Syllabus

Basic issues in optimization stochastic vs. deterministic methods Random search, Recursive methods: LMS, RLS, Kalman filter, Stochastic approximation (SA), Stochastic gradient methods, Finite-difference SA (FDSA), Simultaneous perturbation SA (SPSA), Markov chains, Markov Decision Process (MDP), Dynamic Programming method, Design of approximate controllers for MDPs, Stochastic linear quadratic control, Finite horizon control.

Course Objectives

  1. Students will be competent in critical questioning and analysis of the mathematical tools used in stochastic process and control.

Course Outcomes

At the end of the course, students will be able
(I) To understand basic principles of stochastic optimization theory, probability theory and stochastic dynamical systems including Markov chains.
(II) To formulate stochastic control problems as Markov Decision Process (MDP) problems.
(III) To solve linear stochastic control problems.

Essential Reading

  1. K. J. Astrom, Introduction to Stochastic Control Theory, Dover , 2004
  2. D. Bertsekas, Dynamic programming and Optimal Control, Athena Scientific , 2007

Supplementary Reading

  1. P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification and Adaptive Control, SIAM , 2016
  2. H. Kwakernaak and R. Sivan, Linear Optimal Control, John Wiley , 1972