Stochastic optimal control with imperfect state information over a finite Horizon

Discrete-time stochastic optimal control problems are considered. These problems are stated over a finite number of decision stages. The state vector is assumed to be observed through a noisy measurement channel. Because of the very general assumptions under which the problems are stated, obtaining analytically optimal solutions is practically impossible. Note that the controller has to retain the vector of all the measures and of all the controls in memory, up to the most recent decision stage. Such measures and controls constitute the “information vector” that the control function depends on. The increasing dimension of the information vector makes it practically impossible to use dynamic programming. Then, we resort to the “extended Ritz method” (ERIM). The ERIM consists in substituting the admissible functions with fixed-structure parametrized functions containing vectors of “free” parameters. Of course, if the number of decision stages is large, the application of the ERIM is also impossible. Therefore, an approximate approach is followed by truncating the information vector and retaining in the memory only a suitable “limited-memory information vector.”