Deterministic optimal control over a finite horizon

This chapter addresses discrete-time deterministic problems, where optimal controls have to be generated at a finite number of decision stages. No random variables influence either the dynamic system or the cost function. Then, there is no necessity of estimating the state vector. Such optimization problems are stated for their intrinsic practical importance and to describe the basic concepts of dynamic programming. As the problems are formulated under very general assumptions, their optimal solutions cannot be found in an analytical form. Therefore, we resort to an approximation consisting of the discretization of the state space into suitable grids at each decision stage. The discretization by regular grids is the simplest approach (and the one most widely used until some time ago). However, unless a small dimension of the state space is considered, this approach leads to an exponential growth of the number of samples, and thus to the curse of dimensionality. Therefore, the discretization by deterministic sequences of samples is addressed, which spread the samples in the most uniform way. Specifically, low-discrepancy sequences are considered, like quasi-Monte Carlo sequences. We also point out that the optimization problem can even be viewed as a nonlinear programming problem solvable by gradient-based descent techniques.