HJB equations for the optimal control of differential equations with delays and state constraints, I: Regularity of viscosity solutions

We study a class of optimal control problems with state constraints, where the state equation is a differential equation with delays. This class includes some problems arising in economics, in particular, the so-called models with time to build; see [P. K. Asea and P. J. Zak, J. Econom. Dynam. Control, 23 (1999), pp. 1155-1175; M. Bambi, J. Econom. Dynam.. Control, 32 (2008), pp. 1015-1040; F. E. Kydland and E. C. Prescott, Econometrica, 50 (1982), pp. 1345-1370]. We embed the problem in a suitable Hilbert space H and consider the associated Hamilton-Jacobi-Bellman (HJB) equation. This kind of infinite dimensional HJB equation has not been previously studied and is difficult due to the presence of state constraints and the lack of smoothing properties of the state equation. Our main result on the regularity of solutions to such an HJB equation seems to be entirely new. More precisely, we prove that the value function is continuous in a sufficiently big open set of H, that it solves in the viscosity sense the associated HJB equation, and that it has continuous classical derivative in the direction of the "present." This regularity result is the starting point to define a feedback map in the classical sense, which gives rise to a candidate optimal feedback strategy. © 2010 Society for Industrial and Applied Mathematics.