The basic infinite-dimensional or functional optimization problem

In infinite-dimensional or functional optimization problems, one has to minimize (or maximize) a functional with respect to admissible solutions belonging to infinite-dimensional spaces of functions, often dependent on a large number of variables. As we consider optimization problems characterized by very general conditions, optimal solutions might not be found analytically and classical numerical methods might fail. In these cases, one must use suitable approximation methods. This chapter describes an approximation method that uses families of nonlinear approximators – including commonly used shallow and deep neural networks as special cases – to reduce infinite-dimensional problems to finite-dimensional nonlinear programming ones. It is named “Extended Ritz Method” (ERIM). This term originates from the classical Ritz method, which uses linear combinations of functions as an approximation tool. It does not seem that the Ritz method has met with important successes as regards problems whose admissible solutions depend on large number of variables. This might be ascribed to the fact that this method can be subject to one of the forms of the so-called “curse of dimensionality.” However, theoretical and numerical results show that the ERIM might mitigate this drawback. Examples of infinite-dimensional optimization problems are presented that can be approximated by nonlinear programming ones.