Approximating networks for the solution of T-stage stochastic optimal control problems

Approximate solution of a general T-stage stochastic optimal control problem is considered. It is known that discretizing uniformly the state components may lead dynamic programming to incur the "curse of dimensionality". Approximating networks, i.e., linear combinations of parametrized basis functions provided with density properties in suitable normed spaces, are then defined and used in two approximate methods (examples of such networks are one hidden layered feedforward neural networks, Radial Basis Functions, etc.). The first one consists in approximating the optimal cost-to-go functions in dynamic programming. The second method reduces the original functional optimization problem to a nonlinear programming one that is solved by means of stochastic approximation. Approximating networks of suitable types benefit by the property that, for the approximation of members of some classes of smooth functions, the number of parameters to be optimized and the number of samples to be used increase moderately with the dimensions of the arguments of the functions. The two methods are tested and compared in a test problem involving a 10-dimension state vector. Copyright ((C)) 2001 IFAC.