As the Riccati equation for control of linear systems, the
Hamilton-Jacobi-Bellman (HJB) equations play a fundamental role for optimal
control of nonlinear systems. For infinite-horizon control problem, the optimal
control can be represented by the stable manifold of the characteristic
Hamiltonian system of HJB equation. In this paper, we study the deep neural
network (NN) semiglobal approximation of the stable manifold. Our main
contribution includes twofold: firstly, from the mathematical point of view, we
theoretically prove that if an approximation is sufficiently close to the exact
stable manifold of the HJB equation, then the corresponding control derived
from this approximation stabilizes the system and is nearly optimal. Secondly,
based on the theoretical result, we propose a deep learning approach to
approximate the stable manifold and compute optimal feedback control
numerically. Our algorithm is based on the geometric features of the stable
manifold and relies on adaptive data generation through finding trajectories
randomly within the stable manifold. To achieve this, we solve two-point
boundary value problems (BVPs) locally near the equilibrium and extend the
local solutions using initial value problems (IVPs) for the characteristic
Hamiltonian system. We randomly choose a number of samples along each
trajectory, and adaptively select additional samples near points with large
errors from the previous round of training. Our algorithm is causality-free
basically, hence it has the potential to apply to a wide range of
high-dimensional nonlinear systems. We demonstrate the effectiveness of our
method through two examples: stabilizing the Reaction Wheel Pendulums and
controlling the parabolic Allen-Cahn equation.