Robust control for affine nonlinear system with unknown time‐varying uncertainty under reinforcement learning framework

Abstract This paper investigates the adaptive robust control problem based on reinforcement learning for an affine nonlinear system with unknown time‐varying uncertainty. Inspired by the ability to estimate uncertainty of neural network, a novel policy iteration algorithm is proposed which alternates between the value evaluation, uncertainty estimation, and policy update steps until the adaptive robust control law is obtained. Especially during the step of uncertainty estimation, the unknown time‐varying uncertainty is approximated by a radial basis function neural network and introduce it into the reinforcement learning framework. By designing an appropriate utility function, the algorithm improves both convergence rate and final approximate error comparing with existing reinforcement learning algorithm. The Lyapunov stability theorem provides theoretical demonstrations of the stability and convergence. Furthermore, the uniformly ultimately bounded stability of the affine nonlinear system is demonstrated with unknown time‐varying uncertainty. Finally, the performance of the proposed algorithm is demonstrated through a torsion pendulum system.