Comparison of Variational Methods in Optimal Control: Newton's Method, Gradient Descent, and Conjugate Gradient

In this semester's optimal control assignment, we conducted a comparative analysis of three variational methods: Newton's Method, Gradient Descent, and Conjugate Gradient, yielding several noteworthy conclusions. Our investigation revealed that while these methods demonstrate similar performance under certain conditions, their behavior can diverge significantly in other scenarios. Newton's Method operates as an iterative algorithm commonly employed for optimization problems, implementing second-order Taylor series expansions to rapidly converge toward global optima. However, in large-scale problems, its computational demand for Hessian matrix inversion may impose memory constraints. Gradient Descent employs a first-order optimization approach where solutions are updated along the negative gradient direction during each iteration, typically implemented with a step size parameter (learning rate) for stability. Although scalable to large problems, this method is susceptible to convergence at local optima due to its greedy search strategy. The Conjugate Gradient method specializes in solving symmetric positive-definite linear systems, utilizing an orthogonal direction selection mechanism that prevents cyclic search patterns and accelerates convergence. Implementation typically involves iterative residual calculations and direction vector updates without matrix storage requirements. In summary, while each variational method presents distinct advantages and limitations, the selection process must consider problem-specific characteristics such as dimensionality, convexity, and computational resources.