MATLAB Implementation of Conjugate Gradient Method and Newton's Method

This document provides MATLAB implementations of both the Conjugate Gradient Method and Newton's Method. The Conjugate Gradient Method is an iterative algorithm designed for solving large-scale linear systems. It typically outperforms traditional direct methods by operating on vectors rather than full matrices, requiring only O(n) storage complexity. The MATLAB implementation demonstrates key steps including gradient calculation, conjugate direction updates using the Fletcher-Reeves formula, and optimal step size selection through line search techniques. In contrast, Newton's Method is an optimization algorithm for solving nonlinear equations that leverages both first and second derivatives to achieve quadratic convergence rates. The provided code illustrates the complete Newton iteration process: function evaluation, Jacobian matrix computation, Hessian matrix calculation for second-order methods, and the core update formula x_{k+1} = x_k - H^{-1}∇f(x_k). The implementation includes safeguards for handling ill-conditioned Hessian matrices through regularization techniques. Both programs feature practical considerations such as convergence criteria checking (gradient norms or function value changes), maximum iteration limits, and numerical stability enhancements. The code structure follows MATLAB best practices with clear function modularization, allowing easy adaptation to specific problem domains including machine learning optimization and scientific computing applications.