Least Squares Simulation Functions

Least squares is a classic mathematical optimization method commonly used for data fitting problems. Its core principle involves finding the optimal function match by minimizing the sum of squared errors. In polynomial fitting scenarios, the implementation of simulation functions typically follows three key steps: Data Generation First, simulate data points by adding random noise to a ground truth function (such as a sine curve). Introducing noise helps mimic real-world data acquisition uncertainties. In code implementation, this can be achieved using numpy.random.normal() in Python or randn() in MATLAB to generate Gaussian-distributed noise around the true function values. Polynomial Modeling For cubic polynomial fitting, the model includes constant, linear, quadratic, and cubic terms - totaling 4 undetermined coefficients. A 9th-degree polynomial extends to higher-order terms. As polynomial degree increases, model flexibility improves but may lead to overfitting - where the model adapts excessively to training noise rather than underlying patterns. Programmatically, polynomial features can be generated using libraries like sklearn.preprocessing.PolynomialFeatures or polyfit() functions. Solution and Evaluation Solve the least squares problem by constructing the Normal Equation or using matrix operations to obtain polynomial coefficients. The normal equation (X^T * X) * coefficients = X^T * y provides the analytical solution, while numerical methods like numpy.linalg.lstsq() offer computational stability. Evaluation typically compares training and testing errors: cubic polynomials may exhibit underfitting, while 9th-degree polynomials often show classic overfitting characteristics (extremely low training error but sharply increased testing error). In practice, plotting fitted curves visually demonstrates different polynomial degrees' ability to capture data trends. Cross-validation techniques help validate regularization methods (like Ridge or Lasso regression) for suppressing overfitting in high-degree terms, implemented through regularization parameters that penalize large coefficients.