Polynomial Model y(i) = b0 + b1*x + b2*x^2 + b3*x^3 + ... with Recursive Least Squares Implementation

Polynomial fitting is a fundamental modeling technique in data analysis, where polynomial functions are constructed to approximate observed data. For a given n-th degree polynomial model, we can employ the Recursive Least Squares (RLS) method to dynamically update coefficient estimates. This approach is particularly suitable for streaming data applications or scenarios requiring real-time updates. The algorithm can be implemented by initializing parameter vectors and covariance matrices, then updating them iteratively using matrix operations.

The core concept of recursive least squares involves transforming traditional batch-based least squares computations into incremental updates. Algorithm initialization requires setting initial parameter vectors and covariance matrices (typically using small positive values for numerical stability). When new data points arrive, the system calculates prediction errors and adjusts current parameter estimates through gain factors. Key implementation steps include: computing the Kalman gain vector using covariance matrices, updating parameter estimates with prediction errors, and recursively modifying the covariance matrix.

Several critical implementation aspects require attention: the construction of polynomial basis functions directly impacts parameter estimation efficiency; the introduction of forgetting factors handles time-varying systems; numerical stability issues must be addressed through normalization or matrix decomposition techniques like Cholesky decomposition. Compared with batch least squares, the recursive version conserves memory and provides real-time outputs, but requires more careful parameter tuning. Code implementation typically involves maintaining and updating a covariance matrix P and parameter vector theta at each iteration.

In practical applications, this method is widely used in signal processing, system identification, and control engineering. By selecting appropriate polynomial degrees and regularization strategies, one can balance model goodness-of-fit with generalization capability. The RLS algorithm can be implemented using matrix operations that efficiently handle the polynomial basis expansion while maintaining numerical stability through proper initialization and update rules.