Iterative Methods for Solving Linear Systems: Richardson, Jacobi, and Beyond

Here is a comprehensive list of iterative algorithms for solving linear equation systems: - Richardson Iteration: A fundamental stationary method using a constant parameter to update solutions iteratively. Implementation typically involves matrix-vector multiplication and additive updates. - Parameterized Richardson Iteration: An enhanced version with optimized relaxation parameters for improved convergence rates. - GRS Iteration: A generalized Richardson scheme offering flexible parameter selection strategies. - Jacobi Iteration: A component-wise updating method where each variable is solved independently using previous iteration values. Code implementation requires diagonal matrix inversion. - Gauss-Seidel Iteration: An improved sequential updating technique using newly computed values immediately. Algorithm features faster convergence than Jacobi through successive substitutions. - SOR (Successive Over-Relaxation): An accelerated Gauss-Seidel variant using a relaxation factor ω to control convergence speed. Optimal ω selection is critical for performance. - SSOR (Symmetric SOR): A symmetric version applying SOR in forward and backward passes for symmetric matrices. - JOR (Jacobi Over-Relaxation): A relaxed Jacobi method incorporating acceleration parameters similar to SOR. - Two-Step Iteration: A multi-stage method combining different iteration schemes for enhanced stability. - Steepest Descent: A gradient-based optimization method for linear systems, minimizing residual norms through directional updates. - Conjugate Gradient: A Krylov subspace method providing optimal convergence for symmetric positive-definite systems. Implementation features orthogonal direction vectors and exact convergence in n steps. - Preconditioned Conjugate Gradient: An enhanced CG version using preconditioners to improve condition numbers and accelerate convergence. - Block Jacobi Iteration: A block-wise generalization of Jacobi method for partitioned systems, enabling parallel computation. - Block Gauss-Seidel Iteration: A block version of Gauss-Seidel updating subvectors sequentially while maintaining internal simultaneity. - Block Successive Over-Relaxation: A blocked SOR implementation combining partitioning with over-relaxation techniques. These algorithms are applicable to linear systems of various scales and are widely used in numerical computing and scientific computation domains. Implementation considerations include convergence criteria, matrix conditioning, and computational complexity analysis for different problem structures.