Multi-Period Newsvendor Problem: Solving MDP Models with Value Iteration, Policy Iteration, and Reinforcement Learning Algorithms in MATLAB

On the MATLAB platform, we implement value iteration, policy iteration, and reinforcement learning algorithms to solve the multi-period newsvendor problem using MDP modeling. The value iteration algorithm solves for optimal policies by iteratively updating the optimal value function for each state through Bellman equation computations, typically implemented using matrix operations and convergence checks. Policy iteration alternates between policy evaluation (solving linear systems for current policy values) and policy improvement (greedily selecting better actions), ensuring monotonic convergence. Reinforcement learning employs a trial-and-error approach where Q-learning or SARSA algorithms interact with the environment to learn optimal policies through reward-based updates, requiring careful design of state representations and reward functions. By comprehensively applying these three algorithms with proper state-space discretization and inventory level management, we achieve robust solutions for the multi-period newsvendor problem, expanding the possibilities for MDP model resolution in supply chain optimization scenarios.