MDP modeling for multi-stage stochastic programs
Abstract
Abstract We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include decision-dependent uncertainty for one-step transition probabilities as well as a limited form of statistical learning. We focus on the expressiveness of our modeling approach, illustrating ideas with a series of examples of increasing complexity. As a solution method, we develop new variants of stochastic dual dynamic programming, including approximations to handle non-convexities.
Fields of Study & Subjects
Software Reliability and Analysis Research
Scheduling and Optimization Algorithms
Supply Chain and Inventory Management
Publication details
- Journal / source
- Mathematical Programming
- Publisher
- Springer Science and Business Media LLC
- Volume
- 217
- Issue
- 1-2
- Pages
- 43–78
- Publication date
- 1 May 2026
- Language
- EN