Policy stability and ultimate stationarity in discounted risk-sensitive stochastic control

Read full text
Authors: Nicole Bäuerle, Marcin Pitera, Łukasz Stettner
Cited by: 0

Abstract

We study discrete-time Markov Decision Processes (MDPs) on finite state-action spaces and analyze the stability of optimal policies and value functions in the long-run discounted risk-sensitive objective setting. Our analysis addresses robustness with respect to perturbations of the risk-aversion parameter and the discount factor, the emergence of ultimate stationarity, and the interaction between discounted and averaged formulations under suitable mixing assumptions. We further investigate limiting regimes associated with vanishing discount and vanishing risk sensitivity, and discuss the role of Blackwell-type stability properties in the discounted setting. Finally, we provide numerical illustrations that highlight the intrinsic non-stationarity of optimal discounted risk-sensitive policies.

Fields of Study & Subjects

Adaptive Dynamic Programming Control Reinforcement Learning in Robotics Probability and Risk Models

Publication details

Journal / source
arXiv (Cornell University)
Publication date
11 January 2026
Document type
Preprint