Bayesian Adaptive Deep Reinforcement Learning for Personalized Treatment in Multiple Myeloma

Personalized management of multiple myeloma relies on accurate assessment of the patient’s current disease state, while accounting for its potential evolution, to guide follow-up and treatment adaptation. Piecewise Deterministic Markov Processes (PDMPs) offer an appropriate framework for modelling cancer progression. When treatment or monitoring decisions must be made, these models naturally lead to impulse-controlled PDMPs formulated as continuous-space Partially Observed Markov Decision Processes (POMDPs). Existing solution methods, however, assume perfectly known model parameters of the POMDP, which is unrealistic, given the likely variability between patient responses to treatment. We address this challenge by developing a Bayes-Adaptive POMDP (BAPOMDP) framework for controlling partially-observed \\ PDMPs with uncertain model parameters,enabling the decision-maker to explore a wide range of beliefs about individual patients’ disease dynamics while planning, without requiring large cohort data. We solve the resulting BAPOMDP problem using simulation-based deep reinforcement learning algorithms to compute near-optimal policies in the resulting high-dimensional continuous model. This demonstrates the practical relevance of Bayes-adaptive control for personalized cancer follow-up.