Exploratory mean-variance portfolio selection with constant elasticity of variance models in regime-switching markets

We studied a continuous-time exploratory mean-variance portfolio optimization problem using a reinforcement learning framework. The problem was set in a Markov regime-switching financial market, which captured time-varying market characteristics. Under different regimes, the risky asset followed constant elasticity of variance dynamics with regime-dependent parameters. Within this setting, the exploratory mean-variance portfolio optimization problem was formulated as a stochastic control problem. Stochastic dynamic programming techniques were employed to derive the Hamilton–Jacobi–Bellman equation associated with the exploratory control problem. We first derived analytical solutions for the optimal investment strategy and the corresponding value function. Although these solutions admitted closed-form representations, they were expressed in an integral form, which made them difficult to implement directly in practical numerical computations. Because of this, we developed a reinforcement learning algorithm to approximate the optimal investment policy and the corresponding value function. Based on the structural properties of the analytical solutions, we established the convergence of both the investment policy and the value function by invoking the policy improvement theorem. This result provided a rigorous theoretical foundation for the proposed algorithm. In the algorithmic implementation, linear function approximation was employed to parameterize both the value function and the investment policy. Finally, numerical experiments were conducted to verify the convergence behavior of the proposed algorithm and to demonstrate its effectiveness in solving the considered portfolio optimization problem.