Almost Sure Convergence of Nonhomogeneous Switching Markov Chains With Absorbing States: A Graph-Based Approach

This article investigates the problem of almost sure convergence for nonhomogeneous switching Markov chains with absorbing states. Two types of graphs are used to describe the switching walks of state components in the Markov chain under deterministic transfer-restricted switching and arbitrary switching. The transfer-restricted switching among nonhomogeneous Markov chains is characterized by a switching directed graph, while the stochastic transitions within each Markov chain are depicted by state transition component graphs. The reachability relationship between nonabsorbing and absorbing state components is then analyzed by using the Lyapunov function constructed from the state space of Markov chains. Building on the two types of graphs, cycle-dependent switching strategies for almost sure convergence to the target absorbing state are established. Furthermore, necessary and sufficient conditions for almost sure convergence to the common absorbing state under arbitrary switching are derived, which utilize the properties of absorbing Markov chains and state transition component graphs with self-loops. Finally, the effectiveness of the proposed method is validated through two examples.