A Factorized Markov Skeleton for Adaptive Dynamics

Read full text
Authors: Kim, Tae-Hun
Cited by: 0

Abstract

We introduce a factorized Markov skeleton for adaptive dynamical systems. The framework represents any adaptive algorithm as a controlled Markov kernel on an extended state space Z = X x H x G x P, decomposed via a fixed DAG factorization into four conditional kernels with prescribed parent sets. Deterministic operator interfaces are embedded as Dirac kernels in the concrete Markov category Stoch_SB of kernels on standard Borel spaces, and Prime admissibility is enforced by measurable idempotent projections. A generic representation theorem establishes that every kernel satisfying the DAG factorization and Prime admissibility admits a skeleton realization (operator interface plus noise kernel). Six canonical algorithms—stochastic variational inference (SVI), policy-gradient reinforcement learning, stochastic gradient Langevin dynamics (SGLD), Adam, Hamiltonian Monte Carlo (HMC), and Expectation-Maximization (EM)—are shown to be clean special cases. An indexed (coproduct) extension handles dynamics that change model class, architecture, or state type at runtime. A Lyapunov stability criterion for skeleton dynamics is also provided.

Fields of Study & Subjects

Reinforcement Learning in Robotics Stochastic Gradient Optimization Techniques Model Reduction and Neural Networks

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
8 February 2026
Document type
Preprint