C 2 -SeqRL: Reinforcement Learning for Conflict-Constrained Sequence Construction

Sequences underpin numerous applications across communication systems, scheduling, symbolic reasoning, and beyond. However, constructing sequences that satisfy complex structural constraints remains a persistent challenge. As sequence length and constraint complexity increase, the feasible solution space grows exponentially, rendering conventional searchbased or rule-driven approaches computationally infeasible. To overcome these limitations, we introduce $\mathbf{C}^{\mathbf{2}}$-SeqRL, a deep reinforcement learning framework that formulates constrained sequence construction as an episodic decision-making problem. An agent progressively fills sequence positions while strictly adhering to conflict-avoidance constraints, enabling the discovery of latent structural patterns that characterize valid solutions. Furthermore, $\mathbf{C}^{\mathbf{2}}$-SeqRL leverages maskable reinforcement learning to proactively eliminate infeasible actions and integrates expert-guided policy learning to enhance solution quality and accelerate convergence. By jointly exploiting constraint awareness and informed exploration, $C^{2}$-SeqRL offers a scalable and adaptive paradigm for constrained combinatorial sequence generation, showing strong potential for real-world applications where structural validity is critical. Code is available at: github.com/pangay/C2_SeqRL