First-Order (Coarse) Correlated Equilibria in Non-concave Games

We investigate first-order notions of correlated equilibria in smooth games, in which players do not incur any regret against small modifications of their actions prescribed by some vector field. We define two such notions, based on local deviations and on stationarity of the distribution, and identify the notion of coarseness as the setting where the strategy modifications are prescribed by gradient fields. For coarse equilibria, we prove that online (projected) gradient ascent has a universal approximation property for both variants of equilibrium; in the self-play setting, every differentiable function induces an equilibrium constraint, the approximation error of which depends on the modulus of continuity and magnitude of the gradient. In the adversarial setting, we instead obtain a characterisation of regret guarantees against continuous strategy modifications satisfied by projected gradient ascent; these are precisely deviations induced by gradient fields tangent to the action set. We also provide a generalisation of the Lagrangian Hedging framework, which identifies a novel refinement of correlated equilibrium which is tractable to approximate.