atellou/RlArcProject: v0.1.1 (Thesis release)

v0.1.1 — ARC RL experiment runner + sweep docs polish Algorithms included D4PG (Distributed Distributional DDPG) via rlarcworld.algorithms.d4pg.D4PGCore distributional critic (categorical projection) target networks + soft updates n-step return support (via config) replay-buffer-based off-policy training (TorchRL) CARSM regularization integrated as an optional knob in the D4PG core/config (documented as a sweep stage) Modular components (architecture highlights) Gymnasium-first ARC environments rlarcworld.environments.arc_batch_grid_env.ArcBatchGridEnv pixel-level gym.spaces.Dict action space (location + color + submit) Steppers for rollout logic rlarcworld.environments.steppers.arc_stepper.PixelWiseArcStepper separates "episode simulation / data collection" from the algorithm core Wrappers as the main extension mechanism rlarcworld.environments.wrappers.* for curriculum/reward shaping/termination variations Decoupled training loop rlarcworld.trainers.core.Trainer orchestrates sampling, replay buffer usage, updates, and hooks ARC-specific experiment wiring rlarcworld.trainers.arc_d4pg.ArcD4PG wires models + steppers + replay buffer + trainer + callbacks Config-driven experiments experiments/run_experiment.py and experiments/run_experiments.sh YAML overrides under experiments/configs/ for repeatable sweeps Docs / experiment execution improvements Updated top-level README.md describing the modular RL-for-ARC design and uv setup. Updated experiments/README_old.md to match current config surface area. Updated experiments/EXPERIMENT_RUN_ORDER.md with concrete, copy/paste sweep commands. Added convenience config aliases: experiments/configs/rd_exp.yaml experiments/configs/world_model_experiment.yaml