MixtureKit: A General Framework for Composing, Training, and Visualizing Mixture-of-Experts Models

We introduce MixtureKit, a modular opensource framework for constructing, training, and analyzing Mixture-of-Experts (MoE) models from arbitrary pre-trained or fine-tuned checkpoints.MixtureKit supports three complementary strategies: (i) Traditional MoE, using a single router per transformer block to select experts; (ii) BTX (Branch-Train-Mix), adding routers at user-specified sub-layers for fine-grained token routing; and (iii) BTS (Branch-Train-Stitch), preserving experts intact and introducing lightweight stitch layers for controlled hub-expert information exchange.Given a single configuration dictionary, MixtureKit automatically modifies model configuration, patches decoder and causal LM classes, and exports a unified transformerscompatible checkpoint ready for inference or further fine-tuning.We also provide a visualization interface to inspect token routing, expert weight distributions, and layer-wise contributions.Experiments on multilingual codeswitched (Arabic-Latin) data show that BTX models built with MixtureKit can outperform dense baselines across multiple benchmarks.