Compact Domain-Specialized Language Models for Industrial Automation: A Synthetic Data Approach

We address a fundamental challenge in industrial automation: the absence of public datasets for training AI systems specialized in Programmable Logic Controller (PLC) programming. This limitation forces reliance on large general-purpose models exceeding 70 billion parameters, resulting in prohibitive computational and energy costs. We present two compact language models (3B and 4B parameters) specialized for industrial automation programming following IEC 61131-3 standards. Our approach combines algorithmic templating with multi-LLM diversification strategies to generate 40,000 synthetic training conversations. The 4B model achieves 58% on our custom PLC-MMLU benchmark, outperforming significantly larger general-purpose models while requiring 17x fewer parameters and 3x less energy. Both models demonstrate comprehensive coverage of IEC 61131-3 languages and can be deployed on conventional hardware, making specialized industrial AI economically viable.