An Intelligent Sorting System of Coal-Gangue Based on Machine Vision and Convolutional Neural Network

Gangue is inevitably mixed with coal during mining and transportation. Currently, the manual sorting and conventional mechanical separation technologies widely adopted in the coal mining industry are plagued by low efficiency, poor identification accuracy, severe environmental pollution, and other drawbacks. This paper proposes a machine vision-based intelligent coal gangue sorting robot system. Firstly, the OpenMV captures images of coal gangue and utilizes the MobileNetV2 0.35 lightweight convolutional neural network to train the FOMO (Faster Objects, More Objects) target detection model in the cloud. This enables prediction and recognition of gangue, along with the acquisition of its center point pixel coordinates. Secondly, the position information of the gangue is sent to the STM32 microcontroller using the serial communication protocol for coordinate system conversion, pose algorithm, and path planning. Finally, the STM32 microcontroller controls the start and stop of the conveyor belt through the working status of the relay. When the relay is absorbed, the conveyor belt stops, and at the same time, the robotic arm grasps the gangue for transfer action, thus realizing the sorting of coal and gangue. The experimental results demonstrate that the cloud-trained FOMO neural network model achieves an F 1 score of 95.5% and a recall of 91.3%, with a test accuracy of 97.56%. The quantified model deployed on OpenMV can accurately identify multiple gangues and output their position information. The success rate of the robotic arm in tracking and sorting gangue reaches 90.13%, and the positioning error of the robotic arm is [9,12.5] mm. This system realizes high-precision identification, positioning, and intelligent sorting of coal and gangue, meeting the basic requirements for gangue sorting in coal mines.