Balancing Exploration and Exploitation: Insights from ETC, UCB, and Thompson Sampling

The multi-armed bandit problem is a core issue that cannot be ignored in online decision-making. It is widely applied in areas such as advertising recommendation, clinical medical experiments, and financial investment optimization. This paper examines three classic algorithms for multi-armed slot machines: Explore-Then-Commit (ETC), Upper Confidence Bound (UCB), and Thompson Sampling (TS). First, the basic principles and operation mechanisms of each algorithm will be introduced, and the related cumulative regrets will be analyzed. By simulating different environments for different algorithms, the stability related to these algorithms can be verified. This further validates the advantages of the ETC, UCB and TS algorithms in the exploration-exploitation process. The results show that the ETC algorithm is simple and easy to implement, the UCB algorithm has excellent theoretical performance and does not require prior information, and the TS algorithm is more efficient in exploration, making it suitable for dynamic environments. This article provides theoretical support and practical examples for understanding and applying the multi-armed slot machine algorithm.