Games: Slot Machines

This chapter gives a gentle introduction to statistics. An important property of AI algorithms is the exploration versus exploitation trade-off. This concept is illustrated with slot machines (“one-armed bandits”). Given multiple machines, you can observe how well each one performs. Exploitation is where you use the machine giving the highest payout. But you might have been lucky with that machine. Hence, you need to occasionally consider other machines, exploring, to see if there is something better. People’s IQ and height are used as two examples of a “normal” distribution. When you plot all the data about people’s IQ (or height), there is a midpoint. 68% of the population will have their IO be within what’s called one “standard deviation” (34% above and 34% below). Going further above and below, two standard deviations correspond to 95% and three encompass 99.7%. You can think of the midpoint as being the height of the distribution, and the width of a standard deviation as being how skinny or broad the distribution is. These properties are useful in knowing whether your data conforms to the expected results or is statistically unusual.