Multi-Armed Bandit

Understanding Multi-Armed Bandit

The "multi-armed bandit" name comes from a metaphor about a gambler at a row of slot machines (each one a "one-armed bandit"). With limited pulls and unknown payoffs, the gambler must balance exploration (trying each machine to learn its payout) with exploitation (pulling the best-performing machine). Bandit algorithms apply this trade-off mathematically to A/B test traffic allocation.

In ecommerce, bandit algorithms are most useful at low to medium traffic where fixed-split A/B tests would take months to reach significance. The algorithm continuously updates its estimate of each variant's conversion rate and allocates more traffic to better-performing variants over time. By the end of the test, most traffic has gone to the winner — minimizing the opportunity cost of testing.

The trade-off is that bandit algorithms produce slightly biased estimates of variant performance (because allocation is correlated with performance) and are less suited to definitive yes/no decisions about specific hypotheses. They excel at "find the best of N options quickly" and underperform at "prove that variant B is meaningfully better than variant A".