Extended Understanding of Bayesian A/B Testing
How to use Bayesian A/B Testing Efficiently
In my earlier post, I talked about the fundamentals of Bayesian A/B testing and described how it is able to answer one of the most fundamental questions in all A/B tests — Is treatment better than control? Beyond that, I have also left a few unanswered questions that you might be facing when actually applying bayesian A/B testing.
- What happens if I want to carry out tests with more than 2 groups, e.g. A/B/C/D …?
- How to reduce the computational cost?
- Is there any technique to choose a prior distribution?
In this post, I will try to answer the above questions. Now let’s get into it.
Table of Contents
- Multiple Treatments
- Monte Carlo Sampling
- Technique for Selecting a Prior
- Conclusion
- Credit and References
Multiple Treatments
In our last post, we derived this final cost function
Where it is only applicable to two groups, e.g. one treatment and one control. Naturally, you would think about what to do with multiple treatments. Actually, it is not…