Trustworthy Online Controlled Experiments

A Practical Guide to A/B Testing

; Diane Tang ; Ya Xu

Trustworthy Online Controlled Experiments

Getting numbers is easy; getting numbers you can trust is hard. This practical guide by experimentation leaders at Google, LinkedIn, and Microsoft will teach you how to accelerate innovation using trustworthy online controlled experiments, or A/B tests. Les mer
Vår pris
419,-

(Paperback) Fri frakt!
Leveringstid: Usikker levering*
*Vi bestiller varen fra forlag i utlandet. Dersom varen finnes, sender vi den så snart vi får den til lager

Vår pris: 419,-

(Paperback) Fri frakt!
Leveringstid: Usikker levering*
*Vi bestiller varen fra forlag i utlandet. Dersom varen finnes, sender vi den så snart vi får den til lager

Om boka

Trustworthy Online Controlled Experiments

Getting numbers is easy; getting numbers you can trust is hard. This practical guide by experimentation leaders at Google, LinkedIn, and Microsoft will teach you how to accelerate innovation using trustworthy online controlled experiments, or A/B tests. Based on practical experiences at companies that each run more than 20,000 controlled experiments a year, the authors share examples, pitfalls, and advice for students and industry professionals getting started with experiments, plus deeper dives into advanced topics for practitioners who want to improve the way they make data-driven decisions. Learn how to * Use the scientific method to evaluate hypotheses using controlled experiments * Define key metrics and ideally an Overall Evaluation Criterion * Test for trustworthiness of the results and alert experimenters to violated assumptions * Build a scalable platform that lowers the marginal cost of experiments close to zero * Avoid pitfalls like carryover effects and Twyman's law * Understand how statistical issues play out in practice.

Preface - how to read this book; 1. Introduction and motivation; 2. Running and analyzing experiments: an end-to-end example; 3. Twyman's law and experimentation trustworthiness; 4. Experimentation platform and culture; Part II: 5. Speed matters: an end-to-end case study; 6. Organizational metrics; 7. Metrics for experimentation and the Overall Evaluation Criterion (OEC); 8. Institutional memory and aeta-analysis; 9. Ethics in controlled experiments; Part III: 10. Complementary techniques; 11. Observational causal studies; Part IV: 12. Client-side experiments; 13. Instrumentation; 14. Choosing a randomization unit; 15. Ramping experiment exposure: trading off speed, quality, and risk; 16. Scaling experiment analyses; Part V: 17. The statistics behind online controlled experiments; 18. Variance estimation and improved sensitivity: pitfalls and solutions; 19. The A/A test; 20. Triggering for improved sensitivity; 21. Guardrail metrics; 22. Leakage and interference between variants; 23. Measuring long-term treatment effects.

This practical guide for students, researchers and practitioners offers real world guidance for data-driven decision making and innovation.

Fakta