Your A/B Test Dropped Conversion 8% in 6 Days… Now What?

Koalatative's Gerda Thomas and Experiment Nation's Rommil Santiago answer your CRO questions

An ongoing series that will go on until one of us is tired of answering questions.

Koalatative's Gerda Thomas and Experiment Nation's Rommil Santiago answer CRO questions from the community.

Chapters

• 00:00 Burger Menu Debate: Left vs Right

• 01:28 Mystery Meat Navigation & UX History

• 03:03 Why Modern Phones Break UX

• 05:08 Community CRO Scenarios Begin

• 06:18 Experiment Tanks Conversion by 8%

• 07:33 Root Cause Analysis & Leadership Pressure

• 09:17 QA, Tracking Errors & Failed Tests

• 10:03 Building a CRO Analysis Tool

• 12:18 Probabilistic Root Cause Detection

• 14:26 Managing Panic During Bad Experiments

• 15:21 Guardrail Metrics Explained

• 16:40 Using Control Limits in Experimentation

• 18:18 Building an Experimentation Knowledge Bank

• 20:05 Why You Don’t Need AI for Everything

• 21:08 Why Process Changes Create Admin Pain

• 23:08 Migrating Between Experimentation Tools

• 24:31 Why Teams Always Go Back to Spreadsheets

• 26:50 CRO Teams vs Developers & Designers

• 29:09 Why Early Experiments Should Be Simple

• 30:41 Why Developers Hate “Throwaway Code”

• 32:16 Designers vs Experiment Constraints

• 33:02 Outro & Next Community Questions