Recent Presentations, EVents & Interviews

Ask a CRO

Koalatative’s Gerda Thomas and Experiment Nation’s Rommil Santiago answer questions from the community in a new YouTube series.

Conferences

2025

Topics include: AI, and Career Advice

2024

Topics include: Working with HiPPOs, AI and PLG

2023

Topics include: Politics, Personalization and Low-Traffic

EVENTS

Trends that have to stop

Product Experimentation

Interviews

See our recent episodes below or watch them all on YouTube.

Season 5

Season 4

Season 3

Recent Episodes

Some episodes from the last couple of years

2021-03-27 2021-03-27

Specsavers' Melanie Kyrklund on How to run a high-impact optimization program

In this episode, Rommil and Tracy interview Melanie Kyrklund from Specsavers on how to run a high-impact optimization program, how to align the program with a product's vision, and how she likes black licorice.

Podcast: Gartner's Shiva Manjunath on how Conversion Rate Optimization and Best Practices are terribly named

YouTube Video Index

We have so many Videos, I asked ChatGPT to sort them by topic. To be honest, I think it got it wrong. But it’s better than nothing.

Analytics

Topic	Subtopic	Summary	Related Videos
Anomaly & Quality	Data QA & alerts	Catching anomalies and maintaining trust in data.	CRO QA: Why Implementation QA Matters with Daniel Ramirez and Liz King Convert QA Wizard: Easy Experiment QA with Ruben de Boer Dealing with revenue metrics outliers with Ishan Goel No QA wasted $300,000
Cohorts & Retention	Time-based cohorts	Measuring retention and cohort behaviors.	#shorts - The CRO Survival guide with Carlos Trujillo - 💪 #shorts : What’s the best time to send an email with Deborah O'Malley 5 Ways Onboarding Drives User Retention featuring Aarushi Raizada Return to Basics: Analytics & Interpretation in A/B Testing Presented by Ellie Hughes The CRO Survival guide with Carlos Trujillo What’s the best time to send an email? ft. Deborah O'Malley
Dashboards & Viz	Reporting & storytelling	Turning analysis into decisions and action.	Essential tools and strategies for modern experimentation featuring Olga Kosyreva Experiment Nation's 2024 CRO Salary Report and how to paid what you're worth with Tracy Laranjo Optimizing Freemium Models: Welcome to the land of flat results ft. Dorian Gonzalez Optimizing Subscription Models ft. Cristina Molina Maestro The Digital Experimentation Model with Marylin Montoya
Event Tracking	Schema & governance	Clean, governed event tracking for analysis.	#shorts - Coming soon: Experiment Nation's Product #Experimentation Event! Coming soon! Our Product Experimentation Event! SRM and Experiment Tracking (Jonas Alves)
Experiment Readouts	Narrative & action	Telling a crisp story and driving action from results.	Are you leveraging experimentation to make smarter decisions? #productmanagement #experimentation
Funnel & Pathing	Drop-offs & sequences	Analyzing user paths and funnel drop-offs.	Experimentation post-conversion: Switching from buying journey to customer journey - Juliana Jackson Funnel experimentation within app environments with Trang Quach HIGHLIGHTS: Use automation to build optimal flow states with Shawn David Making an impact when you only own a small part of the user journey Optimize the Journey: Captivating User Experience at Every Turn with Achille Vaulot Optimizing the entire customer journey vs just the website Use automation to build optimal flow states with Shawn David

CRO

Topic	Subtopic	Summary	Related Videos
A/B & Multivariate Testing	Test design	Designing and running controlled experiments (A/B and MVT).	#shorts - How to be good at coding A/B tests with Jereon Wieresema #shorts - How to be good at coding A/B tests with Jeroen Wiersma Avoid AB Test issues: AB testing fallacies and practical tips on how to avoid them ft. Ellie Hughes Creating an experimentation program that’s more than A/B testing (Annika Thompson and Paul Randall) Don't A/B Test?! featuring Solomon Kahn Ground Rules for A/B Test Validity with Aline Roux How SiteSpect handles ad blockers when running A/B tests How to be good at coding A/B tests with Jeroen Wiersma How to make a better sandwich: An introduction to multivariate testing (Danielle Gunraj) How to make a better sandwich: An introduction to multivariate testing (Danielle Gunraj) Not every insight is derived from an A/B Test (Daniela Marquez) Optimize Your Testing Program via the Inclusion of a Shared Control Group w/ H Alexander & L Youtsey Return to Basics: Analytics & Interpretation in A/B Testing Presented by Ellie Hughes Seven lessons from running 22,000 AB tests with Ayat Shukairy Stop False Winners in A/B Tests: Bonferroni Correction Explained Stop running simple A/B tests wrong! A/B testing basics with Deborah O'Malley Targeting and breaking down a/b tests by multiple audiences with CROCT Track A/B Tests in Real Time with Convert’s Live Duration Insights Why Your A/B Testing Stats Are WRONG and How to Fix them! Sequential Testing with Ishan Goel
Accessibility	Inclusive UX	Ensuring accessible experiences that also convert.	Always start by testing your best variation with Zach Lebovics How to make a better sandwich: An introduction to multivariate testing (Danielle Gunraj) How to make a better sandwich: An introduction to multivariate testing (Danielle Gunraj) Standing-up a CRO Program: How to build experimentation from ground up ft. Maria Luiza de Lange
Analytics Foundations	Attribution & tracking	Ensuring reliable measurement and attribution.	SRM and Experiment Tracking (Jonas Alves)
Checkout & Payments	Payments UX	Improving cart and payment success rates.	3-column checkout layout: Verifone case study by Nicoleta Danilet 5 Shopify Checkout Ideas to Experiment With ft. Charlotte Bomford Experiment Cartography :: Building the Perfect Roadmap (Kenya Davis and Krissy Tripp) How to optimize your shopping cart: Tips and Live teardown with Adrian Timar
Experiment Operations	Testing workflow	How teams plan, QA, and ship tests consistently.	Build a testing backlog with the business team featuring Marylisa Santos CRO QA: Why Implementation QA Matters with Daniel Ramirez and Liz King Convert QA Wizard: Easy Experiment QA with Ruben de Boer No QA wasted $300,000 Stop struggling with testing - start pre-test planning with Haley Carpenter
Forms & Inputs	Error states & validation	Reducing form friction with better validation and UX.	Bridging the Discovery Validation Gap featuring Ellie Hughes Data-driven Form Optimization with Alun Lucas Get the most out of form optimization! Ft. Ana Kerkovic Getting fast feedback to inform growth marketing #experimentation #cro #motivation #podcast Misinformation spreads fast Profitable CRO: Transforming Internal Experimentation Team into a Profit Center with Alisa Antypova
Funnel Optimization	Friction & drop-off	Finding and fixing funnel leaks to lift conversion.	5 Steps to build a structured experimentation program for your product ft. Dennis Nederlof Creating CRO Processes from Scratch with Steph Le Prevost Ethical Testing & Design in Experimentation: Leaks vs. Puddles Framework (w/ Namata Agarwal) Funnel experimentation within app environments with Trang Quach Realizing women's full potential in 6 steps with Bithika Mehra
Landing Pages & Copy	Messaging & layout	Optimizing landing page hierarchy, copy, and CTAs.	#shorts - Tearing down others helps no one in the CRO industry with Matt Scaysbrook 2 Experimentation ideas that helped my Product Management Career with Olabanji Ewenla 3-column checkout layout: Verifone case study by Nicoleta Danilet Headlines for December 14, 2020 Landing Page Optimization: Your landing page should end your ad story ft. Gerda Vogt-Thomas What you need to know about Landing pages with Claire More Why designing before writing the copy is a bad idea featuring Magdalena Baciu
Micro-Interactions	UI polish	Small UI cues that clarify and motivate action.	Tooltips: Experiment across Devices featuring CROCT Tooltips: Group Sequential Testing featuring SiteSpect
Mobile UX	App & responsive	Optimizing mobile and app conversion patterns.	Boost Your App's Growth #podcast #experimentation #cro Customer Value Optimization Explained with Defiant's Jason Chappel Funnel experimentation within app environments with Trang Quach Instant Mobile App Changes without Code Releases featuring SiteSpect Navigating App Optimization with Ekaterina (Shpadareva) Gamsriegler Navigating App Optimization with Ekaterina (Shpadareva) Gamsriegler Run Overlapping Experiments Without Conflicts with Croct Staying true to experimentation: Navigating the scrappy world of mobile apps featuring Jason Gossett Why Users Uninstall Apps The Login Struggle w/ Juliana Jackson #experimentation #appoptimization
Performance	Core Web Vitals	Improving speed and stability for higher CVR.	#shorts : Moving Beyond Spaghetti Testing with Haley Carpenter Getting Buy-in For Experiment Ideas with Shiva Manjunath Getting buy-in on your optimization program with Andra Baragan Getting fast feedback to inform growth marketing #experimentation #cro #motivation #podcast Getting started leading an experimentation programme with Cristina Pujol Learn how to speed up your CRO program Moving Beyond Spaghetti Testing (Haley Carpenter) Moving Beyond Spaghetti Testing (Haley Carpenter) Save time with Convert's Import & Export Project Settings
Personalization	Segmentation & targeting	Tailoring experiences to segments to lift CVR.	All you need to know about Personalization with Munir Al-Dajani Have a strict definition of your target audience with Dr. Else van der Berg Make Personalization Personal with Care and Competence with David Mannheim Making Personalization Personal ft. David Mannheim Personalization timing matters - Avoid these common mistakes ft. Eric Melchor Targeting and breaking down a/b tests by multiple audiences with CROCT
Survey Analysis	Voice of customer	Turning survey data into testable hypotheses.	7 tips for CRO Program Success with Chris Marsh Access to the 2023 conference Best practices might be killing your experimentation culture – 3 reasons why (Ana Catarina Cizilio) CRO Organizational Structure- Centralized Team vs Process: with Oliver Paton Creating CRO Processes from Scratch with Steph Le Prevost How to rally your team around Voice of Customer research (Tracy Laranjo) How to rally your team around Voice of Customer research (Tracy Laranjo) How to rally your team around Voice of Customer research (Tracy Laranjo) Leave your Best Practices at home Optimizing organizational processes with Ruben de Boer Semantic Textual Analysis (Natural Language Processing) Ft. Juliana Jackson & Krasimir Bambalov Targeting and breaking down a/b tests by multiple audiences with CROCT The art and science of improving experiences with Namrata Agarwal and Luiza de Lange Tooltips: Experiment across Devices featuring CROCT Unlocking Success The 3 Peaks of Experimentation #experimentation #business #changemanagement

Experimentation

Topic	Subtopic	Summary	Related Videos
Analysis & Debugging	Outliers & sanity checks	Post-test analysis and debugging wins/losses.	Dealing with revenue metrics outliers with Ishan Goel
Hypotheses & Design	Hypothesis frameworks	Crafting hypotheses and prioritizing experiments.	Advice for new Experimenters Best practices might be killing your experimentation culture – 3 reasons why (Ana Catarina Cizilio) Create a more impactful testing roadmap with this framework - ft. Edmund Beggs Ethical Testing & Design in Experimentation: Leaks vs. Puddles Framework (w/ Namata Agarwal) How to rally your team around Voice of Customer research (Tracy Laranjo) How to rally your team around Voice of Customer research (Tracy Laranjo) How to rally your team around Voice of Customer research (Tracy Laranjo) How to use questions in CRO and how to build a Questioning framework with Riccardo Vandra Leave your Best Practices at home Mixed Methods: How UX Research can supercharge your experimentation practice with Harriet Swan Solve Problems with the Unstructured Brain Framework with Juliana Jackson Tooltips: Experiment across Devices featuring CROCT
MVT & Factorials	Interactions & DOE	Designing tests to learn interactions efficiently.	How to make a better sandwich: An introduction to multivariate testing (Danielle Gunraj) How to make a better sandwich: An introduction to multivariate testing (Danielle Gunraj)
Metrics & Guardrails	North-star & safety	Balancing primary metrics with guardrails.	Dealing with revenue metrics outliers with Ishan Goel Embracing Failure Metrics in Experimentation #experimentation #business #podcast #motivation Guardrail Metrics explained by Ishan Goel Guardrail Metrics with Ishan Goel Stop Cherry Picking Metrics with Jyoti Malik The Importance of Diversifying Your Experiment Metrics with Carolanne Sinacore
Multi-armed Bandits	Adaptive allocation	Adaptive allocation strategies vs classical A/B.	Creating an experimentation program that’s more than A/B testing (Annika Thompson and Paul Randall)
Power & Sizing	Sample size & MDE	Planning power and MDE before launch.	Get your Sample Size right featuring Ishan Goel Low-traffic testing: How to test with a low sample size ft. Rosie Hoggmascall The power of CRO mentorship with Tracy Laranjo
Program & Governance	Process & ethics	Operating a scalable, ethical program.	Challenges of a center of excellence with Rommil Santiago and Ben Lebay
Sequential Testing	Peeking & alpha spending	Controlling error while monitoring experiments.	Tooltips: Group Sequential Testing featuring SiteSpect Why Your A/B Testing Stats Are WRONG and How to Fix them! Sequential Testing with Ishan Goel

Growth/PLG

Topic	Subtopic	Summary	Related Videos
Freemium & Trials	Upgrade mechanics	Designing freemium/trial flows that convert.	Optimizing Freemium Models: Welcome to the land of flat results ft. Dorian Gonzalez
Lifecycle Messaging	Email & in-product	Targeted messaging to move users forward.	#shorts : What’s the best time to send an email with Deborah O'Malley AI Integration in the Product Experimentation Lifecycle with Nils Stotz AI and the Experimentation Lifecycle featuring Joey Herzberg Accelerating your business through Customer Lifecycle Understanding with Kritika Jalan Experimentation post-conversion: Switching from buying journey to customer journey - Juliana Jackson Funnel experimentation within app environments with Trang Quach Highlights: AI and the Experimentation Lifecycle featuring Joey Herzberg Making an impact when you only own a small part of the user journey Multi Faceting Your DTC Brand to Optimize Email Revenue with Trina Moitra Optimize the Journey: Captivating User Experience at Every Turn with Achille Vaulot Optimizing the entire customer journey vs just the website What’s the best time to send an email? ft. Deborah O'Malley
Monetization	Pricing & packaging	Optimizing pricing models and packaging.	Stop struggling with testing - start pre-test planning with Haley Carpenter
Onboarding	Guides & checklists	Designing onboarding that accelerates activation.	#shorts - The CRO Survival guide with Carlos Trujillo - 💪 5 Ways Onboarding Drives User Retention featuring Aarushi Raizada A Beginner’s Guide to CRO: How to Get Started Ft. Carla Quizon Experiment Nation: 2021's Worst CRO Jargon Tournament - Coming soon! Maximizing Results: A Comprehensive Guide to Ad Network Experimentation ft. João Marcelo Rodrigues The CRO Survival guide with Carlos Trujillo Tooltips: Experiment across Devices featuring CROCT Tooltips: Group Sequential Testing featuring SiteSpect
Paid Acquisition	CAC & ROAS	Scaling paid channels while maintaining unit economics.	Experiment Nation's 2024 CRO Salary Report and how to paid what you're worth with Tracy Laranjo
Retention	Habit loops & churn	Strengthening product habits and reducing churn.	#shorts - How to change visitor's habits with running server-side tests with Lucia van den Brink 5 Ways Onboarding Drives User Retention featuring Aarushi Raizada How to change visitor's habits with running server-side tests with Lucia van den Brink
SEO & Content	Acquisition via content	Compounding organic acquisition via content.	A pointless debate: Long vs Short Content Real Time Content Replacement Without Touching Code featuring SiteSpect Using a Modular Content System to Streamline Testing and Optimize Conversions w/ Kay del Rosario Which converts better? AI or human-created content? with Deborah O'Malley
Self-Serve	Checkout & procurement	Optimizing self-serve purchase experiences.	Scaling a self-serve Experimentation Program with Sabrina Naqvi
Usage Funnels	DAU/WAU & feature use	Measuring and improving product usage funnels.	Leveraging Workshops for Experimentation Buy-In and Engagement with Gertrud Vahtra
Virality & Referrals	Loops & incentives	Building referral loops and share incentives.	Optimize Your Testing Program via the Inclusion of a Shared Control Group w/ H Alexander & L Youtsey