Recent Presentations, EVents & Interviews

Ask a CRO

Koalatative’s Gerda Thomas and Experiment Nation’s Rommil Santiago answer questions from the community in a new YouTube series.

Conferences

2025

Topics include: AI, and Career Advice

2024

Topics include: Working with HiPPOs, AI and PLG

2023

Topics include: Politics, Personalization and Low-Traffic

EVENTS

Trends that have to stop

Product Experimentation

Interviews

See our recent episodes below or watch them all on YouTube.

Season 5

Season 4

Season 3

Recent Episodes

Some episodes from the last couple of years

Don’t rely on macro goals with Gursimran (Simar) Gujral

Simar, CEO and Founder of OptiPhoenix, shared invaluable insights on A/B testing and CRO at Experiment Nation. Here are 5 key takeaways: Don’t rely solely on macro goals: Micro goals reveal user behavior, driving better testing strategies and informing future hypotheses. Iterative testing trumps scrapping: Learn from failed tests by adjusting hypotheses and running new experiments, boosting your win rate. QA is critical but often neglected: Invest time in thorough QA, ensuring your test is accurate and integrated correctly. Outsourcing AB test development can be beneficial: Consider outsourcing for faster scale-up, access to expert resources, and reduced development burden. Prioritize experimentation within your workflow: Make A/B testing an integral part of your product development process, leading to better decisions and faster results.

Read More

Navigating Experimentation at Startups with Avishek Basu Mallick

Avishek Basu Mallick, a Senior PM at CheapOair talks about: - Experimentation at scale: Even with high traffic, rigorous experimentation is crucial. - Idea generation: User research, NPS surveys, and internal stakeholders are valuable sources of experiment ideas. - Communicating progress: Regular updates with leadership build trust and ensure alignment. - Personalization is nuanced: Distinguish between customization, segmentation, and true personalization. - Generative AI’s potential: Explore its use for simplifying the travel planning experience.

Read More

People are the biggest challenge featuring Michael St Laurent

Michael St Laurent, a seasoned CRO expert, shared his insights on large-scale testing Here are 5 key takeaways for marketers: Prioritization is key: Large-scale testing requires prioritizing tests based on potential impact and aligning with overall business goals. AI is the future, but humans guide it: Mike emphasizes the power of AI in automating processes and analyzing data, but highlights the importance of human expertise and guidance. Data quality is crucial: AI models rely heavily on the quality of data. Invest in building a robust data infrastructure to ensure accurate results. People are the biggest challenge: Navigating internal politics and managing stakeholder expectations is often the biggest hurdle for large-scale testing. Building a testing culture is a win: The ultimate goal of large-scale testing is to create a data-driven culture that fosters continuous improvement and better user experiences.

Read More

Start Small and Stay Curious featuring Ellie Hughes

Ellie Hughes, a top CRO expert and Experimentor of the Year, shared some incredible insights in her recent Experiment Nation interview. Here are 5 takeaways to boost your experimentation game:- Start Small and Stay Curious: Ellie emphasizes that experimentation is a journey, not a destination. Start small, stay curious, and never stop learning!- Prioritize Value, Not Tasks: Focus your time on activities that truly drive client value, like uncovering insights and ideation, rather than spending hours on tedious tasks.- Set Clear Expectations: Clearly define reporting timelines and communicate about peeking, managing client expectations, and setting the stage for success.- Embrace Iteration: Experimentation is an iterative process. Be open to changing your process, adapting to new learnings, and continuously improving.- Maximize Impact Through Big Ideas: Larger changes can be easier to detect, resulting in faster results and more impactful growth. Encourage clients to think big!

Read More

Using measurement to test on low-traffic sites featuring Chris Mercer

Join us for a deep dive into conversion rate optimization on low-traffic sites with these actionable insights Chris MercerMeasure before optimizing: Understand the current user journey and behaviors to identify areas for improvement.Test sequentially: Iterate faster on low-traffic websites by testing changes sequentially instead of using AB tests.Focus on the right metrics: Prioritize actions that drive meaningful outcomes, such as lead generation or revenue.Listen to the market: Gather user feedback to inform your optimization strategy and meet their needs.Automate measurement: Leverage Google Analytics 4 to collect and analyze data effectively to guide decision-making.

Read More

Build a Learning Library featuring Nick Murphy

Discover the power of experimenting to learn and win! ?Nick Murphy shares his expertise on building learning libraries. Here are 5 key takeaways:- Learning libraries capture the “why” behind successful and unsuccessful tests.- Organize your learning library with consistent and digestible fields.- Use “broad” and “specific” learning columns for a layered understanding.- Encourage cross-team collaboration by sharing insights company-wide.- Learning libraries foster a culture of experimentation and growth.

Read More

How To Build An Unstoppable Alliance With CX Feat. Courtney Leblanc

In this episode, senior customer experience leader, Courtney LeBlanc, reveals the ideal partnership between CX and experimenters and shares invaluable insights on:Forging powerful partnerships between CX and experimentersNavigating the daily demands of a CX leaderSimplifying CX workflows with customer insights... and so much more!Don't miss out on this enlightening conversation from an expert in the CX game!

Read More

Use automation to build optimal flow states with Shawn David

Learn about human-first engineering in this in-depth interview where Shawn David discusses strategies to organize work and to break down tasks into smaller digestible sessions. He explained the most effective ways for teams to discover the work that should be automated the most by focusing on the desires and strengths of individuals, to remove work that is menial especially to them. We explored the impacts on company culture that menial tasks can cause and discussed some of the pitfalls with new technologies as well as lessons to keep in mind while reviewing new tools and technologies.

Read More

The Challenges of Building a Culture of Experimentation: Insights from Industry Leader Nima Yassini

Join Nima Yassini as he dives into experimentation culture, the journey of building a brand UX and digital marketing agency focused on conversion rate optimization, the integration of digital marketing and experimentation culture, the struggles and benefits of adopting a culture of experimentation, and the need for leadership and reinvestment in the experimentation industry.

Read More

Always start by testing your best variation with Zach Lebovics

Zach Lebovics shares insights on experimentation and product management. He discusses the importance of documenting assumptions, testing the best-case scenario first, and understanding that experiment failure isn't solely tied to the outcome. Zach also shares personal experiences and learnings from working in companies like Zynga, Ritual, and Tonal, providing valuable tips for product managers.

Read More

Product Management vs. CRO with Bryce York

In an insightful interview at Experiment Nation’s Conference, Bryce York, Director of Product Management at Tatari, delves into several key insights. He explores the parallels and distinctions between CRO and product management, especially in smaller companies where overlapping skills are crucial. Bryce shares his personal journey from entrepreneurship to product management, underscoring the significance of both qualitative and quantitative data in experiment design. Additionally, he emphasizes the importance of documentation and knowledge sharing in cultivating an experiment-driven culture.

Read More

Create a more impactful testing roadmap with this framework - ft. Edmund Beggs

Documentation. The unsung, yet incredibly important part of any experimentation program. Great documentation allows you to look back at past experiments and find patterns of what were the most effective levers to inform your testing roadmap and levers you probably can deprioritize. In today's episode of Experiment Nation, our own Charlotte April Bomford recently spoke to Edmund Beggs about the "Levers Framework" which allows Convert.com to deliver results for their clients in shorter amounts of time. If you'd like to learn more about this interesting framework, watch the full interview.

Read More

YouTube Video Index

We have so many Videos, I asked ChatGPT to sort them by topic. To be honest, I think it got it wrong. But it’s better than nothing.

Analytics

Topic Subtopic Summary Related Videos
Anomaly & Quality Data QA & alerts Catching anomalies and maintaining trust in data.
Cohorts & Retention Time-based cohorts Measuring retention and cohort behaviors.
Dashboards & Viz Reporting & storytelling Turning analysis into decisions and action.
Event Tracking Schema & governance Clean, governed event tracking for analysis.
Experiment Readouts Narrative & action Telling a crisp story and driving action from results.
Funnel & Pathing Drop-offs & sequences Analyzing user paths and funnel drop-offs.

CRO

Topic Subtopic Summary Related Videos
A/B & Multivariate Testing Test design Designing and running controlled experiments (A/B and MVT).
Accessibility Inclusive UX Ensuring accessible experiences that also convert.
Analytics Foundations Attribution & tracking Ensuring reliable measurement and attribution.
Checkout & Payments Payments UX Improving cart and payment success rates.
Experiment Operations Testing workflow How teams plan, QA, and ship tests consistently.
Forms & Inputs Error states & validation Reducing form friction with better validation and UX.
Funnel Optimization Friction & drop-off Finding and fixing funnel leaks to lift conversion.
Landing Pages & Copy Messaging & layout Optimizing landing page hierarchy, copy, and CTAs.
Micro-Interactions UI polish Small UI cues that clarify and motivate action.
Mobile UX App & responsive Optimizing mobile and app conversion patterns.
Performance Core Web Vitals Improving speed and stability for higher CVR.
Personalization Segmentation & targeting Tailoring experiences to segments to lift CVR.
Survey Analysis Voice of customer Turning survey data into testable hypotheses.

Experimentation

Topic Subtopic Summary Related Videos
Analysis & Debugging Outliers & sanity checks Post-test analysis and debugging wins/losses.
Hypotheses & Design Hypothesis frameworks Crafting hypotheses and prioritizing experiments.
MVT & Factorials Interactions & DOE Designing tests to learn interactions efficiently.
Metrics & Guardrails North-star & safety Balancing primary metrics with guardrails.
Multi-armed Bandits Adaptive allocation Adaptive allocation strategies vs classical A/B.
Power & Sizing Sample size & MDE Planning power and MDE before launch.
Program & Governance Process & ethics Operating a scalable, ethical program.
Sequential Testing Peeking & alpha spending Controlling error while monitoring experiments.

Growth/PLG

Topic Subtopic Summary Related Videos
Freemium & Trials Upgrade mechanics Designing freemium/trial flows that convert.
Lifecycle Messaging Email & in-product Targeted messaging to move users forward.
Monetization Pricing & packaging Optimizing pricing models and packaging.
Onboarding Guides & checklists Designing onboarding that accelerates activation.
Paid Acquisition CAC & ROAS Scaling paid channels while maintaining unit economics.
Retention Habit loops & churn Strengthening product habits and reducing churn.
SEO & Content Acquisition via content Compounding organic acquisition via content.
Self-Serve Checkout & procurement Optimizing self-serve purchase experiences.
Usage Funnels DAU/WAU & feature use Measuring and improving product usage funnels.
Virality & Referrals Loops & incentives Building referral loops and share incentives.