This course was created with the
course builder. Create your online course today.
Start now
Create your course
with
Autoplay
Autocomplete
Previous Lesson
Complete and Continue
A/B Test like a Pro! (online course)
#0 - WELCOME
Welcome to the course (0:55)
About me (1:17)
What's in the course? How to get the most out of it. (4:15)
#1 - INTRODUCTION
What is A/B testing exactly? (3:01)
The correlation vs. causation issue (4:47)
Why do you need A/B testing? (a story and the real reason) (4:23)
What can you A/B test? (8:52)
EXTRA: Hand-picked A/B Testing case studies
ACTION: The A/B tests you've run so far -- and the A/B tests you want to run in the future
#2 - IMPORTANT INITIAL BUSINESS QUESTIONS
Important A/B Testing mindset: Conversion Rate Optimization or Research? (3:42)
Two key questions before starting an A/B Test. (And when is A/B testing a really bad idea?) (6:45)
When you don't need to run experiments (6:32)
When you need to run experiments (5:30)
A/B test only when you need it (2:09)
EXTRA: the "can I run this A/B test" checklist
ACTION: Filter out the bad A/B test ideas from your backlog!
#3 - STATISTICAL BACKGROUND
Statistics is important (1:30)
Randomization (we all are terrible "random-generators") (3:01)
Statistical significance and the certainty of your results (4:37)
An intuitive way of interpreting the importance of statistical significance (2:02)
80%? 95%? 99%? What's the right confidence level? (6:45)
Calculating the required sample size before the test (4:57)
EXTRA: online calculators (statistical significance and sample size)
ACTION: Review your A/B testing backlog!
VAULT: Q&A
Your questions answered.
#4 - EXECUTION - STEP 1: PREPARING YOUR A/B TEST(S)
The four steps of executing an A/B test (1:39)
The Research Framework (14:14)
Your success metric(s) to evaluate your test (7:36)
A few words about setting a hypothesis (5:05)
EXTRA: A/B Testing Hypothesis Form
ACTION: Run your first research round!
#4 - EXECUTION - STEP 2: IMPLEMENTATION
My favorite A/B testing tools (6:06)
Tool demo (Google Optimize) (16:01)
Avoiding website flickering in an A/B test (4:38)
The change-one-thing-at-a-time myth, multivariate tests, A/B/n tests and unusual audience splits (7:04)
A few typical mistakes at the implementation phase (3:13)
EXTRA: The "can I start the A/B test?" checklist
ACTION: Try out the simplest ever A/B test!
#4 - EXECUTION - STEP 3: RUNNING A TEST
How long should an A/B test run? (2:42)
Which part of the year/week shouldn't you run an A/B test? (1:53)
Typical mistakes while running an A/B test (5:57)
#4 - EXECUTION - STEP 4: EVALUATING YOUR EXPERIMENTS
Don't lie to yourself! (4:36)
What to do when a test didn't perform the way you expected (2:28)
What to do after evaluating your experiment (6:29)
ACTION: Start your own knowledge base!
#5 - A FEW SMALLER ADDITIONS
The limitations of A/B testing (2:26)
The art and science of A/B testing (3:15)
Segmentation (2:37)
EXTRA: Further reads and videos
#6 - SUMMARY
Summary & The right mindset to be successful with A/B testing (2:29)
Where to go next
VAULT: Q&A
Your questions answered
VAULT #2: Running a redirect A/B test without 3rd party tools -- by writing your own code.
Why?
1) Traffic allocation: sending visitors to the old/new landing pages (with JavaScript)
2) Data collection and visualization (Bash + SQL + Google Data Studio)
3) Evaluation: significance calculations (using Python)
Conclusion
The change-one-thing-at-a-time myth, multivariate tests, A/B/n tests and unusual audience splits
Lesson content locked
If you're already enrolled,
you'll need to login
.
Enroll in Course to Unlock
course builder. Create your online course today.
Start now
course builder. Create your online course today.
Start now
course builder. Create your online course today.
Start now