- Write the hypothesis before you open PostHog
- Starting the wizard
- Step 1: Description
- Step 2: Variant rollout
- Step 3: Analytics
- Save, then launch
- What not to change once it's live
- A worked example for a SaaS signup page
- Statsig and GrowthBook as other options
- My take for product teams
- Frequently asked questions
In PostHog, go to Experiments and click New experiment. The wizard has three steps: Description (name, hypothesis and feature flag key), Variant rollout (variants, split and rollout percentage) and Analytics (inclusion criteria and metrics). Click Save as draft, then launch the experiment from its detail page.
Write the hypothesis before you open PostHog
An experiment is only as useful as the question behind it. Before you click anything, write one sentence: if we change this, we expect that metric to move, because of this reason. For example, a shorter signup form will raise completed signups because fewer people give up halfway. That sentence becomes your hypothesis in step one and your primary metric in step three.
Starting the wizard
Go to the Experiments section in the PostHog app and click the New experiment button in the top right. A guided wizard walks you through three steps, with a Guide panel on the right giving tips for each one.
Step 1: Description
Name the experiment and write your hypothesis. Each experiment is backed by a feature flag, which decides which variant each person sees. By default the wizard creates a new flag when you save, with a key generated from the experiment name. You can customise the key, or click Select existing flag to link one you already have. An existing flag must be multivariate, with 2 to 20 variants.
Step 2: Variant rollout
| Setting | Default | What you can change |
|---|---|---|
| Variants | A control and a test variant | Add more, up to 20 in total |
| Split | Equal between variants | Adjust the split |
| Rollout | All participants | Lower the Rollout percentage |
| Level | User | Group-targeted, if you've created groups |
Experiments need at least 2 variants. If you linked an existing flag in step one, this step becomes read-only, because the flag already defines the variants.
Step 3: Analytics
Here you decide who counts and what success means. By default, a user is considered exposed when the feature flag called event is recorded, and you can switch to a custom event if exposure happens somewhere more specific. Multiple variant handling defaults to excluding anyone exposed to more than one variant, which keeps results clean.
Then add metrics. Primary metrics decide whether the experiment succeeded. Secondary metrics watch for side effects, such as more signups but fewer people finishing onboarding. Metrics you use often can be saved as shared metrics at project level. PostHog lets you add metrics later, but it recommends defining them upfront.
Save, then launch
On the final step, click Save as draft. Nothing goes live yet. Check the experiment's detail page, then launch it from there when you're ready to start collecting data.
What not to change once it's live
PostHog Docs flag the change that quietly spoils results. Changing the split or adding variants after rollout can give users inconsistent experiences, since someone who saw one version may now see another. Increasing the rollout percentage doesn't reassign users, so widening the audience is safe. Plan the variants and split before launch, and treat them as fixed.
A worked example for a SaaS signup page
Say your hypothesis is that removing the company name field raises completed signups. Create the experiment with a control (current form) and one test variant (shorter form), keep the equal split, and start the rollout at a share of visitors you're comfortable with. Set signup completed as the primary metric and onboarding finished as a secondary one. Once you're confident nothing's broken, raise the rollout percentage; that won't reassign anyone. My guide on how to run A/B tests effectively covers how long to let it run.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Statsig and GrowthBook as other options
Statsig describes an experiment as a randomised controlled trial that splits traffic into a control group and test groups, with deterministic assignment so each user sees the same variant. GrowthBook's documentation lists delivery methods including Feature Flag Experiments, an AI Visual Editor and URL Redirects. For more choices, see my roundups of A/B testing tools and product analytics tools.
My take for product teams
Write the hypothesis first, keep the default control and test unless you've a reason for more, and set a primary metric before you save. Launch from the detail page, raise the rollout freely, but never change the split or variants mid-experiment.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
Frequently asked questions
How do I create an experiment in PostHog?
Go to Experiments, click New experiment and follow the three-step wizard.
How many variants are allowed?
At least 2 and up to 20.
Do I need a feature flag?
Each experiment is backed by one; the wizard creates it by default, or you can link an existing multivariate flag.
Can I launch without metrics?
Yes. You can add metrics later, though PostHog recommends defining them upfront.
Can I run experiments on groups instead of users?
Yes, if you've created groups.
