What is statistical significance in A/B testing?

Statistical significance tells you whether the difference in conversion rates between your control and variant is likely due to a real effect rather than random chance. A result is typically considered significant when the p-value falls below your chosen threshold (e.g. p < 0.05 for 95% confidence).

What confidence level should I use for my A/B test?

95% confidence (p < 0.05) is the industry standard for most CRO experiments. Use 99% when the change is high-stakes (pricing, checkout flow). Use 90% for low-risk experiments where you need faster decisions with less traffic.

How much traffic do I need for a statistically significant A/B test?

It depends on your baseline conversion rate, expected lift, and desired confidence level. As a rule of thumb, aim for at least 100 conversions per variant before drawing conclusions. Small conversion rates and small expected lifts require much larger sample sizes.

What is a p-value and how do I interpret it?

The p-value is the probability of observing a difference this large (or larger) purely by chance if there were no real effect. A p-value of 0.05 means there's a 5% chance the result is a false positive. Lower p-values indicate stronger evidence against the null hypothesis.

What does relative lift mean?

Relative lift (or uplift) is the percentage change in conversion rate from control to variant: ((variant CVR − control CVR) / control CVR) × 100. A +10% relative lift means your variant converts 10% better than the original, not that conversions increased by 10 percentage points.

Is my data stored or sent anywhere?

No. All calculations happen entirely in your browser using JavaScript. No data is ever sent to our servers. Your experiment data stays completely private.

Free A/B Testing Tool

Is Your A/B Test
Statistically Significant?

Enter your control and variant data to instantly calculate statistical significance, p-value, z-score, and relative lift. Make data-driven decisions with confidence.

🔬

Z-Score & P-Value

Industry-standard two-proportion z-test gives you exact p-values and z-scores in milliseconds.

🎯

Confidence Levels

Choose 90%, 95%, or 99% confidence thresholds to match your risk tolerance and sample size.

📈

Relative Lift

See the percentage improvement (or decline) of your variant vs control conversion rate at a glance.

Enter Experiment Data

ControlOriginal version

Visitors

Conversions

VariantTest version

Visitors

Conversions

Confidence Level

Frequently Asked Questions

Everything you need to know about A/B Test Calculator.

Is Your A/B TestStatistically Significant?

Z-Score & P-Value

Confidence Levels

Relative Lift

Frequently Asked Questions

Is Your A/B Test
Statistically Significant?