Minimum Detectable Effect (MDE)
Minimum Detectable Effect (MDE) is the smallest difference between two A/B test variants that you can reliably detect given your sample size, baseline conversion rate, and statistical confidence level.
Understanding Minimum Detectable Effect (MDE)
MDE is the practical answer to "what size lift can my test actually find?" It depends on three inputs: (1) your baseline conversion rate, (2) your sample size (sessions per variant), and (3) your statistical confidence and power requirements (typically 95% confidence, 80% power).
For a 2.5% baseline CVR with 95% confidence, the relationship is approximately: detecting a 5% relative lift needs ~110,000 sessions per variant; a 10% lift needs ~28,000; a 15% lift needs ~12,500; a 25% lift needs ~4,500. Lower baseline CVRs require larger samples; higher baselines require fewer.
Most merchants underestimate MDE. They expect to "test color of the button" with 1,000 sessions, but realistically can only detect a 25-35% relative effect at that volume — far larger than typical button color changes produce. The mismatch is why most low-traffic A/B tests run inconclusively for months.
The fix is either testing bigger changes (effects of 25%+ that low traffic can actually detect), aggregating across products to multiply effective sample size, or using bandit and genetic-algorithm approaches that don't require the binary fixed-split decision A/B tests do.
Why It Matters for E-Commerce
Setting unrealistic MDE expectations is the most common reason A/B tests fail to deliver value. Stores with 500 sessions/day cannot detect 5% effects in any reasonable time; trying to do so wastes weeks. Correctly setting MDE upfront prevents this and reframes the optimization strategy toward changes the test can actually find.
Related Terms
A/B testing is an experiment where two versions of a page, element, or experience are shown to different segments of visitors simultaneously to determine which version performs better against a defined metric.
Statistical significance is the probability that an observed difference between test variants is not due to random chance. In A/B testing it is usually expressed as a p-value below 0.05 or equivalently a confidence level of 95% or higher.
Conversion Rate Optimization (CRO) is the systematic process of increasing the percentage of website visitors who take a desired action, such as making a purchase, adding to cart, or signing up for a newsletter.
Related Articles
More about Minimum Detectable Effect (MDE)
Review Strategy for Fashion & Apparel Stores
Optimize reviews for fashion stores with fit photos, size info, and visual proof.
GuideMaximizing Revenue Per Visitor
Why RPV beats conversion rate, how to read RPV data, and optimization strategies.
How-toHow to A/B Test Review Layouts on Shopify
Learn how to A/B test different review layouts on your Shopify store. Find the review display format that drives the highest conversion rate.
How-toHow to Measure Review ROI for Your Store
Calculate the return on investment of your review strategy. Quantify how reviews impact conversion rate, revenue, and customer acquisition on Shopify.
ArticleWhy A/B Testing Fails on Low-Traffic Shopify Stores (And What Works Instead)
A/B testing needs more traffic than most Shopify stores have. Here is the sample-size math on why low-traffic A/B tests fail, and the continuous-optimization approach that works at low volume.
ArticleDoes Conversion Rate Optimization Actually Work? An Honest Look at the Data (2026)
Does CRO actually work? An honest, data-backed answer: where conversion rate optimization reliably pays off, why most CRO fails, and what kind of optimization delivers real lift on Shopify.
TipAdd Star Rating Filters to Your Review Section
Rating filters let shoppers find the reviews they trust most. Learn why letting customers filter by star rating increases conversion, not decreases it.
TipTurn Sizing Complaints Into a Conversion Tool
Sizing complaints in reviews reduce returns when surfaced properly. Shoppers who read "runs small, size up" buy with confidence and return less.
ProblemPoor Mobile Conversion Rate
Mobile shoppers convert at half the rate of desktop visitors. Learn how AI-optimized mobile review layouts can close the gap and boost Shopify mobile sales.
ProblemLow Checkout Completion Rate
Shoppers enter checkout but do not complete their purchase. Learn how building stronger product confidence through optimized reviews increases completions.
GlossaryStatistical Significance
Statistical significance is the probability that an observed difference between test variants is not due to random chance. In A/B testing it is usually expressed as a p-value below 0.05 or equivalently a confidence level of 95% or higher.
GlossaryMulti-Armed Bandit
A multi-armed bandit is an optimization algorithm that allocates traffic between variants dynamically — gradually shifting more traffic to better-performing options while continuing to test the others, instead of running a fixed split until a winner is declared.
Ready to optimize your reviews?
Eevy AI uses genetic algorithms to continuously optimize how reviews are displayed on your Shopify store — maximizing revenue per visitor.
Get Started Free