What is a sufficient number of people to do A/B testing with? 10, 20, 100, 1000? Why? Should this be a closed group initially or not? - Clarity Answers

Loading...

What is a sufficient number of people to do A/B testing with? 10, 20, 100, 1000? Why? Should this be a closed group initially or not?

Market Research

Cauvee [call-vay] answered

There's no set right answer. The more people the better the data. Traditionally most sample sizes are 100 people but think about it like this. Which has more power? 90% of 100 or 90% of 1000 all vote in your favor. Which would you pick? My recommendation would be use a sample of at least 100 people. 👊🏽

Tim Titan answered

Simple Answer - 1000. However, dump a losing ad quickly. For example, if ad A is pulling in a 5 to 1 response over ad B, come up with a new ad B that attempts to beat ad A. You may see these results almost immediately. Examine the winning ad and beat it. Do not get hung up on the sample size if an ad stinks.

I hope this helps.
Best of Luck,
Mike

PS I may assist you as my time permits. Right now I am doing something that I haven't done for many years. I am actively soliciting new select clients due to selling one of my businesses and freeing myself up a little. But as you might expect, I am still very busy. That is how I like it.
TheLittleGuysWayToRiches.com

Robert-Jan van Veen answered

Statistically speaking, a pool of 30 respondents is enough to start seeing a trend. 100 is about the maximum. Beyond that it's just more of the same pattern and it might even start getting confusing, because the differences will start to level out.