Loading...
Answers
MenuWhat is a sufficient number of people to do A/B testing with? 10, 20, 100, 1000? Why? Should this be a closed group initially or not?
This question has no further details.
Answers
There's no set right answer. The more people the better the data. Traditionally most sample sizes are 100 people but think about it like this. Which has more power? 90% of 100 or 90% of 1000 all vote in your favor. Which would you pick? My recommendation would be use a sample of at least 100 people. 👊🏽
Simple Answer - 1000. However, dump a losing ad quickly. For example, if ad A is pulling in a 5 to 1 response over ad B, come up with a new ad B that attempts to beat ad A. You may see these results almost immediately. Examine the winning ad and beat it. Do not get hung up on the sample size if an ad stinks.
I hope this helps.
Best of Luck,
Mike
PS I may assist you as my time permits. Right now I am doing something that I haven't done for many years. I am actively soliciting new select clients due to selling one of my businesses and freeing myself up a little. But as you might expect, I am still very busy. That is how I like it.
TheLittleGuysWayToRiches.com
Statistically speaking, a pool of 30 respondents is enough to start seeing a trend. 100 is about the maximum. Beyond that it's just more of the same pattern and it might even start getting confusing, because the differences will start to level out.
Related Questions
-
How can I determine the TAM and SAM of an SAAS company I am looking to startup?
Total Addressable (at least in the US) can be built based on market data sets, both free and for fee. For early child care centers, Market Data Retrieval and NCES (publicly available data set) are two recommended sources. For Serviceable, that depends on a) your own business / operating model and b) the current and projected penetration of comparable competitors in market. For the former, consider things like technology usage and implementation capabilities. For the latter, if you have identified clear competitors, it is unlikely that you will have access to accurate info. So, if you can estimate these numbers and reverse-engineer, that's a good starting point. This would also be a great time to consider how your SAAS product is differentiated and will give you a chance to think through different scenario models for customer "switching" (from an existing competitor product to your own).DS
-
How much is LinkedIn really useful in terms of job search?
some information related to success rate or similar quantification will be appreciated!
I am not sure of the specifics you're asking, it seems like a lot of technical data. This might help in your search for these statistics though: http://jobsearch.about.com/od/networking/a/linkedin.htm http://www.forbes.com/sites/nextavenue/2014/01/21/how-linkedin-is-thwarting-your-job-search/ http://mashable.com/2014/06/19/linkedin-job-search-app/ http://www.cio.com/article/2403150/linkedin/10-linkedin-tips-to-boost-your-job-search.htmlPM
-
Other than appdata.com, where can I find download & revenue numbers of successful iOS apps?
I believe Appannie.com is what you are looking for.AC
-
Is there a way to use paid advertising to test a domain name?
Yes. Use the "Link Text" in your facebook ads to A/B test various domain names. To do this, simply launch a few identical ads, and only change the link text to say the variation of the domain name you are considering. The ad with the highest CTR (not conversion rate or other KPI) will be the one that sparks the most interest in your target audience. Note: one domain may have a high relevance to one audience, but low relevance with another. So this tactic should only be used if you have a very specific demo you are targeting.AG
-
How do I do the right Market Research for a Product or App?
The best way to do validation on this idea would be to acquire customers through Facebook ads (rich targeting makes you able to cohort users nicely). Then, you could manually pair people via email or through a simple text messaging exchange facilitated by Twilio. In this scenario, you could prompt them to take the actions you think will be at the core of your app's functionality. Observing their engagement rate and what they do with each other would give you some insight into how viable this is. Happy to talk this through in a call with you. I've helped a lot of clarity members through similar customer development exercises.TW
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.