Loading...
Answers
MenuWhat is a sufficient number of people to do A/B testing with? 10, 20, 100, 1000? Why? Should this be a closed group initially or not?
This question has no further details.
Answers
There's no set right answer. The more people the better the data. Traditionally most sample sizes are 100 people but think about it like this. Which has more power? 90% of 100 or 90% of 1000 all vote in your favor. Which would you pick? My recommendation would be use a sample of at least 100 people. 👊🏽


Simple Answer - 1000. However, dump a losing ad quickly. For example, if ad A is pulling in a 5 to 1 response over ad B, come up with a new ad B that attempts to beat ad A. You may see these results almost immediately. Examine the winning ad and beat it. Do not get hung up on the sample size if an ad stinks.
I hope this helps.
Best of Luck,
Mike
PS I may assist you as my time permits. Right now I am doing something that I haven't done for many years. I am actively soliciting new select clients due to selling one of my businesses and freeing myself up a little. But as you might expect, I am still very busy. That is how I like it.
TheLittleGuysWayToRiches.com
Statistically speaking, a pool of 30 respondents is enough to start seeing a trend. 100 is about the maximum. Beyond that it's just more of the same pattern and it might even start getting confusing, because the differences will start to level out.
Related Questions
-
Other than appdata.com, where can I find download & revenue numbers of successful iOS apps?
I believe Appannie.com is what you are looking for.
-
Is there a way to use paid advertising to test a domain name?
Yes. Use the "Link Text" in your facebook ads to A/B test various domain names. To do this, simply launch a few identical ads, and only change the link text to say the variation of the domain name you are considering. The ad with the highest CTR (not conversion rate or other KPI) will be the one that sparks the most interest in your target audience. Note: one domain may have a high relevance to one audience, but low relevance with another. So this tactic should only be used if you have a very specific demo you are targeting.
-
How do I do the right Market Research for a Product or App?
The best way to do validation on this idea would be to acquire customers through Facebook ads (rich targeting makes you able to cohort users nicely). Then, you could manually pair people via email or through a simple text messaging exchange facilitated by Twilio. In this scenario, you could prompt them to take the actions you think will be at the core of your app's functionality. Observing their engagement rate and what they do with each other would give you some insight into how viable this is. Happy to talk this through in a call with you. I've helped a lot of clarity members through similar customer development exercises.
-
How do you do market research?
Hi First of all, the fact that you are talking about market research ('validation') is great. I've seen so many startups invest time and money, only to find out later that no ones wants/needs their product, or that they only want a certain aspect of it (whilst the startup spent money developing a whole bunch of other features), or that they aren't willing to pay the requested price. So you're already one step ahead. Also, the sooner you validate your product, the better! The best ways to validate your idea is to see whether people are willing to actually pay for your product/service. The best way to get investors, is to show (1) that you are solving a need, (2) that you have traction, and (3) that the investors will get a nice return on their investment. As for methods of validation, here are the most efficient methods. 1. Check that people will actually buy/pay for the product. You can do this by setting up a Wix or Wordpress website (this can be done for free / very low costs). If your venture is a mobile app, then you can create what’s called a Clickable Prototype (“CP”) (a visual of the app in which the images change when you click on something - you can do so using the Apple Keynote tool or Microsoft PowerPoint). On the website, include the price of the product/service, and enable people to order it (YES, even if it doesn't really exist yet!). I am happy to explain how this can be done whilst still being fair to your potential customers (the people who click the "buy/download” button). 2. Determine your target market/customers. 3. Spend a small amount (say $100 - depending on your budget) on the most relevant platforms and promote your product/service - linking to the website that you created. This way, after only spending a very small amount, you will be able to know (if you did it right): a. Do people like your product. b. Do people want/need your product (not the same as 'a'). c. Are people willing to pay for your product? (not the same as 'b' - and this being the most important stage) d. How much they are willing to pay. (you can check this by having 2-3 landing pages with different prices on each). 2 last important points: a) In order to rule out external factors like an unattractive landing page or advertising campaign, and assuming you have the time, create multiple landing pages / advertising campaigns, with different designs. b) During the above process, don't forget to check how much it costs you for each user/customer that clicks the "buy" button. If for example each click on your promotion/advertisement costs you $2, and only every 10 people who click go on to the "buy" page - that means each sale is costing you $20. Then check what your average profit per sale is, and then you'll know if your service/product is worth pursuing (obviously there are additional factors like return customers, referrals etc, but you will get a good estimated/validation of the idea/business). Regarding questionnaires and talking with people, these methods are also very good, but it is crucial that you implement them correctly, otherwise your results might be misleading. I’ve successfully helped over 300 entrepreneurs and I’d be happy to help you. Doing validation correctly can save you a lot of time and money. Before the call, and in order to make the best of your time, please send me more information about your product/service and your target market. Best of luck!
-
How much is LinkedIn really useful in terms of job search?
some information related to success rate or similar quantification will be appreciated!
I am not sure of the specifics you're asking, it seems like a lot of technical data. This might help in your search for these statistics though: http://jobsearch.about.com/od/networking/a/linkedin.htm http://www.forbes.com/sites/nextavenue/2014/01/21/how-linkedin-is-thwarting-your-job-search/ http://mashable.com/2014/06/19/linkedin-job-search-app/ http://www.cio.com/article/2403150/linkedin/10-linkedin-tips-to-boost-your-job-search.html