Loading...
Answers
MenuWhat is a sufficient number of people to do A/B testing with? 10, 20, 100, 1000? Why? Should this be a closed group initially or not?
This question has no further details.
Answers
There's no set right answer. The more people the better the data. Traditionally most sample sizes are 100 people but think about it like this. Which has more power? 90% of 100 or 90% of 1000 all vote in your favor. Which would you pick? My recommendation would be use a sample of at least 100 people. 👊🏽
Simple Answer - 1000. However, dump a losing ad quickly. For example, if ad A is pulling in a 5 to 1 response over ad B, come up with a new ad B that attempts to beat ad A. You may see these results almost immediately. Examine the winning ad and beat it. Do not get hung up on the sample size if an ad stinks.
I hope this helps.
Best of Luck,
Mike
PS I may assist you as my time permits. Right now I am doing something that I haven't done for many years. I am actively soliciting new select clients due to selling one of my businesses and freeing myself up a little. But as you might expect, I am still very busy. That is how I like it.
TheLittleGuysWayToRiches.com
Statistically speaking, a pool of 30 respondents is enough to start seeing a trend. 100 is about the maximum. Beyond that it's just more of the same pattern and it might even start getting confusing, because the differences will start to level out.
Related Questions
-
How much time do you spend on google (or other search engine) when you are searching informations on a specific topic ?
As a tech marketer - I am always looking for the fastest, cheapest yet reliable research in order to create competitive intelligence internal documents, stats for blog posts, etc. Some people live by the "if it isn't on the first page, it doesn't matter" when it comes to leveraging search engines as a research tool. However, often when we are looking for value, we have to realize that anything REALLY worthwhile these days from a content perspective is typically gated behind a web form, or nestled inside a blog post. So, expect to spend a few hours in order to get through all the form gates, and also make sure you're using as specific and direct search terms as possible - then getting more generic if results are not ideal. Some other tips: -Have a few "burner" email addresses setup with gmail or hotmail, etc. - these allow you to get content when filling out forms to get White Papers, Reports, Case Studies, etc. without your primary inbox getting stuffed with marketing emails. -In the form fill process, if available put "student" or "researcher" as typically sales reps pass by these "leads" when the form gets dumped into the CRM queue. -Don't forget specialty search sites and tools like Wolfram Alpha for more "numbers" derived searches. -Also, like your question here - always seek the wisdom of crowds in addition to machine learning algorithms! Hope this is helpful! Search on!MS
-
Google for Online Shopping: How do I calculate market size and TAM, SOM and SAM? Do the three have to be different? (Please refer to details)
First of all, starting a business on revenue generated from affiliate sales or incidental revenue is going to be minor and not enough to scale much less build a business. Unless you're a "solo-preneur" you'll need some additional form of revenue if you intend to add to the software-as-service website after it's launched. Also, you need to determine a) if you have a budget allocated for completion of this project, and you would be able to estimate this based on completing a business plan (if you have not already done so), and then calling several agencies for estimates. Expect to pay (at least five grand to get it off the ground). As a developer, I used eCommerce daily, and would be interested in a related service that could measure factors such as tracking visitor peregrinations through my site and what drove them to place one order over another, for example, Most other data I get through the eCommerce provider. So I would say that one of the reason's it's difficult to come up with some of these factors is not yet know enough about the business model itself. You can't determine size of a market without first identifying specifically what it is, and how that may change. You'd need to try a trial run, talk to some similar ventures, (especially similar ones that are local so you can meet them personally). I have no issue with all the jargon you're using, but I do not agree with the sentiment that traditional business references won't help. You need a business plan if you have not already completed one. The next step is narrowing in on target audience; you're coming up with numbers and estimations based on theory and not on actual product launch and tests - and this can be fatal for any kind of launch where presumptions dominate over exact data. I think once you narrow this in more, focus on niche market(s), find some ways to test your concept (so you can get what's called proof of concept), figure out how you are going to finance this while it's struggling for its first years, figure out how to scale and measure ROI, you might have something to work with. I would go to Score.org, and look up some different business plans, business model mock-ups, find references to software as service, then I'd look up similar businesses on Manta, Dunn & Bradstreet, or other corporate intelligence sites, look for similar entities (most of which flopped statistically) and look for local businesses that are similar in practice or intent and break bread with them. There's simply not enough meat on the bones for me to speculate on possible income streams; it's like going in front of investors and explaining to them how big a market share is and how much it generates annually and then that your business will exploit that market share. You need to explain how, why it's different and unique, how you'll compete, and what your revenue sources will be and show some kind of sustained profit through at least 1 to 2 years before they'll part with a nickel. So by the same token you have to assume the onus of conducting more due diligence.DS
-
Is there a way to use paid advertising to test a domain name?
Yes. Use the "Link Text" in your facebook ads to A/B test various domain names. To do this, simply launch a few identical ads, and only change the link text to say the variation of the domain name you are considering. The ad with the highest CTR (not conversion rate or other KPI) will be the one that sparks the most interest in your target audience. Note: one domain may have a high relevance to one audience, but low relevance with another. So this tactic should only be used if you have a very specific demo you are targeting.AG
-
How do you get access to the right people to validate an idea?
The art of the "cold pitch" is definitely something that needs to be worked on, and doesn't come naturally to everyone. A couple of quick tips: 1. 4 is not a big enough target group, you've got to cast a bigger net. Try pitching 20, and aim to get 3-5 responses. 2. When sending a cold email, really think about what you are offering them. Whilst you'll get the odd good egg that simply wants to help - you can't expect entrepreneurs and small biz owners to take time out of their day to answer your questions. To counter, why not tell them you're conducting research in the space, and would be happy to send the finds/reports back to them in a nice format, which could in turn help their business. 3. Don't give up, keep hustling. Try changing around the emails slightly, track which emails convert into responses, and fine tune from there. Good luck and feel free to book a call if you want to chat more. Best James.JP
-
My direct competitor is growing fast, my start-up is still at an early stage. Do I need to cancel the initiative and try something new?
Even if your competitor continuously capitalizes on your ideas, by imitating your product features, etc., and even if he grows exponentially faster than you, it shouldn't intimidate you. Similar product, less money, whatever else, doesn't matter. Get awesomely creative with your marketing. Focus on the user experience and forget the competition. That's one thing no competitor can ever take away from you. And, the only thing you've got to mediate risk. I'd love to share more with you over a 15 minute call. I can guide you through getting a better understanding of your customers, and developing an awesome user experience, by moving beyond conventional marketing. Looking forward to it. AdrianAC
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.