Loading...
Answers
MenuBuilding a database of A/B testing examples
Answers
Yes definitely. As you know these elements are super important pertaining to to this information. There are plethora of small businesses that cannot the technical expertise needed in some cases to increase in engagements on websites, which for most websites are the best promotion tools. This would be a great asset for small to medium size business who want to increase the proficiency of engagements for one. Some other ideas I could think of too. I would explore those engagements with those users to see if there is an noticeable difference before and after using the information provided from your db.
Great idea - I'd be interested, especially in pricing A/B test results. Are you up and running already?
There are already a couple of products in market like this, goodui.org being the main one.
However, personally, I don't believe that this kind of meta-analysis of experiments from different sites has too much value, for a couple of reasons. I'm only really outlining this stuff for you because it might be food for thought if you want to create products for the experimentation community:
Whilst the theory of meta-analysis is technically sound, it is also fraught with issues relating to the complexity of web experiences:
Tests must be subjectively categorized and then analysed as 'patterns' when there is zero evidence that that thing was the cause of the result. For example, you might think a test won because you removed the voucher code box, but it equally could have won because of the change in page layout.
No two tests are in any way the same in any controlled way. Meta analysis mainly comes from sciences where experiments can be repeated under (relatively) identical controlled conditions. This is impossible and A/B testing cannot really be compared to that kind of controlled experimentation. By comparing two different experiments you instantly remove any control you had around audiences etc because they will not be the same.
The greatest benefit to be had from experimentation is by integrating it with business and brand strategy. A good business strategy is aimed at carving out a unique and differentiated direction of travel for that brand. The opportunity with experimentation is to iteratively develop and innovate the business through learning and adaptation. This means that experimentation should aim at being meaningful within the context of the brand strategy. Meta-analysis of experiments from different sites is, in many ways, the opposite of this because it seeks to make the ‘lowest common denominator’ changes which would be beneficial to any business anywhere. Conversion uplift and revenue might be achieved but what has been furthered in terms of strategy and differentiation? This is not to say that it isn’t valuable if you simply want to do ‘CRO’ but my argument is that nobody should merely do CRO.
Related Questions
-
How should we plan a well-executed SaaS product launch to an existing customer base?
I'm a product developer, startup veteran, and advisor to SaaS companies. Hopefully you've been already developing this new product with input from your existing customers, letting them beta test it and give feedback. (If not, my advice is to STOP immediately and get enough pilot customers involved to be sure that you're delivering something really valuable to them, that works the way they expect it to work, is easy to understand and get started with, etc.. The last thing you want is to do a big splashy launch of a product that is D.O.A. because you built what you assumed the customers wanted instead of they actually demonstrated that they wanted.) OK, so let's assume that you've got customers in the loop. Interview the heck out of them. Really understand how they use the product, why they use the product, what makes it valuable to them, what they can do with it that they couldn't do before, etc. If the product's not done enough for them to be best testing it yet and getting results, at least get some insights into how they see themselves getting results from it. How does it/will it change their lives? As you do this, be on the lookout for things that really resonate. Emotional language, for example. "It's such a relief that I don't have to worry about sending invoices manually anymore." (or whatever pain it is that your software solves) Also look for (and try to elicit) specific result statements: "This new software saves me [or is going to save me] 15 hours a week. Now I can spend that time where I really want to, with my kids ( ... my cat ... my golf buddies ... )" You're doing this for three reasons: 1) This stuff makes for phenomenal testimonials; 2) it helps you come up with great ideas for pre-launch content; and 3) it generates *PURE SOLD GOLD* you'll use in writing the copy for your launch offer. OK, launch mechanics. There are people who teach huge long expensive courses on this stuff. I'll give you the Cliff's notes. While I haven't personally run a major product launch, I have been trained in the strategy and am very familiar with it. - Plan your launch period in advance. You might want to do a pre-launch sequence that lasts 1, 2, even 3 months depending on the magnitude of your product and how much effort you're willing to put into creating content for the launch. - Create some teaser content of interest to your customers who might want to buy this product. Offer to teach them something, or offer to give them a sneak-peak behind-the-scenes of your new product. - Send an enticing offer for this content out to your list. Get people who are interested in this content to sign up for it. This creates your launch email list. - Send your launch list weekly updates: development milestones, sneak-peak screenshots, videos, educational material, interviews with/testimonials from beta users, and so on. - You're not trying to sell here yet (not hard sell at least). Drop some hints that there is going to be a special offer when the product launches, just for special loyal customers like them. - Create at least three videos on topics that are really, really interesting to your prospective customers... not necessarily about your new product itself, but teach them about what they can achieve with it, or what others have achieved with it already. As you publish these videos, send the link out to your launch list. - Also send out an offer to see these videos to your main list, to entice more people to sign up for your launch list. - As you get closer to launch time, keep sending frequent updates to the pre-launch list, and send another email out to your main list to let them know that the product is launching soon, and that if they're interested in the special one-time-only launch pricing, they need to sign up for the "early bird list" (your launch list). - Send out a 24-hour notice that the launch is going to happen soon, and the launch pricing will only be available for a limited time (potentially, to a limited number of customers ... to increase scarcity and urgency). - I recommend that even if you plan to open the product up to all your customers that at launch time you limit it to a smaller number. This makes the inevitable post-launch gremlins less painful to deal with because you have fewer customers, and it motivates people to buy because they fear that they'll lose the opportunity to do so. You can open the product up to more people later... the delay will result in pent-up demand and easier sales. - Start the launch. Tell your early-bird launch list a few hours early, then tell your main list. Direct them to a web page with a video and long-form sales copy of your launch offer. - Send out 2-day, 1-day, 12-hour, etc. notices that the launch is ending soon and reminding people what they're missing out on if they don't act now. If you're offering a limited number of spots, tell people what percentage has already sold out. Remind people that if they're "on the fence" about this, that this is the time to make a decision. - Send out an email letting people know that the launch is over and thanking them for their support and their vote of confidence. Tell the people who didn't buy (or didn't get in) that you'll let them know that the product will be opening up for new registrations some time in the future. (You may get people sending you emails begging to be let in at this point, if your product is desirable and your marketing was executed well.) And, of course, you don't just have to promote your launch content to your existing customer list ... you can post it to social media (and encourage your customers to do so) to attract brand new customers into your world. If you'd like to go into more detail about launch planning for your specific product and market, I'd be happy to jump on a call and talk about ways to make this work for you.BB
-
Freemium v.s. free trial for a marketplace?
It depends on a number of factors but I'd boil it down to two key things to start: 1) What is your real cost to provide a free plan or trial? 2) Who exactly is your customer and what are they used to paying and who and how do they pay today? When you say "online workforce marketplace" it sounds as though you're placing virtual workers. If that's the case, or if you're paying for the supply side of the marketplace, the question is how much can you subsidize demand? Depending on where you're at in the process, I'd also question how much you can learn about the viability of your marketplace by offering a free version, assuming again, that free is actually a real cost to you. I was part of a SaaS project that started charging people for early access based mostly on just a good landing page (we clearly stated they were pre-paying) and were amazed at the response. I've also run a SaaS product that offered free trials and realized that the support costs and hand-holding and selling required to convert from free trial to paid wasn't worth it, this despite the product's significant average ARR. You might be better off providing a "more information" sign-up form (to capture more leads) and let them ask for a free trial while only showing your paid options. I've been amazed at the lead capture potential from a simple "have questions? Click here and we'll contact you" This is all the generalized advice I can offer based on the limited information I have, but happy to dive-in further if you'd like on a call.TW
-
How important is coding knowledge in starting a SAAS business? Should I start by learning code or just get started on the idea? Book suggestions?
I started a large SaaS Company for B2B where perfection in code is as importante as it gets. So here is my advice, DON'T CODE until you know what the Saas Really is. First start understanding what the problem REALLY is. Interview people and actually spend 100% of your time doing Customer Discovery. (This sounds easy but it is a skill you'll have to develop far more important than coding). Once you understand what the problem is, come up with a value proposition. Still no code. Then make a sell. If you can actually find things already existing that you can Hack and put it together then use that. Then make another sell. If you can sell it to at least 50 people if you are B2C, or if you are B2B you should have at least 1 customer. Once you do that then start automating some parts of the solution that you have hacked and so on. But THE most important thing is to be in constant conversations with your customers and hot leads. Remember you are a customer making machine not a coding machine, the first one is where the money is. Hope this helped you, if you want to talk more about customer discovery and customer development, just give me a call.JC
-
What is the point of having multi-year contracts in SAAS if the customer does not pay upfront for the 2nd year?
If you have an enforceable contract, the client is obligated to pay for the services received. As a business owner, I would be very concerned if a SAAS was demanding upfront payment for 2 years.SN
-
For a SaaS, I find that Stripe is not available to Indian companies. What are other Stripe-like payment gateway options for Indian companies?
there is Balanced, Dwolla, Braintree but none of them seem to work in India yet.HJ
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.