Loading...
Answers
MenuBuilding a database of A/B testing examples
Answers
Yes definitely. As you know these elements are super important pertaining to to this information. There are plethora of small businesses that cannot the technical expertise needed in some cases to increase in engagements on websites, which for most websites are the best promotion tools. This would be a great asset for small to medium size business who want to increase the proficiency of engagements for one. Some other ideas I could think of too. I would explore those engagements with those users to see if there is an noticeable difference before and after using the information provided from your db.
Great idea - I'd be interested, especially in pricing A/B test results. Are you up and running already?
There are already a couple of products in market like this, goodui.org being the main one.
However, personally, I don't believe that this kind of meta-analysis of experiments from different sites has too much value, for a couple of reasons. I'm only really outlining this stuff for you because it might be food for thought if you want to create products for the experimentation community:
Whilst the theory of meta-analysis is technically sound, it is also fraught with issues relating to the complexity of web experiences:
Tests must be subjectively categorized and then analysed as 'patterns' when there is zero evidence that that thing was the cause of the result. For example, you might think a test won because you removed the voucher code box, but it equally could have won because of the change in page layout.
No two tests are in any way the same in any controlled way. Meta analysis mainly comes from sciences where experiments can be repeated under (relatively) identical controlled conditions. This is impossible and A/B testing cannot really be compared to that kind of controlled experimentation. By comparing two different experiments you instantly remove any control you had around audiences etc because they will not be the same.
The greatest benefit to be had from experimentation is by integrating it with business and brand strategy. A good business strategy is aimed at carving out a unique and differentiated direction of travel for that brand. The opportunity with experimentation is to iteratively develop and innovate the business through learning and adaptation. This means that experimentation should aim at being meaningful within the context of the brand strategy. Meta-analysis of experiments from different sites is, in many ways, the opposite of this because it seeks to make the ‘lowest common denominator’ changes which would be beneficial to any business anywhere. Conversion uplift and revenue might be achieved but what has been furthered in terms of strategy and differentiation? This is not to say that it isn’t valuable if you simply want to do ‘CRO’ but my argument is that nobody should merely do CRO.
Related Questions
-
For a SaaS, I find that Stripe is not available to Indian companies. What are other Stripe-like payment gateway options for Indian companies?
there is Balanced, Dwolla, Braintree but none of them seem to work in India yet.HJ
-
How should we plan a well-executed SaaS product launch to an existing customer base?
I'm a product developer, startup veteran, and advisor to SaaS companies. Hopefully you've been already developing this new product with input from your existing customers, letting them beta test it and give feedback. (If not, my advice is to STOP immediately and get enough pilot customers involved to be sure that you're delivering something really valuable to them, that works the way they expect it to work, is easy to understand and get started with, etc.. The last thing you want is to do a big splashy launch of a product that is D.O.A. because you built what you assumed the customers wanted instead of they actually demonstrated that they wanted.) OK, so let's assume that you've got customers in the loop. Interview the heck out of them. Really understand how they use the product, why they use the product, what makes it valuable to them, what they can do with it that they couldn't do before, etc. If the product's not done enough for them to be best testing it yet and getting results, at least get some insights into how they see themselves getting results from it. How does it/will it change their lives? As you do this, be on the lookout for things that really resonate. Emotional language, for example. "It's such a relief that I don't have to worry about sending invoices manually anymore." (or whatever pain it is that your software solves) Also look for (and try to elicit) specific result statements: "This new software saves me [or is going to save me] 15 hours a week. Now I can spend that time where I really want to, with my kids ( ... my cat ... my golf buddies ... )" You're doing this for three reasons: 1) This stuff makes for phenomenal testimonials; 2) it helps you come up with great ideas for pre-launch content; and 3) it generates *PURE SOLD GOLD* you'll use in writing the copy for your launch offer. OK, launch mechanics. There are people who teach huge long expensive courses on this stuff. I'll give you the Cliff's notes. While I haven't personally run a major product launch, I have been trained in the strategy and am very familiar with it. - Plan your launch period in advance. You might want to do a pre-launch sequence that lasts 1, 2, even 3 months depending on the magnitude of your product and how much effort you're willing to put into creating content for the launch. - Create some teaser content of interest to your customers who might want to buy this product. Offer to teach them something, or offer to give them a sneak-peak behind-the-scenes of your new product. - Send an enticing offer for this content out to your list. Get people who are interested in this content to sign up for it. This creates your launch email list. - Send your launch list weekly updates: development milestones, sneak-peak screenshots, videos, educational material, interviews with/testimonials from beta users, and so on. - You're not trying to sell here yet (not hard sell at least). Drop some hints that there is going to be a special offer when the product launches, just for special loyal customers like them. - Create at least three videos on topics that are really, really interesting to your prospective customers... not necessarily about your new product itself, but teach them about what they can achieve with it, or what others have achieved with it already. As you publish these videos, send the link out to your launch list. - Also send out an offer to see these videos to your main list, to entice more people to sign up for your launch list. - As you get closer to launch time, keep sending frequent updates to the pre-launch list, and send another email out to your main list to let them know that the product is launching soon, and that if they're interested in the special one-time-only launch pricing, they need to sign up for the "early bird list" (your launch list). - Send out a 24-hour notice that the launch is going to happen soon, and the launch pricing will only be available for a limited time (potentially, to a limited number of customers ... to increase scarcity and urgency). - I recommend that even if you plan to open the product up to all your customers that at launch time you limit it to a smaller number. This makes the inevitable post-launch gremlins less painful to deal with because you have fewer customers, and it motivates people to buy because they fear that they'll lose the opportunity to do so. You can open the product up to more people later... the delay will result in pent-up demand and easier sales. - Start the launch. Tell your early-bird launch list a few hours early, then tell your main list. Direct them to a web page with a video and long-form sales copy of your launch offer. - Send out 2-day, 1-day, 12-hour, etc. notices that the launch is ending soon and reminding people what they're missing out on if they don't act now. If you're offering a limited number of spots, tell people what percentage has already sold out. Remind people that if they're "on the fence" about this, that this is the time to make a decision. - Send out an email letting people know that the launch is over and thanking them for their support and their vote of confidence. Tell the people who didn't buy (or didn't get in) that you'll let them know that the product will be opening up for new registrations some time in the future. (You may get people sending you emails begging to be let in at this point, if your product is desirable and your marketing was executed well.) And, of course, you don't just have to promote your launch content to your existing customer list ... you can post it to social media (and encourage your customers to do so) to attract brand new customers into your world. If you'd like to go into more detail about launch planning for your specific product and market, I'd be happy to jump on a call and talk about ways to make this work for you.BB
-
How important is coding knowledge in starting a SAAS business? Should I start by learning code or just get started on the idea? Book suggestions?
I started a large SaaS Company for B2B where perfection in code is as importante as it gets. So here is my advice, DON'T CODE until you know what the Saas Really is. First start understanding what the problem REALLY is. Interview people and actually spend 100% of your time doing Customer Discovery. (This sounds easy but it is a skill you'll have to develop far more important than coding). Once you understand what the problem is, come up with a value proposition. Still no code. Then make a sell. If you can actually find things already existing that you can Hack and put it together then use that. Then make another sell. If you can sell it to at least 50 people if you are B2C, or if you are B2B you should have at least 1 customer. Once you do that then start automating some parts of the solution that you have hacked and so on. But THE most important thing is to be in constant conversations with your customers and hot leads. Remember you are a customer making machine not a coding machine, the first one is where the money is. Hope this helped you, if you want to talk more about customer discovery and customer development, just give me a call.JC
-
What's the best way to sell a SaaS prior to launching?
I was involved with a SaaS product that launched a landing page and made clear that the product was still in development, but that we would give earliest access to people who pre-paid for the product. We also allowed people to choose what they paid, and promised them that payment would stay in-effect for several months. We generated revenue the first day of posting the landing-page publicly and increased revenue month-over-month. However, we discontinued the product as it was simply not big enough of a market for us to justify continued time and energy. But I would encourage you to pursue a similar model in that it's a great way to test and validate the pain others experience for the problem and a great way to ensure you're building the product to satisfy real customers. Happy to talk this through in more detail in a callTW
-
How can I manage my developers' performance if I don't understand IT?
Whenever you assign them a task, break down the task into small chunks. Make the chunks as small as you can (within reason, and to the extent that your knowledge allows), and tell your devs that if any chunks seem large, that they should further break those chunks down into bite size pieces. For instance, for the overall task of making a new webpage, _you_ might break it down as follows: 1) Set up a database 2) Make a form that takes user email, name, and phone number and adds them to database 3) Have our site send an email to everyone above the age of 50 each week When your devs take a look at it, _they_ might further break down the third step into: A) Set up an email service B) Connect it to the client database C) Figure out how to query the database for certain users D) Have it send emails to users over 50 You can keep using Asana, or you could use something like Trello which might make more sense for a small company, and might be easier to understand and track by yourself. In Trello you'd set up 4 columns titled, "To Do", "Doing", "Ready for Review", "Approved" (or combine the last two into "Done") You might want to tell them to only have tasks in the "Doing" column if they/re actually sitting at their desk working on it. For instance: not to leave a task in "Doing" overnight after work. That way you can actually see what they're working on and how long it takes, but that might be overly micro-manager-y At the end of each day / week when you review the tasks completed, look for ones that took a longer time than average (since, on average, all the tasks should be broken down into sub-tasks of approximately the same difficulty). Ask them about those tasks and why they took longer to do. It may be because they neglected to further break it down into chunks as you had asked (in which case you ask them to do that next time), or it may be that some unexpected snag came up, or it may be a hard task that can't be further broken down. In any case, listen to their explanation and you should be able to tell if it sounds reasonable, and if it sounds fishy, google the problem they say they encountered. You'll be able to get a better feel of their work ethic and honesty by how they answer the question, without worrying as much about what their actual words are. Make sure that when you ask for more details about why a task took longer, you don't do it in a probing way. Make sure they understand that you're doing it for your own learning and to help predict and properly plan future timelines.LV
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.