Just wondering how to run a closed-beta for an ios app.
1. How many users?
2. What kind of tools? (TestFlight? - recommend me!)
3. Other things to keep in mind
Apple will allow a developer to register 100 UDID devices per 12 month cycle to test via TestFlight or HockeyApp. Having started with TestFlight, I would really encourage you NOT to use it, and go directly to HockeyApp. HockeyApp is a much better product. There is also enterprise distribution which allows you far more UDID's but whether you qualify for enterprise distribution is difficult to say.
As part of your testing, I'd encourage to explicitly ask your testers to only register one device. One of the things we experienced was some testers registering 3 devices but only used one, essentially wasting those UDID's where we could have given to other testers.
Who you invite to be a tester should be selective as well. I think you should have no more than 10 non-user users. These people should be people who have either built successful mobile apps or who are just such huge consumers of similar mobile apps to what you're building, that they can give you great product feedback even though they aren't your user. Specifically, they can help point out non obvious UI problems and better ways to implement particular features.
The rest of your users should be highly qualified as actually wanting what you're building. If they can't articulate why they should be the first to use what you're building, they are likely the wrong tester. The more you can do to make them "beg" to be a tester, the higher the sign that the feedback you're getting from them can be considered "high-signal."
In a limited beta test, you're really looking to understand the biggest UX pain-points. For example, are people not registering and providing you the additional permissions you are requiring? Are they not completing an action that could trigger virality? How far are they getting in their first user session? How much time are they spending per user session?
Obviously, you'll be doing your fair share of bug squashing, but the core of it is around improving the core flows to minimize friction as much as possible.
Lastly, keep in mind that even with highly motivated users, their attention spans and patience for early builds is limited, so make sure that each of your builds really make significant improvements.
Happy to talk through any of this and more about mobile app testing.
We really benefited from allowing ourselves ample time to test the app. Firstly, it helps you perfect the app but we found more importantly, it actually helped with the marketing.
We used TestFlight but HockeyApp is also great. (neither solution is a joy to use, mainly due to limitations Apple places on distribution of test builds). Getting 10-50 users to test is usually sufficient unless you're testing a multi-player game that requires more players to test the in-game dynamics. In those cases, it's often better to go straight to a soft-launch in a smaller English-speaking country (see http://blog.tapstream.com/post/71538606229/the-art-of-soft-launching )
I mentioned how your beta test can actually help your marketing efforts: use the beta to create buzz and aura of exclusivity around your app. Get people with large social followings to test it, enjoy it and talk about it before it's released. Have a LaunchRock or similar lander to capitalize on that buzz and to collect emails for a waiting list to use for launch. This can give you a serious ranking boost on the launch day!
Keep in mind that regardless of the tool you choose you will need to setup each device and register it on iTunes Connect. Note also that there are limits:
* 100 users/devices per Developer Apple ID (Apple seems to me migrating to a 200 devices limit)
* Devices can be removed/replaced just once each membership year
* A time limit of 90 days
There are some alternatives:
* Ad-hoc installation (needs physical access to the device)
* Testflight (https://testflightapp.com/) the one I use and recommend +++
* Pieceable (https://www.pieceable.com/) lets you test over the web browser
* AppHance (http://www.utest.com/apphance) is a TestFlight like service
* HockeyKit (https://github.com/TheRealKerni/HockeyKit) is an opensource stack to host beta apps on the web (still needs the device registration on iTunes Connect)
* HockeyApp (http://hockeyapp.net/features/) an TestFlight like service
Go for fabric.io by Twitter, simple delivery, crash analytics etc... Way better than the iOS Beta program
The other solution is to get an iOS Developer Enterprise program (200$/y), you'll have unlimited devices.
In any cases, I recommend fabric.io for crash analytics, and high level user engagement stats
There is a tool called Daiwa that a number of developers use for beta testing the IOS apps. you have to share your UUID of your device and then they can send you an install link.
You can also get added as a tester in Itunes and pre test you app from there. This takes a little more effort to setup and get it to work.
Another tool I found that will let you install apps without iTunes is FunBox.
Beta testing is an important part of the app launch process. After developers test their app (that’s a rigorous testing process called alpha testing), they push it forward to real users to test it. At this point, they can get big flaws resolved before making the app available for the wider audience. Here are the steps you need to follow:
1. Decide how many testers you need: How many people will you engage during the beta testing period? Keep in mind that not all testers will give you valuable feedback. Some of them will not detect bugs, and some of them will tell you they liked the app even though they faced problems with it. It is just easier to write “everything is okay” than to provide detailed feedback. You cannot have only 10 testers. However, you should not get too many testers, either. If, for example, you get 1,000 testers for an app, collecting feedback will be difficult to process. In general, it is advised to get 100-300 beta testers to make sure your app works well before you launch it. This gives you enough feedback to work with, but you will not get overwhelmed by it.
2. Decide what type of testers you need: Depending on your goals, you must decide what type of person to select as a beta tester based on their background and experience. Developing a beta tester persona will help you choose the right users for this stage of development. There are two types of beta testers: technical (these detect bugs and give you constructive feedback) and marketing (they are usually influencers, who increase the awareness before your app gets launched). In this article, we focused on technical beta testers.
3. Develop a beta tester persona
During the planning part of the app development process, you probably defined your user persona. You developed the entire concept around that final user, and probably had several questions on your mind:
i. What problems do they have?
ii. What solutions do they need?
iii. What solutions can I provide?
iv. What would make this person use an app daily?
v. What features are they looking for?
You will need that user persona when developing the marketing campaign, too. You also need a persona when looking for beta testers. At this point, you have additional requirements:
i. What goals do you have for the beta-testing phase?
ii. Do you want to get feature request? Discover bugs? Develop an audience of early users? Bring awareness to the app?
Based on your needs, answer this question: what skills do you want this person to have? From there, it is easier to build the persona you are looking for.
4. Give testing a timeframe: If you do not plan a timeframe for your test period, it can go on for ages. TacoBot, for example, was announced back in 2016, but it still has not reached its audience. You do not want to extend the beta testing page to an indefinite period. This does not mean that you should rush through the beta test. It must be thorough. You should give the beta testers enough time to try all features of the app and detect all its issues. How long should the beta test last? According to Centercode, the answer to that question will depend on few factors: your goals, resources, the tester limitations, and the number of test phases you opt for.
5. Find your beta testers: So where do you find the people who will test your app and provide valuable feedback? There are few places you can try:
i. Erli Bird
ii. BetaList
iii. Betabound
iv. BetaTesters
Those were few specialized platforms that can connect you with beta testers. You can also search for testers via Twitter, Reddit, and Quora. Use an appropriate hashtag, such as #testmyapp on Twitter, and people will start contacting you. You can share the news on specific subreddits, such as /r/TestMyApp.
6. Deliver the app to your beta testers: You got to the point of actual distribution? That is great! If this is your first time dealing with beta testing, you must be wondering: how will you do that? There are app distributing platforms you can choose from. These are only some of them:
i. AppBlade
ii. Google Play Console
iii. Appaloosa
iv. TestFlight
7. Keep your testers engaged: So, you developed the beta tester persona, you found your beta testers, and you distributed the app with instructions on what you expect. Now what? You must make sure this process will deliver the results you need. You want their input, so you must motivate them to use your app and share feedback. You can do that by providing various incentives. For example, you can give them codes so they can get premium features for free. You will not lose a lot of money, but you will encourage the testers to spend more time with your app.
8. Take their opinions into consideration: When your beta testers inform you about bugs, you will clearly do everything to fix them. But what happens when they give you tips for improvement? You worked hard for some of the features, but it turns out that most beta testers do not like them. You found your community. The least you can do is trust these people. Whatever comments or questions they provide, take them into consideration as you are improving your app before its release.
9. Be selective: Does the above point mean you should make every single change that your beta testers suggest? Absolutely not. You should listen to them and consider them, but you may proceed only with those suggestions that make sense.
10. Keep the connections: Once you go through a successful beta testing stage, you will realize how important it is for the overall success of your app. When you make connections with passionate beta testers, who enjoy trying out new apps, maintain them. Contact them whenever you develop a new app that needs testing.
Besides if you do have any questions give me a call: https://clarity.fm/joy-brotonath
