Loading...
Answers
MenuWhat's your opinion on using something like usertesting.com vs. real time usability testing (online and offline)?
This question has no further details.
Answers
UserTesting can be instructive in terms of understanding whether people understand your copy, CTAs, and intended flows but generally, I've found the quality of their panels to be pretty low. You're almost always getting people who are not your actual users, so the feedback can only be generally applied as above. I find whatever web analytics package or packages you're using are generally able to provide much better insights.
I also really do believe in *real* user panels. Buying pizza or offering small financial incentives to real users to click through new flows where they are talking out loud or answering specific questions is going to give far more actionable insights than anything else.
What I like to do is take my best guesses as to what's not working or what I'm looking to improve and then discover/validate via real in-person customer panels.
Happy to talk through this in more detail with you in a call.
I do both. We setup usertesting.com every Friday to run through a preset script for both our web & mobile apps and then review the videos as a team and add a couple items to our backlog. The users are definitely not our ideal customers, but their perspective can be quite enlightening and it's just a great thing to have automated and run for $80/week.
As for user tests, we actually use Clarity to find target customers (not hard for us) and then schedule calls and use join.me to either show them clickeable prototypes (made with Keynote -> saved to PDF) or watch them interact with our app and ask questions. We typically only request calls with people who are $1/min, and most calls last 20 minutes - so it's pretty cheap testing :)
Hope that helps.
P.S. I still do manual / in person testing with anyone who's a target customer and has a laptop or iphone - but that's just my personality :)
Usabilitytesting.com recruits users from a pre-selected panel of users who have given consent to be a part of usability tests in general. When you test with these set of users, what you hope to catch are low-lying usability fruits like fonts, colors, labels, widgets, etc.
Key issues like navigation and intended behaviour nuances will not get addressed as the users are not 'intrinsically motivated' to complete the tasks. The users are neither motivated to apply for a Certificate of Deposit online nor interested in buying a pair of hiking shoes online for the upcoming K2 climb.
Usabilitytesing.com is great for things which most people regularly do online - buy pizza, check account balance, etc. However, it is not good to conduct in-lab usability testing when specialized people with intrinsic motivation for completing a task are required.
There is benefits and drawbacks to every usability testing tool. People like usertesting.com because they can "set it and forget it" meaning they recruit all of the participants for you and offer automated unmoderated tests, then send you the results. It's an easy way to get a lot of feedback quickly.
While this may seem like a great way to save time and money, how does it really get you closer to achieving your goals of better understanding your users? Do you think that a test conducted in this manner is really attaining high quality results and valid data?
Empathy cannot be outsourced. It requires time and effort to get to know people at an intimate level. If you have stakeholders demanding usability testing with 100 participants and you're considering using a tool like this to save time, first consider educating your stakeholders on why it is unnecessary to conduct usability testing with that many participants. Five in-person test sessions will cost a heck of a lot less, take even less time, and result in very high quality and valuable feedback you can implement immediately. Plus it will help you and your team better internalize the flaws of your design and the needs of your users.
If you have more questions about these tools, usability testing, or other research methods, please feel free to call me on Clarity.
Related Questions
-
How should the dynamic between a ux designer and a developer who are working together look like?
It depends a lot of in the skill sets and experience of both people but in most cases the ux designer should be controlling the developer pretty heavily in order to make sure his ideas come through properly. The UX designer may just need to work on his approach so people don't feel bossed around and more like they are working together. In an ideal world, there would be a project manager who makes sure everything is communicated well and keeps the dynamic feeling great.JM
-
What is a good way to find a niche group of people, who already converse on a regular basis, to test my app?
Look on Google Communities for technology focused BETA testing groups... Look on Linkedin Groups for business focused potential testers... Look on Facebook Pages for people who like apps/companies like what you are trying to bring to market... That should get you started.DW
-
How do I run a successful closed beta for my web application that is almost done with development?
Create an ideal customer profile. Create some questions that will allow to you survey a potential tester to determine if they fit your profile. Design simple landing page with very clear value proposition that speaks to your ideal customer. Ask for a minimum of information up-front (email), but ask for more info after they commit by submitting the first piece of info. (KISSmetrics does a good job of this on their current website trial signup). Use the their answers to these profiling questions to put the applicants into buckets. Let in the most ideal bucket first, or split them into groups if they're big enough. Try and measure engagement the best you can. Measure qualitative and quantitative data. Schedule calls with your beta testers to find out more, especially with the ones who's user behavior seemed to indicate that they didn't get value from your product. Find out why. Make sure they are indeed your ideal customer. Pick up the phone and get to know your customers inside and out. Meet them in person if possible. Incorporate their feedback quickly and get more feedback. Rinse repeat.DH
-
If you launch a product to a few different countries, does it make sense to do usability testing in all those different countries?
Perhaps. You have to consider how your product is used and how the users will sensorially interact with it. If you have a website, for example, you could probably assume similar US and UK usability -- but now consider layout for Semitic languages which are read right-to-left -- or consider ideographic languages where the ideographs could interact with other graphical elements in a way wholly irrelevant with the languages you're familiar with. If you have a physical product, the question is conceptually the same, but the actions are different. You have to consider how users in a given culture use and interact with products of that sort. Different cultural memes, ways of learning and expectations can effect usability. There are some simple ways to do these things if you have a small budget. You could easily go to craigslist and recruit 10-15 natives of a given country or region. Give them the product or have them use the website (for 60-90 minutes), video it (with their permission of course and for reasonable compensation), and you'll get a potentially useful indicator of usability. If you're P&G or Microsoft, you've got the budget to test against statistically valid sample sizes pretty much everywhere in the world. You don't have that, so you need to use some heuristics. Good luck to you. Should you have any questions, I'd be happy to discuss.SM
-
Is having "HOME" button in navigation menu necessary if I have a clickable logo? What makes most sense from UX POV?
We have been collecting usage data on the home button from about 750 websites we manage across North America in an effort to try to determine if it is necessary or not. While each website is different, and much of the data is statistically insignificant, we have started to operate with a few assumptions. 1.) Most users, particularly younger users, recognize the logo as a way to get to the home page. 2.) Websites without home buttons seem to get a comparable amount of traffic to the home page as those that do not. We don't see a significant difference between having a home button or not. 3.) Websites without a home button often will see an increase in direct traffic from returning users during a session indicating that users who do not know the logo is a route to the home page will instead clear the address bar back to the root domain to get back home. Based on our research, we have decided to omit the home button in most instances. Although when it is present, it is often used, most users seem to understand how to get to the home page regardless of the inclusion of a home button. With the complexity of modern websites, we are usually pressed for space in the header and can better use the real estate that would be dedicated to the home button for other UI elements. That said, if the audience for our website skews older, we will still include the home button. Our research has indicated older users are less familiar with the concept of the logo being a home button.RS
the startups.com platform
Copyright © 2025 Startups.com. All rights reserved.