Are you running usability testing correctly?
Topics covered in this post:
When I learnt the importance of usability testing
What usability testing is
When is usability testing important?
How to do usability testing
Tips when running testing sessions
When I learnt the importance of usability testing
During my first job as a UX designer I spent most of my time creating UI designs and usability testing them with users. This was pretty much my entire design process (let’s ignore the fact that I was missing out a huge part of the design process - discovery, definition etc 😆) However running these tests were extremely valuable!
The company I worked for had never invited users in to observe how they use the product, so we had no idea if people understood what we had designed. What we saw and found was fascinating..
One example when we ran usability testing on the screen which had the biggest impact on the companies revenue:
We had a product screen where we listed all our products users could apply for and they were ordered in a way where the top product was something the user was guaranteed to be accepted for (these were credit products, so users had to apply for them with a risk of not getting them).
I usability tested this screen and every single user said they didn’t understand the icon or the label we used to highlight that product so they completely missed that they were guaranteed to be accepted for it!😱 This was a massive and shocking learning for us and the first thing we did after the testing was redesign that small element and push it live in the next few weeks (especially as that was how we made our money as a business!).. You can imagine the increase it had on our revenue figures! And with such a small change which we were blind to.
So now I make sure I do usability testing with every new thing I design especially if it is. revenue generating screen.
Analysing the results from running usability testing
So what exactly is usability testing?
Usability testing is a method to evaluate a product or app. Let’s start by explaining what the word ‘usability’ actually means. ‘Usability’ means the ‘degree to which something is able or fit to be used’. So in other words, if something is usable then it means it is easy to use.
The testing allows the researcher to evaluate whether users are able to understand the app and therefore able to use it to complete relevant tasks and actions. Usability testing helps identify if there are specific things the user isn’t able to do. Designers are then able to take the learnings from the usability testing and make improvements on the elements users struggled with.
When is usability testing important?
Usability testing is very important in certain scenarios.. I am a strong believer that you shouldn’t do usability testing on every single thing you design. For example if I am designing an e-commerce app, and the app uses patterns and styles which are commonly used across other e-commerce products then there’s a big chance I don’t need to test it. This is because using common patterns reduces the risk of a user not understanding how to use your product because they have already learnt how to use it from other products on the market. E-commerce experiences typically have the same list of products to scroll through, a more detailed view when you click on the product, a way to add to your basket, a way to save, a way to see more information etc. (See the screens below)
However if I am designing something completely new. For example the Duolingo screens below:
As this is quite an innovative way to progress through an app, it makes sense to do usability testing to ensure users know how to begin, how to complete an action, how to pause progress and resume etc.
How you can do usability testing in the most simplified way?
Let’s imagine you need to do usability testing for a food delivery app such as Deliveroo. Here are the steps you would take:
Identify what you want to learn from the usability testing e.g does a user understand how to search for specific cuisines, does a user know how to edit the toppings on certain meals, does a user know how to add their payment details (these tasks will be based on what you are trying to achieve with your specific project)
Once you have identified the areas you want to explore you need to write your tasks for your testing. For example if you want to test if users know how to add their payment details - your first task will be “Can you show me how you would add your card details to your deliveroo account.. Can you show me how you would pay by apple pay.. Can you show me how you would update your card details if your card had expired.. etc”
Recruit participants. You need to ensure you recruit participants who are representative of your target market. An extreme example may be that you recruit participants who aren’t very mobile savvy whereas the target market for a Deliveroo type of app would be users who are comfortable with using apps.
Organise the team. If you are running these sessions at work, you will need to organise who will be taking notes as it will be difficult for you to run the sessions and take notes at the same time. You may decide you may just record the sessions and take notes afterwards. You also need to organise if you will have team members observing the sessions of helping you to facilitate if you have a lot of sessions to run
Conduct the tests. It is important to do a dry run of the session with a team member so you can test all the equipment, questions and set up. You don’t want to have technical difficulties on the day. I have always come across problems with tech when I have done my dry run so I really recommend making sure you make time for this!
Analyse your results. There are multiple ways you can analyse the results - some ways can you take you a few hours and some can take you a few days. Decide which is the most relevant for your project. A common tool to use here is affinity mapping.
Make changes to your designs. Once you have analysed the results you need to prioritise the changes to decide which ones to make in order to improve your product. Sometimes you may have a lot of improvements and it isn’t possible to implement them all. Other times you may only have a few.
Tips when you are running the sessions
If the user asks you what something means because they don’t understand - don’t tell them. Instead put the question back on them and ask them what they think it could mean. Because you don’t want to invalidate the results by telling them.
Try not to use positive reinforcing language when users do something right as you don’t want to bias the results and encourage them to do things they may not.
Try not to fill the empty silences - this is a difficult one because if feels very unnatural and pretty awkward! However the users will be thinking through things and may give you something insightful at the end of their thought process so give them the time.
A great question to ask to get the user talking is “Talk me through what you see”. This highlights what first grabs their attention without you biasing them by guiding them to a specific feature.
Before starting the session remind the users to ‘talk out aloud’ so you can understand their thoughts. Feel free to say this to users throughout if they go silent and start interacting with the screens.
Before starting the session, remind the users that they should feel comfortable to be as open as possible and that there are no wrong answers! Explain to them how their honest feedback is what will help you to improve the experience so you want to know if they don’t like or understand anything.
Conclusion
A lot or organisations try to avoid doing usability testing because it is ‘time consuming’ as they are unable to understand the benefits of conducting these tests. Remember when communicating the benefits of running these sessions to include how improving a potential problem before development is much less time consuming than making changes once the feature is live. So in the long run, running these sessions are much more cost effective then letting the mistakes go live.
