January 20, 2017
User Experience and Usability Testing Explained
You Should Know
- There are two types of testing: User experience testing (or UX testing) and usability testing.
- Whether or not testing is right for your website will come down to whether the value outweighs the cost.
- There are alternatives to testing that will make sure your website is doing its job well.
User experience testing. Usability testing. What are they and when are they valuable? This post will answer these basic questions by exploring these topics:
Let’s start with a common scenario: Your organization is poised for a website redesign. You’ve invested much time and effort understanding who your users are, outlining your objectives, and are beginning to lay out the necessary steps for your project. The process is clear: You will move through an information gathering process, into planning your sites content and features (information architecture), into reviewing design mockups, then the technical build, and so on.
But, one question remains: Do you need to test your website with your users? With so much effort going into your new website, this is a natural question to ask. After all, it needs to serve both you and your users, so when the dust settles, how can you be sure it will?
To answer this question, let’s first understand the two basic types of testing.
Types of testing
There are two categories of testing: User experience testing and usability testing. For the purposes of this article, we will not cover specific methods which apply to each category, but focus on the fundamental purposes of each.
- User experience testing will provide insights into the experience your user had while on the site. User experience testing (often referred to as UX testing) can yield valuable qualitative insights into your website such as how they felt while using your site and what impressions they were left with.
- Usability testing will provide insights into how easily your users accomplish specific tasks on your website. Usability testing can yield valuable quantitative insights into your website such as how many clicks it took to get from point A to point B.
You may choose one type of testing or a mix of both. Often times a combination tailored to the unique objectives of your website will yield the specific insights you wish to gain. Because you want to focus your effort on the types of testing that will create the most value, let’s explore the two types of testing a little deeper.
User experience testing is valuable if you are trying to shape specific impressions. Examples might include creating awareness of your mission or your approach in addressing specific issues. Do users find your new site engaging and informative? Did they find the experience frustrating and unfulfilling? These are important questions because the experience a user has on your site will shape lasting impressions about your organization. A positive user experience can compel your users to take actions like donating, signing up for membership, or simply immersing themselves in the purpose of your mission.
Usability testing is valuable if you want to monitor how (or if) users complete specific tasks. An example might include understanding if users can navigate from point A to B. Can users find your donation page and complete a donation? Is your membership application form easy to find and complete? Are they using the site’s navigation as intended? Usability testing can be valuable for organizations wanting to better understand the pathways users are taking through their site in order to connect with information or complete individual tasks.
As you can see, these are not independent concepts. If your goal is to inspire your users to take an action, you must first compel them to do so (user experience) while simultaneously making it easy for them to complete that action (usability).
Now that we have defined what types of testing there are and when they are appropriate, I’d like to address the elephant in the room….
Will the value of testing justify the investment?
Assuming you have limited time and budget, your decision to do testing will come down to cost. Testing takes time and your decision to test is likely going to boil down to whether you have the resources to support the types of testing that will return the most value.
Time: Testing will take time to thoughtfully plan and execute, so you will need to consider how it will affect your project timeline. Testing is most valuable when the website is complete (versus testing wireframes or flat design mockups). At this point in the project you have poured over informed decisions related to content, design, and features and the site is nearly ready for public consumption. Can you afford to take the time necessary to both test and, more importantly, act on the insights gathered through testing?
Money: There is no magic equation for calculating the financial cost for testing. It will be determined by the types of testing you need, the methods you choose, the scale* of the testing and what you plan to do with the results. Will your budget allow you to change course in order to revisit aspects of your site once it is nearly complete? Because testing is optional, it’s likely that this cost is not included in the original scope of your project and any impact to the current site architecture, design, or build will likely result in additional cost.
*Regardless of method, the more participants the better: A larger sample size will allow you to distil your findings in order to prioritize your course of action post-testing.
So, do you need to test your website?
To put it simply, the value of testing may not outweigh the costs associated with it. Both user experience testing and usability testing are strong toolsets that help us discern how successfully we have executed on specific objectives, but they aren’t always a good fit.
Consider the type of website you are building and its purpose. If there are actions your users must take in order to support your organization’s bottom-line, the value of testing may be justified. Testing is most effective if the the purpose of your website is to sell a product or service. This can also include organizations who depend on donations or membership signups. If your website is focused on informing and connecting you users with information, it’s likely that (if properly planned) the site will do this well without the need for testing and iteration prior to launch.
Here is the best case scenario: You have made informed decisions about your audiences and objectives (more below) throughout the project. The copy has been written, the designs have been vetted, the build is complete, and the the site is ready to accomplish it’s goals. You test to discover that the website fulfills its intended purpose well—that was the goal all along wasn’t it?
Fortunately there are other valuable ways to make sure your website is doing its job.
Alternatives to Testing
Many organizations forgo testing, choosing to adopt practices that can monitor and improve the usability and experience your site delivers—both before and after launch.
User Research: In order to connect with your users, you must first understand them. It can be assumed that you already understand who your audiences, but do you understand their habits and motivations? By conducting user research at the outset of your project, you can better understand your users unique behaviours and needs. This understanding will inform the creation of a website that speaks to these nuances and helps you facilitate your organization’s objectives.
Google Analytics: One of the most powerful (and free!) tools we have to monitor user behavior is Google Analytics. It is bar-none the best tool for understanding the usability because it monitors actual user behaviour in in the context of your live site. This customizable platform will monitor your site’s traffic and deliver valuable insights like:
- Where are users coming from (via search, social, direct, or other sources)
- What pages are most trafficked and for what lengths of time (engagement)
- How well specific types of content and features are performing
- What usability pitfalls and successes exist
Plan your installation of Analytics during the early stages of your project. Out-of-the-box, Google Analytics will deliver incredible data points on your website’s many touch-points, but it can be customized with advanced goal tracking. For a better understanding of what is possible, check out our posts on 5 ways Nonprofits Can Use Google Analytics to Better Measure Engagement and Better Google Analytics.
Iteration: Here’s the good news: You’re website is not set-in-stone! It is a dynamic platform that can evolve and change with your (and your user’s) needs. If you’ve recognized certain aspects of your website are not performing as desired, it’s not too late. Revisit and refine the experience based on information gathered post-launch driven by insights gleaned through your use of Google Analytics or through anecdotal feedback.
Both user experience testing and usability testing are invaluable toolsets that are often misunderstood. My hope is that this article sheds light on the difference between the two and can help you make an informed decision on whether or not they make sense within the lifecycle of your project.
In a perfect world, we would not be constrained by time or budget, but the reality is that the costs often outweigh the value for some organizations. Fortunately, with proper research, planning and execution, most sites will hit the mark reasonably well, and if any aspect of the experience is falling short of expectations, a website can always be improved.
Are you interested in learning more about this? Let us know and we’ll plan additional posts in the future!