What is User Acceptance Testing?
Once, I worked at a large organization with state-wide responsibility for providing consumers with a specific kind of information. I don’t want to give away the nature of the organization by telling you what this information was, but suffice it to say it impacted the majority of the adult population, and several large industries, in my state.
This organization knew it’s customer-facing website was clunky, and decided to upgrade. They spent a huge number of hours gathering input from all business departments about what each needed the system to do. Then, the developers spent a huge amount of time structuring the new website, based on the requirements they were given. Finally, the developers proclaimed testing was done (everything worked as it should) and the new site went live.
The Problems of Not Performing Proper Software Testing
Almost immediately, complaints came rolling in. Users who previously had no difficulty finding the information they needed now couldn’t find anything. Links were either simply gone, or reformatted under ambiguous headers. The whole interface was a series of icons instead of descriptive texts. The business departments were livid…their call rate had doubled! The dev team was scratching their heads….in their opinion, they’d done what the business unit requirements asked them to do.
Instead of saving time by making things easier for the user, the new release was costing time (and money) as users struggled to understand the interface. The reason? User acceptance testing hadn’t been performed.
Why Perform User Acceptance Testing?
It’s a truism that users think differently than developers. As such, they’ll likely do things differently as well. In the above case, the developers understood the sites’ information hierarchy perfectly. After all, they designed it. But they hadn’t thought about using affordances that would clarify what was where for the average user. Jakob Neilsen writes “it’s a disaster to guess at the users’ needs. Since designers are so different from the majority of the target audience, it’s not just irrelevant what you like or what you think is easy to use — it’s often misleading to rely on such personal preferences.”
User Acceptance Testing, or UAT, strives to reconcile this difference through testing software to make sure it meets user expectations, as well as business requirements. Developers build based on their understanding, and interpretation, of the requirements. This, as we see, is often very different from user expectations based on the fact that developers “think differently” and enjoy the intimacy of knowing their product inside and out. Another facet to consider is that what the business thinks it needs (it’s requirements) aren’t always aligned with user expectations. Business requirements may also evolve during the development process, and unless these changes have been clearly communicated to the development team, final platform configuration may fail to meet business requirements and expectations.
What is User Acceptance Testing?
The goal of User Acceptance Testing is simple — to assess if the system can support day-to-day business and user scenarios and ensure the system is sufficient and correct for business usage. The obvious place for this to occur is near the end of the dev cycle, once a functional UI is in place. It’s an unfortunate truth that positioning user acceptance testing as last before deployment often means it gets short shrift, or doesn’t get done at all. (And look how well that worked in my example!)
The Components of User Acceptance Testing
While the testing itself may take place late in the dev cycle, a strong user acceptance testing plan should be created prior to reaching that point. There’s more to UAT than meets the eye. A wise program administrator creates efficiency and targeted testing through developing their UAT plan in advance.
1. Define Acceptance Criteria
The best criteria to start with will be the business requirements. These are the things the software must do, without fail. Business documentation that will help build acceptance criteria use cases, process flow diagrams, system requirements and specifications, and business requirements documentation. Using these as groundwork for defining UAT acceptance criteria allows the testing to confirm the end-to-end business needs are met. It’s within the framework of the business needs that the user expectations will be tested.
2. Select the Testers
This can vary depending with each situation, but at a minimum you should have representatives from both the business and user perspective. It’s not a bad idea to include the development and software QA perspective, as well. Don’t assume the tester will simply be a “non-technical end-user”. If each testers skill level is understood your materials and test sessions can be made more effective. Especially when creating any kind of written documentation for the test, the audience has direct impact on the materials to be prepared. In considering your UAT testers, ask yourself “Who is the audience, and what do they need?”
3. Prepare Test Scenarios
The materials created for user acceptance testing vary as much as the testers do. You might target a feature review with checklists, or provide a series of steps the user should follow to check for defects. As you create these materials with the audience in mind, it helps to remember that during UAT you want to challenge the application, not simply confirm function. Try creating test scenarios that use complex progressions or data input to both challenge the capabilities of the app and help spot where user flows begin to deteriorate.
4. Conduct the UAT Session
It’s a good idea to begin user acceptance testing with a kick-off meeting. Here, it’s wise to start with a review of the product, and the test expectations and deadlines. You’ll also have the opportunity to review acceptance criteria and sign-off procedures. Another good idea is to provide testers with contact information for support or help resources. This might be a contact in the software QA department or on the UAT team. You’ll also want to schedule an individual follow-up meeting, as this can be a great way to find out about issues early.
5. Report Test Results
Once the tests are run, it’s time to compile and report the results. This report will act as an exit ticket from the testing process if test criteria and user expectations are met. Once this has been signed off on by the client, the product can begin production or be released.
User Acceptance Testing Provides Valuable Insights
While UAT is designed to confirm performance of business requirements in the user environment, it has many benefits. Insight from User Acceptance Testing can inform future iterations of the project, and translates into developing an overall understanding of user expectations that can be applied across many platforms.