What is User Acceptance Testing?

Once, I worked at a large organization with state-wide responsibility for providing consumers with a specific kind of information. I don’t want to give away the nature of the organization by telling you what this information was, but suffice it to say it impacted the majority of the adult population, and several large industries, in my state.

This organization knew it’s customer-facing website was clunky, and decided to upgrade. They spent a huge number of hours gathering input from all business departments about what each needed the system to do. Then, the developers spent a huge amount of time structuring the new website, based on the requirements they were given. Finally, the developers proclaimed testing was done (everything worked as it should) and the new site went live.

The Problems of Not Performing Proper Software Testing

Almost immediately, complaints came rolling in. Users who previously had no difficulty finding the information they needed now couldn’t find anything. Links were either simply gone, or reformatted under ambiguous headers. The whole interface was a series of icons instead of descriptive texts. The business departments were livid…their call rate had doubled! The dev team was scratching their heads….in their opinion, they’d done what the business unit requirements asked them to do.

Instead of saving time by making things easier for the user, the new release was costing time (and money) as users struggled to understand the interface. The reason? User acceptance testing hadn’t been performed.

Why Perform User Acceptance Testing?

It’s a truism that users think differently than developers. As such, they’ll likely do things differently as well. In the above case, the developers understood the sites’ information hierarchy perfectly. After all, they designed it. But they hadn’t thought about using affordances that would clarify what was where for the average user. Jakob Neilsen writes “it’s a disaster to guess at the users’ needs. Since designers are so different from the majority of the target audience, it’s not just irrelevant what you like or what you think is easy to use — it’s often misleading to rely on such personal preferences.”

User Acceptance Testing, or UAT, strives to reconcile this difference through testing software to make sure it meets user expectations, as well as business requirements. Developers build based on their understanding, and interpretation, of the requirements. This, as we see, is often very different from user expectations based on the fact that developers “think differently” and enjoy the intimacy of knowing their product inside and out. Another facet to consider is that what the business thinks it needs (it’s requirements) aren’t always aligned with user expectations. Business requirements may also evolve during the development process, and unless these changes have been clearly communicated to the development team, final platform configuration may fail to meet business requirements and expectations.

What is User Acceptance Testing?

The goal of User Acceptance Testing is simple — to assess if the system can support day-to-day business and user scenarios and ensure the system is sufficient and correct for business usage. The obvious place for this to occur is near the end of the dev cycle, once a functional UI is in place. It’s an unfortunate truth that positioning user acceptance testing as last before deployment often means it gets short shrift, or doesn’t get done at all. (And look how well that worked in my example!)

The Components of User Acceptance Testing

While the testing itself may take place late in the dev cycle, a strong user acceptance testing plan should be created prior to reaching that point. There’s more to UAT than meets the eye. A wise program administrator creates efficiency and targeted testing through developing their UAT plan in advance.

1. Define Acceptance Criteria

The best criteria to start with will be the business requirements. These are the things the software must do, without fail. Business documentation that will help build acceptance criteria use cases, process flow diagrams, system requirements and specifications, and business requirements documentation. Using these as groundwork for defining UAT acceptance criteria allows the testing to confirm the end-to-end business needs are met. It’s within the framework of the business needs that the user expectations will be tested.

2. Select the Testers

This can vary depending with each situation, but at a minimum you should have representatives from both the business and user perspective. It’s not a bad idea to include the development and software QA perspective, as well. Don’t assume the tester will simply be a “non-technical end-user”. If each testers skill level is understood your materials and test sessions can be made more effective. Especially when creating any kind of written documentation for the test, the audience has direct impact on the materials to be prepared. In considering your UAT testers, ask yourself “Who is the audience, and what do they need?”

3. Prepare Test Scenarios

The materials created for user acceptance testing vary as much as the testers do. You might target a feature review with checklists, or provide a series of steps the user should follow to check for defects. As you create these materials with the audience in mind, it helps to remember that during UAT you want to challenge the application, not simply confirm function. Try creating test scenarios that use complex progressions or data input to both challenge the capabilities of the app and help spot where user flows begin to deteriorate.

4. Conduct the UAT Session

It’s a good idea to begin user acceptance testing with a kick-off meeting. Here, it’s wise to start with a review of the product, and the test expectations and deadlines. You’ll also have the opportunity to review acceptance criteria and sign-off procedures. Another good idea is to provide testers with contact information for support or help resources. This might be a contact in the software QA department or on the UAT team. You’ll also want to schedule an individual follow-up meeting, as this can be a great way to find out about issues early.

5. Report Test Results

Once the tests are run, it’s time to compile and report the results. This report will act as an exit ticket from the testing process if test criteria and user expectations are met. Once this has been signed off on by the client, the product can begin production or be released.

Examples of User Acceptance Test Scenarios

User acceptance testing is a crucial step in ensuring a product meets both business and customer expectations. I thought it may be helpful to share some example scenarios that might be part of a UAT process:

Scenario 1: Small Order for New Customers

A first-time shopper purchases products totaling less than $20 in a single transaction. This tests whether the system correctly applies any policies like “no free shipping” for smaller orders.

Scenario 2: Small Order for Returning Customers

A regular customer places a similar small order under $20. This situation assesses if returning clients receive consistent order handling without unexpected privileges.

Scenario 3: Medium Order for New Customers

A new user places an order exceeding $20 but under $50. This scenario ensures that the platform handles such orders correctly, particularly regarding any applicable shipping fees.

Scenario 4: Medium Order for Existing Customers

A frequent customer orders more than $20 but less than $50 worth of products. This tests for consistency in order processing, ensuring that whether a customer is new or returning does not alter the handling of medium-sized orders.

These scenarios assist in validating that the system supports various customer categories and order sizes effectively, offering a seamless user experience across different shopping conditions.

Frequency of UAT in Major Projects

User acceptance testing is generally conducted once during the lifecycle of a major project. This singular execution requires in-depth analysis at the test strategy phase.

Key Considerations:

• Test Design Effort: It is vital to assess how much effort should go into designing the tests. If there’s potential for these tests to be repurposed, for instance, in regression testing, or if the project entails significant risk, investing in thorough test design is advisable.
• Efficiency: In situations where detailed tests aren’t strictly necessary, a simple User Acceptance Testing checklist aligned with the project’s acceptance criteria may suffice.

Ultimately, deciding on how extensively to design UAT hinges on these strategic evaluations.

Purpose of a UAT Test Plan Template

A UAT plan template can work as a roadmap for ensuring that software meets the needs and expectations of its users – and matches what you had in mind when developing your software. The template can provide a structured framework to define and manage the testing process, focusing on user-centric goals rather than system specifications.

Some of the main reasons we’ll include a UAT test plan template include:

1. Alignment with Business Objectives:

The primary goal of the UAT plan is to verify whether the software aligns with the real-world workflows and business processes. It ensures that the final product supports the necessary business functions, such as customer management and reporting.

2. Structured Test Planning:

A UAT template offers a consistent guide to planning, organizing, and executing tests. It helps teams identify what specific aspects should be tested, how the testing will be conducted, and what success looks like—all in relation to user experience.

3. Clarifying Roles and Responsibilities:

The template outlines who will conduct the tests and defines the different roles involved in the UAT process, thereby ensuring accountability and clarity within the team.

4. Risk Management:

By highlighting potential risks and outlining mitigation strategies, the plan helps prevent any hindrances that could derail the testing process or affect software deployment.

5. Standardization Across Projects:

Using a UAT test plan template fosters consistency across different projects within an organization. This standardization helps streamline processes and reduce variation, ultimately leading to more predictable results.

6. Comprehensive Test Coverage:

The template serves to ensure thorough coverage of testing areas. This includes checking critical business operations, like generating custom reports, and making sure that each function works as intended from a user’s perspective.

7. Facilitating Communication:

By detailing the entire testing process, including schedules and environments, the template acts as a communication tool that helps synchronize all stakeholders involved in the testing phase.

Ultimately, the template is not just a document but a strategic tool that helps make sure the software products fulfill business needs effectively. Think of it as bridging the gap between technical features and business objectives, making it indispensable for successful software validation.

User Acceptance Testing Provides Valuable Insights

While UAT is designed to confirm performance of business requirements in the user environment, it has many benefits. Insight from User Acceptance Testing can inform future iterations of the project, and translates into developing an overall understanding of user expectations that can be applied across many platforms.

Interested in Learning More About UAT?