Automated Testing vs. Manual Testing – Getting the Right Mix
Automation is ideal for continuously running sanity checks to verify that the development crew didn’t unintentionally break something. The problem with automation is one of resource availability. Placing heavy emphasis on test automation means allocating highly skilled resources to it and, even if they are initially available, the odds are that they will be drained away by the press of code development.
Maintaining Automated Test Scripts
That leaves the question of test script maintenance up in the air. An alternative approach is to maintain a script library that operates and verifies all the features at a fairly shallow level. The tests exercise the code without stressing it in any way. This ensures nothing was severely damaged by a change such that a feature stops operating or does something unintended.
The requirement for new features to be tested in detail can be filled by manual testing where a test tech explores the new feature the same way a user would. This places a strong focus on carefully testing the newest code, which is most likely to harbor new bugs.
The make-or-break for test automation coverage is resource allocation. How many skilled developers is your organization willing to task with automated script creation and maintenance? Writing test scripts requires the same skill set as code generation in an individual who has a proper understanding of the product’s intentions and a tester’s approach to verifying implementation. Assigning the necessary staff to maintain full coverage of a system can quickly approach the same cost level as creating it in the first place.
Failure to maintain the script libraries and update them alongside the release of new features risks their becoming obsolete and reporting false failures that eventually discredit the entire automation process.
Resource Allocation for Automated Testing
Most companies work to strike a balance between automation for repetitive, predictable tests and manual, exploratory testing for newly coded features. This allows a sanity check to catch collateral damage and detailed verification to get the new code just right.
Successful test automation hinges on allocating sufficient resources to script maintenance. A winning strategy is to backstop your automation effort with manual testing where it makes more economic and skill application sense to do so. An external test services company can provide flexible test techs and engineers who can come up to speed quickly to get new code and features verified while the in-house team is putting test scripts in place for sanity checks and regression testing.
Check out the white paper for a full discussion of manual versus automated testing and where each excels.