Agile QA Testing for eCommerce: Case Study
The client is a leader in web services with revenue derived from eCommerce. The company has grown dramatically over its 10-year history, and this growth is based on the company’s ability to provide a functional, attractive customer experience on a site that continuously presents new marketing initiatives and campaigns. In addition to registration and payment processes, the site requires the user to complete a complex questionnaire to access its services. Maintaining the customer’s interest through this screening requires attention to detail and near perfection in code development and verification.
The Problem: Agile QA
Typical software development organizations run a ratio of one quality engineer to three or four development engineers, due to the slow roll of new code… months, even a year per version. An eCommerce organization must roll new releases every one to two weeks. This pares the traditional quality/development personnel ratio to something more like one-to-two.
Highly competent web services development engineers aren’t easy to find, and finding software quality engineers who can keep up with them is even more difficult. Because of this, development engineers end up testing their own code or depending on code reviews and integration tests to catch flaws. For an eCommerce company that has to manage the Agile development process, this is courting disaster. The client had to find a way for QA engineer recruiting to keep pace with its explosive growth in order to maintain the customer satisfaction that fueled that growth.
The Solution: Agile QA Staff Augmentation
QualityLogic consulted with the client’s QA director and put together a team of software quality engineers to augment the in-house QA staff. Initially, a nucleus of three experienced senior QA engineers was shared among four major development projects to become familiar with the client’s data infrastructure and augment the resources where QA needs were critical.
As additional QualityLogic QA engineers came aboard, they took on more permanent positions in the client’s projects. This let the client focus the roles of their in-house QA staff on system attributes, such as marketing campaigns and feature additions, rather than mundane but necessary regression testing and sanity checks. Most importantly, QualityLogic staff automated repetitive and time consuming test jobs, creating quick-turnover sanity and ‘smoke’ tests. This let the client’s developers ramp up work on marketing initiatives and new features without the concern that the release of code to the live system would be bottle-necked at the verification stage.
The Outcome: Agile QA Support That Kept Up With Business Growth
One of QualityLogic’s principle contributions was fielding a qualified, capable agile QA engineering staff on short order. The QualityLogic seed team immediately went on-site at the client’s facility and gathered the operational knowledge necessary to bring staff engineers on line and make them productive within days.
QualityLogic’s on-site project manager integrated additional engineers onto the project so that they could become productive resources immediately. The manager gained insights into the code development and testing processes from which he developed recommendations and plans for improvements.
QualityLogic created automated testing systems to perform repetitive test processes unattended. An eCommerce system’s stability relies on rigorous regression tests of all the system components and their connections each time new code is installed. Automation is the best way to ensure this, but it takes high value engineering talent away from development or testing. QualityLogic studied the standing system and brought engineering resources in to work up the test automation harness code without detracting from either the development or QA processes.
QualityLogic’s engineers were prepared to work within the client’s Agile development process; they are well versed in the Agile operational concepts of quick, short-term turnaround of proposed features into working code without disturbing long-term system stability. The staff in QualityLogic’s lab participated via IP audio links in the stand-up Scrum meetings held at the end of each sprint, and the project manager attended them in person. The on-site manager responded to requirement changes and directed the remote engineering staff’s focus on the client’s quality priorities. This synthesis of on-site management with remote staffing allowed QualityLogic’s team to be extremely functional and cost effective.