Yes, Virginia, Software Fails – a journey of software failures and testing solutions

The power and fatal weakness of software is its ultimate malleability. It can be quickly and easily changed by drastic degrees. This one inescapable aspect would be good cause to abandon software entirely if it weren’t for its extreme power to make our machines do our bidding. In this day and age, software runs nearly everything of any importance in our world.

The elemental issue with software is its ephemeral quality. Physical products have a substance which can be

examined by all our five senses to see that they meet the measure of their requirements and that the requirements themselves make sense. Software has no such constraints.

Software is composed of millions of lines of text that must be written with ultimate precision. The slightest coding mistake can have massive repercussions when the system is used. Back in the 1960s, a satellite launch was turned into a fiery explosion by the absence of a comma in a line of code.

Something so impactful must be carefully designed, but design work consumes time without yielding the appearance of project progress. Numerous ‘software failures’

can be laid at the doorstep of poor or insufficient design. So, given all this, software development should be a rigorous, careful, deliberate, tightly controlled process… right?

Producing Quality Software in an Agile World

The preceding sentence loosely describes what was called the waterfall software development method. Features were described in detail by marketing, turned into designs and code structures by engineering, verified by quality, fixed by engineering, and then released every couple of years. Literally, code releases came at intervals of 12 to 24 months and, even at this pace, defect-free code was a fond desire rather than a reality.

Today, the Agile methodology has won the hearts and minds of modern software development organizations. As commerce moved onto the Internet, web software became the gateway to sales of just about everything. At the very least, a current, eye-catching website is required to announce the existence of the product or service to its prospective customer base.

This puts marketing on a bubble of making continuous changes to site code to show off new products and sales campaigns. This push to beat the competition to the web is what has propelled Agile into becoming the premier development method. Agile is predicated on releasing operational code every one to two weeks. Reducing the scope and quantity of these changes down to no more than four or five per release is supposed to allow for a series of small feature additions/bug fixes to be propelled into the functioning system code very quickly.

Sounds good doesn’t it?

The reality is that the constant push toward what is called Continuous Release — releasing each change as it is made — has resulted in the abrogation of most of the controls on software development. The intrinsic re-usability of code has led to the creation of what are called frameworks which are simply tool boxes of software modules that can be tied together to perform various common software processes. These frameworks are so useful that development organizations have standardized them, and they have become commercially available development tools.

So, we have a process that focuses tightly on a few system aspects to release incremental changes rapidly using pre-fabricated code building blocks as much as possible. What could go wrong?

Sanity Checks for Quality Software

Design

It is not enough for the code to follow the design, the design itself must be carefully considered and explained to the programmers. It is vital for the product managers to attend the Agile planning meetings to make sure that both development and quality understand what the intention of the new feature is and, especially, how it should work for the user.

Schedule

Market windows may wait for no one, but there still has to be time allocated for the entire development process. Non-negotiable deadlines are the arch enemies of good design, thorough documentation, and careful verification. Being first to market with a product or service that doesn’t work is not a win.

Budget

Software development takes longer and costs more than anyone thinks it should. This should be a given in the planning of any software development project. Low cost estimates or quotes for any project activity or tool set should be instantly suspect. If the project budget is so tight that it has no flexibility, perhaps the project itself needs to be reconsidered.

Documentation

Unfortunately, documentation has become the bane of the Agile methodology. It is typically viewed as useless history, the creation of which consumes time and resources better used for development work. It is still a necessity to leave a trail of what was done and why in order to facilitate the next time that code has to be modified. In particular, defect reports have to be detailed and contain exact descriptions of how to replicate symptoms.

Management

Quality used to be the conduit of information flow between product management and code development. Agile has shifted this responsibility to product management. Those responsible for the final product have to stay on top of getting the design right, communicated to engineering, and properly understood by those testing the results. With Agile, the opportunity for corrective guidance is a very narrow window that management must seize and use to fullest effect.

Verification

A perception exists that QA is an expense rather than a revenue-generating investment. That may be the accounting take but sacrificing testing to meet a market window is a prescription for disaster. With Agile, quality has truly become everyone’s responsibility. As such, it must be planned for in each sprint and any test plan, automated test script or sanity check must be updated in order to maintain its relevance. Development and management must become the direct allies of quality working together to see that the release meets the design, performs the function fully, doesn’t damage other parts of the system, and passes use acceptance as a working addition to the system.

Yes, Virginia, Software Fails But All is Not Lost

Software is a mutable solution to a vast array of challenges. It is easy to create and easy to change. But because of its power, it requires attention to detail like nothing else in our world. Development organizations need to pay close attention to process as they plunge into the break-neck pace of Agile and Continuous Release.

Contact Us