How Data Drives the Value of Software Testing Solutions

Building a Case for Data in Software Testing

Technology in the twenty-first century has evolved at blistering speeds, but today, the most remarkable facet of that evolution is how much data is captured from the devices we use and the software we engage with. The amount of data that is collected every second based on the clicks, swipes, or likes is staggering, and it’s not just stagnant information. Data collection has proven to be a vital asset for every industry in the world, from marketing and real estate to healthcare and human resources. In this article, we’ll discuss how data can help to build robust software testing solutions.

A study by DOMO, developers of a cloud-based operating system suggest that Americans use 2,657,700 GB of Internet data every second. So what does that translate to?

Here’s a breakdown of how that data is used every 60 seconds:

• Texts: 15,220,700 texts sent
• Twitter: 456,000 tweets sent
• Google: 3,607,080 searches
• Venmo: 51,892 peer-to-peer transactions
• Netflix: 69,444 hours streamed
• YouTube: 4,146,600 videos watched

Those statistics magnify the remarkable amount of data generated, stored, and analyzed. They also show how critical a software testing solution has become to everyone from OTT service providers like Netflix and Hulu as well as healthcare systems, financial institutions, and government agencies.

But before we look at the specifics of software testing, it’s important to understand the basics of data, the analytics of data, and the value that analytics can deliver.

Data Analytics Discovers Trends and Solves Problems

Data analytics is the process of gathering insights from data. That process uses tools, techniques, and methods for collecting, organizing, and storing data to discover trends and solving problems.

Data has been around for decades, but it started the meteoric rise – both in use and consumer awareness – in 2005 when Facebook and YouTube presented overwhelming uses and benefits from it. Since then, the widespread adoption of analytics has demonstrated that data analysis is a fundamental component of business strategy.

Data Analytics Informs Key Business Strategies

Businesses across all sectors and industries use data analytics to inform key business strategies and increase revenue. Internet tech company CIO gives these examples to demonstrate how analytics can apply to very different business models.

• La-Z-Boy uses data analytics to improve operations: International furniture retailer La-Z-Boy has used analytics to improve operations in 20 departments, including HR, finance, supply chain, and sales. Analytics helps the company manage pricing, SKU performance, warranty, shipping, and other information, as well as forecasting inventory levels.
• Predictive analytics helps Owens Corning develop turbine blades: Manufacturer Owens Corning, with the help of its analytics center of excellence, has used predictive analytics to streamline the process of testing the binders used in the creation of glass fabrics for wind turbine blades. Analytics has helped the company reduce the testing time for any given new material from 10 days to about two hours.
• Kaiser Permanente reduces waiting times with analytics: Kaiser Permanente has been using a combination of analytics, machine learning, and AI to overhaul the data operations of its 39 hospitals and more than 700 medical offices in the U.S. since 2015. It uses analytics to better anticipate and resolve potential bottlenecks, allowing it to provide better patient care while improving the efficiency of daily operations.

Extracting information from software development, data analytics relies on mathematics, and statistics to examine data to predict, and improve performance. Techniques, including data mining, data modeling, data cleansing, and data transformation, help to discover trends and solve existing or potential problems. Making sure that business software is capturing the data accurately is vital and why software testing is key.

While the examples above show diversity in business deliverables, they also demonstrate a common feature—software. The efficacy of the data is dependent on the robust nature of the software and the type of analytics used.

Four Types of Analytics

Different methods of analytics deliver different kinds of information. What is good for Healthcare may not align with the needs of a retail marketer while a software testing company relies on a predictive method to deliver the right solution.

1. Descriptive analytics: What has happened and what is happening right now? Descriptive analytics uses historical and current data from multiple sources to describe the present state by identifying trends and patterns. In business analytics, this is the purview of business intelligence (BI).
2. Diagnostic analytics: Why is it happening? Diagnostic analytics uses data (often generated via descriptive analytics) to discover the factors or reasons for past performance.
3. Predictive analytics: What is likely to happen in the future? Predictive analytics applies techniques such as statistical modeling, forecasting, and machine learning to the output of descriptive and diagnostic analytics to make predictions about future outcomes. Predictive analytics is often considered a type of “advanced analytics,” and frequently depends on machine learning and/or deep learning.
4. Prescriptive analytics: What do we need to do? Prescriptive analytics is a type of advanced analytics that involves the application of testing and other techniques to recommend specific solutions that will deliver desired outcomes. In business, predictive analytics uses machine learning, business rules, and algorithms.

Analytics has become a way to shape business processes and inform decision-making to improve business results and accelerate speed to market. That speed is essential in the software development arena so staying ahead of the game is vital. Combining numerical data with a humanistic approach is critical in understanding the big picture.

How is Analytics Used in Software Testing?

Predictive analytics (PA) is a method most often used in software testing for one primary reason — it helps predict issues that can lead to bigger problems. The speed of software development and release requires software testing companies to perform at breakneck speeds as well. By using artificial intelligence, machine learning, algorithms, mining, and modeling, predictive analytics can help not only keep up but stay ahead. Tech consultancy, TechDecisions captured how predictive analysis works with four important points:

1. Data is used for predicting possible scenarios, which helps to understand what event led to the other.
2. Predict how a user will react to a specific event based on their previous pattern.
3. Possible areas of bugs, possible reasons for encountering those bugs, and events leading to those bugs can be predicted through predictive analytics.
4. Modify testing methodologies by analyzing the user’s behavioral patterns. Understanding these patterns, we can provide focus to other major areas.

By using predictive analysis (PA), software companies can predict issues and engineer solutions to meet technical requirements while providing a focus on customer behavior as well. The benefits for better software testing solutions are as follows:

1. More Data, Better Software Testing Solutions
Software testing generates data. Testing creates log files. Log files highlight defects showing how the results impact the user. The more data, the more informed developer and more satisfied the user.

Software testing is also meant to find issue patterns which also leads to data points. The data is combined with predictive analytics algorithms to identify patterns that help make accurate predictions about future failures. Additionally, software testing solutions utilize machine learning algorithms. These algorithms help to optimize regression suites and determine redundant cases. Based on the previous test results, predictive analytics help in forecasting the future pass rate.

2. Customer-Centric Testing
Predictive analytics can enable the tester to monitor customer behavior and feedback. In social media monitoring, sentimental analysis helps in understanding the feedback of customers about various applications and products making the process quicker and easier.

3. Better Defect Detection
Detecting defects is one of the first steps towards improving quality. With the help of available data, predictive analytics can uncover defects that would otherwise go undetected. At its core, predictive analytics can help the software team reach the root cause of the failures faster.

4. Efficiencies and Growth
Predictive analytics helps the testing team better understand what is working and what they can improve. It can help drive better application efficiencies by using the information gathered in the development and testing process.

5. Saves Time and Money
Predictive analytics can help find defects faster which means, saving time by increasing speed to market and money by saving time. Predictive analytics is also efficient at discovering what kind of bugs exist and when they originate, increasing efficiencies in the end-to-end development process as well.

Predictive analytic solutions forecast future outcomes by reading and interpreting historical data. Through the use of statistics, advanced algorithms, and machine learning, quantitative and qualitative information is transformed into predictions. — Wonderflow

Differences in the Data

Data insights seem clear when using numbers that provide specific measurements, but when the research is not based on data sets or numbers, the analysis can be more challenging. The difference between quantitative and qualitative data is significant and important to understand especially when it is applied in context. Data collection company Formplus defines the difference as follows:

Quantitative Data
Quantitative data’s value is measured in the form of numbers or counts, with a unique numerical value associated with each data set. (e.g. How many? How often? How much?)

This data type can also be defined as a group of quantifiable information that can be used for mathematical computations and statistical analysis which informs real-life decisions. For example, a manufacturing company will need an answer to the question, “How much does the production cost?”.

Quantitative data of the company’s cost of production will be collected through this question and will inform the company’s selling cost (where selling cost = production cost + profit).

Qualitative Data
Qualitative data is a type of data that describes the information. It is investigative and often open-ended, allowing respondents to express themselves.
This data type isn’t measured using numbers but rather categorized based on properties, attributes, labels, and other identifiers. Numbers like national identification numbers, phone numbers, etc. are however regarded as qualitative data because they are categorical and unique to one individual.

Examples of qualitative data include name, state of origin, citizenship, etc. A more practical example is a case whereby a teacher gives the whole class an essay that was assessed by giving comments on spelling, grammar, and punctuation rather than score.

An example of qualitative data used in software testing could include measuring the value of an Onshore vs. an Offshore software testing company.

Because quantitative data can provide the exact ROI through profit and loss, an offshore company might look like a more profitable choice. Benchmarking the ROI for a testing company based in the home country is a challenge because there is value applied to aspects other than the hourly fee or cost of business.

Qualitative data can show cost-saving of an onshore software testing company, and therefore profit, based on the efficiencies of a testing company based in the home country, speaking the same language, and working in the same time-zones. Qualitative data represented illustrating these benefits can be invaluable.

The Value of Customer-centric Software Testing Solutions

Offshore testing often relies on that methodical step-by-step approach without much oversight or collaboration with the customer. This approach has been popular as a way to keep overhead costs low. However, there has been a shift lately and the positive results can be measured.

Keith Klain is the head of the Global Test Center for corporate and investment banking and wealth management at Barclays Bank. Responsible for managing hundreds of software testers in the United States, Europe, and the Asia-Pacific region, he believes that the shift away from so-called “factory methods” of testing is a good thing.

Assembly line methods where one group plans the work and another executes in a detailed step-by-step manner misses the whole picture and important aspects of the customer experience.

Factory testing eliminates the testers’ ability to react, learn and change approach. It’s the kind of thing that happens in chess every time an opponent makes an unexpected move. —Keith Klain, Global Testing, Barclays Bank

What this kind of thinking suggests is that a smaller and more nimble team can provide more focused and detailed thinking, along with a more human-centric approach. The ability to learn and adapt is more successful in design and execution.

While the factory method is typically used with offshore resources where daily oversight and close collaboration is impossible, a nimble onshore team partnering with the developers in the same country and in the same time zone leads to some of the best testing software solutions.

Qualitative Data Helps Close the Gap in Pricing

Barclay’s Keith Klain explains that while outsourcing to offshore resources has been widely adopted for the last 15 years, the financial models are shifting, and along with rising wages, cost of living increases, and currency fluctuations, prioritizing offshore resources has also declined.

“Most of the improvement models used to rationalize the commoditized testing approach use strictly quantitative metrics to assess the quality or measure improvement, an approach which breaks down rather quickly beyond any first-order metrics,” says Klain. “There is an increased focus on business value and testing skills, which means you have to bring more to the table than just the ability to do it cheaper.” — Keith Klain, Global Testing, Barclays Bank

An experienced software testing company would follow this model by making sure test technicians are recruited and trained with both quantitative and qualitative research and reporting in mind. As a result, a comprehensive onshore software test company will work harder and faster to compress test cycles and accelerate releases. The hourly fee that a more efficient onshore testing company charges will offer more and quickly close the gap in the pricing of an offshore counterpart.

The QualityLogic Example

With all of this in mind, onshore companies like QualityLogic bring more to the table. With highly trained experts, QualityLogic understands the need to approach software testing with detailed quantitative data for better machine learning and prediction, but we also believe that communication and relationship are everything. The relationships we develop with clients and the understanding we have for the user helps to inform everything we do.

Without a qualitative approach, we would miss opportunities in delivering the best software testing solutions possible. We think the benefits are clear and the points below show some of the qualitative value that comes from working onshore.

Qualitative Benefits of Onshore Testing

• Face-to-face communication: Onshore testing enables real-time detection and communication on emerging issues allowing for more efficient problem-solving. Face-to-face, meaning Zoom, Microsoft Teams, Google, or Slack calls.
• More effective communication: With no significant time zone or cultural differences, the risk of misunderstandings within teams is significantly reduced. This also means that onshore testers are more likely to find edge cases in the testing process.
• Fewer security risks: While both onshore and offshore testing companies have strict security protocols in place, countries, where onshore companies are based, will typically have stricter security laws.
• Improved time to market: Based on the above benefits, the speed to market is greatly improved.
• Longer contract terms: Offshore companies may push for a full-time team. If your requirements are going to be more project-based, an onshore software testing company may be a better fit as they can easily provide both project and longer-term testing. Offshore testing companies may also require change fees when you look to adjust your testing requirements.

Proof That Qualitative Value Drives Better Business

To show that qualitative value is more than just words, QualityLogic did a case study to prove that the qualitative value of our work led to better business results. Below is some of the qualitative feedback that came out of the project.

A global industrial PC company needed a flexible and independent solution to help manage an inconsistent volume of work. QualityLogic was able to acclimate to operational needs and provide a partnership that aligned with the company culture and accommodated the needs of a small but nimble in-house QA department.

QualityLogic’s ability to meet the QA requirements and in some cases accelerate release schedules, offered value that could not be measure with numbers. Flexibility in collaborating with the internal development team and the ability to align with the company values also provided real qualitative value inefficiencies and cost savings.

The four-person QualityLogic team provided the flexibility and resources needed to get features and fixes to production fast, while still maintaining a high level of product quality. One of the most surprising and appreciated aspects of the collaboration was the speed at which the QualityLogic team was able to understand and adopt the business processes, both in terms of the technical needs and the ERP system.

Additionally, as is the case in the fast-paced tech industry, communication is vital. The client was surprised by how transparent, straightforward, and easy the communication was. QualityLogic’s ability to make a challenging job seem so easy is what made them a truly integrated and trustworthy partner.

Again, these kinds of benefits could not be measure by numbers but would deliver qualitative data helping to prove financial gains.

QualityLogic’s inclusive and invested approach resulted in the confidence of internal resources, as well as the leadership team who were impressed by how quickly the QualityLogic team was able to ramp up and deliver.

As a result, the backlog of testing needs was met within 6 months from contract initiation. These results demonstrated value based on qualitative data.

Quantitative + Qualitative Data = Better Business

While that case study is just one example from one company, the method that was employed can be applied to any process, and the results will be the same. It is just as much a mantra as a method, and we are diligent at making sure our approach is effective and approachable.

The goal is to capture qualitative customer data and turn it into actionable insights on future decisions. With each project, there is even more learning to foster continuous improvement for more robust software testing solutions.

Our adaptability and new technologies set up apart from other companies. We combine manual and automated testing to provide comprehensive QA coverage. If you are looking to leverage quantitative and qualitative data for better business value, call us today. With a quick, informal call, we can help you assess your software testing options and clear the clutter or quell the chaos for the straightest line to success.