Load and Performance Testing: A Super Bowl Cautionary Tale
The 2017 Super Bowl came with an ad from a virtually unknown construction supplies company, 84 Lumber. It was intended as a recruitment ad but contained a strong message about immigration. The 90-second ad aired just before the halftime show and directed viewers to a website for more information.
Inside of a minute, the website, journey84.com, received more than 300,000 hits. This was more than enough to slow the site’s performance and display a ‘service unavailable’ message to visitors. It seems that 84 Lumber’s site was only able to serve 150,000 requests per minute. After one hour, the site had received more than 6 million requests to load.
Following the 2016 Super Bowl halftime performance by Coldplay and Bruno Mars, a commercial announced that Beyoncé would be going on a world tour. The commercial for her Formation tour instructed viewers to go to either the official Beyoncé or Live Nation websites. Ten minutes after the commercial aired, Beyoncé’s site was down.
This idea of instant site loading even led Frito-Lay to invite its customers to create their own Doritos ad spots in a ‘Crash the Super Bowl’ ad campaign. At least one fan-made commercial was guaranteed to air during the Super Bowl each year of the campaign. From 2006 through 2016, eight editions of the Crash the Super Bowl commercial contest were held. Doritos fans submitted more than 36,000 entries showing just how much traction a major event can generate.
Great Content is Only Half the Game
Obviously, site content is intensely important. That said, site visitors have to be able to load the pages in order for that content to have any effect. Site performance is key to conversion ratios, the rate at which site access turns into intended results.
According to Invesp, the global average conversion rate for ecommerce sites, from the 4th quarter 2015 to the 4th quarter 2016 was 2.95%. While that number depends heavily on your market, it is still representative of the relatively low rate at which site access becomes a successful sales experience.
For your ad campaign to be successful, you need both outstanding content and smooth, fast page loading. One wins you very little without the other especially when less than 10% of your visitor encounters pay off with the desired results.
Gear Up: Making Sure Your Website Can Handle The Traffic
Load and performance testing are commonly confused terms that are even used interchangeably. In its most basic definition, load testing attempts to load a site through its user interface to determine its crash point and its performance degradation as that point is approached. One of the tests is to see that the site ‘fails gracefully’ such that it can autonomously recover and provides reasonably useful information to the user.
This type of testing typically loads up the user interface of a site or application with virtual users who are really manifestations of a test system. Such systems can run hundreds or even hundreds of thousands of these users to login and exercise specified paths through the system using specified features. These paths can be apportioned between groups of virtual users to vary the load on all parts of the system during a single test.
Performance testing tends more towards a focus on individual aspects of the system under test. Stimulation of specific middle-ware segments, database access and third party API calls is used to specifically measure the response times of these code units. Commonly used to troubleshoot load test results, this type of testing can often be performed by code modules written specifically for the test in order to manage selection of features and the range of parameter data used.
Marketing + Sales + Website Testing = Touchdown!
Marketing and development need to closely coordinate sales campaigns with application testing. If a publicly accessible system such as an information or ecommerce site is going to receive massive PR, it needs to be ready for massive access. That means loading it with user logins and carefully using performance tests to weed out any bottlenecks in the precious flow of data between the user and the system.