I have run hundreds of load tests in the last few years. The majority of customers run tests only once or twice a year – normally, in advance of expected heavy use of their site. While these customers are to be applauded for running performance tests, their sites would get a lot more benefit from running regular tests. In this blog post, I will enumerate some of the reasons for running tests regular and repeated testing.
First let’s talk about what I mean by regular and repeated. Regular testing means that testing needs to be baked into your development process. Testing should occur at various points in the development process. An example would be that you would test all code changes before implementing them into production. This would include all point releases, as well as any patches that are produced.
When I say repeated, I mean that most tests should be run at least three times. The reason is that each test is going to uncover your site’s biggest bottlenecks. Once you perform remediation to improve these major offenders, new bottlenecks will rise to the top to take their place. By testing and improving a few times, much greater performance will be realized. As an example, a customer of mine ran a test, and then took corrective measures based on the results. They were able to realize 20 percent better performance on their site. After running two additional sets of tests, the performance improvement was increased by another 25 percent.
When should you test?
Prior to any expected spike in your web traffic
Before each major and minor release
Before rolling out any patch
Before and after an update/upgrade of your application server
Before and After rolling out new hardware
Before and after moving content off of your servers to a Content Delivery Network (CDN)
These are a few examples of times when it is necessary to test. An even better idea is to test all the time. If you are constantly testing, you will minimize any unpleasant surprises.