Multiple Scripts: Balancing Realism with Complexity and Effort

No Comments

One way that you can make your performance tests more realistic is to run multiple scripts at the same time – forming one test representing a blend of use cases.  Afterall, all of the visitors to your website do not follow the same set of steps upon arrival and there may be a variety of “roles” represented by these visitors.  By creating multiple scripts – each representing a particular use-case, your test will more accurately resemble your site traffic.  However, you will need to strike a balance between coverage of all possible use-cases on one hand and complexity and effort on the other.

Some load testing software has a built-in limit on the number of simultaneous scripts that can be run in a single test.  The number may be limited to 1 or 3 or 10 but, even in cases where there is no defined limit it makes sense top carefully weight the pluses and minuses.

On the plus side, you have realism.  The more use-cases covered, the more like your actual traffic your test will be.

Minuses:

There is overhead associated with each script.

— Time and effort to create, validate and possibly parameterize each script

— Memory and CPU on each load generator

— Additional resources used by the controller to keep track of additional scripts

I am very careful to make absolutely sure that the testing software is not the bottleneck.  It seems to me that running a very large number of scripts (and monitoring each of them during the test) is one way to overload the testing software.  Yes, each software package is designed differently, and may be more or less affected – but, really, keeping it simple is a good policy.

I have run tests with 10 scripts running at the same time (rarely) and the test had difficulties that were very possibly due to the number of scripts.  As a general rule, I try to keep the number of scripts at 5 or below – and most tests have either one or three scripts running.

 

Michael's LinkedIn Profile: Click Here

    About us and this blog

    We are all about web application performance, and use this blog as a way to disseminate relevant information.

    More from our blog

    See all posts