Frequently Asked Questions

  • 1. Testing
  • 2. General
Performance Testing
Expand All | Collapse All
  • 1. Are there restrictions on the time of day that you run tests?
     

    Absolutely not!
    We run our tests when it makes sense for your application(s).  We are happy to run tests at a time that least impacts your customers or users.

    Was this answer helpful ? Yes / No
    Viewed 14 Times
  • 2. Is there a limit to the number of virtual users you can generate?
     

    We can generate load with more than 1,000,000 (one million) simultaneous users that execute real Web sessions – including login to protected Web pages and full support for user specific session data.

    Was this answer helpful ? Yes / No
    Viewed 41 Times
  • 3. Can the load you generate come from different geographic regions?
     

    Yes. We use Amazon EC2, and can generate load from any or all of the following geographic regions:

    • us-west-2 (Oregon)
    • us-west-1 (N. California)
    • us-east-1 (Virginia)
    • sa-east-1 (Sao Paulo)
    • eu-west-1 (Ireland)
    • eu-central-1 (Frankfurt)
    • ap-southeast-2 (Sydney)
    • ap-southeast-1 (Singapore)
    • ap-northeast-1 (Tokyo)

     

    Was this answer helpful ? Yes / No
    Viewed 248 Times
  • 4. Can each virtual user login with different credentials?
     

    Yes.  Provide us with a comma-delimited file containing:

    userid, password

    We will use those credentials during the test so that each virtual user can log in as a different person.

    We can parameterize other things as well (search terms, form fields...etc.).

    Was this answer helpful ? Yes / No
    Viewed 15 Times
  • 5. How far in advance do I need to schedule a test?
     

    A good rule of thumb would be 48 hours.  We might be able to work with less lead time - but, it can not be guaranteed.

    Was this answer helpful ? Yes / No
    Viewed 3 Times
  • 6. Do you offer discounts for running multiple tests?
     

    Yes we do.  Give us a call and we will put together a price package for you.

    Was this answer helpful ? Yes / No
    Viewed 10 Times
  • 7. What is different about your approach to performance testing?
     

    At FTL Metrics, we know that a load test is not the complete answer.  That is why we have to two-pronged approach:

    1. We analyze your homepage from a front end perspective and give you an actionable list of items to improve performance.  Items covered in our report are things like image compression, placement of javascript and .css on the page, enabling compression and leveraging browser caching.
    2. Our performance testing brings bottlenecks in your application, application server and infrastructure to the surface and makes it easy to see where improvements can be made.
    Was this answer helpful ? Yes / No
    Viewed 25 Times
  • 8. What errors do you typically see during tests?
     

    The following HTTP STATUS codes highlight typical errors during testing:

    400: Bad Request – A VUser sent a request that is not properly formatted.
    401: Unauthorized – A VUser failed authentication to access the target server.
    403: Forbidden – A VUser lacked the permission to get a resource.
    404: Not Found – Usually the link is invalid due to a typo, stale token, or the file does not exist.
    500: Internal Server Error – Usually indicates the target server is having trouble with the load or something is wrong with the request.
    502: Bad Gateway – Could be a configuration issue with target server.
    503: Service Unavailable – Your server is barely functioning, and can only respond with this status code.

    Was this answer helpful ? Yes / No
    Viewed 8 Times
  • 9. Why ramp up the number of virtual users?
     

    It is much better to ramp up the number of virtual users over time as opposed to starting them all at once. The following reasons are why:

      Having them all start at the same time can introduce an artificial bottleneck. Example: 2,000 users all running a login at the same time might cause a problem. The same 2,000 users - if spread-out within the application might run just fine. Not to mention: staggering virtual users is also more realistic.
      Starting slow, allows cache to fill and database connects get made before the system is straining.
    Was this answer helpful ? Yes / No
    Viewed 12 Times
  • 10. Will the load test show up in our Google Analytics reports?
     

    Most customers request that 3rd-party requests such as the ones to Google Analytics be removed from the test. While the choice is yours, we recommend removing them as leaving them in would inflate your numbers artificially.

    Was this answer helpful ? Yes / No
    Viewed 14 Times
  • 11. Can your test be stopped in case our application crashes?
     

    Yes. We can stop a test immediately.

    Was this answer helpful ? Yes / No
    Viewed 9 Times
  • 1. Where are you located?
     

    Our office is in South Florida, but our customers are all over the world.

    Was this answer helpful ? Yes / No
    Viewed 81 Times
  • 2. Can I try before I buy?
     

    Yes!

    We will run a 10-minute sample test with 10 virtual users (small enough to not adversely affect your customers) and provide you with a PDF report.

    Contact us to schedule your sample test today!

    Was this answer helpful ? Yes / No
    Viewed 217 Times
  • 3. Do you offer discounts for running multiple tests?
     

    Yes we do.  Give us a call and we will put together a price package for you.

    Was this answer helpful ? Yes / No
    Viewed 10 Times
  • 4. What is different about your approach to performance testing?
     

    At FTL Metrics, we know that a load test is not the complete answer.  That is why we have to two-pronged approach:

    1. We analyze your homepage from a front end perspective and give you an actionable list of items to improve performance.  Items covered in our report are things like image compression, placement of javascript and .css on the page, enabling compression and leveraging browser caching.
    2. Our performance testing brings bottlenecks in your application, application server and infrastructure to the surface and makes it easy to see where improvements can be made.
    Was this answer helpful ? Yes / No
    Viewed 25 Times
  • 5. Can I pay with a credit card?
     

    Yes.  FTL Metrics accepts major credit cards and bank transfers.

    Was this answer helpful ? Yes / No
    Viewed 182 Times