Guidelines for testscripts

For the performance test automation to work properly, be sure to implement some basic guidelines.

Script

• transactions that are reported on should have at least a number/string pattern in it: (0001_login, BSM_01_login)
• load parameters should be configured in a 'user defined variables' component, named WORKLOAD_ for. we execute productie, duurtest, stresstest. So WORKLOAD_productie would be a valid name. For specific test patterns or load levels also free format names can be used, eg: WORKLOAD_christmas
• be sure to define an extra load level called 'Verificatie' for pre-test check (WORKLOAD_Verificatie) workload components will be enabled/disabled automatically depending on the load level that is executed
• transaction controllers are supported, be sure to disable 'Generate parent sample'. This option is due to removal from the component.

Test patterns

• productie (loadtest): be sure to execute enough iterations to have all transactions execute for minimum of 100 times, typical productie level test is 60mins
• stresstest: be sure to test at least 30mins to enable the algorytm to calculate response time baseline between 5 and 10 minutes of the test
• stresstest: be sure to slowly raise the load to ensure the metrics to give the algorythm the chance to detect a breaking point
• endurance: be sure to configure a duration that is long enough to see stability problems if any exist, no minimum, typically runs 6 hours
• validation: be sure to configure the script in such a way that the Validation test is short and touches all transactions once. This test is used as pre-test that executes before each normal test (load, stress, duration) to ensure the application is up and the script matches the application version.

General

Fast response times When dealing with an application with fast response times (eg around 100ms) be prepared to see response times fluctuate more due to virtualization and environmental noise. Possible solution is to cluster transactions into transaction controllers with clusters of transactions under it.