Executing Load Test Programs

After the load test program has been called by the Project Navigator, you must enter the test input parameters for the test run (a single execution of the load test program is also called “test run”).

The most important parameters are Number of Concurrent Users and Load Test Duration. You should also enter a small comment about the test run into the input field Annotation.

Input Fields
save as template stores all load test input parameters additionally inside a XML template . Later, this template can be used to rerun (repeat) the same load test.
Execute Test Form denotes from which computer or load releasing cluster the load test will be executed. If you did not define additional remote Exec Agents or Exec Agent Clusters , only the option “Host: Local Exec Agent“ is available, indicating that the load test program is executed by your local system.
Number of Concurrent Users number of users which are simulated during the load test.
Load Test Duration planned test duration. After the test duration has elapsed, each user will terminate the current loop (repetition of the web surfing session) before the test run completes; thus, the duration of the test run will be a little bit longer than the planned test duration given here. If the value of the input field Max. Loops per User is not set to unlimited, the test run may complete before the planned test duration elapses because all users have already executed their maximum number of loops.
Max. Loops per User maximum number of surf session repetitions per user. If the value of the input field Load Test Duration is not set to unlimited, the test run may complete before the planned test duration elapses because all users have already executed their maximum number of loops.
Startup Delay per User usually concurrent users are not started at exactly the same time because this, rather unusual, scenario will overload the web server immediately. This parameter controls how much time will elapse before an additional user will be started (ramp-up of load at start).
Max. Network Bandwidth per User allows you to reduce the maximum network bandwidth per user in order to simulate slow network connections to the web server; for example, connections over DSL or modem lines. The downlink and the uplink speeds can be adjusted separately to simulate asymmetric network bandwidths.
Request Timeout per URL timeout in seconds per single URL call. If this timeout expires, the URL call will be reported as failed (no response from web server), and the emulated user will abort the current loop and continue with the next loop.
Max. Error-Snapshots limits the maximum number of error snapshots taken during load test execution . Either the maximum memory used to store error snapshots can be configured (recommended - for cluster jobs: value overall cluster members), or alternatively the maximum number of error snapshots per URL can be configured (not recommended - for cluster jobs: value per Exec Agent).
Statistic Sampling Interval statistic sampling interval during the load test in seconds (interval-based sampling). Used for time-based overall diagrams like for example the measured network throughput. If you run a load test over several hours, it is required that you increase the statistic sampling interval up to 10 minutes (600 seconds) to save memory. If the load test runs only some minutes you may decrease the statistic sampling interval.
Additional Sampling Rate per Page Call captures the measured response time of a web page each time when a simulated user calls a web page (event based sampling). Used to display the response time diagrams at real-time as well as in the Analyze Load Test Details menu. For endurance tests over several hours it is strongly recommended that the sampling rate for web pages is set between 1% and 5%. For shorter tests 100% sampling rate is recommended.
Additional Sampling Rate per URL Call captures the measured response time of a URL each time when a simulated user calls a URL (event based sampling). Used to display the response time diagrams at real-time as well as in the Analyze Load Test Details menu. For endurance tests over several hours it is strongly recommended that the sampling rate for URL calls is disabled or set to 1% or 2%. For shorter tests 100% sampling rate is recommended. In addition to capturing the response time of the URL calls further data can be captured by using one of the following Add options.
recommended no additional data are captured.
Performance Details per Call additionally collects the network connect time, the request transmit time, the response header wait time, the response header receive time, and the response content receive time of the URL calls.
Request Headers additionally collects the request headers of the URL calls.
Request Content (Form Data) additionally collects the request content (form data) of the URL calls.
Req. Headers & Content additionally collects the request headers and request content (form data) of the URL calls.
Response Headers additionally collects the response headers of the URL calls.
Resp. Headers & Content additionally collects the response headers and the response content of the URL calls.
All - without Resp. Content additionally collects the request headers, the request content, and the response headers of the URL calls.
All - full URL Snapshots additionally collects all data of the URL calls.
Debug Options these options allow you to debug the inner workings of the load- test program. The result is written to the job_*.out file, which is usually only used to analyze internal errors in the load test program:.
none - recommended recommended default value. Note that all measured performance data, and all error snapshots, are already stored inside the result file (*.prxres); therefore, special debug options are not necessary in order to analyze the load test result.
debug failed loops writes the log data of all executed web surfing sessions (loops) that have failed to standard output, including information about dynamically-extracted session parameters and Input Files.
debug loops writes the log data of all executed web surfing sessions (loops) to standard output, including information about dynamically-extracted session parameters and Input Files.
debug headers & loops includes the above option “debug loops” and, in addition, writes out all transmitted and received HTTP headers to standard output.
debug content & loops includes the above option “debug loops” and, in addition, writes out all transmitted and received HTTP content data to standard output; however, this option only writes out data which has been transmitted or received in ASCII format, such as HTML form parameters and HTML, XML, SOAP, or CSS style sheet data - but no binary data, such as images.
debug cookies & loops includes the above option “debug loops” and, in addition, writes out all received and transmitted cookies to standard output.
debug keep-alive & loops includes the above option “debug loops” and, in addition, writes out additional debug information about re-used network connections to standard output.
debug SSL handshake & loops includes the above option “debug loops” and, in addition, writes out additional debug information about SSL handshakes to standard output.
Additional Options these options allow you to enter special options. All special options keywords begin with a minus sign. Several options can also be combined (separated by space characters):.
multihomed Forces the Exec Agent(s) to use multiple local IP addresses when executing the load test. This option is only used by the Exec Agent(s) if multiple IP addresses are configured at the operating system level, and are assigned to the Exec Agent configuration . The effect of this option is that each user uses, during the load test, its own client IP address. If fewer IP addresses are available than concurrent users are running, the IP addresses are averaged across the users.
dnshosts <file-name> Effects that the load test job uses an own DNS hosts file to resolve host names - rather than using the hosts file of the underlying operating system. Note that you have to ZIP the hosts file together with the compiled class of the load test program. To automate the ZIP it's recommended to declare the hosts file as an external resource (w/o adding it to the CLASSPATH).
dnssrv <IP-name-server-1 [,<IP-name-server-N ] Effects that the load test job uses specific (own) DNS server(s) to resolve host names - rather than using the DNS library of the underlying operating system. When using this option, at least one IP address of a DNS server must be specified. Multiple DNS servers can be configured separated by commas. If a resolved DNS host name contains multiple IP addresses the stressed Web servers are called in a round-robin order (user 1 uses resolved IP Address no. 1, user 2 uses resolved IP Address no. 2, etc.).
dnsenattl Enable consideration of DNS TTL by using the received TTL-values from the DNS server(s). This option cannot be used in combination with the option -dnsperloop. \ Note: when using this option the resolved IP addresses (and therefore the stressed Web servers) may alter inside the executed loop of a simulated user at any time - suddenly from one URL call to the next one.
dnsfixttl <seconds> Enable DNS TTL by using a fixed TTL-value of seconds for all DNS resolves. This option cannot be used in combination with the option -dnsperloop. \ Note: when using this option the resolved IP addresses (and therefore the stressed Web servers) may alter inside the executed loop of a simulated user at any time - suddenly from one URL call to the next one.
dnsperloop Perform new DNS resolves for each executed loop. All resolves are stable within the same loop (no consideration of DNS TTL within a loop). This option cannot be used in combination with the options -dnsenattl or -dnsfixttl. \ Note: consider when using this option that the default or the configured DNS servers are stressed more than usual because each executed loop of each simulated user will trigger one or more DNS queries.
dnsstatistic Effects that statistical data about DNS resolutions are measured and displayed in the load test result, by using an own DNS stack on the load generators. \ Note: there is no need to use this option if any other, more specific DNS option is enabled because all (other) DNS options also effect implicitly that statistical data about DNS resolutions are measured. If you use this option without any other DNS option, the (own) DNS stack on the load generators will communicate with the default configured DNS servers of the operating system - but without considering the "hosts" file.
mtpu <number> Allows to configure how many threads per simulated user are used to process URLs in parallel (simultaneously). Note: this value applies only for URLs which have been configured to be executed in parallel.
nosdelayCluster Effects for Cluster Jobs that the Startup Delay per User is applied per Exec Agent Job instead of applying it overall simulated users of the Cluster Job. Thus a faster ramp up of load can be achieved.
setuseragent "<text>" Replaces the recorded value of the HTTP request header field User-Agent with a new value. The new value is applied for all executed URL calls.
collect <measuring agent host [:port][,<measuring agent host [:port]]…> Forces the load test program to collect additional data from external measuring agents. Such data contain for example system operating values like CPU usage and memory consumption of the Web server and the database server.
sslcache <seconds> Alters the timeout of the user-related SSL session cache. The default value is 300 seconds. A value of 0 (zero) indicates that the cache is disabled.
sscreset Resets the user-related SSL session cache per loop (default: no reset per loop).
sslcmode Applies SSL (https) compatibility workarounds for buggy SSL servers. You may try this option if you consistently receive the error message "Network Connection aborted by Server" for all https calls when executing the load test.
tz <timezone> Allows you to set another time zone to be used during the load test, For a list of supported time zones: see the Application Reference Manual, Chapter 6.
Xbootclasspath/a:<path> Specify for the load test job a path of JAR archives and ZIP archives to append to the default bootstrap class path.
Xbootclasspath/p:<path> Specify for the load test job a path of JAR archives and ZIP archives to prepend in front of the default bootstrap class path.
SSL specifies which HTTPS/SSL protocol version should be used:.
All allows you to detect the best SSL protocol version automatically. The TLS 1.2 protocol is preferred; however, if it is not supported by the web server, an older TLS version or SSL v3 is used. This is standard behavior implemented by many commercial web browser products.
TLS12 sets the SSL protocol version to TLS version 1.2.
TLS11 sets the SSL protocol version to TLS version 1.1.
TLS sets the SSL protocol version to TLS version 1.0.
v3 sets the SSL protocol version to SSLv3.
Annotation here you should enter a short comment about the test run, such as purpose, current web server configuration, and so on. This annotation will be displayed on the result diagrams.

Full Snapshots

Warning: capturing additional URL data takes much memory and uses also much CPU. Therefore the test duration should not exceed 10 minutes if you use one of these add-options in combination with 100% sampling rate per URL call. Reducing the sampling rate to 10% may allow a load test duration up to 30 minutes.

Tip: these additional URL data can be displayed and/or exported in the form of an HTML table when the test run has been completed.

If you have specified that the load test program be executed by a single Exec Agent (but not by an Exec Agent Cluster), the load test program is transmitted to the local or remote Exec Agent, and a corresponding load test job - with a job number - is created locally within the Exec Agent. The job is now in the state “configured”; that is, ready to run, but the job is not yet started.

image

Tip: each Exec Agent always executes load test jobs as separate background processes, and is also able to execute more than one job at the same time. The option Display Real-Time Statistic only means that the GUI opens an additional network connection to the Exec Agent, which reads the real time data directly from the memory space of the corresponding executed load test program.

  • Click the Start Load Test Job button to start the job.

If you have de-selected the checkbox Display Real-Time Statistic, the window will close after a few seconds; however, you can - at any time - access the real time statistic data, or the result data, of the job by using the Jobs menu which can be called from the Main Menu and also from the Project Navigator.

Alternatively, the load test program can also scheduled to be executed at a predefined time. However, the corresponding Exec Agent process must be available (running) at the predefined time, because the scheduling entry is stored locally inside the Exec Agent jobs working directory which is monitored by the Exec Agent itself. Especially, if you have started the local Exec Agent implicitly by using the ZebraTester Console - AND if the scheduled job should run on that local Exec Agent, you must keep the ZebraTester Console Window open in order that the job will be started ¹.

¹ This restriction can be avoided by installing the Exec Agent as a Windows Service or as a Unix Daemon.

Note: if you close the window without clicking on the Start Load Test Job button, the job remains in the state "configured" or “scheduled”. Afterwards you can use the Jobs menu to start or delete the job, or to schedule or cancel the schedule of this job.

image

Real-Time Job Statistics (Exec Agent Jobs)

image

Real-time statistics shown in this window are updated every 5 seconds for as long as the load test job is running.

You may abort a running load test job by clicking on the Abort Job button. This will take a few seconds because the job writes out the statistic result file (*.prxres) before it terminates.

Note: closing this window will not stop the load test job. If you close this window you can later acquire the load test result or return to this dialogue (if the load test is still running) by clicking on the Jobs icon in the Main Menu or in the Project Navigator window

image

Item Description
<Exec Agent Name> or <Cluster Name> The name of the Exec Agent - or the name of the Exec Agent Cluster - which executes the load test job.
Job <number> Unique job ID (unique per Exec Agent, or unique cluster job ID).
Real-Time Comment If real-time comments are entered during test execution, these comments are later displayed inside all time-based diagrams of the load test result detail menu.
Job Parameter The name of the load test program, and the program arguments - test input parameter.

The Web Transaction Rate Diagram shows the actual number of (successful) completed URL calls per second, counted overall simulated users. By clicking on this diagram, the Response Time Overview Diagrams are shown .

image

  • Total Passed URL Calls - The total number of passed URL calls since the load test job was started.
  • Total Failed URL Calls - The total number of failed URL calls since the load test job was started.
  • Keep-Alive Efficiency (%) - The efficiency in percent about how often a network-connection to the web server was successfully re-used, instead of creating a new network connection. This (floating) average value is calculated since the load test job was started.
  • AV Web Trans. Rate (URL calls/sec) - The (floating) average number of (successful) completed URL calls per second, calculated since the load test job was started

The Session Failures / Ignored Errors Diagram shows the actual number of non-fatal errors (yellow bars) as well as the number of fatal errors (red bars = failed sessions), counted over all simulated users.

image

  • Total Passed Loops - The total number of passed loops (repetitions of web surfing sessions) since the load test was started.
  • Total Failed Loops - The total number of failed loops (repetitions of web surfing sessions) since the load test was started.
  • Σ User's Think Time per Loop (sec): The total user's think time in seconds for one loop per simulated user.
  • Session Time per Loop (sec): The average session time for one loop per simulated user. This value is the sum of "the average response time of all URLs and all user's think times" per successful completed loop.

The Number of Users / Waiting Users Diagram shows the total number of currently simulated users (red bars) as well as the actual number of users which are waiting for response from the web server (purple bars). The users waiting for response is a subset of the currently simulated users.

image

By clicking on this diagram, the Statistical Overview Diagrams are shown .

  • Users Waiting For Response - the actual number of users which are waiting for response from the web server, compared to ("of") the total number of currently simulated users.
  • TCP Socket Connect Time (ms) - The time in milliseconds (per URL call) to open a new network connection to the web server.
  • AV Network Throughput (Mbit/s) - The total network traffic which is generated by this load test job, measured in megabits per second. This (floating) average value is calculated since the load test job was started.
  • Total Transmitted Bytes - The total number of transmitted bytes, measured since the load test job was started.

More actual measurement details are available by clicking on the Detailed Statistic button. Especially, an overview about the current execution steps of the simulated users is shown:

image

By clicking on the magnifier icon of a page, the most relevant measured values of the URLs are shown for the selected page.

  • Using this menu, you can also display and analyze error snapshots by clicking on the magnifier icon next to the failure counter . In this way, you can begin analyzing errors immediately as they occur - during the running load test.
  • By clicking on a URL, the corresponding URL Response Time Diagram is shown .

All of these detail data, including all error data, are also stored inside the final result file (.prxres) which can be accessed when the load test job has completed.

Response Time Overview Diagrams (Real-Time)

image

Description: displays during the load test (at real-time) a diagram per web page about the measured response times.

Please consider that maybe only a fraction of the response times is shown, depending on the Additional Sampling Rate per Page Call which was selected when the load test was started. For example: only every fifth response time is shown if the Additional Sampling Rate per Page Call was set to 20%.

Input Fields
Response Time (drop-down list) Allows to select the period, from the current time back to the past, within the response times are shown in the diagrams.
Time Bars (drop-down list) Allows to select if the bars inside the diagrams are shown as average values or as max. values. Please note that there if only a difference between the max. values and the average values if multiple measured samples of the response time fall inside the same pixel (inside the same displayed bar):.

image

The tables at the right side of the diagrams contain the response times for all URLs of the web page. Also these response times are either average values or max. values, depending on the selection in the Time Bars drop-down list. However these values are calculated since the load test was started, and always "accurately" measured which means that they do not depend on the value chosen for the "Additional Sampling Rate per Page Call".

You can click on a URL response time to show the corresponding URL Response Time Diagram :

image

image

At the left side inside the diagram, the average response time of the web page is shown as red colored text, calculated since the load test was started. But depending on the selected period this value may not be displayed in every case. At the right side inside the diagram, the last measured value is shown:

URL Response Time Diagram (Real-Time)

image

Description: displays during the load test (at real-time) the response times of a URL, and also a summary diagram about the measured errors of the URL.

Please consider that maybe only a fraction of the response times is shown, depending on the Additional Sampling Rate per URL Call which was selected when the load test was started. For example: only every fifth response time is shown if the "Additional Sampling Rate per URL Call" was set to 20%.

Input Fields
Response Time (drop-down list) Allows to select the period, from the current time back to the past, within the response times are shown inside the diagram.
Time Bars (drop-down list) Allows to select if the bars inside the diagram are shown as average values or as max. values. Please note that there if only a difference between the max. values and the average values if multiple measured samples of the response time fall inside the same pixel (inside the same displayed bar).

Info Box / Measured Values

image

All values in this info box are calculated overall successful completed calls of the URL, measured since the load test was started. These values are always "accurately" measured which means that they do not depend on the value chosen for the "Additional Sampling Rate per URL Call".

Total Passed URL Calls total number of passed calls for this URL.
Total Failed URL Calls total number of failed calls for this URL.
Average Size (Req. + Resp.) the average size of the transmitted + received data per URL call.
Max. Response Time the maximum response time ever measured.
Min. Response Time the minimum response time ever measured.
Av. TCP Socket Connect Time the average time to open a new network connection to the web server, measured for this URL. --- instead of a value means that never a new network connection was opened for this URL because HTTP Keep-Alive (re-using of cached network connections) was always successful. The additional percentage value shown in brackets at the left hand displays the percentage about how often a new network connection was opened to the web server, in comparison to how often this was not necessary. This percentage value is also called the reverse keep-alive efficiency.
Av. Request Transmit Time the average time to transmit the HTTP request header + (optionally) the HTTP request content data (form data or file upload data) to the web server, measured after the network connection was already established.
Av. Response Header Wait Time the average time for waiting for the first byte of the web server response (-header), measured since the request has (completely) transmitted to the web server.
Av. Response Header Receive Time the average time for receiving the remaining data of the HTTP response header, measured since the first byte of the response header was received.
Av. Response Content Receive Time the average time for receiving the response content data, for example HTML data or the data of a GIF image.
Average Response Time the average response time for this URL. This value is calculated as:\\.

URL Errors / Real-Time Profile of Error Types: : This diagram shows an overview about what kind of errors did occur for the URL at which time, measured since the load test was started. This "basic error information" is always "accurately" measured independently of the value chosen for the "Additional Sampling Rate per URL Call" - and captured in every case, also if no more memory is left to store full error snapshots.

Error Overview Diagrams (Real-Time)

Description

image

displays during the load test (at real-time) an overview about all occurred errors.

image

Failure Diagrams: : The first diagram shows an overview about what kind of errors did occur at which time, counted overall URLs and measured since the load test was started. This "basic error information" is always captured in every case, also if no more memory is left to store full error snapshots.

The succeeding diagrams which are shown per web page provide only information at which time errors did occur. The tables at the right side of the diagrams are showing the number of errors which did occur on the URLs of the web page. You can click on a error counter to show the error detail information (error snapshots) for the corresponding URL: : First Error Snapshots: : Displays a list about errors which did occur at first (at the start of the load test). By clicking on a magnifier icon the corresponding error detail information (error snapshot) is shown.   : Latest Error Snapshots: : Displays a list about the latest (newest) errors. By clicking on a magnifier icon the corresponding error detail information (error snapshot) is shown:

image

 

Input Fields
All failed URL Calls effects that all errors about failed URL calls are shown (non-fatal and fatal errors).
Session Failures only effects that only fatal errors about failed URL calls are shown (session failures).

Error Snapshot Memory: % used + : By clicking on the + (plus sign), you can increase the amount of memory available to store error snapshots. Please Note: when the memory is already 50% or more used, no additional error snapshots for non-fatal errors are captured. This means that increasing the memory may also re-enable capturing for non-fatal errors:

image

Statistical Overview Diagrams (Real-Time)

image

Description: displays statistical overview diagrams (at real-time) about a load test job.

Note:** the values shown in the diagrams are captured at regular intervals, depending on the **Statistic Sampling Interval**" which was selected when the load test was started.  ** : Diagrams

Concurrent Users The total number of simulated users.
Users Waiting For Response The number of users which are waiting for response from the web server.
Session Failures The number of failed sessions - which is the same as the number of fatal errors.
Session Time per User - per Loop The session time for one loop per simulated user. This value is the sum of "the response time of all URLs and all user's think times" per successful completed loop.
Web Transaction Rate The number of (successful) completed URL calls per second, measured overall simulated users.
Completed Loops per Minute The number of (successful) completed loops (sessions) per minute, measured overall simulated users.
TCP Socket Connect Time The time in milliseconds (per URL call) to open a new network connection to the web server.
Network Throughput The total network traffic which is generated by this load test job, measured in megabits per second.

Real-Time Comments

Description: supports to enter comments during the load test execution.

Real-time comments are notes or Tips, which you can enter during the load test execution:

image

These comments are later displayed inside all time-based diagrams of the load test result detail menu :

image

You can also modify, delete or add real-time comments before you generate the PDF report. However, all retroactively entered real-time comments are not permanently stored inside the result data.  

image

Loading the Statistics File

After the load test job has completed, the statistic results file is stored in the job directory of the local or remote Exec Agent. In order to access this results file, you must transfer it back to the (local) Project Navigator directory from which the load test program was started.

image

image

image

This menu shows all files of the load test job; however, only the statistics results file is usually needed, and this is already selected. The "*.out" file contains debug information, and the "*.err" file is either empty, or contains internal error messages from the load test program itself.

By clicking on the Acquire Selected Files button, all selected files are transferred (back) to the (local) Project Navigator directory.

If the checkbox Load *.prxres File on Analyze Load Test Menu is selected, the statistics results file is also loaded into the memory area of the Analyze Load Tests menu where the statistics and diagrams of the measured data can be shown, analyzed, and compared with results of previous test runs.

If you have specified that the load test program be executed by an Exec Agent Cluster , the load test program is transmitted to the local cluster job controller which coordinates all cluster members (Exec Agents). The cluster job controller creates a cluster job, and allocates a cluster job number. The cluster job is now in the state “configured” (ready to run, but not yet started).

image

The number of concurrent users will be automatically distributed across the cluster members, depending on the capability of the individual computer systems - called "load factor".

In cases where the load test program uses Input Files, you are asked - for each Input File - if you wish to split the content of the Input File. This can be useful, for example, if the Input File contains user accounts (usernames/passwords) but the web application does not allow duplicate logins. In this case, each cluster member must use different user accounts. By clicking on the corresponding magnifier icon, you can view how the Input File data would be distributed across the cluster members. If you do not use the split functionality, each cluster member would receive an entire copy of the Input File.

The distribution of users across the cluster members can also be modified manually; however, this is useful only if a cluster member is currently not available (marked with light red background), in which case the cluster job can not be started. In this case, you can assign the users of the unavailable cluster member to other cluster members, and then try to start the cluster job again. This redistribution may take a few seconds to complete.

Alternatively, the load test program can also scheduled to be executed at a predefined time. However, the local Job Controller process must be available (running) at the predefined time, because the scheduling entry for the cluster job is stored inside the Job Controller working directory which is monitored by the Job Controller itself. Especially, if you have started the Job Controller implicitly by using the ZebraTester Console you must keep the ZebraTester Console Window open in order that the cluster job will be started ¹. ¹ This restriction can be avoided by installing the local Job Controller as a Windows Service or as a Unix Daemon. 

After the cluster job has been scheduled you can leave this menu by closing the window and you can use later the Jobs menu to cancel or modify the schedule of this job.

Real-Time Cluster Job Statistics

The real-time statistics of a cluster job show the most important measured values, similar to the values which are shown in the Real Time Statistic of Exec Agent Jobs . The cluster job itself contains Exec Agent jobs which have been created by the local cluster job controller. By clicking on the magnifier icon of a cluster member, the real-time statistics of the corresponding Exec Agent job can be displayed in its own window.

image

If you want to abort the cluster job, you must do it at this level, as this will also abort all Exec Agent jobs. Aborting a single Exec Agent job will not interrupt the cluster job.

The same applies to the statistics result file (*.prxres), which must be accessed at this level.

Loading the Statistics File of Cluster Jobs

The statistics result file of a cluster job contains the consolidated (merged) measurements for all cluster members. The calculations for merging the results are extensive; therefore, it may take up to 60 seconds for the result file to be shown. The individual measurements of the Exec Agents are embedded separately inside the same consolidated result file.

image

The consolidated statistics result file is marked with a blue background and is already selected for you.

By clicking on the magnifier icon, you have access to the "*.out" and "*.err" files of the corresponding Exec Agent jobs.

image

Usually, you would work inside the Analyze Load Tests menu with the consolidated measurement results only. However, it is also possible to expand the measurement results to access the results of each individual Exec Agent job:

image

This feature can be used to check if all cluster members have measured approximately the same response times; however, variations in a range of ± 20% or more may be normal:

image

image

image

image

All load test programs which are started from the Project Navigator are always executed as "batch jobs" by an (external) Exec Agent process or by an Exec Agent Cluster. This means, that it is not required to wait for the completion of a load test program on the “Execute Load Test” window: you can close the "Execute Load Test" window at any time and you can check later the result, or the actual effort, of all load test jobs by using this menu.

If a load test job has completed you are disposed to acquire the corresponding statistic result file (*.prxres). If a load test job is still running you are disposed to the temporary live-statistic window of the job.

Input Fields
Display Cluster Jobs shows all Exec Agent Cluster jobs.
Display Exec Agent Jobs of allows to select the Exec Agent from which a list of all load test jobs is displayed.
Clean Up: Delete All Non-Running Jobs deletes all jobs except running and scheduled jobs.\\.
Clean Up: Delete Old Completed Jobs deletes all completed jobs except the newest one. This button is only shown if at minimum two jobs have been completed.

Columns of the job list:

Item Description
 Job  Each job has its unique ID which was automatically assigned when the job was defined. However the ID is only unique per Exec Agent. Cluster jobs have an own, separate ID (own enumeration counter).
   Allows to acquire the statistic result file (.prxres) of an already completed load test job - or reconnects to the temporary statistic of the load test job if the job is still running – or allows to cancel the schedule of the job.
   Deletes all data (-files) of a completed load test job. Take into consideration that you must first acquire the statistic result file (*.prxres) of the job before you delete all files of a job - otherwise the result data of the job are lost.
 Date  Displays the date and time when the job has been defined or when the job has been completed, or - for scheduled jobs - the planned point in time when the job will be started.
 State  Displays the current job state: configured (ready to run), scheduled, running or completed. The state "???" means that the job data are corrupted - you should delete all jobs which have the state "???" because they delay the display of all jobs in this list.
 Load Test Program & Arguments  Displays the name of the load test program and the arguments of the load test program.
 Released from GUI (IP)  Displays the TCP/IP address (remote computer) from which the job has been initiated.

Load Test Program Arguments

Argument / Parameter Meaning
u <number> Number of concurrent users
d <seconds> Planned test duration in seconds. 0 = unlimited
t <seconds> Request timeout per URL call in seconds
sdelay <milliseconds> Startup delay between creating concurrent users in milliseconds
maxloops <number> Max. number of loops (repetitions of web surfing session) per user. 0 = unlimited
downlink <Kbps> Network bandwidth limitation per concurrent user in kilobits per second for the downlink (web server to web browser)
uplink <Kbps> Network bandwidth limitation per concurrent user in kilobits per second for the uplink (web browser to web server)
sampling <seconds> Statistical sampling interval in seconds (interval-based sampling). Used for time-based overall diagrams like for example the measured network throughput
percpage <percent> Additional sampling rate in percent for response times of web pages (event based sampling, each time when a web page is called)
percurl <percent> Additional sampling rate in percent for response times of URL calls (event based sampling, each time when a URL is called)
maxerrsnap <number> Max. number of error snapshots per URL (per Exec Agent), 0 = unlimited
maxerrmem <megabytes> Max. memory in megabytes which can be used to store error snapshots, -1 = unlimited
setuseragent "<text>" Replaces the recorded value of the HTTP request header field User-Agent with a new value. The new value is applied for all executed URL calls.
nostdoutlog Disables writing any date to the *.out file of the load test job. Note that the *.out file is nevertheless created but contains zero bytes.
dfl Debug failed loops
dl Debug loops
dh Debug headers &amp; loops
dc Debug content &amp; loops
dC Debug cookies &amp; loops
dK Debug keep-alive for re-used network connections &amp; loops
dssl Debug information about the SSL protocol and the SSL handshake &amp; loops
multihomed Forces the Exec Agent(s) to use multiple client IP addresses
ipperloop Using this option in combination with the option -multihomed effects that an own local IP address is used for each executed loop rather than for each simulated user. This option is considered only if also the option -multihomed is used.
ssl <version> Use fixed SSL protocol version: v3, TLS, TLS11 or TLS12
sslcache <seconds> Timeout of SSL cache in seconds. 0 = cache disabled
nosni Disable support for TLS server name indication (SNI)
dnshosts <file-name> Effects that the load test job uses an own DNS hosts file to resolve host names - rather than using the hosts file of the underlying operating system.Note that you have to ZIP the hosts file together with the compiled class of the load test program. To automate the ZIP it's recommended to declare the hosts file as an external resource (w/o adding it to the CLASSPATH).
dnssrv <IP-name-server-1>[,<IP-name-server-N>] Effects that the load test job uses specific (own) DNS server(s) to resolve host names – rather than using the DNS library of the underlying operating system.
dnsenattl Enable consideration of DNS TTL by using the received TTL-values from the DNS server(s). This option cannot be used in combination with the option -dnsperloop.
dnsfixttl <seconds> Enable DNS TTL by using a fixed TTL-value of seconds for all DNS resolves. This option cannot be used in combination with the option -dnsperloop.
dnsperloop Perform new DNS resolves for each executed loop. All resolves are stable within the same loop (no consideration of DNS TTL within a loop). This option cannot be used in combination with the options -dnsenattl or -dnsfixttl.
dnsstatistic Effects that statistical data about DNS resolutions are measured and displayed in the load test result, by using an own DNS stack on the load generators. Note: There is no need to use this option if any other, more specific DNS option is enabled because all (other) DNS options also effect implicitly that statistical data about DNS resolutions are measured. If you use this option without any other DNS option, the (own) DNS stack on the load generators will communicate with the default configured DNS servers of the operating system - but without considering the "hosts" file.
tz <value> Time zone (see Application Reference Manual)
annotation <text> Comment about the test-run

Several load test jobs can be started from the GUI at the same time. However, the GUI does not have the ability to automatically run sequences of load test jobs, synchronize load test jobs, or automatically start several jobs, with a single mouse click.

To perform these kinds of activities, you must program load test job scripts which are written in the “natural” scripting language of your operating system (Windows: *.bat files, Unix: *.sh, *.ksh, *.csh … files). Inside these scripts, the PrxJob utility is used as the interface to the ZebraTester system. When the Windows version of ZebraTester is installed, the installation kit creates the directory ScriptExamples within the Project Navigator, and this directory contains some example scripts.

The PrxJob utility allows you to start load test jobs on the local as well as on a remote system. It also provides the capability to create cluster jobs, to synchronize jobs, to obtain the current state of jobs, and to acquire the statistics result files of jobs. More information about the PrxJob utility can be found in the Application Reference Manual, Chapter 4.

Every time when a load test is started, an additional job definition template file is stored in the actual Project Navigator directory (in XML format). Such a job definition template file contain all configuration date which are needed to rerun the same load test job again. If you click the corresponding button of a job definition template file in Project Navigator, the load test job inclusive all of its input parameter is automatically transferred to the Exec Agent or to the Exec Agent Cluster and immediately ready-to-run.

image

Additionally, if you wish to trigger several load test jobs at the same time to be ready-to-run (by using only one mouse click), you can zip several templates to one zip archive. After this click the corresponding button of the zip archive:

image

XML Load Test Template Attributes

Attribute Name Description
loadTestProgramPath Absolute file path to compiled load test program (*.class) or load test program ZIP archive
startFromExecAgentName Name of the Exec Agent on which the load test is started (empty value if cluster job)
startFromClusterName Name of the Exec Agent Cluster on which the load test is started (empty value if no cluster job)
concurrentUsers Number of concurrent users
testDuration Planned test duration in seconds (0 = unlimited)
loopsPerUser Number of planned loops per user (0 = unlimited)
startupDelayPerUser Startup delay per user in milliseconds
downlinkBandwidth Downlink bandwidth per user in kilobits per second (0 = unlimited)
uplinkBandwidth Uplink bandwidth per user in kilobits per second (0 = unlimited)
requestTimeout Request timeout per URL call in seconds
maxErrorSnapshots Limits the number of error snapshots taken during load test execution (0 = unlimited). Negative value: maximum memory in megabytes used to store all error snapshots, counted overall Exec Agents (recommended). Positive value: maximum number of error snapshots per URL, per Exec Agent (not recommended).
statisticSamplingInterval Statistic sampling interval in seconds
percentilePageSamplingPercent Additional sampling rate per Web page in percent (0..100)
percentileUrlSamplingPercent Additional sampling rate per URL call in percent (0..100)
percentileUrlSamplingPercentAddOption Additional URL sampling options per executed URL call (numeric value):<br>0: no options<br>1: all URL performance details (network connect time, request transmit time, …)2: request header3: request content (form data)4: request header &amp; request content5: response header6: response header &amp; response content7: all – but without response content8: all – full URL snapshot
debugOptions Debug options: (string value)-dl”: debug loops (including var handler)-dh”: debug headers &amp; loops-dc”: debug content &amp; loops-dC”: debug cookies &amp; loops-dK”: debug keep-alive &amp; loops-dssl”: debug SSL handshake &amp; loops
additionalOptions Additional options (string)
sslOptions SSL/HTTPS options: (string value)<br>“all”: automatic SSL protocol detection (TLS preferred)tls”: SSL protocol fixed to TLSv3”: SSL protocol fixed to v3v2”: SSL protocol fixed to V2
testRunAnnotation Annotation for this test-run (string)
userInputFields Label, variable name and default value of User Input Fields