Tutorials
So, you’ve installed Yandex.Tank to a proper machine, it is close to target, access is permitted and server is tuned. How to make a test?
Note
This guide is for phantom
load generator.
Create a file on a server with Yandex.Tank: load.yaml
phantom:
address: 203.0.113.1:80 # [Target's address]:[target's port]
uris:
- /
load_profile:
load_type: rps # schedule load by defining requests per second
schedule: line(1, 10, 10m) # starting from 1rps growing linearly to 10rps during 10 minutes
console:
enabled: true # enable console output
telegraf:
enabled: false # let's disable telegraf monitoring for the first time
And run:
$ yandex-tank -c load.yaml
phantom
have 3 primitives for describing load scheme:
step (a,b,step,dur)
makes stepped load, where a,b are start/end load values, step - increment value, dur - step duration.
- Examples:
step(25, 5, 5, 60)
- stepped load from 25 to 5 rps, with 5 rps steps, step duration 60s.
step(5, 25, 5, 60)
- stepped load from 5 to 25 rps, with 5 rps steps, step duration 60s
line (a,b,dur)
makes linear load, wherea,b
are start/end load,dur
- the time for linear load increase from a to b.
- Examples:
line(10, 1, 10m)
- linear load from 10 to 1 rps, duration - 10 minutes
line(1, 10, 10m)
- linear load from 1 to 10 rps, duration - 10 minutes
const (load,dur)
makes constant load.load
- rps amount,dur
- load duration.
- Examples:
const(10,10m)
- constant load for 10 rps for 10 minutes.
const(0, 10)
- 0 rps for 10 seconds, in fact 10s pause in a test.
Note
- You can set fractional load like this:
line(1.1, 2.5, 10)
- from 1.1rps to 2.5 for 10 seconds.
Note
step
and line
could be used with increasing and decreasing intensity:
You can specify complex load schemes using those primitives.
- Example:
schedule: line(1, 10, 10m) const(10,10m)
linear load from 1 to 10rps during 10 minutes, then 10 minutes of 10rps constant load.
Time duration could be defined in seconds, minutes (m) and hours (h).
For example: 27h103m645
For a test with constant load at 10rps for 10 minutes, load.yaml
should
have following lines:
phantom:
address: 203.0.113.1:80 # [Target's address]:[target's port]
uris:
- /uri1
- /uri2
load_profile:
load_type: rps # schedule load by defining requests per second
schedule: const(10, 10m) # starting from 1rps growing linearly to 10rps during 10 minutes
console:
enabled: true # enable console output
telegraf:
enabled: false # let's disable telegraf monitoring for the first time
Preparing requests
- There are several ways to set up requests:
Access mode
URI-style
URI+POST
request-style.
Note
Request-style is default ammo type.
Note
Regardless of the chosen format, resulted file with requests could be gzipped - tank supports archived ammo files.
To specify external ammo file use ammofile
option.
Note
You can specify URL to ammofile, http(s). Small ammofiles (~<100MB) will be downloaded as is,
to directory /tmp/<hash>
, large files will be read from stream.
Note
If ammo type is uri-style or request-style, tank will try to guess it.
Use ammo_type
option to explicitly specify ammo format. Don’t forget to change ammo_type
option
if you switch format of your ammo, otherwise you might get errors.
Example:
phantom:
address: 203.0.113.1:80
ammofile: https://yourhost.tld/path/to/ammofile.txt
URI-style, URIs in load.yaml
YAML-file configuration: Don’t specify ammo_type
explicitly for this type of ammo.
Update configuration file with HTTP headers and URIs:
phantom:
address: 203.0.113.1:80
load_profile:
load_type: rps
schedule: line(1, 10, 10m)
header_http: "1.1"
headers:
- "[Host: www.target.example.com]"
- "[Connection: close]"
uris:
- "/uri1"
- "/buy"
- "/sdfg?sdf=rwerf"
- "/sdfbv/swdfvs/ssfsf"
console:
enabled: true
telegraf:
enabled: false
Parameter uris
contains uri, which should be used for requests generation.
Note
Pay attention to the sample above, because whitespaces in multiline uris
and headers
options are important.
URI-style, URIs in file
YAML-file configuration: ammo_type: uri
Create a file with declared requests: ammo.txt
[Connection: close]
[Host: target.example.com]
[Cookie: None]
/?drg tag1
/
/buy tag2
[Cookie: test]
/buy/?rt=0&station_to=7&station_from=9
File consists of list of URIs and headers to be added to every request defined below.
Every URI must begin from a new line, with leading /
.
Each line that begins from [
is considered as a header.
Headers could be (re)defined in the middle of URIs, as in sample above.
- Example:
Request
/buy/?rt=0&station_to=7&station_from=9
will be sent withCookie: test
, notCookie: None
.
Request may be marked by tag, you can specify it with whitespace following URI.
URI+POST-style
YAML-file configuration: ammo_type: uripost
Create a file with declared requests: ammo.txt
[Host: example.org]
[Connection: close]
[User-Agent: Tank]
5 /route/?rll=50.262025%2C53.276083~50.056015%2C53.495561&origin=1&simplify=1
class
10 /route/?rll=50.262025%2C53.276083~50.056015%2C53.495561&origin=1&simplify=1
hello!clas
7 /route/?rll=37.565147%2C55.695758~37.412796%2C55.691454&origin=1&simplify=1
uripost
File begins with optional lines […], that contain headers which will be added to every request. After that section there is a list of URIs and POST bodies. Each URI line begins with a number which is the size of the following POST body.
Request-style
YAML-file configuration: ammo_type: phantom
Full requests listed in a separate file. For more complex requests, like POST, you’ll have to create a special file. File format is:
[size_of_request] [tag]\n
[request_headers]
[body_of_request]\r\n
[size_of_request2] [tag2]\n
[request2_headers]
[body_of_request2]\r\n
where size_of_request
– request size in bytes. ‘rn’ symbols after
body
are ignored and not sent anywhere, but it is required to
include them in a file after each request. Pay attention to the sample above
because ‘r’ symbols are strictly required.
Note
Parameter ammo_type
is unnecessary, request-style is default ammo type.
sample GET requests (null body)
73 good
GET / HTTP/1.0
Host: xxx.tanks.example.com
User-Agent: xxx (shell 1)
77 bad
GET /abra HTTP/1.0
Host: xxx.tanks.example.com
User-Agent: xxx (shell 1)
78 unknown
GET /ab ra HTTP/1.0
Host: xxx.tanks.example.com
User-Agent: xxx (shell 1)
sample POST requests (binary data)
904
POST /upload/2 HTTP/1.0
Content-Length: 801
Host: xxxxxxxxx.dev.example.com
User-Agent: xxx (shell 1)
^.^........W.j^1^.^.^.²..^^.i.^B.P..-!(.l/Y..V^. ...L?...S'NR.^^vm...3Gg@s...d'.\^.5N.$NF^,.Z^.aTE^.
._.[..k#L^ƨ`\RE.J.<.!,.q5.F^՚iΔĬq..^6..P..тH.`..i2
.".uuzs^^F2...Rh.&.U.^^..J.P@.A......x..lǝy^?.u.p{4..g...m.,..R^.^.^......].^^.^J...p.ifTF0<.s.9V.o5<..%!6ļS.ƐǢ..㱋....C^&.....^.^y...v]^YT.1.#K.ibc...^.26... ..7.
b.$...j6.٨f...W.R7.^1.3....K`%.&^..d..{{ l0..^\..^X.g.^.r.(!.^^...4.1.$\ .%.8$(.n&..^^q.,.Q..^.D^.].^.R9.kE.^.$^.I..<..B^..^.h^^C.^E.|....3o^.@..Z.^.s.$[v.
527
POST /upload/3 HTTP/1.0
Content-Length: 424
Host: xxxxxxxxx.dev.example.com
User-Agent: xxx (shell 1)
^.^........QMO.0^.++^zJw.ر^$^.^Ѣ.^V.J....vM.8r&.T+...{@pk%~C.G../z顲^.7....l...-.^W"cR..... .&^?u.U^^.^.....{^.^..8.^.^.I.EĂ.p...'^.3.Tq..@R8....RAiBU..1.Bd*".7+.
.Ol.j=^.3..n....wp..,Wg.y^.T..~^..
sample POST multipart:
533
POST /updateShopStatus? HTTP/1.0
User-Agent: xxx/1.2.3
Host: xxxxxxxxx.dev.example.com
Keep-Alive: 300
Content-Type: multipart/form-data; boundary=AGHTUNG
Content-Length:334
Connection: Close
--AGHTUNG
Content-Disposition: form-data; name="host"
load-test-shop-updatestatus.ru
--AGHTUNG
Content-Disposition: form-data; name="user_id"
1
--AGHTUNG
Content-Disposition: form-data; name="wsw-fields"
<wsw-fields><wsw-field name="moderate-code"><wsw-value>disable</wsw-value></wsw-field></wsw-fields>
--AGHTUNG--
sample ammo generators you may find on the Ammo generators page.
Run Test!
Request specs in load.yaml – run as
yandex-tank -c load.yaml
Request specs in ammo.txt – run as
yandex-tank -c load.yaml ammo.txt
Yandex.Tank detects requests format and generates ultimate requests versions.
yandex-tank
here is an executable file name of Yandex.Tank.
If Yandex.Tank has been installed properly and configuration file is correct, the load will be given in next few seconds.
Results
During test execution you’ll see HTTP and net errors, answer times
distribution, progressbar and other interesting data. At the same time
file phout.txt
is being written, which could be analyzed later.
If you need more human-readable report, you can try Report plugin, You can found it here
If you need to upload results to an external storage, such as Graphite or InfluxDB, you can use one of existing artifacts uploading modules Modules
SSL
To activate SSL add phantom: {ssl: true}
to load.yaml
.
Now, our basic config looks like that:
phantom:
address: 203.0.113.1:443
load_profile:
load_type: rps
schedule: line(1, 10, 10m)
ssl: true
Note
Do not forget to specify ssl port to address. Otherwise, you might get ‘protocol errors’.
Autostop
Autostop is an ability to automatically halt test execution if some conditions are reached.
HTTP and Net codes conditions
There is an option to define specific codes (404,503,100) as well as code groups (3xx, 5xx, xx). Also you can define relative threshold (percent from the whole amount of answer per second) or absolute (amount of answers with specified code per second).
Examples:
autostop: http(4xx,25%,10)
– stop test, if amount of 4xx http codes in every second of last 10s period exceeds 25% of answers (relative threshold).
autostop: net(101,25,10)
– stop test, if amount of 101 net-codes in every second of last 10s period is more than 25 (absolute threshold).
autostop: net(xx,25,10)
– stop test, if amount of non-zero net-codes in every second of last 10s period is more than 25 (absolute threshold).
Average time conditions
- Example:
autostop: time(1500,15)
– stops test, if average answer time exceeds 1500ms.
So, if we want to stop test when all answers in 1 second period are 5xx plus some network and timing factors - add autostop line to load.yaml:
phantom:
address: 203.0.113.1:80
load_profile:
load_type: rps
schedule: line(1, 10, 10m)
autostop:
autostop:
- time(1s,10s)
- http(5xx,100%,1s)
- net(xx,1,30)
Logging
Looking into target’s answers is quite useful in debugging. For doing
that use parameter writelog, e.g. add phantom: {writelog: all}
to load.yaml
to log all messages.
Note
Writing answers on high load leads to intensive disk i/o usage and can affect test accuracy.**
Log format:
<metrics>
<body_request>
<body_answer>
Where metrics are:
size_in size_out response_time(interval_real) interval_event net_code
(request size, answer size, response time, time to wait for response
from the server, answer network code)
Example:
user@tank:~$ head answ_*.txt
553 572 8056 8043 0
GET /create-issue HTTP/1.1
Host: target.yandex.net
User-Agent: tank
Accept: */*
Connection: close
HTTP/1.1 200 OK
Content-Type: application/javascript;charset=UTF-8
For load.yaml
like this:
phantom:
address: 203.0.113.1:80
load_profile:
load_type: rps
schedule: line(1, 10, 10m)
writelog: all
autostop:
autostop:
- time(1,10)
- http(5xx,100%,1s)
- net(xx,1,30)
Results in phout
phout.txt - is a per-request log. It could be used for service behaviour
analysis (Excel/gnuplot/etc) It has following fields:
time, tag, interval_real, connect_time, send_time, latency, receive_time, interval_event, size_out, size_in, net_code proto_code
Phout example:
1326453006.582 1510 934 52 384 140 1249 37 478 0 404
1326453006.582 others 1301 674 58 499 70 1116 37 478 0 404
1326453006.587 heavy 377 76 33 178 90 180 37 478 0 404
1326453006.587 294 47 27 146 74 147 37 478 0 404
1326453006.588 345 75 29 166 75 169 37 478 0 404
1326453006.590 276 72 28 119 57 121 53 476 0 404
1326453006.593 255 62 27 131 35 134 37 478 0 404
1326453006.594 304 50 30 147 77 149 37 478 0 404
1326453006.596 317 53 33 158 73 161 37 478 0 404
1326453006.598 257 58 32 106 61 110 37 478 0 404
1326453006.602 315 59 27 160 69 161 37 478 0 404
1326453006.603 256 59 33 107 57 110 53 476 0 404
1326453006.605 241 53 26 130 32 131 37 478 0 404
Note
contents of phout depends on phantom version installed on your Yandex.Tank system.
net codes are system codes from errno.h, on most Debian-based systems those are:
1 EPERM Operation not permitted
2 ENOENT No such file or directory
3 ESRCH No such process
4 EINTR Interrupted system call
5 EIO Input/output error
6 ENXIO No such device or address
7 E2BIG Argument list too long
8 ENOEXEC Exec format error
9 EBADF Bad file descriptor
10 ECHILD No child processes
11 EAGAIN Resource temporarily unavailable
12 ENOMEM Cannot allocate memory
13 EACCES Permission denied
14 EFAULT Bad address
15 ENOTBLK Block device required
16 EBUSY Device or resource busy
17 EEXIST File exists
18 EXDEV Invalid cross-device link
19 ENODEV No such device
20 ENOTDIR Not a directory
21 EISDIR Is a directory
22 EINVAL Invalid argument
23 ENFILE Too many open files in system
24 EMFILE Too many open files
25 ENOTTY Inappropriate ioctl for device
26 ETXTBSY Text file busy
27 EFBIG File too large
28 ENOSPC No space left on device
29 ESPIPE Illegal seek
30 EROFS Read-only file system
31 EMLINK Too many links
32 EPIPE Broken pipe
33 EDOM Numerical argument out of domain
34 ERANGE Numerical result out of range
35 EDEADLOCK Resource deadlock avoided
36 ENAMETOOLONG File name too long
37 ENOLCK No locks available
38 ENOSYS Function not implemented
39 ENOTEMPTY Directory not empty
40 ELOOP Too many levels of symbolic links
42 ENOMSG No message of desired type
43 EIDRM Identifier removed
44 ECHRNG Channel number out of range
45 EL2NSYNC Level 2 not synchronized
46 EL3HLT Level 3 halted
47 EL3RST Level 3 reset
48 ELNRNG Link number out of range
49 EUNATCH Protocol driver not attached
50 ENOCSI No CSI structure available
51 EL2HLT Level 2 halted
52 EBADE Invalid exchange
53 EBADR Invalid request descriptor
54 EXFULL Exchange full
55 ENOANO No anode
56 EBADRQC Invalid request code
57 EBADSLT Invalid slot
59 EBFONT Bad font file format
60 ENOSTR Device not a stream
61 ENODATA No data available
62 ETIME Timer expired
63 ENOSR Out of streams resources
64 ENONET Machine is not on the network
65 ENOPKG Package not installed
66 EREMOTE Object is remote
67 ENOLINK Link has been severed
68 EADV Advertise error
69 ESRMNT Srmount error
70 ECOMM Communication error on send
71 EPROTO Protocol error
72 EMULTIHOP Multihop attempted
73 EDOTDOT RFS specific error
74 EBADMSG Bad message
75 EOVERFLOW Value too large for defined data type
76 ENOTUNIQ Name not unique on network
77 EBADFD File descriptor in bad state
78 EREMCHG Remote address changed
79 ELIBACC Can not access a needed shared library
80 ELIBBAD Accessing a corrupted shared library
81 ELIBSCN .lib section in a.out corrupted
82 ELIBMAX Attempting to link in too many shared libraries
83 ELIBEXEC Cannot exec a shared library directly
84 EILSEQ Invalid or incomplete multibyte or wide character
85 ERESTART Interrupted system call should be restarted
86 ESTRPIPE Streams pipe error
87 EUSERS Too many users
88 ENOTSOCK Socket operation on non-socket
89 EDESTADDRREQ Destination address required
90 EMSGSIZE Message too long
91 EPROTOTYPE Protocol wrong type for socket
92 ENOPROTOOPT Protocol not available
93 EPROTONOSUPPORT Protocol not supported
94 ESOCKTNOSUPPORT Socket type not supported
95 ENOTSUP Operation not supported
96 EPFNOSUPPORT Protocol family not supported
97 EAFNOSUPPORT Address family not supported by protocol
98 EADDRINUSE Address already in use
99 EADDRNOTAVAIL Cannot assign requested address
100 ENETDOWN Network is down
101 ENETUNREACH Network is unreachable
102 ENETRESET Network dropped connection on reset
103 ECONNABORTED Software caused connection abort
104 ECONNRESET Connection reset by peer
105 ENOBUFS No buffer space available
106 EISCONN Transport endpoint is already connected
107 ENOTCONN Transport endpoint is not connected
108 ESHUTDOWN Cannot send after transport endpoint shutdown
109 ETOOMANYREFS Too many references: cannot splice
110 ETIMEDOUT Connection timed out
111 ECONNREFUSED Connection refused
112 EHOSTDOWN Host is down
113 EHOSTUNREACH No route to host
114 EALREADY Operation already in progress
115 EINPROGRESS Operation now in progress
116 ESTALE Stale file handle
117 EUCLEAN Structure needs cleaning
118 ENOTNAM Not a XENIX named type file
119 ENAVAIL No XENIX semaphores available
120 EISNAM Is a named type file
121 EREMOTEIO Remote I/O error
122 EDQUOT Disk quota exceeded
Graph and statistics
Use Report plugin OR use your favorite stats packet, R, for example.
Thread limit
instances: N
in load.yaml
limits number of simultanious
connections (threads).
Example with 10 threads limit:
phantom:
address: 203.0.113.1:80
load_profile:
load_type: rps
schedule: line(1, 10, 10m)
instances: 10
Dynamic thread limit
You can specify load_type: instances
instead of ‘rps’ to schedule a number of active instances
which generate as much rps as they manage to.
Bear in mind that active instances number cannot be decreased
and final number of them must be equal to instances
parameter value.
Example:
phantom:
address: 203.0.113.1:80
load_profile:
load_type: instances
schedule: line(1,10,10m)
instances: 10
loop: 10000 # don't stop when the end of ammo is reached but loop it 10000 times
Note
When using load_type: instances
you should specify how many loops of
ammo you want to generate because tank can’t find out from the schedule
how many ammo do you need
Custom stateless protocol
In necessity of testing stateless HTTP-like protocol, Yandex.Tank’s HTTP parser could be switched off, providing ability to generate load with any data, receiving any answer in return. To do that use tank_type parameter:
phantom:
address: 203.0.113.1:80
load_profile:
load_type: rps
schedule: line(1, 10, 10m)
instances: 10
tank_type: none
Note
Indispensable condition: Connection close must be initiated by remote side
Gatling
If server with Yandex.Tank have several IPs, they may be
used to avoid outcome port shortage. Use gatling_ip
parameter for
that. load.yaml:
phantom:
address: 203.0.113.1:80
load_profile:
load_type: rps
schedule: line(1, 10, 10m)
instances: 10
gatling_ip: IP1 IP2