Φορτίο (fortio) is Istio's load testing tool. Fortio runs at a specified query per second (qps) and records an histogram of execution time and calculates percentiles (e.g. p99 ie the response time such as 99% of the requests take less than that number (in seconds, SI unit)). It can run for a set duration, for a fixed number of calls, or until interrupted (at a constant target QPS, or max speed/load per connection/thread).
The name fortio comes from greek φορτίο which is load/burden.
Fortio is a reusable, embeddable go library as well as a command line tool and server process, the server includes a simple web UI and graphical representation of the results (both a single latency graph and a multiple results comparative min, max, avg and percentiles graphs).
- Install go (golang 1.8 or later)
go get -u istio.io/fortio- you can now run
fortio(from your gopath bin/ directory)
Or use docker, for instance:
docker run -p 8080:8080 -p 8079:8079 istio/fortio server & # For the server
docker run istio/fortio load http://www.google.com/ # For a test runOr download the binary distribution, for instance:
curl -L https://github.com/istio/fortio/releases/download/0.4.0/fortio-linux_x64-0.4.0.tgz \
| sudo tar -C / -xvzpf -Once fortio server is running, you can visit its web UI at http://localhost:8080/fortio/
You can get a preview of the reporting/graphing UI at https://fortio.istio.io/
Fortio can be an http or grpc load generator, gathering statistics using the load subcommand, or start simple http and grpc ping servers, as well as a basic web UI, result graphing and https redirector, with the server command or issue grpc ping messages using the grpcping command. It can also fetch a single URL's for debugging when using the curl command (or the -curl flag to the load command). You can run just the redirector with redirect. Lastly if you saved JSON results (using the web UI or directly from the command line), you can browse and graph those results using the report command.
$ fortio
Φορτίο 0.6.7 usage:
fortio command [flags] target
where command is one of: load (load testing), server (starts grpc ping and
http echo/ui/redirect servers), grpcping (grpc client), report (report only UI
server), redirect (redirect only server), or curl (single URL debug).
where target is a url (http load tests) or host:port (grpc health test)
and flags are:
-H value
Additional Header(s)
-a Automatically save JSON result with filename based on labels & timestamp
-allow-initial-errors
Allow and don't abort on initial warmup errors
-c int
Number of connections/goroutine/threads (default 4)
-compression
Enable http compression
-curl
Just fetch the content once
-data-dir string
Directory where JSON results are stored/read (default ".")
-echo-debug-path string
http echo server URI for debug, empty turns off that part (more secure)
(default "/debug")
-gomaxprocs int
Setting for runtime.GOMAXPROCS, <1 doesn't change the default
-grpc
Use GRPC (health check) for load testing
-grpc-port int
grpc port (default 8079)
-halfclose
When not keepalive, whether to half close the connection (only for fast
http)
-health
client mode: use health instead of ping
-healthservice string
which service string to pass to health check
-http-port int
http echo server port (default 8080)
-http1.0
Use http1.0 (instead of http 1.1)
-httpbufferkb int
Size of the buffer (max data size) for the optimized http client in kbytes
(default 128)
-httpccch
Check for Connection: Close Header
-httpreqtimeout duration
Http request timeout value (default 15s)
-json string
Json output to provided file or '-' for stdout (empty = no json output,
unless -a is used)
-keepalive
Keep connection alive (only for fast http 1.1) (default true)
-labels string
Additional config data/labels to add to the resulting JSON, defaults to
target URL and hostname
-logcaller
Logs filename and line number of callers to log (default true)
-loglevel value
loglevel, one of [Debug Verbose Info Warning Error Critical Fatal]
(default Info)
-logprefix string
Prefix to log lines before logged messages (default "> ")
-n int
Run for exactly this number of calls instead of duration. Default (0) is
to use duration (-t). Default is 1 when used as grpc ping count.
-p string
List of pXX to calculate (default "50,75,99,99.9")
-payload string
Payload string to send along
-profile string
write .cpu and .mem profiles to file
-qps float
Queries Per Seconds or 0 for no wait/max qps (default 8)
-quiet
Quiet mode: sets the loglevel to Error and reduces the output.
-r float
Resolution of the histogram lowest buckets in seconds (default 0.001)
-redirect-port int
Redirect all incoming traffic to https URL (need ingress to work properly)
-1 means off. (default 8081)
-static-dir string
Absolute path to the dir containing the static files dir
-stdclient
Use the slower net/http standard client (works for TLS)
-sync string
index.tsv or s3/gcs bucket xml URL to fetch at startup for server modes.
-t duration
How long to run the test or 0 to run until ^C (default 5s)
-ui-path string
http server URI for UI, empty turns off that part (more secure)
(default "/fortio/")
- Start the internal servers:
$ fortio server &
Https redirector running on :8081
UI starting - visit:
http://localhost:8080/fortio/
Fortio 0.6.1 grpc ping server listening on port 8079
Fortio 0.6.1 echo server listening on port 8080
- Simple grpc ping:
$ fortio grpcping localhost
02:29:27 I pingsrv.go:116> Ping RTT 305334 (avg of 342970, 293515, 279517 ns) clock skew -2137
Clock skew histogram usec : count 1 avg -2.137 +/- 0 min -2.137 max -2.137 sum -2.137
# range, mid point, percentile, count
>= -4 < -2 , -3 , 100.00, 1
# target 50% -2.137
RTT histogram usec : count 3 avg 305.334 +/- 27.22 min 279.517 max 342.97 sum 916.002
# range, mid point, percentile, count
>= 250 < 300 , 275 , 66.67, 2
>= 300 < 350 , 325 , 100.00, 1
# target 50% 294.879
- Load (low default qps/threading) test:
$ fortio load http://www.google.com
Fortio 0.3.6 running at 8 queries per second, 8->8 procs, for 5s: http://www.google.com
19:10:33 I httprunner.go:84> Starting http test for http://www.google.com with 4 threads at 8.0 qps
Starting at 8 qps with 4 thread(s) [gomax 8] for 5s : 10 calls each (total 40)
19:10:39 I periodic.go:314> T002 ended after 5.056753279s : 10 calls. qps=1.9775534712220633
19:10:39 I periodic.go:314> T001 ended after 5.058085991s : 10 calls. qps=1.9770324224999916
19:10:39 I periodic.go:314> T000 ended after 5.058796046s : 10 calls. qps=1.9767549252963101
19:10:39 I periodic.go:314> T003 ended after 5.059557593s : 10 calls. qps=1.9764573910247019
Ended after 5.059691387s : 40 calls. qps=7.9056
Sleep times : count 36 avg 0.49175757 +/- 0.007217 min 0.463508712 max 0.502087879 sum 17.7032725
Aggregated Function Time : count 40 avg 0.060587641 +/- 0.006564 min 0.052549016 max 0.089893269 sum 2.42350566
# range, mid point, percentile, count
>= 0.052549 < 0.06 , 0.0562745 , 47.50, 19
>= 0.06 < 0.07 , 0.065 , 92.50, 18
>= 0.07 < 0.08 , 0.075 , 97.50, 2
>= 0.08 <= 0.0898933 , 0.0849466 , 100.00, 1
# target 50% 0.0605556
# target 75% 0.0661111
# target 99% 0.085936
# target 99.9% 0.0894975
Code 200 : 40
Response Header Sizes : count 40 avg 690.475 +/- 15.77 min 592 max 693 sum 27619
Response Body/Total Sizes : count 40 avg 12565.2 +/- 301.9 min 12319 max 13665 sum 502608
All done 40 calls (plus 4 warmup) 60.588 ms avg, 7.9 qps
- Curl like (single request) mode
$ fortio load -curl -H Foo:Bar http://localhost:8080/debug
14:26:26 I http.go:133> Setting regular extra header Foo: Bar
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Date: Mon, 08 Jan 2018 22:26:26 GMT
Content-Length: 230
Φορτίο version 0.6.0 echo debug server up for 39s on ldemailly-macbookpro - request from [::1]:65055
GET /debug HTTP/1.1
headers:
Host: localhost:8080
User-Agent: istio/fortio-0.6.0
Foo: Bar
body:
- Report only UI
If you have json files saved from running the full UI, you can serve just the reports:
$ fortio report
Browse only UI starting - visit:
http://localhost:8080/
Https redirector running on :8081
Fortio server - has the following feature - http listening on 8080 (all paths and ports are configurable through flags above):
- A simple echo server which will echo back posted data (for any path not mentioned below).
For instance
curl -d abcdef http://localhost:8080/returnsabcdefback. It supports the following optional query argument parameters:
| Parameter | Usage, example |
|---|---|
| delay | duration to delay the response by. Can be a single value or a coma separated list of probabilities, e.g delay=150us:10,2ms:5,0.5s:1 for 10% of chance of a 150 us delay, 5% of a 2ms delay and 1% of a 1/2 second delay |
| status | http status to return instead of 200. Can be a single value or a coma separated list of probabilities, e.g status=404:10,503:5,429:1 for 10% of chance of a 404 status, 5% of a 503 status and 1% of a 429 status |
/debugwill echo back the request in plain text for human debugging./fortio/A UI to- Run/Trigger tests and graph the results.
- A UI to browse saved results and single graph or multi graph them (comparative graph of min,avg, median, p75, p99, p99.9 and max).
- Proxy/fetch other URLs
/fortio/data/index.tsvan tab separated value file conforming to Google cloud storage URL list data transfer format so you can export/backup local results to the cloud.- Download/sync peer to peer JSON results files from other Fortio servers (using their
index.tsvURLs) - Download/sync from an Amazon S3 or Google Cloud compatible bucket listings XML URLs
The report mode is a readonly subset of the above directly on /.
Fortio is written in the Go language and includes a scalable semi log histogram in stats.go and a periodic runner engine in periodic.go with specializations for http and grpc. The http/ package includes a very high performance specialized http 1.1 client. You may find fortio's logger useful as well.
You can run the histogram code standalone as a command line in histogram/, a basic echo http server in echosrv/, or both the http echo and GRPC ping server through fortio server, the fortio command line interface lives in this top level directory fortio_main.go
With 5k qps: (includes envoy and mixer in the calls)
$ time fortio load -qps 5000 -t 60s -c 8 -r 0.0001 -H "Host: perf-cluster" http://benchmark-2:9090/echo
2017/07/09 02:31:05 Will be setting special Host header to perf-cluster
Fortio running at 5000 queries per second for 1m0s: http://benchmark-2:9090/echo
Starting at 5000 qps with 8 thread(s) [gomax 4] for 1m0s : 37500 calls each (total 300000)
2017/07/09 02:32:05 T004 ended after 1m0.000907812s : 37500 calls. qps=624.9905437680746
2017/07/09 02:32:05 T000 ended after 1m0.000922222s : 37500 calls. qps=624.9903936684861
2017/07/09 02:32:05 T005 ended after 1m0.00094454s : 37500 calls. qps=624.9901611965524
2017/07/09 02:32:05 T006 ended after 1m0.000944816s : 37500 calls. qps=624.9901583216429
2017/07/09 02:32:05 T001 ended after 1m0.00102094s : 37500 calls. qps=624.9893653892883
2017/07/09 02:32:05 T007 ended after 1m0.001096292s : 37500 calls. qps=624.9885805003184
2017/07/09 02:32:05 T003 ended after 1m0.001045342s : 37500 calls. qps=624.9891112105419
2017/07/09 02:32:05 T002 ended after 1m0.001044416s : 37500 calls. qps=624.9891208560392
Ended after 1m0.00112695s : 300000 calls. qps=4999.9
Aggregated Sleep Time : count 299992 avg 8.8889218e-05 +/- 0.002326 min -0.03490402 max 0.001006041 sum 26.6660543
# range, mid point, percentile, count
< 0 , 0 , 8.58, 25726
>= 0 < 0.001 , 0.0005 , 100.00, 274265
>= 0.001 < 0.002 , 0.0015 , 100.00, 1
# target 50% 0.000453102
WARNING 8.58% of sleep were falling behind
Aggregated Function Time : count 300000 avg 0.00094608764 +/- 0.0007901 min 0.000510522 max 0.029267604 sum 283.826292
# range, mid point, percentile, count
>= 0.0005 < 0.0006 , 0.00055 , 0.15, 456
>= 0.0006 < 0.0007 , 0.00065 , 3.25, 9295
>= 0.0007 < 0.0008 , 0.00075 , 24.23, 62926
>= 0.0008 < 0.0009 , 0.00085 , 62.73, 115519
>= 0.0009 < 0.001 , 0.00095 , 85.68, 68854
>= 0.001 < 0.0011 , 0.00105 , 93.11, 22293
>= 0.0011 < 0.0012 , 0.00115 , 95.38, 6792
>= 0.0012 < 0.0014 , 0.0013 , 97.18, 5404
>= 0.0014 < 0.0016 , 0.0015 , 97.94, 2275
>= 0.0016 < 0.0018 , 0.0017 , 98.34, 1198
>= 0.0018 < 0.002 , 0.0019 , 98.60, 775
>= 0.002 < 0.0025 , 0.00225 , 98.98, 1161
>= 0.0025 < 0.003 , 0.00275 , 99.21, 671
>= 0.003 < 0.0035 , 0.00325 , 99.36, 449
>= 0.0035 < 0.004 , 0.00375 , 99.47, 351
>= 0.004 < 0.0045 , 0.00425 , 99.57, 290
>= 0.0045 < 0.005 , 0.00475 , 99.66, 280
>= 0.005 < 0.006 , 0.0055 , 99.79, 380
>= 0.006 < 0.007 , 0.0065 , 99.82, 92
>= 0.007 < 0.008 , 0.0075 , 99.83, 15
>= 0.008 < 0.009 , 0.0085 , 99.83, 5
>= 0.009 < 0.01 , 0.0095 , 99.83, 1
>= 0.01 < 0.012 , 0.011 , 99.83, 8
>= 0.012 < 0.014 , 0.013 , 99.84, 35
>= 0.014 < 0.016 , 0.015 , 99.92, 231
>= 0.016 < 0.018 , 0.017 , 99.94, 65
>= 0.018 < 0.02 , 0.019 , 99.95, 26
>= 0.02 < 0.025 , 0.0225 , 100.00, 139
>= 0.025 < 0.03 , 0.0275 , 100.00, 14
# target 50% 0.000866935
# target 75% 0.000953452
# target 99% 0.00253875
# target 99.9% 0.0155152
Code 200 : 300000
Response Body Sizes : count 300000 avg 0 +/- 0 min 0 max 0 sum 0
Or you can get the data in JSON format (using -json result.json)
Or graphically (through the http://localhost:8080/fortio/ web UI):
Simple form/UI:
Sample requests with responses delayed by 250us and 0.5% of 503 and 1.5% of 429 simulated http errors.
Run result:
Code 200 : 2939 (98.0 %)
Code 429 : 47 (1.6 %)
Code 503 : 14 (0.5 %)
Contributions whether through issues, documentation, bug fixes, or new features are most welcome !
Please also see Contributing to Istio and Getting started contributing to Fortio in the FAQ.
And make sure to go format and run those commands successfully before sending your PRs:
make test
make lint
make webtest
When modifying Javascript, check with standard:
standard --fix ui/static/js/fortio_chart.js
Our wiki and the Fortio FAQ (including for instance differences between fortio and wrk or httpbin)