Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker image alternative using Ubuntu #2205

Merged
merged 11 commits into from
Mar 2, 2019
Merged

Conversation

Alkarex
Copy link
Member

@Alkarex Alkarex commented Jan 1, 2019

Ubuntu seems to be faster, cf. e.g. PHPBench on:

This image is also different from our Alpine-based image as it is using Apache Event and PHP-FPM as FastCGI. They are also supposed to be faster for some types of workload than mod_php but with other drawbacks (poorer control of flush, more processes running...).
So some concreate performance tests would be needed to check whether this is relevant at all.
It is quite usual for Docker images to offer both a Debian/Ubuntu and an Alpine version, so we could also do the same if there is any need.

Follow-up of #2202

Ubuntu seems to be faster, cf. e.g. PHPBench on:

https://www.phoronix.com/scan.php?page=article&item=freebsd-12-windows&num=5

https://www.phoronix.com/scan.php?page=article&item=docker-summer-2018&num=4
This image is also different from our Alpine-based image as it is using
Apache Event and PHP-FPM as FastCGI. They is also supposed to be faster
for some types of workload than mod_php but with other drawbacks (poorer
control of flush, more processes running...).
So some concreate performance tests would be needed to check whether
this is relevant at all.
It is quite usual for Docker images to offer both a Debian/Ubuntu and an
Alpine version, so we could also do the same if there is any need.
Follow-up of FreshRSS#2202
@Alkarex Alkarex added this to the 1.14.0 milestone Jan 1, 2019
Docker/Dockerfile-Ubuntu Outdated Show resolved Hide resolved
Fix double-logging issue
Default Apache configuration in Ubuntu, which was removed when disabling
all confs.
It is also the default Apache configuration in Alpine.
@Alkarex
Copy link
Member Author

Alkarex commented Jan 7, 2019

The Ubuntu version (with php-fpm) of our Docker image seems to be about 3 times faster than the Alpine version (with php-mod).

That is probably significant enough to care. Additional tests are needed to distinguish the Alpine vs. Ubuntu from the mod-php vs. php-fpm.

Details:

Tests done with Apache Bench from local network with the system running on a Raspberry Pi 3B+, comparing our Alpine (using mod_php) image with this Ubuntu one (using php-fpm). Using SQLite, HTTP authentication, and FreshRSS configured to return 32 articles per page. Otherwise default settings of everything else.

Protocol: run ab -n 1000 -c 10 to warm up (discard the results), and then run 10000 request with 1, 10, 50, and then 100 concurrent requests.

  • Alpine (mod-php): about 200ms per request, about 15 requests per second. Full system is using ~520MB of RAM during load with 100 concurrent requests and then down to ~216MB.
  • Ubuntu (php-fpm): about 48ms per request, about 45 requests per second. Full system is using ~250MB of RAM during load with 100 concurrent requests and then down to ~224MB.

Raw data:

Alpine mod-php

$ ab -n 10000 -c 1 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.35
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302000 bytes

Concurrency Level:      1
Time taken for tests:   2003.033 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3024030000 bytes
HTML transferred:       3020000000 bytes
Requests per second:    4.99 [#/sec] (mean)
Time per request:       200.303 [ms] (mean)
Time per request:       200.303 [ms] (mean, across all concurrent requests)
Transfer rate:          1474.34 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.1      1       4
Processing:   172  200  71.0    195    3873
Waiting:       72   85  68.0     82    3759
Total:        172  200  71.0    196    3873

Percentage of the requests served within a certain time (ms)
  50%    196
  66%    198
  75%    202
  80%    206
  90%    220
  95%    238
  98%    247
  99%    256
 100%   3873 (longest request)

----

$ ab -n 10000 -c 10 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.35
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302000 bytes

Concurrency Level:      10
Time taken for tests:   655.669 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3024030000 bytes
HTML transferred:       3020000000 bytes
Requests per second:    15.25 [#/sec] (mean)
Time per request:       655.669 [ms] (mean)
Time per request:       65.567 [ms] (mean, across all concurrent requests)
Transfer rate:          4504.03 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    1   0.3      1      13
Processing:   230  654 252.5    636    6187
Waiting:       94  290 217.1    275    5700
Total:        232  655 252.5    637    6188

Percentage of the requests served within a certain time (ms)
  50%    637
  66%    688
  75%    725
  80%    752
  90%    819
  95%    876
  98%    934
  99%   1010
 100%   6188 (longest request)

----

$ ab -n 10000 -c 50 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.35
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302000 bytes

Concurrency Level:      50
Time taken for tests:   646.817 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3024030000 bytes
HTML transferred:       3020000000 bytes
Requests per second:    15.46 [#/sec] (mean)
Time per request:       3234.086 [ms] (mean)
Time per request:       64.682 [ms] (mean, across all concurrent requests)
Transfer rate:          4565.67 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    1   0.8      1      17
Processing:   804 3230 513.0   3047   10047
Waiting:      295 1308 281.5   1244    8375
Total:        805 3231 513.1   3048   10048

Percentage of the requests served within a certain time (ms)
  50%   3048
  66%   3145
  75%   3278
  80%   3678
  90%   4090
  95%   4228
  98%   4367
  99%   4544
 100%  10048 (longest request)

----

$ ab -n 10000 -c 100 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.35
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302000 bytes

Concurrency Level:      100
Time taken for tests:   648.992 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3024030000 bytes
HTML transferred:       3020000000 bytes
Requests per second:    15.41 [#/sec] (mean)
Time per request:       6489.918 [ms] (mean)
Time per request:       64.899 [ms] (mean, across all concurrent requests)
Transfer rate:          4550.37 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    1   3.0      1      37
Processing:   285 6477 1133.1   6100   38129
Waiting:      141 2597 703.2   2429   34561
Total:        322 6478 1132.3   6101   38129

Percentage of the requests served within a certain time (ms)
  50%   6101
  66%   6247
  75%   6475
  80%   7307
  90%   8239
  95%   8511
  98%   8790
  99%   9150
 100%  38129 (longest request)

Ubuntu php-fpm

$ ab -n 10000 -c 1 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302146 bytes

Concurrency Level:      1
Time taken for tests:   475.629 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3026590000 bytes
HTML transferred:       3021460000 bytes
Requests per second:    21.02 [#/sec] (mean)
Time per request:       47.563 [ms] (mean)
Time per request:       47.563 [ms] (mean, across all concurrent requests)
Transfer rate:          6214.20 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.1      1       2
Processing:    34   47  61.6     41    4764
Waiting:        9   13  60.7     11    4710
Total:         35   47  61.6     41    4764

Percentage of the requests served within a certain time (ms)
  50%     41
  66%     50
  75%     51
  80%     52
  90%     53
  95%     54
  98%     63
  99%     73
 100%   4764 (longest request)

----

$ ab -n 10000 -c 10 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302146 bytes

Concurrency Level:      10
Time taken for tests:   217.459 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3026590000 bytes
HTML transferred:       3021460000 bytes
Requests per second:    45.99 [#/sec] (mean)
Time per request:       217.459 [ms] (mean)
Time per request:       21.746 [ms] (mean, across all concurrent requests)
Transfer rate:          13591.80 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   1.2      1      20
Processing:    89  215  86.2    202    2415
Waiting:       26  135  73.4    132    2207
Total:         91  217  86.1    203    2416

Percentage of the requests served within a certain time (ms)
  50%    203
  66%    222
  75%    238
  80%    249
  90%    283
  95%    315
  98%    345
  99%    369
 100%   2416 (longest request)

----

$ ab -n 10000 -c 50 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302146 bytes

Concurrency Level:      50
Time taken for tests:   218.575 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3026590000 bytes
HTML transferred:       3021460000 bytes
Requests per second:    45.75 [#/sec] (mean)
Time per request:       1092.875 [ms] (mean)
Time per request:       21.858 [ms] (mean, across all concurrent requests)
Transfer rate:          13522.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    2   1.4      1      22
Processing:   125 1088 111.1   1065    1911
Waiting:       51 1008 101.2    992    1748
Total:        131 1090 110.9   1067    1912

Percentage of the requests served within a certain time (ms)
  50%   1067
  66%   1090
  75%   1108
  80%   1121
  90%   1166
  95%   1219
  98%   1517
  99%   1638
 100%   1912 (longest request)

----

$ ab -n 10000 -c 100 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.X.XXX (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.X.XXX
Server Port:            8X8X

Document Path:          /i/?a=normal
Document Length:        302146 bytes

Concurrency Level:      100
Time taken for tests:   226.370 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      3026590000 bytes
HTML transferred:       3021460000 bytes
Requests per second:    44.18 [#/sec] (mean)
Time per request:       2263.703 [ms] (mean)
Time per request:       22.637 [ms] (mean, across all concurrent requests)
Transfer rate:          13056.72 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    2   2.0      1      21
Processing:   131 2251 386.5   2161    5169
Waiting:       64 2169 382.7   2085    4995
Total:        152 2253 386.1   2163    5169

Percentage of the requests served within a certain time (ms)
  50%   2163
  66%   2191
  75%   2216
  80%   2239
  90%   2385
  95%   2978
  98%   3339
  99%   4959
 100%   5169 (longest request)

@Alkarex
Copy link
Member Author

Alkarex commented Mar 2, 2019

I am finally finding time to do additional performance testing, this time comparing 3 variants:

  1. Alpine mod-php
  2. Ubuntu mod-php
  3. Ubuntu php-fpm

Done on a new FreshRSS version and new database content, still on Raspberry Pi 3 B+. First results with only 1000 requests, to quickly confirm the findings from previous tests: The Alpine version is about 3 times slower than the Ubuntu version.

There does not seem to be a big difference between Ubuntu mod-php and Ubuntu php-fpm, so I will run again the tests with 10k requests for a better precision.

Raw data

Alpine mod-php

ab -n 1000 -c 1 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.38
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      1
Time taken for tests:   211.523 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      410014000 bytes
HTML transferred:       409611000 bytes
Requests per second:    4.73 [#/sec] (mean)
Time per request:       211.523 [ms] (mean)
Time per request:       211.523 [ms] (mean, across all concurrent requests)
Transfer rate:          1892.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.1      1       2
Processing:   184  211  78.1    204    2034
Waiting:       87  103  77.5     98    1922
Total:        185  211  78.1    204    2034

Percentage of the requests served within a certain time (ms)
  50%    204
  66%    210
  75%    213
  80%    213
  90%    228
  95%    245
  98%    258
  99%    264
 100%   2034 (longest request)

----

ab -n 1000 -c 10 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.38
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      10
Time taken for tests:   72.722 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      410014000 bytes
HTML transferred:       409611000 bytes
Requests per second:    13.75 [#/sec] (mean)
Time per request:       727.220 [ms] (mean)
Time per request:       72.722 [ms] (mean, across all concurrent requests)
Transfer rate:          5505.96 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.3      1       3
Processing:   297  725 228.0    693    2390
Waiting:      124  356 159.3    340    1920
Total:        298  726 228.0    694    2391

Percentage of the requests served within a certain time (ms)
  50%    694
  66%    749
  75%    811
  80%    844
  90%    951
  95%   1052
  98%   1411
  99%   1687
 100%   2391 (longest request)

Ubuntu mod-php

ab -n 1000 -c 1 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      1
Time taken for tests:   69.369 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      409990000 bytes
HTML transferred:       409611000 bytes
Requests per second:    14.42 [#/sec] (mean)
Time per request:       69.369 [ms] (mean)
Time per request:       69.369 [ms] (mean, across all concurrent requests)
Transfer rate:          5771.76 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.1      1       1
Processing:    59   69  21.1     62     576
Waiting:       22   26  16.6     23     517
Total:         60   69  21.1     63     576

Percentage of the requests served within a certain time (ms)
  50%     63
  66%     64
  75%     69
  80%     71
  90%     85
  95%    106
  98%    117
  99%    119
 100%    576 (longest request)

----

ab -n 1000 -c 10 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      10
Time taken for tests:   27.198 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      409990000 bytes
HTML transferred:       409611000 bytes
Requests per second:    36.77 [#/sec] (mean)
Time per request:       271.983 [ms] (mean)
Time per request:       27.198 [ms] (mean, across all concurrent requests)
Transfer rate:          14720.78 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   2.3      1      15
Processing:    87  269  86.0    263     637
Waiting:       33   93  35.2     87     303
Total:         90  271  86.0    265     638

Percentage of the requests served within a certain time (ms)
  50%    265
  66%    297
  75%    314
  80%    331
  90%    377
  95%    420
  98%    501
  99%    540
 100%    638 (longest request)

----

ab -n 1000 -c 100 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      100
Time taken for tests:   32.668 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      409990000 bytes
HTML transferred:       409611000 bytes
Requests per second:    30.61 [#/sec] (mean)
Time per request:       3266.767 [ms] (mean)
Time per request:       32.668 [ms] (mean, across all concurrent requests)
Transfer rate:          12256.18 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1   13  28.0      2     137
Processing:   178 3201 1556.4   2798    8727
Waiting:       65 1757 1308.8   1309    6266
Total:        193 3214 1554.3   2807    8728

Percentage of the requests served within a certain time (ms)
  50%   2807
  66%   3137
  75%   3438
  80%   3679
  90%   6010
  95%   7188
  98%   7776
  99%   8064
 100%   8728 (longest request)

Ubuntu php-fpm

ab -n 1000 -c 1 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      1
Time taken for tests:   72.217 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      410111000 bytes
HTML transferred:       409611000 bytes
Requests per second:    13.85 [#/sec] (mean)
Time per request:       72.217 [ms] (mean)
Time per request:       72.217 [ms] (mean, across all concurrent requests)
Transfer rate:          5545.77 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.8      1      20
Processing:    58   71  73.3     62    1744
Waiting:       23   29  48.6     25     967
Total:         59   72  73.3     63    1745

Percentage of the requests served within a certain time (ms)
  50%     63
  66%     69
  75%     71
  80%     72
  90%     74
  95%     95
  98%    117
  99%    136
 100%   1745 (longest request)

----

ab -n 1000 -c 10 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      10
Time taken for tests:   30.930 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      410111000 bytes
HTML transferred:       409611000 bytes
Requests per second:    32.33 [#/sec] (mean)
Time per request:       309.304 [ms] (mean)
Time per request:       30.930 [ms] (mean, across all concurrent requests)
Transfer rate:          12948.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   1.1      1       7
Processing:   128  306  92.3    288    1036
Waiting:       60  215  66.2    210     937
Total:        130  308  92.1    290    1037

Percentage of the requests served within a certain time (ms)
  50%    290
  66%    316
  75%    340
  80%    352
  90%    400
  95%    428
  98%    466
  99%    915
 100%   1037 (longest request)

----

ab -n 1000 -c 100 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 1000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        409611 bytes

Concurrency Level:      100
Time taken for tests:   30.197 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      410111000 bytes
HTML transferred:       409611000 bytes
Requests per second:    33.12 [#/sec] (mean)
Time per request:       3019.715 [ms] (mean)
Time per request:       30.197 [ms] (mean, across all concurrent requests)
Transfer rate:          13262.81 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    3   4.2      1      20
Processing:   136 2871 504.6   3005    3247
Waiting:       86 2783 496.0   2930    3092
Total:        155 2874 501.1   3006    3248

Percentage of the requests served within a certain time (ms)
  50%   3006
  66%   3035
  75%   3058
  80%   3072
  90%   3118
  95%   3160
  98%   3187
  99%   3210
 100%   3248 (longest request)

@Frenzie
Copy link
Member

Frenzie commented Mar 2, 2019

I still find the difference astounding. Does Alpine ship a more vanilla Apache and/or PHP configuration or something?

@Alkarex
Copy link
Member Author

Alkarex commented Mar 2, 2019

@Frenzie Indeed, I am also quite surprised. Independent tests by phoronix.com (see links in first comment) are also showing big differences in the same direction (although not as much as what I am observing here). The Apache + PHP configurations should be extremely similar between the two images. On the one hand is Ubuntu known to be very well optimised for speed, and on the other hand is Alpine focusing on a small footprint with a musl libc that is producing smaller but sometimes less speedy binaries. We should reproduce the tests on another CPU architecture such as AMD64.

@Alkarex
Copy link
Member Author

Alkarex commented Mar 2, 2019

On Raspberry Pi 3B+, Ubuntu mod-php wins, being ~14% faster than Ubuntu php-fpm (which was ~300% faster than Alpine mod-php).

Maybe this could be partially due to the fact that php-fpm does not support flush() optimisations with HTTP compression like mod-php does, and to the fact that there is one additional layer to go through with php-fpm.

Raw data

Ubuntu mod-php

ab -n 10000 -c 1 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        411161 bytes

Concurrency Level:      1
Time taken for tests:   710.056 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      4115400000 bytes
HTML transferred:       4111610000 bytes
Requests per second:    14.08 [#/sec] (mean)
Time per request:       71.006 [ms] (mean)
Time per request:       71.006 [ms] (mean, across all concurrent requests)
Transfer rate:          5660.04 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.3      1      14
Processing:    59   70  31.0     67    2020
Waiting:       22   27  28.5     24    1947
Total:         59   71  31.0     67    2021

Percentage of the requests served within a certain time (ms)
  50%     67
  66%     70
  75%     71
  80%     72
  90%     84
  95%     93
  98%    115
  99%    118
 100%   2021 (longest request)

----

ab -n 10000 -c 10 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        411161 bytes

Concurrency Level:      10
Time taken for tests:   277.798 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      4115400000 bytes
HTML transferred:       4111610000 bytes
Requests per second:    36.00 [#/sec] (mean)
Time per request:       277.798 [ms] (mean)
Time per request:       27.780 [ms] (mean, across all concurrent requests)
Transfer rate:          14467.14 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   2.2      1      15
Processing:    85  275 120.0    261    1956
Waiting:       30   99  82.6     87    1721
Total:         86  278 120.0    263    1957

Percentage of the requests served within a certain time (ms)
  50%    263
  66%    295
  75%    317
  80%    334
  90%    394
  95%    459
  98%    535
  99%    609
 100%   1957 (longest request)

----

ab -n 10000 -c 100 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        411161 bytes

Concurrency Level:      100
Time taken for tests:   281.004 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      4115400000 bytes
HTML transferred:       4111610000 bytes
Requests per second:    35.59 [#/sec] (mean)
Time per request:       2810.037 [ms] (mean)
Time per request:       28.100 [ms] (mean, across all concurrent requests)
Transfer rate:          14302.11 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    6  19.0      1     163
Processing:   140 2798 747.8   2579    8094
Waiting:       62  879 438.3    767    4357
Total:        180 2804 747.5   2583    8099

Percentage of the requests served within a certain time (ms)
  50%   2583
  66%   2719
  75%   2875
  80%   3064
  90%   3777
  95%   4503
  98%   5186
  99%   5540
 100%   8099 (longest request)

Ubuntu php-fpm

ab -n 10000 -c 1 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        411161 bytes

Concurrency Level:      1
Time taken for tests:   745.311 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      4116610000 bytes
HTML transferred:       4111610000 bytes
Requests per second:    13.42 [#/sec] (mean)
Time per request:       74.531 [ms] (mean)
Time per request:       74.531 [ms] (mean, across all concurrent requests)
Transfer rate:          5393.89 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.5      1      24
Processing:    58   74  37.4     68    2181
Waiting:       24   30  33.4     28    2109
Total:         59   74  37.4     68    2182

Percentage of the requests served within a certain time (ms)
  50%     68
  66%     72
  75%     78
  80%     79
  90%     83
  95%     86
  98%    101
  99%    113
 100%   2182 (longest request)

----

ab -n 10000 -c 10 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        411161 bytes

Concurrency Level:      10
Time taken for tests:   326.227 seconds
Complete requests:      10000
Failed requests:        2763
   (Connect: 0, Receive: 0, Length: 2763, Exceptions: 0)
Total transferred:      4181128813 bytes
HTML transferred:       4176128813 bytes
Requests per second:    30.65 [#/sec] (mean)
Time per request:       326.227 [ms] (mean)
Time per request:       32.623 [ms] (mean, across all concurrent requests)
Transfer rate:          12516.25 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   1.4      1      59
Processing:   146  324 209.1    295    4259
Waiting:       61  233 201.8    214    4070
Total:        147  326 209.0    297    4260

Percentage of the requests served within a certain time (ms)
  50%    297
  66%    324
  75%    345
  80%    361
  90%    407
  95%    442
  98%    489
  99%    627
 100%   4260 (longest request)

----

ab -n 10000 -c 100 -A 'user:password' 'http://192.168.X.XXX:8X8X/i/?a=normal&state=3'
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.1.101 (be patient)
Finished 10000 requests


Server Software:        Apache/2.4.34
Server Hostname:        192.168.1.101
Server Port:            8282

Document Path:          /i/?a=normal&state=3
Document Length:        434512 bytes

Concurrency Level:      100
Time taken for tests:   321.267 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      4350120000 bytes
HTML transferred:       4345120000 bytes
Requests per second:    31.13 [#/sec] (mean)
Time per request:       3212.669 [ms] (mean)
Time per request:       32.127 [ms] (mean, across all concurrent requests)
Transfer rate:          13223.16 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    2   1.9      1      40
Processing:   167 3196 352.0   3117    4724
Waiting:       93 3100 346.1   3030    4560
Total:        187 3198 351.3   3119    4725

Percentage of the requests served within a certain time (ms)
  50%   3119
  66%   3156
  75%   3191
  80%   3217
  90%   3342
  95%   4099
  98%   4368
  99%   4476
 100%   4725 (longest request)

@Alkarex Alkarex merged commit fc8fb0a into FreshRSS:dev Mar 2, 2019
@Alkarex Alkarex deleted the Docker-Ubuntu branch March 2, 2019 20:49
Alkarex added a commit to Alkarex/FreshRSS that referenced this pull request Mar 2, 2019
FreshRSS#2205
(And removed proxy_fcgi forgotten from php-fpm)
@Alkarex Alkarex mentioned this pull request Mar 2, 2019
Alkarex added a commit that referenced this pull request Mar 2, 2019
#2205
(And removed proxy_fcgi forgotten from php-fpm)
@Alkarex Alkarex mentioned this pull request Mar 23, 2019
@Alkarex Alkarex mentioned this pull request Aug 20, 2019
Alkarex added a commit to Alkarex/FreshRSS that referenced this pull request Aug 20, 2019
@Alkarex Alkarex mentioned this pull request Aug 20, 2019
Alkarex added a commit that referenced this pull request Aug 20, 2019
javerous pushed a commit to javerous/FreshRSS that referenced this pull request Jan 20, 2020
* Docker image alternative using Ubuntu

Ubuntu seems to be faster, cf. e.g. PHPBench.
It is quite usual for Docker images to offer both a Debian/Ubuntu and an
Alpine version, so we could also do the same if there is any need.
Follow-up of FreshRSS#2202

* Only explicit use of Apache confs

Fix double-logging issue

* Explicit ServerTokens OS

Default Apache configuration in Ubuntu, which was removed when disabling all confs.
It is also the default Apache configuration in Alpine.

* Take advantage of syslog for actualization

Same than for Alpine
FreshRSS@43ab629

* COPY_SYSLOG_TO_STDERR

Same as FreshRSS#2213

* Update to libapache2-mod-php instead of php-fpm

* Default to Ubuntu

Ubuntu mod-php wins, being ~14% faster than Ubuntu php-fpm (which was ~300% faster than Alpine mod-php)
javerous pushed a commit to javerous/FreshRSS that referenced this pull request Jan 20, 2020
FreshRSS#2205
(And removed proxy_fcgi forgotten from php-fpm)
javerous pushed a commit to javerous/FreshRSS that referenced this pull request Jan 20, 2020
mdemoss pushed a commit to mdemoss/FreshRSS that referenced this pull request Mar 25, 2021
* Docker image alternative using Ubuntu

Ubuntu seems to be faster, cf. e.g. PHPBench.
It is quite usual for Docker images to offer both a Debian/Ubuntu and an
Alpine version, so we could also do the same if there is any need.
Follow-up of FreshRSS#2202

* Only explicit use of Apache confs

Fix double-logging issue

* Explicit ServerTokens OS

Default Apache configuration in Ubuntu, which was removed when disabling all confs.
It is also the default Apache configuration in Alpine.

* Take advantage of syslog for actualization

Same than for Alpine
FreshRSS@43ab629

* COPY_SYSLOG_TO_STDERR

Same as FreshRSS#2213

* Update to libapache2-mod-php instead of php-fpm

* Default to Ubuntu

Ubuntu mod-php wins, being ~14% faster than Ubuntu php-fpm (which was ~300% faster than Alpine mod-php)
mdemoss pushed a commit to mdemoss/FreshRSS that referenced this pull request Mar 25, 2021
FreshRSS#2205
(And removed proxy_fcgi forgotten from php-fpm)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants