kube-proxy appears to have some bad long-tail latency issues #3734
Labels
priority/awaiting-more-evidence
Lowest priority. Possibly useful, but not yet enough support to actually get it done.
sig/network
Categorizes an issue or PR as relevant to SIG Network.
sig/scalability
Categorizes an issue or PR as relevant to SIG Scalability.
using redis-benchmark as a test:
with the proxy:
====== INCR ======
10000 requests completed in 0.67 seconds
50 parallel clients
3 bytes payload
keep alive: 1
1.16% <= 1 milliseconds
8.60% <= 2 milliseconds
72.41% <= 3 milliseconds
95.61% <= 4 milliseconds
98.83% <= 5 milliseconds
99.02% <= 6 milliseconds
99.25% <= 7 milliseconds
99.28% <= 8 milliseconds
99.35% <= 9 milliseconds
99.54% <= 10 milliseconds
99.58% <= 11 milliseconds
99.59% <= 13 milliseconds
99.60% <= 20 milliseconds
99.61% <= 21 milliseconds
99.62% <= 23 milliseconds
99.63% <= 25 milliseconds
99.64% <= 27 milliseconds
99.65% <= 29 milliseconds
99.66% <= 31 milliseconds
99.67% <= 32 milliseconds
99.68% <= 34 milliseconds
99.69% <= 35 milliseconds
99.70% <= 39 milliseconds
99.71% <= 40 milliseconds
99.72% <= 42 milliseconds
99.73% <= 43 milliseconds
99.74% <= 49 milliseconds
99.75% <= 52 milliseconds
99.76% <= 53 milliseconds
99.77% <= 55 milliseconds
99.78% <= 57 milliseconds
99.79% <= 60 milliseconds
99.80% <= 61 milliseconds
99.81% <= 64 milliseconds
99.82% <= 66 milliseconds
99.83% <= 68 milliseconds
99.84% <= 70 milliseconds
99.85% <= 73 milliseconds
99.86% <= 74 milliseconds
99.87% <= 77 milliseconds
99.88% <= 80 milliseconds
99.89% <= 82 milliseconds
99.90% <= 85 milliseconds
99.91% <= 89 milliseconds
99.92% <= 92 milliseconds
99.93% <= 95 milliseconds
99.94% <= 98 milliseconds
99.95% <= 100 milliseconds
99.96% <= 103 milliseconds
99.97% <= 107 milliseconds
99.98% <= 110 milliseconds
99.99% <= 113 milliseconds
100.00% <= 116 milliseconds
14880.95 requests per second
without:
====== INCR ======
10000 requests completed in 0.43 seconds
50 parallel clients
3 bytes payload
keep alive: 1
0.20% <= 1 milliseconds
67.35% <= 2 milliseconds
94.11% <= 3 milliseconds
97.21% <= 4 milliseconds
98.23% <= 5 milliseconds
99.04% <= 6 milliseconds
99.51% <= 14 milliseconds
99.82% <= 15 milliseconds
100.00% <= 15 milliseconds
23255.81 requests per second
We're basically at the same point for the 99%, but past that the proxy falls off a cliff. Would be interesting to understand why.
The text was updated successfully, but these errors were encountered: