Skip to content

jdockerty/gruglb

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Grug Load Balancer (gruglb)

A simplistic L4/L7 load balancer, written in Rust, for grug brained developers (me).

Why?

This is largely a toy project and not intended for production use, but also provides a segue into being able to use a simple load balancer without many frills for my own projects and to learn about writing more complex systems in Rust.

Install

Using cargo you can install via

cargo install --git https://github.com/jdockerty/gruglb --bin gruglb

Once installed, pass a YAML config file using the --config flag, for example

gruglb --config path/to/config.yml

Features

  • Round-robin load balancing of HTTP/HTTPS/TCP connections.
  • Health checks for HTTP/HTTPS/TCP targets.
  • Graceful termination.
  • TLS via termination, backends are still expected to be accessible over HTTP.

How does it work?

Given a number of pre-defined targets which contains various backend servers, gruglb will route traffic between them in round-robin fashion.

When a backend server is deemed unhealthy, by failing a GET request to the specified health_path for a HTTP target or failing to establish a connection for a TCP target, it is removed from the routable backends for the specified target. This means that a target with two backends will have all traffic be directed to the single healthy backend until the other server becomes healthy again.

Health checks are conducted at a fixed interval upon the application starting and continue throughout its active lifecycle.

The configuration is defined in YAML, using the example-config.yaml that is used for testing, it looks like this:

# The interval, in seconds, to conduct HTTP/TCP health checks.
health_check_interval: 2

# Run a graceful shutdown period of 30 seconds to terminate separate worker threads.
# Defaults to true.
graceful_shutdown: true

# Log level information, defaults to 'info'
logging: info

# Defined "targets", the key simply acts as a convenient label for various backend
# servers which are to have traffic routed to them.
targets:

  # TCP target example
  tcpServersA:
    # Either TCP or HTTP, defaults to TCP when not set.
    protocol: 'tcp'

    # Port to bind to for this target.
    listener: 9090

    # Statically defined backend servers.
    backends:
      - host: "127.0.0.1"
        port: 8090
      - host: "127.0.0.1"
        port: 8091

  # HTTP target example
  webServersA:
    protocol: 'http'
    listener: 8080
    backends:
      - host: "127.0.0.1"
        port: 8092
        # A `health_path` is only required for HTTP backends.
        health_path: "/health"
      - host: "127.0.0.1"
        port: 8093
        health_path: "/health"

Using the HTTP bound listener of 8080 as our example, if we send traffic to this we expect to see a response back from our configured backends under webServersA. In this instance, the fake_backend application is already running.

# In separate terminal windows (or as background jobs) run the fake backends
fake_backend --id fake-1 --protocol http --port 8092
fake_backend --id fake-2 --protocol http --port 8093

# In your main window, run the load balancer
gruglb --config tests/fixtures/example-config.yaml

# Send some traffic to the load balancer
for i in {1..5}; do curl localhost:8080; echo; done

# You should have the requests routed in a round-robin fashion to the backends.
# The output from the above command should look like this
Hello from fake-2
Hello from fake-1
Hello from fake-2
Hello from fake-1
Hello from fake-2

Performance

These tests are not very scientific and were simply ran as small experiments to see comparative performance between my implementation and something that I know is very good.

Using bombardier as the tool of choice.

gruglb

Running on localhost

CPU: Intel i7-8700 (12) @ 4.600GHz

Using two simplebenchserver servers as backends for a HTTP target:

bombardier http://127.0.0.1:8080 --latencies --fasthttp -H "Connection: close"
Bombarding http://127.0.0.1:8080 for 10s using 125 connection(s)
[========================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     42558.30    3130.17   47446.16
  Latency        2.93ms   427.72us    29.29ms
  Latency Distribution
     50%     2.85ms
     75%     3.17ms
     90%     3.61ms
     95%     4.01ms
     99%     5.22ms
  HTTP codes:
    1xx - 0, 2xx - 425267, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
Running on AWS with m5.xlarge nodes

This test was performed with 4 nodes: 1 for the load balancer, 2 backend servers running a slightly modified version of simplebenchserver which allows binding to 0.0.0.0, and 1 node used to send traffic internally to the load balancer.

bombardier http://172.31.22.113:8080 --latencies --fasthttp -H "Connection: close"
Bombarding http://172.31.22.113:8080 for 10s using 125 connection(s)
[======================================================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     16949.53    9201.05   29354.53
  Latency        7.37ms     6.62ms   103.98ms
  Latency Distribution
     50%     4.99ms
     75%     6.14ms
     90%    14.03ms
     95%    22.23ms
     99%    42.41ms
  HTTP codes:
    1xx - 0, 2xx - 169571, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    20.14MB/s

nginx

Configuration
events {
    worker_connections 1024;
}

http {
    server {
        listen 8080;
        location / {
            proxy_pass http://backend;
        }
    }
    upstream backend {
        server 172.31.21.226:8091;
        server 172.31.27.167:8092;
    }
}
Running on localhost

Using the same two backend servers and a single worker process for nginx

bombardier http://127.0.0.1:8080 --latencies --fasthttp -H "Connection: close"
Bombarding http://127.0.0.1:8080 for 10s using 125 connection(s)
[========================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     11996.59     784.99   14555.03
  Latency       10.42ms     2.91ms   226.42ms
  Latency Distribution
     50%    10.37ms
     75%    10.72ms
     90%    11.04ms
     95%    11.22ms
     99%    11.71ms
  HTTP codes:
    1xx - 0, 2xx - 119862, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    14.29MB/s

Something to note is that gruglb does not have the concept of worker_processes like nginx does.

This was ran with the default of a single process, it performs even better with multiple (~85k req/s).

Running on AWS with m5.xlarge nodes

This test was performed with 4 nodes: 1 for the load balancer, 2 backend servers running a slightly modified version of simplebenchserver which allows binding to 0.0.0.0, and 1 node used to send traffic internally to the load balancer.

Again, using the default of worker_processes 1;

bombardier http://172.31.22.113:8080 --latencies --fasthttp -H "Connection: close"
Bombarding http://172.31.22.113:8080 for 10s using 125 connection(s)
[======================================================================================================================================================] 10s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      8207.42    2301.56   11692.24
  Latency       15.22ms     6.57ms   100.47ms
  Latency Distribution
     50%    14.71ms
     75%    15.88ms
     90%    18.67ms
     95%    25.69ms
     99%    49.37ms
  HTTP codes:
    1xx - 0, 2xx - 82117, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     9.93MB/s

Releases

No releases published

Packages

No packages published