eBPF-based autoinstrumentation of HTTP/HTTPS/GRPC Go services.

Documentation

Part of the code is taken from: https://github.com/open-telemetry/opentelemetry-go-instrumentation

Differences:

• No need to maintain old Go versions (e.g. stack-based parameters)
• standard HTTP instrumentation works
  • Original didn't work in Go 1.17+ because it uses registers odigos-io/opentelemetry-go-instrumentation#45
  • We use a pointer to the goroutine as map key
• Using ringbuffer instead of perf_buffer
  • despite the name, ringbuffer is faster
• We return status code
• We provide goroutine wait time (since the connection is accepted until the request is processed) for more accurate metrics on overload scenarios.
• Capturing request sizes for HTTP

The simplest way is to use Kubernetes and the files in the deployments/ folder.

1. Provide your Grafana credentials. Use the following K8s Secret template to introduce the endpoints, usernames and API keys for Mimir and Tempo:

   $ cp deployments/01-example-k8s-agentconfig.yml.template deployments/01-example-k8s-agentconfig.yml
   $ # EDIT the fields
   $ vim deployments/01-example-k8s-agentconfig.yml.template
   $ kubectl apply -f deployments/01-example-k8s-agentconfig.yml 
   

2. Deploy the Grafana Aent:

   kubectl apply -f deployments/02-grafana-agent.yml
   

3. Deploy a demo app with the auto-instrumenter as a sidecar. You can use the blog example in the deployments/03-instrumented-app.yml file.

   $ kubectl apply -f ./deployments/03-instrumented-app
   $ kubectl port-forward service/goblog 8443:8443
   

You should be able to query traces and metrics in your Grafana board.

The eBPF program is embedded into the pkg/ebpf/bpf_* generated files. This step is generally not needed unless you change the C code in the bpf folder.

If you have Docker installed, you just need to run:

make docker-generate

If you can't install docker, you should locally install the following required packages:

dnf install -y kernel-devel make llvm clang glibc-devel.i686
make generate

Tested in Fedora 35, 38 and Red Hat Enterprise Linux 8.