Export metrics from Logstash to Prometheus. The project was created as rewrite of existing awesome application logstash_exporter, which was also written in Go, but it was not maintained for a long time. A lot of code was reused from the original project.

The application can be run in two ways:

• using the binary executable
• using the Docker image

Additionally Helm chart is provided for easy deployment to Kubernetes.

The binary executable can be downloaded from the releases page. Linux binary is available under https://github.com/kuskoman/logstash-exporter/releases/download/v${VERSION}/logstash-exporter-linux. The binary can be run without additional arguments, as the configuration is loaded from the .env file and environment variables.

Each binary should contain a SHA256 checksum file, which can be used to verify the integrity of the binary.

VERSION="test-tag" \
OS="linux" \
wget "https://github.com/kuskoman/logstash-exporter/releases/download/${VERSION}/logstash-exporter-${OS}" && \
wget "https://github.com/kuskoman/logstash-exporter/releases/download/${VERSION}/logstash-exporter-${OS}.sha256" && \
sha256sum -c logstash-exporter-${OS}.sha256

It is recommended to use the binary executable in combination with the systemd service. The application should not require any of root privileges, so it is recommended to run it as a non-root user.

The unstable version of the application can be downloaded from the GitHub Actions. The latest successful build can be found under the Go application CI/CD workflow (already selected in the link). To download the binary, simply go to the link location, click on the latest successful build, and download the binary from the Artifacts section on the bottom of the page.

You are able to download artifact from any workflow run, not only master branch. To do that, go to GitHub Actions without master filter, select the workflow run you want to download artifact from, and download the binary from the Artifacts section.

The Docker image is available under kuskoman/logstash-exporter:<tag>. You can pull the image using the following command:

docker pull kuskoman/logstash-exporter:<tag>

You can browse tags on the Docker Hub.

The Docker image can be run using the following command:

docker run -d \
    -p 9198:9198 \
    -e LOGSTASH_URL=http://logstash:9600 \
    kuskoman/logstash-exporter:<tag>

The unstable version of the Docker image can be downloaded from the GitHub Container Registry. To pull the image from command line simply use:

    docker pull ghcr.io/kuskoman/logstash-exporter:master

The Helm chart has its own README.

• /metrics: Exposes metrics in Prometheus format.
• /health: Returns 200 if app runs properly.

The application can be configured using the following environment variables, which are also loaded from .env file:

All configuration variables can be checked in the config directory.

• make all: Builds binary executables for Linux, macOS, and Windows and saves them in the out directory.
• make run: Runs the Go Exporter application.
• make build-<OS>: Builds a binary executable for the specified OS (<OS> can be linux, darwin, or windows).
• make build-docker: Builds a Docker image for the Go Exporter application.
• make build-docker-multi: Builds a multi-arch Docker image (amd64 and arm64) for the Go Exporter application.
• make clean: Deletes all binary executables in the out directory.
• make test: Runs all tests.
• make compose: Starts a Docker-compose configuration.
• make wait-for-compose: Starts a Docker-compose configuration and waits for it to be ready.
• make compose-down: Stops a Docker-compose configuration.
• make verify-metrics: Verifies the metrics from the Go Exporter application.
• make pull: Pulls the Docker image from the registry.
• make logs: Shows logs from the Docker-compose configuration.
• make minify: Minifies the binary executables.
• make help: Shows the available commands.

The main Go Exporter application is located in the cmd/exporter/main.go file. The binary executables are saved in the out directory.

Build binary executables for all supported operating systems:

make all

Run the Go Exporter application:

make run

Build a binary executable for macOS:

make build-darwin

Build the Docker image:

make build-docker

Build the multi-arch Docker image:

make build-docker-multi

Delete all binary executables:

make clean

Run all tests:

make test

Start the Docker-compose configuration:

make compose

Start the Docker-compose configuration and wait for it to be ready:

make wait-for-compose

Stop the Docker-compose configuration:

make compose-down

Verify the metrics from the Exporter application:

make verify-metrics

Pull the Docker image from the registry:

make pull

Show logs from the Docker-compose configuration:

make logs

Minify the binary executables:

make minify

Show the available commands:

make help

Application repository contains some helper scripts, which can be used to improve process of building, testing, and running the application. These scripts are not useful for the end user, but they can be useful for all potential contributors. The helper scripts are located in the scripts directory.

This script is used to add metrics table to the README.md file. Usage:

./scripts/add_metrics_to_readme.sh

This script is used to create release notes for the GitHub release. Used primarily by the CI workflow.

This script is used to generate Helm chart README.md file. The readme contains all the configuration variables from the values.yaml file.

This script is used to install readme-generator-for-helm tool. The tool is used to generate Helm chart README.md file. The script installs the tool under helm-generator directory.

This script is used to verify the metrics from the Go Exporter application. Can be used both locally and in the CI workflow.

./scripts/verify_metrics.sh

The application contains both unit and integration tests. All the tests are executed in the CI workflow.

Unit tests are located in the same directories as the tested files. To run all unit tests, use the following command:

make test

Integration tests checks if Prometheus metrics are exposed properly. To run them you must setup development docker-compose file.

make wait-for-compose

Then you can run the tests:

make verify-metrics

This projects code was reviewed by Boldly Go in an awesome video, which in a huge way helped me to improve the code quality.

These are the features that are planned to be implemented in the future:

• Parse missing metrics (if you find any useful missing metrics, please create an issue)
• Add description to all metrics
• Improve test coverage
• Build Helm chart (see #66 for progress)
• Automatically add release notes to GitHub release
• Provide example Grafana dashboard
• Improve chart value descriptions (current one is generated by GPT-4 and not always accurate)

Feel free to create an issue if you have any suggestions, ideas or questions.

If you want to contribute to this project, please read the CONTRIBUTING.md file.

Table of exported metrics: