Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ruler: correct logging of alert name & template data #15093

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

dimitarvdimitrov
Copy link
Contributor

@dimitarvdimitrov dimitarvdimitrov commented Oct 4, 2024

The error messages we see in the embedded ruler in Mimir right now are contain alert="unsupported value type" and data="unsupported value type"

ts=2024-10-04T09:59:56.900015691Z caller=alerting.go:384 level=warn component=ruler insight=true user=XXX alert="unsupported value type" msg="Expanding alert template failed" err="error executing template __alert_service_mesh_down_alert: template: __alert_service_mesh_down_alert:1:139: executing \"__alert_service_mesh_down_alert\" at <.registry_name>: can't evaluate field registry_name in type struct { Labels map[string]string; ExternalLabels map[string]string; ExternalURL string; Value interface {} }" data="unsupported value type"

now the log lines looks like this

time=2024-12-24T11:11:57.024+02:00 level=WARN msg="Expanding alert template failed" err="error parsing template __alert_HTTPRequestRateLow: template: __alert_HTTPRequestRateLow:1: bad character U+0022 '\"'" data="<Labels: {__name__=\"http_requests\", group=\"canary\", instance=\"1\", job=\"app-server\", severity=\"overwrite-me\"}, ExternalLabels: {__name__=\"http_requests\", group=\"canary\", instance=\"1\", job=\"app-server\", severity=\"overwrite-me\"}, ExternalURL: , Value: 80>"

The error messages we see in the embedded ruler in Mimir right now are contain `alert="unsupported value type"` and `data="unsupported value type"`

```
ts=2024-10-04T09:59:56.900015691Z caller=alerting.go:384 level=warn component=ruler insight=true user=XXX alert="unsupported value type" msg="Expanding alert template failed" err="error executing template __alert_service_mesh_down_alert: template: __alert_service_mesh_down_alert:1:139: executing \"__alert_service_mesh_down_alert\" at <.registry_name>: can't evaluate field registry_name in type struct { Labels map[string]string; ExternalLabels map[string]string; ExternalURL string; Value interface {} }" data="unsupported value type"
```

Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
@dimitarvdimitrov dimitarvdimitrov changed the title ruler: correct logging of alert name ruler: correct logging of alert name & template data Oct 4, 2024
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
rules/alerting.go Outdated Show resolved Hide resolved
Copy link
Member

@bboreham bboreham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out. At the bug-scrub we didn't like %#v.

rules/alerting.go Outdated Show resolved Hide resolved
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
@dimitarvdimitrov
Copy link
Contributor Author

sorry for taking so long. I made the changes you suggested @bboreham @krajorama . Please take another look when you get a minute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants