Scrapes cAdvisor port for metrics in Kubernetes 1.7 #477
Conversation
|
Can one of the admins verify this patch? |
|
ok to test |
| @@ -10,6 +10,9 @@ spec: | |||
| - port: http-metrics | |||
| interval: 30s | |||
| honorLabels: true | |||
There was a problem hiding this comment.
This can be removed now as it only was needed for cAdvisor metrics.
Those being exposed on separate endpoints now is actually good from a monitoring perspective.
|
Can you run |
|
@fabxc Thanks for the review. I've adjusted the diff based on your feedback. |
|
@alindeman @fabxc The CI failure is a false negative. Linting, unit and e2e tests succeeded. Sorry for the trouble. |
|
Heads up that kubernetes/kubernetes#48483 has been fixed by kubernetes/kubernetes#49079. |
|
lgtm, thanks a lot @alindeman ! It's a good idea to separate out cAdvisor anyways. |
|
@danp it seems like this is a solution that will continue working the same way as the separate path, but this is available now, so right now the better option. |
Perfect, thanks. |
Kubernetes 1.7 inadvertently stopped reporting cAdvisor metrics on the main /metrics endpoint. They are now only reported on the cAdvisor /metrics endpoint. As a temporary solution, I've added the cAdvisor port to the kubelet endpoint and servicemonitor so they get scraped as well.
I'm not totally sure it's a good idea to merge this, since the original behavior might be restored soon, but I did want to bring it up in this repository in an attempt to save other folks some headdesking trying to figure out why pod-level metrics are not being reported in Kubernetes 1.7.
As a stopgap for my use case, I've built my own container with this fix and pushed it to a registry. Hopefully the original behavior will be restored in an upcoming point release of Kubernetes.