Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Instrument Kubelet for monitoring #4604

Closed
vmarmol opened this issue Feb 19, 2015 · 4 comments
Closed

Instrument Kubelet for monitoring #4604

vmarmol opened this issue Feb 19, 2015 · 4 comments
Assignees
@vmarmol
Labels
area/introspection priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability.
Milestone
v1.0

Comments

@vmarmol
Copy link
Contributor

vmarmol commented Feb 19, 2015

There are a few areas of the Kubelet that we should instrument to allow us to monitor them for v1. This will also be very useful to measure and meet our scalability goals.

  • Pod startup latency histogram (from sync to running)
  • Run container histogram
  • Image pull histogram
  • Current tasks counter
  • Restarts per task counter

We should use Prometheus metrics as the master has started to use them too (see @a-robinson's PRs in that area).
@vmarmol vmarmol added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability. sig/node Categorizes an issue or PR as relevant to SIG Node. labels Feb 19, 2015
@dchen1107 dchen1107 added this to the v1.0 milestone Feb 19, 2015
@dchen1107
Copy link
Member

Also see #1625 for more discussion

@vmarmol
Copy link
Contributor Author

vmarmol commented Feb 19, 2015

We'll use this issue for the node components of that umbrella issue.

@pires
Copy link
Contributor

pires commented Feb 20, 2015

+1

vmarmol added a commit to vmarmol/kubernetes that referenced this issue Feb 24, 2015
@vmarmol
Adding image pull latency metric.
c40a60e 
Also adding TODOs for other metrics.

Part of kubernetes#4604.
@vmarmol vmarmol mentioned this issue Feb 24, 2015
Merged
@vmarmol vmarmol mentioned this issue Feb 25, 2015
Merged
vmarmol added a commit to vmarmol/kubernetes that referenced this issue Feb 25, 2015
@vmarmol
Add Kubelet metrics for pod and container counts.
9546434 
Part of kubernetes#4604.
vmarmol added a commit to vmarmol/kubernetes that referenced this issue Feb 26, 2015
@vmarmol
Adding sync pod latency metric.
3f85ba1 
Latency is broken down by create, update, and sync pods.

Part of #kubernetes#4604.
vmarmol added a commit to vmarmol/kubernetes that referenced this issue Feb 26, 2015
@vmarmol
Adding sync pod latency metric.
744f33d 
Latency is broken down by create, update, and sync pods.

Part of kubernetes#4604.
@alex-mohr alex-mohr modified the milestones: Preview, v1.0 Feb 26, 2015
nikhiljindal pushed a commit to nikhiljindal/kubernetes that referenced this issue Feb 26, 2015
@vmarmol @nikhiljindal
Add Kubelet metrics for pod and container counts.
36a6ac7 
Part of kubernetes#4604.
nikhiljindal pushed a commit to nikhiljindal/kubernetes that referenced this issue Feb 26, 2015
@vmarmol @nikhiljindal
Adding sync pod latency metric.
9c8a803 
Latency is broken down by create, update, and sync pods.

Part of kubernetes#4604.
vmarmol added a commit to vmarmol/kubernetes that referenced this issue Feb 27, 2015
@vmarmol
Adding sync pod latency metric (again).
ed0f588 
Latency is broken down by create, update, and sync pods.

Part of kubernetes#4604.
@vmarmol vmarmol mentioned this issue Feb 27, 2015
Merged
vmarmol added a commit to vmarmol/kubernetes that referenced this issue Feb 27, 2015
@vmarmol
Adding metrics to measure latency of Docker operations.
cbf9382 
Part of kubernetes#4604.
@vmarmol vmarmol mentioned this issue Feb 27, 2015
Merged
@vmarmol
Copy link
Contributor Author

vmarmol commented Feb 27, 2015

After the slew of PRs, this should be complete. The Kubelet now exports a /metrics endpoint with a slew of metrics tracking the timing of certain operations and the counds of pods, containers, and containers per pod.

@vmarmol vmarmol closed this as completed Feb 27, 2015
@a-robinson a-robinson mentioned this issue Mar 16, 2015
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vmarmol vmarmol

Labels
area/introspection priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/scalability Categorizes an issue or PR as relevant to SIG Scalability.
Projects
None yet
Milestone
v1.0
Development

No branches or pull requests
4 participants
@vmarmol @pires @dchen1107 @alex-mohr