Skip to content
This repository has been archived by the owner on Nov 8, 2022. It is now read-only.

Commit

Permalink
Updated documentation for profiling tools
Browse files Browse the repository at this point in the history 
Follow-up with the new profiling tools flag (#1306). We can now easily
profile Snap: daemon and plugins.
  • Loading branch information
@kindermoumoute
kindermoumoute committed Nov 10, 2016
1 parent 7f7af43 commit 0a7ad73
Showing 1 changed file with 66 additions and 60 deletions.
126 changes: 66 additions & 60 deletions docs/PROFILING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,24 @@

Using pprof and go-torch is a good way to bring to light on how your application performs. If you've never heard about profiling with the Go programming language, feel free to take a look at [Debugging performance issues in Go programs](https://software.intel.com/en-us/blogs/2014/05/10/debugging-performance-issues-in-go-programs).

## Set up ...
### FlameGraph
Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately. FlameGraph needs to be installed with Perl before trying to use go-torch.
## Getting started
If you have **Docker** installed, jump to [generate a profile](#generating-a-profile).

You can install Perl with your program manager.
### System Requirements
Nativaly supported OS:
- Linux
- OS X 10.11+ ([patch for 10.6 - 10.10](https://github.com/rsc/pprof_mac_fix))


### go-torch
In order to use go-torch, we need:
- FlameGraph.
- Perl (required for FlameGraph).


Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.

#### Install Perl
*`brew` on OS X*
```bash
brew install perl
Expand All @@ -21,7 +33,8 @@ apt install perl
yum install perl
```

Then download flamegraph.pl from the [flamegraph repository](https://github.com/brendangregg/FlameGraph):
#### Install FlameGraph
Then download `flamegraph.pl` from the [flamegraph repository](https://github.com/brendangregg/FlameGraph):
```
wget https://raw.githubusercontent.com/brendangregg/FlameGraph/master/flamegraph.pl
```
Expand All @@ -37,78 +50,71 @@ You'll also have to make flamegraph executable:
chmod 755 ~/bin/flamegraph
```

### graphviz
Graphviz will allow you to generate graphs with pprof.
*`brew` on OS X*
```bash
brew install graphviz
```
*`apt` on Debian or Ubuntu*
```bash
apt install graphviz
```
*`yum` on CentOS*
```bash
yum install graphviz
```

### go-torch

#### Install go-torch
(requires [Go](https://golang.org/doc/install))
```bash
go get github.com/uber/go-torch
```

## Generating a profile
Before exploiting any result with go-torch and pprof we need to generate a profile - in our case a CPU profile. In this example, we'll use the package `github.com/pkg/profile`.

### Implement the code
So on your main function start the profile:
```go
import "github.com/pkg/profile"

var p interface {
Stop()
}

func main() {
p = profile.Start(profile.CPUProfile, profile.ProfilePath("."), profile.NoShutdownHook)
...
}
### Run snapd
Start snapd with the `--pprof` flag:
```bash
snapd -t 0 --pprof
```

Then you must run `p.Stop()` just before exiting the program. In the snap daemon case it's just after handling a SIGINT (found in [snapd.go](../snapd.go)):
```go
func startInterruptHandling(modules ...coreModule) {
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, os.Kill, syscall.SIGTERM)

//Let's block until someone tells us to quit
go func() {
sig := <-c
p.Stop() // Stop profiling
...
```
Just doing that will create endpoints on the port used by Snap API. If you use the default port like in the example above, you should be able to access this url from your web browser: http://127.0.0.1:8181/debug/pprof

## Launch your program
Now launch your program - in this example, we started snapd and ran a task. The longer you run your task, the deeper your graph will go into the subroutines.
Next steps of this documentation will be focused on profiling tools, which is one part of what is exposed by `net/http/pprof` package. Find more information about other tools on the [official documentation](https://golang.org/pkg/net/http/pprof/#pkg-overview).

When your program quits you should have a `cpu.pprof` file generated in your current folder.
### Give some work to Snap
If you don't give work to Snap before generating profile, the **profile will be empty**. Some example of work:
- Run a task
- Send a lot of request to the API

(If the size of this file is 0 bytes, it means you didn't properly stop profiling with `p.Stop()` in your code.)
### Generate a profile
You can now start profiling for a defined period of time (e.g. 120 seconds):
```bash
curl "http://127.0.0.1:8181/debug/pprof/profile?seconds=120" > cpu.pprof
```

During this step you actually have to wait (here 120 seconds), while Snap is doing work. **The longer is the period, the more accurate is the profile.**

## Use pprof and go-torch
From your terminal, you can now generate the flame graphs. In this example we assume that the `snapd` binary and cpu.pprof are in the same folder:
```
go-torch --binaryinput cpu.pprof --binaryname snapd
From your terminal, you can now generate the flame graphs:

### Without Docker
```bash
go-torch --binaryinput cpu.pprof --binaryname `which snapd`
```

Same with pprof:
### For Docker user
```bash
cp `which snapd` .
docker run -v `pwd`:/tmp uber/go-torch --binaryinput /tmp/cpu.pprof --binaryname /tmp/snapd -p > torch.svg
```
go tool pprof snapd cpu.pprof

### Pprof tool
Once again, I highly encourage you to read about pprof in [this blog post](https://software.intel.com/en-us/blogs/2014/05/10/debugging-performance-issues-in-go-programs).
(requires [Go](https://golang.org/doc/install))
```bash
go tool pprof `which snapd` cpu.pprof
```

Note that pprof allows you to watch the profile of any Go program executed during the profiling:
## BONUS: using pprof tools for plugin
Plugins that use the [new Go library](https://github.com/intelsdi-x/snap-plugin-lib-go/tree/master/v1/plugin) will have their own endpoint.
```bash
$ snapctl plugin list --running
NAME HIT COUNT LAST HIT TYPE PPROF PORT
mock-grpc 54 Tue, 08 Nov 2016 13:08:38 PST collector 62143
statistics 266 Tue, 08 Nov 2016 13:08:38 PST processor 62148
influxdb 266 Tue, 08 Nov 2016 13:08:38 PST publisher
```
go tool pprof $SNAP_PLUGIN/snap-plugin-type-myplugin cpu.pprof

On the column "pprof port" you can find the port used by the plugin to expose pprof tools. e.g.:
```bash
$curl "http://127.0.0.1:62148/debug/pprof/profile?seconds=10" > cpu.pprof
```


If there is no port, it means that plugin is compiled with an old library, or it's not a Go plugin.

0 comments on commit 0a7ad73

Please sign in to comment.