Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new Servicegraph visualization. #3318

Merged
merged 2 commits into from
Feb 9, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ $(MIXER_GO_BINS):
bin/gobuild.sh $@ istio.io/istio/pkg/version ./mixer/cmd/$(@F)

servicegraph:
bin/gobuild.sh $@ istio.io/istio/pkg/version ./mixer/example/servicegraph/cmd/server
bin/gobuild.sh $@ istio.io/istio/pkg/version ./addons/servicegraph/cmd/server

${ISTIO_OUT}/servicegraph:
bin/gobuild.sh $@ istio.io/istio/pkg/version ./addons/$(@F)/cmd/server
Expand Down
56 changes: 36 additions & 20 deletions addons/servicegraph/README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,44 @@
# Istio ServiceGraph
# Istio Servicegraph

*WARNING WARNING WARNING WARNING*
Servicegraph is a small app that generates and visualizes graph
representations of your Istio service mesh. Servicegraph is dependent
on the
[Prometheus](https://istio.io/docs/tasks/telemetry/querying-metrics.html)
addon and the standard metrics configuration. The documentation for
deploying and using Servicegraph is
[here](https://istio.io/docs/tasks/telemetry/servicegraph.html).

These services are examples ONLY. This code may change at will, or be removed
entirely without warning. Taking any dependency on this code is done at your own
peril.
## Visualizations

## Services
- `/force/forcegraph.html` is an interactive
[D3.js](https://d3js.org/) visualization.

### Servicegraph service
- `/dotviz` is a static [Graphviz](https://www.graphviz.org/)
visualization.

Defined in `servicegraph/cmd/server`, this provides a basic HTTP API for
generating servicegraphs. It exposes the following endpoints:
- `/graph` which provides a JSON serialization of the servicegraph
- `/dotgraph` which provides a dot serialization of the servicegraph
- `/dotviz` which provides a visual representation of the servicegraph
## Serializations

All endpoints take an optional argument of `time_horizon`, which controls the
timespan to consider for graph generation.
- `/dotgraph` provides a
[DOT](https://en.wikipedia.org/wiki/DOT_(graph_description_language))
serialization.

All endpoints also take an optional arugment of `filter_empty=true`, which will
restrict the nodes and edges shown to only those that reflect non-zero traffic
levels during the specified `time_horizon`.
- `/d3graph` provides a JSON serialization for D3 visualization.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder: do we need two JSON serializations for the same graph?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was more comfortable doing the transformation in Go. I think the core Static/Dynamic graph structure is better for internal representation than the one that D3 wants, so I didn't want to change that. The /graph endpoint could be removed, as it is not used, but It is trivial to keep it, and someone might find a way to use it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. fair enough.


### Demosvc service
Defined in `servicegraph/cmd/demosvc`, this provides a simple HTTP endpoint that
generates prometheus metrics. This can be used to test the servicegraph service.
- `/graph` provides a JSON serialization.

## Query Parameters

All endpoints take these query parameters:

- `time_horizon` controls the timespan to consider for graph
generation. Format is a number plus a time unit. Example `15s` or
`1m`. Default is `5m`.

- `filter_empty=true` will restrict the nodes and edges shown to only
those that reflect non-zero traffic levels during the specified
`time_horizon`. Deafult is `false`.

# Demosvc service
Defined in `servicegraph/cmd/demosvc`, this provides a simple HTTP
endpoint that generates Prometheus metrics. This can be used to test
the servicegraph service.
3 changes: 2 additions & 1 deletion addons/servicegraph/cmd/server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ type state struct {
func main() {
bindAddr := flag.String("bindAddr", ":8088", "Address to bind to for serving")
promAddr := flag.String("prometheusAddr", "http://localhost:9090", "Address of prometheus instance for graph generation")
assetDir := flag.String("assetDir", "example/servicegraph", "directory find assets to serve")
assetDir := flag.String("assetDir", "./", "directory find assets to serve")
flag.Parse()

s := &state{staticGraph: &servicegraph.Static{Nodes: make(map[string]struct{})}}
Expand All @@ -87,6 +87,7 @@ func main() {
http.HandleFunc("/node", s.addNode)
http.Handle("/dotgraph", promgen.NewPromHandler(*promAddr, s.staticGraph, dot.GenerateRaw))
http.Handle("/dotviz", promgen.NewPromHandler(*promAddr, s.staticGraph, dot.GenerateHTML))
http.Handle("/d3graph", promgen.NewPromHandler(*promAddr, s.staticGraph, servicegraph.GenerateD3JSON))

log.Printf("Starting servicegraph service at %s", *bindAddr)
log.Fatal(http.ListenAndServe(*bindAddr, nil))
Expand Down
82 changes: 82 additions & 0 deletions addons/servicegraph/d3graph.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
// Copyright 2017 Istio Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package servicegraph defines the core model for the servicegraph service.
package servicegraph

import (
"encoding/json"
"errors"
"io"
)

type (
// d3Graph is a graph representation for JSON serialization to be
// consumed easily by the D3.js library.
d3Graph struct {
Nodes []d3Node `json:"nodes"`
Links []d3Link `json:"links"`
}

d3Node struct {
Name string `json:"name"`
}

d3Link struct {
Source int `json:"source"`
Target int `json:"target"`
Labels Attributes `json:"labels"`
}
)

func indexOf(nodes []d3Node, name string) (int, error) {
for i, v := range nodes {
if v.Name == name {
return i, nil
}
}
return 0, errors.New("invalid graph")
}

// GenerateD3JSON converts the standard Dynamic graph to d3Graph, then
// serializes to JSON.
func GenerateD3JSON(w io.Writer, g *Dynamic) error {
graph := d3Graph{
Nodes: make([]d3Node, 0, len(g.Nodes)),
Links: make([]d3Link, 0, len(g.Edges)),
}
for k := range g.Nodes {
n := d3Node{
Name: k,
}
graph.Nodes = append(graph.Nodes, n)
}
for _, v := range g.Edges {
s, err := indexOf(graph.Nodes, v.Source)
if err != nil {
return err
}
t, err := indexOf(graph.Nodes, v.Target)
if err != nil {
return err
}
l := d3Link{
Source: s,
Target: t,
Labels: v.Labels,
}
graph.Links = append(graph.Links, l)
}
return json.NewEncoder(w).Encode(graph)
}
3 changes: 2 additions & 1 deletion addons/servicegraph/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ FROM scratch

WORKDIR /tmp/
COPY servicegraph /usr/local/bin/
COPY viz /tmp/js/viz/
COPY js /tmp/js/
COPY force /tmp/force/

EXPOSE 8088
ENTRYPOINT ["/usr/local/bin/servicegraph", "--assetDir=/tmp"]
3 changes: 2 additions & 1 deletion addons/servicegraph/docker/Dockerfile.debug
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ FROM gcr.io/istio-testing/ubuntu_xenial_debug:3f57ae2aceef79e4000fb07ec850bbf4bc

WORKDIR /tmp/
COPY servicegraph /usr/local/bin/
COPY viz /tmp/js/viz/
COPY js /tmp/js/
COPY force /tmp/force/

EXPOSE 8088
CMD ["/usr/local/bin/servicegraph", "--assetDir=/tmp"]
24 changes: 0 additions & 24 deletions addons/servicegraph/docker/gcloud_build.sh

This file was deleted.

83 changes: 83 additions & 0 deletions addons/servicegraph/force/forcegraph.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
.background {
stroke: white;
stroke-width: 1px;
fill: white;
}

.node {
stroke: black;
stroke-width: 1.5px;
cursor: move;
fill: #466BB0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is way way down in the weeds, but I think we might want to consider a different color (non-red). would using istio blue make sense?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is Istio blue now, I had it as "lightcoral" before.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ha. ok. i need to learn hex colors ;).

}

.node.darken {
stroke: grey;
}

.link {
fill: none;
stroke: #000;
stroke-width: 3px;
opacity: 0.7;
marker-end: url(#end-arrow);
}

.link.darken {
opacity: 0.3;
}

.label {
fill: white;
font-family: Verdana;
font-size: 14px;
text-anchor: middle;
cursor: move;
}

.label.darken {
fill: lightgrey;
}

#graph {
overflow-y: scroll;
}

#info {
height: 300px;
display: none;
overflow-y: scroll;
}

html, body, #total{
height: 100%;
width: 100%;
margin: 0;
font-family: Verdana;
font-size: 14px;
}

a {
font-size: 18px;
color: #3F51B5;
}

table {
border-collapse: collapse;
width: 80%;
table-layout: fixed;
}

table, th, td {
border: 1px solid black;
text-align: left;
padding: 4px;
}

.conn-table {
margin: 20px;
}

div.float {
position:absolute;
}
50 changes: 50 additions & 0 deletions addons/servicegraph/force/forcegraph.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
<!DOCTYPE html>
<head>
<title>Istio Service Graph</title>
<meta charset="utf-8">
<link rel="stylesheet" href="forcegraph.css">
</head>
<body>
<div id="title" class="float">
<h2>Istio Service Graph</h2>
Click on a service for traffic information
</div>
<div id="total">
<div id="graph"></div>
<div id="info"></div>
</div>
<script id="info-template" type="text/x-handlebars-template">
<a onclick="hideInfo()">Close</a>

<h2>{{name}}</h2>
<div id="incoming" class="conn-table"><table>
<tr>
<th>Incoming Connections</th>
<th>Reqs/sec</th>
</tr>
{{#each incoming}}
<tr>
<td>{{this.source}}</td>
<td>{{this.ops}}</td>
</tr>
{{/each}}
</table></div>

<div id="outgoing" class="conn-table"><table>
<tr>
<th>Outgoing Connections</th>
<th>Reqs/sec</th>
</tr>
{{#each outgoing}}
<tr>
<td>{{this.destination}}</td>
<td>{{this.ops}}</td>
</tr>
{{/each}}
</table></div>
</script>
<script src="/js/d3/d3.v4.min.js"></script>
<script src="/js/cola/cola.min.js"></script>
<script src="/js/handlebars/handlebars.min.js"></script>
<script src="forcegraph.js"></script>
</body>
Loading