Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin/trace: Fix zipkin json_v2 and #4160 #4180

Merged
merged 1 commit into from
Nov 10, 2020

Conversation

nyodas
Copy link
Contributor

@nyodas nyodas commented Oct 5, 2020

1. Why is this pull request needed and what does it do?

In #4171 i noticed that the client_server was not being used anymore.
It appears that it should still be there.

While testing that change i noticed that the zipkin provider was borked and couldn't push traces anymore
Tcpdump showed that the zipkin server was responding with

Expected a JSON_V1 encoded list, but received: JSON_V2

There was also an issue (#4160) with


	recorder, err := zipkin.NewEndpoint(t.serviceName, t.serviceEndpoint)

As t.serviceEndpoint is :53 in most cases
Zipkin can't parse the ip:port tuple and send a warning with

[WARNING] build Zipkin endpoint found err: lookup : no such host

cf: openzipkin/endpoint.go

To actively set a correct values in serviceEndpoint you need to use the bind plugin

.:53 {
    bind 127.0.0.1
    trace
}

2. Which issues (if any) are related?

#4160
#4109
#4171

3. Which documentation changes (if any) need to be made?

Indicate that the zipkin api v1 is not working as of 1.7.1

4. Does this introduce a backward incompatible change or deprecation?

Yes the api version of zipkin is now using v2
Though that fix a bug introduced with #4109

@@ -47,6 +47,8 @@ You can run Zipkin on a Docker host like this:
docker run -d -p 9411:9411 openzipkin/zipkin
```

:warning: The zipkin provider does not support the v1 API since coredns 1.7.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this useful information for the reader or can we just not mention it?

Copy link
Contributor Author

@nyodas nyodas Oct 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a "breaking" change compared to pre-1.7.1
I don't mind removing the note, I made it in response to this comment

@@ -35,7 +35,9 @@ func traceParse(c *caddy.Controller) (*trace, error) {
)

cfg := dnsserver.GetConfig(c)
tr.serviceEndpoint = cfg.ListenHosts[0] + ":" + cfg.Port
if cfg.ListenHosts[0] != "" {
tr.serviceEndpoint = cfg.ListenHosts[0] + ":" + cfg.Port
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we not add the port here? As you say in the description this doesn't work with zipkin...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Zipkin does understand the port when associated with a resolvable hostname / an IP, and there is always a port in the config.

I assume to track the the correct listener you need the right couple of bind + port

That would also work with no port but would be less precise

every uint64
serviceName string
clientServer bool
serviceEndpoint string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serviceEndpoint is empty in all tests? In general if you want to test a little new thing, just add a small new test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed it.

@codecov-commenter
Copy link

codecov-commenter commented Oct 5, 2020

Codecov Report

Merging #4180 into master will decrease coverage by 0.04%.
The diff coverage is 75.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4180      +/-   ##
==========================================
- Coverage   55.99%   55.94%   -0.05%     
==========================================
  Files         225      225              
  Lines        9930     9943      +13     
==========================================
+ Hits         5560     5563       +3     
- Misses       3905     3912       +7     
- Partials      465      468       +3     
Impacted Files Coverage Δ
plugin/trace/setup.go 66.66% <33.33%> (-2.69%) ⬇️
plugin/trace/trace.go 76.59% <100.00%> (-2.48%) ⬇️
plugin/ready/ready.go 75.75% <0.00%> (-9.43%) ⬇️
plugin/auto/auto.go 0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 275a62c...7601659. Read the comment docs.

@nyodas nyodas force-pushed the bob/fix-zipkin-trace branch 3 times, most recently from 0d3e362 to 7601659 Compare October 5, 2020 20:27
@nyodas nyodas requested a review from miekg October 6, 2020 18:15
@johnbelamaric
Copy link
Member

@nyodas can you resolve conflicts please?

@nyodas nyodas force-pushed the bob/fix-zipkin-trace branch from 7601659 to 1940c43 Compare October 26, 2020 23:46
@nyodas
Copy link
Contributor Author

nyodas commented Oct 26, 2020

Done.
My mistake for all the new reviewer.
My rebase added many commit unrelated to this and auto-added Codeowner as reviewer.
I can't seem to remove the new ones :/

In coredns#4171 i noticed that the client_server was not being used anymore.
It appears that it should still be there.

While testing that change i noticed that the zipkin provider was borked and couldn't push traces anymore
Tcpdump showed that the zipkin server was responding with

```
Expected a JSON_V1 encoded list, but received: JSON_V2
```

There was also an issue (coredns#4160)  with
```

	recorder, err := zipkin.NewEndpoint(t.serviceName, t.serviceEndpoint)
```
As `t.serviceEndpoint` is `:53` in most cases
Zipkin can't parse the ip:port tuple and send a warning with
```
[WARNING] build Zipkin endpoint found err: lookup : no such host
```
cf: [openzipkin/endpoint.go](https://github.com/openzipkin/zipkin-go/blob/master/endpoint.go#L27-L80)

To actively set a correct values in serviceEnpoint you need to use the bind plugin
```
.:53 {
    bind 127.0.0.1
    trace
}
```

Signed-off-by: bob <bob.bouteillier@datadoghq.com>
@nyodas nyodas force-pushed the bob/fix-zipkin-trace branch from 1940c43 to 36957e5 Compare October 26, 2020 23:57
@codecov-io
Copy link

Codecov Report

Merging #4180 into master will decrease coverage by 0.00%.
The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4180      +/-   ##
==========================================
- Coverage   56.05%   56.04%   -0.01%     
==========================================
  Files         223      223              
  Lines        9850     9852       +2     
==========================================
+ Hits         5521     5522       +1     
  Misses       3867     3867              
- Partials      462      463       +1     
Impacted Files Coverage Δ
plugin/trace/setup.go 66.21% <33.33%> (-2.28%) ⬇️
plugin/trace/trace.go 67.30% <100.00%> (-3.29%) ⬇️
plugin/kubernetes/metrics.go 100.00% <0.00%> (+23.07%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e37750c...36957e5. Read the comment docs.

@mindw
Copy link
Contributor

mindw commented Nov 9, 2020

@nyodas The bind workaround + explicit URL works with 1.8.0:

. {
    ...

    bind 0.0.0.0

    # enable to output traces to jaeger
    trace zipkin http://{$HOST_IP}:30411/api/v2/spans {
      every 100
      client_server
    }
   ...
}

@johnbelamaric
Copy link
Member

Thanks!

/lgtm
/merge

Copy link

@corbot corbot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by johnbelamaric

@corbot corbot bot merged commit b781420 into coredns:master Nov 10, 2020
@nyodas nyodas deleted the bob/fix-zipkin-trace branch November 12, 2020 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants