Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v2] Add legacy formats into e2e kafka integration tests #5824

Merged
merged 16 commits into from
Aug 18, 2024

Conversation

joeyyy09
Copy link
Member

@joeyyy09 joeyyy09 commented Aug 9, 2024

Description of the changes

  • Add Kafka tests for legacy Jaeger formats, because people migrating from v1 are not able to start using otlp in v1

Checklist

Signed-off-by: joeyyy09 <menteharshith@gmail.com>
@joeyyy09 joeyyy09 changed the title Add legacy formats into the tests Add legacy formats into the e2e kafka integration tests Aug 9, 2024
Copy link

codecov bot commented Aug 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.79%. Comparing base (4e6c510) to head (a572ce4).
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5824   +/-   ##
=======================================
  Coverage   96.79%   96.79%           
=======================================
  Files         342      342           
  Lines       16523    16523           
=======================================
  Hits        15994    15994           
  Misses        341      341           
  Partials      188      188           
Flag Coverage Δ
badger_v1 8.05% <ø> (ø)
badger_v2 1.81% <ø> (ø)
cassandra-3.x-v1 16.61% <ø> (ø)
cassandra-3.x-v2 1.74% <ø> (ø)
cassandra-4.x-v1 16.61% <ø> (ø)
cassandra-4.x-v2 1.74% <ø> (ø)
elasticsearch-6.x-v1 18.82% <ø> (+0.04%) ⬆️
elasticsearch-7.x-v1 18.89% <ø> (+0.06%) ⬆️
elasticsearch-8.x-v1 19.08% <ø> (+0.06%) ⬆️
elasticsearch-8.x-v2 1.81% <ø> (+0.01%) ⬆️
grpc_v1 9.52% <ø> (ø)
grpc_v2 7.14% <ø> (+0.01%) ⬆️
kafka-v1 9.74% <ø> (ø)
kafka-v2 1.81% <ø> (ø)
memory_v2 1.81% <ø> (ø)
opensearch-1.x-v1 18.94% <ø> (+0.06%) ⬆️
opensearch-2.x-v1 18.94% <ø> (+0.04%) ⬆️
opensearch-2.x-v2 1.81% <ø> (ø)
unittests 95.27% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@joeyyy09 joeyyy09 marked this pull request as ready for review August 9, 2024 17:11
@joeyyy09 joeyyy09 requested a review from a team as a code owner August 9, 2024 17:11
@joeyyy09 joeyyy09 requested a review from jkowall August 9, 2024 17:11
@dosubot dosubot bot added the storage/kafka label Aug 9, 2024
@yurishkuro yurishkuro added the changelog:ci Change related to continuous integration / testing label Aug 9, 2024
joeyyy09 and others added 7 commits August 11, 2024 10:46
…ncoding

Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Copy link
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am surprised this works without changing the topic name for each test - doesn't it create a conflict with the same topic used for different data format?

cmd/jaeger/internal/integration/kafka_test.go Outdated Show resolved Hide resolved
cmd/jaeger/internal/integration/kafka_test.go Outdated Show resolved Hide resolved
@joeyyy09
Copy link
Member Author

I am surprised this works without changing the topic name for each test - doesn't it create a conflict with the same topic used for different data format?

Yeah, my next plan would be to generate random topic names to avoid any future conflicts but most probably the reason why it's not throwing up errors is that the tests are not running in parallel, there's no overlap in the reading and writing of messages to the Kafka topic. I'm assuming our sequential execution ensures that the data format in the topic aligns with the format expected by the test.

Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Copy link
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code mostly looks good, any idea why it's failing? It was not clear from the run's logs.

cmd/jaeger/internal/integration/kafka_test.go Show resolved Hide resolved
@joeyyy09
Copy link
Member Author

The code mostly looks good, any idea why it's failing? It was not clear from the run's logs.

Not sure why the test fails at this point, I'll try to debug and see if I can get something from the logs. Else, I'll revisit the code and come up with a different logic or approach.

Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: joeyyy09 <menteharshith@gmail.com>
@joeyyy09
Copy link
Member Author

joeyyy09 commented Aug 16, 2024

The code mostly looks good, any idea why it's failing? It was not clear from the run's logs.

Not sure why the test fails at this point, I'll try to debug and see if I can get something from the logs. Else, I'll revisit the code and come up with a different logic or approach.

The Jaeger thrift and OTLP JSON for some reason fail, from the logs can say that the v2 binary refuses to start. So I'm assuming that there's something off with the configs or maybe the formats aren't supported the way we've been trying to use. I'll see if I can come up with something more accurate.

yurishkuro and others added 2 commits August 17, 2024 19:59
Signed-off-by: Yuri Shkuro <github@ysh.us>
@yurishkuro yurishkuro changed the title Add legacy formats into the e2e kafka integration tests Add legacy formats into e2e kafka integration tests Aug 18, 2024
@yurishkuro yurishkuro added changelog:exprimental Change to an experimental part of the code and removed changelog:ci Change related to continuous integration / testing labels Aug 18, 2024
@yurishkuro yurishkuro changed the title Add legacy formats into e2e kafka integration tests [v2] Add legacy formats into e2e kafka integration tests Aug 18, 2024
@yurishkuro yurishkuro added the v2 label Aug 18, 2024
@yurishkuro yurishkuro enabled auto-merge (squash) August 18, 2024 00:02
@yurishkuro yurishkuro merged commit 34a8aa5 into jaegertracing:main Aug 18, 2024
48 checks passed
@joeyyy09 joeyyy09 deleted the legacy-format branch August 18, 2024 05:41
JaredTan95 pushed a commit to JaredTan95/jaeger that referenced this pull request Aug 28, 2024
…ng#5824)

## Description of the changes
- Add Kafka tests for legacy Jaeger formats, because people migrating
from v1 are not able to start using otlp in v1

## Checklist
- [x] I have read
https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
- [x] I have signed all commits
- [x] I have added unit tests for the new functionality
- [x] I have run lint and test steps successfully
  - for `jaeger`: `make lint test`
  - for `jaeger-ui`: `yarn lint` and `yarn test`

---------

Signed-off-by: joeyyy09 <menteharshith@gmail.com>
Signed-off-by: Yuri Shkuro <github@ysh.us>
Co-authored-by: Yuri Shkuro <github@ysh.us>
Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog:exprimental Change to an experimental part of the code storage/kafka v2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants