Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Remove test spam Continued #15881

Merged
merged 5 commits into from
Jul 23, 2020

Conversation

thedeveloperr
Copy link
Collaborator

#1587 This is the 3rd PR in series of PR which aims to hunt and remove spam from test-backend output.

Testing Plan:

GIFs or Screenshots:

Uses assertLogs to prevent spam in test-backend output.
This avoids spam in test-backend output.
This removes spam in test-backend output caused by print statement.
This is to avoid spam in test-backend output.
This will prevent it to propagating to root logger and causing flaky
behavior in tests which verify logs by root logger using assertLogs.
@timabbott timabbott merged commit 7bbba74 into zulip:master Jul 23, 2020
@timabbott
Copy link
Member

These changes are all great, merged, thanks @thedeveloperr!

How close are we to finishing this effort?

@thedeveloperr
Copy link
Collaborator Author

@timabbott

  1. Few log verify are causing failures which I can't figure out why they are happening
  2. I noticed few new spam which was not coming before I started this, I am not sure why this new spam is coming now and not before, maybe buffering issue in stdout, stderr ? I will check it

@thedeveloperr
Copy link
Collaborator Author

test: Running zerver.tests.test_logging_handlers.RateLimitFilterTest.test_recursive_filter_handling
 spam: 2020-07-23 06:11:50.989 ERR  [] Log an error to trigger recursive filter() calls in _RateLimitFilter.

2020-07-23 06:11:50.989 ERR  [] Log an error to trigger initial _RateLimitFilter.filter() call.

test: Running zerver.tests.test_push_notifications.PushBouncerNotificationTest.test_push_bouncer_api
 spam: 2020-07-23 06:13:04.687 ERR  [django.request] Bad Gateway: /json/users/me/apns_device_token

2020-07-23 06:13:04.703 ERR  [django.request] Bad Gateway: /json/users/me/apns_device_token

2020-07-23 06:13:04.753 ERR  [django.request] Bad Gateway: /json/users/me/android_gcm_reg_id

2020-07-23 06:13:04.766 ERR  [django.request] Bad Gateway: /json/users/me/android_gcm_reg_id

These are the stuff which i am not able to solve right now

@thedeveloperr
Copy link
Collaborator Author

@timabbott 14 more spam found (few of them just started appearing in the output, they were not appearing before) I think 1 or 2 day more. The slowest step in doing this is running the test again and again. Running single module test is not enough because single module produces different spam and that spam doesn't appear when running test for all modules.
Also some extra spam comes when parallel flag have value > 1

@timabbott
Copy link
Member

For zerver.tests.test_push_notifications.PushBouncerNotificationTest.test_push_bouncer_api, I'll just do a quick fix; it's just a loop: 4a7eb47.

@mateuszmandera may be able to help with adding an appropriate assertLogs for test_recursive_filter_handling.

Some spam may be nondeterministic; happy to help figure out a plan for any of those. But I'm excited we're close!

@thedeveloperr
Copy link
Collaborator Author

For zerver.tests.test_push_notifications.PushBouncerNotificationTest.test_push_bouncer_api, I'll just do a quick fix; it's just a loop: 4a7eb47.

@timabbott I made a similar fix but it was failing sometimes. I am not sure why. 9861937

@timabbott
Copy link
Member

It's possible the issue is the way you nested things? I guess we'll see if the version I did fails; I wouldn't expect it to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants