Skip to content
This repository has been archived by the owner on Sep 12, 2023. It is now read-only.

Timeout on response receiver and more logging #3209

Merged
merged 3 commits into from
Oct 20, 2022
Merged

Conversation

da-kami
Copy link
Contributor

@da-kami da-kami commented Oct 20, 2022

After some debugging with @luckysori we cannot understand why the receiver would sometimes not finish.
To make this more resilient we apply a timeout and add more logging to understand better what is happening.

Let's run this on testnet for a while to see how it behaves. This solution is not perfect, but it should be resilient enough to do the job.

This is relevant debugging information.
If we don't get a response on the receiver for `10` seconds we stop processing responses.
ew expect responses to arrive continuously because we process the requests in parallel.

Upon a timeout we print the amount of responses received (script histories received) and the requests sent (scripts processed).
Note that we still process the histories received knowing that we might not have all responses; it is still valid to update the script history for what we have receive.

Note that it *can* theoretically happen that a sender sends the responses to the receiver but the receiver is gone, but only if for some reason we are stuck while processing. We can't block indefinitely so this is a way forward for now. If we keep seeing a large amount of timeouts we will have to investigate further otherwise this solution is resilient enough for now, because we trigger sync frequently and the next sync should deliver the complete responses.
@da-kami da-kami changed the title More monitor logs Timeout on response receiver and more logging Oct 20, 2022
@da-kami da-kami marked this pull request as ready for review October 20, 2022 05:53
@da-kami
Copy link
Contributor Author

da-kami commented Oct 20, 2022

bors r+

@bors
Copy link
Contributor

bors bot commented Oct 20, 2022

Build succeeded:

@bors bors bot merged commit a784d12 into master Oct 20, 2022
@bors bors bot deleted the more-monitor-logs branch October 20, 2022 06:44
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants