Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement check retry thresholds and interval #17

Merged
merged 3 commits into from
Mar 25, 2022

Conversation

jacobtomlinson
Copy link
Member

@jacobtomlinson jacobtomlinson commented Mar 25, 2022

This PR factors the execution of the individual checks out into a dispatcher. This allowed cleaner implementation of the retry mechanisms. The configurable timeout, delay, thresholds and interval have now been implemented.

checks:
  - name: uid
    description: User ID is 1234
    probe:
      exec:
        command: [...]
      initialDelaySeconds: 0  # Delay after starting the container before the check should be run
      timeoutSeconds: 30  # Overall timeout for the check
      successThreshold: 1  # Number of times the check must pass before moving on
      failureThreshold: 1  # Number of times the check is allowed to fail before giving up
      periodSeconds: 1  # Interval between runs if threasholds are >1

One benefit is that now instead of having a flaky 10 second delay to wait for the container to start for the validation examples we can set the failure threshold to some high number and have the check retry every second. This resulted in the test suite going from 14 seconds to 4 seconds.

@jacobtomlinson jacobtomlinson added the enhancement New feature or request label Mar 25, 2022
Signed-off-by: Jacob Tomlinson <jtomlinson@nvidia.com>
Signed-off-by: Jacob Tomlinson <jtomlinson@nvidia.com>
@jacobtomlinson jacobtomlinson enabled auto-merge (squash) March 25, 2022 15:25
Signed-off-by: Jacob Tomlinson <jtomlinson@nvidia.com>
@jacobtomlinson jacobtomlinson merged commit 8996838 into NVIDIA:main Mar 25, 2022
@jacobtomlinson jacobtomlinson deleted the implement-retries branch March 25, 2022 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant