-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add real-time counter for fuzz/invariant runs #585
Comments
I could see the run count, along with mean and median too - implemented in this line. Am I missing something here, or this feature is already implemented? |
Oh, you are right. Perhaps the request is probably moreso the real-time update of the counter (if supported by your terminal) than the count itself? Generally we should work on streaming in test results where possible which is (sort of) tracked in #141, so I'll close this as a duplicate. Do let me know if something was misunderstood here! |
Ah yea it does show the number of fuzz runs after a run already. The main motivation for opening this issue was to show a counter during the fuzz runs. I recently ran 10k fuzz runs with ffi, so it was quite slow, and it looks like your tests are hanging since there's nothing printed to the terminal during execution. A real-time fuzz run counter seems to be a simpler scope than #141, and I don't see this functionality mentioned in there (though I only skimmed all the comments), so I feel like they could be tracked separately. But either way is fine with me 🙂 |
I agree it would be valuable to have some sort of real-time counter. Let's track it here then 😅 Can you update the original issue to reflect it? |
done! |
I would love to see this for invariants |
This would also be useful for long-winded end-to-end fork tests that also fuzz inputs, since they can take a long time to run. |
We should add coverage % for invariant runs. Basically, record locations of
|
dapptools has a counter like the one in the screenshot below so you can track progress in fuzz runs while those fuzz runs are executing. I recently ran a 10k fuzz run test that uses ffi, so it was quite slow and appears as if tests hang. A counter that updates live like the one below would be helpful for monitoring fuzz/invariant run progress and having confidence tests are still running as expected
The text was updated successfully, but these errors were encountered: