Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regen tasks should only overwrite existing files if successful #2454

Open
Manishearth opened this issue Aug 25, 2022 · 2 comments
Open

Regen tasks should only overwrite existing files if successful #2454

Manishearth opened this issue Aug 25, 2022 · 2 comments
Labels
C-test-infra Component: Integration test infrastructure good first issue Good for newcomers help wanted Issue needs an assignee S-small Size: One afternoon (small bug fix or enhancement) T-enhancement Type: Nice-to-have but not required
Milestone
Backlog ⟨P4⟩

Comments

@Manishearth
Copy link
Member

We have a bunch of regen-style tasks (testdata, diplomat-gen, fingerprint, ffi-coverage) where we generate a bunch of files on top of checked-in files (and typically, check the difference on CI)

They typically work by deleting the existing files, running the task, and then overwriting the file. This means that if the task fails, the files will be in a weird state that often needs to be reset from git. Furthermore, for testdata, git status is basically useless during the long-running testdata regeneration.

I think these tasks should run the generation to some temporary folder first, and only if the generation step succeeds, delete the old files and copy over new ones.
@Manishearth Manishearth added T-enhancement Type: Nice-to-have but not required good first issue Good for newcomers help wanted Issue needs an assignee C-test-infra Component: Integration test infrastructure S-small Size: One afternoon (small bug fix or enhancement) labels Aug 25, 2022
@sffc
Copy link
Member

sffc commented Aug 26, 2022

Questions:

  1. Should we delete the temp directory if codegen failed, or not delete it for debugging purposes but leak disk space?
  2. How to make it work consistently on Windows and Unix?

@Manishearth
Copy link
Member Author

I think it's fine to delete if codgen failed, but we can have an env var disable it perhaps.

I assume duckscript has enough stuff to make this work cross platform.

@sffc sffc added the backlog label Sep 8, 2022
@sffc sffc added this to the Backlog milestone Dec 22, 2022
@sffc sffc removed the backlog label Dec 22, 2022
@sffc sffc mentioned this issue Feb 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-test-infra Component: Integration test infrastructure good first issue Good for newcomers help wanted Issue needs an assignee S-small Size: One afternoon (small bug fix or enhancement) T-enhancement Type: Nice-to-have but not required
Projects
None yet
Milestone
Backlog ⟨P4⟩
Development

No branches or pull requests
2 participants
@sffc @Manishearth