Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a python script that does a size check for third_party. #5310

Closed
seanlip opened this issue Jul 16, 2018 · 7 comments
Closed

Create a python script that does a size check for third_party. #5310

seanlip opened this issue Jul 16, 2018 · 7 comments

Comments

@seanlip
Copy link
Member

seanlip commented Jul 16, 2018

This is a suggestion by @hoangviet1993 (thanks @hoangviet1993!):

Let's include a python script that does a size-check (i.e. number of files) for third_party. Should It get above , let's say 7k?, the Travis build should fail.

The aim of this is to prevent us accidentally breaching the 10k file limit on App Engine. See also #5201 which is related.

/cc @vojtechjelinek

@seanlip
Copy link
Member Author

seanlip commented Jul 16, 2018

Also /cc @apb7 since this relates to dev workflow.

@seanlip
Copy link
Member Author

seanlip commented Aug 10, 2018

Note: this is meant to be a count of files, not the size of the files.

@seanlip seanlip changed the title Create a shell script that does a size check for third_party. Create a python script that does a size check for third_party. Oct 4, 2018
@tonadev
Copy link
Contributor

tonadev commented Oct 6, 2018

Hi, is anyone working in this issue? If not, I would like to work on it. But I have some questions about the task. The explanation says that the script causes a crash in Travis build when there are more than 7k files, I was looking at the others scripts that make Travis build crash and they have a shell script that runs the python script and if it fails then the scripts fails and finally the .travis.yaml builds fails. My question is, the task includes creating that shell scrips and the code necessary in the .travis.yaml or only the python script? Or is there any other way of making the Travis build fail using only the python script? And also, does directories count or just the files?

@seanlip
Copy link
Member Author

seanlip commented Oct 7, 2018

Hi @tonadev, go for it!

The task includes creating whatever code is needed to achieve the desired result -- i.e. if we get to more than a certain file count then Travis should fail.

Also, only the files count, not the directories.

Thanks!

@hoangviet1993
Copy link
Contributor

hoangviet1993 commented Oct 7, 2018

I think you can include the check in install_third_party.py, after _install_third_party_libs() is run. Does it make sense? (or just run a separate python script after install_third_party.py has finished installing is fine too)

@seanlip
Copy link
Member Author

seanlip commented Oct 7, 2018

Hi @tonadev, there is a Travis build that runs on every PR. The build for that PR should catch the issue and prevent that PR from being merged (right?) So there shouldn't be a cascading effect on other PRs...

We could also just make this a regular presubmit check, but only if it doesn't take a long time, otherwise it slows down folks a lot.

I'm not sure whether including the check in install_third_party.py suffices though. The developer may just miss the error message...

@tonadev
Copy link
Contributor

tonadev commented Oct 7, 2018

I realized about that I was wrong, that why supposedly I deleted the comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants