Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the number of files that get deployed to production. #5201

Open
1 of 4 tasks
seanlip opened this issue Jul 6, 2018 · 3 comments
Open
1 of 4 tasks

Reduce the number of files that get deployed to production. #5201

seanlip opened this issue Jul 6, 2018 · 3 comments
Labels
enhancement Label to indicate an issue is a feature/improvement Impact: Medium Will improve quality-of-life for at least 30% of users. Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet.

Comments

@seanlip
Copy link
Member

seanlip commented Jul 6, 2018

(This was originally filed as #5182. Some immediate work has been done in #5196 -- thanks @vojtechjelinek! -- but there's still some more long term stuff to do, so I'm refiling.)

There is a limit of 10,000 on the number of files that can be uploaded to the production server. We have breached this limit enough times that it seems like it would be better to take active steps to control the number of files we deploy to production.

Here are some avenues for investigation:

  • Remove jquery-ui-themes altogether from manifest.json -- it looks like we don't need it. (I think @vojtechjelinek is doing this?)
  • Enforce that each dependency in manifest.json has a "bundle" option stating which of its files should be deployed to production. In general, each third-party library should not contribute more than one or two files to the release.
  • Find a way to whitelist the subset of files in MathJax that are deployed (note that the MathJax library weighs in at 2k+ files, and some of these are loaded from the JS on-demand).
  • We duplicate a bunch of files in build/ or backend_prod_files/, and we should not deploy those to production. We should create a reference doc explaining what we are doing in the build process so that we can exclude the unneeded files -- or, even better, find a way to organize things so that all the stuff we ignore / all the stuff we deploy automatically goes in a clearly-labelled folder.
@seanlip
Copy link
Member Author

seanlip commented Jul 6, 2018

Assigning @vojtechjelinek for the first checkbox.

@hoangviet1993
Copy link
Contributor

Another solution that I came across is to host our own "minified" version of each dependency on a separate Github. This will ensure a constant upstream as well as control over file count.

Updating the self-hosted dependency is going to be an issue. However, each dependency update is a little journey by itself so it should be treated with great care.

@hoangviet1993
Copy link
Contributor

hoangviet1993 commented Aug 25, 2018

At the point of writing this comment, app.yaml has a bunch of handler entries for dev mode development(bash scripts/start.sh without any flag).

Upon running production deployment, a different app.yaml (say we put it in /scripts/build/app.yaml to be copied to the final directory ) can be used in place of the current app.yaml to only use files within the final directory that contains all of the built files. This will help cut down the duplicated file count a ton.

Per Sean's recommendation:

But one thing we could do is include app.yaml in the build process and put it in the build/ folder? Then we deploy only from the build/ folder?
So the build/ (or whatever) folder becomes the new root for the production push

@U8NWXD U8NWXD moved this to To Do (Low Priority) in Developer Workflow Team Aug 1, 2022
@U8NWXD U8NWXD moved this from To Do (Low Priority) to Backlog in Developer Workflow Team Aug 16, 2022
@kevintab95 kevintab95 added the enhancement Label to indicate an issue is a feature/improvement label Aug 30, 2022
@U8NWXD U8NWXD added Impact: Medium Will improve quality-of-life for at least 30% of users. Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet. and removed talk-to: @seanlip labels Sep 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Label to indicate an issue is a feature/improvement Impact: Medium Will improve quality-of-life for at least 30% of users. Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet.
Projects
Status: Todo
Development

No branches or pull requests

7 participants