Skip to content

Consolidate node_modules #112718

Closed
Closed
@alexdima

Description

Our build times are pretty bad. A large part is played by installing node modules or restoring them from a previous run.

Our extensions already use scripts defined in the vscode core (for compilation, webpacking, building, packaging), so they are already completely reliant on the core vscode package.json, etc. Having each extension have separate dev dependencies only increases the footprint of node modules (and duplicates a lot of node modules) for what is IMHO no componentization benefit.

cc @joaomoreno

Initial situation
PATH                                            BYTES    FILES
root                                      743_481_709    35813
build                                     172_837_622    11452
remote                                    172_608_051     1759
extensions/markdown-language-features      86_835_034     8632
extensions/typescript-language-features    71_349_956     1942
test/smoke                                 71_016_776     8903
extensions/microsoft-authentication        69_289_291     1368
extensions/github-authentication           69_204_063     1218
extensions/vscode-notebook-tests           63_824_873     2693
test/automation                            60_712_649     4728
test/integration/browser                   53_567_321      484
extensions                                 30_564_829       84
extensions/vscode-api-tests                27_829_723     3527
extensions/emmet                           17_208_956     4319
extensions/html-language-features/server   16_940_631     4387
extensions/git                             15_738_298     2793
extensions/vscode-colorize-tests           13_720_268     3508
extensions/css-language-features/server    11_319_654     2488
extensions/vscode-test-resolver            10_358_999     2237
extensions/css-language-features            7_960_524     2305
extensions/vscode-custom-editor-tests       7_806_176     1965
remote/web                                  5_728_559      267
extensions/github                           4_186_258      399
extensions/json-language-features           3_492_388      682
extensions/json-language-features/server    2_921_100      418
extensions/html-language-features           2_860_474      563
extensions/npm                              2_564_903      399
extensions/extension-editing                2_218_244      249
build/lib/watch                             1_249_570      512
extensions/image-preview                      881_680      235
extensions/configuration-editing              846_610       72
extensions/debug-auto-launch                  678_927       56
extensions/debug-server-ready                 678_927       56
extensions/grunt                              678_927       56
extensions/gulp                               678_927       56
extensions/jake                               678_927       56
extensions/merge-conflict                     678_927       56
extensions/php-language-features              678_927       56
extensions/git-ui                             654_993       50
TOTAL                                   1_826_532_671   110843

Final situation
----------------------------------
size saving: 22.65%
file count saving: 40.50%
----------------------------------
PATH                                             BYTES      FILES
root                                       798,750,218     37,863
remote                                     172,606,391      1,759
build                                       97,236,856      6,496
test/smoke                                  63,143,603      5,028
extensions/typescript-language-features     62,153,672        682
test/automation                             60,751,880      4,732
test/integration/browser                    53,607,097        488
extensions                                  30,564,829         84
extensions/html-language-features/server     6,336,406        418
extensions/emmet                             6,231,918        438
remote/web                                   5,728,559        267
extensions/css-language-features/server      4,889,165        342
extensions/markdown-language-features        4,771,902      1,140
extensions/git                               4,662,508        409
extensions/microsoft-authentication          4,557,879        667
extensions/github                            4,298,776        404
extensions/json-language-features            3,566,298        686
extensions/json-language-features/server     3,078,395        422
extensions/html-language-features            2,934,384        567
extensions/github-authentication             2,897,205        467
extensions/npm                               2,638,813        403
extensions/extension-editing                 2,292,154        253
extensions/css-language-features             2,223,462        339
build/lib/watch                              1,249,570        512
extensions/configuration-editing               920,520         76
extensions/vscode-colorize-tests               895,212         70
extensions/image-preview                       881,680        235
extensions/vscode-api-tests                    822,295         58
extensions/debug-auto-launch                   752,837         60
extensions/debug-server-ready                  752,837         60
extensions/grunt                               752,837         60
extensions/gulp                                752,837         60
extensions/jake                                752,837         60
extensions/merge-conflict                      752,837         60
extensions/php-language-features               752,837         60
extensions/vscode-custom-editor-tests          743,453         67
extensions/git-ui                              728,903         54
extensions/vscode-notebook-tests               728,903         54
extensions/vscode-test-resolver                728,903         54
TOTAL                                    1,412,891,668     65,954

Azure Pipeline node_modules cache size / time

OS Before After Time saved
Build 20201217.3 Build 20201222.14
Windows 363.1 MB / 1m49s 278.9 MB / 1m18s 31s
Linux 305.0 MB / 49s 224.9 MB / 33s 16s
macOS 308.1 MB / 1m02s 227.2 MB / 43s 19s

GitHub Actions node_modules cache size / time

OS Before After Time saved
Build 18621 Build 18835
Windows 373 MB / 1m38s 285 MB / 57s 41s
Linux 277 MB / 22s 199MB / 13s 9s
macOS 276 MB / 47s 196 MB / 34s 13s

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

engineeringVS Code - Build / issue tracking / etc.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions