libopenblas64_.so is by far the biggest fish here. For a regular x86-64 manylinux wheel, we're at 14.8 MB for numpy 1.26.2, and >50% of that is in numpy.libs.

If you build a wheel without vendored libopenblas (through python -m build), then it's about 7.8 MB. If you unzip it, you find the following:

• total unzipped size: 28.9 MB
• _multiarray_umath.so: 8.2 MB
• _simd.so: 3.4 MB
• _core/tests/: 3.3 MB
• all other tests/ combined: ~3.4 MB
• numpy/random/*.so: 3.2 MB

It looks like separating out the tests could be the easiest and most significant win after separating out libopenblas. The two static libraries are only ~200 kb together so not that relevant.

Would we make the tests optionally installable, or how would users validate that their NumPy installation is working correctly?

Would we make the tests optionally installable, or how would users validate that their NumPy installation is working correctly?

We could publish a numpy-tests package indeed. And/or a build-time switch like -Dinclude-tests=true.

So the idea would be that most "only CI testing" people would get a non-optimized NumPy without tests, and "normal users" would download OpenBLAS?

I don't see a realistic way to do that, one cannot distinguish between CI jobs and other regular uses. Also, it's not at all clear that most usage is for CI. It seems unlikely actually - all we know is that ~90% of jobs are for Linux. 200 million CI jobs/month would be a lot. The distribution over Python versions, with 3.12 still very low 4 months after it was released, rather suggests that it's large-scale production deployments of applications and data pipelines:

Another alternative for the test suite, if we want it to be included by default, is to compress it.

I think I've figured out how to remove the tests for pandas, if you're interested.

pandas-dev/pandas#53007 is my attempt.

Basically my strategy is use .gitattributes to remove tests from the sdist (and use setuptools to build wheels for the tests separetely), then build the wheel from the sdist without tests.

Interesting, thanks for sharing @lithomas1.

I'd be more inclined to keep tests in the sdist (distros need it after all) and use a build flag to allow removing tests from wheels. But I'm certainly not sure that that's the way to go before trying.

Another alternative for the test suite, if we want it to be included by default, is to compress it.

That could help a bit, but probably not all that much compared to not shipping them - wheels are already compressed after all (they're zip archives).

Yeap, sorry for the noise. Compression advantage of something like zstd over gz is only a few 100k.

I wish wheels supported better compression. Looks like they considered it but never got zstd or similar into the standard lib.

pandas-dev/pandas#53007 is my attempt.

@lithomas1 I think we (@czgdp1807 and I) have just figured out a much cleaner way of doing this, using install tags, which are specifically meant for this kind of thing. I think the diff will end up being quite small, and it retains the flexibility to install tests yes/no with a single CLI flag. It should also allow producing two wheels (separate numpy_tests one for the tests package) without a rebuild. You may want to hold off on merging your Pandas PR to see whether this will work out for numpy. We should have a PR to look at sometime within the next week.

gh-26274 allows removing the tests, and looks quite clean. Reduction in wheel sizes will be 1.6 MB.

A very brief summary of the benefits of binary size reductions:

• The reduction in PyPI network traffic is very large. The reductions from dropping the tests (~1.6 MB wheel size change) alone will be ~4 PB at current download rates, which is massive. The CDN egress cost of that at current commercial rates (looks like ~$0.025/GB from glancing at AWS et al. pricing pages) is $100,000/year. Sponsored by PyPI's CDN providers so not a real cost for the community now, but at least a liability. And it also hints at the environmental costs, which is important.
• Usability for the average user improves, smaller is better (faster environment construction etc.)
• Conda-forge split out the tests from SciPy a year ago (Separate tests into their own package conda-forge/scipy-feedstock#184), and I don't think we've seen any complaints.

Reducing the size of the vendored OpenBLAS wheel (xref MacPython/openblas-libs#144) will help more, and succeeding in splitting it off completely even more. The macOS Accelerate wheels show what is possible: they are now (for 2.0.0rc1) 5.0-5.2 MB, and without tests they are 3.4-3.6 MB. The manylinux2014_x86_64 wheels for 2.0.0rc1 are 19.0-19.3 MB.

For packages that make pyinstaller bundles it's a big win making scipy and numpy smaller. Two lots of openblas that dont have to be bundled, tests that don't have to be shipped, etc. I think some of the binaries I was releasing were on the 100Mb level and higher.

I'm not a huge fan of the extra complexity re: separate test suite "package" FWIW, but looks like I'll be outvoted, fair enough.

_simd.so: 3.4 MB

Thanks to @Mousius I just realized that this is a test-only file, so we can remove it from the installed package as well if tests aren't requested. That's another 10% won, at least on x86-64 (gain on other platforms is significantly smaller).