Skip to content

proposal: net/http: add support for RFC8878 Content-Encoding: zstd to http.DefaultTransport #62492

Open
@Jorropo

Description

Currently the default http transport will magically send Accept-Encoding: gzip header and handle decompression if the consumer is not handling that themselves.
Chrome recently added opt-in support for Content-Encoding: zstd https://chromestatus.com/feature/6186023867908096.

I propose that the default transport sends Accept-Encoding: gzip, zstd and handles the decompression following the same logic that gzip have.


Implementation wise internal/zstd's performance sounds good enough, for a rough estimation I copied the BenchmarkLarge from internal/zstd and made a compress/gzip version, the compression ratio were 3.21 vs 3.24 respectively (very comparable) and on my Ryzen 3600 CPU the output's throughput are really similar:

name      gzip time/op    zstd time/op    delta
Large-12    3.08ms ± 4%    3.15ms ± 2%    +2.02%  (p=0.008 n=15+13)

name      gzip speed      zstd speed      delta
Large-12   184MB/s ± 3%   194MB/s ± 2%    +5.41%  (p=0.000 n=15+13)

name      gzip alloc/op   zstd alloc/op   delta
Large-12    7.54kB ± 0%    5.52kB ± 3%   -26.79%  (p=0.000 n=15+13)

name      gzip allocs/op  zstd allocs/op  delta
Large-12      40.0 ± 0%       0.0       -100.00%  (p=0.000 n=15+15)

Neither is a particularly high performance implementation, but if compress/gzip is good enough zstd should also be (except maybe for memory usage ?).


Adding internal/zstd to a binary adds 469KiB with 2186909. I guess this could be optimized by sharing some of the lz77 routines between gzip and zstd if this were a concern.
This is true but missleading, most of it is reflect.
Tested with (GOOS=linux GOARCH=amd64 GOAMD64=v3 and without stripping debug information):

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    • Status

      Incoming

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions