Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow compression of json response body using Accept-Encoding header #110349

Closed
ilackarms opened this issue Apr 17, 2017 · 16 comments
Closed

Allow compression of json response body using Accept-Encoding header #110349

ilackarms opened this issue Apr 17, 2017 · 16 comments
Labels
help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/backlog Higher priority than priority/awaiting-more-evidence. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@ilackarms
Copy link
Contributor

JSON payload returned by Kubernetes can be very large on GET requests when there are a lot of managed resources in the cluster.

Currently it is not possible to specify that large response bodies should be compressed; Accept-Encoding header is ignored by the apiserver.

Specifying the Accept-Encoding header on a GET request should cause the k8s apiserver to respond with response Body compressed with the given encoding sceme (e..g gzip, bzip2, etc.)

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 23, 2017
@cben
Copy link

cben commented Dec 27, 2017

ref issue #44164

This got implemented in #45666, reverted, and implemeted again (gated off by default) in #46966 (commit b74e5942e2a7ef9f73874b7e5a48c030ff7f1189 in this repo).

There is #51508 still open to enable it by default.

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 27, 2017
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 27, 2018
@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 26, 2018
@cben
Copy link

cben commented May 3, 2018

/remove-lifecycle rotten

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label May 3, 2018
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 1, 2018
@cben
Copy link

cben commented Aug 1, 2018 via email

@k8s-ci-robot k8s-ci-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Aug 1, 2018
@ilackarms
Copy link
Contributor Author

@dims
Copy link
Member

dims commented Jun 2, 2022

/transfer kubernetes

@k8s-ci-robot k8s-ci-robot transferred this issue from kubernetes/apiserver Jun 2, 2022
@k8s-ci-robot k8s-ci-robot added needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jun 2, 2022
@sftim
Copy link
Contributor

sftim commented Jun 2, 2022

BTW, for extra credit: support hop-by-hop compression too.

Of course, kubectl doesn't yet support hop-by-hop compression, but some generic tools (eg curl) do.

Clients can combine what they negotiate, too: for example, preferring application/vnd.kubernetes.protobuf data and also advertising that they support either identity or br hop-by-hop transfer encoding.

You don't have to restrict compression to JSON response bodies (per the issue title)

…or even to responses. The API server can allow clients to hop-by-hop compress a request body, too (using the Expect: 100-continue mechanism).

Because clients that understand hop-by-hop compression typically implement it properly, and only for HTTP/1.1 requests, adding hop-by-hop compression support is low impact / low risk. We could implement that first to get feedback on the implementation and then later migrate to end-to-end compression (which clients and proxies often do get wrong).

PS. https://datatracker.ietf.org/doc/html/rfc7540 forbids HTTP/2 clients from requesting hop-by-hop response body compression, and it looks like HTTP/3 will forbid it too. I'm not suggesting that hop-by-hop compression for those protocols is likely; that's a different and bigger question.

@neolit123
Copy link
Member

/sig api-machinery

@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 6, 2022
@fedebongio
Copy link
Contributor

/priority backlog

@k8s-ci-robot k8s-ci-robot added the priority/backlog Higher priority than priority/awaiting-more-evidence. label Jun 7, 2022
@fedebongio
Copy link
Contributor

/triage accepted
/help

@k8s-ci-robot
Copy link
Contributor

@fedebongio:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

  • Why are we solving this issue?
  • To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
  • Does this issue have zero to low barrier of entry?
  • How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/triage accepted
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the triage/accepted Indicates an issue or PR is ready to be actively worked on. label Jun 7, 2022
@k8s-ci-robot k8s-ci-robot added help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jun 7, 2022
@smarterclayton
Copy link
Contributor

Interestingly the original issue should be fixed, accept encoding is what we key compression off of. Possible to close?

@smarterclayton
Copy link
Contributor

As per #77449 this has been in Kube for a while. Please reopen if the original request was different. Re: hop by hop that should be a separate issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/backlog Higher priority than priority/awaiting-more-evidence. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

9 participants