go-client: Use httpcache client for all requests, even though only openapi returns ETags for caching #47353
Conversation
k8s-ci-robot
added
the
cncf-cla: yes
k8s-github-robot
added
size/XXL
|
/release-note |
k8s-ci-robot
added
release-note
|
/lgtm Don't you think keeping "add dependency" commit separate (not squashing) is cleaner? |
k8s-ci-robot
added
the
lgtm
I don't have a super strong opinion, I'll reword the commit. |
k8s-github-robot
removed
the
lgtm
apelisse
force-pushed
the
http-cache
branch
2 times, most recently
from
d33ec70 to
b4d5af8
Compare
k8s-github-robot
added
size/XL
|
/assign @shiywang |
|
I found why the tests are broken, the httpcache dependency has a bug with infinite streams (and that happens for watch for example). I've submitted a patch on that library, will report when we have news. |
apelisse
added
the
do-not-merge
k8s-github-robot
added
needs-rebase
k8s-github-robot
removed
the
needs-rebase
apelisse
force-pushed
the
http-cache
branch
2 times, most recently
from
3c596b0 to
2f6e7b8
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: apelisse, mbohlool, pwittrock, thockin Associated issue: 38637 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these OWNERS Files:
You can indicate your approval by writing |
k8s-github-robot
added
the
approved
|
/retest |
|
Automatic merge from submit-queue |
|
This broke kubectl proxy in #49534 - disabling the cache transport makes it work. It's possible that the kubectl proxy is reusing a cache with this code. Either way, we need to understand why kubectl proxy broke. This was definitely client side (I tested with an older server that doesn't have the change) so it's not related to anything server side. I'm concerned about the implications of this change and the cache always being present, given that it is a potentially large change to behavior of kubectl. I do not like that multiple instances of kubectl running at the same time with different kubeconfigs could overlap. Has that been explicitly tested? |
|
It's also possible the cache transport is not upgrade aware, and that the negotiation is being intercepted somehow. |
| @@ -56,6 +59,9 @@ func HTTPWrappersForConfig(config *Config, rt http.RoundTripper) (http.RoundTrip | |||
| len(config.Impersonate.Extra) > 0 { | |||
| rt = NewImpersonatingRoundTripper(config.Impersonate, rt) | |||
| } | |||
| if len(config.CacheDir) > 0 { | |||
| rt = NewCacheRoundTripper(config.CacheDir, rt) | |||
There was a problem hiding this comment.
This is a really significant change. Wrapping the whole transport affects every aspect of the clients, for streaming, spdy, etc. Given that we immediately saw issues with this for websocket use, I think we should revert this until the wrapping transports are more thoroughly reviewed, or we can scope this to just the discovery/openapi endpoints for now.
There was a problem hiding this comment.
Looks like this also results in all get/list resources getting persisted to disk:
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-gqj6t kubernetes.io/service-account-token 3 13m
$ more .kube/http-cache/499e4e6a4f2a2b827ce93a2349098106
HTTP/2.0 200 OK
Content-Length: 3680
Content-Type: application/json
Date: Mon, 07 Aug 2017 00:54:25 GMT
{"kind":"SecretList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/namespaces/default/secrets","resourceVersion":"1105"},"items":[{"metadata":{"name":"default-token-gqj6t"...
| // DefaultClientConfig represents the legacy behavior of this package for defaulting | ||
| // DEPRECATED will be replace | ||
| DefaultClientConfig = DirectClientConfig{*clientcmdapi.NewConfig(), "", &ConfigOverrides{ | ||
| ClusterDefaults: ClusterDefaults, | ||
| CacheDir: cacheDirDefault, |
There was a problem hiding this comment.
Does this get set as the default for clients that get built from a kubeconfig (for example, the controller manager and kubelet)? If so, this needs more vetting than if it just affected kubectl
|
it also looks like the disk cache component is not safe to use from multiple processes (it doesn't use atomic fs operations), and can leave partially written responses in the cache (if |
|
Ok, definitely needs to be disabled. The concurrency and caching is
problematic at well - kubectl can be heavily concurrent.
On Aug 6, 2017, at 9:18 PM, Jordan Liggitt <notifications@github.com> wrote:
it also looks like the disk cache component is not safe to use from
multiple processes (it doesn't use atomic fs operations), and can leave
partially written responses in the cache (if writeStreamWithLock encounters
an error while copying the stream, it leaves the file with the partial
response in the cache dir, and returns it as if it were a successful
response to future requests)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#47353 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABG_p4z9VAPequlGf6mOeVe7rWiBStxTks5sVmX7gaJpZM4N3YNn>
.
|
liggitt
added
release-note-none
Automatic merge from submit-queue (batch tested with PRs 50254, 50174, 50179) Revert "Merge pull request #47353 from apelisse/http-cache" Some issues were discovered with the caching merged in #47353: * uses a disk-based cache that is not safe between processes (does not use atomic fs operations) * writes get/list responses to disk that should not be cached (like `kubectl get secrets`) * is vulnerable to partially written cache responses being used as responses to future requests * breaks uses of the client transport that make use of websockets * defaults to enabling the cache for any client builder using RecommendedConfigOverrideFlags or DefaultClientConfig which affects more components than just kubectl This reverts commit fc89743, reversing changes made to 29ab38e.
…p-cache"" This reverts commit 4ee72eb.
…p-cache"" This reverts commit 4ee72eb.
…p-cache"" This reverts commit 4ee72eb.
…p-cache"" This reverts commit 4ee72eb.
…p-cache"" This reverts commit 4ee72eb.
What this PR does / why we need it: Use HTTP ETag for caching Swagger spec download
This also adds a new command-line flag "cachedir" to specify where the cache should keep its file. It defaults to
$HOME/.kube/http-cache.Which issue this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close that issue when PR gets merged): partly #38637Special notes for your reviewer:
Because this adds a bunch of dependencies, and removes a couple of files, I do recommend reading each commit individually.
Release note: