Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automated cherry pick of #100959: DelegatingAuthenticationOptions TokenReview request timeout #101103

Merged
merged 1 commit into from
Apr 28, 2021
Merged

Automated cherry pick of #100959: DelegatingAuthenticationOptions TokenReview request timeout #101103

merged 1 commit into from
Apr 28, 2021
+42 −24

Conversation

p0lyn0mial
Copy link
Contributor

@p0lyn0mial p0lyn0mial commented Apr 14, 2021 •
edited by liggitt
Loading

Cherry pick of #100959 on release-1.20.

#100959: DelegatingAuthenticationOptions TokenReview request timeout

For details on the cherry pick process, see the cherry pick requests page.

What type of PR is this?

/kind bug

What this PR does / why we need it:

It turns out that setting a timeout on HTTP client affects watch requests made by the delegated authentication component.
With a 10 second timeout watch requests are being re-established exactly after 10 seconds even though the default request timeout for them is ~5 minutes.

This is because if multiple timeouts were set, the stdlib picks the smaller timeout to be applied, leaving others useless.
For more details see https://github.com/golang/go/blob/a937729c2c2f6950a32bc5cd0f5b88700882f078/src/net/http/client.go#L364

Instead of setting a timeout on the HTTP client, we should use context for cancellation.

This has the potential of being scattered across the codebase, perhaps we should seek a broader solution.

Here is a reproducer for re-establish watch requests when http.Client.Timeout is set with standard client-go library
p0lyn0mial/simple-watch#2

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

fixes regression in 1.20 in DelegatingAuthenticationOptions to set the timeout only for the token review client. Previously the timeout was also applied to watches making them reconnecting every 10 seconds.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@p0lyn0mial
DelegatingAuthenticationOptions TokenReview request timeout
e9372dc 
it turns out that setting a timeout on HTTP client affect watch requests made by the delegated authentication component.
with a 10 second timeout watch requests are being re-established exactly after 10 seconds even though the default request timeout for them is ~5 minutes.

this is because if multiple timeouts were set, the stdlib picks the smaller timeout to be applied, leaving other useless.
for more details see https://github.com/golang/go/blob/a937729c2c2f6950a32bc5cd0f5b88700882f078/src/net/http/client.go#L364

instead of setting a timeout on the HTTP client we should use context for cancellation.
@k8s-ci-robot k8s-ci-robot added this to the v1.20 milestone Apr 14, 2021
@k8s-ci-robot k8s-ci-robot added do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. area/apiserver labels Apr 14, 2021
@k8s-ci-robot k8s-ci-robot added area/cloudprovider sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Apr 14, 2021
@p0lyn0mial
Copy link
Contributor Author

/kind bug

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Apr 14, 2021
@p0lyn0mial
Copy link
Contributor Author

/retest

@dims
Copy link
Member

dims commented Apr 14, 2021

/approve

@p0lyn0mial
Copy link
Contributor Author

/retest

@deads2k
Copy link
Contributor

deads2k commented Apr 14, 2021

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 14, 2021
@deads2k
Copy link
Contributor

deads2k commented Apr 14, 2021

/approve

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 14, 2021
@fedebongio
Copy link
Contributor

/triage accepted

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Apr 15, 2021
@deads2k
Copy link
Contributor

deads2k commented Apr 20, 2021

This resolves a significant load problem.

/priority important-soon

@k8s-ci-robot k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Apr 20, 2021
@xmudrii
Copy link
Member

xmudrii commented Apr 27, 2021

/kind regression

@k8s-ci-robot k8s-ci-robot added the kind/regression Categorizes issue or PR as related to a regression from a prior release. label Apr 27, 2021
xmudrii
xmudrii approved these changes Apr 27, 2021
View reviewed changes
Copy link
Member

@xmudrii xmudrii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@xmudrii xmudrii added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Apr 27, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. label Apr 27, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, dims, p0lyn0mial, xmudrii

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@p0lyn0mial p0lyn0mial mentioned this pull request Apr 27, 2021
Merged
@xmudrii
Copy link
Member

xmudrii commented Apr 27, 2021

/test pull-kubernetes-bazel-test

@xmudrii
Copy link
Member

xmudrii commented Apr 27, 2021

/test pull-kubernetes-e2e-kind

@xmudrii
Copy link
Member

xmudrii commented Apr 27, 2021

/test pull-kubernetes-e2e-gce-ubuntu-containerd

@xmudrii
Copy link
Member

xmudrii commented Apr 27, 2021

/test pull-kubernetes-e2e-kind

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

4 similar comments
@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit 6e0f1fe into kubernetes:release-1.20 Apr 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xmudrii xmudrii xmudrii approved these changes

@cheftako cheftako Awaiting requested review from cheftako

@deads2k deads2k Awaiting requested review from deads2k

Assignees

@deads2k deads2k

@xmudrii xmudrii

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver area/cloudprovider cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. kind/regression Categorizes issue or PR as related to a regression from a prior release. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.20
Development

Successfully merging this pull request may close these issues.
7 participants
@p0lyn0mial @dims @deads2k @fedebongio @xmudrii @k8s-ci-robot @fejta-bot