Consistent timeout of pull-kubernetetes-verify on a PR touching hack/ directory. #50319
Comments
|
@bskiba
Note: Method 1 will trigger an email to the group. You can find the group list here and label list here. |
k8s-github-robot
added
the
needs-sig
|
/sig testing |
k8s-ci-robot
added
the
sig/testing
k8s-github-robot
removed
the
needs-sig
|
Looks like I managed to merge the PR as another retest fit under the limit (2017-08-08 16:52 CEST took 56m43s) but raising the timeout for pull-kubernetes-verify might be worth considering for cases similar to mine. |
|
I am facing the same issue for my pr https://k8s-gubernator.appspot.com/pr/46090 |
|
/assign @fejta |
|
here is a link to the issue on |
|
it's not that specific task, I don't think... the verify job just got slower in the past couple days (it's timing out at an hour every time on #49642 now) these times are from a failed run today: before today the individual tasks were running faster, but were still close to the timeout. these times were from a successful run earlier: the time spent in godeps is sort of ridiculous (we restore godeps three times during the verify run, each of which takes ~3 minutes) |
|
this bit in verify-godeps.sh is likely to be the issue if you touch readonly branch=${1:-${KUBE_VERIFY_GIT_BRANCH:-master}}
if ! [[ ${KUBE_FORCE_VERIFY_CHECKS:-} =~ ^[yY]$ ]] && \
! kube::util::has_changes_against_upstream_branch "${branch}" 'Godeps/' && \
! kube::util::has_changes_against_upstream_branch "${branch}" 'vendor/' && \
! kube::util::has_changes_against_upstream_branch "${branch}" 'hack/'; then
exit 0
fiskipping that script drops ~15 minutes from the run time of the job |
|
opened #50418 to skip unrelated changes in |
Automatic merge from submit-queue (batch tested with PRs 50418, 49830, 49206, 49061, 49912) Target godep script change verifications helps with #50319 `hack/verify-godeps.sh` takes ~15 minutes of the verify job time. We should not run it if not required, especially since the job is regularly timing out at 1 hour when this check is included. #48653 added a check to run it if *anything* under `hack` was changed. This targetes just changes to the godep scripts
|
timeout was lengthened to 75 minutes, and fewer things should be subject to godep verification since #50418 merged |
Automatic merge from submit-queue (batch tested with PRs 49615, 49321, 49982, 49788, 50355) Simplify hack/verify-flags-underscore.py **What this PR does / why we need it**: This PR removes the need for `hack/verify-flags/known-flags.txt` and verify-flags-underscore.py will always parse the flags from go files to check if they have underscore. It is much faster compared to earlier checks and it does its job to check for underscore in flags. Now: ``` # time ./hack/verify-flags-underscore.py real 0m1.638s user 0m1.560s sys 0m0.076s ``` Before: ``` # time ./hack/verify-flags-underscore.py real 0m22.585s user 0m22.464s sys 0m0.112s ``` It has become a pain to keep adding new flag to `known-flags.txt` whenever a new flag is introduced. with this PR this is step is not required anymore. **Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #40329 #50319 **Special notes for your reviewer**: **Release note**: ``` NONE ``` /cc @fejta @mtaufen
/kind bug
I have a PR that I am trying to merge that ads a flag to hack/verify-flags/known-flags.txt (#49382). I can't get pull-kubernetes-verify to pass as it keeps timing out (https://k8s-gubernator.appspot.com/builds/kubernetes-jenkins/pr-logs/pull/49382/pull-kubernetes-verify/)
Comparing with logs other PRs that do pass the check I have an additional 15 minutes spent on fetching godeps due to change in hack/ directory. As it usually takes ~45mins to run a successfull pull-kubernetes-verify, my chances of merging are close to 0.
Example excerpt from https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/49382/pull-kubernetes-verify/43937/build-log.txt
Verifying hack/make-rules/../../hack/verify-godeps.sh
I0807 13:48:00.045] Checking for 'Godeps/' changes against 'bootstrap-upstream/master'
I0807 13:48:00.267] No 'Godeps/' changes detected.
I0807 13:48:00.271] Checking for 'vendor/' changes against 'bootstrap-upstream/master'
I0807 13:48:00.506] No 'vendor/' changes detected.
I0807 13:48:00.510] Checking for 'hack/' changes against 'bootstrap-upstream/master'
I0807 13:48:02.323] +++ [0807 13:48:02] Starting to download all kubernetes godeps. This takes a while
I0807 13:50:53.413] +++ [0807 13:50:53] Download finished
I0807 13:50:53.714] Running godep save. This will take around 15 minutes.
I0807 14:05:21.760]
I0807 14:05:21.761] Don't forget to run:
I0807 14:05:21.761] - hack/update-bazel.sh to recreate the BUILD files
I0807 14:05:21.761] - hack/update-godep-licenses.sh if you added or removed a dependency!
I0807 14:05:22.247] Godeps Verified.
I0807 14:05:22.248] Removing /tmp/gopath.jlBk34
I0807 14:05:24.930] �[0;32mSUCCESS�[0m hack/make-rules/../../hack/verify-godeps.sh 1044s
The text was updated successfully, but these errors were encountered: