Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kubernetes-e2e-gce job to git #18454

Merged
merged 1 commit into from
Dec 11, 2015
Merged

Add kubernetes-e2e-gce job to git #18454

merged 1 commit into from
Dec 11, 2015

Conversation

spxtr
Copy link
Contributor

@spxtr spxtr commented Dec 9, 2015

I think it might be a good idea to add this job on its own, make sure it's working right, then add the rest of the GCE E2E tests in a second PR. For the most part they have the same settings, except for easy-to-substitute things like timeout duration or branch. I have backed up the current config.

Changes to the config:

  • We no longer block if kubernetes-build is unstable or failing. I can add this back in, if necessary.
  • We no longer publish to slack. The Jenkins slack plugin puts authentication info in every job config, which is unacceptable here. See this issue. It looks like work in in progress to fix this.
    Get Jenkins job configs under source control #18122

@k8s-github-robot
Copy link

Labelling this PR as size/L

@k8s-github-robot k8s-github-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Dec 9, 2015
@k8s-bot
Copy link

k8s-bot commented Dec 9, 2015

GCE e2e build/test failed for commit 61b87d8f3538cca86cab1f9015d08b5e2b2b25bb.

@spxtr
Copy link
Contributor Author

spxtr commented Dec 9, 2015

@ihmccreery #18116

@@ -224,6 +224,7 @@ E2E_OPT=${E2E_OPT:-""}
# Set environment variables shared for all of the GCE Jenkins projects.
if [[ ${JOB_NAME} =~ ^kubernetes-.*-gce ]]; then
KUBERNETES_PROVIDER="gce"
: ${GCE_SERVICE_ACCOUNT:=$(cat /var/lib/jenkins/.gce/service_account)}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file doesn't exist on all of our VMs. (In particular, none of the PR VMs have it.)

I wonder if using gcloud auth would be better?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would that work? It definitely sounds better than having to copy this file around.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's almost certainly a better way, but my 60s PoC (which works for now?):

gcloud auth list 2>/dev/null | grep active | cut -f3 -d' '

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that much more, thanks.

@ixdy
Copy link
Member

ixdy commented Dec 10, 2015

It's probably fine to no longer block on kubernetes-build failing, as long as we make it clear to everyone that tests continue with the latest successful build.

@k8s-bot
Copy link

k8s-bot commented Dec 10, 2015

GCE e2e test build/test passed for commit 4954508.

@ikehz
Copy link
Contributor

ikehz commented Dec 10, 2015

It's probably fine to no longer block on kubernetes-build failing, as long as we make it clear to everyone that tests continue with the latest successful build.

+1

@spxtr
Copy link
Contributor Author

spxtr commented Dec 10, 2015

Is it a good idea to pull the environment variables out of e2e.sh and into the yaml file? It looks like we can only pull some of them out, the SKIP_TEST stuff is set at the top of e2e.sh. The rest we can move here.

The EnvInject plugin is already installed on Jenkins and would make this quite easy.

@ikehz
Copy link
Contributor

ikehz commented Dec 10, 2015

Is it a good idea to pull the environment variables out of e2e.sh and into the yaml file?

Let's wait on that. One thing at a time. 😄

@spxtr
Copy link
Contributor Author

spxtr commented Dec 10, 2015

Sounds good. Is there any reason not to make this replacement? Edit: The second one won't fail if the curl fails. Looks like a dead-end.

curl -fsS -o upload-to-gcs.sh --retry 3 "..." && source upload-to-gcs.sh; rm -f upload-to-gcs.sh

->

source <(curl -fsS --retry 3 "...")

@ixdy
Copy link
Member

ixdy commented Dec 11, 2015

LGTM.

I think I tried doing

curl ... | source -

before and it didn't work for some reason. Maybe your variation will... but let's maybe try changing one thing at a time?

Which actually makes me wonder: how do we test changes to configs in this new system, given that latency is much higher than just tweaking something on the Jenkins UI?

@spxtr
Copy link
Contributor Author

spxtr commented Dec 11, 2015

The source <(curl ...) variation works, but there's no way to fail if the curl fails. It's process substitution.

To test changes to the configs I think it's reasonable to make a new jenkins job with "test-" before the name or something. If that's not going to work, I think it's reasonable to manually merge config changes in order to reduce the latency.

The only thing that shouldn't happen is for someone to go into jenkins, make a config change, then forget about it. I'll sit there until someone touches that file in git, then it will be reverted. We could switch on ignore_cache in jenkins_jobs.ini to make it always update jobs, so that their change will be reverted next time the update job runs.

@spxtr
Copy link
Contributor Author

spxtr commented Dec 11, 2015

Merging manually to watch Jenkins.

spxtr added a commit that referenced this pull request Dec 11, 2015
Add kubernetes-e2e-gce job to git
@spxtr spxtr merged commit 8535f13 into kubernetes:master Dec 11, 2015
@spxtr spxtr deleted the e2e-gce branch December 11, 2015 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/test-infra size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants