Skip to content

Commit

Permalink
Merge pull request kubernetes#17996 from spxtr/jenkins-jobs
Browse files Browse the repository at this point in the history
Check Jenkins job configs into git
  • Loading branch information
fabioy committed Dec 7, 2015
2 parents ec1ba74 + 1bc9980 commit 0fe724f
Show file tree
Hide file tree
Showing 5 changed files with 117 additions and 7 deletions.
32 changes: 25 additions & 7 deletions hack/jenkins/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,23 @@
integration system. The Google team is running a Jenkins server on a
private GCE instance for the Kubernetes project in order to run longer
integration tests, continuously, on different providers. Currently, we
(Google) are only running Jenkins on our own providers (GCE and GKE)
in different flavors.
are running tests on GCE, GKE, and AWS.

## General flow
The flow of the Google Jenkins server:
* Under the `kubernetes-build` job: Every 5 minutes, Jenkins polls for a batch of new commits, after which it runs the `build.sh` script (in this directory) on the latest tip. This results in build assets getting pushed to GCS and the `latest.txt` file in the `ci` bucket being updated. That job then triggers `kubernetes-e2e-*`.
* On trigger, and every half hour (which effectively means all the time, unless we're failing cluster creation), e2e variants run, on the latest build assets in GCS:
* `kubernetes-e2e-gce`: Standard GCE e2e
* `kubernetes-e2e-gke`: GKE provider e2e, with head k8s client and GKE creating clusters at its default version
* `kubernetes-e2e-gke-ci`: GKE provider e2e, with head k8s client and GKE creating clusters at the head k8s version
* Under the `kubernetes-build` job: Every 2 minutes, Jenkins polls for a batch
of new commits, after which it runs the `build.sh` script (in this directory)
on the latest tip. This results in build assets getting pushed to GCS and the
`latest.txt` file in the `ci` bucket being updated.
* On trigger, and every half hour (which effectively means all the time, unless
we're failing cluster creation), e2e variants run, on the latest build assets
in GCS:
* `kubernetes-e2e-gce`: Standard GCE e2e.
* `kubernetes-e2e-gke`: GKE provider e2e, with head k8s client and GKE
creating clusters at its default version.
* `kubernetes-e2e-gke-ci`: GKE provider e2e, with head k8s client and GKE
creating clusters at the head k8s version.
* `kubernetes-e2e-aws`: AWS provider e2e.
* Each job will not run concurrently with itself, so, for instance,
Jenkins executor will only ever run one `kubernetes-build`
job. However, it may run the jobs in parallel,
Expand All @@ -22,6 +29,8 @@ The flow of the Google Jenkins server:
pushed to our GCS bucket rapidly, but they may take some time to
fully work through Jenkins. Or you may get lucky and catch the
train in 5 minutes.
* There are many jobs not listed here, including upgrade tests, soak tests, and
tests for previous releases.

## Scripts

Expand All @@ -32,5 +41,14 @@ outside this repository, it's tricky to keep documentation for it up
to date quickly. However, the scripts themselves attempt to provide
color for the configuration(s) that each script runs in.

## Job Builder

New jobs should be specified as YAML files to be processed by [Jenkins Job
Builder](http://docs.openstack.org/infra/jenkins-job-builder/). The YAML files
live in `job-configs` and its subfolders. Jenkins runs Jenkins Job Builder
in a Docker container defined in `job-builder-image`, and triggers it using
`update-jobs.sh`. Jenkins Job Builder uses a config file called
[jenkins_jobs.ini](http://docs.openstack.org/infra/jenkins-job-builder/execution.html)
which contains the location and credentials of the Jenkins server.

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/hack/jenkins/README.md?pixel)]()
33 changes: 33 additions & 0 deletions hack/jenkins/job-builder-image/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# This docker image runs Jenkins Job Builder (JJB) for automatic job reconciliation.

FROM ubuntu:14.04
MAINTAINER Joe Finney <spxtr@google.com>

RUN mkdir /build
WORKDIR /build

# Dependencies for JJB
RUN apt-get update && apt-get install -y \
wget \
git \
python-dev \
python-pip \
libyaml-dev \
python-yaml
RUN pip install PyYAML python-jenkins
# Required since JJB supports python 2.6, which doesn't have ordereddict built-in. We have 2.7.
RUN wget https://pypi.python.org/packages/source/o/ordereddict/ordereddict-1.1.tar.gz \
&& tar -xvf ordereddict-1.1.tar.gz \
&& cd ordereddict-1.1 \
&& python setup.py install

RUN git clone https://git.openstack.org/openstack-infra/jenkins-job-builder \
&& cd jenkins-job-builder \
&& python setup.py install

# JJB configuration lives in /etc/jenkins_jobs/jenkins_jobs.ini
RUN mkdir -p /etc/jenkins_jobs

WORKDIR /
RUN git clone https://github.com/kubernetes/kubernetes.git
WORKDIR kubernetes
5 changes: 5 additions & 0 deletions hack/jenkins/job-builder-image/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
TAG = 1

all:
docker build -t gcr.io/google_containers/kubekins-job-builder:$(TAG) .
gcloud docker push gcr.io/google_containers/kubekins-job-builder:$(TAG)
9 changes: 9 additions & 0 deletions hack/jenkins/job-configs/kubernetes-update-jenkins-jobs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
- job:
name: kubernetes-update-jenkins-jobs
description: "Update Jenkins jobs"

triggers:
- timed: "H/15 * * * *"

builders:
- shell: "curl -fsS https://raw.githubusercontent.com/kubernetes/kubernetes/master/hack/jenkins/update-jobs.sh | /bin/bash -"
45 changes: 45 additions & 0 deletions hack/jenkins/update-jobs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/bin/bash

# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Update all Jenkins jobs in a folder. If no folder is provided in $1,
# defaults to hack/jenkins/job-configs.

if [[ $# -eq 1 ]]; then
config_dir=$1
else
config_dir="hack/jenkins/job-configs"
fi

# Run the container if it isn't present.
if ! docker inspect job-builder &> /dev/null; then
# jenkins_jobs.ini contains administrative credentials for Jenkins.
# Store it in the workspace of the Jenkins job that calls this script.
if [[ -e jenkins_jobs.ini ]]; then
docker run -idt \
--net host \
--name job-builder \
--restart always \
gcr.io/google_containers/kubekins-job-builder:1
docker cp jenkins_jobs.ini job-builder:/etc/jenkins_jobs
else
echo "jenkins_jobs.ini not found in workspace" >&2
exit 1
fi
fi

docker exec job-builder git checkout master
docker exec job-builder git pull
docker exec job-builder jenkins-jobs test ${config_dir}

0 comments on commit 0fe724f

Please sign in to comment.