Skip to content

Commit

Permalink
Rework hack/ and build/ directories.
Browse files Browse the repository at this point in the history 
* Rewrite a bunch of the hack/ directory with modular reusable bash libraries.
* Have 'build/*' build on 'hack/*'.  The stuff in build now just runs hack/* in a docker container.
* Use a docker data container to enable faster incremental builds.
* Standardize output to _output/{local,dockerized}/bin/OS/ARCH/*.  This regularized placement makes cross compilation work.
* Move travis specific scripts under hack/travis

With new dockerized incremental builds, I can do a no-op `make quick-release` in ~30s.  This is a significant improvement.
  • Loading branch information
@jbeda
jbeda committed Nov 2, 2014
1 parent 203a3d0 commit 5d33ce4
Show file tree
Hide file tree
Showing 51 changed files with 1,009 additions and 793 deletions.
2 changes: 1 addition & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@ network_closure.sh
/third_party/pkg

# also ignore etcd installed by hack/install-etcd.sh
/third_party/etcd
/third_party/etcd*
8 changes: 4 additions & 4 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ go:

install:
- go get code.google.com/p/go.tools/cmd/cover
- ./hack/install-etcd.sh
- ./hack/travis/install-etcd.sh
- ./hack/verify-gofmt.sh
- ./hack/verify-boilerplate.sh
- ./hack/install-std-race.sh
- ./hack/travis/install-std-race.sh
- ./hack/build-go.sh
- go get ./contrib/podex

script:
- KUBE_TIMEOUT='-timeout 60s' ./hack/test-go.sh
- PATH=$HOME/gopath/bin:./third_party/etcd/bin:$PATH ./hack/test-cmd.sh
- PATH=$HOME/gopath/bin:./third_party/etcd/bin:$PATH ./hack/test-integration.sh
- PATH=$HOME/gopath/bin:./third_party/etcd:$PATH ./hack/test-cmd.sh
- PATH=$HOME/gopath/bin:./third_party/etcd:$PATH ./hack/test-integration.sh
- ./hack/benchmark-go.sh

notifications:
Expand Down
11 changes: 6 additions & 5 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
OUT_DIR = _output
GODEPS_PKG_DIR = Godeps/_workspace/pkg

export GOFLAGS
KUBE_GOFLAGS = $(GOFLAGS)
export KUBE_GOFLAGS

# Build code.
#
Expand Down Expand Up @@ -49,7 +50,7 @@ check test:
# make test_integration
test_integration test_integ:
hack/test-integration.sh
.PHONY: integration
.PHONY: test_integration test_integ

# Build and run end-to-end tests.
#
Expand Down Expand Up @@ -94,10 +95,10 @@ release:
.PHONY: release

# Build a release, but skip tests
#
#
# Example:
# make release-skip-tests
release-skip-tests:
release-skip-tests quick-release:
KUBE_RELEASE_RUN_TESTS=n build/release.sh
.PHONY: release-skip-tests
.PHONY: release-skip-tests quick-release

17 changes: 9 additions & 8 deletions build/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,13 @@ There is also early support for building Docker "run" containers

## Key scripts

* `make-server.sh`: This will compile all of the Kubernetes server binaries for linux/amd64
* `make-client.sh`: This will make all cross-compiled client binaries
* `run-tests.sh`: This will run the Kubernetes unit tests
* `run-integration.sh`: This will build and run the integration test
* `copy-output.sh`: This will copy the contents of `_output/build` from any remote Docker container to the local `_output/build`. Right now this is only necessary on Mac OS X with `boot2docker`.
* `make-clean.sh`: Clean out the contents of `_output/build` and remove any local built container images.
* `run.sh`: Run a command in a build docker container. Common invocations:
* `run.sh hack/build-go.sh`: Build just linux binaries in the container. Pass options and packages as necessary.
* `run.sh hack/build-cross.sh`: Build all binaries for all platforms
* `run.sh hack/test-go.sh`: Run all unit tests
* `run.sh hack/test-integration.sh`: Run integration test
* `copy-output.sh`: This will copy the contents of `_output/dockerized/bin` from any remote Docker container to the local `_output/dockerized/bin`. Right now this is only necessary on Mac OS X with `boot2docker` when your git repo isn't under `/Users`.
* `make-clean.sh`: Clean out the contents of `_output/dockerized` and remove any local built container images.
* `shell.sh`: Drop into a `bash` shell in a build container with a snapshot of the current repo code.
* `release.sh`: Build everything, test it, and (optionally) upload the results to a GCS bucket.

Expand Down Expand Up @@ -81,10 +82,10 @@ If the release script is set to upload to GCS, it'll do the following:

These are in no particular order

* [ ] Harmonize with scripts in `hack/`. How much do we support building outside of Docker and these scripts?
* [X] Harmonize with scripts in `hack/`. How much do we support building outside of Docker and these scripts?
* [ ] Get a cluster up and running with the Docker images. Perhaps start with a local cluster and move up to a GCE cluster.
* [ ] Implement (#186)[https://github.com/GoogleCloudPlatform/kubernetes/issues/186]. This will make it easier to develop Kubernetes.
* [ ] Deprecate/replace most of the stuff in the hack/
* [X] Deprecate/replace most of the stuff in the hack/
* [ ] Create an install script that'll let us do a `curl https://[URL] | bash` to get that tarball down and ensure that other dependencies (cloud SDK?) are installed and configured correctly.
* [ ] Support/test Windows as a client.
* [ ] Finish support for the Dockerized runtime. Issue (#19)[https://github.com/GoogleCloudPlatform/kubernetes/issues/19]. A key issue here is to make this fast/light enough that we can use it for development workflows.
Expand Down
8 changes: 8 additions & 0 deletions build/build-image/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ ENV GOARCH amd64
# Get the code coverage tool and godep
RUN go get code.google.com/p/go.tools/cmd/cover github.com/tools/godep

# We use rsync to copy some binaries around. It is faster (0.3s vs. 1.1s) on my
# machine vs. `install`
RUN apt-get update && apt-get install -y rsync

# Download and symlink etcd. We need this for our integration tests.
RUN mkdir -p /usr/local/src/etcd &&\
cd /usr/local/src/etcd &&\
Expand All @@ -39,6 +43,10 @@ WORKDIR /go/src/github.com/GoogleCloudPlatform/kubernetes

# Propagate the git tree version into the build image
ADD kube-version-defs /kube-version-defs
ENV KUBE_GIT_VERSION_FILE /kube-version-defs

# Make output from the dockerized build go someplace else
ENV KUBE_OUTPUT_SUBPATH _output/dockerized

# Upload Kubernetes source
ADD kube-source.tar.gz /go/src/github.com/GoogleCloudPlatform/kubernetes
81 changes: 0 additions & 81 deletions build/build-image/common.sh
View file

This file was deleted.

39 changes: 0 additions & 39 deletions build/build-image/make-client.sh
View file

This file was deleted.

29 changes: 0 additions & 29 deletions build/build-image/make-server.sh
View file

This file was deleted.

Loading

0 comments on commit 5d33ce4

Please sign in to comment.