Skip to content

Commit

Permalink
Split hack/{verify,update}-* files so we don't always go build
Browse files Browse the repository at this point in the history
Right now some of the hack/* tools use `go run` and build almost every
time. There are some which expect you to have already run `go install`.
And in all cases the pre-commit hook, which runs a full build wouldn't
want to do either, since it just built!

This creates a new hack/after-build/ directory and has the scripts which
REQUIRE that the binary already be built. It doesn't test and complain.
It just fails miserably. Users should not be in this directory. Users
should just use hack/verify-* which will just do the build and then call
the "after-build" version. The pre-commit hook or anything which KNOWS
the binaries have been built can use the fast version.
  • Loading branch information
eparis committed Aug 11, 2015
1 parent 985fa94 commit 3b7c396
Show file tree
Hide file tree
Showing 25 changed files with 495 additions and 258 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,6 @@ www/master/shared/config/development.json
# Karma output
www/test_out

# precommit temporary directories created by ./hack/verify-gendocs.sh and ./hack/lib/util.sh
# precommit temporary directories created by ./hack/verify-generated-docs.sh and ./hack/lib/util.sh
_tmp/
doc_tmp/
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ script:
- ./hack/verify-description.sh
- PATH=$GOPATH/bin:$PATH ./hack/verify-generated-conversions.sh
- PATH=$GOPATH/bin:$PATH ./hack/verify-generated-deep-copies.sh
- PATH=$GOPATH/bin:./third_party/etcd:$PATH ./hack/verify-gendocs.sh
- PATH=$GOPATH/bin:./third_party/etcd:$PATH ./hack/verify-generated-docs.sh
- PATH=$GOPATH/bin:./third_party/etcd:$PATH ./hack/verify-swagger-spec.sh
- PATH=$GOPATH/bin:./third_party/etcd:$PATH ./hack/verify-linkcheck.sh
- godep go test ./cmd/mungedocs
Expand Down
2 changes: 1 addition & 1 deletion build/versionize-docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -71,5 +71,5 @@ done
# Update API descriptions to match this version.
$SED -ri -e "s|(releases.k8s.io)/[^/]+|\1/${NEW_VERSION}|" pkg/api/v[0-9]*/types.go

${KUBE_ROOT}/hack/run-gendocs.sh
${KUBE_ROOT}/hack/update-generated-docs.sh
${KUBE_ROOT}/hack/update-swagger-spec.sh
2 changes: 1 addition & 1 deletion docs/devel/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ See [conformance-test.sh](http://releases.k8s.io/HEAD/hack/conformance-test.sh).
## Regenerating the CLI documentation

```sh
hack/run-gendocs.sh
hack/update-generated-docs.sh
```


Expand Down
4 changes: 4 additions & 0 deletions hack/after-build/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
These should be called only by a script (or a user) who knows that the binaries are built and correct. Normal users should just have the versions in ../ which should just always do a build.


[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/hack/after-build/README.md?pixel)]()
57 changes: 57 additions & 0 deletions hack/after-build/update-generated-conversions.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/bin/bash

# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
source "${KUBE_ROOT}/hack/lib/init.sh"

kube::golang::setup_env

genconversion=$(kube::util::find-binary "genconversion")

function generate_version() {
local version=$1
local TMPFILE="/tmp/conversion_generated.$(date +%s).go"

echo "Generating for version ${version}"

sed 's/YEAR/2015/' hack/boilerplate/boilerplate.go.txt > "$TMPFILE"
cat >> "$TMPFILE" <<EOF
package ${version}
// AUTO-GENERATED FUNCTIONS START HERE
EOF

"${genconversion}" -v "${version}" -f - >> "$TMPFILE"

cat >> "$TMPFILE" <<EOF
// AUTO-GENERATED FUNCTIONS END HERE
EOF

mv "$TMPFILE" "pkg/api/${version}/conversion_generated.go"
}

VERSIONS="v1"
for ver in $VERSIONS; do
# Ensure that the version being processed is registered by setting
# KUBE_API_VERSIONS.
KUBE_API_VERSIONS="${ver}" generate_version "${ver}"
done

# ex: ts=2 sw=2 et filetype=sh
79 changes: 79 additions & 0 deletions hack/after-build/update-generated-deep-copies.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#!/bin/bash

# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
source "${KUBE_ROOT}/hack/lib/init.sh"

kube::golang::setup_env

gendeepcopy=$(kube::util::find-binary "gendeepcopy")

function result_file_name() {
local version=$1
if [ "${version}" == "api" ]; then
echo "pkg/api/deep_copy_generated.go"
else
echo "pkg/api/${version}/deep_copy_generated.go"
fi
}

function generate_version() {
local version=$1
local TMPFILE="/tmp/deep_copy_generated.$(date +%s).go"

echo "Generating for version ${version}"

sed 's/YEAR/2015/' hack/boilerplate/boilerplate.go.txt > $TMPFILE
cat >> $TMPFILE <<EOF
package ${version}
// AUTO-GENERATED FUNCTIONS START HERE
EOF

"${gendeepcopy}" -v "${version}" -f - -o "${version}=" >> "$TMPFILE"

cat >> "$TMPFILE" <<EOF
// AUTO-GENERATED FUNCTIONS END HERE
EOF

gofmt -w -s "$TMPFILE"
mv "$TMPFILE" `result_file_name ${version}`
}

function generate_deep_copies() {
local versions="api v1"
# To avoid compile errors, remove the currently existing files.
for ver in ${versions}; do
rm -f `result_file_name ${ver}`
done
apiVersions=""
for ver in ${versions}; do
# Ensure that the version being processed is registered by setting
# KUBE_API_VERSIONS.
if [ "${ver}" != "api" ]; then
apiVersions="${ver}"
fi
KUBE_API_VERSIONS="${apiVersions}" generate_version "${ver}"
done
}

generate_deep_copies

# ex: ts=2 sw=2 et filetype=sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,17 @@ set -o errexit
set -o nounset
set -o pipefail

KUBE_ROOT=$(dirname "${BASH_SOURCE}")/..
KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
source "${KUBE_ROOT}/hack/lib/init.sh"

kube::golang::setup_env
"${KUBE_ROOT}/hack/build-go.sh" cmd/gendocs cmd/genman cmd/genbashcomp cmd/mungedocs

# Find binary
gendocs=$(kube::util::find-binary "gendocs")
genman=$(kube::util::find-binary "genman")
genbashcomp=$(kube::util::find-binary "genbashcomp")
mungedocs=$(kube::util::find-binary "mungedocs")

if [[ ! -x "$gendocs" || ! -x "$genman" || ! -x "$genbashcomp" || ! -x "$mungedocs" ]]; then
{
echo "It looks as if you don't have a compiled gendocs, genman, genbashcomp or mungedocs binary"
echo
echo "If you are running from a clone of the git repo, please run"
echo "'./hack/build-go.sh cmd/gendocs cmd/genman cmd/genbashcomp cmd/mungedocs'."
} >&2
exit 1
fi

kube::util::gen-doc "${gendocs}" "${KUBE_ROOT}" "docs/user-guide/kubectl/" '###### Auto generated by spf13/cobra'
kube::util::gen-doc "${genman}" "${KUBE_ROOT}" "docs/man/man1"
kube::util::gen-doc "${genbashcomp}" "${KUBE_ROOT}" "contrib/completions/bash/"
Expand Down
75 changes: 75 additions & 0 deletions hack/after-build/update-swagger-spec.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/bin/bash

# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Script to fetch latest swagger spec.
# Puts the updated spec at swagger-spec/

set -o errexit
set -o nounset
set -o pipefail

KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
SWAGGER_ROOT_DIR="${KUBE_ROOT}/api/swagger-spec"
source "${KUBE_ROOT}/hack/lib/init.sh"

function cleanup()
{
[[ -n ${APISERVER_PID-} ]] && kill ${APISERVER_PID} 1>&2 2>/dev/null

kube::etcd::cleanup

kube::log::status "Clean up complete"
}

trap cleanup EXIT SIGINT

kube::golang::setup_env

apiserver=$(kube::util::find-binary "kube-apiserver")

ETCD_HOST=${ETCD_HOST:-127.0.0.1}
ETCD_PORT=${ETCD_PORT:-4001}
API_PORT=${API_PORT:-8050}
API_HOST=${API_HOST:-127.0.0.1}
KUBELET_PORT=${KUBELET_PORT:-10250}

kube::etcd::start

# Start kube-apiserver
kube::log::status "Starting kube-apiserver"
KUBE_API_VERSIONS="v1" "${KUBE_OUTPUT_HOSTBIN}/kube-apiserver" \
--address="127.0.0.1" \
--public-address-override="127.0.0.1" \
--port="${API_PORT}" \
--etcd-servers="http://${ETCD_HOST}:${ETCD_PORT}" \
--public-address-override="127.0.0.1" \
--kubelet-port=${KUBELET_PORT} \
--runtime-config=api/v1 \
--service-cluster-ip-range="10.0.0.0/24" >/dev/null 2>&1 &
APISERVER_PID=$!

kube::util::wait_for_url "http://127.0.0.1:${API_PORT}/healthz" "apiserver: "

SWAGGER_API_PATH="http://127.0.0.1:${API_PORT}/swaggerapi/"
kube::log::status "Updating " ${SWAGGER_ROOT_DIR}
curl -fs ${SWAGGER_API_PATH} > ${SWAGGER_ROOT_DIR}/resourceListing.json
curl -fs ${SWAGGER_API_PATH}version > ${SWAGGER_ROOT_DIR}/version.json
curl -fs ${SWAGGER_API_PATH}api > ${SWAGGER_ROOT_DIR}/api.json
curl -fs ${SWAGGER_API_PATH}api/v1 > ${SWAGGER_ROOT_DIR}/v1.json

kube::log::status "SUCCESS"

# ex: ts=2 sw=2 et filetype=sh
47 changes: 47 additions & 0 deletions hack/after-build/verify-generated-conversions.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/bin/bash

# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
source "${KUBE_ROOT}/hack/lib/init.sh"

kube::golang::setup_env

APIROOT="${KUBE_ROOT}/pkg/api"
TMP_APIROOT="${KUBE_ROOT}/_tmp/api"
_tmp="${KUBE_ROOT}/_tmp"

mkdir -p "${_tmp}"
cp -a "${APIROOT}" "${TMP_APIROOT}"

"${KUBE_ROOT}/hack/update-generated-conversions.sh"
echo "diffing ${APIROOT} against freshly generated conversions"
ret=0
diff -Naupr -I 'Auto generated by' "${APIROOT}" "${TMP_APIROOT}" || ret=$?
cp -a ${TMP_APIROOT} "${KUBE_ROOT}/pkg"
rm -rf "${_tmp}"
if [[ $ret -eq 0 ]]
then
echo "${APIROOT} up to date."
else
echo "${APIROOT} is out of date. Please run hack/update-generated-conversions.sh"
exit 1
fi

# ex: ts=2 sw=2 et filetype=sh
49 changes: 49 additions & 0 deletions hack/after-build/verify-generated-deep-copies.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/bin/bash

# Copyright 2015 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

KUBE_ROOT=$(dirname "${BASH_SOURCE}")/../..
source "${KUBE_ROOT}/hack/lib/init.sh"

kube::golang::setup_env

gendeepcopy=$(kube::util::find-binary "gendeepcopy")

APIROOT="${KUBE_ROOT}/pkg/api"
TMP_APIROOT="${KUBE_ROOT}/_tmp/api"
_tmp="${KUBE_ROOT}/_tmp"

mkdir -p "${_tmp}"
cp -a "${APIROOT}" "${TMP_APIROOT}"

"${KUBE_ROOT}/hack/update-generated-deep-copies.sh"
echo "diffing ${APIROOT} against freshly generated deep copies"
ret=0
diff -Naupr -I 'Auto generated by' "${APIROOT}" "${TMP_APIROOT}" || ret=$?
cp -a ${TMP_APIROOT} "${KUBE_ROOT}/pkg"
rm -rf "${_tmp}"
if [[ $ret -eq 0 ]]
then
echo "${APIROOT} up to date."
else
echo "${APIROOT} is out of date. Please run hack/update-generated-deep-copies.sh"
exit 1
fi

# ex: ts=2 sw=2 et filetype=sh
Loading

0 comments on commit 3b7c396

Please sign in to comment.