Skip to content

Commit

Permalink
Merge pull request kubernetes#25978 from thockin/dont-checkin-generat…
Browse files Browse the repository at this point in the history 
…ed-code

Automatic merge from submit-queue

Don't check in generated code, part 1

This PR is a first step towards not commiting generated files, which make up a huge portion of "needs rebase" errors.  It only handles deep-copy generation and conversion generation.  More will come later, if the model passes muster.

This is a mega-PR.  Sorry.  It was necessary to do 2 generators to convince myself it worked, and the evolution of the techniques warranted multiple commits.  I have tried to keep the commits self-contained and reviewable.  

A quick summary of the major points in the series:

- Start by making everything call `make` rather than the various hack/* scripts.  The hack scripts still exist, but give a warning to use make instead, and then they do what they did before, so it should be compatible.

- Move deepcopy generation into the Makefile, so it is done automatically

- Move conversion generation into the Makefile, so it is done automatically

- Optimize makefile for faster rebuilds

- Make CI pass


Net result: if you run "make", it will rebuild any deepcopy or conversion files it needs.  It takes a few seconds to figure out there's nothing to do, but it should be a net savings.  There is more to do, and we can follow this up with other generators being converted, some of which are MUCH slower than these 2.


@wojtek-t @lavalamp @smarterclayton @bgrant0607 @mikedanese @madhusudancs
  • Loading branch information
k8s-merge-robot authored Jul 13, 2016
2 parents ec47f5c + fef16dd commit eecbfb1
Show file tree
Hide file tree
Showing 117 changed files with 3,957 additions and 29,880 deletions.
6 changes: 6 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,9 @@ network_closure.sh

# Downloaded kubernetes binary release tar ball
kubernetes.tar.gz

# generated files in any directory
zz_generated.*

# make-related metadata
/.make/
173 changes: 125 additions & 48 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,55 @@
# See the License for the specific language governing permissions and
# limitations under the License.

DBG_MAKEFILE ?=
ifeq ($(DBG_MAKEFILE),1)
$(warning ***** starting makefile for goal(s) "$(MAKECMDGOALS)")
$(warning ***** $(shell date))
else
# If we're not debugging the Makefile, don't echo recipes.
MAKEFLAGS += -s
endif


# Old-skool build tools.
#
# Targets (see each target for more information):
# Commonly used targets (see each target for more information):
# all: Build code.
# check: Run tests.
# test: Run tests.
# clean: Clean up.

OUT_DIR = _output
# It's necessary to set this because some docker images don't make sh -> bash.
SHELL := /bin/bash

# We don't need make's built-in rules.
MAKEFLAGS += --no-builtin-rules
.SUFFIXES:

# We want make to yell at us if we use undefined variables.
MAKEFLAGS += --warn-undefined-variables

# Constants used throughout.
OUT_DIR ?= _output
BIN_DIR := $(OUT_DIR)/bin
PRJ_SRC_PATH := k8s.io/kubernetes

# Metadata for driving the build lives here.
META_DIR := .make

#
# Define variables that we use as inputs so we can warn about undefined variables.
#

WHAT ?=
TESTS ?=

GOFLAGS ?=
KUBE_GOFLAGS = $(GOFLAGS)
export KUBE_GOFLAGS
export KUBE_GOFLAGS GOFLAGS

GOLDFLAGS ?=
KUBE_GOLDFLAGS = $(GOLDFLAGS)
export KUBE_GOLDFLAGS
export KUBE_GOLDFLAGS GOLDFLAGS

# Build code.
#
Expand All @@ -41,16 +75,16 @@ export KUBE_GOLDFLAGS
# make
# make all
# make all WHAT=cmd/kubelet GOFLAGS=-v
all:
hack/build-go.sh $(WHAT)
.PHONY: all
all: generated_files
hack/make-rules/build.sh $(WHAT)

# Build ginkgo
#
# Example:
# make ginkgo
ginkgo:
hack/build-go.sh vendor/github.com/onsi/ginkgo/ginkgo
hack/make-rules/build.sh vendor/github.com/onsi/ginkgo/ginkgo
.PHONY: ginkgo

# Runs all the presubmission verifications.
Expand All @@ -61,9 +95,9 @@ ginkgo:
# Example:
# make verify
# make verify BRANCH=branch_x
verify:
KUBE_VERIFY_GIT_BRANCH=$(BRANCH) hack/verify-all.sh -v
.PHONY: verify
verify:
KUBE_VERIFY_GIT_BRANCH=$(BRANCH) hack/make-rules/verify.sh -v

# Build and run tests.
#
Expand All @@ -78,91 +112,134 @@ verify:
# make check
# make test
# make check WHAT=pkg/kubelet GOFLAGS=-v
check test:
hack/test-go.sh $(WHAT) $(TESTS)
.PHONY: check test
check test: generated_files
hack/make-rules/test.sh $(WHAT) $(TESTS)

# Build and run integration tests.
#
# Example:
# make test_integration
test_integration:
hack/test-integration.sh
.PHONY: test_integration test_integ
# make test-integration
.PHONY: test-integration
test-integration: generated_files
hack/make-rules/test-integration.sh

# Build and run end-to-end tests.
#
# Example:
# make test_e2e
test_e2e:
# make test-e2e
.PHONY: test-e2e
test-e2e: ginkgo generated_files
go run hack/e2e.go -v --build --up --test --down
.PHONY: test_e2e

# Build and run node end-to-end tests.
#
# Args:
# FOCUS: regexp that matches the tests to be run. Defaults to "".
# SKIP: regexp that matches the tests that needs to be skipped. Defaults to "".
# RUN_UNTIL_FAILURE: Ff true, pass --untilItFails to ginkgo so tests are run repeatedly until they fail. Defaults to false.
# REMOTE: If true, run the tests on a remote host instance on GCE. Defaults to false.
# IMAGES: for REMOTE=true only. Comma delimited list of images for creating remote hosts to run tests against. Defaults to "e2e-node-containervm-v20160321-image".
# LIST_IMAGES: If true, don't run tests. Just output the list of available images for testing. Defaults to false.
# HOSTS: for REMOTE=true only. Comma delimited list of running gce hosts to run tests against. Defaults to "".
# DELETE_INSTANCES: for REMOTE=true only. Delete any instances created as part of this test run. Defaults to false.
# ARTIFACTS: for REMOTE=true only. Local directory to scp test artifacts into from the remote hosts. Defaults to ""/tmp/_artifacts".
# REPORT: for REMOTE=false only. Local directory to write juntil xml results to. Defaults to "/tmp/".
# CLEANUP: for REMOTE=true only. If false, do not stop processes or delete test files on remote hosts. Defaults to true.
# IMAGE_PROJECT: for REMOTE=true only. Project containing images provided to IMAGES. Defaults to "kubernetes-node-e2e-images".
# INSTANCE_PREFIX: for REMOTE=true only. Instances created from images will have the name "${INSTANCE_PREFIX}-${IMAGE_NAME}". Defaults to "test"/
# FOCUS: Regexp that matches the tests to be run. Defaults to "".
# SKIP: Regexp that matches the tests that needs to be skipped. Defaults
# to "".
# RUN_UNTIL_FAILURE: If true, pass --untilItFails to ginkgo so tests are run
# repeatedly until they fail. Defaults to false.
# REMOTE: If true, run the tests on a remote host instance on GCE. Defaults
# to false.
# IMAGES: For REMOTE=true only. Comma delimited list of images for creating
# remote hosts to run tests against. Defaults to a recent image.
# LIST_IMAGES: If true, don't run tests. Just output the list of available
# images for testing. Defaults to false.
# HOSTS: For REMOTE=true only. Comma delimited list of running gce hosts to
# run tests against. Defaults to "".
# DELETE_INSTANCES: For REMOTE=true only. Delete any instances created as
# part of this test run. Defaults to false.
# ARTIFACTS: For REMOTE=true only. Local directory to scp test artifacts into
# from the remote hosts. Defaults to ""/tmp/_artifacts".
# REPORT: For REMOTE=false only. Local directory to write juntil xml results
# to. Defaults to "/tmp/".
# CLEANUP: For REMOTE=true only. If false, do not stop processes or delete
# test files on remote hosts. Defaults to true.
# IMAGE_PROJECT: For REMOTE=true only. Project containing images provided to
# IMAGES. Defaults to "kubernetes-node-e2e-images".
# INSTANCE_PREFIX: For REMOTE=true only. Instances created from images will
# have the name "${INSTANCE_PREFIX}-${IMAGE_NAME}". Defaults to "test".
#
# Example:
# make test_e2e_node FOCUS=kubelet SKIP=container
# make test_e2e_node REMOTE=true DELETE_INSTANCES=true
# make test-e2e-node FOCUS=kubelet SKIP=container
# make test-e2e-node REMOTE=true DELETE_INSTANCES=true
# Build and run tests.
test_e2e_node: ginkgo
hack/e2e-node-test.sh
.PHONY: test_e2e_node
.PHONY: test-e2e-node
test-e2e-node: ginkgo generated_files
hack/make-rules/test-e2e-node.sh

# Build and run cmdline tests.
#
# Example:
# make test-cmd
test-cmd:
@hack/make-rules/test-cmd.sh
.PHONY: test-cmd

# Remove all build artifacts.
#
# Example:
# make clean
clean:
.PHONY: clean
clean: clean_generated clean_meta
build/make-clean.sh
rm -rf $(OUT_DIR)
rm -rf Godeps/_workspace # Just until we are sure it is gone
.PHONY: clean

# Remove make-related metadata files.
#
# Example:
# make clean_meta
.PHONY: clean_meta
clean_meta:
rm -rf $(META_DIR)

# Run 'go vet'.
#
# Args:
# WHAT: Directory names to vet. All *.go files under these
# directories will be vetted. If not specified, "everything" will be
# vetted.
# TESTS: Same as WHAT.
# GOFLAGS: Extra flags to pass to 'go' when building.
# GOLDFLAGS: Extra linking flags to pass to 'go' when building.
#
# Example:
# make vet
# make vet WHAT=pkg/kubelet
vet:
hack/verify-govet.sh $(WHAT) $(TESTS)
.PHONY: vet
vet:
hack/make-rules/vet.sh $(WHAT)

# Build a release
#
# Example:
# make release
release:
build/release.sh
.PHONY: release
release: generated_files
build/release.sh

# Build a release, but skip tests
#
# Example:
# make release-skip-tests
release-skip-tests quick-release:
KUBE_RELEASE_RUN_TESTS=n KUBE_FASTBUILD=true build/release.sh
.PHONY: release-skip-tests quick-release
release-skip-tests quick-release: generated_files
KUBE_RELEASE_RUN_TESTS=n KUBE_FASTBUILD=true build/release.sh

# Cross-compile for all platforms
#
# Example:
# make cross
.PHONY: cross
cross:
hack/make-rules/cross.sh

# Add rules for all directories in cmd/
#
# Example:
# make kubectl kube-proxy
.PHONY: $(notdir $(abspath $(wildcard cmd/*/)))
$(notdir $(abspath $(wildcard cmd/*/))): generated_files
hack/make-rules/build.sh cmd/$@

# Include logic for generated files.
include Makefile.generated_files
Loading

0 comments on commit eecbfb1

Please sign in to comment.