Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Kubernetes binaries with valid Semantic Version #57445

Merged
merged 1 commit into from
Feb 14, 2018

Conversation

chenzhiwei
Copy link
Contributor

@chenzhiwei chenzhiwei commented Dec 20, 2017

What this PR does / why we need it:

Refuse to build Kubernetes when a version string like v1.8.3+xx+xx.

This PR is to restrict the KUBE_GIT_VERSION, if the version string does match the rule of Semantic Version, then refuse to build.

Since Kubernetes Conformance test needs the KUBE_GIT_VERSION to be a valid Semantic Version, so I think it's better to restrict the version string.

Release note:

NONE

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Dec 20, 2017
@k8s-ci-robot
Copy link
Contributor

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.


Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 20, 2017
@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Dec 20, 2017
@chenzhiwei
Copy link
Contributor Author

I just signed the CLA.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Dec 20, 2017
@dims
Copy link
Member

dims commented Dec 20, 2017

/ok-to-test

@k8s-ci-robot k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 20, 2017
@eparis
Copy link
Contributor

eparis commented Dec 20, 2017

What created an invalid version which caused this to be a problem and require a patch?

@eparis
Copy link
Contributor

eparis commented Dec 20, 2017

Is this related to #57420 ?

@rphillips
Copy link
Member

This PR doesn't tweak the previous stanza to pull out the minor and major versions like in #57420. In addition, the regex doesn't match on v1.111.0+me.0.

@chenzhiwei
Copy link
Contributor Author

@eparis The K8s Conformance test will fail if the version string contains two + symbol, such as v1.8.3+identifer1+identifer2.

@rphillips It does match the version string v1.111.0+me.0, see here: https://regexr.com/3ia5j

As I understand, the v1.8.3+identifier is a stable release and v1.8.3-identifier is not, since v1.8.3-identifier will cause the Minor version to 8+.

I know this is not a perfect one, but covers more cases.

@rphillips
Copy link
Member

@chenzhiwei I tried this regex in bash, and it failed to parse. see an issue with the script?

#!/usr/bin/env bash

if [[ "v1.111.0+me.0" =~ ^v([0-9]+)\.([0-9]+)(\.[0-9]+)?([-][0-9A-Za-z+-.]*|[+][0-9A-Za-z.]+)?$ ]] ; then
  echo "matched"
fi

@chenzhiwei
Copy link
Contributor Author

chenzhiwei commented Dec 20, 2017

@rphillips here is my result, I just copied your code:

zhiwei@macbook:~$ cat aa.sh
#!/usr/bin/env bash

if [[ "v1.111.0+me.0" =~ ^v([0-9]+)\.([0-9]+)(\.[0-9]+)?([-][0-9A-Za-z+-.]*|[+][0-9A-Za-z.]+)?$ ]] ; then
  echo "matched"
fi
zhiwei@macbook:~$ bash aa.sh
matched

By the way, seems v1.111.0+me.0 is not a valid Semantic Version string, the identifiers after + should be [0-9A-Za-z-].

I just submitted another patch which covers more cases.

^v([0-9]+)\.([0-9]+)(\.[0-9]+)?([-][0-9A-Za-z.-]+|[-][0-9A-Za-z.-]+[+][0-9A-Za-z.-]+|[+][0-9A-Za-z.-]+)?$

Let me describe the last element in ().

[-][0-9A-Za-z.-]+   ==> v1.8.3-alpha.1-rc0
[-][0-9A-Za-z.-]+[+][0-9A-Za-z.-]+   ==> v1.8.3-alpha.1-rc0+identifiers
[+][0-9A-Za-z.-]+  ==>> v1.8.3+identifiers

As said in https://semver.org/, the identifiers after + sign should be [0-9A-Za-z-], but it also says 1.0.0-beta+exp.sha.5114f85 is a valid version string. So, I made a change to identifiers to support . sign.

@chenzhiwei chenzhiwei force-pushed the version-check branch 2 times, most recently from 426c265 to bbfd48b Compare December 20, 2017 15:37
@rphillips
Copy link
Member

the tweak to the regex fixed the parse issue for me. Thanks!

@chenzhiwei
Copy link
Contributor Author

/retest

@eparis
Copy link
Contributor

eparis commented Dec 20, 2017

My question is 'where did the 'illegal' version come from that caused you to write this PR? You say:

The K8s Conformance test will fail if the version string contains two + symbol, such as v1.8.3+identifer1+identifer2.

What is creating something with 2 +'s? It seems to me like the whole point of this PR is to break whatever is doing that, not fix that thing and prevent it from coming back. Show/explain to me how/why this is not a regression.

@chenzhiwei
Copy link
Contributor Author

This is to restrict the KUBE_GIT_VERSION to fully match the Semantic Version rules.

Sometimes, when a user build a Kubernetes binary and he may specify a wrong version such as v1.8.3+id1+id2 by exporting KUBE_GIT_VERSION=v1.8.3+id1+id2.

And Conformance tests will fail when the version string is v1.8.3+id1+id2.

This patch is used to fail early in build steps, not in running the Conformance tests cases. Catch the issue early and fix it early.

Another thing is, if users do not export the KUBE_GIT_VERSION environment variable, the build script will automatically get the version string from git repo. If users on a detached/special branch, the KUBE_GIT_VERSION will be a strange string such as branch-xxx+xxx-dirty.

I want to avoid such cases at the very beginning of the build process, so users can fix it early.

@lavalamp lavalamp assigned ixdy and david-mcmahon and unassigned lavalamp and eparis Jan 25, 2018
@david-mcmahon
Copy link
Contributor

This seems reasonable, but I'm not an OWNER that can approve.
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 26, 2018
@@ -96,6 +96,13 @@ kube::version::get_version_vars() {
KUBE_GIT_MINOR+="+"
fi
fi

# If KUBE_GIT_VERSION is not a valid Semantic Version, then refuse to build.
if ! [[ "${KUBE_GIT_VERSION}" =~ ^v([0-9]+)\.([0-9]+)(\.[0-9]+)?([-][0-9A-Za-z.-]+|[-][0-9A-Za-z.-]+[+][0-9A-Za-z.-]+|[+][0-9A-Za-z.-]+)?$ ]]; then
Copy link
Member

@ixdy ixdy Jan 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit: couldn't the last part of this be simplified to
(-[0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)??

i.e. "an optional pre-release version followed by an optional build metadata section"?

Full regex would look something like ^v([0-9]+)\.([0-9]+)(\.[0-9]+)?(-[0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Thanks, this regx is better.

@k8s-github-robot k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 31, 2018
@ixdy
Copy link
Member

ixdy commented Jan 31, 2018

/lgtm
/retest

unit test failure looks like a flake. cross failure is not you.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 31, 2018
@chenzhiwei
Copy link
Contributor Author

/test pull-kubernetes-cross

@chenzhiwei
Copy link
Contributor Author

/test pull-kubernetes-e2e-kubeadm-gce-canary

/test pull-kubernetes-cross

@ixdy
Copy link
Member

ixdy commented Jan 31, 2018

tracking issue for crossbuild failure is #59121

@ixdy
Copy link
Member

ixdy commented Feb 13, 2018

/retest

@ixdy
Copy link
Member

ixdy commented Feb 13, 2018

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chenzhiwei, david-mcmahon, ixdy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these OWNERS Files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 13, 2018
@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel comment for consistent failures.

@chenzhiwei
Copy link
Contributor Author

/test pull-kubernetes-e2e-kubeadm-gce-canary

@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit 1aa2a02 into kubernetes:master Feb 14, 2018
@k8s-ci-robot
Copy link
Contributor

@chenzhiwei: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-kubeadm-gce-canary 3b8b9e6 link /test pull-kubernetes-e2e-kubeadm-gce-canary
pull-kubernetes-e2e-gce-device-plugin-gpu 3b8b9e6 link /test pull-kubernetes-e2e-gce-device-plugin-gpu

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants