add junit recording in e2e runner #28133

freehan · 2016-06-27T22:10:34Z

2nd part of #27825

Injected junit recorder at a few places.

lavalamp · 2016-06-27T22:18:43Z

hack/jenkins/e2e-runner.sh

@@ -21,8 +21,53 @@ set -o nounset
 set -o pipefail
 set -o xtrace

+# include shell2junit library
+source <(curl -fsS --retry 3 'https://raw.githubusercontent.com/kubernetes/kubernetes/master/third_party/forked/shell2junit/sh2ju.sh')


Why do you need to curl this? Shouldn't this file be included in whatever PR is getting tested?

IMO it's not good to get it from master-- if it breaks at head it breaks everywhere...

The reason we need to curl it is because e2e-runner is running as standalone script on jenkins. It does not come with the whole repo.

How is this running standalone? Don't you need to clone this repo to get access to the script in the first place?

Nope. e2e-runner will pull k8s binary from gcs.
@ixdy

@freehan is correct. We curl down e2e-runner.sh which downloads the kubernetes tar from GCS.

lavalamp · 2016-06-27T22:20:32Z

hack/jenkins/e2e-runner.sh

    echo "${difference}"
    echo "!!! FAIL: Google Cloud Platform resources leaked while running tests!"
-    exit 1


Does the presence of the junit file cause a failure now? Or is an exit 1 still needed somewhere?

we do not need to exit here because the record_command will exit upon failure. false is treated as failure.

I typically like to return control to main and exit from there, but it's a style choice.

errexit is set, hence e2e runner may exit anywhere.

k8s-bot · 2016-06-27T22:44:02Z

GCE e2e build/test passed for commit 5753d0b13a875fafbd808319f46d0b2667d1d283.

k8s-bot · 2016-06-27T22:54:09Z

GCE e2e build/test passed for commit 0b2922834a2adc130539f915f2ffb23f8e404f71.

k8s-bot · 2016-06-27T22:57:14Z

GCE e2e build/test passed for commit cb9f966e0a34f7650edfcbb32ef8087cf9d99e0a.

freehan · 2016-06-27T23:01:16Z

FYI, the junit result showed up.

See TEST-CLEANUP.xml at here

david-mcmahon · 2016-06-28T00:23:34Z

hack/jenkins/e2e-runner.sh

+function record_command() {
+    set +o xtrace
+    set +o nounset
+    set +o errexit


Why set + here and then re-set - below?

for xtrace, if we leave it on, it will trace all commands in juLog, which is way too noisy.

juLog made the assumption that nounset and errexit were not set. I want to ensure it won't exit accidentally.

ixdy · 2016-06-28T21:20:23Z

I think our tooling looks for junit files named junit*.xml, so maybe we want to change the names?

freehan · 2016-06-28T21:26:32Z

Oh yes. Will change that.

ixdy · 2016-06-28T21:33:49Z

hack/jenkins/e2e-runner.sh

 : ${KUBE_GCS_RELEASE_BUCKET:="kubernetes-release"}

+# record_command records the command output and error message in junit format


it's not super clear to me from the name/comments that this actually runs the command too,

lavalamp · 2016-06-28T22:23:50Z

hack/jenkins/e2e-runner.sh

+    local -r found=$(echo "${ret}" | grep -c "+++ exit code: 0")
+    if [[ ${found} -le 0 ]]; then
+        echo "Last command failed, exiting..."
+        exit 1


The goal here is to get a build marked "unstable" instead of "failed" which I understand means, now that I unfortunately learned about jenkins, that we shouldn't exit at all (we want to run all the commands) and should print "Test Suite Failed" so that jenkins will mark it unstable (@ixdy is about to turn that plugin on). @ixdy, sound correct?

Hold on that. @fejta is trying to convince us that we shouldn't care about
the difference and that it's perfectly valid to fail like this.

On Tue, Jun 28, 2016 at 3:23 PM, Daniel Smith notifications@github.com
wrote:

In hack/jenkins/e2e-runner.sh
#28133 (comment)
:

shift

local name=$1

shift

echo "Recording: ${class} ${name}"

echo "Running command: $@"

local -r ret=$(juLog -output="${ARTIFACTS}" -class="${class}" -name="${name}" "$@")

set -o nounset

set -o errexit

set -o xtrace

since record_command disable errexit, exit script on failure.

local -r found=$(echo "${ret}" | grep -c "+++ exit code: 0")

if [[ ${found} -le 0 ]]; then

echo "Last command failed, exiting..."

exit 1

The goal here is to get a build marked "unstable" instead of "failed"
which I understand means, now that I unfortunately learned about jenkins,
that we shouldn't exit at all (we want to run all the commands) and should
print "Test Suite Failed" so that jenkins will mark it unstable (@ixdy
https://github.com/ixdy is about to turn that plugin on). @ixdy
https://github.com/ixdy, sound correct?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/kubernetes/pull/28133/files/498735ea3364f419c967944e9188dba4268b6ca4#r68855336,
or mute the thread
https://github.com/notifications/unsubscribe/AAnglgA-mudyhnqfiFTu9cPQ0RlIcgr5ks5qQZ7_gaJpZM4I_jtj
.

k8s-bot · 2016-06-28T22:43:35Z

GCE e2e build/test passed for commit 498735ea3364f419c967944e9188dba4268b6ca4.

k8s-bot · 2016-06-30T00:22:58Z

GCE e2e build/test passed for commit 586bb1f.

freehan · 2016-06-30T17:12:59Z

ping

lavalamp · 2016-07-08T17:11:22Z

Whoops, sorry, didn't realize this was waiting on me. LGTM

k8s-bot · 2016-07-09T06:13:12Z

GCE e2e build/test passed for commit 586bb1f.

spxtr · 2016-07-09T06:22:37Z

    <testsuite failures="0" assertions="" name="CLEANUP" tests="1" errors="0" time="">

    <testcase assertions="1" name="gcp_resource_leak_check" time="" classname="CLEANUP">

    <system-out>
<![CDATA[

+++ Running case: CLEANUP.gcp_resource_leak_check 
+++ working dir: /var/lib/jenkins/workspace/kubernetes-pull-build-test-e2e-gce/kubernetes
+++ command: true
+++ exit code: 0
]]>
    </system-out>
    <system-err>
<![CDATA[

]]>
    </system-err>
    </testcase>

    </testsuite>

That's pretty strange looking. I suppose it's valid.

k8s-bot · 2016-07-09T06:50:56Z

GCE e2e build/test passed for commit 586bb1f.

k8s-github-robot · 2016-07-09T08:19:53Z

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

k8s-bot · 2016-07-09T08:54:39Z

GCE e2e build/test passed for commit 586bb1f.

k8s-github-robot · 2016-07-09T08:55:07Z

Automatic merge from submit-queue

freehan added the release-note-none Denotes a PR that doesn't merit a release note. label Jun 27, 2016

freehan assigned lavalamp Jun 27, 2016

googlebot added the cla: yes label Jun 27, 2016

k8s-github-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 27, 2016

freehan force-pushed the e2ejunit branch from 5753d0b to cb9f966 Compare June 27, 2016 22:15

lavalamp reviewed Jun 27, 2016
View reviewed changes

freehan force-pushed the e2ejunit branch from cb9f966 to 0b29228 Compare June 27, 2016 22:19

lavalamp reviewed Jun 27, 2016
View reviewed changes

david-mcmahon reviewed Jun 28, 2016
View reviewed changes

ixdy reviewed Jun 28, 2016
View reviewed changes

freehan force-pushed the e2ejunit branch from 0b29228 to 498735e Compare June 28, 2016 22:03

lavalamp reviewed Jun 28, 2016
View reviewed changes

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 29, 2016

freehan added 2 commits June 29, 2016 16:46

add junit recording in e2e runner

a5c1e02

rename junit file

586bb1f

freehan force-pushed the e2ejunit branch from 498735e to 586bb1f Compare June 29, 2016 23:49

k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 29, 2016

krousey mentioned this pull request Jun 30, 2016

Cleanup third party (pt 2) #28204

Merged

lavalamp assigned ixdy and david-mcmahon Jul 8, 2016

lavalamp added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 9, 2016

k8s-github-robot merged commit 6eae13e into kubernetes:master Jul 9, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add junit recording in e2e runner #28133

add junit recording in e2e runner #28133

freehan commented Jun 27, 2016

lavalamp Jun 27, 2016

lavalamp Jun 27, 2016

freehan Jun 27, 2016

david-mcmahon Jun 28, 2016

freehan Jun 28, 2016

spxtr Jul 8, 2016

lavalamp Jun 27, 2016

freehan Jun 27, 2016

david-mcmahon Jun 28, 2016

freehan Jun 28, 2016

k8s-bot commented Jun 27, 2016

k8s-bot commented Jun 27, 2016

k8s-bot commented Jun 27, 2016

freehan commented Jun 27, 2016 •

edited

Loading

david-mcmahon Jun 28, 2016

freehan Jun 28, 2016

ixdy commented Jun 28, 2016

freehan commented Jun 28, 2016

ixdy Jun 28, 2016

lavalamp Jun 28, 2016

lavalamp Jun 28, 2016

since record_command disable errexit, exit script on failure.

k8s-bot commented Jun 28, 2016

k8s-bot commented Jun 30, 2016

freehan commented Jun 30, 2016

lavalamp commented Jul 8, 2016

k8s-bot commented Jul 9, 2016

spxtr commented Jul 9, 2016

k8s-bot commented Jul 9, 2016

k8s-github-robot commented Jul 9, 2016

k8s-bot commented Jul 9, 2016

k8s-github-robot commented Jul 9, 2016

		: ${KUBE_GCS_RELEASE_BUCKET:="kubernetes-release"}

		# record_command records the command output and error message in junit format

add junit recording in e2e runner #28133

add junit recording in e2e runner #28133

Conversation

freehan commented Jun 27, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-bot commented Jun 27, 2016

k8s-bot commented Jun 27, 2016

k8s-bot commented Jun 27, 2016

freehan commented Jun 27, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ixdy commented Jun 28, 2016

freehan commented Jun 28, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

since record_command disable errexit, exit script on failure.

k8s-bot commented Jun 28, 2016

k8s-bot commented Jun 30, 2016

freehan commented Jun 30, 2016

lavalamp commented Jul 8, 2016

k8s-bot commented Jul 9, 2016

spxtr commented Jul 9, 2016

k8s-bot commented Jul 9, 2016

k8s-github-robot commented Jul 9, 2016

k8s-bot commented Jul 9, 2016

k8s-github-robot commented Jul 9, 2016

freehan commented Jun 27, 2016 •

edited

Loading