V2: Add e2e tests for NFS and Gluster #7435

jsafrane · 2015-04-28T14:29:08Z

Based on PR #6755 and #7223

there is some common scaffolding to write additional volume tests easily
there are no services for NFS/Gluster servers, the test mounts directly IP address of the server pod
now with Makefiles and Dockerfiles in /contrib/for-tests/volumes/*
- the images are based on Ubuntu 14.04, like several /cluster/addons and number of examples
- can someone please push the images to gcr.io/google_containers/?

The test must be run with --allow_privileged=True everywhere, the server pods won't work without that. Should it be made default for e2e tests?

googlebot · 2015-04-28T14:29:09Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project, in which case you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please let us know the company's name.

a-robinson · 2015-04-28T17:03:26Z

@thockin @vishh

roberthbailey · 2015-05-04T22:25:26Z

@thockin if you won't have time to review this yourself, can you please find someone to take a look at it?

jayunit100 · 2015-05-11T03:43:21Z

test/e2e/volumes.go

+		var err error
+		c, err = loadClient()
+		Expect(err).NotTo(HaveOccurred())
+		namespace = "e2e-ns-" + randomSuffix()


You'll want to use namespace lifecycle as we do in networking.go....

In BeforeEach

By("Building a namespace api object") namespace, err = createTestingNS("nettest", c) Expect(err).NotTo(HaveOccurred())

And in AfterEach

if err := c.Namespaces().Delete(namespace.Name); err != nil { Failf("Couldn't delete ns %s", err) }

timothysc · 2015-05-14T16:17:40Z

@thockin, @zmerlynn, @roberthbailey, @quinton-hoole, - reping.

I know everyone is busy, but this is pretty important from our side, can we delegate here?

ghost · 2015-05-14T17:55:50Z

I can take a look at this today.

ghost · 2015-05-14T19:25:44Z

Build failed on both Travis and Shippable for the same reaon:

./hack/verify-boilerplate.sh
Boilerplate header is wrong for: ./test/e2e/volumes.go

jayunit100 · 2015-05-14T19:34:47Z

Fyi it's easy to fix just copy the new header over From any current files the owner changed to kubernetes from google. This happens on prs which predate the owner change

ghost · 2015-05-14T20:32:20Z

contrib/for-tests/volumes/gluster/Dockerfile

@@ -0,0 +1,11 @@
+FROM ubuntu:14.04


You're going to need to include the standard copyright and license blurb at the head of every new file that you add.

Given that every other directory under for-tests/ follows the naming convention of xxx-tester, is there any reason not to follow that convention here?

As per previous comment, can we not get by with a smaller base container image here. This one includes "ubuntu universe" which seems excessive.

Copyright header added to all Dockerfiles.

jsafrane · 2015-05-20T07:18:32Z

It's hard to believe that both NFS and Gluster are each twice the size of a whole operating system base image? But if that is really the case, I guess switching to Debian as the base OS is the best we can do. But you can defer that to a separate PR if you like. Consider it a nit, given that the difference in size between Ubuntu and Debian base is only about 50MB, less than 10% of the total, assuming that your numbers are correct. If you decide to do that, please open a separate issue and cross-reference this one.

Ok, I'll re-base the images to Debian in separate PR when this one is merged.

I don't really understand why you cannot run an NFS or Gluster server in a non-privileged container? If you can get the servers to run in non-privileged containers, then we can enable these tests by default.

NFS container needs to mount /proc/fs/nfsd. I don't know exactly what Gluster container needs, but it does not work in unprivileged mode.

jsafrane · 2015-05-20T07:20:08Z

Shippable — Builds failed on Shippable

Any idea what's wrong? Shippable.com shows me endless "Waiting for build information...".

timothysc · 2015-05-20T18:38:31Z

@jsafrane - Shippable is a false positive.

ghost · 2015-05-22T17:37:33Z

Please squash into a single commit, and then this is LGTM to merge.

timothysc · 2015-05-22T17:43:04Z

/cc @eparis

timothysc · 2015-05-22T18:00:19Z

contrib/for-tests/volumes-tester/gluster/Dockerfile

+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM ubuntu:14.04


Centos please.

- add appropriate server containers into contrib/for-tests/volumes-tester - the tests are off by default (they need kubelet --allow_privileged=True) - enable by 'go run hack/e2e.go ... --ginkgo.focus=Volume' - add glusterfs tools to list of installed packages on each node

jsafrane · 2015-05-25T10:06:08Z

Squashed. I'll look into squeezing the size of test images in a separate PR.

timothysc · 2015-05-26T20:49:47Z

@jsafrane great!
@quinton-hoole PTAL por favor.

ghost · 2015-05-26T21:13:18Z

LGTM. @saad-ali will merge as soon as is practical. There is a significant PR backlog, so it might take a few hours.

timothysc · 2015-05-26T21:18:04Z

ack, thx.

V2: Add e2e tests for NFS and Gluster

timothysc · 2015-05-27T14:11:13Z

@saad-ali @brendanburns - could we get an image upload for the tests to gcr.io/google_containers loc?

saad-ali · 2015-05-27T20:28:37Z

@timothysc I'll take a crack at pushing the gluster and nfs images to gcr.io

timothysc · 2015-05-27T21:09:09Z

@saad-ali many thanks.

saad-ali · 2015-05-27T21:50:48Z

Pushed volume-gluster and volume-nfs to gcr.io/google_containers.
The following commands should work now:

gcloud preview docker pull gcr.io/google_containers/volume-gluster:0.1
gcloud preview docker pull gcr.io/google_containers/volume-nfs:0.1

timothysc · 2015-05-28T14:22:43Z

test/e2e/volumes.go

+			config := VolumeTestConfig{
+				namespace:   namespace.Name,
+				prefix:      "nfs",
+				serverImage: "gcr.io/google_containers/volume-nfs",


@saad-ali, @jsafrane, @childsb
So both the nfs and gluster tests still fail b/c tag is missing.

docker pull gcr.io/google_containers/volume-nfs

Pulling repository gcr.io/google_containers/volume-nfs
FATA[0001] Tag latest not found in repository gcr.io/google_containers/volume-nfs

docker pull gcr.io/google_containers/volume-nfs:0.1

Pulling repository gcr.io/google_containers/volume-nfs
7faef5ba165a: Download complete
e9e06b06e14c: Download complete
a82efea989f9: Download complete
37bea4ee0c81: Download complete
07f8e8c5e660: Download complete
a61fbd375433: Download complete
36ed446fd761: Download complete
5c06e7e161bd: Download complete
087a9dde26b9: Download complete
56da8d2b187a: Download complete
8cb003e95f03: Download complete
9570a95e2d95: Download complete
Status: Downloaded newer image for gcr.io/google_containers/volume-nfs:0.1

saad-ali · 2015-05-28T19:00:12Z

@timothysc Yep, the tagging is screwed up, Since the nfs and gluster Makefiles manually sets a version, docker doesn't automatically attach the latest tag. I'll take a crack at fixing this today.

saad-ali · 2015-05-28T19:36:22Z

Ok, the following commands should all work now. I'll send out a PR with the updated Makefiles

gcloud preview docker pull gcr.io/google_containers/volume-gluster:0.1
gcloud preview docker pull gcr.io/google_containers/volume-nfs:0.1
gcloud preview docker pull gcr.io/google_containers/volume-gluster:latest
gcloud preview docker pull gcr.io/google_containers/volume-nfs:latest
gcloud preview docker pull gcr.io/google_containers/volume-gluster
gcloud preview docker pull gcr.io/google_containers/volume-nfs

(Note for the last two, they default to :latest)

googlebot added the cla: no label Apr 28, 2015

This was referenced Apr 28, 2015

Add end-to-end test for NFS volumes. #6755

Closed

RFC: E2E test for GlusterFS #7223

Closed

roberthbailey assigned thockin May 4, 2015

jayunit100 reviewed May 11, 2015
View reviewed changes

ghost assigned ghost and unassigned thockin May 14, 2015

ghost reviewed May 14, 2015
View reviewed changes

googlebot added cla: no and removed cla: yes labels May 20, 2015

timothysc reviewed May 22, 2015
View reviewed changes

jsafrane force-pushed the devel/volume-tests branch from 6dc23f9 to 20004e0 Compare May 25, 2015 10:04

ghost added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 26, 2015

saad-ali added a commit that referenced this pull request May 27, 2015

Merge pull request #7435 from jsafrane/devel/volume-tests

bed9f82

V2: Add e2e tests for NFS and Gluster

saad-ali merged commit bed9f82 into kubernetes:master May 27, 2015

timothysc reviewed May 28, 2015
View reviewed changes

saad-ali mentioned this pull request May 28, 2015

Fix makefiles for NFS and Gluster images #8942

Merged

markturansky mentioned this pull request Jun 4, 2015

Fixes nfs-server security issue in image #9019

Merged

rootfs mentioned this pull request Jun 12, 2015

add Ceph FS volume plugin #6649

Merged

jsafrane unassigned ghost Aug 12, 2015

jsafrane deleted the devel/volume-tests branch August 19, 2016 11:10

ixdy mentioned this pull request Sep 10, 2018

Fix small typo - "nfds" #68401

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V2: Add e2e tests for NFS and Gluster #7435

V2: Add e2e tests for NFS and Gluster #7435

jsafrane commented Apr 28, 2015

googlebot commented Apr 28, 2015

a-robinson commented Apr 28, 2015

roberthbailey commented May 4, 2015

jayunit100 May 11, 2015

jsafrane May 11, 2015

timothysc commented May 14, 2015

ghost commented May 14, 2015

ghost commented May 14, 2015

jayunit100 commented May 14, 2015

ghost May 14, 2015

ghost May 14, 2015

jsafrane May 15, 2015

jsafrane commented May 20, 2015

jsafrane commented May 20, 2015

timothysc commented May 20, 2015

ghost commented May 22, 2015

timothysc commented May 22, 2015

timothysc May 22, 2015

jsafrane commented May 25, 2015

timothysc commented May 26, 2015

ghost commented May 26, 2015

timothysc commented May 26, 2015

timothysc commented May 27, 2015

saad-ali commented May 27, 2015

timothysc commented May 27, 2015

saad-ali commented May 27, 2015

timothysc May 28, 2015

saad-ali commented May 28, 2015

saad-ali commented May 28, 2015

V2: Add e2e tests for NFS and Gluster #7435

V2: Add e2e tests for NFS and Gluster #7435

Conversation

jsafrane commented Apr 28, 2015

googlebot commented Apr 28, 2015

a-robinson commented Apr 28, 2015

roberthbailey commented May 4, 2015

jayunit100 May 11, 2015

Choose a reason for hiding this comment

jsafrane May 11, 2015

Choose a reason for hiding this comment

timothysc commented May 14, 2015

ghost commented May 14, 2015

ghost commented May 14, 2015

jayunit100 commented May 14, 2015

ghost May 14, 2015

Choose a reason for hiding this comment

ghost May 14, 2015

Choose a reason for hiding this comment

jsafrane May 15, 2015

Choose a reason for hiding this comment

jsafrane commented May 20, 2015

jsafrane commented May 20, 2015

timothysc commented May 20, 2015

ghost commented May 22, 2015

timothysc commented May 22, 2015

timothysc May 22, 2015

Choose a reason for hiding this comment

jsafrane commented May 25, 2015

timothysc commented May 26, 2015

ghost commented May 26, 2015

timothysc commented May 26, 2015

timothysc commented May 27, 2015

saad-ali commented May 27, 2015

timothysc commented May 27, 2015

saad-ali commented May 27, 2015

timothysc May 28, 2015

Choose a reason for hiding this comment

docker pull gcr.io/google_containers/volume-nfs

docker pull gcr.io/google_containers/volume-nfs:0.1

saad-ali commented May 28, 2015

saad-ali commented May 28, 2015