@smarterclayton

GCE e2e build/test failed for commit 60fe71a049c49129e03fc23a9c2ce4e40af724d3.

• Build Log
• Test Artifacts
• Internal Jenkins Results

Needs a unit test somehow... no idea how to do that easily though.

http://www.infoq.com/articles/Web-Sockets-Proxy-Servers seems to cover this for WS

@smarterclayton I'm not sure how you'd unit test it, since everything would be running on the same host. But I agree some sort of way to test is highly desirable.

You'd need a connect level proxy - which you could run in a container
during e2e

On Fri, Jul 24, 2015 at 1:18 PM, Andy Goldstein notifications@github.com
wrote:

@smarterclayton https://github.com/smarterclayton I'm not sure how
you'd unit test it, since everything would be running on the same host. But
I agree some sort of way to test is highly desirable.

—
Reply to this email directly or view it on GitHub
#11694 (comment)
.

Clayton Coleman | Lead Engineer, OpenShift

I'm thinking that the way to e2e test this is to do the following:

1. Create a pod with a proxy server such as squid in it
2. Create a pod with kubectl in it
3. Configure the kubectl container such that it's only allowed to talk to the squid pod (or a service fronting it)
4. Have the kubectl container run http_proxy=$squid kubectl -s $apiServer exec $somePod $someCommand

Questions:

1. Do you think this will work? Is there an easier way?
2. Can we use iptables to restrict the kubectl container's outgoing packets?
   1. Is it safe to assume iptables will work inside a container wherever e2e is running?
   2. This means the kubectl container will need to be privileged - is that ok?
   3. Is there some other way to do this?
3. Ideally I'd like to test http_proxy and https_proxy - can we test both http and https with e2e?

@smarterclayton @thockin @bgrant0607 @brendandburns

@kubernetes/rh-cluster-infra @kubernetes/rh-networking

cc @lavalamp

Why do you want to restrict packets?

I'd like to prove without a doubt that proxying is working. If we can't restrict packets, I could write a test that runs http_proxy=... kubectl exec ... and succeeds with the current code, which doesn't support proxying.

regarding iptables - it should work inside containers, but it does require
privileges.

On Wed, Aug 26, 2015 at 6:48 AM, Andy Goldstein notifications@github.com
wrote:

I'd like to prove without a doubt that proxying is working. If we can't
restrict packets, I could write a test that runs http_proxy=... kubectl
exec ... and succeeds with the current code, which doesn't support
proxying.

—
Reply to this email directly or view it on GitHub
#11694 (comment)
.

regarding iptables - it should work inside containers, but it does require
privileges.

Yes, it seems somewhat backwards to require a privileged container to be able to test that proxying is working, but that's all I've been able to come up with so far...

@ncdc could you elaborate? The intended behaviour of the code in portforward is the same, with the only difference that the old one was broken.

@mvdan sorry that comment was meant for the go1.5 race pr

Why not just examine the kubectl log to verify that all traffic went through its proxy?

@lavalamp good idea

Do you all have a preference for what image to use for the proxy? Something with squid or polipo in it?

Actually I'll try nginx since I know e2e already uses that

Looks like nginx doesn't support CONNECT when acting as a forward proxy. That leaves me with squid, polipo, ...

GCE e2e build/test failed for commit 43f4590318ec10fbd9cd47f7364db6751f7d4ba3.

• Build Log
• Test Artifacts
• Internal Jenkins Results

GCE e2e build/test passed for commit b9290c46be53cde9eb9611bdde0e7ca3d1ad67aa.

• Build Log
• Test Artifacts
• Internal Jenkins Results

If we run the forward proxy as a pod, and we run kubectl exec from outside the cluster, there's no easy way for kubectl to use the proxy pod as its http_proxy, since the pod's IP isn't routable from outside the cluster.

We could try to find a way to run kubectl inside the cluster, but that requires that kubectl be built locally and then somehow run in the cluster, perhaps in a custom image that's built on the fly, pushed to a publicly accessible registry, and then run as a pod. I don't like that idea because of the need to build and push a kubectl image somewhere.

Another option would be to run the proxy pod with a hostPort, perhaps, and have kubectl connect to that.

Does anyone have any thoughts about how to put together an e2e test where kubectl uses an http proxy to talk to the apiserver?

Run kubectl in a pod

On Aug 27, 2015, at 5:26 PM, Andy Goldstein notifications@github.com
wrote:

If we run the forward proxy as a pod, and we run kubectl exec from outside
the cluster, there's no easy way for kubectl to use the proxy pod as its
http_proxy, since the pod's IP isn't routable from outside the cluster.

We could try to find a way to run kubectl inside the cluster, but that
requires that kubectl be built locally and then somehow run in the cluster,
perhaps in a custom image that's built on the fly, pushed to a publicly
accessible registry, and then run as a pod. I don't like that idea because
of the need to build and push a kubectl image somewhere.

Another option would be to run the proxy pod with a hostPort, perhaps, and
have kubectl connect to that.

Does anyone have any thoughts about how to put together an e2e test where
kubectl uses an http proxy to talk to the apiserver?

—
Reply to this email directly or view it on GitHub
#11694 (comment)
.

@krousey not fixed yet, @ashcrow is working on something else at the moment, but we'll come back to it soon.

@ncdc Thanks. I mainly wanted to get that status in the main comment stream so that I don't get confused again and again 😄

GCE e2e build/test failed for commit cd497a6f5768a1b3253740261d7bd3128b080abc.

• Build Log
• Test Artifacts
• Internal Jenkins Results

@krousey jumping back over now 😄

GCE e2e test build/test passed for commit 57fc4bf.

• Build Log
• Test Artifacts
• Internal Jenkins Results

@k8s-teamcity-mesosphere failure seems like a flake

[13:54:14][Step 4/4] Error response from daemon: Cannot start container 49b2e6b7cb245420fc0b4b270842a38f781329fabbdb123d1afcac79b12de43c: Cannot link to a non running container: /docker_apiserver_1 AS /prickly_bose/apiserver
[13:54:14][Step 4/4] Dumping logs to '/tmp/kubernetes/log'
[13:54:15][Step 4/4] + timeout 5m ./cluster/kube-down.sh
[13:54:15][Step 4/4]  [1BBringing down cluster using provider: mesos/docker
[13:54:15][Step 4/4] Verifying required commands
[13:54:15][Step 4/4] Stopping mesos/docker cluster

Shippable also looks like a flake. One of them passed in 40 minutes and the other failed in 2 hours.

@krousey mind taking a look now that this no longer forces a static kubectl for everything?

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

GCE e2e build/test failed for commit 57fc4bf.

• Build Log
• Test Artifacts
• Internal Jenkins Results

Unrelated Jenkins e2e error:

• Failure [94.077 seconds]
EmptyDir volumes
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/empty_dir.go:95
  should support (root,0644,tmpfs) [Conformance] [It]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/empty_dir.go:46

  Oct 14 21:49:39.061: Failed to create pod: Pod "pod-243d789a-72f8-11e5-bdf9-42010af0226b" is forbidden: no API token found for service account e2e-tests-emptydir-vjc6c/default, retry after the token is automatically created and added to the service account

  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/util.go:1112

@k8s-bot test this please

GCE e2e test build/test passed for commit 57fc4bf.

• Build Log
• Test Artifacts
• Internal Jenkins Results

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

GCE e2e test build/test passed for commit 57fc4bf.

• Build Log
• Test Artifacts
• Internal Jenkins Results

Automatic merge from submit-queue