boot2docker is Linux. It doesn't support iptables or the commands are different? AFAICT, Tiny Core Linux does support iptables: http://gr8idea.info/os/tutorials/tiny-core/, and boot2docker even relies on them: boot2docker/boot2docker#561.

Ah ok to clarify - boot2docker is a light weight linux vm for docker which makes it feel similar to running docker natively. So you would run Kubernetes / Openshift / development tools etc as normal on the OS X host and point DOCKER_HOST to the boot2docker VM. So in the setup I had working before everything apart from docker ran on my Mac and therefore no iptables command.

Yeah - the scenario we're most interested in is being able to run local Kube clusters on Mac/Windows pointing to a VM with Docker:

• A developer can test and develop local Kube configurations without having to use another cluster
• A developer of Kubernetes can iterate on Kubernetes itself on Mac

The key gaps I'm aware of are

• IP per service being hard to emulate without something like iptables
• Volumes being mapped into the VM

In order to make this work we'd need some simple abstractions around kube-proxy initialization (use a different iptables mechanism), or look at having a very simple mode for IP allocation that is designed for a single host. The volume problem would potentially work today by choosing a --volumes dir.

We'd be willing to help contribute to make this possible since want to bring Windows/Java workloads into Kubernetes and offer local environments for that sort of testing.

That's great and I'll help in any way I can. If you need me to do any investigation / testing into the two key gaps then let me know.

osx/darwin uses pf (http://www.openbsd.org/faq/pf/) instead of iptables. I'd recommend we abstract things even further so that e.g. proxier.go can support more than just iptables (https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/proxy/proxier.go#L536 etc).

@thockin would you support more abstraction here?

The abstraction is really AddPortal RemovePortal, Cleanup, right? If we could do the refactor to get down to that interface, we could do the simple implementation for other platforms.

Actually, the simplest possible stub (for development style environments) is IPAllocator returning one address (the host) and the proxy listening on the host. For a lot of people that would be enough to do simple Kube testing. Adding PF is a smaller step up.

In either case, we're willing to do the work to support those folks, just want to get signoff on a minimal abstraction to do it.

I'm not against abstracting that (heck, Docker and the whole damn thing
should run on MacOS). That said, wouldn't it be easier to run the master
in another VM?

On Thu, Nov 6, 2014 at 7:35 AM, Clayton Coleman notifications@github.com
wrote:

The abstraction is really AddPortal RemovePortal, Cleanup, right? If we
could do the refactor to get down to that interface, we'd do the simple
stub implementation for other platforms.

Actually, the simplest possible stub (for development style environments)
is IPAllocator returning one address (the host) and the proxy listening on
the host. For a lot of people that would be enough to do simple Kube
testing. Adding PF is a smaller step up.

In either case, we're willing to do the work to support those folks, just
want to get signoff on a minimal abstraction to do it.

Reply to this email directly or view it on GitHub
#2170 (comment)
.

What ever is easiest is definitely good. The issue is at the moment for people that don't develop on linux there are a number of extra steps and working with vms has its own challenges especially when all the dev tools run on the host. These are some of the recent attempts on the Openshift vagrant image openshift/origin#365 and I've had similar issues with the Kubernetes vagrant file. Performance of the vms is also an issue and for the standard java developer it would be good to find a quick, easy and reliable way to get them up and running so they can reap the benefits of Kubernetes early on. If running a kube master in a boot2docker way is possible so that it feels similar to running natively then thats a big +1

We can use arbitrary remote docker hosts now for building, even over TLS, on master. I'm looking at what it would take to get the rest of the dev setup going locally on Mac.

Has any progress or decisions been made on the above issues so far?

It's still something we could go do - for OpenShift we were able to run standalone in a docker image and some of us cribbed together remote docker flows for pushing binaries into the image as well. I'm not aware of anything added to master that makes this more difficult except the public ip work Brendan did around services (adds more iptables rules).

On Dec 2, 2014, at 5:33 PM, James Kyle notifications@github.com wrote:

We can use arbitrary remote docker hosts now for building, even over TLS, on master. I'm looking at what it would take to get the rest of the dev setup going locally on Mac.

Has any progress or decisions been made on the above issues so far?

—
Reply to this email directly or view it on GitHub.

Volumes in boot2docker work but are fairly limited because b2d only mounts /Users into the VM.

So long as your volume lives beneath /Users, Kube on OS X w/ boot2docker works except for services.

cc @vmarmol