The corresponding engine-api PR: docker/engine-api#375

I agree with @cpuguy83. I'm not sure why you would not set it in case you needed it later. Is there a reason?

FWIW I also tested this across a multinode cluster and it works (container attaches to net and can ping, etc).

I agree with @cpuguy83. I'm not sure why you would not set it in case you needed it later. Is there a reason?

If you need it later you would set it, but in a typical deployment the cluster admin can decide and plan ahead of time which networks they want to allow container attaches and which containers are solely for services.

Is a boolean enough here?
Since it is security-related would it make more since to have a full policy per network?

I'm bad at naming things but something like:

type NonSwarmAttachmentPolicy struct {
    MaxAttachments int // max number of containers that can be attached
    IPRange  cidr // sub-set of the full network range
    AttachFromWorker bool // allow attaching from worker node
}

Also thinking it should just allowed on manager nodes (maybe the example struct needs some tweaking there) by default. Otherwise we'll just get blowback from users about weird behavior. This way we can at least say "we don't allow it on worker nodes by default for security reasons, but you can do it from manager nodes".

Also thinking it should just allowed on manager nodes (maybe the example struct needs some tweaking there) by default.

I think it makes simpler for users to reason about a flag without any additional context. The behavior of a command differing based on the role of the node just make things a bit more complicated IMO. Once we properly document the purpose of the flag, the users will always be able to reason out quickly that in swarm mode they need to add this flag if they have to attach a regular container to that network.

@mrjana We already have behavior differences on non-manager nodes.

@cpuguy83 @mrjana The reason is operations such as creating a service are part of swarm management by their nature, therefore it made sense to have them restricted to managers. Creating a container and attaching it to a network falls into the "payload" category - if a user wants to manually run a mysql (non-managed container), it would be very restricting to force to run it on manager nodes. Service creation has no impact on the underlying node while docker run does.

@ehazlett @cpuguy83 @mrjana Regarding that flag - I agree that it might be useful by default, however we decided not to do so for the time being (but this is still debatable). How would you feel about:

• It's disabled by default
• We allow users to change it using docker network update foo --manually-attachable=true
• We may consider to enable it by default in the future depending on user reception of 1.13.

/cc @diogomonica @icecrime

We allow users to change it using docker network update foo --manually-attachable=true

We did discuss this option and this is definitely something that we want to in the future.

We allow users to change it using docker network update foo --manually-attachable=true

+1

Related: moby/libnetwork#1414

We allow users to change it using docker network update foo --manually-attachable=true

We can certainly discuss about mutability of the network object and reusing docker update. It is not just for this flag. There are other configurations that can be updated as well. But I dont think this PR should be the place for making network as a mutable object.

@aluzzardi I strongly recommend against enabling this by default. That will be essentially disabling the whole security model, because no one will ever enable it, and we'll be in a situation where workers can join whatever networks they want (which is exactly what we designed our model against).

@ehazlett the reason is: you should always use least-privilege. If you don't need random nodes attaching to random networks right now, don't enable it. If you ever need it, enable it temporarily. What I don't want is people to always enable it, since they will have a less secure cluster because of it.

And we should make it explicit that: if you should avoid using this feature. This makes this network vulnerable to the compromise of any worker node in the cluster, since any worker node can attach to it.

@diogomonica cool thx. i didn't know what the attaching affected -- thx for clarifying.

Indeed, thanks for clarifying @diogomonica. I've heard lots of feedback from people trying to use docker that way (docker run in swarm mode overlay) though.

OK, I think I understand the intent here...
What about --scope=global and --scope=swarm?

btw, I choose these values b/c we're already using it for networks and it pretty much matches what we're doing here.

@cpuguy83

What about --scope=global and --scope=swarm?

Though not directly related to --manually-attachable idea.... I like the suggestion, especially because it captures the intent and is inline with the definition of scope in swarm and non-swarm-mode. In non-swarm-mode, overlay networks are automatically placed in global scope and in swarm-mode it is placed in swarm scope. But letting the user specify that it also makes it possible (in the future) for the driver to accept or reject such a configuration.

1 drawback with this suggestion though is that we are overloading the concept of scope with this new requirement. And also scope is a concept managed and handled by drivers/plugins. Whereas this requirement has no involvement from the drivers.

This is going to be on 1.12-rc2, right?

@rogaha no. the milestone is for 1.13.0.

ok, sounds good. thanks for the heads up @mavenugo.

Did it make it in 1.13rc1? Because I still have the exact same problem using that version.
@mrjana

@gprivitera can you open an issue, and steps to reproduce? That makes it easier to track and prioritize if needed

#23901 It already exist, do you want me to create one for 1.13rc1?
edit: done.