Skip to content

Instantly share code, notes, and snippets.

@nerdalert

nerdalert/macvlan_ipvlan_driver_notes.md

Last active May 9, 2022 02:21
Show Gist options
  • Save nerdalert/28168b016112b7c13040 to your computer and use it in GitHub Desktop.
Save nerdalert/28168b016112b7c13040 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. nerdalert revised this gist Mar 5, 2016. 1 changed file with 117 additions and 23 deletions.
    140 changes: 117 additions & 23 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -17,10 +17,12 @@

    - [IPv6 Example Usage](#ipv6)

    - [Advanced Networking Options](#advanced-networking-options)

    - [Manual 802.1q Link Creation Example](#manually-creating-8021q-links)

    **The PR can be found at**: [https://github.com/docker/libnetwork/pull/964](https://github.com/docker/libnetwork/pull/964). This branch contains both `macvlan` and `ipvlan` as the mechanics are structure are very similar. These two drivers do not use bridges, e.g. `docker0`. Nor is there any NAT/PAT. All addresses are placed directly onto the wire via the `--parent=` interface.

    **The PR can be found at: [https://github.com/docker/libnetwork/pull/964](https://github.com/docker/libnetwork/pull/964)**. This branch contains both `macvlan` and `ipvlan` as the mechanics are structure are very similar.

    **NOTE: when you start containers on both Macvlan and Ipvlan networks the container CAN NOT ping or communicate with any interface on the Docker host. This is how Ipvlan and Macvlan Linux interfaces are designed. It is to isolate the default namespace/underlying OS from the containers running in namespaces. You can communicate with other containers as described in each section or anything external to the Docker host, just not the Docker host itself.**

    @@ -70,7 +72,12 @@ Create the macvlan network and run a container attaching to it:

    ```
    # Macvlan (--macvlan_mode= Defaults to Bridge mode if not specified)
    docker network create -d macvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 -o host_iface=eth0 macnet100
    docker network create -d macvlan \
    --subnet=192.168.1.0/24 \
    --gateway=192.168.1.1 \
    -o host_iface=eth0 macnet100
    # Start a container on the new network
    docker run --net=macnet100 -it --rm busybox
    ```

    @@ -97,25 +104,29 @@ Create the ipvlan network and run a container attaching to it:

    ```
    # Ipvlan (--ipvlan_mode= Defaults to L2 mode if not specified)
    docker network create -d ipvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 -o host_iface=eth0 ipnet100
    docker network create -d ipvlan \
    --subnet=192.168.1.0/24 \
    --gateway=192.168.1.1 \
    -o host_iface=eth0 ipnet100
    # Start a container on the new network
    docker run --net=ipnet100 -it --rm busybox
    ```

    You can explicitly specify the `l2` mode option `-o ipvlan_mode=l2`. It is the default so will be in `l2` mode either way.




    ### Macvlan 802.1q Trunk Bridge Mode Example Usage ###


    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="center" width="350" ></a>


    Start two containers to test a ping. The default namespace is not reachable per macvlan design in order to isolate container namespaces from the underlying host.

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.
    The Linux sub-interface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the sub-interface. Parent interfaces such as `eth0` are not deleted, only sub-interface with a netlink parent index > 0.

    For the driver to add/delete the vlan subinterfaces the format needs to be `interface_name.vlan_tag`.
    For the driver to add/delete the vlan sub-interface the format needs to be `interface_name.vlan_tag`.

    For example: `eth0.50` denotes a parent interface of `eth0` with a slave of `eth0.50` tagged with vlan id `50`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.50 type vlan id 50`.

    @@ -127,7 +138,10 @@ In the first network tagged and isolated by the Docker host, `eth0.50` is the pa

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged
    docker network create -d macvlan --subnet=192.168.50.0/24 --gateway=192.168.50.1 -o host_iface=eth0.50 macvlan50
    docker network create -d macvlan \
    --subnet=192.168.50.0/24 \
    --gateway=192.168.50.1 \
    -o host_iface=eth0.50 macvlan50
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=macvlan50 -it --name macvlan_test5 --rm busybox
    @@ -152,25 +166,28 @@ docker run --net=macvlan60 -it --name macvlan_test8 --rm ubuntu

    Architecturally, Ipvlan L2 mode trunking is the same as Macvlan with regard to gateways and L2 path isolation. There are nuances that can be advantageous for CAM table exhaustion in ToR switches, one MAC per port and MAC exhaustion on a host's parent NIC to name a few.

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="center" width="350" ></a>
    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776sub-interfaces/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="center" width="350" ></a>

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.
    The Linux sub-interface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the sub-interface. Parent interfaces such as `eth0` are not deleted, only sub-interface with a netlink parent index > 0.

    For the driver to add/delete the vlan subinterfaces the format needs to be `interface_name.vlan_tag`. Other subinterface naming can be used as the spacified parent, but the link will not be deleted automatically when `docker network rm` is invoked.
    For the driver to add/delete the vlan sub-interface the format needs to be `interface_name.vlan_tag`. Other sub-interface naming can be used as the specified parent, but the link will not be deleted automatically when `docker network rm` is invoked.

    The option to use either existing parent vlan subinterfaces or let Docker manage them enables the user to either completely manage the Linux interfaces and networking or let Docker create and delete the Vlan parent subinterfaces (netlink `ip link`) with no effort from the user.
    The option to use either existing parent vlan sub-interface or let Docker manage them enables the user to either completely manage the Linux interfaces and networking or let Docker create and delete the Vlan parent sub-interface (netlink `ip link`) with no effort from the user.

    For example: `eth0.10` or `eth0:10` to denote a subinterface of `eth0` tagged with vlan id `10`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.10 type vlan id 10`.
    For example: `eth0.10` or `eth0:10` to denote a sub-interface of `eth0` tagged with vlan id `10`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.10 type vlan id 10`.

    The example creates the vlan tagged networks and then start two containers to test connectivity between containers. Different Vlans cannot ping one another without a router routing between the two networks. The default namespace is not reachable per ipvlan design in order to isolate container namespaces from the underlying host.
    The example creates the vlan tagged networks and then start two containers to test connectivity between containers. Different VLANs cannot ping one another without a router routing between the two networks. The default namespace is not reachable per ipvlan design in order to isolate container namespaces from the underlying host.

    **Vlan ID 20**

    In the first network tagged and isolated by the Docker host, `eth0.20` is the parent interface tagged with vlan id `20` specified with `-o host_iface=eth0.20`. Other naming formats can be used, but the links need to be added and deleted manually using `ip link` or Linux configuration files. As long as the `--host_iface` exists anything can be used if compliant with Linux netlink.

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged
    docker network create -d ipvlan --subnet=192.168.20.0/24 --gateway=192.168.20.1 -o host_iface=eth0.20 ipvlan20
    docker network create -d ipvlan \
    --subnet=192.168.20.0/24 \
    --gateway=192.168.20.1 \
    -o host_iface=eth0.20 ipvlan20
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=ipvlan20 -it --name ivlan_test1 --rm busybox
    @@ -183,7 +200,11 @@ In the second network, tagged and isolated by the Docker host, `eth0.30` is the

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged.
    docker network create -d ipvlan --subnet=192.168.30.0/24 --gateway=192.168.30.1 -o host_iface=eth0.30 --ipvlan_mode=l2 ipvlan30
    docker network create -d ipvlan \
    --subnet=192.168.30.0/24 \
    --gateway=192.168.30.1 \
    -o host_iface=eth0.30 \
    --ipvlan_mode=l2 ipvlan30
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=ipvlan30 -it --name ivlan_test3 --rm ubuntu
    @@ -198,8 +219,6 @@ default via 192.168.30.1 dev eth0
    192.168.30.0/24 dev eth0 src 192.168.30.2
    ```



    ### IPVlan L3 Mode Example Usage ###

    -Ipvlan L3 mode drops all broadcast and multicast traffic.
    @@ -220,7 +239,12 @@ The mode ` -o ipvlan_mode=l3` must be explicitly specified since the default ipv

    ```
    # Create the Ipvlan L3 network
    docker network create -d ipvlan --subnet=192.168.120.0/24 -o host_iface=eth0 -o ipvlan_mode=l3 ipvlan120
    docker network create -d ipvlan \
    --subnet=192.168.120.0/24 \
    -o host_iface=eth0 \
    -o ipvlan_mode=l3 ipvlan120
    # Start a couple of containers
    docker run --net=ipvlan120 -it --rm busybox
    # Or specify a specific IPv4 address
    @@ -269,10 +293,13 @@ Example: start two containers and ping one another:

    ```
    # Create a v6 network
    docker network create -d macvlan --subnet=fe90::/64 --gateway=fe90::22 -o host_iface=eth0 macnetv6
    docker network create -d macvlan \
    --subnet=fe90::/64 \
    --gateway=fe90::22 \
    -o host_iface=eth0 macnetv6
    # Start a container on the network
    docker run --net=macnetv6 -it --rm busybox
    ```

    View the container eth0 interface:
    @@ -309,16 +336,83 @@ PING fe90::2 (fe90::1): 56 data bytes
    64 bytes from fe90::1: seq=0 ttl=64 time=0.044 ms
    ```

    ### Dummy Interface Default and Internal Network


    One of the most important parameters a user passes for functionality is the `-o parent=` interface. In order to have functionality in all cases even if missing the parent interface is if the parent link is not passed, a dummy Linux interface will be setup. The dummy parent link is named with the first 12 numbers in the network ID.

    **Important Note:** This configuration can also be setup using the Libnetwork `--internal` flag. The **ONLY** things a container on a dummy or internal network can communicate with is other containers on the docker network (containers that share the same `--parent` link).

    **Example:** No parent interface specified - will default to creating a new Dummy interface for any container on that network. There is no way for those containers on that network to talk to anything other then eachother as they are completely isolated.

    ```
    # Create v4 dummy network
    docker network create -d macvlan --subnet=192.168.242.0/24 macnet242
    # Start two containers and verify they can ping one another
    docker run --net=macnet242 -itd alpine /bin/sh
    docker run --net=macnet242 -itd alpine /bin/sh
    ```

    Will acheive the same results as using the `--internal` flag:

    ```
    # Create v4 dummy network
    docker network create -d macvlan --subnet=192.168.246.0/24 macnet242
    # Start two containers and verify they can ping one another
    docker run --net=macnet246 -itd alpine /bin/sh
    docker run --net=macnet246 -itd alpine /bin/sh
    ```

    All of the same options can be used with no `--parent=eth0` specified ot `--internal`

    ```
    docker network create -d macvlan \
    --subnet=192.168.222.0/24 --subnet=192.168.224.0/24 \
    --subnet=fd18::/64 \
    --aux-address="exclude1=fd18::2" \
    --aux-address="exclude2=192.168.222.2" \
    --aux-address="exclude3=192.168.224.2" \
    --ip-range=192.168.222.0/25 \
    --ip-range=192.168.224.0/25 \
    -o macvlan_mode=bridge macnet222
    # Start containers on the network
    docker run --net=macnet222 -itd alpine /bin/sh
    docker run --net=macnet222 --ip=192.168.222.10 --ip6=fd18::10 -itd alpine /bin/sh
    docker run --net=macnet222 --ip6=fd18::11 -itd alpine /bin/sh
    ```

    ### Advanced Networking Options

    All of the applicable Docker networking options are available to the macvlan and ipvlan drivers. The ones that aren't are likely bridge related. Since we have cut out the need for a bridge with ipvlan/macvlan modes.

    **Example**: The following demonstrates excluding addresses along with dual stack. Let's say your eth0 in on the `192.168.131.0/24` and has a secondary address that is also routed by the network for `192.168.133.0/24`. Containers will be placed directly onto that network and any address between `192.168.131.2-192.168.131.254` and `192.168.133.2-192.168.133.254` are usable. In some cases the host may have an address on the Docker host `eth0` in this example. Since you don't want that address to be used, Libnetwork has the ability to blacklist that address from IPAM handing it out using `--aux-address="exclude2=192.168.131.254"`. The word `exclude2` in the `--aux-address` is completely arbitrary. The only requirement is it needs to be unique from the others in the network since it is a map whose keys will overwrite eachother in the case of collision.


    ```
    docker network create -d ipvlan \
    --subnet=192.168.131.0/24 --subnet=192.168.133.0/24 \
    --subnet=192.168.131.1 --subnet=192.168.133.1 \
    --subnet=fd14::/64 \
    --aux-address="exclude1=fd14::2" \
    --aux-address="exclude2=192.168.131.254" \
    --aux-address="exclude3=192.168.133.254" \
    --ip-range=192.168.131.0/25 \
    --ip-range=192.168.133.0/25 \
    -o host_iface=eth0 \
    -o ipvlan_mode=l3 ipnet131
    ```

    ### Manually Creating 802.1q Links

    **Vlan ID 40**

    If you do not want the driver to create the vlan subinterface it simply needs to exist prior to the `docker network create`.
    If you do not want the driver to create the vlan sub-interface it simply needs to exist prior to the `docker network create`.

    ```
    # create a new subinterface tied to dot1q vlan 40
    # create a new sub-interface tied to dot1q vlan 40
    ip link add link eth0 name eth0.40 type vlan id 40
    # enable the new sub-interface
  2. nerdalert revised this gist Feb 29, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -22,7 +22,7 @@

    **The PR can be found at: [https://github.com/docker/libnetwork/pull/964](https://github.com/docker/libnetwork/pull/964)**. This branch contains both `macvlan` and `ipvlan` as the mechanics are structure are very similar.

    **NOTE: when you start containers on both Macvlan and Ipvlan networks the container CAN NOT ping any interface on the Docker host. This is how Ipvlan and Macvlan Linux interfaces are designed. It is to isolate the default namespace/underlying OS from the containers running in namespaces.**
    **NOTE: when you start containers on both Macvlan and Ipvlan networks the container CAN NOT ping or communicate with any interface on the Docker host. This is how Ipvlan and Macvlan Linux interfaces are designed. It is to isolate the default namespace/underlying OS from the containers running in namespaces. You can communicate with other containers as described in each section or anything external to the Docker host, just not the Docker host itself.**

    - The driver caches `NetworkCreate` callbacks to the boltdb datastore along with populating `*networks`. In the case of a restart, the driver initializes the datastore with `Init()` and populates `*networks` since `NetworkCreate()` is only called once.
    - There can only be one (ipvlan or macvlan) driver type bound to a host interface with running containers at any given time. Currently the driver does not prevent ipvlan and macvlan networks to be created with the same `--host_interface` but will throw an error if you try to start an ipvlan container and a macvlan container at the same time on the same `--host_interface`. A mix of host interfaces and macvlan/ipvlan types can be used with running containers, each interface just needs to use the same type. Example: Macvlan Bridge or IPVlan L2. There is no mixing of running containers on the same host interface.
  3. nerdalert revised this gist Feb 29, 2016. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -20,8 +20,9 @@
    - [Manual 802.1q Link Creation Example](#manually-creating-8021q-links)


    Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan). This branch contains both macvlan and ipvlan as the mechanics are structure are very similar.
    **The PR can be found at: [https://github.com/docker/libnetwork/pull/964](https://github.com/docker/libnetwork/pull/964)**. This branch contains both `macvlan` and `ipvlan` as the mechanics are structure are very similar.

    **NOTE: when you start containers on both Macvlan and Ipvlan networks the container CAN NOT ping any interface on the Docker host. This is how Ipvlan and Macvlan Linux interfaces are designed. It is to isolate the default namespace/underlying OS from the containers running in namespaces.**

    - The driver caches `NetworkCreate` callbacks to the boltdb datastore along with populating `*networks`. In the case of a restart, the driver initializes the datastore with `Init()` and populates `*networks` since `NetworkCreate()` is only called once.
    - There can only be one (ipvlan or macvlan) driver type bound to a host interface with running containers at any given time. Currently the driver does not prevent ipvlan and macvlan networks to be created with the same `--host_interface` but will throw an error if you try to start an ipvlan container and a macvlan container at the same time on the same `--host_interface`. A mix of host interfaces and macvlan/ipvlan types can be used with running containers, each interface just needs to use the same type. Example: Macvlan Bridge or IPVlan L2. There is no mixing of running containers on the same host interface.
  4. nerdalert revised this gist Feb 29, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -4,7 +4,7 @@
    ### Contents

    - [Download the Binary](#download-the-binary)
    -

    - [MacVlan Bridge Mode Example Usage](#macvlan-bridge-mode-example-usage)

    - [Ipvlan L2 Mode Example Usage](#ipvlan-l2-mode-example-usage)
  5. nerdalert revised this gist Feb 29, 2016. 1 changed file with 6 additions and 3 deletions.
    9 changes: 6 additions & 3 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,6 +3,8 @@

    ### Contents

    - [Download the Binary](#download-the-binary)
    -
    - [MacVlan Bridge Mode Example Usage](#macvlan-bridge-mode-example-usage)

    - [Ipvlan L2 Mode Example Usage](#ipvlan-l2-mode-example-usage)
    @@ -29,12 +31,13 @@ Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https:/
    - **Note:** In both Macvlan and Ipvlan you are not able to ping or communicate with the default namespace IP address. For example, if you create a container and try to ping the Docker host's `eth0` it will **not** work. That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.

    - More information about Ipvlan & Macvlan can be found in the upstream [readme](https://github.com/torvalds/linux/blob/master/Documentation/networking/ipvlan.txt)


    - If you just want to test the Docker binary with the drivers compiled in download: [docker-ipvlan-macvlan-binary.tar](https://www.dropbox.com/s/tsz9ktobg4dkwuy/docker-1.11.0-dev.zip?dl=1)

    - For a list of manual tests you can paste in and give a whirl see: [macvlan_ipvlan_docker_driver_manual_tests.txt](https://gist.github.com/nerdalert/9dcb14265a3aea336f40)

    ### Download the Binary

    - If you don't want to build the binary yourself you can simple download this test binary: [docker-ipvlan-macvlan-binary.tar](https://www.dropbox.com/s/tsz9ktobg4dkwuy/docker-1.11.0-dev.zip?dl=1)

    - The Docker build is the following:

    ```
  6. nerdalert revised this gist Feb 29, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,5 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes
    # Macvlan, Ipvlan and 802.1q Trunk Native Docker Support Notes

    ### Contents

  7. nerdalert revised this gist Feb 22, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,7 +1,7 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    ### TOC
    ### Contents

    - [MacVlan Bridge Mode Example Usage](#macvlan-bridge-mode-example-usage)

  8. nerdalert revised this gist Feb 22, 2016. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,8 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    ### TOC

    - [MacVlan Bridge Mode Example Usage](#macvlan-bridge-mode-example-usage)

    - [Ipvlan L2 Mode Example Usage](#ipvlan-l2-mode-example-usage)
    @@ -256,10 +258,8 @@ L3 (layer 3) or ip routing has the ability to scale well beyond L2 (layer 2) net

    ### IPv6

    v6:
    - Default to dual stack? Currently does with generic v4 and gateway services
    - Default to dual stack? Currently does with generic v4 and gateway services. If ipv4 is not specified should we disable gateway services and provision v6 only etc.
    - Should `EnableIPv6` be used?
    - If ipv4 is not specified should we disable gateway services and provision v6 only?

    Example: start two containers and ping one another:

  9. nerdalert revised this gist Feb 22, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -257,7 +257,7 @@ L3 (layer 3) or ip routing has the ability to scale well beyond L2 (layer 2) net
    ### IPv6

    v6:
    - What are our dual stack plans?
    - Default to dual stack? Currently does with generic v4 and gateway services
    - Should `EnableIPv6` be used?
    - If ipv4 is not specified should we disable gateway services and provision v6 only?

  10. nerdalert revised this gist Feb 22, 2016. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -11,9 +11,9 @@

    - [IPVlan L3 Mode Example Usage](#ipvlan-l3-mode-example-usage)

    - [IPv6 Example](#ipv6)
    - [IPv6 Example Usage](#ipv6)

    - [Manual 802.1q Link Creation](#manually-creating-8021q-links)
    - [Manual 802.1q Link Creation Example](#manually-creating-8021q-links)


    Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan). This branch contains both macvlan and ipvlan as the mechanics are structure are very similar.
  11. nerdalert revised this gist Feb 22, 2016. 1 changed file with 7 additions and 7 deletions.
    14 changes: 7 additions & 7 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,19 +1,19 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    1. [MacVlan Bridge Mode Example Usage](#MacVlan-Bridge-Mode-Example-Usage)
    - [MacVlan Bridge Mode Example Usage](#macvlan-bridge-mode-example-usage)

    2. [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)
    - [Ipvlan L2 Mode Example Usage](#ipvlan-l2-mode-example-usage)

    3. [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)
    - [Macvlan 802.1q Trunk Bridge Mode Example Usage](#macvlan-8021q-trunk-bridge-mode-example-usage)

    4. [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)
    - [Ipvlan 802.1q Trunk L2 Mode Example Usage](#ipvlan-8021q-trunk-l2-mode-example-usage)

    5. [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    - [IPVlan L3 Mode Example Usage](#ipvlan-l3-mode-example-usage)

    6. [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    - [IPv6 Example](#ipv6)

    7. [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)
    - [Manual 802.1q Link Creation](#manually-creating-8021q-links)


    Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan). This branch contains both macvlan and ipvlan as the mechanics are structure are very similar.
  12. nerdalert revised this gist Feb 22, 2016. 1 changed file with 1 addition and 2 deletions.
    3 changes: 1 addition & 2 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,7 +1,7 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    1. [MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)
    1. [MacVlan Bridge Mode Example Usage](#MacVlan-Bridge-Mode-Example-Usage)

    2. [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)

    @@ -18,7 +18,6 @@

    Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan). This branch contains both macvlan and ipvlan as the mechanics are structure are very similar.

    [Macvlan/IPVlan Usage Diagrams](https://cloud.githubusercontent.com/assets/1711674/13032315/099ddbb2-d2bb-11e5-86c1-a526ade612cc.jpg)

    - The driver caches `NetworkCreate` callbacks to the boltdb datastore along with populating `*networks`. In the case of a restart, the driver initializes the datastore with `Init()` and populates `*networks` since `NetworkCreate()` is only called once.
    - There can only be one (ipvlan or macvlan) driver type bound to a host interface with running containers at any given time. Currently the driver does not prevent ipvlan and macvlan networks to be created with the same `--host_interface` but will throw an error if you try to start an ipvlan container and a macvlan container at the same time on the same `--host_interface`. A mix of host interfaces and macvlan/ipvlan types can be used with running containers, each interface just needs to use the same type. Example: Macvlan Bridge or IPVlan L2. There is no mixing of running containers on the same host interface.
  13. nerdalert revised this gist Feb 22, 2016. 1 changed file with 7 additions and 7 deletions.
    14 changes: 7 additions & 7 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,19 +1,19 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    1.[MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)
    1. [MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)

    2.[Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)
    2. [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)

    3.[Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)
    3. [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)

    4.[Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)
    4. [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)

    5.[IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    5. [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    6.[IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    6. [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    - [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)
    7. [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)


    Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan). This branch contains both macvlan and ipvlan as the mechanics are structure are very similar.
  14. nerdalert revised this gist Feb 22, 2016. 1 changed file with 5 additions and 5 deletions.
    10 changes: 5 additions & 5 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,15 +3,15 @@

    1.[MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)

    2. [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)
    2.[Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)

    3. [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)
    3.[Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)

    4. [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)
    4.[Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)

    5. [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    5.[IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    6. [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    6.[IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    - [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)

  15. nerdalert revised this gist Feb 22, 2016. 1 changed file with 5 additions and 0 deletions.
    5 changes: 5 additions & 0 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -2,10 +2,15 @@
    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    1.[MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)

    2. [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)

    3. [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)

    4. [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)

    5. [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    6. [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    - [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)
  16. nerdalert revised this gist Feb 22, 2016. 1 changed file with 6 additions and 11 deletions.
    17 changes: 6 additions & 11 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,17 +1,12 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    - [MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)

    - [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)

    - [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)

    - [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)

    - [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    - [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    1.[MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)
    2. [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)
    3. [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)
    4. [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)
    5. [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    6. [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    - [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)

  17. nerdalert revised this gist Feb 22, 2016. 1 changed file with 13 additions and 7 deletions.
    20 changes: 13 additions & 7 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,13 +1,19 @@

    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    [MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)
    [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)
    [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)
    [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)
    [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)
    - [MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)

    - [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)

    - [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)

    - [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)

    - [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    - [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)

    - [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)


    Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan). This branch contains both macvlan and ipvlan as the mechanics are structure are very similar.
  18. nerdalert revised this gist Feb 22, 2016. 1 changed file with 11 additions and 2 deletions.
    13 changes: 11 additions & 2 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,14 @@

    ### Macvlan, Ipvlan and 802.1q Trunk Driver Notes
    # Macvlan, Ipvlan and 802.1q Trunk Driver Notes

    [MacVlan Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-bridge-mode-example-usage)
    [Ipvlan L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l2-mode-example-usage)
    [Macvlan 802.1q Trunk Bridge Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#macvlan-8021q-trunk-bridge-mode-example-usage)
    [Ipvlan 802.1q Trunk L2 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-8021q-trunk-l2-mode-example-usage)
    [IPVlan L3 Mode Example Usage](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    [IPv6 Example](https://gist.github.com/nerdalert/28168b016112b7c13040#ipvlan-l3-mode-example-usage)
    [Manual 802.1q Link Creation](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links)


    Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan). This branch contains both macvlan and ipvlan as the mechanics are structure are very similar.

    @@ -103,7 +112,7 @@ Replace the `macvlan` with `ipvlan` in the `-d` driver argument to create macvla

    **Vlan ID 50**

    In the first network tagged and isolated by the Docker host, `eth0.50` is the parent interface tagged with vlan id `50` specified with `-o host_iface=eth0.50`. Other naming formats can be used, but the links need to be added and deleted manually using `ip link` or Linux configuration files. As long as the `--host_iface` exists anything can be used if compliant with Linux netlink.
    In the first network tagged and isolated by the Docker host, `eth0.50` is the parent interface tagged with vlan id `50` specified with `-o host_iface=eth0.50`. Other naming formats can be used, but the links need to be added and deleted [manually](https://gist.github.com/nerdalert/28168b016112b7c13040#manually-creating-8021q-links) using `ip link` or Linux configuration files. As long as the `--host_iface` exists anything can be used if compliant with Linux netlink.

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged
  19. nerdalert revised this gist Feb 22, 2016. 1 changed file with 5 additions and 3 deletions.
    8 changes: 5 additions & 3 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -95,9 +95,9 @@ Start two containers to test a ping. The default namespace is not reachable per

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.

    For the driver to add/delete the vlan subinterfaces the format needs to be either `interface_name.vlan_tag` or `interface_name:vlan_tag`.
    For the driver to add/delete the vlan subinterfaces the format needs to be `interface_name.vlan_tag`.

    For example: `eth0.50` or `eth0:50` to denote a subinterface of `eth0` tagged with vlan id `50`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.50 type vlan id 50`.
    For example: `eth0.50` denotes a parent interface of `eth0` with a slave of `eth0.50` tagged with vlan id `50`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.50 type vlan id 50`.

    Replace the `macvlan` with `ipvlan` in the `-d` driver argument to create macvlan 802.1q trunks.

    @@ -136,7 +136,9 @@ Architecturally, Ipvlan L2 mode trunking is the same as Macvlan with regard to g

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.

    For the driver to add/delete the vlan subinterfaces the format needs to be either `interface_name.vlan_tag` or `interface_name:vlan_tag`.
    For the driver to add/delete the vlan subinterfaces the format needs to be `interface_name.vlan_tag`. Other subinterface naming can be used as the spacified parent, but the link will not be deleted automatically when `docker network rm` is invoked.

    The option to use either existing parent vlan subinterfaces or let Docker manage them enables the user to either completely manage the Linux interfaces and networking or let Docker create and delete the Vlan parent subinterfaces (netlink `ip link`) with no effort from the user.

    For example: `eth0.10` or `eth0:10` to denote a subinterface of `eth0` tagged with vlan id `10`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.10 type vlan id 10`.

  20. nerdalert revised this gist Feb 22, 2016. 1 changed file with 4 additions and 4 deletions.
    8 changes: 4 additions & 4 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -30,7 +30,7 @@ Docker version 1.11.0-dev, build ac9d1b7-unsupported

    - In this example, `eth0` on the docker host has an IP on the `192.168.1.0/24` network and a default gateway of `192.168.1.1`. The gateway is an external router with an address of `192.168.1.1`. An IP address is not required on the Docker host interface `eth0` in `bridge` mode, it merely needs to be on the proper upstream network to get forwarded by a network switch or network router.

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232710/5c1c7d2a-d97e-11e5-99e9-f19d1aa0f87f.jpg" align="left" width="350" ></a>
    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232710/5c1c7d2a-d97e-11e5-99e9-f19d1aa0f87f.jpg" align="center" width="350" ></a>

    **Note** For Macvlan bridge mode and Ipvlan L2 mode the subnet values need to match the NIC's interface of the Docker host. For example, Use the same subnet and gateway of the Docker host ethernet interface that is specified by the `-o host_iface=` option.

    @@ -62,7 +62,7 @@ You can explicitly specify the `bridge` mode option `-o macvlan_mode=bridge`. It

    The ipvlan `L2` mode example is virtually identical to the macvlan `bridge` mode example. The driver is specified with `-d driver_name` option. In this case `-d ipvlan`

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232749/7999fdf0-d97e-11e5-9e62-8793ccdcb061.jpg" align="left" width="350" ></a>
    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232749/7999fdf0-d97e-11e5-9e62-8793ccdcb061.jpg" align="center" width="350" ></a>


    The parent interface `-o host_iface=eth0` is configured as followed:
    @@ -88,7 +88,7 @@ You can explicitly specify the `l2` mode option `-o ipvlan_mode=l2`. It is the d

    ### Macvlan 802.1q Trunk Bridge Mode Example Usage ###

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="left" width="350" ></a>
    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="center" width="350" ></a>


    Start two containers to test a ping. The default namespace is not reachable per macvlan design in order to isolate container namespaces from the underlying host.
    @@ -132,7 +132,7 @@ docker run --net=macvlan60 -it --name macvlan_test8 --rm ubuntu

    Architecturally, Ipvlan L2 mode trunking is the same as Macvlan with regard to gateways and L2 path isolation. There are nuances that can be advantageous for CAM table exhaustion in ToR switches, one MAC per port and MAC exhaustion on a host's parent NIC to name a few.

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="left" width="350" ></a>
    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="center" width="350" ></a>

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.

  21. nerdalert revised this gist Feb 22, 2016. 1 changed file with 11 additions and 1 deletion.
    12 changes: 11 additions & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -30,6 +30,8 @@ Docker version 1.11.0-dev, build ac9d1b7-unsupported

    - In this example, `eth0` on the docker host has an IP on the `192.168.1.0/24` network and a default gateway of `192.168.1.1`. The gateway is an external router with an address of `192.168.1.1`. An IP address is not required on the Docker host interface `eth0` in `bridge` mode, it merely needs to be on the proper upstream network to get forwarded by a network switch or network router.

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232710/5c1c7d2a-d97e-11e5-99e9-f19d1aa0f87f.jpg" align="left" width="350" ></a>

    **Note** For Macvlan bridge mode and Ipvlan L2 mode the subnet values need to match the NIC's interface of the Docker host. For example, Use the same subnet and gateway of the Docker host ethernet interface that is specified by the `-o host_iface=` option.

    - The parent interface used in this example is `eth0` and it is on the subnet `192.168.1.0/24`. The containers in the `docker network` will also need to be on this same subnet as the parent `--host_iface=`. The gateway is an external router on the network, not any ip masquerading or any other local proxy (see [diagrams for topologies](https://cloud.githubusercontent.com/assets/1711674/13032315/099ddbb2-d2bb-11e5-86c1-a526ade612cc.jpg)).
    @@ -60,6 +62,9 @@ You can explicitly specify the `bridge` mode option `-o macvlan_mode=bridge`. It

    The ipvlan `L2` mode example is virtually identical to the macvlan `bridge` mode example. The driver is specified with `-d driver_name` option. In this case `-d ipvlan`

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232749/7999fdf0-d97e-11e5-9e62-8793ccdcb061.jpg" align="left" width="350" ></a>


    The parent interface `-o host_iface=eth0` is configured as followed:

    ```
    @@ -83,6 +88,9 @@ You can explicitly specify the `l2` mode option `-o ipvlan_mode=l2`. It is the d

    ### Macvlan 802.1q Trunk Bridge Mode Example Usage ###

    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="left" width="350" ></a>


    Start two containers to test a ping. The default namespace is not reachable per macvlan design in order to isolate container namespaces from the underlying host.

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.
    @@ -120,9 +128,11 @@ docker run --net=macvlan60 -it --name macvlan_test8 --rm ubuntu
    ```


    ### Ipvlan 802.1q Trunk L2 Mode Example Usage ###

    Architecturally, Ipvlan L2 mode trunking is the same as Macvlan with regard to gateways and L2 path isolation. There are nuances that can be advantageous for CAM table exhaustion in ToR switches, one MAC per port and MAC exhaustion on a host's parent NIC to name a few.

    ### Ipvlan 802.1q Trunk L2 Mode Example Usage ###
    <a href="url"><img src="https://cloud.githubusercontent.com/assets/1711674/13232776/9e93b966-d97e-11e5-8803-e30e0d913e47.jpg" align="left" width="350" ></a>

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.

  22. nerdalert revised this gist Feb 22, 2016. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -12,6 +12,9 @@ Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https:/
    - Containers on separate networks cannot reach one another without an external process routing between the two networks/subnets.
    - **Note:** In both Macvlan and Ipvlan you are not able to ping or communicate with the default namespace IP address. For example, if you create a container and try to ping the Docker host's `eth0` it will **not** work. That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.

    - More information about Ipvlan & Macvlan can be found in the upstream [readme](https://github.com/torvalds/linux/blob/master/Documentation/networking/ipvlan.txt)


    - If you just want to test the Docker binary with the drivers compiled in download: [docker-ipvlan-macvlan-binary.tar](https://www.dropbox.com/s/tsz9ktobg4dkwuy/docker-1.11.0-dev.zip?dl=1)

    - For a list of manual tests you can paste in and give a whirl see: [macvlan_ipvlan_docker_driver_manual_tests.txt](https://gist.github.com/nerdalert/9dcb14265a3aea336f40)
  23. nerdalert revised this gist Feb 21, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -12,7 +12,7 @@ Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https:/
    - Containers on separate networks cannot reach one another without an external process routing between the two networks/subnets.
    - **Note:** In both Macvlan and Ipvlan you are not able to ping or communicate with the default namespace IP address. For example, if you create a container and try to ping the Docker host's `eth0` it will **not** work. That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.

    - Download this Docker binary with IPVlan/MacVlan native drivers to try out the following: [docker-ipvlan-macvlan-binary.tar](https://www.dropbox.com/s/tsz9ktobg4dkwuy/docker-1.11.0-dev.zip?dl=1)
    - If you just want to test the Docker binary with the drivers compiled in download: [docker-ipvlan-macvlan-binary.tar](https://www.dropbox.com/s/tsz9ktobg4dkwuy/docker-1.11.0-dev.zip?dl=1)

    - For a list of manual tests you can paste in and give a whirl see: [macvlan_ipvlan_docker_driver_manual_tests.txt](https://gist.github.com/nerdalert/9dcb14265a3aea336f40)

  24. nerdalert revised this gist Feb 21, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -15,7 +15,7 @@ Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https:/
    - Download this Docker binary with IPVlan/MacVlan native drivers to try out the following: [docker-ipvlan-macvlan-binary.tar](https://www.dropbox.com/s/tsz9ktobg4dkwuy/docker-1.11.0-dev.zip?dl=1)

    - For a list of manual tests you can paste in and give a whirl see: [macvlan_ipvlan_docker_driver_manual_tests.txt](https://gist.github.com/nerdalert/9dcb14265a3aea336f40)
    -

    - The Docker build is the following:

    ```
  25. nerdalert revised this gist Feb 21, 2016. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -14,6 +14,8 @@ Branch at: [https://github.com/nerdalert/libnetwork/tree/ipvlan_macvlan](https:/

    - Download this Docker binary with IPVlan/MacVlan native drivers to try out the following: [docker-ipvlan-macvlan-binary.tar](https://www.dropbox.com/s/tsz9ktobg4dkwuy/docker-1.11.0-dev.zip?dl=1)

    - For a list of manual tests you can paste in and give a whirl see: [macvlan_ipvlan_docker_driver_manual_tests.txt](https://gist.github.com/nerdalert/9dcb14265a3aea336f40)
    -
    - The Docker build is the following:

    ```
  26. nerdalert revised this gist Feb 21, 2016. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -107,7 +107,7 @@ In the second network, tagged and isolated by the Docker host, `eth0.60` is the

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged.
    docker network create -d macvlan --subnet=192.168.60.0/24 --gateway=192.168.60.1 -o host_iface=eth0.60 --macvlan_mode=bridge macvlan60
    docker network create -d macvlan --subnet=192.168.60.0/24 --gateway=192.168.60.1 -o host_iface=eth0.60 -o --macvlan_mode=bridge macvlan60
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=macvlan60 -it --name macvlan_test7 --rm ubuntu
  27. nerdalert revised this gist Feb 21, 2016. 1 changed file with 29 additions and 0 deletions.
    29 changes: 29 additions & 0 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -230,13 +230,22 @@ v6:
    - Should `EnableIPv6` be used?
    - If ipv4 is not specified should we disable gateway services and provision v6 only?

    Example: start two containers and ping one another:

    ```
    # Create a v6 network
    docker network create -d macvlan --subnet=fe90::/64 --gateway=fe90::22 -o host_iface=eth0 macnetv6
    # Start a container on the network
    docker run --net=macnetv6 -it --rm busybox
    ```

    View the container eth0 interface:

    ```
    # IP config of the container once it starts:
    $ ip addr show eth0
    eth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
    link/ether ce:26:6b:b8:27:11 brd ff:ff:ff:ff:ff:ff
    inet 172.19.0.2/16 scope global eth0
    @@ -247,6 +256,26 @@ eth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
    valid_lft forever preferred_lft forever
    ```

    Start a second container and ping the first containers v6 address at `fe90::1` from it's address of `fe90::2`.

    ```
    $ ip a show eth0
    48: eth0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
    link/ether 06:09:43:a2:91:c3 brd ff:ff:ff:ff:ff:ff
    inet 172.19.0.3/16 scope global eth0
    valid_lft forever preferred_lft forever
    inet6 fe80::409:43ff:fea2:91c3/64 scope link
    valid_lft forever preferred_lft forever
    inet6 fe90::2/64 scope link flags 02
    valid_lft forever preferred_lft forever
    $ ping -6 fe90::1
    PING fe90::2 (fe90::1): 56 data bytes
    64 bytes from fe90::1: seq=0 ttl=64 time=0.044 ms
    ```



    ### Manually Creating 802.1q Links

    **Vlan ID 40**
  28. nerdalert revised this gist Feb 21, 2016. 1 changed file with 18 additions and 4 deletions.
    22 changes: 18 additions & 4 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -177,7 +177,7 @@ ip a show eth0
    inet 192.168.1.251/24 brd 192.168.1.255 scope global eth0
    ```

    -A traditional gateway doesn't mean much to an L3 mode Ipvlan interface since there is no broadcast traffic allowed. Because of this we simply point the the containers `eth0` device as the default gateway. See below for CLI output of `ip route` from inside an L3 container.
    -A traditional gateway doesn't mean much to an L3 mode Ipvlan interface since there is no broadcast traffic allowed. Because of this we simply point the the containers `eth0` device as the default gateway. See below for CLI output of `ip route` from inside an L3 container.

    The mode ` -o ipvlan_mode=l3` must be explicitly specified since the default ipvlan mode is `l2`.

    @@ -190,7 +190,7 @@ docker run --net=ipvlan120 -it --rm busybox
    docker run --net=ipvlan120 --ip=192.168.120.10 -it --rm busybox
    ```

    Take a look at the container routing table from inside of the container:
    You will notice there is no `--gateway=` option in the network create. The field is ignored if one is specified since it doesn't have a use in `l3` mode. Take a look at the container routing table from inside of the container:

    ```
    # Inside the container
    @@ -199,14 +199,28 @@ default dev eth0
    192.168.120.0/24 dev eth0 src 192.168.120.2
    ```

    L3 mode requires a route to be injected both on the local host pointing the container network to the master interface. The IP stack is then processed inside of the namespace rather then in the default namespace.

    -Example: This route is added to the Docker host not the Docker container. `eth0` in the route has to be the same interface as specified in the `-o host_iface=eth0` network create.

    In order to ping the container from a remote host or the container be able to ping a remote host, the remote host needs to have a route pointing to the host IP address of the container's Docker host eth interface.

    -If you have two hosts and one of them has a ipvlan l3 mode container running, the second host needs to have a route added to ping the container. The following are the addresses of the example of two hosts, host1 running docker with a container in ipvlan l3 mode and *host2* a regular host we will be pinging from the container and vice versa:

    -*Host1:* Docker Host eth0: 192.168.1.249/24
    -*Host1:* Docker ipvlan l3 mode container network: 192.168.130.0/24
    -*Host1:* Docker ipvlan l3 mode container IP: 192.168.130.2
    -*Host2:* Remote Host eth0: 192.168.1.251/24

    First, add a route to the remote host that tells the host, in order to reach the container network `192.168.130.0/24` on *Host1* use the next hop *Host1* eth0 `192.168.1.249`. By adding the following route to

    ```
    $ ip route add 10.1.48.0/24 dev eth0
    # On Host2 add a route for the L3 container network to be reached via the Docker host eth0 address (192.168.1.249).
    ip route add 192.168.130.0/24 via 192.168.1.249
    ```

    After adding the above route to *Host1* it should be able to ping the L3 mode container at `192.168.130.2` from its eth0 interface with the IP `192.168.1.251` as should the container be able to ping `192.168.1.249` from its address at `192.168.130.2`.

    L3 (layer 3) or ip routing has the ability to scale well beyond L2 (layer 2) networks. The Internet is made up of a collection of interconnected L3 networks. This is attractive when coupled with the density presented by migrations to Docker containers and worth spending some time to understand if new to scaling networks.


    ### IPv6
  29. nerdalert revised this gist Feb 21, 2016. 1 changed file with 36 additions and 35 deletions.
    71 changes: 36 additions & 35 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -76,88 +76,89 @@ You can explicitly specify the `l2` mode option `-o ipvlan_mode=l2`. It is the d



    ### Ipvlan 802.1q Trunk L2 Mode Example Usage ###
    ### Macvlan 802.1q Trunk Bridge Mode Example Usage ###

    Start two containers to test a ping. The default namespace is not reachable per macvlan design in order to isolate container namespaces from the underlying host.

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.

    For the driver to add/delete the vlan subinterfaces the format needs to be either `interface_name.vlan_tag` or `interface_name:vlan_tag`.

    For example: `eth0.10` or `eth0:10` to denote a subinterface of `eth0` tagged with vlan id `10`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.10 type vlan id 10`.
    For example: `eth0.50` or `eth0:50` to denote a subinterface of `eth0` tagged with vlan id `50`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.50 type vlan id 50`.

    The example creates the vlan tagged networks and then start two containers to test connectivity between containers. Different Vlans cannot ping one another without a router routing between the two networks. The default namespace is not reachable per ipvlan design in order to isolate container namespaces from the underlying host.
    Replace the `macvlan` with `ipvlan` in the `-d` driver argument to create macvlan 802.1q trunks.

    **Vlan ID 20**
    **Vlan ID 50**

    In the first network tagged and isolated by the Docker host, `eth0.20` is the parent interface tagged with vlan id `20` specified with `-o host_iface=eth0.20`. Other naming formats can be used, but the links need to be added and deleted manually using `ip link` or Linux configuration files. As long as the `--host_iface` exists anything can be used if compliant with Linux netlink.
    In the first network tagged and isolated by the Docker host, `eth0.50` is the parent interface tagged with vlan id `50` specified with `-o host_iface=eth0.50`. Other naming formats can be used, but the links need to be added and deleted manually using `ip link` or Linux configuration files. As long as the `--host_iface` exists anything can be used if compliant with Linux netlink.

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged
    docker network create -d ipvlan --subnet=192.168.20.0/24 --gateway=192.168.20.1 -o host_iface=eth0.20 ipvlan20
    docker network create -d macvlan --subnet=192.168.50.0/24 --gateway=192.168.50.1 -o host_iface=eth0.50 macvlan50
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=ipvlan20 -it --name ivlan_test1 --rm busybox
    docker run --net=ipvlan20 -it --name ivlan_test2 --rm busybox
    docker run --net=macvlan50 -it --name macvlan_test5 --rm busybox
    docker run --net=macvlan50 -it --name macvlan_test6 --rm busybox
    ```

    **Vlan ID 30**
    **Vlan ID 60**

    In the second network, tagged and isolated by the Docker host, `eth0.30` is the parent interface tagged with vlan id `30` specified with `-o host_iface=eth0.30`. The `ipvlan_mode=` defaults to l2 mode `ipvlan_mode=l2`. It can also be explicitly set with the same result as shown in the next example.
    In the second network, tagged and isolated by the Docker host, `eth0.60` is the parent interface tagged with vlan id `60` specified with `-o host_iface=eth0.60`. The `macvlan_mode=` defaults to `macvlan_mode=bridge`. It can also be explicitly set with the same result as shown in the next example.

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged.
    docker network create -d ipvlan --subnet=192.168.30.0/24 --gateway=192.168.30.1 -o host_iface=eth0.30 --ipvlan_mode=l2 ipvlan30
    docker network create -d macvlan --subnet=192.168.60.0/24 --gateway=192.168.60.1 -o host_iface=eth0.60 --macvlan_mode=bridge macvlan60
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=ipvlan30 -it --name ivlan_test3 --rm ubuntu
    docker run --net=ipvlan30 -it --name ivlan_test4 --rm ubuntu
    docker run --net=macvlan60 -it --name macvlan_test7 --rm ubuntu
    docker run --net=macvlan60 -it --name macvlan_test8 --rm ubuntu
    ```

    The gateway is set inside of the container as the default gateway. That gateway would typically be an external router on the network.

    ```
    $ ip route
    default via 192.168.30.1 dev eth0
    192.168.30.0/24 dev eth0 src 192.168.30.2
    ```



    ### Macvlan 802.1q Trunk Bridge Mode Example Usage ###

    Start two containers to test a ping. The default namespace is not reachable per macvlan design in order to isolate container namespaces from the underlying host.
    ### Ipvlan 802.1q Trunk L2 Mode Example Usage ###

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.

    For the driver to add/delete the vlan subinterfaces the format needs to be either `interface_name.vlan_tag` or `interface_name:vlan_tag`.

    For example: `eth0.50` or `eth0:50` to denote a subinterface of `eth0` tagged with vlan id `50`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.50 type vlan id 50`.
    For example: `eth0.10` or `eth0:10` to denote a subinterface of `eth0` tagged with vlan id `10`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.10 type vlan id 10`.

    Replace the `macvlan` with `ipvlan` in the `-d` driver argument to create macvlan 802.1q trunks.
    The example creates the vlan tagged networks and then start two containers to test connectivity between containers. Different Vlans cannot ping one another without a router routing between the two networks. The default namespace is not reachable per ipvlan design in order to isolate container namespaces from the underlying host.

    **Vlan ID 50**
    **Vlan ID 20**

    In the first network tagged and isolated by the Docker host, `eth0.50` is the parent interface tagged with vlan id `50` specified with `-o host_iface=eth0.50`. Other naming formats can be used, but the links need to be added and deleted manually using `ip link` or Linux configuration files. As long as the `--host_iface` exists anything can be used if compliant with Linux netlink.
    In the first network tagged and isolated by the Docker host, `eth0.20` is the parent interface tagged with vlan id `20` specified with `-o host_iface=eth0.20`. Other naming formats can be used, but the links need to be added and deleted manually using `ip link` or Linux configuration files. As long as the `--host_iface` exists anything can be used if compliant with Linux netlink.

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged
    docker network create -d macvlan --subnet=192.168.50.0/24 --gateway=192.168.50.1 -o host_iface=eth0.50 macvlan50
    docker network create -d ipvlan --subnet=192.168.20.0/24 --gateway=192.168.20.1 -o host_iface=eth0.20 ipvlan20
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=macvlan50 -it --name macvlan_test5 --rm busybox
    docker run --net=macvlan50 -it --name macvlan_test6 --rm busybox
    docker run --net=ipvlan20 -it --name ivlan_test1 --rm busybox
    docker run --net=ipvlan20 -it --name ivlan_test2 --rm busybox
    ```

    **Vlan ID 60**
    **Vlan ID 30**

    In the second network, tagged and isolated by the Docker host, `eth0.60` is the parent interface tagged with vlan id `60` specified with `-o host_iface=eth0.60`. The `macvlan_mode=` defaults to `macvlan_mode=bridge`. It can also be explicitly set with the same result as shown in the next example.
    In the second network, tagged and isolated by the Docker host, `eth0.30` is the parent interface tagged with vlan id `30` specified with `-o host_iface=eth0.30`. The `ipvlan_mode=` defaults to l2 mode `ipvlan_mode=l2`. It can also be explicitly set with the same result as shown in the next example.

    ```
    # now add networks and hosts as you would normally by attaching to the master (sub)interface that is tagged.
    docker network create -d macvlan --subnet=192.168.60.0/24 --gateway=192.168.60.1 -o host_iface=eth0.60 --macvlan_mode=bridge macvlan60
    docker network create -d ipvlan --subnet=192.168.30.0/24 --gateway=192.168.30.1 -o host_iface=eth0.30 --ipvlan_mode=l2 ipvlan30
    # in two separate terminals, start a Docker container and the containers can now ping one another.
    docker run --net=macvlan60 -it --name macvlan_test7 --rm ubuntu
    docker run --net=macvlan60 -it --name macvlan_test8 --rm ubuntu
    docker run --net=ipvlan30 -it --name ivlan_test3 --rm ubuntu
    docker run --net=ipvlan30 -it --name ivlan_test4 --rm ubuntu
    ```

    The gateway is set inside of the container as the default gateway. That gateway would typically be an external router on the network.

    ```
    $ ip route
    default via 192.168.30.1 dev eth0
    192.168.30.0/24 dev eth0 src 192.168.30.2
    ```


  30. nerdalert revised this gist Feb 21, 2016. 1 changed file with 36 additions and 7 deletions.
    43 changes: 36 additions & 7 deletions macvlan_ipvlan_driver_notes.md
    View file
    Original file line number Diff line number Diff line change
    @@ -23,40 +23,68 @@ Docker version 1.11.0-dev, build ac9d1b7-unsupported

    ### MacVlan Bridge Mode Example Usage ###

    - In this example, `eth0` on the docker host has an IP on the `192.168.1.0/24` network and a default gateway of `192.168.1.1`. An IP address is not required on the Docker host interface, it merely needs to be on the proper upstream network to get forwarded by a network switch or network router.
    - In this example, `eth0` on the docker host has an IP on the `192.168.1.0/24` network and a default gateway of `192.168.1.1`. The gateway is an external router with an address of `192.168.1.1`. An IP address is not required on the Docker host interface `eth0` in `bridge` mode, it merely needs to be on the proper upstream network to get forwarded by a network switch or network router.

    **Note** For Macvlan bridge mode and Ipvlan L2 mode the subnet values need to match the NIC's interface of the Docker host. For example, Use the same subnet and gateway of the Docker host ethernet interface that is specified by the `-o host_iface=` option.

    - The parent interface used in this example is `eth0` and it is on the subnet `192.168.1.0/24`. The containers in the `docker network` will also need to be on this same subnet as the parent `--host_iface=`. The gateway is an external router on the network, not any ip masquerading or any other local proxy (see [diagrams for topologies](https://cloud.githubusercontent.com/assets/1711674/13032315/099ddbb2-d2bb-11e5-86c1-a526ade612cc.jpg)).

    -The parent interface `-o host_iface=eth0` is configured as followed:

    - The driver is specified with `-d driver_name` option. In this case `-d macvlan`

    ```
    ip a show eth0
    ip addr show eth0
    3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:50:56:39:45:2e brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.251/24 brd 192.168.1.255 scope global eth0
    ```

    Create the macvlan network and run a container attaching to it:

    ```
    # Macvlan opts tests w/o mode (Defaults to Bridge)
    # Macvlan (--macvlan_mode= Defaults to Bridge mode if not specified)
    docker network create -d macvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 -o host_iface=eth0 macnet100
    docker run --net=macnet100 -it --rm busybox
    ```

    You can explicitly specify the `bridge` mode option `-o macvlan_mode=bridge`. It is the default so will be in `bridge` mode either way.

    ### Ipvlan 802.1q Trunk L2 Mode Example Usage ###

    Start two containers to test a ping. The default namespace is not reachable per ipvlan design in order to isolate container namespaces from the underlying host.

    ### Ipvlan L2 Mode Example Usage ###

    The ipvlan `L2` mode example is virtually identical to the macvlan `bridge` mode example. The driver is specified with `-d driver_name` option. In this case `-d ipvlan`

    The parent interface `-o host_iface=eth0` is configured as followed:

    ```
    ip addr show eth0
    3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 192.168.1.251/24 brd 192.168.1.255 scope global eth0
    ```

    Create the ipvlan network and run a container attaching to it:

    ```
    # Ipvlan (--ipvlan_mode= Defaults to L2 mode if not specified)
    docker network create -d ipvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 -o host_iface=eth0 ipnet100
    docker run --net=ipnet100 -it --rm busybox
    ```

    You can explicitly specify the `l2` mode option `-o ipvlan_mode=l2`. It is the default so will be in `l2` mode either way.




    ### Ipvlan 802.1q Trunk L2 Mode Example Usage ###

    The Linux subinterface tagged with a vlan can either already exist or will be created when you call a `docker network create`. `docker network rm` will delete the subinterface. Parent interfaces such as `eth0` are not deleted, only subinterfaces with a netlink parent index > 0.

    For the driver to add/delete the vlan subinterfaces the format needs to be either `interface_name.vlan_tag` or `interface_name:vlan_tag`.

    For example: `eth0.10` or `eth0:10` to denote a subinterface of `eth0` tagged with vlan id `10`. The equivalent `ip link` command would be `ip link add link eth0 name eth0.10 type vlan id 10`.

    Replace the `ipvlan` with `macvlan` in the `-d` driver argument to create macvlan 802.1q trunks.
    The example creates the vlan tagged networks and then start two containers to test connectivity between containers. Different Vlans cannot ping one another without a router routing between the two networks. The default namespace is not reachable per ipvlan design in order to isolate container namespaces from the underlying host.

    **Vlan ID 20**

    @@ -139,6 +167,7 @@ docker run --net=macvlan60 -it --name macvlan_test8 --rm ubuntu
    -Ipvlan L3 mode drops all broadcast and multicast traffic.
    -L3 mode needs to be on a separate subnet as the default namespace since it requires a netlink route in the default namespace pointing to the Ipvlan parent interface.
    - The parent interface used in this example is `eth0` and it is on the subnet `192.168.1.0/24`. Notice the `docker network` is **not** on the same subnet as `eth0`.
    - Unlike macvlan bridge mode and ipvlan l2 modes, different subnets/networks can ping one another as long as they share the same parent interface `--host_interface=`.

    ```
    ip a show eth0