Adding tunneling support + PTF tests (#13)

Adding tunneling support + PTF tests
p4lang · mihaibudiu · Mar 4, 2020 · Apr 8, 2019 · Apr 8, 2019 · Apr 13, 2019
commit 5cc9931b5f124fa4108f65fdad2406825841de03
diff --git a/backends/ubpf/CMakeLists.txt b/backends/ubpf/CMakeLists.txt
@@ -22,6 +22,7 @@ set(P4C_UBPF_SOURCES
         ubpfBackend.cpp
         ubpfProgram.cpp
         ubpfParser.cpp
+        ubpfDeparser.cpp
         ubpfControl.cpp
         ubpfType.cpp
         ubpfTable.cpp
@@ -40,9 +41,11 @@ set(P4C_UBPF_SOURCES
         ../../backends/ebpf/lower.cpp)
 
 set(P4C_UBPF_HEADERS
+        codeGen.h
         ubpfProgram.h
         ubpfType.h
         ubpfParser.h
+        ubpfDeparser.h
         ubpfControl.h
         ubpfRegister.h
         ubpfModel.h

diff --git a/backends/ubpf/README.md b/backends/ubpf/README.md
@@ -1,8 +1,10 @@
 # Introduction to uBPF Backend
 
-This is a back-end, which generates the code to be consumed by the userspace BPF VM - https://github.com/iovisor/ubpf .
+This repository implements the uBPF (Userspace BPF) backend for the P4 compiler (https://github.com/p4lang/p4c).
 
-The P4-to-uBPF compiler accepts only the P4_16 programs written for the `ubpf_filter_model.p4` architecture model.
+The **p4c-ubpf** compiler allows to translate P4 programs into the uBPF programs. We use the uBPF implementation provided by [the ubpf project](https://github.com/iovisor/ubpf). 
+
+The P4-to-uBPF compiler accepts only the P4_16 programs written for the `ubpf_model.p4` architecture model.
 
 The backend for uBPF is mostly based on [P4-to-eBPF compiler](../ebpf/README.md). In fact, it implements the same concepts, but
 generates the C code, which is compatible with the user-space BPF implementation. 
@@ -68,8 +70,8 @@ The output file (`out.o`) can be injected to the uBPF VM.
 
 ### Known limitations
 
-* The modification of wide packet's fields has not been tested extensively. Hence, it can not work properly.
-
+* No support for some P4 constructs (meters, counters, etc.)
+* No support for checksum computation.
 
 ### Contact
 

diff --git a/backends/ubpf/codeGen.h b/backends/ubpf/codeGen.h
@@ -0,0 +1,20 @@
+#ifndef P4C_CODEGEN_H
+#define P4C_CODEGEN_H
+
+#include "lib/sourceCodeBuilder.h"
+#include "ebpf/codeGen.h"
+#include "target.h"
+
+namespace UBPF {
+    class UbpfCodeBuilder : public EBPF::CodeBuilder {
+    public:
+        const UbpfTarget *target;
+
+        explicit UbpfCodeBuilder(const UbpfTarget *target) : EBPF::CodeBuilder(target), target(target) {
+
+        }
+    };
+}
+
+
+#endif //P4C_CODEGEN_H
diff --git a/backends/ubpf/docs/EXAMPLES.md b/backends/ubpf/docs/EXAMPLES.md
@@ -1,7 +1,7 @@
 # Introduction
 
 This file contains description of the basic P4 programs, which were used to test the functionality of the P4-to-uBPF compiler.
-All tests have been run on the [Oko](https://github.com/Orange-OpenSource/oko) switch using the [Vagrant prepared for Oko](https://github.com/P4-Research/vagrant-oko).
+All tests have been run on the [Oko](https://github.com/Orange-OpenSource/oko) switch using [Vagrant prepared for Oko](https://github.com/P4-Research/vagrant-oko).
 
 Before any experiment the following commands need to be invoked:
 
@@ -10,14 +10,19 @@ Before any experiment the following commands need to be invoked:
 $ p4c-ubpf -o test.c PROGRAM.p4 
 $ sudo ovs-ofctl del-flows br0
 # compile test.c to BPF machine code
-$ clang-3.8 -O2 -I .. -target bpf -v  -c test.c -o /tmp/test.o
-# Load filter BPF program and setup rules to forward traffic
-$ sudo ovs-ofctl load-filter-prog br0 1 /tmp/test.o
-$ sudo ovs-ofctl add-flow br0 table=1,ip,nw_dst=172.16.0.12,actions=output:1
-$ sudo ovs-ofctl add-flow br0 priority=1,in_port=2,filter_prog=1,actions=goto_table:1
-$ sudo ovs-ofctl add-flow br0 in_port=1,filter_prog=1,actions=output:2
+$ clang-6.0 -O2 -I .. -target bpf -c test.c -o /tmp/test.o
+# Load filter BPF program
+$ sudo ovs-ofctl load-bpf-prog br0 1 /tmp/test.o
+# Setup rules to forward traffic (bidirectional)
+$ sudo ovs-ofctl add-flow br0 in_port=2,actions=prog:1,output:1
+$ sudo ovs-ofctl add-flow br0 in_port=1,actions=prog:1,output:2
 ```
 
+**Note!** The P4-uBPF compiler and Oko work properly with `clang-6.0`. We noticed some problems when using older versions of clang (e.g. 3.9).
+
+# Examples
+
+This section presents how to run and test the P4-uBPF compiler. 
 
 ## Packet modification
 
@@ -43,15 +48,15 @@ This section presents P4 program, which modifies the packet's fields.
 Sample usage:
 
 ```bash
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 0 0 0 0 0 0 0 0 0 0 0 0 # decrements MPLS TTL
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 1 0 0 0 24 0 0 0 0 0 0 0 # sets MPLS label to 24
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 2 0 0 0 24 0 0 0 0 0 0 0 # sets MPLS label to 24 and decrements TTL
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 3 0 0 0 3 0 0 0 0 0 0 0 # modifies MPLS TC (set value to 3)
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 4 0 0 0 24 0 0 0 1 0 0 0 # sets MPLS label to 24 and TC to 1
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 5 0 0 0 1 0 0 0 0 0 0 0 # modifies stack value of MPLS header
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 6 0 0 0 6 0 0 0 0 0 0 0 # changes IP version to 6.
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 7 0 0 0 0 0 0 0 0 0 0 0 # swaps IP addresses
-$ sudo ovs-ofctl update-map br0 1 0 key 14 0 16 172 value 8 0 0 0 1 0 16 172 0 0 0 0 # sets source IP address to 172.16.0.1
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 0 0 0 0 0 0 0 0 0 0 0 0 # decrements MPLS TTL
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 1 0 0 0 24 0 0 0 0 0 0 0 # sets MPLS label to 24
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 2 0 0 0 24 0 0 0 0 0 0 0 # sets MPLS label to 24 and decrements TTL
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 3 0 0 0 3 0 0 0 0 0 0 0 # modifies MPLS TC (set value to 3)
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 4 0 0 0 24 0 0 0 1 0 0 0 # sets MPLS label to 24 and TC to 1
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 5 0 0 0 1 0 0 0 0 0 0 0 # modifies stack value of MPLS header
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 6 0 0 0 6 0 0 0 0 0 0 0 # changes IP version to 6.
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 7 0 0 0 0 0 0 0 0 0 0 0 # swaps IP addresses
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 14 0 16 172 value 8 0 0 0 1 0 16 172 0 0 0 0 # sets source IP address to 172.16.0.1
 ```
 
 ### IPv6 (oko-test-ipv6.p4)
@@ -67,22 +72,24 @@ The match key is source IPv6 address. The P4 program implements three actions:
 Sample usage:
 ```bash
 # Changes destination IPv6 address from fe80::a00:27ff:fe7e:b95 to e00::: (simple, random value)
-$ sudo ovs-ofctl update-map br0 1 0 key 254 128 0 0 0 0 0 0 10 0 39 255 254 21 180 17 value 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 254 128 0 0 0 0 0 0 10 0 39 255 254 21 180 17 value 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 # Swaps source and destination IPv6 addresses
-$ sudo ovs-ofctl update-map br0 1 0 key 254 128 0 0 0 0 0 0 10 0 39 255 254 21 180 17 value 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 254 128 0 0 0 0 0 0 10 0 39 255 254 21 180 17 value 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 # Sets Flow Label to 1.
-$ sudo ovs-ofctl update-map br0 1 0 key 254 128 0 0 0 0 0 0 10 0 39 255 254 21 180 17 value 2 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 254 128 0 0 0 0 0 0 10 0 39 255 254 21 180 17 value 2 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 ```
 
 ## Registers
 
-This section presents P4 program, which uses registers.   
+This section presents P4 programs, which use registers.   
 Register can be declared this way:  
 `Register<value_type, key_type>(number_of_elements) register_t;`  
 where:  
 `value_type` - is bit array type (i.e. bit<32>) or struct like type  
 `key_type` - is bit array type (i.e. bit<32>) or struct like type  
-`number_of_elements` - the maximum number of key-value pairs  
+`number_of_elements` - the maximum number of key-value pairs
+
+Currently registers have a limitation - they are not being initialized automatically. Initialization has to be done by control plane. 
 
 ### Rate limiter (rate-limiter.p4)
 
@@ -94,21 +101,86 @@ For instance now BUCKET_SIZE has value of 10 and WINDOW_SIZE has value of 100.
 It means that 10 packets are passed in 100 ms window. It also means 100 packets per second. 
 If you send 1470 Bytes width packets the bit rate should not exceed 1.176 Mbit/s (1470B * 8 * (10/100ms)).
 
-For measuring the bandwidth use for instance iperf:  
+Due to registers limitation before starting your own tests initialize rate limiter registers with zeros:
+```bash
+# Initalizes timestamp_r register
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 0 0 0 0 value 0 0 0 0
+# Initalizes count_r register
+$ sudo ovs-ofctl update-bpf-map br0 1 1 key 0 0 0 0 value 0 0 0 0
+```
+
+For measuring the bandwidth use for instance iperf tool:  
 
 Start a iperf UDP server
-```
-iperf -s -u
+```bash
+$ iperf -s -u
 ```
 Then run iperf client
-```
-iperf -c <server_ip> -b 10M -l 1470
+```bash
+$ iperf -c <server_ip> -b 10M -l 1470
 ```
 
 ### Rate limiter (rate-limiter-structs.p4)
 
-The same rate limiter as above but implemented with structs.
+The same rate limiter as above, but implemented using structs.
+
+### Simple firewall
+
+This is very simple example of stateful firewall. Every TCP packet is analyzed to track the state of the TCP connection. 
+If the traffic belongs to known connection it is passed. Otherwise, it is dropped.  
+Notice that the example program uses hash function which is constrained to hash only 64 bit values - that's why TCP connection is identified via IP source and destination address.  
+
+Due to registers limitation before starting your own tests initialize simple firewall registers with zeros:
+```bash
+# Initalizes conn_state register (key is a output from a hash function for client(192.168.1.10) and server (192.168.1.1))
+$ sudo ovs-ofctl update-bpf-map br0 1 0 key 172 192 20 5 value 0 0 0 0
+# Initalizes conn_srv_addr register (key is a output from a hash function for client(192.168.1.10) and server (192.168.1.1))
+$ sudo ovs-ofctl update-bpf-map br0 1 1 key 172 192 20 5 value 0 0 0 0
+```  
+
+To test simple firewall you can use as an example `ptf/stateful_test.py` test.
+
+## Tunneling
+
+This section presents more complex examples of packet tunneling operations. There are two P4 programs used:
+
+* `tunneling.p4`, which implements MPLS tunneling,
+* `vxlan.p4`, which implements more complex packet tunneling: VXLAN.
+
+### VXLAN
+
+To run example compile `vxlan.p4` with `p4c` and then `clang-6.0`. 
+
+Sample usage:
+```bash
+# Sets action vxlan_decap() (value 0) for packets matching rule VNI=25 (key 25) 
+# handled by the table upstream_tbl (map id 0) and BPF prog 1.
+sudo ovs-ofctl update-bpf-map br0 1 0 key 25 0 0 0 value 0 0 0 0
+# Sets action vxlan_encap() (value 0) for packets matching rule IP dstAddr=172.16.0.14 (key 14 0 16 172) 
+# handled by the table downstream_tbl (map id 1) and BPF prog 1.
+sudo ovs-ofctl update-bpf-map br0 1 1 key 14 0 16 172 value 0 0 0 0
+```
+
+### GPRS Tunneling Protocol (GTP)
+
+To run example compile `gtp.p4` with `p4c` and then `clang-6.0`. 
+
+To test encapsulation:
+```bash
+# For downstream_tbl (ID 1) sets action gtp_encap() (value 0) and GTP TEID=3 for packets with destination IP address 172.16.0.14.
+$ sudo ovs-ofctl update-bpf-map br0 10 1 key 14 0 16 172 value 0 0 0 0 3 0 0 0
+```
+
+To test decapsulation:
+```bash
+# For upstream_tbl (ID 0) sets action gtp_decap() for packets matching GTP TEID=3.
+$ sudo ovs-ofctl update-bpf-map br0 10 0 key 3 0 0 0 value 0 0 0 0
+```
 
-## Counters
+Scapy can be used to easily test GTP protocol:
 
-TBD
+```python
+>>> load_contrib('gtp')
+>>> p = Ether(dst='08:00:27:7e:0b:95', src='08:00:27:15:b4:11')/IP(dst='172.16.0.14', src='172.16.0.12')/UDP(sport=2152,dport=2152)/GTPHeader(teid=3)/IP(dst='172.16.0.14', src='172.16.0.12')/ICMP()
+>>> sendp(p, iface='eth1')
+```