Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add libbpf to maintain Ubuntu 16.04 compatibility #2403

Merged
merged 17 commits into from
Jun 3, 2020

Conversation

fruffy
Copy link
Collaborator

@fruffy fruffy commented May 28, 2020

This pull request adds libbpf as dependency to make the eBPF back end more stable. It makes the generated code less dependent on the kernel headers of a particular distribution. This may help with the problem that different distributions of Ubuntu cannot compile the code generated by p4c-ebpf. Also fixes #1961.

Let's see if this works out.

Changes:

  • Added libbpf as dependency.
  • Reenabled eBPF kernel tests.
  • Added iproute2 to the Travis build.
  • Runs Travis in privileged mode to allow the creation of network namespaces.
  • Add stf test for lpm_ebpf.p4, also marked this test as xfail.
  • Added the extern object tests to the user-space eBPF tests.
  • Cleaned up the ebpf_kernel.h file, using only the imports from libbpf.
  • Added BPF_USER_MAP_UPDATE_ELEM to the user-space test framework.
  • Cleaned up the eBPF makefiles a little.

backends/ebpf/README.md Show resolved Hide resolved
#include "ebpf_common.h"


/* If we operate in user space we only need to include bpf.h and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this compile with any kernel version?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well this is what libbpf says: "No ties to any specific kernel, transparent handling of older kernels. Libbpf is designed to be kernel-agnostic and work across multitude of kernel versions. It has built-in mechanisms to gracefully handle older kernels, that are missing some of the features, by working around or gracefully degrading functionality. Thus libbpf is not tied to a specific kernel version and can/should be packaged and versioned independently."

I try to use their custom headers everywhere I can.

@@ -141,6 +141,15 @@ else()
set (SUPPORTS_KERNEL False)
endif()

if (EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/runtime/usr/include/bpf")
message(STATUS "Found libbpf")
else()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you test this on MacOS?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see what Travis will say.

({ \
char ____fmt[] = fmt; \
bpf_trace_printk(____fmt, sizeof(____fmt), \
##__VA_ARGS__); \
})

/* helper functions called from eBPF programs written in C */
static void *(*bpf_map_lookup_elem)(void *map, void *key) =
static void *(*bpf_map_lookup_elem)(void *map, const void *key) =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping we would not need this junk anymore.
Isn't there a header with this stuff in libbpf?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will see if I can strip more later. Last time I checked those were still necessary.

backends/ebpf/runtime/runtime.mk Outdated Show resolved Hide resolved
@mihaibudiu
Copy link
Contributor

There are some problems with your Python scripts.

@fruffy
Copy link
Collaborator Author

fruffy commented May 29, 2020

Trying to support kernel eBPF tests in Ubuntu 16.04 is just pure masochism. At this point we also need to build the iproute2 package from source... These are the same problems we had with the xdp back ends so at least we have the fixes already.

@fruffy
Copy link
Collaborator Author

fruffy commented May 29, 2020

Trying to fix the build unearthed a bunch of other issues. Right now, I have no idea what to do about the failing tests. One of them requires a change to how tables are registered, the other is the LPM_TRIE map getting rejected. They are failing for reasons we have missed before.

@mihaibudiu
Copy link
Contributor

I thought we didn't support lpm in the ebpf backends.

@fruffy
Copy link
Collaborator Author

fruffy commented May 29, 2020

There is a test that fails for the kernel
https://github.com/p4lang/p4c/blob/master/testdata/p4_16_samples/lpm_ebpf.p4
We can mark it xfail for now.

@fruffy
Copy link
Collaborator Author

fruffy commented May 29, 2020

The other tests that fail are the extern object ones, they do not play nicely for various reasons.

@fruffy
Copy link
Collaborator Author

fruffy commented Jun 1, 2020

I think this is ready for review now. I also created some issues based on problems I have encountered.
I reenabled the kernel tests, I hope they are stable now. The thing I can see becoming a problem is iproute2.

@fruffy fruffy marked this pull request as ready for review June 1, 2020 09:24
Copy link
Contributor

@mihaibudiu mihaibudiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if anyone else wants to review this.

@mihaibudiu
Copy link
Contributor

@fruffy : is this finished?

@fruffy
Copy link
Collaborator Author

fruffy commented Jun 3, 2020

From my point of view it is. The remaining issues should probably be handled separately. Merging this will break the xdp back end, but I have a pull request ready for it.

@mihaibudiu mihaibudiu merged commit 0abf901 into p4lang:master Jun 3, 2020
@fruffy fruffy deleted the libbpf branch July 10, 2020 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ebpf_common.h should not include linux/bpf.h
3 participants