Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add injection to python rest api #1508

Merged

Conversation

ofirZelig
Copy link
Contributor

@ofirZelig ofirZelig commented Dec 5, 2018

added a functions to the RestApi that supports injections in skydive :
injection_create - support creation of injections, the function returns an InjectionRule.
injection_delete - support deletion of injections.
injection_list - returns a list of all the current injections in the system.

@masco
Copy link
Member

masco commented Dec 5, 2018

it is nice to have some tests and squash the commits.

@lebauce
Copy link
Member

lebauce commented Dec 5, 2018

add to whitelist

@lebauce
Copy link
Member

lebauce commented Dec 5, 2018

run skydive-python-tests

@lebauce
Copy link
Member

lebauce commented Dec 5, 2018

@ofirZelig Thank you for this. Could you please also add support for deletion ?

@safchain safchain requested a review from masco December 14, 2018 10:46
@masco
Copy link
Member

masco commented Dec 14, 2018

Please add some tests, it would be easy to validate the code.

contrib/python/api/skydive/rest/client.py Outdated Show resolved Hide resolved
contrib/python/api/skydive/rest/client.py Outdated Show resolved Hide resolved
contrib/python/api/skydive/rest/client.py Outdated Show resolved Hide resolved
@skydive-bot
Copy link
Member

Can one of the admin review this patch ?

Copy link
Collaborator

@safchain safchain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add test for that ?

contrib/python/api/skydive/rest/client.py Outdated Show resolved Hide resolved
contrib/python/api/skydive/rest/client.py Outdated Show resolved Hide resolved
@hunchback
Copy link
Collaborator

hunchback commented Dec 20, 2018

@ofirZelig - please squash the commits - for that you can use:

git rebase -i HEAD~10

and then place an "s" command on each but your first commit. and then doing so please provide a concise commit message

@hunchback
Copy link
Collaborator

@ofirZelig - please extend the description of what this PR does; and why

@hunchback
Copy link
Collaborator

@ofirZelig - to test your code please use: https://docs.python.org/3/library/unittest.html

the tests should be wrapped by a script which can be invoked via CI:

scripts/ci/run-python-api-tests.sh

if you need any setup run before running these tests - then implement it in:

scripts/ci/install-<XXX>.sh

and as a development environment please use a devmode VM which can be setup as explained here:

http://skydive.network/documentation/contribute#development-box

@ofirZelig
Copy link
Contributor Author

Hi @hunchback ,

We are trying to deploy the development environment on ubuntu 18.04 using virtualbox 5.2.18 and vagrant 2.1.1. The problem is that the files sharing between the guest and the host doesnt work. We cant find the source code inside the guest so we cant build it.

Can you think of any explanation or solution for this?

@ofirZelig
Copy link
Contributor Author

@hunchback
update: We have cloned the repository to our machine but when building it we recieve the following error message:

[vagrant@dev skydive]$ make
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_MEASUREMENT = "he_IL.UTF-8",
LC_PAPER = "he_IL.UTF-8",
LC_MONETARY = "he_IL.UTF-8",
LC_NAME = "he_IL.UTF-8",
LC_ADDRESS = "he_IL.UTF-8",
LC_NUMERIC = "he_IL.UTF-8",
LC_TELEPHONE = "he_IL.UTF-8",
LC_IDENTIFICATION = "he_IL.UTF-8",
LC_TIME = "he_IL.UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
CC= GOARCH= go get github.com/kardianos/govendor
/home/vagrant/bin/govendor sync
patch -p0 < dpdk/dpdk.govendor.patch
patching file vendor/github.com/intel-go/yanff/low/low.go
rm -rf vendor/github.com/weaveworks/tcptracer-bpf/vendor/github.com/
mv /home/vagrant/.skydive-build-tool bin || true; ln -s vendor src || mv vendor src; cd src/gopkg.in/src-d/proteus.v1/cli/proteus; (unset GOARCH; unset CC; GOPATH=$GOPATH/src/github.com/skydive-project/skydive go install gopkg.in/src-d/proteus.v1/cli/proteus); cd -; mv bin /home/vagrant/.skydive-build-tool; unlink src || mv src vendor; proteus proto -f ${GOPATH}/src -p github.com/skydive-project/skydive/flow/layers
/home/vagrant/src/github.com/skydive-project/skydive
error scanning package "github.com/skydive-project/skydive/flow/layers": /home/vagrant/src/github.com/skydive-project/skydive/flow/layers/layers.go:25:8: could not import github.com/google/gopacket/layers (/home/vagrant/src/github.com/skydive-project/skydive/vendor/github.com/google/gopacket/layers/arp.go:11:2: could not import encoding/binary (/home/vagrant/.gimme/versions/go1.9.1.linux.amd64/src/encoding/binary/binary.go:26:2: could not import io (/home/vagrant/.gimme/versions/go1.9.1.linux.amd64/src/io/pipe.go:12:2: could not import sync (/home/vagrant/.gimme/versions/go1.9.1.linux.amd64/src/sync/cond.go:8:2: could not import sync/atomic (/home/vagrant/.gimme/versions/go1.9.1.linux.amd64/src/sync/atomic/value.go:31:37: cannot convert v (variable of type *Value) to /home/vagrant/.gimme/versions/go1.9.1.linux.amd64/src/unsafe.Pointer)))))
make: *** [Makefile:233: flow/layers/generated.proto] Error 1

@safchain
Copy link
Collaborator

safchain commented Jan 2, 2019

Install by hand is pretty simple, you could try this:
http://skydive.network/documentation/build

@lebauce
Copy link
Member

lebauce commented Jan 2, 2019

@ofirZelig It seems your GOPATH is /home/vagrant. Therefore, your GOROOT is inside GOPATH which is known to cause problems. We updated the Vagrant box in this commit 9bb05e8
So you may want to use a newer version of the Vagrant box.

@hunchback
Copy link
Collaborator

@ofirZelig - I recommend that you use our standard test/development environment which can be deployed on your laptop/host by following:

http://skydive.network/documentation/contribute#development-box

Once done I can provide additional details on running the specific tests you have.

@ofirZelig
Copy link
Contributor Author

@safchain I'm currently working on adding InjectionRule to rules.py, but it is not clear which fileds are optional in the injection. Can you say which is optional from the following fields:
uuid, src, dst, srcip, dstip ,srcmac, dstmac, srcport, dstport, type, payload, trackingid, icmpid, count, interval, increment, starttime

Thank you

@ofirZelig ofirZelig force-pushed the Add_Injection_To_Python_Rest_Api branch from a59430b to 33c29c8 Compare January 9, 2019 14:04
@ofirZelig
Copy link
Contributor Author

ofirZelig commented Jan 9, 2019

@safchain
We fixed all the changes which were requested in this PR, and squashed the commits.
The only problem is that in the automatic tests we need to create nodes but they are not able to inject. We are currently trying to solve the problem, do you have an idea what is the best way to get nodes that are able to inject?

  • note: The tests we created which are not automated pass successfully

@safchain
Copy link
Collaborator

safchain commented Jan 9, 2019

@ofirZelig maybe the loopback interface as source and destination

Copy link
Collaborator

@safchain safchain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to be able to test packet injection we need to start an analyzer and an agent however only an analyzer is currently started. In order to have both you need to change
analyzer by allinone

https://github.com/skydive-project/skydive/blob/master/contrib/python/api/tests/tests.py#L51

Then you could use the eth0 interface like this G.V().Has('Name', 'eth0')

I tested by hand and it worked so I guess it should work with python

contrib/python/api/skydive/rules.py Outdated Show resolved Hide resolved
contrib/python/api/skydive/rules.py Outdated Show resolved Hide resolved
contrib/python/api/skydive/rest/client.py Outdated Show resolved Hide resolved
@ofirZelig
Copy link
Contributor Author

@safchain Is there progress in our PR?

@safchain
Copy link
Collaborator

@ofirZelig there are multiple coding style issue to fix, please have a look at the CI result

@safchain
Copy link
Collaborator

run skydive-python-tests

@safchain
Copy link
Collaborator

run skydive-python-tests

1 similar comment
@safchain
Copy link
Collaborator

run skydive-python-tests

@safchain
Copy link
Collaborator

@ofirZelig why did you reverted this one 3b51fa0
? we need to use the allinone mode otherwise it fails

@ofirZelig
Copy link
Contributor Author

@ofirZelig why did you reverted this one 3b51fa0
? we need to use the allinone mode otherwise it fails

@safchain
The ssl tests fail when using "allinone" instead of "analyzer", even the tests we did not touch, do you know what is the reason? Meanwhile we tried a new approach.

@safchain
Copy link
Collaborator

@ofirZelig I'm checking why with TLS it is not working

@safchain
Copy link
Collaborator

@ofirZelig I submitted this #1621 to fix the TLS issue

@ofirZelig
Copy link
Contributor Author

@safchain
I pushed a new version of the tests, hopfully now they would pass.

@safchain
Copy link
Collaborator

@ofirZelig can you please rebase on top of master

add support for creation, deletion, list and added tests
@safchain
Copy link
Collaborator

run skydive-python-tests

@ofirZelig
Copy link
Contributor Author

@ofirZelig can you please rebase on top of master

Fixed the message, and as I understood, squashing the commits is much easier after the PR is accepted

@safchain
Copy link
Collaborator

@ofirZelig you need to rebase on top of master as we just merged the patch that fixes the TLS/allinone issue otherwise the tests won't pass

@ofirZelig
Copy link
Contributor Author

@safchain
Merged with the current master, everything should be updated

@safchain
Copy link
Collaborator

run skydive-python-tests

@ofirZelig
Copy link
Contributor Author

@safchain
Still the test cant find a node with eth0, even though we merged your commit

@safchain
Copy link
Collaborator

@ofirZelig can you apply this https://paste.fedoraproject.org/paste/naOHo3~ylIlW5B-x9HfTJQ

Please squash it if it passes I'll merge it. Thanks

@lebauce
Copy link
Member

lebauce commented Jan 29, 2019

run skydive-python-tests

@lebauce lebauce merged commit 72d7ce8 into skydive-project:master Jan 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants