Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Need assistance in capturing the flow data using nfprobe plugin #846

Open
desuraghu13 opened this issue Jan 1, 2025 · 6 comments
Open

Need assistance in capturing the flow data using nfprobe plugin #846

desuraghu13 opened this issue Jan 1, 2025 · 6 comments

Comments

@desuraghu13
Copy link

desuraghu13 commented Jan 1, 2025 •
edited
Loading

Description
We are planning to use nfprobe plugin to monitor a ubuntu box.
All the flows are getting captured, but the interface index always getting as 0.
Is there any possibility of capturing on which interface the flow has been generated ?
Can we achieve it using tags or label ? I need to add interface name alone to the flow data.

If i specify direction in the pcap_interface.map, able to capture the traffic in one of the direction only.
As i am looking to capture both directions traffic.

Version
Promiscuous Mode Accounting Daemon, pmacctd 1.7.9-git (20240618-0 (a2f3eaa))

Config:
daemonize: false
debug: true
pcap_ifindex: map
pcap_interfaces_map: /etc/pmacct/pcap_interfaces.map
aggregate: src_host, dst_host, in_iface, out_iface, timestamp_start, timestamp_end, src_port, dst_port, proto, tos
plugins: nfprobe
nfprobe_receiver: x.x.x.:xxx
nfprobe_version: 10

Interface map:
ifindex=100 ifname=vpp1
ifindex=200 ifname=vpp2
ifindex=300 ifname=vpp3
ifindex=400 ifname=vpp10
ifindex=500 ifname=vpp14
ifindex=600 ifname=vpp30

Logs:
INFO ( default/core ): Reading configuration file '/etc/pmacct/pmacctd.conf'.
INFO ( default/core ): [/etc/pmacct/pcap_interfaces.map] (re)loading map.
INFO ( default/core ): [/etc/pmacct/pcap_interfaces.map] map successfully (re)loaded.
INFO ( default/core ): [vpp1,100] link type is: 1
INFO ( default/core ): [vpp2,200] link type is: 1
INFO ( default/core ): [vpp3,300] link type is: 1
INFO ( default/core ): [vpp10,400] link type is: 1
INFO ( default/core ): [vpp14,500] link type is: 1
INFO ( default/core ): [vpp30,600] link type is: 1

Output Received:
{
"host": {
"ip": "172.168.100.1"
},
"netflow": {
"ipClassOfService": 0,
"flowStartMilliseconds": "2025-01-01T06:46:40.166Z",
"tcpControlBits": 0,
"octetDeltaCount": 70,
"ingressInterface": 0,
"sourceTransportPort": 50451,
"destinationIPv4Address": "224.0.0.251",
"ipVersion": 4,
"destinationTransportPort": 5353,
"version": 10,
"sourceIPv4Address": "192.168.1.6",
"protocolIdentifier": 17,
"protocolIdentifierString": "UDP",
"egressInterface": 0,
"flowDirection": 0,
"packetDeltaCount": 1,
"flowEndMilliseconds": "2025-01-01T06:46:40.166Z"
},
"@timestamp": "2025-01-01T06:50:01.000Z",
"@Version": "1"
}

Please guide me in correcting the interface index.
@msune
Copy link
Contributor

msune commented Jan 5, 2025

Just to confirm - based on the iface names -, are you using VPP?

@desuraghu13
Copy link
Author

@msune , yes i am using it in VPP

@desuraghu13
Copy link
Author

Also, is there any way i can add hostname dynamically to label as environment variable instead of hardcoding ?
If i have enabled class in the aggregation, i am getting some junk chars in the logstash when flow is getting encoded. Not sure where the problem is. If anyone can help me in resolving these 2 issues, will be a big help for me.

@msune
Copy link
Contributor

msune commented Jan 7, 2025

I think I am able to reproduce this with the HEAD of master, without VPP, with this simplified config configurations:

$ cat test.conf 
daemonize: false
debug: true
pcap_ifindex: map
pcap_interfaces_map: ./pcap_interfaces.map
aggregate: src_host, dst_host, in_iface, out_iface, timestamp_start, timestamp_end, src_port, dst_port, proto, tos
plugins: nfprobe
nfprobe_receiver: 127.0.0.1:2055
nfprobe_version: 10

$ cat pcap_interfaces.map 
ifindex=3 ifname=wlp0s20f3

Execution:

sudo ./pmacctd -f ./test.conf 
INFO ( default/core ): Promiscuous Mode Accounting Daemon, pmacctd 1.7.10-git (20250103-0 (5a5ca2b0))
INFO ( default/core ):  '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'
INFO ( default/core ): Reading configuration file '/home/marc/personal/pmacct/build/src/test.conf'.
INFO ( default/core ): [./pcap_interfaces.map] (re)loading map.
INFO ( default/core ): [./pcap_interfaces.map] map successfully (re)loaded.
INFO ( default/core ): [wlp0s20f3,3] link type is: 1
INFO ( default_nfprobe/nfprobe ): plugin_pipe_size=4096000 bytes plugin_buffer_size=259 bytes
INFO ( default_nfprobe/nfprobe ): ctrl channel: obtained=212992 bytes target=126512 bytes
INFO ( default_nfprobe/nfprobe ): NetFlow probe plugin is originally based on softflowd 0.9.7 software, Copyright 2002 Damien Miller <djm@mindrot.org> All rights reserved.
INFO ( default_nfprobe/nfprobe ):           TCP timeout: 3600s
INFO ( default_nfprobe/nfprobe ):  TCP post-RST timeout: 120s
INFO ( default_nfprobe/nfprobe ):  TCP post-FIN timeout: 300s
INFO ( default_nfprobe/nfprobe ):           UDP timeout: 300s
INFO ( default_nfprobe/nfprobe ):          ICMP timeout: 300s
INFO ( default_nfprobe/nfprobe ):       General timeout: 3600s
INFO ( default_nfprobe/nfprobe ):      Maximum lifetime: 604800s
INFO ( default_nfprobe/nfprobe ):       Expiry interval: 60s
INFO ( default_nfprobe/nfprobe ): Exporting flows to [127.0.0.1]:2055

image

@paololucente?

@desuraghu13
Copy link
Author

desuraghu13 commented Jan 8, 2025 •
edited
Loading

@msune , i have added pcap_interfaces.map as follows to get the interface index

Interface map:
ifindex=100 ifname=vpp1 direction=in
ifindex=200 ifname=vpp2 direction=in

ifindex=100 ifname=vpp1 direction=out
ifindex=200 ifname=vpp2 direction=out

With this combination able to get the interface index.

How ever, facing 2 more challenges.

  1. We are using DHCP to get the host ip. Each time when we restart the device, new IP is getting assigned. So need to change the nfprobe_src_host manually. If i don't specify the value, it will pickup automatically any one interface ip. Which will not work in my case. To mitigate the same, planning to use hostname as part of label. Was looking for some config option to pickup the hostname directly from environment instead of had coding
  2. When i am encoding the data in logstash, if i use IPFIX format, application id is not coming properly. Getting some junk characters. Not sure where the issues is.

@paololucente, can you please help us here.

@msune
Copy link
Contributor

msune commented Jan 10, 2025

RE: With this combination able to get the interface index.

I think this is explained here:

Omitting the direction enables both ingress and egress collection: in this case populating in_iface or out_iface with the supplied ifindex does stops (as it would not be possible anymore to clearly determine direction).

Which I believe it means that it will be 0, so it is consistent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@msune @desuraghu13