Skydive is “an open source real-time network topology and protocols analyzer”. It is a tool (with CLI and web interface) to help analyze and debug your network (OpenStack, OpenShift, containers, …).
The Data Plane Development Kit (DPDK) is a set of data plane libraries and network interface controller drivers for fast packet processing. http://dpdk.org/
As Skydive uses flow probes to analyze packets, thanks to gopacket. As the bandwidth used by Software Defined Networks increases, we need a solution to capture and analyze packets at high speed from switch port mirroring.
Let’s setup a demo environment
The demo setup is a switch with a 10Gbits/s port configured as port mirroring and a dedicated host running Skydive Agent with the DPDK configuration above.
Two hosts running VMs and containers will be named dpdk1 and dpdk3.
First we must re-compile, Skydive. This is because the DPDK flow probe has strong version dependencies between the go-yanff projects and its dependencies.
# cd $GOPATH/src/github.com/skydive-project/skydive # make WITH_DPDK=true # echo 256 | sudo tee /proc/sys/vm/nr_hugepages # sudo modprobe uio # sudo insmod ./dpdk/dpdk-17.08/x86_64-native-linuxapp-gcc-install/lib/modules/$(uname -r)/extra/dpdk/rte_kni.ko # sudo insmod ./dpdk/dpdk-17.08/x86_64-native-linuxapp-gcc-install/lib/modules/$(uname -r)/extra/dpdk/igb_uio.ko
Feel free to install the rte_kni.ko and igb_uio.ko on your system and add a hugepage sysctl rule to /etc/sysctl.d/50-dpdk.conf.
DPDK quick configuration
On fedora userspace helper :
# dnf install -y dpdk-tools # dpdk_nic_bind --status Network devices using DPDK-compatible driver ============================================ 0000:05:00.0 'Ethernet Controller 10-Gigabit X540-AT2' drv=ixgbe unused=igb_uio 0000:05:00.1 'Ethernet Controller 10-Gigabit X540-AT2' drv=ixgbe unused=igb_uio
Bind/map the DPDK driver to the first physical port
# dpdk_nic_bind -b igb_uio 0000:05:00.0
Configure your switch in port mirroring, to send a copy of flow traffic to the first physical port
In the agent yaml file, we need to enable the dpdk probe by adding these lines :
dpdk: # DPDK port listening flows from ports: - 0 # - 1 # nb workers per port workers: 4 # debug message every n seconds # debug: 1
This configures the agent to set up the DPDK flow probe to capture traffic from the first physical port card.
On a VM hosted on dpdk1, generate some ICMP traffic to a VM hosted on dpdk3
vm1: # ping vm2
Now let’s enable traffic capture on the DPDK ports
# skydive --conf /tmp/skydive.yaml client capture create --gremlin "G.V().Has('Type', 'dpdkport')"
Let’s see the captured traffic
# skydive --conf /tmp/skydive.yaml client topology query --gremlin "G.V().Has('Type', 'dpdkport').Flows()"
This expression retrieves all the flows captured by a DPDK port.
Note : These commands could be done and seen on the web interface too, on the Capture, Flows tabs
During DPDK flow probe basic performance testing, I was able to achieve higher throughput on one worker by injecting traffic with tcpreplay.
The DPDK flow probe is configured to dispatch packets between workers based on a calculated L3 hash.
Using the DPDK flow probe is huge step in capturing real traffic behavior on physical infrastructure and have much better performance compared to the default afpacket probe.
Project site: http://skydive.network
The YouTube channel with some demo videos
Mailing list: email@example.com
IRC: #skydive-project on irc.freenode.net