Configuring Cisco and Open Switch Routers and Layer 3 devices to send NetFlow data to a Linux NTOP NetFlow collector to interpret higher layer network flows.
Introduction
Layer 2 and Layer 3 Linux Switching (using kernel-supported utilities) is fast and efficient. Open vSwitch -- when combined with an OpenFlow controller -- is a full-blown Software Defined Networking (SDN) implementation that offers the advantages of offloading topology and switching decisions to a centralized controller, allowing the Open vSwitch devices to primarily switch frames instead of expending processor and memory resources on topology maintenance and decision-making.Open vSwitch devices are also Layer 3 and above aware, able to modify traffic based upon Layer 3 (IP address) and Layer 4 (TCP port) information. However, an administrator needs quality information to make prioritization and routing decisions and OpenFlow controllers provide mostly Layer 2 information. Systems monitoring devices -- MRTG, Cacti, Munin, Nagios/Icinga and Zabbix -- operate primarily at the device level (e.g. interface utilization, interface errors, interface congestion) and are difficult, if not impossible, to configure for detailed analysis. Firewalls and access lists operate at the requisite levels (Layers 2, 3 and 4) for detailed traffic analysis, but require a great deal of manual configuration and interpretation to characterize traffic.
The term used for this kind of detailed traffic characterization is Flows. Each flow has unique end devices, IP addressing and TCP ports. At the flow level, each step in the path between end devices (i.e. intermediary switches and routers) also records traffic. As stated above, OpenFlow controllers report this quite well at Layer 2. Other protocols, such as NetFlow and OpenFlow, provide the higher-level information required to more precisely characterize and manage traffic.
Configuring Open vSwitch Layer 3 Switching
There are several ways to implement Layer 3 functionality, such as fake bridges and VLANs. From a monitoring perspective, these can be somewhat problematical because they do not appear in SNMP MIBs.This article will use multiple bridges on each Layer 3 Open vSwitch to define multiple networks. Bridge and port membership is defined with the ovs-vsctl set of commands. IP address assignments to the bridges is managed at boot time in the /etc/network/interfaces file. Routing is managed using Quagga.
In the topology at the top of the article, there is routing configured on the Cisco R1 gateway and Open vSwitch devices Switch-01 and Switch-05. All are members of OSPF Area 10.128.0.0 (range 10.128.0.0/13 with configured networks 10.128.0.0/24 and 10.128.1.0/24) and the Cisco router is also a member of the host laptop's Backbone Area 0.0.0.0 (range 172.16.0.0/12).
Switch-01 Configuration
The /etc/network/interfaces File
auto eth0iface eth0 inet manual
auto eth1
iface eth1 inet manual
auto eth2
iface eth2 inet manual
...
auto eth7
iface eth7 inet manual
auto br0
iface br0 inet static
address 10.128.0.254
netmask 255.255.255.0
network 10.128.0.0
broadcast 10.128.0.255
gateway 10.128.0.1
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 192.168.1.1
dns-search mydomain.com
auto br1
iface br1 inet static
address 10.128.1.254
netmask 255.255.255.0
network 10.128.1.0
broadcast 10.128.1.255
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 192.168.1.1
dns-search mydomain.com
Open vSwitch Configuration
18ce5ab3-d025-49c2-8cd8-0a41d3927c79Bridge "br1"
Controller "tcp:10.128.0.102:6633"
is_connected: true
Port "eth4"
Interface "eth4"
Port "br1"
Interface "br1"
type: internal
Bridge "br0"
Controller "tcp:10.128.0.102:6633"
is_connected: true
Port "eth1"
Interface "eth1"
Port "eth6"
Interface "eth6"
Port "eth5"
Interface "eth5"
Port "eth3"
Interface "eth3"
Port "br0"
Interface "br0"
type: internal
Port "eth7"
Interface "eth7"
Port "eth2"
Interface "eth2"
Port "eth0"
Interface "eth0"
ovs_version: "2.1.3"
The Quagga OSPF Configuration
router ospfospf router-id 10.128.0.254
network 10.128.0.0/24 area 10.128.0.0
network 10.128.1.0/24 area 10.128.0.0
area 10.128.0.0 range 10.128.0.0/13
Switch-05 Configuration
The /etc/network/interfaces file
auto eth0iface eth0 inet manual
auto eth1
iface eth1 inet manual
auto eth2
iface eth2 inet manual
...
auto eth7
iface eth7 inet manual
auto br0
iface br0 inet static
address 10.128.1.250
netmask 255.255.255.0
network 10.128.1.0
broadcast 10.128.1.255
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 192.168.1.1
dns-search mydomain.com
Open vSwitch Configuration
18ce5ab3-d025-49c2-8cd8-0a41d3927c79Bridge "br0"
Controller "tcp:10.128.0.102:6633"
is_connected: true
Port "eth1"
Interface "eth1"
Port "eth6"
Interface "eth6"
Port "eth0"
Interface "eth0"
Port "eth5"
Interface "eth5"
Port "eth3"
Interface "eth3"
Port "br0"
Interface "br0"
type: internal
Port "eth4"
Interface "eth4"
Port "eth7"
Interface "eth7"
Port "eth2"
Interface "eth2"
ovs_version: "2.1.3"
The Quagga OSPF Configuration
router ospfospf router-id 10.128.1.250
network 10.128.1.0/24 area 10.128.0.0
area 10.128.0.0 range 10.128.0.0/13
Installing NTOP on Ubuntu 14.04 Trusty Tahr
Keep in mind NTOP is not NTOP-NG. NTOP-NG is the current version of the utility, but it decouples NetFlow from the application and requires the paid nProbe package. For this article, we will use the older -- but still useful -- original NTOP.The Easy (and Broken) Way
It's very easy -- apt-get install ntop. You will need to configure the listening interface(s) and supply an administrator password.The problem with this installation is the package does not install the visualization packages required for elements like pie graphs and host details, despite using the last version (5.0.1 dated 2012-08-13). The package does not provide the full range of ntop features.
Compiling from Source
Install dependenciessudo apt-get install libpcap-dev libgdbm-dev libevent-dev librrd-dev python-dev libgeoip-dev automake libtool subversion
sudo apt-get build-dep ntopExtract and compile
tar zxvf ntop-5.0.1.tar.gzCopy libraries to the correct location, change ownership of the executable and restart the service
cd ntop-5.0.1
./autogen
make
sudo make install
sudo cp /usr/local/lib/libntop* /usr/lib/
sudo chown -R ntop.ntop /usr/local/share/ntop
sudo service ntop restart
Configuring NTOP
The installation process specifies one or more listening interfaces. These are available without additional configuration. The overview report provides information about this (ese) interface(s), such as packet distribution and size.
And another for Application Protocols.
In a switched environment, an interface in promiscuous mode will still only capture unicast traffic switched to its own port, broadcast and multicast traffic. The capture does not reflect overall network activity.
One option to capture traffic is port mirroring, in which all traffic on a specified interface is forwarded to another (in this case, the NTOP monitor). The drawbacks to port mirroring are manual configuration, the added bandwidth and processing required. Even using virtio network devices on virtualized machines, the processing and memory bandwidth will contribute to added load on the host.
NetFlow
Netflow is a set of services developed by Cisco. Briefly, a NetFlow capable device summarizes traffic, formats it and forwards it to a collector over UDP. Version 9 is current at the time of writing and its format is detailed here. Essentially, it provides a summary of TCP/IP protocol, address and application information the collector may store, format and present.Configuring NTOP NetFlow Collector
From the main menu, select Plugins and click on the NetFlow option.
NetFlow Device
This is a unique name to identify a network-specific collector.Local Collector UDP Port
This specifies the UDP port on which the collector listens. 2055 is the current default.Virtual NetFlow Interface Network Address
This is NOT an IP address, but a network address that corresponds to the network from which device(s) send information. In the configuration used in this article it is 10.128.0.0/24, which will save as 10.128.0.0/255.255.255.0. This virtual interface may be the destination for more than one collector on each defined network. In this article, we will configure a Cisco router and Ubuntu Open vSwitch to forward NetFlow information to the NTOP server.There are other options available, but they will not be discussed in this article.
Configuring Cisco NetFlow
The following commands begin with configuring NetFlow to operate on interface FastEthernet0/1, collecting inbound and outbound data. Then we configure the router export dource interface, timeouts, NetFlow version and (finally) the destination IP address and UDP port of the NTOP server.
R1#configure terminal
R1(config)#interface f0/1
R1(config-if)#ip route-cache flow
R1(config-if)#ip flow ingress
R1(config-if)#ip flow egress
R1(config-if)#exit
R1(config)#ip flow-export source f0/1
R1(config)#ip flow-cache timeout active 60
R1(config)#ip flow-cache timeout inactive 120
R1(config)#ip flow-export version 5
R1(config)#ip flow-export destination 10.128.1.104 2055
Configuring Open vSwitch Netflow
First some background on Open vSwitch and OpenFlow (e.g. Floodlight) controllers. Open vSwitch alone operates quickly at Layer 2 -- a bridge or switch. The OpenFlow controller centralizes MAC address logic and topologies and is a decision relieves the connected Open vSwitch of logic and topology decisions. Yet Open vSwitch is also aware of higher layers of the network stack -- Layer 3 IP, Layer 4 TCP, etc. The deficiency of existing OpenFlow controllers is a lack of detailed Layer 3 and above information upon which to make configuration decisions such as prioritization. Enter NetFlow.The following command (one statement) configures Open vSwitch NetFlow for bridge br0 and forward it to the NTOP NetFlow collector with an active timeout of 120 seconds:
sudo ovs−vsctl set Bridge br0 netflow=@nf0 -- --id=@nf0 create NetFlow targets=\"10.128.1.104:2055\" active_timeout=120More than one bridge on each device may then be added, such as br1, with the command:
ovs-vsctl set Bridge br1 netflow=[_uuid]
Viewing NetFlow Information on NTOP
Switch to the Netflow interface configured above if you have not already done so. The interface is identical, but now that NetFlow is collecting all port and host information for the network 10.128.0.0/24, there will be much more information.
Top talkers are also identified and graphed.
