ERSPAN on a Nexus 7010 (updated 2017-10-19)

We needed to determine the cause of some faults on our server network which is based on Nexus 7010 equipment.  The ERSPAN ("Encapsulated Remote Source Port ANalyzer") feature proved excellent for this for a number of reasons:

• We didn't need to physically go to the data centre to do the monitoring.  The monitoring is directed across an IP tunnel to the capture device, which can be anywhere else on an IP network.
• ERSPAN (and SPAN in general) on the Nexus platform has very good filtering capability so we can select just the traffic we want and monitor it on a machine with significantly less bandwidth than the capture device (in my case, my office machine monitoring 8x 10Gbit/s links looking for odd packets!).
• Related to the above, actually just physically connecting the 10Gbit/s ports to my laptop would be challenge, ignoring whether it can keep up with the traffic on them!

Setting up ERSPAN

In the Admin VDC (regardless of which VDC traffic is to be monitored in), the following global command needs to be set to define the source IP address for the packets.  This address does not have to available to the particular VDC where the monitoring is done:

  monitor erspan origin ip-address 192.84.5.248 global

The actual mirroring can then be set up as follows:

Command	Meaning
monitor session 1 type erspan-source	Create an ERSPAN session number 1
erspan-id 18	ID number transmitted in the ERSPAN packet to identify this particular SPAN
vrf default	VRF into which ERSPAN traffic is to be transmitted to the destination (not the VRF being monitored)
destination ip 192.0.2.1	IP address of destination (collector of ERSPAN traffic)
source interface port-channel789 both	Interface to be monitored
filter vlan 61,62	VLANs to be selected from the above interface (to monitor a whole VLAN, use 'source vlan ...' instead).  Filters can also reference access lists and other criteria.
filter access-group FILTER-VLANACCMAP	VLAN access-map to use to filter traffic (see below).
no shut	Enable the monitor

Filtering using an ACL

If you want to filter traffic matching things like source IP addresses, port numbers, etc. you can use an access-list to match it.  The Cisco documentation isn't clear on this, and contradicts how it's done, but a good reference is here.

In short, you must create an access-list, then a vlan access-map and apply that to the ERSPAN session.  For example:

  ip access-list ERSPAN-TRAFFIC-ACL4

   permit udp host 192.0.2.6 any eq 53

  !

  vlan access-map ERSPAN-TRAFFIC-VLAM 10

   match ip address ERSPAN-TRAFFIC-ACL4

  !

  monitor session 1 type erspan-source

   filter access-group ERSPAN-TRAFFIC-VLAM

Note that, despite the command "filter access-group ..." you MUST use a VLAN access-map (at least on 7.3.1 and above).

Capturing the traffic

Once enabled, the router will send the monitored traffic to the target host.  This will arrive as GRE traffic with protocol type 0x88be (ERSPAN).  Following the GRE header will be the ERSPAN header, which contains things like the VLAN ID and ERSPAN ID, then the entire L2 frame.

Wireshark recognises the packets as ERSPAN packets without any configuration, and will correctly decode them to show the encapsulated packet directly, allowing you to watch things in real time from the comfort of your desk!