Port mirroring on VMware vSwitch/dvSwitch (dvMirror)

Port mirror is a traditional features available on physical switch to capture port traffic and redirect to a remote destination. A remote destination can be (using the Cisco convention):

  • A physical port located on the same physical switch (Switch port Analyzer or SPAN port). The traffic is simply copied to the destination port too.
  • A physical port located on a remote L2 connected physical switch (Remote SPAN or RSPAN port). The traffic for each RSPAN session is carried over a user-specified RSPAN VLAN that is dedicated for that RSPAN session in all participating switches.
  • A remote host (Encapsulated RSPAN or ERSPAN port). ERSPAN add an ERSPAN header to the original packets and encapsulates it using GRE.

A SPAN/RSPAN/ERSPAN port is usually used for monitoring purpose (network debug, application performance monitoring, security prevention…). On the vSphere world, port mirror is a specific features available on distributed switches. A workaround is available if only standard switches are available.

Mirroring port on a vSwitch

A vSwitch does not have mirroring capabilities, but a simple workaround can be used to implement a SPAN port like. Let’s see the following scenario: cfg_vSwitch0 We want the traffic from and to ubuntu3 VM mirrored to the second vnic of receiver VM. We want also that other VMs do not receive ubuntu3 traffic. Using standard switches the only solution is:

  • create another port group equivalent to the previous one inside the same vSwitch;
  • override on the port group the security policy “Promiscuous mode”;
  • move ubuntu3 VM and receiver vnic to the new port group;
  • both VMs must run on the same vSwitch/ESXi host.

The “Promiscuous mode” configure the vSwitch like a hub: all VMs connected to this port group will broadcast traffic to all connected VMs. vSwitch_mirror Now a software monitor (like tcpdump) running on receiver VM can analyze ubuntu3 VM’s traffic. VM connected to other port groups or vSwitches won’t receive ubuntu3 traffic.

Mirroring port on a dvSwitch (dvMirror)

A distributed switch offers a native feature to mirror a vnic: dvMirror. Five options are available:

  • Distributed Port Mirroring
  • Remote Mirroring Source
  • Remote Mirroring Destination
  • Encapsulated Remote Mirroring (L3) Source
  • Distributed Port Mirroring (legacy)

Distributed Port Mirroring (SPAN)

The first option (Distributed Port Mirroring or SPAN) allows to replicate a virtual port inside a dvSwitch to another port inside the same dvSwitch; both VMs must run on the same host. In the following example traffic from and to ubuntu3 VM running on esxi1, will be replicated to receiver VM running on esxi1 too:

  • add a “Distributed Port Mirroring” session inside DSwitch0;
  • enable the session;
  • select virtual port facing ubuntu3 eth0 as source;
  • select virtual port facing receiver eth1 as destination;
  • be sure that source and destination VMs run on the same ESXi host.

SPAN Now receiver eth1 vnic will receive a copy of ubuntu3 eth0 traffic.

Remote Mirroring Source and Remote Mirroring Destination (RSPAN)

A remote mirroring session is used when source and destination are running on different ESXi host, dvSwitch or physical switch (if source or destination are physical). A RSPAN session is still a L2 protocol, which means that it cannot traverse L3 devices. In the following example traffic from and to ubuntu1 VM running on esxi2, will be replicated to receiver VM running on esxi1:

  • add a VLAN used for this specific RSPAN session on the physical switch (VLAN 4000 will be used):
vlan 4000
 name RSPAN
 remote-span
  • be sure that both ESXi can receive the VLAN 4000 (802.1q must be used);
  • add a “Remote Mirroring Source” session inside DSwitch0;

  • enable the session;
  • use VLAN 4000 ad “Encapsulation VLAN ID” (it’s the destination VLAN used by RSPAN);
  • enable “Normal I/O on destination ports” (destination is a VLAN and normal switching must be allowed to forward frames);
  • check “Preserve original VLAN”;
  • select virtual port facing ubuntu1 eth0 as source;
  • select all available uplinks;
  • confirm, and be aware that if something goes wrong dvSwitch operation can be quiesced (until timeout/rollback); source_rspan

  • add a “Remote Mirroring Destination” session inside DSwitch0;

  • enable the session;
  • add VLAN 4000 as source;
  • select virtual port facing receiver eth1 as destination; destination_rspan

Now receiver VM can monitor ubuntu1 traffic using eth1 interface. Eth1 is dedicated to monitoring operation and normal I/O is disabled.

If the source of a RSPAN session is a physical device connected to a Cisco switch, a VM can still be used for monitoring:

monitor session 1 source interface Gi1/0/48
monitor session 1 destination remote vlan 4000

If roles are inverted and the monitor appliance is physically attached to a Cisco switch:

monitor session 1 source remote vlan 4000
monitor session 1 destination interface Gi1/0/48

Each RSPAN session should have a dedicated VLAN (or monitor sessions will be mixed), and bandwidth inside RSPAN VLANs should not affect production traffic.

Encapsulated Remote Mirroring (L3) Source (ERSPAN)

An encapsulated remote mirroring session is used when source and destination are running on different L3 networks. In the following example traffic from and to ubuntu1 VM running on esxi2, will be replicated to a desktop PC running Wireshark:

  • add an “Encapsulated Remote Mirroring (L3) Source” session inside DSwitch0;
  • enable the session;
  • select virtual port facing ubuntu3 eth0 as source;
  • add the IP address as destination.

ERSPAN

On the monitor station open Wireshark, and filter for the ESXi IP address where ubuntu1 VM is running (Capture Filter: src host 172.31.30.12, Display Filter: ip.proto == 47).

wireshark

ERSPAN packets are automatically decoded and inner packets are showed. I think this is the best way to analyze VM traffic.

References

Posted on 15 Jul 2014 by Andrea.
  • Gmail icon
  • Twitter icon
  • Facebook icon
  • LinkedIN icon
  • Google+ icon