Open vSwitch (OVS) Deep-Dive

Overview

Open vSwitch is a production-quality, multilayer virtual switch used as the dataplane in ML2/OVS Neutron deployments. In an OpenStack compute or network node, OVS is managed by the neutron-openvswitch-agent, which translates Neutron port/network state into OVS flow rules.

OVS consists of:

• ovs-vswitchd: the switch daemon; programs the kernel datapath
• ovsdb-server: stores configuration (bridges, ports, flow tables) in the OVS database
• Kernel datapath module (openvswitch.ko): high-performance packet forwarding in kernel space
• Userspace datapath: used with DPDK for even higher performance

Bridge Architecture

A typical compute node runs three OVS bridges:

VM (qemu tap)
  │ tapXXXXXX
  ▼
br-int  (integration bridge)
  │  patch-tun or patch-int
  ▼
br-tun  (tunnel bridge)
  │  vxlan-<remote-vtep>  (one per remote compute node)
  ▼
Overlay network (UDP encapsulation)

br-int
  │  patch-ext (or phy-br-ex veth pair)
  ▼
br-ex  (provider/external bridge)
  │  eth1 (physical uplink)
  ▼
Physical network

br-int — Integration Bridge

br-int is the central switching fabric for a compute node. Every VM port, router port, DHCP port, and metadata port plugs into br-int. It operates at the local VLAN level — each Neutron network is assigned a local VLAN ID that is only significant within this node.

Key OVS port types on br-int:

Port	Type	Purpose
tapXXXXXX	internal/tap	VM network interface
qr-XXXXXXXX	internal	Router interface (L3 agent)
qdhcp-XXXXXXXX	internal	DHCP tap
int-br-ex	patch	Patch to br-ex (for provider networks)
patch-tun	patch	Patch to br-tun (for overlay networks)

Flow tables on br-int (simplified):

Table 0  (Ingress classification)
  priority=65535, in_port=<local>, actions=NORMAL          # from local ports
  priority=1, actions=drop

Table 23 (Ingress from tunnel bridge)
  dl_vlan=<tun-vlan>, actions=mod_vlan_vid:<local-vlan>,NORMAL

Table 24 (Ingress from external bridge — for provider networks)
  in_port=int-br-ex, dl_vlan=<physnet-vlan>, actions=mod_vlan_vid:<local-vlan>,NORMAL

Table 60 (L2 forwarding)
  priority=2, dl_vlan=<local-vlan>, dl_dst=<mac>, actions=output:<port>
  priority=1, dl_vlan=<local-vlan>, actions=FLOOD

br-tun — Tunnel Bridge

br-tun handles VXLAN (or GRE) encapsulation and decapsulation between compute nodes. Each remote compute node has a dedicated tunnel port:

vxlan-0a000001  →  10.0.0.1 (compute01 VTEP)
vxlan-0a000002  →  10.0.0.2 (compute02 VTEP)

Flow tables on br-tun:

Table 0  (Ingress classification)
  priority=1, in_port=patch-int, actions=goto_table:1     # from br-int
  priority=1, in_port=<tun-port>, actions=goto_table:4    # from remote

Table 1  (Tunnel encap — from br-int outbound)
  dl_dst=ff:ff:ff:ff:ff:ff, actions=goto_table:2          # broadcast → flood
  dl_dst=fa:16:3e:xx:xx:xx, actions=goto_table:20         # unicast → lookup

Table 2  (Flood — with l2population disabled)
  dl_vlan=<local-vlan>, actions=strip_vlan,set_tunnel:<vni>,
           output:vxlan-0a000001,output:vxlan-0a000002

Table 20 (Unicast — with l2population enabled)
  priority=2, dl_vlan=<local-vlan>, dl_dst=fa:16:3e:ab:cd:ef,
    actions=strip_vlan,set_tunnel:<vni>,output:vxlan-0a000002

Table 4  (Ingress from tunnel — decap)
  tun_id=<vni>, actions=mod_vlan_vid:<local-vlan>,goto_table:10

Table 10 (Ingress security / learn)
  actions=learn(table=20,...),output:patch-int

br-ex — Provider/External Bridge

br-ex connects OVS to physical network interfaces for provider networks and external gateway traffic. It has a physical uplink port (e.g., eth1) and a patch port to br-int.

For provider networks, OVS agent configures bridge_mappings = physnet1:br-ex and programs flows to translate between the physical VLAN and the local internal VLAN.

# Show br-ex configuration
ovs-vsctl show
ovs-ofctl dump-flows br-ex

OVS Agent Operation

The neutron-openvswitch-agent runs on every compute/network node. Its main loop:

1. Setup: creates bridges, wires them together with patch ports, programs base flows
2. Polling: uses ovsdb-monitor (or periodic polling) to detect new/removed ports on br-int
3. Port processing: for each new port, looks up its Neutron port record via RPC get_devices_details_list
4. Flow programming: writes OVS flows for the port's local VLAN, security group rules, DHCP/ARP rules
5. RPC handling: listens for port_update, network_update, security_group_rules_for_devices from neutron-server
6. Heartbeat: calls report_state every 30 s

Key configuration options in openvswitch_agent.ini:

[ovs]
local_ip = 10.0.0.11          # VTEP IP for VXLAN tunnels
bridge_mappings = physnet1:br-ex
tunnel_types = vxlan
ovsdb_connection = tcp:127.0.0.1:6640  # OVSDB server

[agent]
l2_population = True          # Enable l2population mechanism driver optimization
arp_responder = True          # Answer ARP locally without flooding
prevent_arp_spoofing = True   # Drop ARP with spoofed source IPs
polling_interval = 2          # Seconds between port scan cycles
minimize_polling = True       # Use OVSDB monitor instead of polling

[securitygroup]
firewall_driver = openvswitch  # OVS-native conntrack firewall (recommended)
# firewall_driver = iptables_hybrid  # Legacy iptables-based approach

Security Group Firewall Drivers

OVS Firewall Driver (Recommended)

Security group rules are translated into OVS flows with conntrack (ct) actions. No Linux bridges or iptables. Works entirely in the kernel OVS datapath.

Flow table layout for OVS firewall:

Table	Purpose
71	Ingress drop (default deny ingress to unknown ports)
72	Accept established inbound traffic (ct_state=+est)
73	Ingress rules (match security group allow rules)
81	Egress drop (default deny egress from unknown ports)
82	Accept established outbound traffic
83	Egress rules

Example flows for a port with tcp:80 ingress rule:

table=73, priority=77, ct_state=+new, tcp, nw_dst=192.168.1.50, tp_dst=80
  actions=ct(commit,zone=<port-zone>,table=0)

table=72, priority=77, ct_state=+est-rel-rpl,
  actions=output:<port>

table=71, priority=10, actions=drop

iptables Hybrid Driver (Legacy)

Each VM port has: - A Linux bridge qbrXXXXXX between the tap device and br-int - iptables FORWARD chain rules applied to the bridge - conntrack for stateful matching

This approach is less efficient because every packet must traverse iptables in addition to OVS. Use the OVS firewall driver for new deployments.

DPDK Support

OVS-DPDK bypasses the kernel entirely using DPDK poll-mode drivers (PMDs) for line-rate performance.

Requirements: - CPUs with hugepages enabled (default_hugepagesz=1G hugepagesz=1G hugepages=8 in kernel cmdline) - NUMA-aware memory allocation - DPDK-compatible NIC (Intel i40e, mlx5, etc.) - ovs-vswitchd built with DPDK support (ovs-vswitchd --version shows DPDK version)

Configure hugepages for OVS-DPDK:

# /etc/default/grub
GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G hugepages=8 iommu=pt intel_iommu=on"

# /etc/dpdk.conf (or equivalent)
EAL_ARGS="-l 2,3,4,5 --socket-mem 2048,2048"

Configure OVS for DPDK:

ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true
ovs-vsctl set Open_vSwitch . other_config:dpdk-socket-mem="2048,2048"
ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x1c  # PMD threads on cores 2,3,4

# Create DPDK bridge
ovs-vsctl add-br br-int
ovs-vsctl set Bridge br-int datapath_type=netdev

# Add DPDK physical port
ovs-vsctl add-port br-int dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:01:00.0

# Add vhost-user port for VMs
ovs-vsctl add-port br-int vhost-user0 \
  -- set Interface vhost-user0 type=dpdkvhostuserclient \
     options:vhost-server-path=/var/run/openvswitch/vhost-user0

DPDK bond for link aggregation:

ovs-vsctl add-bond br-ex dpdkbond dpdk0 dpdk1 \
  bond_mode=active-backup \
  -- set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:01:00.0 \
  -- set Interface dpdk1 type=dpdk options:dpdk-devargs=0000:01:00.1

Nova virt driver configuration for vhost-user ports:

# nova.conf
[DEFAULT]
cpu_shared_set = 0,1
cpu_dedicated_set = 2,3,4,5

[libvirt]
virt_type = kvm

Diagnostic Commands

Show Bridge and Port Configuration

# Show all bridges, ports, and interfaces
ovs-vsctl show

# List all bridges
ovs-vsctl list-br

# List ports on a specific bridge
ovs-vsctl list-ports br-int

# Show detailed interface record
ovs-vsctl list Interface tapabcdef12

Inspect Flow Tables

# Dump all flows on br-int (human-readable)
ovs-ofctl dump-flows br-int

# Dump flows on br-tun sorted by table and priority
ovs-ofctl dump-flows br-tun --sort

# Dump flows matching a specific MAC
ovs-ofctl dump-flows br-int "dl_dst=fa:16:3e:ab:cd:ef"

# Dump flows for a specific table
ovs-ofctl dump-flows br-int "table=0"

# Dump flows with byte/packet counters
ovs-ofctl dump-flows -O OpenFlow13 br-int

Inspect the Forwarding Database

# Show MAC → port mappings learned by OVS
ovs-appctl fdb/show br-int

# Show VXLAN tunnel endpoints
ovs-appctl tnl/neigh/show

# Show all tunnel ports and their remote IPs
ovs-appctl dpif/show

Conntrack Inspection

# Show conntrack table entries
ovs-appctl dpctl/dump-conntrack

# Show conntrack entries for a specific zone
ovs-appctl dpctl/dump-conntrack "zone=<port-zone>"

Packet Tracing

# Trace a packet through OVS (simulated)
ovs-appctl ofproto/trace br-int \
  "in_port=<port-number>,dl_src=fa:16:3e:11:22:33,dl_dst=fa:16:3e:44:55:66,dl_type=0x0800,nw_src=192.168.1.10,nw_dst=192.168.1.20"

OVSDB Queries

# Show all OVS configuration (raw)
ovsdb-client dump

# Monitor OVSDB changes in real time
ovsdb-client monitor Open_vSwitch

# Show Port table
ovs-vsctl list Port

# Show Interface table with statistics
ovs-vsctl list Interface

Performance Inspection

# Show PMD thread CPU usage (DPDK)
ovs-appctl dpif-netdev/pmd-stats-show

# Show PMD thread RX/TX queues
ovs-appctl dpif-netdev/pmd-rxq-show

# Show datapath statistics
ovs-dpctl show
ovs-dpctl dump-flows

Common Issues and Fixes

Symptom	Likely Cause	Fix
Port stuck in BUILD status	OVS agent not running or not receiving RPC	Check neutron-openvswitch-agent logs; verify RabbitMQ connectivity
VM cannot reach another VM on same compute node	br-int flows missing for local VLAN	Restart OVS agent; check ovs-ofctl dump-flows br-int table=0
VM cannot reach VM on another compute node	Tunnel not established or VNI mismatch	Check ovs-vsctl show for tunnel ports; verify local_ip matches
Security group rules not applied	OVS firewall driver misconfigured or iptables hybrid used	Check firewall_driver in agent config; inspect flows in tables 71-83
ARP flooding causing performance issues	l2population or arp_responder disabled	Enable both in openvswitch_agent.ini; ensure l2population in mechanism_drivers