Virtualization and Host Networking

Synopsis #

Configure OpenBSD virtual-machine networking with vmd(8) and vmctl(8) : Layer-2 bridging with bridge(4) , routed segments using vether(4) , and per-VM segmentation with pf.conf(5) anchors. Virtual NICs use tap(4) . Configuration for the hypervisor lives in vm.conf(5) . Use these patterns to attach guests to existing VLANs, to provide isolated routed networks with NAT, and to enforce least-privilege network policies per VM.

Design Considerations #

  • Attachment model.
    • Bridged: VMs appear on an existing L2 segment. Use when DHCP or broadcast discovery must reach guests.
    • Routed (vether): VMs reside behind a virtual L3 interface; the host routes/NATs. Use for isolation and tight control.
  • Failure domains. Avoid extending large L2 domains unnecessarily. Prefer routed segments with vether(4) at scale.
  • VLANs. When bridging into a trunk, add the parent interface that carries VLAN tags to the bridge so tags pass transparently; if you need a single VLAN, bridge its vlan(4) interface instead.
  • MAC accounting. Upstream switches may limit learned MACs. Bridging many VMs through one physical port can exceed limits. Coordinate with switching policy or prefer routed attachment.
  • Security posture. Treat each VM as untrusted. Use PF anchors to apply explicit per-VM policies, and default-deny inter-VM traffic even on the same host.
  • Operations. Keep VM images and network definitions under version control. Enable verbose logging during initial rollout. Apply a consistent naming scheme (for example, switch "lan", switch "dmz").

Configuration #

The host has em0 (WAN/uplink) and em1 (LAN or trunk). The examples show:

  1. Bridging VMs into an existing LAN (or trunk).
  2. Building an isolated routed network with vether0 and optional NAT.
  3. Enforcing per-VM policy with PF anchors.

1) Base hypervisor setup #

Create a bridge for guest attachment, define a vmd switch, and enable the hypervisor.

doas sh <<'ROOT'
cat > /etc/hostname.bridge0 <<'EOF'
add em1
up
EOF
  # Bridge carries em1 toward the LAN/trunk; VMs will join via tap(4)

sh /etc/netstart bridge0
  # Activate the bridge

rcctl enable vmd
  # Start vmd(8) at boot
rcctl start vmd
  # Launch now

install -d -o root -g wheel /var/vm
  # Ensure a place for VM disks
ROOT

Define a vmd switch and a simple VM in vm.conf(5) . The switch ties VMs to bridge0.

## /etc/vm.conf — example switch and guest
switch "lan" {
    interface bridge0
}

vm "web01" {
    memory 1024M
    disk "/var/vm/web01.img"
    interface { switch "lan" }
}

Create the disk and start the VM (install media omitted here; adjust to your workflow).

doas vmctl create -s 20G /var/vm/web01.img
  # Create a sparse disk
doas vmctl reload
  # Load the new definition after creating its disk
doas vmctl start web01
  # Start the VM; vmd(8) attaches a tap(4) to bridge0
doas vmctl status
  # List VM IDs and status; inspect explicit vm.conf tap assignments and bridge membership

2) Bridged guests on an existing LAN (or VLAN trunk) #

With bridge0 carrying em1 and the vmd switch targeting bridge0, guests appear on the physical LAN (or all VLANs traversing em1 if it is a trunk). If you must expose a single VLAN to guests, bridge its VLAN interface instead:

## /etc/hostname.vlan10 — example single-VLAN bridge member
parent em1
vnetid 10
up

doas sh /etc/netstart vlan10
doas ifconfig bridge0 delete em1
  # Remove the parent from the bridge when you only want VLAN 10
doas ifconfig bridge0 add vlan10
  # Add just the VLAN subinterface; guests see only VLAN 10
doas ifconfig bridge0
  # Verify members

Also replace add em1 with add vlan10 in /etc/hostname.bridge0; otherwise a reboot restores the original bridge membership.

Provide DHCP or static addressing from your existing infrastructure. To serve DHCP locally to bridged guests, configure dhcpd(8) on an addressed Ethernet member such as vether0; a bridge interface itself does not carry IP addresses (see the Services chapter).

3) Routed guests with vether(4) and optional NAT #

Create an isolated L3 segment for guests behind vether0. The host routes between vether0 and the uplink.

## /etc/hostname.vether0 — routed guest segment
inet 10.50.0.1 255.255.255.0
up

Create a dedicated bridge for the routed segment and join guest taps to it dynamically (vmd will add taps when guests start).

## /etc/hostname.bridge1 — bridge for the routed segment
add vether0
up

Point a second vmd switch at bridge1 and attach VMs to it.

## /etc/vm.conf — excerpt adding a routed switch and VM
switch "isolated" {
    interface bridge1
}

vm "db01" {
    memory 2048M
    disk "/var/vm/db01.img"
    interface { switch "isolated" }
}

Enable forwarding and, if desired, NAT to the uplink using pf.conf(5) . Manage PF at runtime with pfctl(8) .

## /etc/pf.conf — minimal routed segment with NAT

set skip on lo

wan      = "em0"
isol_net = "10.50.0.0/24"

# NAT the isolated guests out the uplink
match out on $wan from $isol_net to any nat-to ($wan)

block all
pass in on { vether0, tap } from $isol_net to any keep state
pass out on $wan proto { tcp udp icmp } from ($wan) to any keep state

doas sh <<'ROOT'
printf '%s\n' 'net.inet.ip.forwarding=1' >> /etc/sysctl.conf
  # Persist IPv4 forwarding
sysctl net.inet.ip.forwarding=1
  # Apply immediately
pfctl -f /etc/pf.conf
  # Load PF policy
ROOT

Serve addresses to routed guests from the host with dhcpd(8) ) bound to vether0 (see the Services chapter), or configure static addresses in 10.50.0.0/24.

4) Micro-segmentation with PF anchors (per-VM policy) #

The following fragments restrict traffic leaving the host through each guest tap. A tap usually has no host IP address, so use the guest address rather than (tapN). Add separate ingress rules for permitted guest-originated traffic, anti-spoofing checks, and required DHCP/IPv6 control traffic. Integrate these fragments with bridge filtering and test both allowed and denied inter-guest flows before relying on isolation. Fix tap assignments in vm.conf (for example, interface tap3 { switch "isolated" }) when rules depend on a specific tap number.

Main policy:

## /etc/pf.conf — anchor framework (excerpt)

set skip on lo

# Load all per-VM policies from a dedicated directory
block all
anchor "vm/*"
load anchor "vm/web01" from "/etc/pf.vm/web01.conf"
load anchor "vm/db01" from "/etc/pf.vm/db01.conf"

# Firewall-originated outbound traffic
pass out on egress proto { tcp udp icmp } keep state

Per-VM policy files:

## /etc/pf.vm/web01.conf — allow HTTP/HTTPS from anywhere to web01, SSH from admin subnet
# Attach rules to the VM's tap interface (substitute the actual tapN)
intf = "tap3"

# Inbound public service
pass out quick on $intf proto tcp to 10.50.0.20 port { 80, 443 } keep state

# Operations from admin LAN only
pass out quick on $intf proto tcp from 10.10.0.0/24 to 10.50.0.20 port 22 keep state
block out quick on $intf

## /etc/pf.vm/db01.conf — allow MySQL only from web01; block everything else inbound
intf = "tap4"

# Only the web tier may reach the database
pass out quick on $intf proto tcp from 10.50.0.20 to 10.50.0.30 port 3306 keep state
block out quick on $intf

Load and verify:

doas pfctl -f /etc/pf.conf
  # Reload main policy and anchors
doas pfctl -a 'vm/*' -sr
  # Show rules within the vm/* anchors

Maintain a mapping of VM names to tap interfaces (from explicit vm.conf tap assignments and ifconfig bridgeN membership) and reflect it in anchor files, or template anchors during VM deployment.

Verification #

vmctl status
  # VM names and IDs; inspect explicit tap assignments and bridge membership

ifconfig bridge0
  # Expect em1 and one or more tap(4) members for bridged guests
ifconfig bridge1
  # Expect vether0 and guest taps for the routed segment
  • Routing and NAT:
netstat -rn
  # Verify a connected route for 10.50.0.0/24 via vether0
doas pfctl -vvsr | egrep 'match out .* nat-to|\banchor vm/'
  # Confirm NAT rule and anchor loads

doas tcpdump -ni vether0
  # Guest traffic inside the routed segment
doas tcpdump -ni em1 vlan 10
  # VLAN-tagged frames if bridging a single VLAN interface

Troubleshooting #

  • Guests do not get DHCP on bridged networks. Confirm the correct interface is in bridge0 (em1 for the trunk or the specific vlan(4) for a single VLAN). Ensure the upstream DHCP server can reach the segment and that PF is not blocking BOOTP.
  • No connectivity from routed guests. Verify net.inet.ip.forwarding=1, that vether0 has the expected address, and that PF loaded the NAT match rule. Use tcpdump on $wan and vether0 to trace packets.
  • Anchors not applied. Ensure the anchor and load anchor lines exist in the main policy and that per-VM files compile. Inspect with pfctl -a 'vm/*' -sr.
  • VM attached to the wrong network. Check the switch stanza in vm.conf and the interface { switch "…" } block in the VM definition. Reload vmd or restart the VM if you changed the attachment.
  • MAC limit exceeded on upstream switch. Reduce bridged guests on that port or migrate the workload to the routed vether0 pattern.
  • VLAN tagging not preserved. When passing multiple VLANs, bridge the parent physical interface (for example, em1) instead of a single vlan(4) child. To expose only one VLAN, bridge just that vlan(4) interface.

See Also #

Report a bug