I’m going to skip the explanations of these things, and jump right to the how to set them up in Proxmox. I am working with Proxmox 1.4, and a Cisco 4506 switch. The Cisco has several vLANs configured, and the Proxmox host has dual Intel NICs on eth0 and eth1.
On the Cisco switch, we need to configure a trunk, and then add ports to that trunk. In my example, there are no existing Port-channels, so the trunk will be Port-channel1.
interface Port-channel1
description *** Proxmox Host PRX01 bitbud.2009.10.10 ***
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
duplex full
spanning-tree portfast trunk
I will be using ports 1 and 2 on Gigabit slot 2 for the network ports:
interface GigabitEthernet2/1
description *** Proxmox Host PRX01 bitbud.2009.10.10 ***
switchport trunk encapsulation dot1q
switchport mode trunk
speed 1000
duplex full
spanning-tree portfast trunk
channel-group 1 mode desirable
interface GigabitEthernet2/2
description *** Proxmox Host PRX01 bitbud.2009.10.10 ***
switchport trunk encapsulation dot1q
switchport mode trunk
speed 1000
duplex full
spanning-tree portfast trunk
channel-group 1 mode desirable
That takes care of the port configuration on the switch.
Now, in Proxmox, I need to edit the trunk information. In Linux, this is called a network bond. We will also configure vLANs and bridges as well. In Proxmox 1.4, you can have an unlimited number of bridges (bridges are what your virtual NICs connect to), so I like to use a bridge per vLAN (makes sense). In my example, my primary vLAN that most systems use is vlan10, and I’ll also configure vlan20, and vlan30. I’ve included some extra information as an example as well.
All of this configuration is stored in /etc/network/interfaces. This is my complete interfaces file:
# standard net config DW 2009.11.12
#
# update ethx cards available
# update bond0 config (which ethx cards to use)
# update each bond0.x per vlan
# and its associated auto vmbrx per vlan
# exception ___ that vmbr0 should use the bond.x vlan
# that you want to use as the admin interface
# network interface settings
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto eth1
iface eth1 inet manual
auto bond0
iface bond0 inet manual
slaves eth0 eth1
bond_miimon 100
# bond_mode active-backup
bond_mode 4
auto bond0.10
iface bond0.4 inet manual
vlan-raw-device bond0
auto bond0.20
iface bond0.5 inet manual
vlan-raw-device bond0
auto bond0.30
iface bond0.11 inet manual
vlan-raw-device bond0
auto vmbr0
iface vmbr0 inet static
address 10.5.56.71
netmask 255.255.254.0
gateway 10.5.57.4
bridge_ports bond0.10
bridge_stp off
bridge_fd 0
auto vmbr20
iface vmbr20 inet manual
bridge_ports bond0.20
bridge_stp off
bridge_fd 0
auto vmbr30
iface vmbr30 inet static
address 10.101.7.1
netmask 255.255.0.0
bridge_ports bond0.30
bridge_stp off
bridge_fd 0
Let me point out a few things in that example:
- If you have additional interfaces, such as eth2, eth3, etc that you want to add to the bond, simply copy the example with the new interface id.
- Add:
auto eth2
iface eth2 inet manual - Change:
slaves eth0 eth1
To:
slaves eth0 eth1 eth2 - You’ll also need to configure your switch port as well
- Add:
- I have a separate vLAN and IP scheme for my SAN. Using Proxmox’s storage management layer, I connect to these from the Hypervisor. You’ll see this on the bridge configuration vmbr30. Note the additional IP and netmask, but no gateway.
- Proxmox likes to see a vmbr0 as the ‘default’ bridge. Since vLAN10 is my ‘main’ vLAN, I set this to the bridge vmbr0, rather than vmbr10. All other bridges are named after the vLAN they represent (vLAN20 is on bridge vmbr20).
On my production systems, I have over 20 vLANs configured, and they work great. Once you have setup the vLANs in the interfaces file, selecting them from the web interface is easy.
Questions?
Hi, and thank you very much for this article. It matches exactly what I’m about to do in my home lab.
Nevertheles, I stil have one question. Talking about your storage/SAN network/vlan, you underline the fact that you don’t set up a gateway configuring IP on the dedicated bridge. Is there a special reason for that, put aside that I suppose that the storage network is a flat strictly isolated ?
If this is the case, I imagine that all machines connected to it must be multi-homed, with at least another leg plugged in, say a management VLAN, on which a gateway is configured (at least for updates)… Am I wrong ?
Thank you in advance for your answer
Bests
There is already an IP and gateway defined on the primary management interface. The additional bridge and IP assigned on the “storage vLAN” is to facilitate direct access (no routing) to the storage vLAN.
Since both IP’s are assigned to the host computer, there can be only one default gateway, in this case the management interface.
It is true, however, that the storage network is segmented and does not have any routes.