The Nemo Unification and Vanity Naming component of project Clearview has integrated into OpenSolaris build 83, which (among other things) allows administrators to give meaningful names to network datalink interfaces, including VLAN interfaces. I thought I'd share how I used this feature on one of our lab routers here in Sun.
The system has four Ethernet NICs, but needs to be the router for 8 separate lab subnets. The aggregate bandwidth of four Gigabit pipes is plenty for all of the lab subnets combined, so it wasn't really worthwhile to go and add four more NICs to the system (plus, that's not really scalable). Instead, I created a single link aggregation (802.3ad) including all four Ethernet links, and created individual tagged VLAN interfaces (one for each of the 8 subnets) on top of this aggregation.
Step by step, here's what I did. Keep in mind that this is done using a nightly build of OpenSolaris from after January 24th 2008. Here was the list of datalinks on the system before I started changing things (bonus points for anyone who can tell me what kind of system I'm doing this on based on the devices listed below) :-) :
bash-3.2# dladm show-link
LINK CLASS MTU STATE OVER
nge0 phys 1500 up --
nge1 phys 1500 up --
e1000g0 phys 1500 up --
e1000g1 phys 1500 up --
bash-3.2# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
nge0 Ethernet up 1000Mb full nge0
nge1 Ethernet up 1000Mb full nge1
e1000g0 Ethernet up 1000Mb full e1000g0
e1000g1 Ethernet up 1000Mb full e1000g1
First, I unplumbed all IP interfaces on each of these links by issuing appropriate "ifconfig <intf> unplumb" commands. This was necessary since renaming datalinks requires that no IP interfaces be plumbed above them. I then gave each of these interfaces more generic names. The benefit of doing this is that if we replace the Ethernet cards in the future with cards of a different chip set, we won't have to change the interface names associated with that card (one of the big benefits of Clearview UV vanity naming).
bash-3.2# dladm rename-link nge0 eth0
bash-3.2# dladm rename-link nge1 eth1
bash-3.2# dladm rename-link e1000g0 eth2
bash-3.2# dladm rename-link e1000g1 eth3
LINK CLASS MTU STATE OVER
eth0 phys 1500 up --
eth1 phys 1500 up --
eth2 phys 1500 up --
eth3 phys 1500 up --
bash-3.2# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
eth0 Ethernet up 1000Mb full nge0
eth1 Ethernet up 1000Mb full nge1
eth2 Ethernet up 1000Mb full e1000g0
eth3 Ethernet up 1000Mb full e1000g1
Then I created a link aggregation using these four Ethernet links:
bash-3.2# dladm create-aggr -P L2,L3 -l eth0 -l eth1 -l eth2 -l eth3 default0
I named the link "default0" because this is the main untagged subnet for the lab network, and the network to which the default route points. Now the set of links looks like:
bash-3.2# dladm show-link
LINK CLASS MTU STATE OVER
eth0 phys 1500 up --
eth1 phys 1500 up --
eth2 phys 1500 up --
eth3 phys 1500 up --
default0 aggr 1500 up eth0 eth1 eth2 eth3
The next step was to create the VLAN links on top of this aggregation. Our lab subnets have a color-coded naming scheme, which I used when naming the VLAN links. This is convenient when diagnosing network problems with particular systems, as our DNS naming uses a paralell scheme. For example, if a system's hostname is blue-98, I know to do my network snooping on the "blue" link. Creating the VLAN links was as simple as:
bash-3.2# dladm create-vlan -v 2 -l default0 orange0
bash-3.2# dladm create-vlan -v 3 -l default0 green0
bash-3.2# dladm create-vlan -v 4 -l default0 blue0
bash-3.2# dladm create-vlan -v 5 -l default0 white0
bash-3.2# dladm create-vlan -v 6 -l default0 yellow0
bash-3.2# dladm create-vlan -v 7 -l default0 red0
bash-3.2# dladm create-vlan -v 8 -l default0 cyan0
There is now one link for each subnet in the lab (one untagged link, and seven tagged VLAN links).
bash-3.2# dladm show-link
LINK CLASS MTU STATE OVER
eth0 phys 1500 up --
eth1 phys 1500 up --
eth2 phys 1500 up --
eth3 phys 1500 up --
default0 aggr 1500 up eth0 eth1 eth2 eth3
orange0 vlan 1500 up default0
green0 vlan 1500 up default0
blue0 vlan 1500 up default0
white0 vlan 1500 up default0
yellow0 vlan 1500 up default0
red0 vlan 1500 up default0
cyan0 vlan 1500 up default0
bash-3.2# dladm show-vlan
LINK VID OVER FLAGS
orange0 2 default0 -----
green0 3 default0 -----
blue0 4 default0 -----
white0 5 default0 -----
yellow0 6 default0 -----
red0 7 default0 -----
cyan0 8 default0 -----
I then plumbed IP interfaces in each subnet. For example:
bash-3.2# ifconfig orange0 plumb ...
bash-3.2# ifconfig green0 plumb ...
...
Configuring this router also involved configuring IPv4 dynamic routing and forwarding, IPv6 dynamic routing and forwarding, etc. All of these latter steps involved placing the network interface names in some sort of persistent configuration (like /etc/hostname.<intf>, /etc/inet/ndpd.conf, and IP filter rules to name a few). This is where giving meaningful names to network interfaces has the most value. With all of these interface names in various configuration files, we don't want to ever have to go and reconfigure all of those things if the underlying hardware of the system were to change from under them. Before Clearview UV's vanity naming feature, a VLAN interface above the e1000g1 interface would look something like e1000g80001 (for VLAN tag 8), thanks to the moldy "VLAN PPA-hack". This is ridiculous enough as an interface name, but what happens when I replace my e1000g1 card with a Broadcom card which has a device name of bge0? I need to go fetch every piece of configuration on the system that made reference to e1000g1 and e1000g8001, and change everything to bge0 and bge8000.
With Clearview UV's vanity naming feature I could have named the link something meaningful like "private1", and assigned the newly added bge0 card that same name (using the dladm rename-link command I showcased above) to keep all of my network configuration intact.
