Setting up a virtual network with KVM/QEMU and VDE

February 25, 2011 |

I've spent a little time learning how to use debootstrap (via vmbuilder) and VDE (virtual distributed ethernet) to create a virtual network. VDE sounds really neat and has tools (virtual switches, hubs, cables and so on) to create any sort of network configuration you want.

Screenshot

(That's two VMs on the top and the vdeswitch running in the host on the bottom).

Overview

I wish to create two VMs and set them up behind a switch which uplinks via the host to whatever network it can reach (in my home use-case, the internets via another NAT).

Create the VMs

After playing around with debootstrap a little, I found Ubuntu's VMBuilder which a GSoC project has been converting for Debian (works with lenny). You may download the VMBuilder for Debian via git:

git clone git://git.debian.org/git/pkg-escience/vmbuilder.git

This has a couple problems for me: it tries to use grub from the host (lenny uses grub1, but I'm running wheezy so I have grub2) and it uses vol_id which doesn't exist. The first problem has already been solved by this commit (web). You need to mess with it a little as this applies to plugins/ubuntu/distro.py and you want plugins/debian/distro.py instead (just edit the paths in the patch and it applies fine for me with patch -p0).

The second problem needs a tiny patch:

diff --git a/VMBuilder/disk.py b/VMBuilder/disk.py
index 53878bf..e5ed58a 100644
--- a/VMBuilder/disk.py
+++ b/VMBuilder/disk.py
@@ -292,7 +292,11 @@ class Filesystem(object):
         if not self.dummy:
             cmd = self.mkfs_fstype() + [self.filename]
             run_cmd(*cmd)
-            self.uuid = run_cmd('vol_id', '--uuid', self.filename).rstrip()
+            if False:
+                self.uuid = run_cmd('vol_id', '--uuid', self.filename).rstrip()
+            else:
+                ## vol_id is gone in recent Debian releases
+                self.uuid = run_cmd('blkid', '-s', 'UUID', '-o', 'value', self.filename).rstrip()

     def mkfs_fstype(self):
         if self.vm.suite in ['dapper', 'edgy', 'feisty', 'gutsy', 'etch']:

Now we can use VMBuilder to make the two virtual machines we want:

      ./vmbuilder kvm debian --suite lenny --user mike --pass seekrit --rootpass seekrit --hostname atlantis
      mv debian-kvm atlantis-debian-lenny-kvm
      ./vmbuilder kvm debian --suite lenny --user mike --pass seekrit --rootpass seekrit --hostname lemuria
      mv debian-kvm lemuria-debian-lenny-kvm

So now we have two VMs in atlantis-debian-lenny-kvm and lemuria-debian-lenny-kvm directories (actually they're just images, but VMBuilder puts them in a directory to include a run.sh script to launch it)

Virtual Distributed Ethernet

Screenshot

Next, we will configure a virtual network with a switch connected to an uplink providing DHCP and NAT so that we can get out to Teh Intarnets from the VMs. In the above screenshot, there are two VM windows in the upper left and upper right and the vde switch running in a shell on the host in the lower window (that's ratpoison with a horizontal and vertical split).

First, set up the switch (/tmp/vde_switch_0 is both the "name" of the switch and its control socket; you pass this to other commands, but it can be whatever path you feel like):

mike@pangea:~$ vde_switch -s /tmp/vde_switch_0

vde$ port/allprint
0000 DATA END WITH '.'
.
1000 Success

vde$

Next, set up slirpvde (``It acts like a networking router connected to a vde_switch and provides connectivity from the host where it is running to virtual machines inside the virtual network.''). This provides NAT and DHCP (which is why we're starting this up before the VMs). You could do this other ways (e.g. maybe you want DHCP but specifically don't want to provide access to the "real" network), but this provides just what I need.

mike@pangea:~$ slirpvde -s /tmp/vde_switch_0 --dhcp
IP address of your DNS(s): 192.168.XXX.YYY

I have a caching DNS running locally on the router so your output will likely be different. Then, we want to start up the VMs and configure them. I found that with these particular settings, the VMs decided to use eth2 instead of eth0 for reasons I don't understand. So, I had to change /etc/networking/interfaces to reflect this. Note the explicit MAC addresses: kvm/qemu seems to derive an automatic MAC "somehow" (I think it depends on the host's MAC in some way) and these are always the same for both VMs. In any case, use whatever you like, but they should be different.

kvm -net nic,macaddr=00:23:df:ac:36:aa -net vde,sock=/tmp/vde_switch_0 -m 128 -smp 1 -drive file=/home/mike/src/vmbuilder/lemuria-debian-lenny-kvm/disk0.qcow2
kvm -net nic,macaddr=00:23:df:ac:36:bb -net vde,sock=/tmp/vde_switch_0 -m 128 -smp 1 -drive file=/home/mike/src/vmbuilder/atlantis-debian-lenny-kvm/disk0.qcow2

Great! Now, you should have eth2 (or whatever your system decided to call it) up on both VMs and should be able to ping each other (see screenshot). Additionally, both VMs can get out to the "real" internets via vdeslirp so things like apt-get update will work.


name:
email:
URL:
Private mail only; don't publish anywhere.