home | tech | osdev | misc | code | bookmarks (broken) | contact | README


Xen setup NetBSD Dom0 and DomU

Last update on this page was on 2018-10-15.

This is a page I will always try to keep up to date with latest NetBSD and Xen versions. On this version I'm using NetBSD 8.0 and Xen 4.8.

This page is a summary of my experiences with NetBSD and Xen since NetBSD 3.1 and Xen 2.0, although it is a very simple experience, as the user of basic features only. Back to that days of NetBSD 3.1 we needed to install GRUB bootloader to be able to boot Xen. We don't need it anymore, since NetBSD comes with its own good bootloader since version 5.0.

A very useful, more complete and probably up to date reference where I got much of the information is the NetBSD Xen HowTo.

Installing Xen

After installing NetBSD in your computer (with a common kernel like GENERIC), you need to download and compile the xenkernel from pkgsrc:

# cd /tmp
# ftp http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc.tar.gz
# tar zxf pkgsrc.tar.gz -C /usr
# cd /usr/pkgsrc/sysutils/xenkernel48
# make install

Copy the kernel to the root directory:

# cp /usr/pkg/xen48-kernel/xen.gz /

We now have the xen kernel built in /usr/pkg/xen48-kernel/xen.gz that we will use later. Now, build xentools:

# cd /usr/pkgsrc/sysutils/xentools48
# make install

After the package was installed, created special Xen devices in /dev:

# cd /dev/
# ./MAKEDEV xen

Copy daemon scripts in the right place and configure rc.conf to lauch xencommons at the system startup:

# cp /usr/pkg/share/examples/rc.d/xen* /etc/rc.d/
# echo 'xencommons=YES' >> /etc/rc.conf

Using the right kernel

The GENERIC kernel (the one that comes by default) cannot be used with Xen. You need to get another kernel. Just download it:

# cd /
# ftp ftp.netbsd.org/pub/NetBSD/NetBSD-8.0/amd64/binary/kernel/netbsd-XEN3_DOM0.gz

Note

It is important to note that if you are going to load kernel modules with modload(8), you'll have to load the ones specially built for Xen, because of ABI incompatibilities.

Configure bootloader

Since NetBSD 5.0, we can use NetBSD's boot loader (you can optionally use GRUB if you prefer). What we need is to edit the /boot.cfg file, which is a very simple and plain text file. The man page of this file (boot.cfg(5)) has some examples on how to setup it to load the xen kernel and the NetBSD Dom0 kernel to work with Xen.

What we need to do is just to add the following menu entry:

menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=pc

And change the default parameter to where we just put this menu entry.

Reboot, select the right option at the boot menu and see the magic happening!

You can use xl command to get dom0 information like xl dmesg or xl info.

Network configuration

After having Xen up and running, you'll have to create a bridge to connect the interfaces of the virtual machine. I like to think a bridge as a switch internal to the host.

To create a bridge, type:

# ifconfig bridge0 create
# ifconfig bridge0 up

Then, add physical interfaces to that switch, so your virtual machines can talk to the external world. For instance, if your physical network interface is re0, do:

# brconfig bridge0 add re0

This configuration will vanish at the next reboot. So make them persistent. Add the following configuration to /etc/ifconfig.bridge0:

create
up
!brconfig $int add re0

Troubleshooting

Error when bridge is not created

Errors like:

libxl: error: libxl_dm.c:2189:device_model_spawn_outcome: domain 1 device model: spawn failed (rc=-3)
libxl: error: libxl_create.c:1504:domcreate_devmodel_started: device model did not start: -3
libxl: error: libxl_dm.c:2303:kill_device_model: Device Model already exited
libxl: error: libxl.c:1575:libxl__destroy_domid: non-existant domain 1
libxl: error: libxl.c:1534:domain_destroy_callback: unable to destroy guest with domid 1
libxl: error: libxl.c:1463:domain_destroy_cb: destruction of domain 1 failed

Are not very helpful, but can mean that you did not create your bridge device. Check Network configuration for more information.