Xen Live Migration with DRBD on Debian Jessie: Part 1 – Dom0

We have two Debian Jessie servers which we are going to configure as Xen hosts (domain 0) and implement live migration via DRBD. The main goal of this project is to test Xen live migration.

Xen Project Hypervisor

When we say “Xen” here, we reffer to the Xen Project Hypervisor. Don’t get confused with Citrix’s XenServer.

The Xen Project Hypervisor is an open-source type-1 or baremetal hypervisor. Guest virtual machines running on a Xen Project Hypervisor are known as “domains” (domU) where a special domain known as dom0, also called a domain zero, is responsible for controlling the hypervisor and starting other guest operating systems.

Xen hypervisor supports two primary types of virtualisation: para-virtualisation (PV) and hardware virtual machine (HVM) also known as “full virtualisation”.

PV does not require virtualisation extensions from the host CPU. Paravirtualised operating systems are aware that they are being virtualised and as such don’t require virtual “hardware” devices, instead they make special calls to the hypervisor that allow them to access CPUs, storage and network resources.

In contrast HVM requires Intel VT or AMD-V hardware extensions. HVM guests need not be modified as the hypervisor will create a fully virtual set of hardware devices for this machine that resemble a physical x86 computer. This emulation requires much more overhead than the paravirtualisation approach but allows unmodified guest operating systems like Microsoft Windows to run on top of the hypervisor.

Dom0 Installation and Configuration

We are going to use Debian Jessie as our domain 0 system. We have two servers ready, named xen01 and xen02. The servers have 2 CPU cores and 2GB of RAM each.

Both servers have two network interfaces, eth0 used for DRBD communication (non-routable vlan on, and eth1 connected to a LAN (

There is a DHCP/DNS server on the LAN that issues static DHCP leases depending on a client’s MAC address.

The convention followed in the article is that [ALL]# denotes a command that needs to be run on both Xen nodes.

[xen01]# hostnamectl set-hostname xen01.hl.local
[xen02]# hostnamectl set-hostname xen02.hl.local
[ALL]# cat /etc/debian_version

For those who do not have access to (spare) physical hardware to install Xen, VirtualBox guests can be used as dom0.

One thing to note is that VirtualBox does not have the ability to passthrough the VT-x features to a guest, which is necessary to allow nested HVM guests. Therefore only paravirtualised domains can be used as they don’t require virtualisation extensions from the host CPU.

Xen Packages

[ALL]# apt-get update
[ALL]# apt-get install xen-linux-system-amd64 bridge-utils

Prefer Grub to boot Xen:

[ALL]# dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
[ALL]# update-grub


Configure software bridge on both Xen nodes. Our /etc/network/interfaces content can be seen below. Note the static IP addresses used for DRBD.

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

# Leaving allow-hotplug causes the following:
# *** A start job is running for LSB: Raise network interfaces. (15min 10s / no limit)
#allow-hotplug eth1
iface eth1 inet manual

# The default bridge that the toolstack expects is called xenbr0
auto xenbr0
iface xenbr0 inet dhcp
   bridge_ports eth1
   # Other possibly useful options in a virtualised environment
   bridge_stp off       # disable Spanning Tree Protocol
   bridge_waitport 0    # no delay before a port becomes available
   bridge_fd 0          # no forwarding delay

# DRBD heartbeat vlan
auto eth0
iface eth0 inet static
  address #.13 is xen01, .14 is xen02

Enforce network configuration:

[ALL]# systemctl restart networking

Configure Dom0 Memory and CPU

Open both Xen nodes, open the file /etc/default/grub and configure the following:

# Xen boot parameters for all Xen boots 
GRUB_CMDLINE_XEN="dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin"

Remember to apply the change to the grub configuration:

[ALL]# update-grub

Guest Behaviour on Dom0 Reboot

By default, when Xen dom0 shuts down or reboots, it tries to save (read hibernate) the state of the domUs. We want to make sure they get shut down normally.

[ALL]# grep -ve "^#" -ve "^$" /etc/default/xendomains


The xl was introduced in the Xen 4.1 release, however xend remained the default. As of Xen 4.2 however xend/xm are deprecated (and were removed in Xen Project 4.5) and xl should now be used by default.

[ALL]# grep -ve "^#" /etc/default/xen

When finished configuring the dom0, reboot the servers:

[ALL]# systemctl reboot
[xen01]# xl info
host                   : xen01.hl.local
release                : 3.16.0-4-amd64
version                : #1 SMP Debian 3.16.7-ckt25-2 (2016-04-08)
machine                : x86_64
nr_cpus                : 2
max_cpu_id             : 1
nr_nodes               : 1
cores_per_socket       : 2
threads_per_core       : 1
cpu_mhz                : 2294
hw_caps                : 178bfbff:28100800:00000000:00003300:d6d82203:00000000:00000021:00002020
virt_caps              :
total_memory           : 2047
free_memory            : 998
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 4
xen_extra              : .1
xen_version            : 4.4.1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          :
xen_commandline        : placeholder dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin
cc_compiler            : gcc (Debian 4.9.2-10) 4.9.2
cc_compile_by          : jmm
cc_compile_domain      : debian.org
cc_compile_date        : Tue Mar 15 22:37:15 CET 2016
xend_config_format     : 4



Leave a Reply

Your email address will not be published. Required fields are marked *