VM deployment: Kickstart/libvirt/kvm

My aim was to be able to automate the creation of a new virtual machines, starting with:

network config data, such as IP address, hostname
a path/size for disk

and after a short and unattended wait, to be able to log into my new VM using my ssh key.

The Basic Idea

Start with a bare CentOS 6 installation
Create a network bridge so that all new virtual machines can have real addresses on the local network
Install libvirtd to provide admin access to KVM/QEMU
Use virt-install to specify parameters for the new VM
Provide an installation kernel/initrd
Provide a kickstart file to dictate OS installation parameters

Various prerequisites

If the CPU's flags [1] include either vmx (for Intel) or svm (for AMD), it can support virtualization using KVM.

Packages

The host machine should have the following packages installed:

qemu-kvm
qemu-kvm-tools
libvirt
libvirt-client
libvirt-python
python-virtinst

Software repositories/mirrors

The VM install is installed across the network, so I make local mirrors of the following repositories available over http.

CentOS
EPEL
IUS
My own local, allgoodbits.org rpms

Networking

Eliminate the default bridge:

virsh net-destroy default
virsh net-undefine default

Ensure that bridged networking is available

/etc/sysconfig/network:

NETWORKING=yes
HOSTNAME=kvm1.allgoodbits.org
GATEWAY=br0

/etc/sysconfig/network-scripts/ifcfg-eth0:

DEVICE="eth0"
HWADDR="78:2B:CB:03:C2:09"
ONBOOT="yes"
ETHTOOL_OPTS="autoneg off speed 100 duplex full"
BRIDGE=br0

/etc/sysconfig/network-scripts/ifcfg-br0:

DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=10.19.0.32
NETMASK=255.255.252.0
GATEWAY=10.19.0.1
DELAY=0
DNS1=10.19.0.1

Kickstart

The kickstart file specifies exactly how the OS should be installed and what it looks like once it has booted. In other words, it is effectively providing the answers to the various questions asked by the installer during an interactive install of the operating system. For example, questions about disk partitioning, network configuration, the root password, package installation, etc., etc. It also gives the opportunity to run an arbitrary post-installation script. In my case, I use the %post section to install my ssh key, turn off some unnecessary services, start puppet and set an rc.local command so that a message is sent upon first successful boot.

Disks

Here's a sample kickstart fragment that I use as a baseline specification for virtual machines. It will:

remove all filesystems and partitions from all disks visible to the VM, destroying all existing data.
use LVM to create a single physical volume, with a volumegroup containing logical volumes for /, /boot, swap and /var

clearpart --all --initlabel
autopart
bootloader --location=mbr
part /boot --fstype ext2 --size=150
part swap --size=4096
part pv.01 --size=1 --grow
volgroup vg_root pv.01
logvol  /  --vgname=vg_root --fstype ext4 --size=16384  --name=lv_root
logvol  /var  --vgname=vg_root --fstype ext4 --size=1 --grow --name=lv_var

Networking

Fragments to specify boring, I mean 'common', network configuration.

Getting config via DHCP, firewalled, only allowing ssh traffic in:
```
firewall --enabled --ssh
network --device eth0 --bootproto dhcp
```
I likely want to leave out the --device directive and change the ksdevice kernel parameter below to link, indicating that anaconda should use the first interface that has link.

Specifying network configuration, disabled firewall:

network --device eth0 --bootproto static --ip 10.19.0.129 --netmask 255.255.252.0 \
--gateway 10.19.0.1 --nameserver 10.19.0.2 --hostname test.example.com

Yum repositories

A fragment to specify where to find Yum repositories, this creates config in /etc/yum.repos.d/ .

repo --name="centos" --baseurl=http://mirror.example.com/pub/centos/6.0/os/x86_64 --cost=100
repo --name="EPEL" --baseurl=http://mirror.example.com/pub/epel/6/x86_64 --cost=100
repo --name="IUS" --baseurl=http://mirror.example.com/pub/ius/stable/Redhat/6/x86_64 --cost=100
repo --name="LOCAL" --baseurl=http://mirror.example.com/pub/zoll/6/x86_64 --cost=100

Create and Install a VM guest

virt-install allows the creation of new VMs by interacting with various hypervisors using libvirt

virt-install -n test.allgoodbits.org -r 2048 --vcpus=2 \
-w bridge=br0,mac=52:54:00:00:00:01 \
--disk path=/var/lib/libvirt/images/test.allgoodbits.org,size=48 \
-l http://mirror.allgoodbits.org/pub/centos/6.0/os/x86_64 \
--nographics \
-x "ks=http://prime.allgoodbits.org/kickstart/c6-test.cfg ksdevice=eth0 \
ip=10.19.0.253 netmask=255.255.252.0 \
dns=10.15.1.90 gateway=10.19.0.1 console=ttyS0,115200"

-r n              allocate <n> MiB RAM to guest
-w bridge=br0     I want all guests to use bridged networking, because NAT just causes problems and because the virtio network driver which gives much better performance, requires it
--disk opt1=foo   options for path to the disk and the size (in GiB) MUST be included
-x "extra kernel boot params"      passed to the installation kernel; don't forget the console! Consider ``ksdevice=link``
-l <location>    specifies the location of the remote installation media, could also be nfs:<hostname>:/<export>

Using iSCSI storage

If you have created an iSCSI LUN to provide storage for your VM, the kvm storage pool XML description looks like this:

<pool type='iscsi'>
  <name>MYVM</name>
  <source>
      <host name='iscsi.allgoodbits.org' />
      <device path='iqn.2004-04.com.....' />

And you can use it like this:

virsh pool-define <myvm>.xml
virsh pool-start <MYVM>
[virsh pool-autostart <MYVM>]

Then your virt-install command changes its disk parameter:

--disk vol=MYVM/unit:0:0:0

[1]	/proc/cpuinfo contains lots of information about the CPU(s), including details about the feature set of the CPU (in the flags field) and the type/speed and more.

Windows

For Windows, it looks a little different, but basically the same:

virt-install --hvm -n ms1.allgoodbits.org --ram 2048 --vnc \
              --os-type=windows --os-variant=win2k \
              --network bridge \
              --disk path=/var/lib/libvirt/images/ms1.allgoodbits.org,size=10,driver=phy,subdriver=zvol \
               --cdrom /isos/ms-win.iso

Troubleshooting

Configure the firewall correctly (or turn it off)
Configure SElinux to allow VMs (or turn it off)
KVM admin requires r/w access to /dev/kvm

Important filesystem locations

VM disk images: /var/lib/libvirt/images/
Logs: /var/log/libvirt/qemu/
Configurations: /etc/libvirt/

AllGoodBits.org