Inhaltsverzeichnis
KVM With Gentoo
We use as distribution a gentoo system. The KVM will be installed on a raid1 (mirror) with LVM2. We use the standard procedure to install the gentoo system. I start the installation from an already running system (you can even boot from a boot CD, USB stick or any other linux live system).
The system where I will install is located on /dev/sda. We will start with one harddisk, the second harddisk will be plugged into the system after it is running fine the raid system will mirror the data automatically. The harddisk must be completely empty.
Prepare the harddisk
modprobe raid1
Raid Device | /dev/sda | missing | Type |
---|---|---|---|
/dev/sda1 | Bios boot partition | Bios boot partition | 1MiB |
/dev/md0 | Raid for Gentoo-KVM | Raid for Gentoo-KVM | Raid-1 (mirror) |
parted /dev/sda mklabel gpt (say yes if parted asks you) mkpart boot1 1MiB 2MiB mkpart kvm1 2MiB 100% set 1 bios_grub on set 2 raid on quit
mdadm --create /dev/md0 --level=mirror --raid-devices=2 /dev/sda1 missing mdadm --grow /dev/md0 -b internal pvcreate /dev/md0 vgcreate vg-kvm /dev/md0 lvcreate -L100M -nboot vg-kvm lvcreate -L4GB -nswap vg-kvm lvcreate -L30GB -nroot vg-kvm
mkfs.ext2 -L boot /dev/vg-kvm/boot mkfs.ext4 -L root /dev/vg-kvm/root mkswap /dev/vg-kvm/swap swapon /dev/vg-kvm/swap
mount /dev/vg-kvm/root /mnt/gentoo/ mkdir /mnt/gentoo/boot mount /dev/vg-kvm/boot /mnt/gentoo/boot/
Installation of Gentoo
Make sure you have the following use flags defined in /etc/portage/make.conf
USE="bindist mmx sse sse2 -ldap perl python lvm parted qemu virt-network avahi sasl audit nfs" MAKEOPTS="-j5" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64"
Follow the normal gentoo install guide here: http://www.gentoo.org/doc/de/handbook/handbook-amd64.xml?part=1&chap=5
Some changes to the gentoo manual (all to be executed in the chroot environment):
- Use lzo to compress kernel
- Use Grub 2
- Modify fstab to use labels
- Build our own initramfs
echo "=sys-boot/grub-2*">>/etc/portage/package.keywords echo "sys-boot/grub device-mapper">>/etc/portage/package.use emerge -av grub lzop lvm2 mdadm genkernel rc-update add lvm boot rc-update add mdraid boot
Modify /etc/default/grub:
GRUB_CMDLINE_LINUX_DEFAULT="dolvm domdadm intel_iommu=on,igfx_off,pass-through vga=791 splash=silent,theme:gentoo console=tty1 quiet libata.ignore_hpa=1 cgroup_enable=memory swapaccount=1"
To build the initramfs use:
genkernel --lvm --mdadm --iscsi --disklabel --install initramfs
Setup fstab:
blkid /dev/loop0: TYPE="squashfs" /dev/sda2: UUID="9338e0f6-0ed2-8112-a0e2-840b4d9616de" UUID_SUB="fa5fb4fe-6f65-efe6-5cd3-21e7e6c42976" LABEL="Gentoo-20121221:0" TYPE="linux_raid_member" /dev/md0: UUID="PoGeVn-q9Sb-T7YD-Ysro-oRDG-pB7f-tXRWzr" TYPE="LVM2_member" /dev/sdb: LABEL="PENDRIVE" UUID="1105-224F" TYPE="vfat" /dev/mapper/vg--kvm-boot: LABEL="boot" UUID="251d9d9e-2c38-41db-938d-e25ef56afd3a" TYPE="ext2" /dev/mapper/vg--kvm-swap: UUID="9d77c5f7-ebc8-4a9d-afa9-cd215489aae8" TYPE="swap" /dev/mapper/vg--kvm-root: LABEL="root" UUID="7702989f-02c0-48dd-b640-3a921797096b" TYPE="ext4"
Now edit the /etc/fstab based on the information:
UUID="251d9d9e-2c38-41db-938d-e25ef56afd3a" /boot ext2 noatime 1 2 UUID="7702989f-02c0-48dd-b640-3a921797096b" / ext3 noatime 0 1 UUID="9d77c5f7-ebc8-4a9d-afa9-cd215489aae8" none swap sw 0 0
Install GRUB 2
mkdir /boot/grub2 grub-mkconfig -o /boot/grub/grub.cfg grub-install /dev/sda
Configure KVM
Install required software:
emerge -av qemu usbutils bridge-utils usermode-utilities iptables macchanger virt-manager rc-update add libvirtd default gpasswd -a qemu kvm
Enable KSM:
echo "echo 1 > /sys/kernel/mm/ksm/run" >/etc/local.d/ksm.start chmod +x /etc/local.d/ksm.start
Create certificates:
mkdir -p /tmp/certs cd /tmp/certs certtool --generate-privkey > cakey.pem echo "cn = FM-Data" >ca.info echo "ca" >>ca.info echo "cert_signing_key" >>ca.info certtool --generate-self-signed --load-privkey cakey.pem --template ca.info --outfile cacert.pem mkdir -p /etc/pki/CA cp cacert.pem /etc/pki/CA certtool --generate-privkey > serverkey.pem echo "organization = FM-Data" >server.info echo "cn = kvm.idefix.lan" >>server.info echo "tls_www_server" >>server.info echo "encryption_key" >>server.info echo "signing_key" >>server.info certtool --generate-certificate --load-privkey serverkey.pem --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem --template server.info --outfile servercert.pem mkdir -p /etc/pki/libvirt/private cp serverkey.pem /etc/pki/libvirt/private/ cp servercert.pem /etc/pki/libvirt/ certtool --generate-privkey > clientkey.pem echo "country = DE" >client.info echo "state = Bayern" >>client.info echo "locality = Fuerstenfeldbruck" >>client.info echo "organization = FM-Data" >>client.info echo "cn = client1" >>client.info echo "tls_www_client" >>client.info echo "encryption_key" >>client.info echo "signing_key" >>client.info certtool --generate-certificate --load-privkey clientkey.pem --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem --template client.info --outfile clientcert.pem cp clientkey.pem /etc/pki/libvirt/private/clientkey.pem cp clientcert.pem /etc/pki/libvirt/clientcert.pem rm *
Network bridge: Edit vi /etc/conf.d/net
bridge_br0="eth0" brctl_br0="setfd 0 sethello 0 stp off" config_br0="dhcp"
cd /etc/init.d/ ln -s net.lo net.br0 rc-update del net.eth0 rc-update add net.br0 default
Enable PCI passthrough:
echo "options kvm allow_unsafe_assigned_interrupts=1" >kvm_iommu_map_guest.conf
Add Second Harddisk and Build Raid1
Install sgdisk:
emerge -av sys-apps/gptfdisk
Copy the GPT partition schema from the original disk to the new disk. The already existing disk is /dev/sda and the new disk is /dev/sdf:
sgdisk -R=/dev/sdf /dev/sda sgdisk -G /dev/sdf sgdisk -c 1:"boot2" /dev/sdf sgdisk -c 2:"kvm2" /dev/sdf
See disk and raid information:
cat /proc/mdstat blkid /dev/sdf1: PARTLABEL="boot2" PARTUUID="bea234c8-ce71-4c23-8e67-7faf5d8e50d1" /dev/sdf2: PARTLABEL="kvm2" PARTUUID="054de73e-7cd6-4c94-860b-3eda62cabc58"
Add the new disk to the raid:
mdadm -a /dev/md126 /dev/disk/by-partuuid/054de73e-7cd6-4c94-860b-3eda62cabc58
Install boot manager:
grub2-install /dev/sdf
Install Gentoo as Guest
Download the install-amd64-minimal-20130207.iso and place it in the iso directory:
cd /mnt/isos/ wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-iso/default/20130207/install-amd64-minimal-20130207.iso
Now start virt-manager and we create there the new virtual machine. I created a lvm partition with 30GB and mounted the iso we downloaded. Then follow the normal gentoo installation guide or continue here.
Prepare Harddisk
fdisk /dev/sda n p 1 <ENTER> +100M n p 2 <ENTER> <ENTER> t 2 8e a 1 w pvcreate /dev/sda2 vgcreate vg-knx /dev/sda2 lvcreate -L4GB -nswap vg-knx lvcreate -l+100%FREE -nroot vg-knx mkfs.ext2 -L boot /dev/sda1 mkfs.ext4 -L root /dev/vg-knx/root mkswap -L swap /dev/vg-knx/swap swapon /dev/vg-knx/swap mount /dev/vg-knx/root /mnt/gentoo/ mkdir /mnt/gentoo/boot mount /dev/sda1 /mnt/gentoo/boot cd /mnt/gentoo
Install Gentoo
links http://www.gentoo.org/main/en/mirrors.xml (download stage3 in releases/autobuilds/current-stage3/default/..../stage3-amd64....tar.bz2) q tar xjpf stage3-*.tar.bz2 echo 'MAKEOPTS="-j2"' >>etc/portage/make.conf mirrorselect -i -o >> /mnt/gentoo/etc/portage/make.conf mirrorselect -i -r -o >> /mnt/gentoo/etc/portage/make.conf cp -L /etc/resolv.conf /mnt/gentoo/etc/ mount -t proc none /mnt/gentoo/proc mount --rbind /sys /mnt/gentoo/sys mount --rbind /dev /mnt/gentoo/dev chroot /mnt/gentoo /bin/bash source /etc/profile export PS1="(chroot) $PS1" mkdir /usr/portage emerge-webrsync eselect news list eselect news read eselect profile list eselect profile set 3 emerge --sync --quiet cp /usr/share/zoneinfo/Europe/Berlin /etc/localtime echo "Europe/Berlin" > /etc/timezone emerge =gentoo-sources-3.0.35 eix genkernel syslog-ng vixie-cron dhcpcd grub vim rc-update add syslog-ng default rc-update add vixie-cron default rc-update add sshd default
Prepare the kernel
vi /etc/genkernel.conf
SYMLINK="yes" MAKEOPTS="-j2" LVM="yes" DISKLABEL="yes" BOOTLOADER="grub"
Compile the kernel now with:
cd /usr/src/linux genkernel all
Configure the system
vi /etc/fstab LABEL="boot" /boot ext2 noatime 1 2 LABEL="root" / ext4 noatime 0 1 LABEL="swap" none swap sw 0 0
Configure hostname:
vi /etc/conf.d/hostname hostname="knx.idefix.lan"
Configure network:
vi /etc/conf.d/net config_eth0="dhcp"
cd /etc/init.d ln -s net.lo net.eth0 rc-update add net.eth0 default
Locale settings:
vi /etc/conf.d/keymaps vi /etc/locale.gen en_US ISO-8859-1 en_US.UTF-8 UTF-8 de_DE ISO-8859-1 de_DE@euro ISO-8859-15 de_DE.UTF-8 UTF-8 locale-gen vi /etc/env.d/02locale LANG="de_DE.UTF-8" LC_COLLATE="C" env-update && source /etc/profile
Install grub
vi /boot/grub/grub.conf default 0 timeout 0 splashimage=(hd0,0)/grub/splash.xpm.gz title Gentoo Linux root (hd0,0) kernel /kernel real_root=/dev/vg-knx/root dolvm initrd /initramfs
grep -v rootfs /proc/mounts > /etc/mtab grub-install --no-floppy /dev/sda
Setup users
passwd useradd -m -G users,wheel,audio -s /bin/bash idefix passwd idefix rm /stage3-*.tar.bz2*
Reboot the system
exit cd umount -l /mnt/gentoo/dev{/shm,/pts,} umount -l /mnt/gentoo{/boot,/proc,} reboot
Pass Cine-S2 to virtual machine
Find the device with:
lspci
we get:
... 04:00.0 Multimedia controller: Digital Devices GmbH Octopus LE DVB adapter ...
We search for the id of the card:
lspci -n |grep 04:00.0
We get:
04:00.0 0480: dd01:0003
Now we create a startup script to free the card before libvirt tries to start the virtual machine. For this we create a new start script and enable it:
cd /etc/init.d/ echo '#!/sbin/runscript' > pci_passthrough echo "" >> pci_passthrough echo "depend() {" >> pci_passthrough echo " before libvirtd" >> pci_passthrough echo "}" >> pci_passthrough echo "" >> pci_passthrough echo "start () {" >> pci_passthrough echo " echo "dd01 0003" > /sys/bus/pci/drivers/pci-stub/new_id" >> pci_passthrough echo " echo 0000:04:00.0 >/sys/bus/pci/devices/0000\:04\:00.0/driver/unbind" >> pci_passthrough echo " echo 0000:04:00.0 >/sys/bus/pci/drivers/pci-stub/bind" >> pci_passthrough echo "}" >> pci_passthrough echo "" >> pci_passthrough chmod +x pci_passthrough rc-update add pci_passthrough default
If you reboot your system now, it frees the pci card with the given id before libvirt tries to start the virtual machines.
Snapshot and upgrade the guest, maybe roll back
We have now the following setup.
vgs VG #PV #LV #SN Attr VSize VFree boot 1 3 0 wz--n- 148,95g 0 vg-kvm 1 7 0 wz--n- 232,75g 120,30g videostorage 1 1 0 wz--n- 4,00t 20,00g
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert root boot -wi-a--- 30,00g swap boot -wi-a--- 2,00g video1 boot -wi-a--- 116,95g boot vg-kvm -wi-ao-- 100,00m freepbx vg-kvm -wi-a--- 19,53g iso vg-kvm -wi-ao-- 10,00g knx vg-kvm -wi-ao-- 29,30g root vg-kvm -wi-ao-- 30,00g swap vg-kvm -wi-ao-- 4,00g yavdr vg-kvm -wi-ao-- 19,53g video0 videostorage -wi-ao-- 3,98t
Create snapshot
We would like to create now a snapshot of the machine yavdr:
virsh shutdown yavdr
Wait till the virtual machine is shutdown completely. Check it with:
virsh list --all
Create the snapshot:
lvcreate -L4G -s -n /dev/vg-kvm/yavdr-snapshot /dev/vg-kvm/yavdr virsh start yavdr
We can see the new snapshot now:
lvs vg-kvm LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert boot vg-kvm -wi-ao-- 100,00m freepbx vg-kvm -wi-a--- 19,53g iso vg-kvm -wi-ao-- 10,00g knx vg-kvm -wi-ao-- 29,30g root vg-kvm -wi-ao-- 30,00g swap vg-kvm -wi-ao-- 4,00g yavdr vg-kvm owi-aos- 19,53g yavdr-snapshot vg-kvm swi-a-s- 4,00g yavdr 0,05
Roll back to snapshot
We tested now some stuff and we see it does not work. Stop the virtual machine:
virsh destroy yavdr
Rollback the lvm to the snapshot (this will destroy the snapshot):
lvconvert --merge /dev/vg-kvm/yavdr-snapshot
Start the VM again:
virsh start yavdr
Destroy the snapshot
If everything is fine, destroy the snapshot:
lvremove /dev/vg-kvm/yavdr-snapshot