FreeBSD-ZFS
From IdefixWiki
Contents |
Migrate UFS to ZFS
The system is ported in two steps, first step is to generate the first disk and copy the data to this disk, the second step is to add the second disk and start the restore of the raid 1.
Install FreeBSD 8.0 with ZFS Root
To really use ZFS it is recommended to install a AMD64 environment, so boot from USB Memstick and choose Fixit:
Zero your new drive to destroy any existing container:
dd if=/dev/zero of=/dev/ad4 *cancel it after some seconds*
We will use GPT to boot so we create at first these volumes:
gpart create -s gpt ad4 gpart add -b 34 -s 128 -t freebsd-boot ad4 gpart add -b 162 -s 8388608 -t freebsd-swap -l swap0 ad4 gpart add -t freebsd-zfs -l disk0 ad4
Install proteced MBR:
gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad4
Load ZFS:
kldload /mnt2/boot/kernel/opensolaris.ko kldload /mnt2/boot/kernel/zfs.ko kldload /mnt2/boot/kernel/geom_mirror.ko
Create ZFS pool:
mkdir /boot/zfs zpool create zroot /dev/gpt/disk0 zpool set bootfs=zroot zroot
Create the ZFS filessystem hierarchy:
zfs set checksum=fletcher4 zroot
zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp chmod 1777 /zroot/tmp
zfs create zroot/usr zfs create zroot/usr/home cd zroot ; ln -s /usr/home home
zfs create -o compression=lzjb -o setuid=off zroot/usr/ports zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
zfs create zroot/var zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash zfs create -o exec=off -o setuid=off zroot/var/db zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg zfs create -o exec=off -o setuid=off zroot/var/empty zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail zfs create -o exec=off -o setuid=off zroot/var/run zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp chmod 1777 /zroot/var/tmp
Install the FreeBSD to the zroot:
cd /dist/8.0-*
export DESTDIR=/zroot
for dir in base catpages dict doc games info lib32 manpages ports; \
do (cd $dir ; ./install.sh) ; done
cd src ; ./install.sh all
cd ../kernels ; ./install.sh generic
cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/
Make /var/empty readonly
zfs set readonly=on zroot/var/empty
chroot into /zroot
cd / chroot /zroot
echo 'zfs_enable="YES"' > /etc/rc.conf echo 'hostname="freebsdtest.mydomain.local"' >> /etc/rc.conf echo 'ifconfig_em0="DHCP"' >> /etc/rc.conf
Setup the bootloader:
echo 'zfs_load="YES"' > /boot/loader.conf echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf echo 'geom_mirror_load="YES"' >> /boot/loader.conf
Install ZFS aware boor loader
echo 'LOADER_ZFS_SUPPORT=YES' > /etc/src.conf mount -t devfs devfs /dev export DESTDIR="" cd /usr/src/sys/boot/ make obj make depend make cd i386/loader make install
Change the root password:
passwd
Set local time zone:
tzsetup
Create /etc/mail/aliases.db
cd /etc/mail make aliases
Exit from the /zroot
umount /dev exit
Install zpool.cache to the ZFS filesystem
cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache
Create the swap partition:
gmirror label -b prefer swap gpt/swap0
Create the /zroot/etc/fstab
# Device Mountpoint FStype Options Dump Pass# /dev/mirror/swap none swap sw 0 0
export LD_LIBRARY_PATH=/mnt2/lib
Unmount all zfs filesystems:
zfs unmount -a
Change mount points for the zroot pool
zfs set mountpoint=legacy zroot zfs set mountpoint=/tmp zroot/tmp zfs set mountpoint=/usr zroot/usr zfs set mountpoint=/var zroot/var
Exit Fixit mode and the sysinstall. The system should now boot the ZFS environment.
After the system is up again it should say something like this:
zpool status
pool: zroot
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
gpt/disk0 ONLINE 0 0 0
errors: No known data errors
gmirror status
Name Status Components
mirror/swap COMPLETE ad4p2
Copy Old System to ZFS
cd /zroot rsync -av /etc /zroot/ rsync -av /usr/local/etc /zroot/usr/local/ rsync -av /var/amavis /zroot/var/ rsync -av /var/db/DAV /var/db/clamav /var/db/dhcpd.* /var/db/mysql /var/db/openldap-data /var/db/openldap-data.backup /zroot/var/db/ rsync -av /var/log /zroot/var/ rsync -av /var/spool /var/named /zroot/var/ rsync -av /usr/home /zroot/usr/ rsync -av /root /zroot/ rsync -av /usr/src/sys/i386/conf /zroot/usr/src/sys/i386/ rsync -av /usr/local/backup /usr/local/backup_rsync /usr/local/cvs /usr/local/dbdump /usr/local/faxscripts /zroot/usr/local/ rsync -av /usr/local/firewall /usr/local/pgsql /usr/local/cvs /usr/local/psybnc /usr/local/router /zroot/usr/local/ rsync -av /usr/local/squirrelmail_data /usr/local/src /usr/local/ssl /usr/local/svn /usr/local/tftp /zroot/usr/local/ rsync -av /usr/local/var /usr/local/video /usr/local/www /usr/local/idisk /zroot/usr/local/ rsync -av /usr/local/bin/printfax.pl /usr/local/bin/grepm /usr/local/bin/block_ssh_bruteforce /usr/local/bin/learn.sh /zroot/usr/local/bin/ mkdir -p /zroot/usr/local/libexec/cups/ rsync -av /usr/local/libexec/cups/backend /zroot/usr/local/libexec/cups/ rsync -av /usr/local/share/asterisk /zroot/usr/local/share/ rsync -av /usr/local/libexec/mutt_ldap_query /zroot/usr/local/libexec/ rsync -av /usr/local/lib/fax /zroot/usr/local/lib/ mkdir -p /zroot/usr/local/libexec/nagios/ rsync -av /usr/local/libexec/nagios/check_zfs /usr/local/libexec/nagios/check_gmirror.pl /zroot/usr/local/libexec/nagios/
Check your /etc/fstab, /etc/src.conf and /boot/loader.conf after this and adapt it like described above.
Install Software
portsnap fetch portsnap extract cd /usr/ports/lang/perl5.10 && make install && make clean cd /usr/ports/ports-mgmt/portupgrade && make install && make clean portinstall bash zsh screen sudo radvd portinstall sixxs-aiccu security/openvpn quagga isc-dhcp30-server portinstall cyrus-sasl2 mail/postfix clamav amavisd-new fetchmail dovecot-sieve imapfilter p5-Mail-SPF p5-Mail-SpamAssassin procmail portinstall databases/mysql51-server net/openldap24-server databases/postgresql84-server mysql++-mysql51 portinstall asterisk asterisk-addons asterisk-app-ldap portinstall www/apache22 phpMyAdmin phppgadmin mod_perl2 mod_security www/mediawiki smarty portinstall pear-Console_Getargs pear-DB pear-Net_Socket pear php5-extensions squirrelmail squirrelmail-avelsieve-plugin portinstall munin-main munin-node net-mgmt/nagios nagios-check_ports nagios-plugins nagios-spamd-plugin logcheck nrpe portinstall portmaster portaudit portdowngrade smartmontools portinstall awstats webalizer portinstall bazaar-ng subversion git portinstall rsync ipcalc doxygen john security/gnupg nmap unison wol mutt-devel wget miniupnpd portinstall editors/emacs jed portinstall www/tomcat6 hudson portinstall cups portinstall squid adzap portinstall samba portinstall net-snmp portinstall teamspeak_server portinstall scponly
Attach all Disk and Restore them
Insert now the second disk (in my case ad6). We use GPT on the second disk too:
gpart create -s gpt ad6 gpart add -b 34 -s 128 -t freebsd-boot ad6 gpart add -b 162 -s 8388608 -t freebsd-swap -l swap1 ad6 gpart add -t freebsd-zfs -l disk1 ad6
Install MBR:
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad6
Create swap:
gmirror insert swap gpt/swap1
While rebuilding it will show:
gmirror status
Name Status Components
mirror/swap DEGRADED ad4p2
ad6p2 (48%)
After it is finished:
gmirror status
Name Status Components
mirror/swap COMPLETE ad4p2
ad6p2
Create the zfs mirror:
zpool attach zroot /dev/gpt/disk0 /dev/gpt/disk1
It will resilver now the data:
zpool status
pool: zroot
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress for 0h1m, 0.49% done, 4h1m to go
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
mirror ONLINE 0 0 0
gpt/disk0 ONLINE 0 0 0 12.4M resilvered
gpt/disk1 ONLINE 0 0 0 768M resilvered
errors: No known data errors
After the pool in online it shows:
zpool status
pool: zroot
state: ONLINE
scrub: resilver completed after 0h51m with 0 errors on Sat Jan 16 18:27:08 2010
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
mirror ONLINE 0 0 0
gpt/disk0 ONLINE 0 0 0 383M resilvered
gpt/disk1 ONLINE 0 0 0 152G resilvered
errors: No known data errors
Upgrade ZFS to New Version
Upgrade ZFS to a new version is done in two steps.
Upgrade the ZFS is done by:
zpool upgrade zroot zfs upgrade zroot
Now we have to upgrade the GPT bootloader, if you forget this step you will not be able to mount the ZFS anymore! The system will hang before the FreeBSD bootloader can be loaded.
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad4 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad6
To use the zfs as a storage in your network create a new folder:
zfs create -o compression=on -o exec=off -o setuid=off zroot/netshare
Now we define the mountpoint:
zfs set mountpoint=/netshare zroot/netshare
Set up network sharing:
zfs set sharenfs="-mapall=idefix -network=192.168.0/24" zroot/netshare
