Migrate system from i386 (32-bit mode) to amd64 (64-bit mode)

Found here: https://wiki.freebsd.org/amd64/i386Migration

Motivation

If an amd64 hardware has been initially installed in 32-bit mode (i.e. as a plain i386 machine), it might later be desired to turn it into a full 64-bit machine.

The recommended way to do this is to back up all personal and important data, and reinstall using an amd64 installation medium.

However, it’s also possible to migrate the machine through a rebuild from source code.

Much of this description has been inspired by Peter Wemm’s mailing list article.

Prerequisites

• some amd64 hardware, currently running in i386 mode (obviously ;.)
• a swap partition/volume of at least 350 MiB which can be used as a temporary root filesystem (miniroot)
• enough RAM so the machine could at least install a precompiled system without swap
• a recent /usr/src filesystem of whatever branch has been chosen
• a running i386 system matching whatever is installed under /usr/src (just to be sure everything is consistent)
• a safe backup of all important data, in case something goes wrong with the migration
• console access, so you can use the loader prompt and single user mode (ssh access is not sufficient!)

Migration Process

If necessary, define your kernel config file in /etc/make.conf, create and edit it (otherwise, GENERIC will be used). Build amd64 world and kernel using:

make buildworld TARGET=amd64 TARGET_ARCH=amd64

make buildkernel TARGET=amd64 TARGET_ARCH=amd64

This is supposed to pass without any issues. If not: resolve issues, and start over.

Turn your swap into a miniroot:

swapinfo -h — make sure no swap is in use (if the swap is not free, reboot here)
swapoff /dev/ad4s1b (or whatever your swap device is named — replace that name in the steps below if it is different)

edit /etc/fstab to comment out the swap line

newfs -U -n /dev/ad4s1b
mount /dev/ad4s1b /mnt
cd /usr/src && make installworld TARGET=amd64 TARGET_ARCH=amd64 DESTDIR=/mnt
file /mnt/bin/echo — make sure this displays as ELF 64-bit LSB executable
cd /usr/src/etc && make distribution TARGET=amd64 TARGET_ARCH=amd64 DESTDIR=/mnt
cp /etc/fstab /mnt/etc/fstab This completes your miniroot in the swap volume. 

Prepare the /usr/obj tree for later installation: The 64-bit (cross) build should be in /usr/obj/amd64.amd64 now, so remove any leftover 32-bit stuff: rm -rf /usr/obj/usr/src.

cd /usr/obj && ln -s amd64.amd64/* . — this should get symlinks for lib32 and usr (the original file location must still be retained by now) 

Copy the 64-bit kernel into a bootable place, like

cp /usr/obj/amd64.amd64/usr/src/sys/$YOURKERNEL/kernel /boot/kernel.amd64

If any further KLDs are needed that have not been statically compiled into the kernel, copy them over to some place (presumably under /boot), too. Reboot, and stop the system at the loader prompt (in the loader menu, press “2” for FreeBSD 9, or press “6” for FreeBSD 8)

unload to get rid of the (automatically loaded) 32-bit kernel

load /boot/kernel.amd64, possibly followed by loading any essential 64-bit KLDs here

boot -as to boot into single-user mode, with the loader asking for the location of the root filesystem (rather than figuring it out from /etc/fstab)

At the loader prompt asking for the root filesytem, enter ufs:, followed by the name of the swap volume/partition; e.g. ufs:/dev/ad4s1b

Press Enter to get the single-user shell

Mount (at least) the /, /var and /usr filesystems under /mnt; examine /etc/fstab to know which resources to mount Bootstrap the system libraries and utilities from the miniroot:

chflags -R noschg /mnt/*bin /mnt/lib* /mnt/usr/*bin /mnt/usr/lib*

cd / && find *bin lib* usr/*bin usr/lib* | cpio -dumpv /mnt

mount -t devfs devfs /mnt/dev 

chroot /mnt so you can pretend living in the final target filesystem space

cd /usr/src && make installkernel installworld — this should work without any issues

exit so you're back in the miniroot environment

reboot

Boot into single-user mode (in the loader menu, press “6” then Enter for FreeBSD 9, or press “4” for FreeBSD 8)

fsck -p (just make sure all filesystems are allright)

mount -a -t ufs

Edit /etc/fstab to re-enable the swap To give any existing (32-bit) ports a chance to work, do the following:

mkdir /usr/local/lib32

cd /usr/local/lib && find . -type f \( -name "*.so*" -o -name "*.a" \) | cpio -dumpv ../lib32

add the following line to /etc/rc.conf:

ldconfig32_paths="/usr/lib32 /usr/local/lib32" 

Turn /usr/obj into its canonical form (optional):

cd /usr/obj

rm * — just to remove the lib32 and usr symlinks (it will complain and not remove the amd64.amd64 directory)

mv amd64.amd64/* . && rmdir amd64.amd64 

Remove the temporary kernel (optional): rm /boot/kernel.amd64 Exit the single-user shell, bringing the system into multiuser

Now the basic migration has been done.

Some 32-bit ports might not work, despite of the hackery to save their shared libs in a separate place. It’s probably best to reinstall all ports. portupgrade -af doesn’t work as it clobbers its own package database in the course of this operation (when reinstalling ruby and/or ruby-bdb).