Support #769
Updated by Daniel Curtis almost 9 years ago
This is a guide on how I manually setup FreeBSD with a GELI encrypted hard drive underneath of a ZFS root on a GPT formatted hard drive, without the help of a GUI or bsdinstall. This guide is intended to install FreeBSD using the installation DVD and will work offline. * When the FreeBSD Installer Welcome message appears, choose *Shell*. * Get a list of available drives: <pre> camcontrol devlist </pre> * Create the boot partition and install bootcode: <pre> gpart create -s gpt ada0 ada gpart add -l gptboot0 -s 512k -t freebsd-boot -a 4k ada0 ada gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0 da1 gpart set -a bootme -i 1 ada0 da1 </pre> * Create the ZFS *bootpool*: <pre> gpart add -l boot0 -t freebsd-zfs da1 mkdir -p /tmp/mnt/bootpool zpool create -m none -o altroot=/tmp/mnt/bootpool bootpool /dev/gpt/boot0 mkdir -p /tmp/mnt/bootpool/boot/zfs mount_nullfs /tmp/mnt/bootpool/boot/zfs /boot/zfs </pre> * Create the *swap* and *disk0* slices: <pre> gpart create -s gpt ada0 gpart add -s 4G -t freebsd-swap -a 4k -l swap0 ada0 gpart add -t freebsd-zfs -a 4k -l disk0 ada0 </pre> * Encrypt the swap space: <pre> geli onetime -d -e AES-XTS -l 256 -s 4096 /dev/ada0p3 </pre> * Encrypt the OS slice: <pre> mkdir /tmp/mnt/bootpool/boot/metadata_backup geli init -b -s 4096 -e AES-XTS -l 256 -B /tmp/mnt/bootpool/boot/metadata_backup/ada0p4.eli /dev/ada0p4 </pre> * Attach the encrypted slice: <pre> geli attach /dev/ada0p4 </pre> * Create the *xpool* ZFS pool on top of the GELI encrypted slice, then export it: <pre> mkdir -p /tmp/mnt/xpool zpool create -o altroot=/tmp/mnt/xpool -o cachefile=/tmp/zpool.cache -m none -f xpool /dev/ada0p4.eli zpool export xpool </pre> * Next import the *xpool* ZFS pool and create the root dataset and settings: <pre> zpool import -o altroot=/tmp/mnt/xpool -o cachefile=/tmp/zpool.cache xpool zpool set bootfs=xpool xpool zfs set checksum=fletcher4 xpool zfs set atime=off xpool zfs create xpool/ROOT zfs set mountpoint=/ xpool/ROOT </pre> #* Then create some additional system datasets: <pre> zfs create -o canmount=off xpool/ROOT/usr zfs create -o canmount=off xpool/ROOT/var zfs create -o compression=on -o exec=on -o setuid=off xpool/ROOT/tmp zfs create -o compression=gzip -o setuid=off xpool/ROOT/usr/ports zfs create -o compression=off -o exec=off -o setuid=off xpool/ROOT/usr/ports/distfiles zfs create -o compression=off -o exec=off -o setuid=off xpool/ROOT/usr/ports/packages zfs create -o compression=gzip -o exec=off -o setuid=off xpool/ROOT/usr/src zfs create -o compression=lzjb xpool/ROOT/usr/obj zfs create -o compression=lzjb -o exec=off -o setuid=off xpool/ROOT/var/crash zfs create -o compression=off -o exec=off -o setuid=off xpool/ROOT/var/empty zfs create -o compression=lzjb -o exec=on -o setuid=off xpool/ROOT/var/tmp </pre> * Set the permissions of the temp directories in the zfs mount: <pre> chmod 1777 /tmp/mnt/xpool/tmp chmod 1777 /tmp/mnt/xpool/var/tmp </pre> * Remount the *bootpool*: <pre> umount /boot/zfs mkdir /tmp/mnt/xpool/bootpool zfs set mountpoint=/tmp/mnt/xpool/bootpool bootpool zpool export bootpool zpool import bootpool mkdir -p /tmp/mnt/xpool/bootpool/boot/zfs mount_nullfs /tmp/mnt/xpool/bootpool/boot/zfs /boot/zfs </pre> * Extract the base.txz and kernel.txz to the zfs root to install the base system: <pre> cat /usr/freebsd-dist/base.txz | tar --unlink -xpJf - -C /tmp/mnt/xpool cat /usr/freebsd-dist/kernel.txz | tar --unlink -xpJf - -C /tmp/mnt/xpool </pre> h2. Post-Installation Setup * Chroot into the xpool: <pre> chroot /tmp/mnt/xpool </pre> * Copy the install bootload files over to the bootpool, then create a @/boot@ symlink: <pre> cd / rm -r boot/zfs mv boot/* bootpool/boot/ rm -r boot ln -sf bootpool/boot </pre> * Set a root passwd: <pre> passwd root </pre> * Add a new user: <pre> adduser </pre> * Set timezone: <pre> tzsetup </pre> * Create an fstab file: <pre> vi /etc/fstab </pre> #* And add the swap partition definition: <pre> /dev/ada0p3 none swap sw 0 0 </pre> * Add the initial system configuration: <pre> echo 'zfs_enable="YES"' >> /etc/rc.conf echo 'sshd_enable="YES"' >> /etc/rc.conf </pre> * And setup networking using DHCP: <pre> echo 'ifconfig_em0="DHCP"' >> /etc/rc.conf echo 'hostname="freebsd.example.com"' >> /etc/rc.conf </pre> #* (Optional) Setup networking using a static IP address instead: <pre> echo 'ifconfig_em0="inet 192.168.10.70 netmask 255.255.255.0 broadcast 198.100.10.255"' >> /etc/rc.conf echo 'defaultrouter="192.168.10.1"' >> /etc/rc.conf echo 'hostname="freebsd.example.com"' >> /etc/rc.conf echo 'nameserver 192.168.10.1' >> /etc/resolv.conf </pre> * Add the bootloader config: <pre> echo 'geom_eli_load="YES"' >> /boot/loader.conf echo 'zfs_load="YES"' >> /boot/loader.conf echo 'if_em_load="YES"' >> /boot/loader.conf echo 'vfs.root.mountfrom="zfs:xpool/ROOT"' >> /boot/loader.conf echo 'zpool_cache_load="YES"' >> /boot/loader.conf echo 'zpool_cache_type="/boot/zfs/zpool.cache"' >> /boot/loader.conf echo 'zpool_cache_name="/boot/zfs/zpool.cache"' >> /boot/loader.conf </pre> * Exit from the chroot environment: <pre> exit </pre> * Copy the working zpool.cache file to the bootpool: <pre> cp /tmp/zpool.cache /tmp/mnt/xpool/bootpool/boot/zfs/zpool.cache </pre> * Setup the ZFS mountpoints <pre> zfs set mountpoint=legacy xpool/ROOT zfs set mountpoint=/bootpool bootpool </pre> * Unmount the filesystems: <pre> umount /boot/zfs zfs unmount -a zpool export xpool zpool export bootpool </pre> * Reboot the system and eject the FreeBSD install disc: <pre> reboot </pre> h2. Resources * http://www.schmidp.com/2014/01/07/zfs-full-disk-encryption-with-freebsd-10-part-2/ * https://forums.freebsd.org/threads/42773/ * https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE * https://calomel.org/zfs_freebsd_root_install.html * http://daemon-notes.com/articles/system/install-zfs/gpart * http://daemon-notes.com/articles/system/install-zfs/zfs * http://daemon-notes.com/articles/system/install-zfs/finish