GNet Solutions » Hardware Hacking » Raspberry Pi

• Table of contents
• Prepare The System
  • Format the USB drives
  • Install yaourt
• Install ZFS DKMS from the AUR
• Setting Up The Pools
  • Create a storage pool
• Kernel Upgrades
  • Rebuild SPL DKMS
  • Rebuild ZFS DKMS
• Tips
  • Lower ARC size
  • Lower kmem size
  • Lower vdev cache size
• Resources

One of the many projects I had for my pi was installing ZFS. While Raspbian is a a good distro for beginners, I require a bit more control and up to date packages than Raspbian can provide, so I use Arch Linux. This is a guide on how I compiled support for ZFS on my Raspberry Pi Model B+.

WARNING: I have not yet finished this yet, so please treat this as experimental for the time being.

UPDATE: This guide has been tested and works on Arch Linux as of 02/21/15.

NOTE: I have found that installing the zfs-dkms package from the AUR is much easier than trying to roll a custom kernel just for ZFS.

Prepare The System¶

• Update the system:
  

  pacman -Syu
  

• Install base-devel, cmake, and linux-headers packages
  

  pacman -S base-devel cmake linux-headers
  

Format the USB drives¶

This guide is using USB drives for its data drives. Yes, I know this will eventually cause a huge bottleneck in I/O performance.

• Format /dev/sda:
  

  fdisk /dev/sda
  

  • And type the following to format the USB drive as a Solaris root partition:
    

    g
    n
    1
    [Enter]
    [Enter]
    t
    39
    w
    

• Format /dev/sdb:
  

  fdisk /dev/sdb
  

  • And type the following to format the USB drive as a Solaris root partition:
    

    g
    n
    1
    [Enter]
    [Enter]
    t
    39
    w
    

Install yaourt¶

Yaourt isn't necessary, but makes managing AUR packages a lot easier.

• Install yaourt

Install ZFS DKMS from the AUR¶

• Install zfs-dkms:
  

  yaourt zfs-dkms
  

• NOTE: Edit the PKGBUILD for zfs-dkms, spl-dkms, and zfs-utils
  • And modify the arch parameter to match the following; adding "armv6h" and "armv6l":
    

    arch=("i686" "x86_64" "armv6h" "armv6l")
    

• Install zfs-utils:
  

  yaourt zfs-utils
  

  • Force install the zfs-utils package (probably a bad idea, but the only way I could get it to install properly):
    

    pacman -U --force /tmp/yaourt-tmp-username/zfs-utils-0.6.3-1.2-armv6h.pkg.tar.xz
    

  • Replace username with the user that built the zfs-utils package
  • Replace the version with the current version being installed

• Install the zfs kernel module:
  

  depmod -a
  modprobe zfs
  

• Check that the zfs modules were loaded:
  

  lsmod
  

  • Example output:
    

    zfs                  1229845  0 
    zunicode              322454  1 zfs
    zavl                    5993  1 zfs
    zcommon                43765  1 zfs
    znvpair                80689  2 zfs,zcommon
    spl                   165409  5 zfs,zavl,zunicode,zcommon,znvpair
    

---

Setting Up The Pools¶

This guide will be setting up a mirror of 2 USB drives, both will shown as /dev/sda and /dev/sdb, respectively.

Create a storage pool¶

• Get the id's of the drives to add to the zpool. The zfs on Linux developers recommend using device ids when creating ZFS storage pools of less than 10 devices. To find the id's, simply:
  

  ls -lah /dev/disk/by-id/
  

  • Example output:
    

    lrwxrwxrwx 1 root root  9 Aug 12 16:26 usb-SanDisk_Cruzer_20015001801AE2D0432E-0:0-part1 -> ../../sda
    lrwxrwxrwx 1 root root  9 Aug 12 16:26 usb-SanDisk_Cruzer_20022213091FE2A0CC42-0:0-part1 -> ../../sdb
    

• Create the mirrored ZFS pool:
  

  zpool create -f -m /mnt/usbpool usbpool mirror /dev/disk/by-id/usb-SanDisk_Cruzer_20015001801AE2D0432E-0\:0-part1 /dev/disk/by-id/usb-SanDisk_Cruzer_20022213091FE2A0CC42-0\:0-part1
  

  
  NOTE: Make sure the path to the partition is used and not the path for the disk itself, or else an error will occur.

• Check the zpool status:
  

  zpool status
  

  • Example output:
    

      pool: usbpool
     state: ONLINE
      scan: none requested
    config:
    
        NAME                                                   STATE     READ WRITE CKSUM
        usbpool                                                ONLINE       0     0     0
          mirror-0                                             ONLINE       0     0     0
            usb-SanDisk_Cruzer_20015001801AE2D0432E-0:0-part1  ONLINE       0     0     0
            usb-SanDisk_Cruzer_20022213091FE2A0CC42-0:0-part1  ONLINE       0     0     0
    
    errors: No known data errors
    

• Create a mountpoint:
  

  zfs create usbpool/home -o mountpoint=/home
  

• Check the mount point status:
  

  zfs list usbpool/home
  

  • Example output:
    

    NAME          USED  AVAIL  REFER  MOUNTPOINT
    usbpool/home    30K  58.6G    30K  /home
    

• Automatically mount the zfs pool:
  

  mkdir -p /etc/zfs
  zpool set cachefile=/etc/zfs/zpool.cache usbpool
  

  • Enable the service so it is automatically started at boot time:
    

    systemctl enable zfs.target
    

  • To manually start the daemon:
    

    systemctl start zfs.target
    

Kernel Upgrades¶

I found that after upgrading the kernel will not automatically rebuild the ZFS DKMS module, this is to be expected. Rather than reinstalling from the AUR, the DKMS modules just need to be built again.

• Upgrade the kernel:
  

  pacman -Syu
  

• And reboot for the new kernel to take effect:
  

  reboot
  

Rebuild SPL DKMS¶

• Rebuild SPL DKMS module:
  

  dkms build spl/0.6.3
  

• Install SPL DKMS module:
  

  dkms install spl/0.6.3
  

Rebuild ZFS DKMS¶

• Rebuild ZFS DKMS module:
  

  dkms build zfs/0.6.3
  

• Install ZFS DKMS module:
  

  dkms install zfs/0.6.3
  

• Install the zfs kernel module:
  

  depmod -a
  modprobe zfs
  

• Check that the zfs modules were loaded:
  

  lsmod
  

  • Example output:
    

    zfs                  1229845  0 
    zunicode              322454  1 zfs
    zavl                    5993  1 zfs
    zcommon                43765  1 zfs
    znvpair                80689  2 zfs,zcommon
    spl                   165409  5 zfs,zavl,zunicode,zcommon,znvpair
    

Tips¶

Lower ARC size¶

• Edit the cmdline.txt:
  

  nano /boot/cmdline.txt
  

  • And add zfs.zfs_arc_max=64000000 as a kernel parameter to set the ARC to 64MB:
    

    selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 zfs.zfs_arc_max=64000000 elevator=noop rootwait
    

Lower kmem size¶

• Edit the cmdline.txt:
  

  nano /boot/cmdline.txt
  

  • And add vm.kmem_size="256M" vm.kmem_size_max="256M" as a kernel parameter to set the kmem to 256MB:
    

    selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 zfs.zfs_arc_max=64000000 vm.kmem_size="256M" vm.kmem_size_max="256M" elevator=noop rootwait
    

Lower vdev cache size¶

• Edit the cmdline.txt:
  

  nano /boot/cmdline.txt
  

  • And add zfs.vdev.cache.size="4M" as a kernel parameter to set the vdev cache size to 4MB:
    

    selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 zfs.zfs_arc_max=64000000 vm.kmem_size="256M" vm.kmem_size_max="256M" zfs.vdev.cache.size="4M" elevator=noop rootwait
    

---

Resources¶

• https://wiki.archlinux.org/index.php/ZFS
• https://aur.archlinux.org/packages/zfs-dkms/
• https://aur.archlinux.org/packages/zfs-utils/