Navigating unwillful misinformation

A good part of a sysadmin journey is really just navigating thru a bunch of other people's experiences, and my latest sysadmin journey (upgrading an entire vdev in a ZFS pool) has brought me to some... less than accurate information. What I'm hoping writing this down will do (and praying that SEO picks it up) is to bust a myth that I keep seeing on Google - that the Dell H310 Mini HBA does not support 4Kn drives.

The actual answer is that the H310 Mini actually does indeed support 4Kn drives. As proof:

[~] lspci
...
02:00.0 RAID bus controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
...
[~] lstopo
...
      PCIBridge
        PCI 02:00.0 (RAID)
          Block(Disk) "sdf"
          Block(Disk) "sdd"
          Block(Disk) "sdm"
          Block(Disk) "sdb"
          Block(Disk) "sdk"
          Block(Disk) "sdi"
          Block(Disk) "sdg"
          Block(Disk) "sde"
          Block(Disk) "sdc"
          Block(Disk) "sda"
          Block(Disk) "sdj"
          Block(Disk) "sdh"
...
[~] fdisk -l /dev/sdg
Disk /dev/sdg: 10.91 TiB, 12000138625024 bytes, 2929721344 sectors
Disk model: HGST HUH721212AL
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: DEADBEEF-CAFE-CAFE-CAFE-DEADBEEFCAFE

Device     Start        End    Sectors  Size Type
/dev/sdg1    256 2929721087 2929720832 10.9T FreeBSD ZFS

The Twist

Of course, there is no free lunch. The key here is the lspci call which shows the H310 Mini in my R720XD as a LSI SAS2008 device. This is a telltale sign that the H310 is in IT Mode.

There are valid reasons to avoid IT Mode - namely that practically every feature of the H310 is disabled, whic means no RAID from the H310. The H310 will act as a passthrough for the drives. For my purposes, which is using ZFS, this is actually advantageous, but if you need hardware RAID, then you might be out of luck.

I'm not going to cover the process of flashing the H310 Mini into IT mode - there's a great tutorial out there that I can point you to, and once you're in IT mode, the drives will show up just fine.

One final word of caution - your Linux drive letters will change, and if you're using drive letters (which is a bad, bad, bad practice) in /etc/fstab, you'll need to change that.

Upgrading your ZFS storage

My ZFS array started out with 4 2TB drives in RAID-Z2. After a while, I added 4 6TB drives in RAID-Z2 into the pool. My R720xd thus has 8 of it's 12 drive bays filled, but I would really rather not fill all 12 bays right away. (I skimp out on drive costs-wise, so RAID-Z2 is a must on drives I cannot trust wholeheartedly). Fortunately, ZFS allows us to upgrade an entire vdev - once the entire vdev grows in size, the pool gains extra storage.

Firstly you'll need to set autoexpand=on like so:

# zpool set autoexpand=on pool

Then use replace to swap over to the new drives:

# zpool replace /dev/disks/by-id/{old-disk-id} /dev/disks/by-id/{new-disk-id}

You can replace them all at once, but just to be cautious, I'd replace them either one-at-a-time or # of parity drives at a time (which in my case is 2). As of time of writing my vdev is still resilvering, but at the end of this I should have 36TB of capacity on my my server - not too bad!

My wallet will recover from this bad purchase... eventually...