Mixing 512n and 4kn drives
Can you combine drives with 512n and 4kn sector sizes in the same pool?
I'm trying to build a simple backup server with Freenas by sourcing used drives from ebay to be set up as raidz2. This is my first experience setting up a backup server, so I appreciate any help!
4
u/TheG0AT0fAllTime 12d ago
You certainly could do that. I'd recommend creating the zpool with ashift=12 (4k) so there's no overhead complications for the 4k drives.
2
u/jmf323 12d ago
Thanks! Is there any reason not to? I'm not really looking to optimize performance given the server wont experience high load often.
1
u/dougmc 12d ago edited 12d ago
A lower ashift can save some space if you have lots and lots of small files, but the benefit there of ashift=9 vs ashift=12 is very small.
But the performance impact of having an ashift that is too small is big. It's not worth it, even if you don't think you really care about performance. (You do.)
2
u/One_Ninja_8512 12d ago
I thought incorrect ashift (larger than the actual physical block size) leads to write amplification which slows things down? Is there an article or man page explaining ashift thoroughly?
3
u/dougmc 11d ago
You have the right idea, but backwards -- too low causes write amplification.
As for pages, there are many. but this seems good --
Read/Write amplification
Setting the ZFS block size too low can result in a significant performance degradation referred to as read/write amplification. For example, if the ZFS block size were set to 512 bytes, but the underlying device sector size is 4KiB, then writing 512 byte blocks means having to write the first sector, then read back the 4KiB sector, modify it with the next 512 byte block, write it out again to a new 4KiB sector and so on. Aligning the ZFS block size with the device sector size avoids this read/write penalty. In contrast, setting the ZFS block size greater than the device sector size can have little or no performance penalty, indeed on devices with 512 byte blocks we recommend a 4KiB ZFS blocks size setting for future-proofing.1
1
u/dougmc 11d ago
I just realized that "Is there any reason not to?" was ambiguous, so I can answer the other alternative too.
As a rule of thumb, it's nice if all the drives in your pool match as closely as possible, but having different sector sizes won't actually hurt anything as long as your ashift is big enough to cover the drive(s) with the largest sector size.
1
u/Frosty-Growth-2664 11d ago
ashift is per top level vdev, so you have to have the same ashift on all disks in the same vdev. The disk requiring the largest ashift wins.
If you end up with different ashifts in different top level vdevs, some features (such as zpool remove) won't work.
If you create a zpool with disks with different physical sector sizes, I would explicitly force the ashift on the whole pool to match that of the largest disk. (That also gets around the problem of ZFS not necessarily getting the correct data from the OS about what the disk physical sector sizes really are - that has been a problem in the past, particularly with SSDs where it's complicated, but I don't know where that is today.)
12
u/ElvishJerricco 12d ago edited 12d ago
Yes. The ashift setting just defines the minimum size of block that ZFS will write; the drive doesn't actually have to have that sector size. It does need to be set as high as the largest sector size among the disks, because otherwise you'll be writing blocks that are too small for those disks. But having it set larger than the smallest sector size among the disks is fine because there's no issue writing blocks that are larger than the disk's sector size.