# Convert a ZFS mirror to raidz1 using a virtual device https://marc.khouri.ca/posts/2020/zfs-mirror-to-raidz.html It is not possible to convert a zfs vdev’s type (e.g. raidz1 to raidz2). However, with a bit of trickery, we can convert a pool with one mirrored vdev into a pool with one raidz1 vdev, without having an extra hard drive on hand. ## Summary We’re going to create a zfs raidz1 vdev with a virtual device. This will allow us to create the raidz1 vdev with 1 fewer physical hard drive than we really have. In a concrete example, let’s say I have a current pool with one mirrored vdev: ``` zpool old-pool vdev mirror-1 /dev/old_hdd_a /dev/old_hdd_b ``` We’ll remove a device from that mirror: ``` zpool old-pool vdev device-1 /dev/old_hdd_a ``` We’ll use the removed old_hdd_b, along with my newly purchased new_hdd_a and a virtual device /dev/loop1 to create a new pool: ``` zpool old-pool vdev device-1 /dev/old_hdd_a zpool new-pool vdev raidz-1 /dev/old_hdd_b /dev/new_hdd_a /dev/loop1 ``` We can then copy all our data over to the new-pool, destroy the old-pool, and use the old_hdd_a to replace the virtual device /dev/loop1. ## How-to **WARNING:** During this process, you will have no redundancy. ANY DISK FAILURE WILL RESULT IN LOSING ALL OF YOUR DATA FROM YOUR ENTIRE ZFS POOL. Additionally, messing up a command might result in losing ALL YOUR DATA. You should always have backups. Do not use this process for a business-critical pool. I’m just a home user with a bunch of linux isos and home backups, so I can afford to risk data loss in exchange for buying one less hard drive. Ok, with that out of the way, let’s get on to it. This info is cobbled together from a few forum posts around the web, and this blog post on oracle.com (no longer available, except on the Wayback Machine). ### Baseline zfs setup This guide will use device names assuming your initial pool looks like this in `zpool status -v`: ``` NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 old_hdd_a ONLINE 0 0 0 old_hdd_b ONLINE 0 0 0 ``` Detach one of your mirrored devices ``` > sudo zpool offline tank old_hdd_b > sudo zpool detach tank old_hdd_b ``` Now, `zpool status -v` shows: ``` NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 old_hdd_a ONLINE 0 0 0 ``` ### Create a sparse file and mount it Create the sparse file. This file will look like it is the size you specify, but will take up 0 space on your disk: `> dd if=/dev/zero of=disk1.img bs=1 count=0 seek=10T # Use the size of your largest disk for the `seek=` argument` Mount it as a loopback device: `> sudo losetup -f ./disk1.img` Check the device was mounted, and get its path: ``` > losetup -a # Check in the output of this command to verify the disk was mounted /dev/loop6: []: (/home/marckhouri/disk1.img) ``` ### Create the new pool and offline the virtual device Use the loopback device path from the last step to create a new pool. `> sudo zpool create tankier raidz /dev/loop6 /dev/disk/by-id/old_hdd_b /dev/disk/by-id/new_hdd_a` Immediately take the virtual device offline so we don’t write to it `> sudo zpool offline tankier /dev/loop6` Now, `zpool status -v` shows: ``` NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 old_hdd_a ONLINE 0 0 0 NAME STATE READ WRITE CKSUM tankier DEGRADED 0 0 0 raidz1-0 DEGRADED 0 0 0 loop6 OFFLINE 0 0 0 old_hdd_b ONLINE 0 0 0 new_hdd_a ONLINE 0 0 0 ``` ### Transfer your data ``` > sudo zfs snapshot tank@20200710 > sudo zfs send -Rv tank@20200710 | sudo zfs recv -vsF tankier ``` I saw ~350MBPS transfer speeds for this process. Use `-s` on the `recv` command to allow for session resumption in case the transfer is interrupted ### Destroy the old pool and replace the virtual device with the old device ## Conclusion That’s it! Using a sparse file as a loopback device in your zfs pool is definitely a bad idea if you care about your data, but it can be helpful if you’re a home user who cares enough about saving $150 (or physically can’t fit another drive in your chassis 😬). ## Addendum: full output of all commands as I went through this process My setup wasn’t exactly as described above, these are my details: - I had 3x8TB in raidz1 and 2x10TB in mirror (~26TB usable). - I bought 2 more 10TB drives. - I setup a second pool with a 4x10TB raidz1 vdev made up of one of the 10TB drives from my mirror, the 2 new 10TB drives, and a virtual drive (~30TB usable). - My original pool new had 3x8TB in raidz1 and 1x10TB with no redundancy (still ~26TB usable). - I transferred all my data from the old pool to the new pool - I moved all the devices from my old pool to the new pool, so I ended with 3x8TB in raidz1 and 4x10TB in raidz1 (~46TB usable) Here’s a step by step log showing lots of output: ``` marckhouri@mars:~$ zfs list NAME USED AVAIL REFER MOUNTPOINT tank 19.5T 3.32T 19.3T /tank tank@20200426 141G - 15.6T - tank@20200609 26.5G - 18.0T - marckhouri@mars:~$ zpool status -v pool: tank state: ONLINE scan: resilvered 80K in 0h0m with 0 errors on Fri Jul 10 03:28:20 2020 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HK2UN3N ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HJX81NF ONLINE 0 0 0 ata-WDC_WD80EZAZ-11TDBA0_2YJMUYUD ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_2YJGKGGD ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEGVWWZN ONLINE 0 0 0 cache nvme-HP_SSD_EX900_120GB_HBSE18433200255 ONLINE 0 0 0 marckhouri@mars:~$ sudo zpool offline tank ata-WDC_WD100EMAZ-00WJTA0_JEGVWWZN marckhouri@mars:~$ zpool status pool: tank state: DEGRADED status: One or more devices has been taken offline by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scan: resilvered 80K in 0h0m with 0 errors on Fri Jul 10 03:28:20 2020 config: NAME STATE READ WRITE CKSUM tank DEGRADED 0 0 0 raidz1-0 ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HK2UN3N ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HJX81NF ONLINE 0 0 0 ata-WDC_WD80EZAZ-11TDBA0_2YJMUYUD ONLINE 0 0 0 mirror-1 DEGRADED 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_2YJGKGGD ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEGVWWZN OFFLINE 0 0 0 cache nvme-HP_SSD_EX900_120GB_HBSE18433200255 ONLINE 0 0 0 errors: No known data errors marckhouri@mars:~$ sudo zpool detach tank ata-WDC_WD100EMAZ-00WJTA0_JEGVWWZN marckhouri@mars:~$ zpool status pool: tank state: ONLINE scan: resilvered 80K in 0h0m with 0 errors on Fri Jul 10 03:28:20 2020 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HK2UN3N ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HJX81NF ONLINE 0 0 0 ata-WDC_WD80EZAZ-11TDBA0_2YJMUYUD ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_2YJGKGGD ONLINE 0 0 0 cache nvme-HP_SSD_EX900_120GB_HBSE18433200255 ONLINE 0 0 0 errors: No known data errors marckhouri@mars:~$ dd if=/dev/zero of=disk1.img bs=1 count=0 seek=10T 0+0 records in 0+0 records out 0 bytes copied, 0.000302862 s, 0.0 kB/s marckhouri@mars:~$ losetup -a /dev/loop1: []: (/var/lib/snapd/snaps/core18_1754.snap) /dev/loop4: []: (/var/lib/snapd/snaps/core_9289.snap) /dev/loop2: []: (/var/lib/snapd/snaps/core18_1705.snap) /dev/loop0: []: (/var/lib/snapd/snaps/canonical-livepatch_94.snap) /dev/loop5: []: (/var/lib/snapd/snaps/canonical-livepatch_95.snap) /dev/loop3: []: (/var/lib/snapd/snaps/core_9436.snap) marckhouri@mars:~$ sudo losetup -f ./disk1.img marckhouri@mars:~$ losetup -a /dev/loop1: []: (/var/lib/snapd/snaps/core18_1754.snap) /dev/loop6: []: (/home/marckhouri/disk1.img) /dev/loop4: []: (/var/lib/snapd/snaps/core_9289.snap) /dev/loop2: []: (/var/lib/snapd/snaps/core18_1705.snap) /dev/loop0: []: (/var/lib/snapd/snaps/canonical-livepatch_94.snap) /dev/loop5: []: (/var/lib/snapd/snaps/canonical-livepatch_95.snap) /dev/loop3: []: (/var/lib/snapd/snaps/core_9436.snap) marckhouri@mars:~$ sudo zpool create tankier raidz /dev/loop6 /dev/disk/by-id/ata-WDC_WD100EMAZ-00WJTA0_JEGVWWZN /dev/disk/by-id/ata-WDC_WD100EMAZ-00WJTA0_JEK4EW0N /dev/disk/by-id/ata-WDC_WD100EMAZ-00WJTA0_JEK5W8XN marckhouri@mars:~$ zpool status -v pool: tank state: ONLINE scan: resilvered 80K in 0h0m with 0 errors on Fri Jul 10 03:28:20 2020 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HK2UN3N ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HJX81NF ONLINE 0 0 0 ata-WDC_WD80EZAZ-11TDBA0_2YJMUYUD ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_2YJGKGGD ONLINE 0 0 0 cache nvme-HP_SSD_EX900_120GB_HBSE18433200255 ONLINE 0 0 0 errors: No known data errors pool: tankier state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tankier ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 loop6 ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEGVWWZN ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEK4EW0N ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEK5W8XN ONLINE 0 0 0 errors: No known data errors marckhouri@mars:~$ sudo zpool offline tankier /dev/loop6 marckhouri@mars:~$ zpool status -v pool: tank state: ONLINE scan: resilvered 80K in 0h0m with 0 errors on Fri Jul 10 03:28:20 2020 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HK2UN3N ONLINE 0 0 0 ata-WDC_WD80EMAZ-00WJTA0_7HJX81NF ONLINE 0 0 0 ata-WDC_WD80EZAZ-11TDBA0_2YJMUYUD ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_2YJGKGGD ONLINE 0 0 0 cache nvme-HP_SSD_EX900_120GB_HBSE18433200255 ONLINE 0 0 0 errors: No known data errors pool: tankier state: DEGRADED status: One or more devices has been taken offline by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scan: none requested config: NAME STATE READ WRITE CKSUM tankier DEGRADED 0 0 0 raidz1-0 DEGRADED 0 0 0 loop6 OFFLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEGVWWZN ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEK4EW0N ONLINE 0 0 0 ata-WDC_WD100EMAZ-00WJTA0_JEK5W8XN ONLINE 0 0 0 marckhouri@mars:~$ zfs list NAME USED AVAIL REFER MOUNTPOINT tank 19.5T 3.32T 19.3T /tank tank@20200426 141G - 15.6T - tank@20200609 26.5G - 18.0T - tankier 453K 25.5T 140K /tankier marckhouri@mars:~$ sudo zfs send -Rv tank@20200710 | sudo zfs recv -vsF tankier full send of tank@20200426 estimated size is 15.6T send from @20200426 to tank@20200609 estimated size is 2.52T send from @20200609 to tank@20200710 estimated size is 1.38T total estimated size is 19.5T TIME SENT SNAPSHOT receiving full stream of tank@20200426 into tankier@20200426 06:35:04 205M tank@20200426 06:35:05 544M tank@20200426 06:35:06 881M tank@20200426 06:35:07 1.19G tank@20200426 06:35:08 1.50G tank@20200426 06:35:09 1.77G tank@20200426 [...] trimmed # TODO: add the destroy and migration ```