TrueNAS has long had the ability to expand a pool (or vdev) by replacing each of its disks with larger ones. Unfortunately, TrueNAS SCALE 23.10.0 and 23.10.1 have a bug that results in this procedure not working. One way to work around this bug is to do the replacement from the command line. This involves three basic steps:
All of these will need to be done at the command line as the root user.
If you've been bitten by this bug (i.e., if you've already replaced disks in a vdev with larger ones, and haven't seen your pool expand), there's no need to go through the procedure below. Instead, just run lsblk to determine the partition number for data on your disk--it will be the largest partition, and by default with TrueNAS SCALE, it will be partition 1. Then run parted /dev/<device> resizepart <num> 100%. If it's partition 1 on /dev/sdz, the command would be parted /dev/sdz resizepart 1 100%. Repeat for each of the disks.
You'll partition the disks using the parted command. For each disk, start by running parted /dev/<device>. Then make a new GPT partition table by running mklabel gpt. You'll next create a 2 GB swap partition by running mkpart "" linux-swap 1 2048. This partition size is the default for TrueNAS; if you've set your swap partition size to something different, you may want to modify this command (and the next) accordingly. Next, create the main data partition by running mkpart "" zfs 2048 -1--the -1 will use the last sector on the disk as the end of the partition, thereby using all the remaining space on the disk. Run print to make sure the partition table is created correctly, and if it is, quit.
The complete output will look like this:
root@truenas[~]# parted /dev/sdz
GNU Parted 3.5
Using /dev/sdz
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Error: /dev/sdz: unrecognised disk label
Model: ATA ST16000NM000J-2T (scsi)
Disk /dev/sdz: 16.0TB
Sector size (logical/physical): 512B/4096B
Partition Table: unknown
Disk Flags:
(parted) mklabel gpt
(parted) mkpart "" linux-swap 1 2048
(parted) mkpart "" zfs 2048 -1
(parted) print
Model: ATA ST16000NM000J-2T (scsi)
Disk /dev/sdz: 16.0TB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 2048MB 2047MB linux-swap(v1) swap
2 2048MB 16.0TB 16.0TB zfs
(parted) quit
Information: You may need to update /etc/fstab.More information can be found about parted in its manual.
TrueNAS identifies member devices in a ZFS pool using the partition UUIDs, which prevents issues when disks' identifiers change (e.g., from /dev/sdc to /dev/sdg). You'll need to determine the UUIDs of your newly-created partitions before beginning the disk replacement. To do this, run lsblk -o NAME,SIZE,PARTUUID. The command output will look like this:
sdaa 14.6T
|-sdaa1 1.9G 74acda06-1302-4a9f-ad94-23355db0b481
`-sdaa2 14.6T 941cb1d0-f341-47b1-b9e5-2dd21568eee2The second UUID is the one you want, as it refers to the main data partition you created above. Note this UUID, along with those for all the other data partitions you created above.
Now, you'll need to kick off the disk replacement process itself using the zpool replace command. Find the UUID of the first disk you want to replace (the output of zpool status will list all the UUIDs that are currently part of your pool). Then run zpool replace <poolname> /dev/disk/by-partuuid/<olduuid> /dev/disk/by-partuuid/<newuuid> (e.g., zpool replace tank /dev/disk/by-partuuid/15bef7ff-980f-4afb-add9-ffe5c5dd6c6d /dev/disk/by-partuuid/e4ea41e6-230a-4df5-b856-6503c23955fd). This will begin the disk replacement process, resilvering the new disk into your pool, and return you to the shell prompt momentarily.