Move Metadata to Special vdev

[tcpluess]

Hi,
I have a zpool that is >50% full, with more than 30 million files.
I wanted to upgrade my pool a bit by adding a mirrored special vdev. I know that ZFS never rewrites data that is already on disk, so I realise that I will not profit from my special vdev as the existing 30 million files will still keep their metadata on the hard drives.

I also know that it is not possible to manually move the metadata to the new vdev, unless I send and receive the entire pool (or at least the dataset...) and destroy the old pool (or dataset).

However, I am stuck now: I am more than 50% full, i.e. I cannot send because there is nowhere I can send the data to.
I believe I can also somehow force ZFS to "rewrite" the metadata of a file by copying the file, deleting the original and then renaming the file.

Theoretically, this would be acceptable, but with this approach I also have some problem: the backup software being used (IBM Tivoli) will recognise the copied files as modified or new files, even if I use cp --preserve=all (I think because the file's "birth" date is today's date) and will then, in the next run, backup that file. So if I would do this for my entire pool, this would give me some 100 hours of backup runtime and 40TB of data transferred, even though actually nothing changed!

Therefore I am wondering. Is it technically possible to force the moving of metadata to the special device?
I believe it should be possible, because when a vdev is removed, its data is also copied over, so it should also be possible the other way round. No?

My problem at the moment is that I am unsure how much performance for listing/traversing/searching through the files I would gain by adding the special device, and I cannot easily test it.

I also believe such a feature of moving the metadata could be an useful addition, I am certainly not the only one on the world having this problem :-)

[MarcelRobitaille]

@tcpluess I'm having the same problem. Did you find any solution?

[tcpluess]

Good day,
unfortunately, I didn't find a good solution, as you see, I didn't even get a response to my message.
I ended up using this

https://github.com/markusressel/zfs-inplace-rebalancing/blob/master/zfs-inplace-rebalancing.sh

which does re-write your files in-place. It works, but as mentioned in my first post, the file "birth" date is modified, and even though all other parameters, like ACLs and so on, are kept, this might trigger a backup program such that it thinks "this file is new" and needs to be backupped.

So if you can live with this, I can recommend this above linked shell script. If this is not acceptable, then, unfortunately, there is no way except zfs send/recv, with the limitation that it only works when you have enough free space.

[no-usernames-left]

@jimsalterjrs might know the answer; you may wish to post this question at https://discourse.practicalzfs.com/.