options for 2x Intel DC P4510 4TB U.2

[Hr46ph]

Hello everyone!

I would like to seek some advise. While I have an idea what I want I would like to ask what you would do in my case. In particular why and how of your idea.

I have a home server:
Asus Z11PA-U12/10G-2S
Xeon Silver 4210 10C/20T
192 GB ECC RAM
M2 NVMe Boot disk
IBM / LSI M1015 HBA
8x SATA HGST Ultrastar 8TB in 4 mirror pairs

Server runs Arch Linux, LTS kernel with ZFS 2.1.12-1, KVM/libvirt with typically 5 to 10 VMs, and offers SMB and NFS shares for other hosts.

VM's run a docker standalone, a docker swarm, a kubernetes cluster, a couple of VM's. The containers in the docker standalone are important. They host critical things. The rest is playground.

The spinning rust is slow. I don't think it is slower than what we can expect for spinning rust, its just spinning disks and they are slow compared to SSD's.

So I now I have 2x Intel DC P4510 4TB U.2 to play around with. Brand new. They will connect to the motherboard 2 OCUlink ports. Personally I see two viable options:

• Create a new pool with a mirror and use the 4 TB as a separate pool for VM's
• Special vdev for small blocks and metadata to speed up the spinning rust pool

I'd love to hear how you would use them and in particular the how and why.

Cheers!