This does sound like it could be solved with better installDiskSelectors[0]. Talos has done a fair bit of work in improving this and UserVolumeConfigs in the last couple of 1.x revisions.

Alternatively, network booting in some fashion is an option. [1]

[0] https://www.talos.dev/v1.11/reference/configuration/v1alpha1...

[1] https://www.talos.dev/v1.11/talos-guides/install/bare-metal-...