DX12 is totally weird about samplers. At any one time there may be exactly one sampler heap bound to a command buffer. Each sampler heap may contain no more than 2048 sampler descriptors. Currently we only ever use a single sampler heap. This means that the maximum amount of sampler bindings we can allocate in all live bind groups is 2048. This is extremely restrictive, especially when compared to other apis.

You can switch which sampler heap is bound to the command buffer at any time, but according to the command list documentation this may be expensive and cause flushes.

We need a better strategy of dealing with sampler descriptors. I suspect this will involve some kind of sampler descriptor cache where we try to re-use the same sampler heap as much as possible, only overflowing if we run out of space, then just switching sampler heaps as rarely as possible.