The data written to this format has to be in RGBA order (look for "Texel Formats for Storage Textures" table here).

D3D12 swizzles the reads/writes automatically (see gpuweb/gpuweb#3640 (comment)).
Metal also swizzles (see gpuweb/gpuweb#3640 (comment)).

For SPIR-V, use the Unknown image format and request the StorageImageWriteWithoutFormat capability on writes.
I don't think Vulkan swizzles (or I can't find it does) so we will probably have to swizzle the writes ourselves.