Hi @aarnphm 👋 I have a quick question about this PR - I saw that the following line was added to the README in this PR:

To use the vLLM backend, you need a GPU with at least the Ampere architecture or newer and CUDA version 11.8.

I have dug through vLLM's documentation, and vLLM supports pre-Ampere architectures such as volta out-of-the-box.

Is there any documentation about where this limitation came from? My assumption is that it's because the bfloat16 data type is being used, but is it possible that I could add a configuration for vLLM that doesn't use that data type? Or does it have to do with pre-compiled kernels being used?