The default model repository of openllm
This repo (on main branch) is already included by openllm by default.
If you want more up-to-date untested models, please add our nightly branch.
openllm repo add nightly https://github.com/bentoml/openllm-models@nightly| Model | Version | Huggingface Link |
|---|---|---|
| llama3.2 | 1b-instruct-fp16-f6ac | HF Link |
| llama3.2 | 3b-instruct-fp16-f88f | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| llama3.1 | 405b-instruct-awq-4bit-54b7 | HF Link |
| llama3.1 | 70b-instruct-awq-4bit-7c3e | HF Link |
| llama3.1 | 70b-instruct-fp16-c283 | HF Link |
| llama3.1 | 8b-instruct-awq-4bit-c135 | HF Link |
| llama3.1 | 8b-instruct-fp16-44b5 | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| phi3 | 3.8b-instruct-fp16-baed | HF Link |
| phi3 | 3.8b-instruct-ggml-q4-50c9 | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| mistral | 24b-instruct-nemo-e7a4 | HF Link |
| mistral | 7b-instruct-awq-4bit-4175 | HF Link |
| mistral | 7b-instruct-fp16-9926 | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| gemma2 | 27b-instruct-fp16-56d1 | HF Link |
| gemma2 | 9b-instruct-fp16-bf96 | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| qwen2.5 | 0.5b-instruct-fp16-5a57 | HF Link |
| qwen2.5 | 1.5b-instruct-fp16-6c7f | HF Link |
| qwen2.5 | 14b-instruct-fp16-8d25 | HF Link |
| qwen2.5 | 32b-instruct-fp16-0862 | HF Link |
| qwen2.5 | 3b-instruct-fp16-7eb6 | HF Link |
| qwen2.5 | 72b-instruct-fp16-b679 | HF Link |
| qwen2.5 | 7b-instruct-fp16-de92 | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| mixtral | 8x7b-instruct-v0.1-awq-4bit-2117 | HF Link |
| mixtral | 8x7b-instruct-v0.1-fp16-55c3 | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| mistral-large | 123b-instruct-awq-4bit-e339 | HF Link |
| mistral-large | 123b-instruct-fp16-eb4a | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| codestral | 22b-v0.1-fp16-0d5b | HF Link |
| Model | Version | Huggingface Link |
|---|---|---|
| llama3 | 70b-instruct-awq-4bit-e96c | HF Link |
| llama3 | 70b-instruct-fp16-45fe | HF Link |
| llama3 | 8b-instruct-awq-4bit-b159 | HF Link |
| llama3 | 8b-instruct-fp16-72f8 | HF Link |