Skip to content

Commit

Permalink
docs: add cpu benchmark (bitsandbytes-foundation#1366)
Browse files Browse the repository at this point in the history 
* cpu benchmark

* try to fix formatting

* cleanup

* cleanup

---------

Co-authored-by: Titus <9048635+Titus-von-Koeller@users.noreply.github.com>
  • Loading branch information
@jiqing-feng @Titus-von-Koeller @matthewdouglas
2 people authored and matthewdouglas committed Oct 28, 2024
1 parent c6f21bf commit 698645c
Showing 1 changed file with 15 additions and 1 deletion.
16 changes: 15 additions & 1 deletion docs/source/non_cuda_backends.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,18 @@ Thank you for your support!

### Intel

### AMD
The following performance data is collected from Intel 4th Gen Xeon (SPR) platform. The tables show speed-up and memory compared with different data types of [Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf).

#### Inference (CPU)

| Data Type | BF16 | INT8 | NF4 | FP4 |
|---|---|---|---|---|
| Speed-Up (vs BF16) | 1.0x | 0.6x | 2.3x | 0.03x |
| Memory (GB) | 13.1 | 7.6 | 5.0 | 4.6 |

#### Fine-Tuning (CPU)

| Data Type | AMP BF16 | INT8 | NF4 | FP4 |
|---|---|---|---|---|
| Speed-Up (vs AMP BF16) | 1.0x | 0.38x | 0.07x | 0.07x |
| Memory (GB) | 40 | 9 | 6.6 | 6.6 |

0 comments on commit 698645c

Please sign in to comment.