Skip to content

Commit

Permalink
try to fix formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
Titus-von-Koeller committed Sep 21, 2024
1 parent dfaf9a4 commit ee9e6fb
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/source/non_cuda_backends.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,14 @@ Thank you for your support!
The following performance data is collected from Intel 4th Gen Xeon (SPR) platform. The tables show speed-up and memory compared with different data types of [Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf).

For inference:

| CPU | BF16 | INT8 | NF4 | FP4 |
|---|---|---|---|---|
| speed-up | 1.0x | 0.6x | 2.3x | 0.03x |
| memory | 13.1G | 7.6G | 5.0G | 4.6G |

For fine-tune:

| CPU | AMP BF16 | INT8 | NF4 | FP4 |
|---|---|---|---|---|
| speed-up | 1.0x | 0.38x | 0.07x | 0.07x |
Expand Down

0 comments on commit ee9e6fb

Please sign in to comment.