llama.cpp: add IQ3_XXS quantization models (#8)

* doc: add iq3_xxs perf.

---------

Co-authored-by: ymcui <[email protected]>

Author: ymcui

README.md

@@ -188,13 +188,13 @@ Mixtral是一个稀疏混合专家模型。该模型与以往的LLaMA等主流
 
 在llama.cpp下，测试了Chinese-Mixtral量化版模型的性能，如下表所示。
 
-|              |  F16 |   Q8_0 |   Q6_K |   Q5_K |   Q5_0 |   Q4_K |   Q4_0 |   Q3_K |   Q2_K | IQ2_XS | IQ2_XXS |
-| ------------ | ---: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | ------: |
-| Size (GB)    | 87.0 |   46.2 |   35.7 |   30.0 |   30.0 |   24.6 |   24.6 |   19.0 |   16.1 |   12.7 |    11.4 |
-| BPW          | 16.0 |   8.50 |   6.57 |   5.69 |   5.52 |   4.87 |   4.53 |   3.86 |   2.96 |   2.34 |    2.10 |
-| PPL          |    - | 4.4076 | 4.4092 | 4.4192 | 4.4224 | 4.4488 | 4.4917 | 4.5545 | 5.1846 | 6.9784 |  8.5981 |
-| M3 Max Speed |    - |      - |   36.0 |   36.9 |   35.7 |   31.2 |   27.8 |   37.6 |   29.1 |      - |       - |
-| A100 Speed   |    - |      - |   29.9 |   22.6 |   20.5 |   21.7 |   17.1 |   21.7 |   20.3 |   23.7 |    22.5 |
+|              |  F16 |   Q8_0 |   Q6_K |   Q5_K |   Q5_0 |   Q4_K |   Q4_0 |   Q3_K | IQ3_XXS |   Q2_K | IQ2_XS | IQ2_XXS |
+| ------------ | ---: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | ------: | -----: | -----: | ------: |
+| Size (GB)    | 87.0 |   46.2 |   35.7 |   30.0 |   30.0 |   24.6 |   24.6 |   19.0 |    17.1 |   16.1 |   12.7 |    11.4 |
+| BPW          | 16.0 |   8.50 |   6.57 |   5.69 |   5.52 |   4.87 |   4.53 |   3.86 |    3.14 |   2.96 |   2.34 |    2.10 |
+| PPL          |    - | 4.4076 | 4.4092 | 4.4192 | 4.4224 | 4.4488 | 4.4917 | 4.5545 |  4.5990 | 5.1846 | 6.9784 |  8.5981 |
+| M3 Max Speed |    - |      - |   36.0 |   36.9 |   35.7 |   31.2 |   27.8 |   37.6 |       - |   29.1 |      - |       - |
+| A100 Speed   |    - |      - |   29.9 |   22.6 |   20.5 |   21.7 |   17.1 |   21.7 |    20.6 |   20.3 |   23.7 |    22.5 |
 
 > [!NOTE]
 >

README_EN.md

@@ -184,17 +184,17 @@ To evaluate the effectiveness of the related models, this project conducted both
 | Chinese-Alpaca-2-7B-64K                                      |     44.7      |     28.1     |     14.4      |       39.0        |      44.6       |      5.0       | 29.3 |
 | Chinese-LLaMA-2-7B-64K                                       |     27.2      |     16.4     |      6.5      |       33.0        |       7.8       |      5.0       | 16.0 |
 
-### Quantitative Effect Evaluation
+### Quantization Effect Evaluation
 
 Under llama.cpp, the performance of the quantized version of the Chinese-Mixtral model was tested, as shown in the table below.
 
-|              |  F16 |   Q8_0 |   Q6_K |   Q5_K |   Q5_0 |   Q4_K |   Q4_0 |   Q3_K |   Q2_K | IQ2_XS | IQ2_XXS |
-| ------------ | ---: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | ------: |
-| Size (GB)    | 87.0 |   46.2 |   35.7 |   30.0 |   30.0 |   24.6 |   24.6 |   19.0 |   16.1 |   12.7 |    11.4 |
-| BPW          | 16.0 |   8.50 |   6.57 |   5.69 |   5.52 |   4.87 |   4.53 |   3.86 |   2.96 |   2.34 |    2.10 |
-| PPL          |    - | 4.4076 | 4.4092 | 4.4192 | 4.4224 | 4.4488 | 4.4917 | 4.5545 | 5.1846 | 6.9784 |  8.5981 |
-| M3 Max Speed |    - |      - |   36.0 |   36.9 |   35.7 |   31.2 |   27.8 |   37.6 |   29.1 |      - |       - |
-| A100 Speed   |    - |      - |   29.9 |   22.6 |   20.5 |   21.7 |   17.1 |   21.7 |   20.3 |   23.7 |    22.5 |
+|              |  F16 |   Q8_0 |   Q6_K |   Q5_K |   Q5_0 |   Q4_K |   Q4_0 |   Q3_K | IQ3_XXS |   Q2_K | IQ2_XS | IQ2_XXS |
+| ------------ | ---: | -----: | -----: | -----: | -----: | -----: | -----: | -----: | ------: | -----: | -----: | ------: |
+| Size (GB)    | 87.0 |   46.2 |   35.7 |   30.0 |   30.0 |   24.6 |   24.6 |   19.0 |    17.1 |   16.1 |   12.7 |    11.4 |
+| BPW          | 16.0 |   8.50 |   6.57 |   5.69 |   5.52 |   4.87 |   4.53 |   3.86 |    3.14 |   2.96 |   2.34 |    2.10 |
+| PPL          |    - | 4.4076 | 4.4092 | 4.4192 | 4.4224 | 4.4488 | 4.4917 | 4.5545 |  4.5990 | 5.1846 | 6.9784 |  8.5981 |
+| M3 Max Speed |    - |      - |   36.0 |   36.9 |   35.7 |   31.2 |   27.8 |   37.6 |       - |   29.1 |      - |       - |
+| A100 Speed   |    - |      - |   29.9 |   22.6 |   20.5 |   21.7 |   17.1 |   21.7 |    20.6 |   20.3 |   23.7 |    22.5 |
 
 > [!NOTE]
 >