support mistral (mindspore-lab#791)

Geaming2002 · Dec 9, 2023 · 4678ea5 · 4678ea5
1 parent 9bca4b1
commit 4678ea5
Show file tree

Hide file tree

Showing 12 changed files with 1,789 additions and 12 deletions.
diff --git a/README.md b/README.md
@@ -90,50 +90,55 @@ The master branch works with **MindSpore master**.
 
 The table below represents the current support in the library for each of those models, whether they have support in Pynative mode or Graph mode.
 
-| Model                         | Pynative support | Graph Support |
-|-------------------------------|------------------|---------------|
+| Model                         | Pynative support    | Graph Support |
+|-------------------------------|---------------------|---------------|
 | ALBERT                        | ✅                | ✅             |
-| Autoformer                    | TODO              | ❌             |
+| Autoformer                    | ✅ (Inference only)| ❌             |
 | BaiChuan                      | ✅                | ❌             |
 | Bark                          | TODO                | ❌             |
 | BART                          | ✅                | ❌             |
 | BERT                          | ✅                | ✅             |
+| BLIP                          | TODO              | ✅             |
+| BLIP2                         | TODO              | ✅             |
 | BLOOM                         | ✅                | ❌             |
-| CLIP                          | ✅                | ❌             |
 | ChatGLM                       | ✅                | ❌             |
+| CLIP                          | ✅                | ❌             |
 | CodeGen                       | ✅                | ❌             |
 | ConvBERT                      | TODO              | ❌             |
 | CPM                           | ✅                | ❌             |
 | CPM-Ant                       | ✅                | ❌             |
 | CPM-Bee                       | ✅                | ❌             |
 | EnCodec                       | TODO               | ❌             |
 | ERNIE                         | ✅                | ❌             |
-| Falcon                        | TODO                | ❌             |
+| Falcon                        | ✅                | ❌             |
 | GLM                           | ✅                | ❌             |
+| OpenAI GPT                    | ✅                | ❌             |
+| OpenAI GPT-2                  | ✅                | ✅             |
 | GPT Neo                       | ✅                | ❌             |
 | GPT NeoX                      | TODO                | ❌             |
 | GPTBigCode                    | ✅                | ❌             |
-| Graphormer                    | TODO               | ❌             |
+| Graphormer                    | ✅               | ❌             |
 | Llama                         | ✅                | ❌             |
 | Llama2                        | ✅                | ❌             |
 | CodeLlama                     | ✅                | ❌             |
 | Longformer                    | ✅                | ❌             |
-| LongT5                        | TODO               | ❌             |
+| LongT5                        | ✅                | ❌             |
 | LUKE                          | ✅                | ❌             |
 | MaskFormer                    | ✅                | ❌             |
 | mBART-50                      | ✅                | ❌             |
 | Megatron-BERT                 | ✅                | ❌             |
 | Megatron-GPT2                 | ✅                | ❌             |
 | MobileBERT                    | ✅                | ❌             |
 | Moss                          | ✅                | ❌             |
-| OpenAI GPT                    | ✅                | ❌             |
-| OpenAI GPT-2                  | ✅                | ✅             |
+| Nezha                         | ✅                | ❌             |
 | OPT                           | ✅                | ❌             |
 | Pangu                         | ✅                | ❌             |
+| Pop2piano                     | Todo              | ❌             |
 | RoBERTa                       | ✅                | ✅             |
 | RWKV                          | ✅                | ❌             |
 | T5                            | ✅                | ❌             |
 | TimeSformer                   | TODO               | ❌             |
+| Tinybert                      | ✅                | ❌             |
 | Whisper                       | ✅                | ❌             |
 | XLM                           | ✅                | ❌             |
 | XLM-RoBERTa                   | ✅                | ❌             |

diff --git a/mindnlp/configs.py b/mindnlp/configs.py
@@ -16,6 +16,9 @@
 Global configs
 """
 import os
+import mindspore
+
+DEFAULT_DTYPE = mindspore.float32
 
 WEIGHTS_NAME = "mindspore.ckpt"
 PT_WEIGHTS_NAME = "pytorch_model.bin"