Update README.md

AkariAsai · Nov 1, 2023 · ade0af7 · ade0af7
1 parent 6b11901
commit ade0af7
Showing 1 changed file with 11 additions and 3 deletions.
diff --git a/README.md b/README.md
@@ -2,7 +2,7 @@
 
 This includes the original implementation of [SELF-RAG: Learning to Retrieve, Generate and Critique through self-reflection](https://arxiv.org/abs/2310.11511) by Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 
 
-[Website](https://selfrag.github.io/) | [7B Model](https://huggingface.co/selfrag/selfrag_llama2_7b) | [13B Model](https://huggingface.co/selfrag/selfrag_llama2_13b) | [Paper](https://akariasai.github.io/files/adaptive_retrieval_augmented_lm_arxiv.pdf) | [Updates](#updates)
+[Website](https://selfrag.github.io/) | [7B Model](https://huggingface.co/selfrag/selfrag_llama2_7b) | [13B Model](https://huggingface.co/selfrag/selfrag_llama2_13b) | [Paper](https://akariasai.github.io/files/adaptive_retrieval_augmented_lm_arxiv.pdf) | [Training data](https://huggingface.co/datasets/selfrag/selfrag_train_data) | [Twitter summary](https://twitter.com/AkariAsai/status/1715110277077962937) | [Updates](#updates) 
 
 **Self-RAG** (Figure right) is a new framework to train an arbitrary LM to learn to retrieve, generate, and critique to enhance the factuality and quality of generations, without hurting the versatility of LLMs. 
 
@@ -33,7 +33,8 @@ If you find our code, data, models, or the paper useful, please cite the paper:
 2. [Retriever setup](#retriever-setup)
 3. [Training](#training) 
 4. [Inference](#inference)
-5. [Contact](#contact)
+5. [FAQ](#faq)
+6. [Contact](#contact)
 
 
 ## Installation
@@ -98,7 +99,7 @@ git clone git@github.com:AkariAsai/self-rag.git
 cd retrieval_lm
 bash download_demo_corpus.sh
 ```
-If the script does not work, you can download the data from [google drive](https://drive.google.com/file/d/1IYNAkwawfCDiBL27BlBqGssxFQH9vOux/view?usp=share_link). 
+If the script does not work, you can download the data from [google drive](https://drive.google.com/file/d/1IYNAkwawfCDiBL27BlBqGssxFQH9vOux/view?usp=share_link) or [HF dataset](https://huggingface.co/datasets/selfrag/selfrag_train_data). 
 Then, you can run the script under `retrieval_lm`. We tested the script using on 1 RTRTX 6000 with 24GB and 100G RAM (but should be runnable with much smaller RAM).
 
 ```py
@@ -340,5 +341,12 @@ For ASQA, you can run evaluations as follows. Note that ASQA evaluations require
 python eval.py --f YOUR_OUTPUT_FILE --citations --qa --mauve
 ```
 
+## FAQ
+**Q1: How can I train a new pre-trained LM using Self-RAG scheme?** -- If you are using huggingface transformers, you can simply change the `model_name_or_path` and `tokenizer_name` in our training script, [script_finetune_7b.sh](https://github.com/AkariAsai/self-rag/blob/main/retrieval_lm/script_finetune_7b.sh). If you want to use your own fine-tuning script, please make sure to add the special tokens and mask out the paragraph context, as discussed in [this issue](https://github.com/AkariAsai/self-rag/issues/12)
+
+**Q2: Are you planning to release Mistral-7B-based Self-RAG?** -- Right now I have limited bandwidth to do so, but there is a community trained version of Self-RAG [SciPhi-Self-RAG-Mistral-7B-32k](https://huggingface.co/SciPhi/SciPhi-Self-RAG-Mistral-7B-32k) on top of Mistral-7B. We will announce if we can train Self-RAG on Mistral-7B and release the checkpoint.  
+
+
+
 ## Contact
 If you have questions, please open an issue mentioning @AkariAsai or send an email to akari[at]cs.washington.edu.