Skip to content

Commit

Permalink
Merge pull request #17 from IANTHEREAL/patch-1
Browse files Browse the repository at this point in the history
Fix README
  • Loading branch information
AkariAsai authored Nov 14, 2023
2 parents 10bf8de + 5fb6aca commit d4f4b8c
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ cd retrieval_lm
bash download_demo_corpus.sh
```
If the script does not work, you can download the data from [google drive](https://drive.google.com/file/d/1IYNAkwawfCDiBL27BlBqGssxFQH9vOux/view?usp=share_link) or [HF dataset](https://huggingface.co/datasets/selfrag/selfrag_train_data).
Then, you can run the script under `retrieval_lm`. We tested the script using on 1 RTRTX 6000 with 24GB and 100G RAM (but should be runnable with much smaller RAM).
Then, you can run the script under `retrieval_lm`. We tested the script using on 1 RTX 6000 with 24GB and 100G RAM (but should be runnable with much smaller RAM).

```py
from passage_retriever import Retriever
Expand Down Expand Up @@ -171,9 +171,9 @@ for i in {0..3}; do
**Self-RAG** trains two models, *Critic* and *Generator*, both of which expand token vocabularies with reflection tokens and are trained with the standard next token prediction objective.

- [Step 1: Critic Data Creation](#collect-reflection-tokens): Generating Critic training data with GPT4.
- [Step 2: Critic Training](#critic-training): Generating Critic training data with GPT4.
- [Step 2: Critic Training](#critic-training): Training a Critic with new special tokens.
- [Step 3: Generator Data Creation](#generator-data-creation): Generating Generator training data using Critic and Retriever.
- [Step 4: Generator Training](#generator-training): Training a Critic / Generator with new special tokens.
- [Step 4: Generator Training](#generator-training): Training a Generator with new special tokens.

Alternatively, you can download our training data consisting of 150K instances [here](https://drive.google.com/file/d/10G_FozUV4u27EX0NjwVe-3YMUMeTwuLk/view?usp=share_link).

Expand Down

0 comments on commit d4f4b8c

Please sign in to comment.