Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] RLHF dataloading #1309

Merged
merged 25 commits into from
Jun 27, 2023
Merged

[Feature] RLHF dataloading #1309

Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
3f1f0ee
init
vmoens Jun 18, 2023
38a1aef
amend
vmoens Jun 18, 2023
c554967
init
vmoens Jun 23, 2023
ce04a5f
amend
vmoens Jun 23, 2023
efc6216
amend
vmoens Jun 23, 2023
d782ee7
Merge branch 'main' into tensorstorage
vmoens Jun 23, 2023
3ccde0e
Merge branch 'tensorstorage' into rlhf_data
vmoens Jun 23, 2023
21ae0df
amend
vmoens Jun 23, 2023
41b7cc6
amend
vmoens Jun 26, 2023
42c0ac9
amend
vmoens Jun 27, 2023
6044278
amend
vmoens Jun 27, 2023
dc0c06f
amend
vmoens Jun 27, 2023
3e9ab03
fix transformers import
vmoens Jun 27, 2023
77e3d86
lint
vmoens Jun 27, 2023
b109e23
Update test/assets/generate.py
vmoens Jun 27, 2023
735e5bb
Update torchrl/data/rlhf/reward.py
vmoens Jun 27, 2023
2370207
Update torchrl/data/rlhf/dataset.py
vmoens Jun 27, 2023
461c371
TokenizedDatasetLoader
vmoens Jun 27, 2023
48659ee
typo
vmoens Jun 27, 2023
0493d26
lint + doc + tests
vmoens Jun 27, 2023
0fb5514
refactor unzip
vmoens Jun 27, 2023
8fc7dd5
Update torchrl/data/rlhf/dataset.py
vmoens Jun 27, 2023
1a3e6fb
amend
vmoens Jun 27, 2023
b9b0a6d
Merge remote-tracking branch 'origin/rlhf_data' into rlhf_data
vmoens Jun 27, 2023
4ffcf72
amend
vmoens Jun 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
amend
  • Loading branch information
@vmoens
vmoens committed Jun 27, 2023
commit 4ffcf72b59dd794d5a18a26761e1f61b8edc5d33
10 changes: 8 additions & 2 deletions torchrl/data/rlhf/reward.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
from tensordict import tensorclass

from torchrl.data.rlhf.dataset import TensorDictTokenizer, TokenizedDatasetLoader
from tqdm import tqdm

DEFAULT_DATASET = "CarperAI/openai_summarize_comparisons"
_has_datasets = importlib.util.find_spec("datasets") is not None
_has_tqdm = importlib.util.find_spec("tqdm") is not None


@tensorclass
Expand Down Expand Up @@ -197,7 +197,13 @@ def pre_tokenization_hook(dataset, min_length=5):

chosen = []
rejected = []
for sample in tqdm(dataset):
if _has_tqdm:
from tqdm import tqdm

pbar = tqdm(dataset)
else:
pbar = dataset
for sample in pbar:
prompt = sample["prompt"]
chosen_summary = sample["chosen"]
rejected_summary = sample["rejected"]
Expand Down