Code

[Algorithm] RLHF end-to-end, clean#1597

Code

Merged

vmoens merged 52 commits intomainpytorch/rl:mainfrom rlhf-example-refactorpytorch/rl:rlhf-example-refactorCopy head branch name to clipboardOct 5, 2023

+1,402-38

Commits

Commits on Jun 27, 2023

• RLHF end2end example

  apbard

  committed
  f24281c

  Copy full SHA for f24281c

Commits on Jun 28, 2023

• add VmapModule and from_lmhead_model method

  apbard

  committed
  ef3f76f

  Copy full SHA for ef3f76f

• Update examples/rlhf/train_rlhf.py

  Show description for 02a909b
  

  apbard

  and

  vmoens

  authored
  02a909b

  Copy full SHA for 02a909b

• addressing comments

  apbard

  committed
  953e4af

  Copy full SHA for 953e4af

• Merge remote-tracking branch 'origin/main' into rlhf-networks

  vmoens

  committed
  ffb8661

  Copy full SHA for ffb8661

• Update torchrl/modules/tensordict_module/common.py

  vmoens

  authored
  f43faea

  Copy full SHA for f43faea

• Update torchrl/modules/tensordict_module/actors.py

  vmoens

  authored
  69b0588

  Copy full SHA for 69b0588

Commits on Jun 29, 2023

• Add RolloutFromModel class

  tcbegley

  committed
  b6fecbb

  Copy full SHA for b6fecbb

• Add rollout tests

  tcbegley

  committed
  bd8fbb6

  Copy full SHA for bd8fbb6

• Apply suggestions from code review

  Show description for 6fbb603
  

  tcbegley

  and

  apbard

  committed
  6fbb603

  Copy full SHA for 6fbb603

• Address comments

  tcbegley

  committed
  3e80a55

  Copy full SHA for 3e80a55

• Docstring lint

  tcbegley

  committed
  385ac90

  Copy full SHA for 385ac90

• Apply suggestions from code review

  Show description for 8d0a152
  

  tcbegley

  and

  vmoens

  committed
  8d0a152

  Copy full SHA for 8d0a152

• Address comments

  tcbegley

  committed
  fcddc97

  Copy full SHA for fcddc97

• Fix tests

  tcbegley

  committed
  5c7c72e

  Copy full SHA for 5c7c72e

• Handle missing transformers import

  tcbegley

  committed
  92d5757

  Copy full SHA for 92d5757

• Import transformers locally

  tcbegley

  committed
  eec0eaf

  Copy full SHA for eec0eaf

• lint

  

  vmoens

  authored and

  tcbegley

  committed
  87501ea

  Copy full SHA for 87501ea

• Merge branch 'rlhf-rollout' into rlhf-example

  tcbegley

  committed
  043fcf6

  Copy full SHA for 043fcf6

• Merge branch 'rlhf-networks' into rlhf-example

  tcbegley

  committed
  3f53046

  Copy full SHA for 3f53046

• lint

  tcbegley

  committed
  8b69e41

  Copy full SHA for 8b69e41

• Example bugfixes

  tcbegley

  committed
  24eaa3a

  Copy full SHA for 24eaa3a

• Move KL controller logic

  tcbegley

  committed
  fba43a1

  Copy full SHA for fba43a1

Commits on Jul 4, 2023

• Merge branch 'main' into rlhf-example

  Show description for 20fa920
  vmoens

  committed
  20fa920

  Copy full SHA for 20fa920

• amend

  vmoens

  committed
  c07ac93

  Copy full SHA for c07ac93

Commits on Jul 5, 2023

• addressing comments about klcontroller

  apbard

  committed
  f463e0e

  Copy full SHA for f463e0e

Commits on Sep 5, 2023

• Merge remote-tracking branch 'origin/main' into rlhf-example

  vmoens

  committed
  eac5374

  Copy full SHA for eac5374

Commits on Oct 1, 2023

• Merge remote-tracking branch 'origin/main' into rlhf-example

  vmoens

  committed
  8d2dde7

  Copy full SHA for 8d2dde7

Commits on Oct 2, 2023

• Merge branch 'main' into rlhf-example

  vmoens

  committed
  a2ba045

  Copy full SHA for a2ba045

• amend

  vmoens

  committed
  a9b94f0

  Copy full SHA for a9b94f0

Commits on Oct 3, 2023

• init

  vmoens

  committed
  d983ebd

  Copy full SHA for d983ebd

• readme

  vmoens

  committed
  097c443

  Copy full SHA for 097c443

• amend

  vmoens

  committed
  0efd93a

  Copy full SHA for 0efd93a

• amend

  vmoens

  committed
  fba9f03

  Copy full SHA for fba9f03

Commits on Oct 4, 2023

• amend

  vmoens

  committed
  cc535e5

  Copy full SHA for cc535e5

• amend

  vmoens

  committed
  28c116f

  Copy full SHA for 28c116f

• amend

  vmoens

  committed
  0f128a6

  Copy full SHA for 0f128a6

• amend

  vmoens

  committed
  e0ad043

  Copy full SHA for e0ad043

• Merge remote-tracking branch 'origin/main' into rlhf-example-refactor

  vmoens

  committed
  e8cad9b

  Copy full SHA for e8cad9b

• amend

  vmoens

  committed
  c93c134

  Copy full SHA for c93c134

Commits on Oct 5, 2023

• init

  vmoens

  committed
  56f7597

  Copy full SHA for 56f7597

• Merge branch 'refactor_ddpg_loss' into rlhf-example-refactor

  vmoens

  committed
  3fa6ea5

  Copy full SHA for 3fa6ea5

• amend

  vmoens

  committed
  c1c41dc

  Copy full SHA for c1c41dc

• amend

  vmoens

  committed
  880e5b4

  Copy full SHA for 880e5b4

• Update run_test.sh

  vmoens

  authored
  d36ce77

  Copy full SHA for d36ce77

• amend

  vmoens

  committed
  942b311

  Copy full SHA for 942b311

• amend

  vmoens

  committed
  fca9f7b

  Copy full SHA for fca9f7b

• lint

  vmoens

  committed
  6362715

  Copy full SHA for 6362715

• amend

  vmoens

  committed
  e3b2d4f

  Copy full SHA for e3b2d4f

• amend

  vmoens

  committed
  7918f86

  Copy full SHA for 7918f86

• Merge remote-tracking branch 'origin/main' into rlhf-example-refactor

  vmoens

  committed
  9658a44

  Copy full SHA for 9658a44

• lint

  vmoens

  committed
  eb041a4

  Copy full SHA for eb041a4