Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Process reward modeling support #362

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

Process reward modeling support #362

wants to merge 19 commits into from

Conversation

fabrahman
Copy link
Contributor

@fabrahman fabrahman commented Sep 21, 2024 •
edited
Loading

This is a commit to support process reward modeling includes supports for:

  1. processing prm formatted data,
  2. computing process rewards at the end of each step, and
  3. doing cross entropy over predicted logits over end of step tokens wrt labels.

Currently testing it here: https://beaker.org/ex/01J8AK58TPQ48SDH5AKSHVTZMK/tasks/01J8AK58TV4X7Y2VPQ9PG297WP/job/01J8AK5902D3935VH6SQMEMFV0

TODOs: refactoring the evaluation code for this prm task

Faeze Brahman and others added 15 commits August 30, 2024 17:57
modified for handling specific inputs
b37166e
Merge branch 'main' of https://github.com/allenai/open-instruct into …
681a0e1 
…main

Pull recent updates
ierge branch 'main' of https://github.com/allenai/open-instruct into …
1e2ddff 
…main

Merged new evals
Merge branch 'main' of https://github.com/allenai/open-instruct into …
44e1d31 
…main

merge with main
use older image
8f2019f
Merge branch 'main' of https://github.com/allenai/open-instruct into …
768035c 
…main
Merge branch 'main' of https://github.com/allenai/open-instruct into …
1278cff 
…main
use new image
41f8f17
Merge branch 'main' of https://github.com/allenai/open-instruct into …
c44fb66 
…main
Merge branch 'main' of https://github.com/allenai/open-instruct into …
22826ca 
…main
have conda setup
bf1f14d
added support for prm data
3fc06e8
added process reward modeling
2fad09f
added process reward computation
dc8af70
bash for process reward modeling
7de1600
@fabrahman fabrahman requested a review from vwxyzjn September 21, 2024 00:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vwxyzjn vwxyzjn Awaiting requested review from vwxyzjn

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@fabrahman