Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to deal with the generate response is None, which the model directly generate the <im_end> token. #30

Open
unicorneeee opened this issue Dec 11, 2024 · 0 comments

Comments

@unicorneeee
Copy link

I hope this message find you well. When I use the InfiniteBench to compute score, if the response of the model is None , it will raise an mistake,therefore, in our result table with the results of gpt4 and chatglm3 , etc.... Is these evaluate results remove the test example which response is None, and calculate the final score or just set all F1,Recall ,score to 0 for the cases which model's response is None, and calculate the final score?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant