Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔧 fix: Improve the message in case when prompt exceeds the model limit #3836

Conversation

DenisPalnitsky
Copy link
Contributor

@DenisPalnitsky DenisPalnitsky commented Aug 29, 2024

Summary

When message is over the model's limit, chat returns generic error that "something went wrong".
image
This does not tell user what exactly went wrong and how to fix it.

This PR changes message text informing user that his prompt exceeds that max limit and suggest to shorten it.
image

Change Type

Please delete any irrelevant options.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update
  • Translation update

Testing

To test this copy prompt from HugePrompt.txt and send it as a chat message.
You should see the message The prompt is too long and exceeds the token limit. Please shorten your prompt or attach part of it as a text file.

Test Configuration:

Checklist

Please delete any irrelevant options.

  • My code adheres to this project's style guidelines
  • I have performed a self-review of my own code
  • I have commented in any complex areas of my code
  • My changes do not introduce new warnings
  • I have written tests demonstrating that my changes are effective or that my feature works
  • Local unit tests pass with my changes

@DenisPalnitsky DenisPalnitsky marked this pull request as ready for review August 29, 2024 08:00
@danny-avila
Copy link
Owner

Closing due to a few issues, fixed here: #3866

The PR I added would be a good reference for adding more error types in the future, may add it to the docs.

The main point is to keep the error message size small, and leave abortMiddleware file untouched, which is the basis of formatting error messages this way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants