Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle changes in case of output token limit exceeded #979

Open
Anonymousinterpares opened this issue Jan 2, 2025 · 0 comments
Open

handle changes in case of output token limit exceeded #979

Anonymousinterpares opened this issue Jan 2, 2025 · 0 comments

Comments

@Anonymousinterpares
Copy link

Is your feature request related to a problem? Please describe:

Very often the files I work on are significantly larger than the max output limit even of the newest Gemioni models (and adhering to keeping the code as modular as possible). Because of that, given file is never finished within the project structure and the only workaround I see is to request from LLM to answer directly in chat.

Describe the solution you'd like:

Think about adjusting the LLM prompt along with adjusting the app code so that in case output token limit is close to be reached, LLM could leave a defined markdown to allow the application to detect it and let the llm to continue from there in the next output.
Maybe it would also be possible to mark the unfinished file by the app itself - for example, definig a rule that if a file is not being changed anymore while create wheel is still spinning, that place could be marked by the app in a specific way to then, allow LLM to resume from that exact point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant