api: enable passing images in raw
mode
#8209
Open
+100
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I belive this should be considered a bug fix, as it resolves unexpected behavior and clarifies how to use undocumented capabilities - Images were not being passed to multimodal models when
images
were provided andraw
was set to true as described in this issue.This change introduces an optional boolean parameter
prepend_images_to_raw_prompt
, which if set totrue
will add the image placeholder fields necessary to pass the images through to the multimodal model. This parameter only has effect ifraw
is also set to true andimages
are provided. Theprepend_images_to_raw_prompt
parameter can be omitted, and the default behavior remains entirely unchanged to avoid breaking existing users.It also updates the api.md documentation to describe the new functionality (including an example request/response), and also provides and example of manually passing an image placeholder in the text of a raw prompt directly, which offers more control over where the image is inserted in the raw prompt.