Cannot index 5MB PDF with default settings using bedrock #94
Open
Description
I try to upload this file (5MB, 2,384,000 chars) to LibreChat with bedrock API activated
https://pve.proxmox.com/pve-docs/pve-admin-guide.pdf
I tried dev and dev-lite containers but am getting an upload error ("An Error occurred while uploading a file) in the LibreChat GUI but no real error in the logs with DEBUG_RAG_API=true, Strange
If set CHUNK_SIZE=5000 it works however, these are my RAG settings
DEBUG_RAG_API=true
RAG_USE_FULL_CONTEXT=true
PDF_EXTRACT_IMAGES=false # false is default
CHUNK_SIZE=5000 # 1500 is default
AWS_DEFAULT_REGION=us-west-2
AWS_ACCESS_KEY_ID=cc
AWS_SECRET_ACCESS_KEY=cc
EMBEDDINGS_PROVIDER=bedrock
EMBEDDINGS_MODEL=amazon.titan-embed-text-v1
RAG_API_URL=http://host-gateway:8000
Metadata
Assignees
Labels
No labels
Activity
dirkpetersen commentedon Oct 27, 2024
Further testing shows that CHUNK_SIZE=5000 does not fully fix the issue, more testing needed, ChatGPT accepts this document but Claude says it is too big
FinnConnor commentedon Oct 28, 2024
I tested with
CHUNK_SIZE=1500
EMBEDDINGS_PROVIDER=bedrock
EMBEDDINGS_MODEL=amazon.titan-embed-text-v1
PDF_EXTRACT_IMAGES=False
.I was unable to to see any issue with indexing this pdf (5 MB) and querying in both with docker and with only the rag_api.
If you are getting a file upload error. I would run just the rag_api (and database) and see if you are able to use the
\embed
to upload the 5MB pdf. This will help confirm if it is an issue with embedding the file or something else.If your not having an issue with that, it may be that you have
RAG_USE_FULL_CONTEXT=true
this will send the entire context (all text of 5MB PDF) to chat, which very likely exceed the max number of input tokens.Thanks for bringing this up @dirkpetersen
dirkpetersen commentedon Oct 28, 2024
Thanks @ScarFX I set RAG_USE_FULL_CONTEXT=false but the problem persists.
It seems there is a timeout: Next is trying RAG API standalone
FinnConnor commentedon Nov 18, 2024
@dirkpetersen were you able to get RAG API to work?
dvejsada commentedon Jan 14, 2025
We have been experiencing this issue as well. From around 3MB, the file upload fails. For smaller files, it works fine. Here I attach one of the failed files for replication purposes (saved article to PDF from website).
Clanek_SeznamZpravy.pdf. @danny-avila could you please have a look what may be causing this? We use Azure OpenAI embeddings, with text-embedding-3-large model.