Fix memmory availability check for embedding generation #109
Open
Description
There may be an issue with the way we check memory availability from rust for embedding generation.
If all of RAM is filled with cache that can be swapped out upon memory pressure, I think it still reports low memory error
postgres=# select text_embedding('BAAI/bge-small-en', 'haha');
ERROR: Not enough free memory to run the model. Memory needed: 137.70MB, Memory available: 53.87MB
note that available memory below is much larger than free memory
n@cb28478e2f23:/$ free -h
total used free shared buff/cache available
Mem: 125Gi 1.4Gi 1.1Gi 93Gi 123Gi 29Gi
Metadata
Assignees
Labels
No labels