The latest updates on your projects. Learn more about Vercel for Git ↗︎

@pablocastro @farzad528 Kindly request you to please take a look.

@Sidchat95 happy to help but you may have the wrong folks, @farzad528 and I work on Azure Cognitive Search, not on FAISS.

self.relevance_score_fn is relevant for _similarity_search_with_relevance_scores

Hi @hwchase17 , as I agree that self.relevance_score_fn is majorly used for _similarity_search_with_relevance_scores function but it is also effecting this stage as well.

Consider the following scenario where we have a vectorstore built out of FAISS vectorstore:

1. We get the retriever as retriever = vectorstore.as_retriever() which is mostly used for conversational chains.
2. Given that the search type is set to "similarity_score_threshold", when we call retriever.get_relevant_documents(query) by setting the score threshold to say 0.8, the following workflow is triggered chronologically. (Let us keep in mind that 0.8 threshold set here is the threshold set on the basis of a score of 0 to 1 which ideally is the output of relevance_score_fn.

The steps called are:

•  self.vectorstore.similarity_search_with_relevance_scores is triggered with the threshold of 0.8 (Line 410 File langchain.vectorstore.base)
  

•  The next step involves calling self._similarity_search_with_relevance_scores (Line 156 File langchain.vectorstore.base)
  

•  Function _similarity_search_with_relevance_scores is called from file langchain.vectorstore.faiss which in turn calls self.similarity_search_with_score function with the same query and the same threshold of 0.8
  

• Function similarity_search_with_score is called which in turn calls self.similarity_search_with_score_by_vector with the same query and threshold of 0.8
  

• self.similarity_search_with_score_by_vector is called with the threshold of 0.8 and the query. This function first fetches the docs till line 229 of langchain.vectorstore.faiss and then gets the score_threshold set which is still 0.8 in this case. It then matches and filters according to the threshold which is 0.8. 
  
• The score returned in the list docs with the similarities is not between 0 and 1 and thus comparing it to 0.8 will essentially not fetch any results. For example a score of 0.312 in the docs list similarity is actually 0.779 by using the function converter 1.0 - 0.312 / math.sqrt(2) which is why we need to first convert the similarity score fetched by docs and then compare it to the incoming threshold.

@Sidchat95 happy to help but you may have the wrong folks, @farzad528 and I work on Azure Cognitive Search, not on FAISS.

Thanks no problem. My bad !

@hwchase17 Request you to please look into this and approve the workflows

updated so only affects similarity_search_with_relevance_score

updated so only affects similarity_search_with_relevance_score

Thanks @baskaryan for the collaboration but changing it in similarity_search_with_relevance_score in FAISS renders Line 227 to 238 in method similarity_search_with_score_by_vector completely useless as this would always pop the threshold from the arguments. Should we remove the redundant code then ?

updated so only affects similarity_search_with_relevance_score

Thanks @baskaryan for the collaboration but changing it in similarity_search_with_relevance_score in FAISS renders Line 227 to 238 in method similarity_search_with_score_by_vector completely useless as this would always pop the threshold from the arguments. Should we remove the redundant code then ?

it's useless if someone calls similarity_search_with_relevance_score but not useless if someones calls similarity_search_with_score_by_vector directly

updated so only affects similarity_search_with_relevance_score

Thanks @baskaryan for the collaboration but changing it in similarity_search_with_relevance_score in FAISS renders Line 227 to 238 in method similarity_search_with_score_by_vector completely useless as this would always pop the threshold from the arguments. Should we remove the redundant code then ?

it's useless if someone calls similarity_search_with_relevance_score but not useless if someones calls similarity_search_with_score_by_vector directly

Agreed. Thank you @baskaryan . Can we merge this now? I do not have the permissions to merge this actually and seems all checks have passed?

done, thanks @Sidchat95!

done, thanks @Sidchat95!

Thank you @baskaryan as a part of this PR, can I get the recognition for this twitter account if possible for this? @procoretech