-
|
What is an appropriate normalization method for normalizing this score? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 2 replies
-
|
Most importantly, is the scoring standard affected by the query itself (e.g., longer sentences scoring higher overall, shorter sentences scoring lower overall)? |
Beta Was this translation helpful? Give feedback.
-
|
If you are are asking about how BM25 score works, i highly recommend reading the review by one of the original authors: https://www.staff.city.ac.uk/~sbrp622/papers/foundations_bm25_review.pdf |
Beta Was this translation helpful? Give feedback.
-
|
Thank you. In fact, what I want is to set a threshold such that for a query, if the score is higher than the threshold, the two are highly likely to be related. However, I have noticed that for different queries, even when the content is relevant to the query, the score can differ by several orders of magnitude. |
Beta Was this translation helpful? Give feedback.
-
|
That's interesting! NOt totally sure why it's happening, likely due to how the score is computed. feel free to try different score |
Beta Was this translation helpful? Give feedback.
Thank you. In fact, what I want is to set a threshold such that for a query, if the score is higher than the threshold, the two are highly likely to be related. However, I have noticed that for different queries, even when the content is relevant to the query, the score can differ by several orders of magnitude.