Open
Description
Describe the bug
Since sort_heap
is not stable and topk_queue
only sorts on score, the order of results that have the same scores can differ between runs. This is harmful for the repeatbility of results.
To Reproduce
Steps to reproduce the behavior:
- Follow the steps to index and execute queries. Be sure to include a query that will result in a score tie for the ranking model. For instance, the query "chemical reactions" over the vaswani dataset.
- Observe the order of documents with tied scores.
- Repeat and re-observe several times.
Error message
No error message.
Expected behavior
I expect results to be ordered ascending by DocId as a secondary sort when scores result in ties.
Environment info
Operating System: Ubuntu 22.04.1
Compiler name: gcc
Compiler version: 9