Skip to content

stability of document order when tied #508

Open
@seanmacavaney

Description

Describe the bug
Since sort_heap is not stable and topk_queue only sorts on score, the order of results that have the same scores can differ between runs. This is harmful for the repeatbility of results.

To Reproduce
Steps to reproduce the behavior:

  1. Follow the steps to index and execute queries. Be sure to include a query that will result in a score tie for the ranking model. For instance, the query "chemical reactions" over the vaswani dataset.
  2. Observe the order of documents with tied scores.
  3. Repeat and re-observe several times.

Error message
No error message.

Expected behavior
I expect results to be ordered ascending by DocId as a secondary sort when scores result in ties.

Environment info
Operating System: Ubuntu 22.04.1
Compiler name: gcc
Compiler version: 9

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions