Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update bm25Vectorizer #323

Merged
merged 9 commits into from
Jan 17, 2021
Merged

update bm25Vectorizer #323

merged 9 commits into from
Jan 17, 2021

Conversation

MXueguang
Copy link
Member

  • fix bug: Each time of initializing, the BM25 Vectorizer gives vocabulary list with different order, which will make results not replicable. Solved by sort the vocabulary list.
  • change: remove normalize. If we want to normalize, it should be done externally.
  • add: get_query_vector

@MXueguang MXueguang requested a review from lintool January 16, 2021 03:30
lintool
lintool reviewed Jan 16, 2021
View reviewed changes
Copy link
Member

@lintool lintool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turn norm into optional parameter?

pyserini/vectorizer/_base.py Show resolved Hide resolved
@MXueguang MXueguang requested a review from lintool January 16, 2021 17:57
@lintool
Copy link
Member

lintool commented Jan 16, 2021

@MXueguang One more thing... can we add a test case please? Can probably be added here: https://github.com/castorini/pyserini/blob/master/tests/test_index_reader.py

lintool
lintool approved these changes Jan 16, 2021
View reviewed changes
Copy link
Member

@lintool lintool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@MXueguang MXueguang merged commit 98f3236 into castorini:master Jan 17, 2021
@MXueguang MXueguang deleted the densify_sparse_index branch March 6, 2022 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lintool lintool lintool approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@MXueguang @lintool