Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add to onboarding reproduction logs #2462

Merged
merged 2 commits into from
Apr 23, 2024
Merged

Add to onboarding reproduction logs #2462

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
91168cf
Add to onboarding reproduction logs
DanielKohn1208 Apr 23, 2024
9e369c7
Added missed changes to docs for removal of app assembler plugin
DanielKohn1208 Apr 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Added missed changes to docs for removal of app assembler plugin
  • Loading branch information
@DanielKohn1208
DanielKohn1208 committed Apr 23, 2024
commit 9e369c7d2f0ca6adba06f956a298898ff724f651
6 changes: 3 additions & 3 deletions docs/experiments-msmarco-passage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ In many retrieval applications, average precision and recall@1000 are the two me
You can use `trec_eval` to compute the MRR@10 also, which gives results identical to above (just fewer digits of precision):

```
target/appassembler/bin/trec_eval -c -M 10 -m recip_rank \
bin/trec_eval -c -M 10 -m recip_rank \
collections/msmarco-passage/qrels.dev.small.trec \
runs/run.msmarco-passage.dev.small.trec
```
Expand All @@ -272,15 +272,15 @@ It's a different command-line incantation of `trec_eval` to compute MRR@10.
And if you add `-q`, the tool will spit out the MRR@10 _per query_ (for all 6980 queries, in addition to the final average).

```
target/appassembler/bin/trec_eval -q -c -M 10 -m recip_rank \
bin/trec_eval -q -c -M 10 -m recip_rank \
collections/msmarco-passage/qrels.dev.small.trec \
runs/run.msmarco-passage.dev.small.trec
```

We can find the MRR@10 for `qid` 1048585 above:

```bash
$ target/appassembler/bin/trec_eval -q -c -M 10 -m recip_rank \
$ bin/trec_eval -q -c -M 10 -m recip_rank \
collections/msmarco-passage/qrels.dev.small.trec \
runs/run.msmarco-passage.dev.small.trec | grep 1048585

Expand Down
Loading