From 74a87e4951c98d7b066273140576d3cccd9ea0ed Mon Sep 17 00:00:00 2001 From: Larry Li Date: Fri, 29 Jan 2021 15:23:49 -0500 Subject: [PATCH] Fixing issues where Pyserini failed to produce metrices with TREC evaluation tool on MS MAROC passgae retrieval (#342) --- docs/experiments-msmarco-passage.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/experiments-msmarco-passage.md b/docs/experiments-msmarco-passage.md index de67245a2..d9a7e5311 100644 --- a/docs/experiments-msmarco-passage.md +++ b/docs/experiments-msmarco-passage.md @@ -105,8 +105,11 @@ For that we first need to convert the run file into TREC format: ```bash $ python tools/scripts/msmarco/convert_msmarco_to_trec_run.py \ --input runs/run.msmarco-passage.bm25tuned.txt --output runs/run.msmarco-passage.bm25tuned.trec +$ python tools/scripts/msmarco/convert_msmarco_to_trec_qrels.py \ + --input tools/topics-and-qrels/qrels.msmarco-passage.dev-subset.txt --output collections/msmarco-passage/qrels.dev.small.trec ``` + And then run the `trec_eval` tool: ```bash