Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring of SearchCollection #951

Merged
merged 15 commits into from
Jan 18, 2020
Merged

Refactoring of SearchCollection #951

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
ad0c5ab
SearchCollection cleanup
lintool Dec 10, 2019
c074018
Merge branch 'master' into search-refactoring
lintool Dec 10, 2019
66562a5
Changed parameter names.
lintool Dec 10, 2019
8e461eb
Merge branch 'master' into search-refactoring
lintool Dec 10, 2019
0863e42
Merge branch 'master' into search-refactoring
lintool Dec 28, 2019
1a13088
Updated YAML for core17, core18, and disk12
lintool Dec 29, 2019
73a552b
Merge branch 'master' into search-refactoring
lintool Jan 10, 2020
295ef69
Fixed params.
lintool Jan 11, 2020
5e8dfc1
Fixed fine-tuning yaml.
lintool Jan 15, 2020
f4ad457
Merge branch 'master' into search-refactoring
lintool Jan 15, 2020
badb3a5
Merge branch 'master' into search-refactoring
lintool Jan 15, 2020
4225a1a
Updated parameters.
lintool Jan 15, 2020
a2b78aa
Addressed CR re: order of rankCutoff parameter.
lintool Jan 17, 2020
f399439
Tweaked params in docs.
lintool Jan 17, 2020
439755d
Merge branch 'master' into search-refactoring
lintool Jan 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/regressions-car17v1.5.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,19 +40,19 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v1.5.pos

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v1.5.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v1.5.benchmarkY1test.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -output run.car17v1.5.bm25+ax.topics.car17v1.5.benchmarkY1test.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -output run.car17v1.5.bm25+ax.topics.car17v1.5.benchmarkY1test.txt &
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a more general question, the parameter -rerankCutoff only applies when we specify isRerank to true. That means we should first see args.rm3 || args.axiom || args.bm25prf before seeing -rerankCutoff. So I guess we should put rm3, axiom, bm25prf before -rerankCutoff (the original way)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do rankCutoff explicitly for each model? E.g., rm3.rerankCutoff?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My original motivation was to keep all the "groups" together.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or I could make the rankCutoff parameter to the end?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think rerankCutoff for each model is necessary as they will probably not present at the same time? (what is the semantic for that?)

Putting rerankCutoff at the end is fine


nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v1.5.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v1.5.benchmarkY1test.txt \
-ql -output run.car17v1.5.ql.topics.car17v1.5.benchmarkY1test.txt &
-qld -output run.car17v1.5.ql.topics.car17v1.5.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v1.5.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v1.5.benchmarkY1test.txt \
-ql -rm3 -output run.car17v1.5.ql+rm3.topics.car17v1.5.benchmarkY1test.txt &
-qld -rm3 -output run.car17v1.5.ql+rm3.topics.car17v1.5.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v1.5.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v1.5.benchmarkY1test.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -output run.car17v1.5.ql+ax.topics.car17v1.5.benchmarkY1test.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -output run.car17v1.5.ql+ax.topics.car17v1.5.benchmarkY1test.txt &
```

Evaluation can be performed using `trec_eval`:
Expand Down
8 changes: 4 additions & 4 deletions docs/regressions-car17v2.0-doc2query.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,19 +46,19 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0-doc

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0-doc2query.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -output run.car17v2.0-doc2query.bm25+ax.topics.car17v2.0.benchmarkY1test.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -output run.car17v2.0-doc2query.bm25+ax.topics.car17v2.0.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0-doc2query.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-ql -output run.car17v2.0-doc2query.ql.topics.car17v2.0.benchmarkY1test.txt &
-qld -output run.car17v2.0-doc2query.ql.topics.car17v2.0.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0-doc2query.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-ql -rm3 -output run.car17v2.0-doc2query.ql+rm3.topics.car17v2.0.benchmarkY1test.txt &
-qld -rm3 -output run.car17v2.0-doc2query.ql+rm3.topics.car17v2.0.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0-doc2query.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -output run.car17v2.0-doc2query.ql+ax.topics.car17v2.0.benchmarkY1test.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -output run.car17v2.0-doc2query.ql+ax.topics.car17v2.0.benchmarkY1test.txt &
```

Evaluation can be performed using `trec_eval`:
Expand Down
8 changes: 4 additions & 4 deletions docs/regressions-car17v2.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,19 +40,19 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0.pos

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -output run.car17v2.0.bm25+ax.topics.car17v2.0.benchmarkY1test.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -output run.car17v2.0.bm25+ax.topics.car17v2.0.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-ql -output run.car17v2.0.ql.topics.car17v2.0.benchmarkY1test.txt &
-qld -output run.car17v2.0.ql.topics.car17v2.0.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-ql -rm3 -output run.car17v2.0.ql+rm3.topics.car17v2.0.benchmarkY1test.txt &
-qld -rm3 -output run.car17v2.0.ql+rm3.topics.car17v2.0.benchmarkY1test.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.car17v2.0.pos+docvectors+rawdocs \
-topicreader Car -topics src/main/resources/topics-and-qrels/topics.car17v2.0.benchmarkY1test.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -output run.car17v2.0.ql+ax.topics.car17v2.0.benchmarkY1test.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -output run.car17v2.0.ql+ax.topics.car17v2.0.benchmarkY1test.txt &
```

Evaluation can be performed using `trec_eval`:
Expand Down
8 changes: 4 additions & 4 deletions docs/regressions-core17.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,19 +39,19 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.core17.pos+do

nohup target/appassembler/bin/SearchCollection -index lucene-index.core17.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core17.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -output run.core17.bm25+ax.topics.core17.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -output run.core17.bm25+ax.topics.core17.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.core17.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core17.txt \
-ql -output run.core17.ql.topics.core17.txt &
-qld -output run.core17.ql.topics.core17.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.core17.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core17.txt \
-ql -rm3 -output run.core17.ql+rm3.topics.core17.txt &
-qld -rm3 -output run.core17.ql+rm3.topics.core17.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.core17.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core17.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -output run.core17.ql+ax.topics.core17.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -output run.core17.ql+ax.topics.core17.txt &
```

Evaluation can be performed using `trec_eval`:
Expand Down
8 changes: 4 additions & 4 deletions docs/regressions-core18.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,19 +39,19 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.core18.pos+do

nohup target/appassembler/bin/SearchCollection -index lucene-index.core18.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core18.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -output run.core18.bm25+ax.topics.core18.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -output run.core18.bm25+ax.topics.core18.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.core18.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core18.txt \
-ql -output run.core18.ql.topics.core18.txt &
-qld -output run.core18.ql.topics.core18.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.core18.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core18.txt \
-ql -rm3 -output run.core18.ql+rm3.topics.core18.txt &
-qld -rm3 -output run.core18.ql+rm3.topics.core18.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.core18.pos+docvectors+rawdocs \
-topicreader Trec -topics src/main/resources/topics-and-qrels/topics.core18.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -output run.core18.ql+ax.topics.core18.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -output run.core18.ql+ax.topics.core18.txt &
```

Evaluation can be performed using `trec_eval`:
Expand Down
24 changes: 12 additions & 12 deletions docs/regressions-cw09b.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,43 +56,43 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+doc

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.51-100.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.bm25+ax.topics.web.51-100.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.bm25+ax.topics.web.51-100.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.101-150.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.bm25+ax.topics.web.101-150.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.bm25+ax.topics.web.101-150.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.151-200.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.bm25+ax.topics.web.151-200.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.bm25+ax.topics.web.151-200.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.51-100.txt \
-ql -output run.cw09b.ql.topics.web.51-100.txt &
-qld -output run.cw09b.ql.topics.web.51-100.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.101-150.txt \
-ql -output run.cw09b.ql.topics.web.101-150.txt &
-qld -output run.cw09b.ql.topics.web.101-150.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.151-200.txt \
-ql -output run.cw09b.ql.topics.web.151-200.txt &
-qld -output run.cw09b.ql.topics.web.151-200.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.51-100.txt \
-ql -rm3 -output run.cw09b.ql+rm3.topics.web.51-100.txt &
-qld -rm3 -output run.cw09b.ql+rm3.topics.web.51-100.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.101-150.txt \
-ql -rm3 -output run.cw09b.ql+rm3.topics.web.101-150.txt &
-qld -rm3 -output run.cw09b.ql+rm3.topics.web.101-150.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.151-200.txt \
-ql -rm3 -output run.cw09b.ql+rm3.topics.web.151-200.txt &
-qld -rm3 -output run.cw09b.ql+rm3.topics.web.151-200.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.51-100.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.ql+ax.topics.web.51-100.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.ql+ax.topics.web.51-100.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.101-150.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.ql+ax.topics.web.101-150.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.ql+ax.topics.web.101-150.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw09b.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.151-200.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.ql+ax.topics.web.151-200.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw09b.ql+ax.topics.web.151-200.txt &
```

Evaluation can be performed using `trec_eval` and `gdeval.pl`:
Expand Down
8 changes: 4 additions & 4 deletions docs/regressions-cw12.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,17 +46,17 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12.pos+docv

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.201-250.txt \
-ql -output run.cw12.ql.topics.web.201-250.txt &
-qld -output run.cw12.ql.topics.web.201-250.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.251-300.txt \
-ql -output run.cw12.ql.topics.web.251-300.txt &
-qld -output run.cw12.ql.topics.web.251-300.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.201-250.txt \
-ql -rm3 -output run.cw12.ql+rm3.topics.web.201-250.txt &
-qld -rm3 -output run.cw12.ql+rm3.topics.web.201-250.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.251-300.txt \
-ql -rm3 -output run.cw12.ql+rm3.topics.web.251-300.txt &
-qld -rm3 -output run.cw12.ql+rm3.topics.web.251-300.txt &
```

Evaluation can be performed using `trec_eval` and `gdeval.pl`:
Expand Down
16 changes: 8 additions & 8 deletions docs/regressions-cw12b13.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,31 +46,31 @@ nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+d

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.201-250.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.bm25+ax.topics.web.201-250.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.bm25+ax.topics.web.201-250.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.251-300.txt \
-bm25 -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.bm25+ax.topics.web.251-300.txt &
-bm25 -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.bm25+ax.topics.web.251-300.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.201-250.txt \
-ql -output run.cw12b13.ql.topics.web.201-250.txt &
-qld -output run.cw12b13.ql.topics.web.201-250.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.251-300.txt \
-ql -output run.cw12b13.ql.topics.web.251-300.txt &
-qld -output run.cw12b13.ql.topics.web.251-300.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.201-250.txt \
-ql -rm3 -output run.cw12b13.ql+rm3.topics.web.201-250.txt &
-qld -rm3 -output run.cw12b13.ql+rm3.topics.web.201-250.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.251-300.txt \
-ql -rm3 -output run.cw12b13.ql+rm3.topics.web.251-300.txt &
-qld -rm3 -output run.cw12b13.ql+rm3.topics.web.251-300.txt &

nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.201-250.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.ql+ax.topics.web.201-250.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.ql+ax.topics.web.201-250.txt &
nohup target/appassembler/bin/SearchCollection -index lucene-index.cw12b13.pos+docvectors+rawdocs \
-topicreader Webxml -topics src/main/resources/topics-and-qrels/topics.web.251-300.txt \
-ql -axiom -rerankCutoff 20 -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.ql+ax.topics.web.251-300.txt &
-qld -rerankCutoff 20 -axiom -axiom.deterministic -axiom.beta 0.1 -output run.cw12b13.ql+ax.topics.web.251-300.txt &
```

Evaluation can be performed using `trec_eval` and `gdeval.pl`:
Expand Down
Loading