Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Represent suggestion results as sparse arrays #681

Merged
merged 51 commits into from
Apr 14, 2023
Merged

Refactor: Represent suggestion results as sparse arrays #681

Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
6941bcc
implement SuggestionBatch class, although it doesn't do much yet
osma Mar 10, 2023
33b9694
implement SparseSuggestionResult
osma Mar 10, 2023
df80194
remove EvaluationBatch.evaluate() to make transition easier
osma Mar 10, 2023
172ce3c
use SuggestionBatch directly in annif.eval module (but still have to …
osma Mar 14, 2023
d8d5133
add unit tests for true/false positives/negatives eval metrics
osma Mar 14, 2023
d0331f5
reimplement true_positives/false_positives/false_negatives calculatio…
osma Mar 14, 2023
00c952d
test for eval results file in test_eval module
osma Mar 16, 2023
2d6b0b4
reimplement eval results per subject calculation using sparse array ops
osma Mar 16, 2023
e945d9e
reimplement Precision@{1,3,5} metrics without using precision_at_k_sc…
osma Mar 16, 2023
7e3571a
reimplement nDCG calculation using sparse array ops
osma Mar 16, 2023
13f4e13
remove LRAP metric & remaining use of dense arrays in annif.eval
osma Mar 16, 2023
830b3a5
remove unused imports
osma Mar 16, 2023
47b4f1c
create SuggestionBatch already in AnnifBackend._suggest_batch() inste…
osma Mar 17, 2023
0d74f93
more efficient filter_pred_top_k that supports both limit and threshold
osma Mar 17, 2023
8342bde
rename filter_pred_top_k to filter_suggestion and move it to annif.ut…
osma Mar 17, 2023
2bf8c65
implement SuggestionBatch.filter and .from_sequence classmethod
osma Mar 17, 2023
881ba97
use SuggestionBatch.filter() in annif.parallel
osma Mar 17, 2023
1844ad4
implement SuggestionResults and use it in CLI suggest and index commands
osma Mar 17, 2023
818ba21
reimplement CLI optimize command using SuggestionResults class; now a…
osma Mar 17, 2023
3a8eecd
remove remaining use of SuggestionFilter, along with the class and tests
osma Mar 17, 2023
d250de1
remove unused class LazySuggestionResult
osma Mar 17, 2023
96588cf
move filter_suggestion to annif.suggestion where it fits better than …
osma Mar 20, 2023
e2c6577
refactor: replace SuggestionResult.as_list with an iterator
osma Mar 20, 2023
f1e74f4
avoid use of .filter() in backends; apply limit in SuggestionBatch.fr…
osma Mar 20, 2023
5b266a4
remove SuggestionResult.filter() as it is not needed anymore (filteri…
osma Mar 20, 2023
ec0260c
skip deprecated subjects when creating SuggestionBatch from a sequence
osma Mar 20, 2023
ddd6f40
remove ListSuggestionResult class which is not necessary anymore
osma Mar 20, 2023
82f1b28
enforce score ranges in SuggestionBatch.from_sequence
osma Mar 20, 2023
911e149
remove unnecessary .as_vector() methods/variants
osma Mar 20, 2023
37d2251
Merge branch 'main' into issue678-refactor-suggestionresult-sparse-array
osma Mar 31, 2023
8af1fb2
silence NumPy warnings about invalid numbers
osma Mar 31, 2023
ca80d4f
refactor: remove VectorSuggestionResult and the SuggestionResult abst…
osma Mar 31, 2023
749a8c7
refactor: rename SparseSuggestionResult to SuggestionResult
osma Mar 31, 2023
46ba348
remove unused import abc
osma Mar 31, 2023
17e5b98
refactor: remove unnecessary size parameters
osma Mar 31, 2023
f120abf
remove unused function generate_suggestions
osma Mar 31, 2023
5b9518a
minor fixes
osma Mar 31, 2023
548b8b2
move imports inline to avoid startup overhead
osma Mar 31, 2023
05549be
avoid useless work creating SuggestionBatch from vectors + add tests
osma Apr 3, 2023
0ebf71f
improve tests for PAV backend
osma Apr 3, 2023
17f52a1
add test for fasttext suggest with empty text
osma Apr 3, 2023
0e1c79d
refactor: turn vector_to_suggestions into utility function
osma Apr 4, 2023
cc0b6b9
faster vector_to_suggestions using np.argpartition
osma Apr 4, 2023
3112407
refactor _normalize_suggestion_batch using list comprehensions
osma Apr 5, 2023
0d70034
reimplement ensemble hyperopt making better use of sparse arrays
osma Apr 6, 2023
d7e7fa4
refactor ensemble suggest functionality to better support batching
osma Apr 6, 2023
573b64f
remove now-unused merge_hits and WeightedSuggestionsBatch
osma Apr 6, 2023
ec4e37d
avoid extra .filter() operation in nn_ensemble
osma Apr 6, 2023
98fb87c
faster creation of csr_array in .from_sequence
osma Apr 6, 2023
bdb4d6e
remove unnecessary import
osma Apr 11, 2023
3edab48
reimplement filter_suggestion without using slow(?) dok_array
osma Apr 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove unused import abc
  • Loading branch information
@osma
osma committed Mar 31, 2023
commit 46ba348c8c28d10fce24b9a4c07b11e8f958df74
1 change: 0 additions & 1 deletion annif/suggestion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""Representing suggested subjects."""

import abc
import collections
import itertools

Expand Down