Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor KNN graph handling and outlier detection in issue managers #1155

Merged
merged 14 commits into from
Jun 24, 2024
Merged

Refactor KNN graph handling and outlier detection in issue managers #1155

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
b87e44c
feat: Refactor knn setup code in near-duplicate issue manager
elisno Jun 19, 2024
4960555
rename imported functions being tested from knn_graph_helpers
elisno Jun 19, 2024
1c2ba70
Refactor KNN graph setup in Non-IID Issue Manager
elisno Jun 20, 2024
31d1053
Refactor KNN graph setup in unferperforming group issue manager
elisno Jun 20, 2024
6b202fc
Refactor KNN graph setup in data valuation issue manager
elisno Jun 20, 2024
190e903
Merge branch 'master' into refactor-knn-graph-recomputation
elisno Jun 20, 2024
fb775e3
address mypy and flake8 issues
elisno Jun 20, 2024
8762259
Refactor OutlierIssueManager for improved flexibility and consistency
elisno Jun 21, 2024
7b1a3f9
feat: Add xfail reason to test_default_issue_types
elisno Jun 21, 2024
5f82ab4
rename variables in _process_knn_graph_from_inputs for clarity
elisno Jun 24, 2024
3f487f1
remove unused needs_recompute variable for provided knn_graph
elisno Jun 24, 2024
38cf94f
_check_pred_probs method in NonIIDIssueManager renamed to _determine_…
elisno Jun 24, 2024
3cd60a5
improve error handling for insufficient knn graph in OutlierIssueManager
elisno Jun 24, 2024
6ccfc76
Remove unused _score_with_features method from OutlierIssueManager
elisno Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
address mypy and flake8 issues
  • Loading branch information
@elisno
elisno committed Jun 20, 2024
commit fb775e3e4374ec29f151d296683498e6d193303f
9 changes: 5 additions & 4 deletions cleanlab/datalab/internal/issue_manager/knn_graph_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
from scipy.sparse import csr_matrix


from typing import Any, Dict, Optional, Tuple
from typing import Any, Dict, Optional, Tuple, cast

from cleanlab.internal.neighbor.knn_graph import create_knn_graph_and_index
from cleanlab.typing import Metric


def num_neighbors_in_knn_graph(knn_graph: csr_matrix) -> int:
Expand Down Expand Up @@ -44,10 +45,10 @@ def knn_exists(kwargs: Dict[str, Any], statistics: Dict[str, Any], k_needed: int
def set_knn_graph(
features: Optional[npt.NDArray],
find_issues_kwargs: Dict[str, Any],
metric: Optional[str],
metric: Optional[Metric],
k: int,
statistics: Dict[str, Any],
) -> Tuple[csr_matrix, str]:
) -> Tuple[csr_matrix, Metric]:
# This only fetches graph (optionally)
knn_graph = _process_knn_graph_from_inputs(
find_issues_kwargs, statistics, k_for_recomputation=k
Expand All @@ -60,4 +61,4 @@ def set_knn_graph(
assert features is not None, "Features must be provided to compute the knn graph."
knn_graph, knn = create_knn_graph_and_index(features, n_neighbors=k, metric=metric)
metric = knn.metric
return knn_graph, metric
return cast(csr_matrix, knn_graph), cast(Metric, metric)
1 change: 0 additions & 1 deletion tests/datalab/datalab/test_datalab.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@
from cleanlab.datalab.datalab import Datalab
from cleanlab.datalab.internal.report import Reporter
from cleanlab.datalab.internal.task import Task
from cleanlab.internal.neighbor.knn_graph import create_knn_graph_and_index


SEED = 42
Expand Down
Loading