Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add precision classification metric #2293

Merged
merged 31 commits into from
Nov 20, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c73438f
Implement confusion matrix and precision, first draft
Aug 21, 2024
63f4a1d
Implement confusion matrix
Sep 9, 2024
b9d71b6
format :D
Sep 9, 2024
eac29aa
add agg type to cm, reformat debug representation add testing.
Sep 20, 2024
59db68b
formating and tiny refactor
Sep 21, 2024
4261bd8
add ClassificationMetric trait, rename variables and types, move test…
Sep 21, 2024
5431a2f
change unwrap to expect
Sep 21, 2024
fd2e585
update book
Sep 21, 2024
56965e8
remove unused code
Sep 22, 2024
419438a
changes to make reusing code easier
Sep 22, 2024
dfac847
format :D
Sep 22, 2024
ea4b29c
change to static data tests
Sep 24, 2024
e23aa7b
remove classification metric trait, add auxiliary code for classific…
Oct 14, 2024
60a246b
move classification objects to classification.rs, use rstest, remove …
Oct 21, 2024
c145531
review docstring, add top_k for multiclass tasks.
Oct 23, 2024
0c984c4
move class averaging and metric computation to metric implementation,…
Oct 25, 2024
b0a2939
change struct and var names
Oct 25, 2024
f18e321
Merge branch 'main' into add-to-metrics
Oct 26, 2024
386802c
rename params, enforce nonzero for top_k param, optimize one_hot for …
Oct 30, 2024
b525527
add adaptor por classification input, correct one hot function
Nov 1, 2024
ff7611a
define default for ClassReduction, derive new for Precision metric wi…
Nov 8, 2024
4cbcff2
Merge branch 'main' into add-to-metrics
Nov 8, 2024
eeab0d3
expose PrecisionMetric, change metric initialization
Nov 8, 2024
aea207f
check one_hot input tensor has more than 1 classes and correct it's i…
Nov 16, 2024
410f273
Merge branch 'main' into add-to-metrics
Nov 16, 2024
746fa9d
implement adaptor for MultilabelClassificationOutput and Classificati…
Nov 16, 2024
7428b86
change with_top_k to take usize
Nov 18, 2024
58e1902
Merge branch 'main' into add-to-metrics
Nov 18, 2024
d598f00
Add precision config for binary, multiclass and multilabel
laggui Nov 18, 2024
1542ee9
Fix dummy_classification_input
laggui Nov 18, 2024
03ebe1d
make PrecisionMetric public
Nov 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
define default for ClassReduction, derive new for Precision metric wi…
…th class_reduction as default and new setter implementation, move NonZerousize boundary to confusion_stats
  • Loading branch information
Tiago Sanona committed Nov 8, 2024
commit ff7611a84448aee3e23ace6801b86a68ac86884d
3 changes: 2 additions & 1 deletion crates/burn-train/src/metric/classification.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ impl<B: Backend> From<ClassificationInput<B>> for (Tensor<B, 2>, Tensor<B, 2, Bo

/// Class Averaging types for Classification metrics.
#[derive(Copy, Clone)]
#[allow(dead_code)]
#[derive(Default)]
pub enum ClassReduction {
///Computes the statistics over all classes before averaging
#[default]
Micro,
///Computes the statistics independently for each class before averaging
Macro,
Expand Down
15 changes: 9 additions & 6 deletions crates/burn-train/src/metric/precision.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,10 @@ pub struct PrecisionMetric<B: Backend> {
#[new(default)]
state: NumericMetricState,
_b: PhantomData<B>,
#[new(default)]
class_reduction: ClassReduction,
threshold: Option<f64>,
top_k: Option<NonZeroUsize>,
top_k: Option<usize>,
}

impl<B: Backend> PrecisionMetric<B> {
Expand All @@ -40,6 +41,11 @@ impl<B: Backend> PrecisionMetric<B> {
};
avg_tensor.into_scalar().to_f64()
}

pub fn with_class_reduction(mut self, class_reduction: ClassReduction) -> Self {
self.class_reduction = class_reduction;
self
}
}

impl<B: Backend> Metric for PrecisionMetric<B> {
Expand All @@ -53,7 +59,7 @@ impl<B: Backend> Metric for PrecisionMetric<B> {
predictions,
targets,
self.threshold,
self.top_k,
self.top_k.map(NonZeroUsize::new).flatten(),
self.class_reduction,
);
let metric =
Expand Down Expand Up @@ -88,10 +94,8 @@ mod tests {
ClassificationType::{self, *},
THRESHOLD,
};
use crate::TestBackend;
use burn_core::tensor::TensorData;
use rstest::rstest;
use std::num::NonZeroUsize;

#[rstest]
#[case::binary_micro(Binary, Micro, Some(THRESHOLD), None, 0.5)]
Expand All @@ -111,9 +115,8 @@ mod tests {
) {
let input = dummy_classification_input(&classification_type);
let mut metric = PrecisionMetric::new(
class_reduction,
threshold,
top_k.map(NonZeroUsize::new).flatten(),
top_k
);
let _entry = metric.update(&input, &MetricMetadata::fake());
TensorData::from([metric.value()])
Expand Down