All committers have signed the CLA.

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8b8bf78) 96.70% compared to head (cfa26aa) 96.82%.
Report is 3 commits behind head on master.

@@            Coverage Diff             @@
##           master     #856      +/-   ##
==========================================
+ Coverage   96.70%   96.82%   +0.12%     
==========================================
  Files          65       66       +1     
  Lines        5091     5229     +138     
  Branches      875      905      +30     
==========================================
+ Hits         4923     5063     +140     
+ Misses         86       85       -1     
+ Partials       82       81       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Thanks for the contribution @abhijitpal1247!

To make this easier to review, can you:

1. Add a unit test that for now gets this PR to at least 60-70% codecov (ie. a basic unit test that evaluates the end-to-end usage of your new issue manager from the Datalab level).

2. Add a comment to your PR description showing example code to test this out, and what the output looks like exactly (copy/paste the resulting Datalab report() and its issues, issue_summary, and info attributes.

Thanks, we can get somebody to review this PR once those basics are provided. After your code structure has been verified to be good, we'll ask you to add a few more unit tests in order to get the codecov of this new code up to 100%.

@jwmueller
inputs:

embeddings:
[[ nan 0.95071431 0.73199394], [0.59865848 0.15601864 0.15599452], [0.05808361 nan 0.60111501], [0.70807258 0.02058449 0.96990985]]

outputs:

report:

----------------------- null issues ------------------------

Number of examples with this issue: 2
Overall dataset quality in terms of this issue: 0.2500

Examples representing most severe instances of this issue:
   is_null_issue  null_score
1          False    0.000000
3          False    0.000000
0           True    0.333333
2           True    0.666667

Additional Information: 
average_null_score: 0.25

issues:

[{'is_null_issue': True, 'null_score': 0.3333333432674408}, {'is_null_issue': False, 'null_score': 0.0}, {'is_null_issue': True, 'null_score': 0.6666666865348816}, {'is_null_issue': False, 'null_score': 0.0}]

summary:

[{'issue_type': 'null', 'score': 0.25}]

info:

{'average_null_score': 0.25}

P.S.: Converted dataframes to dict and presented here

@elisno will leave this for your review. The main criteria to ensure are simply that:

• The quality score for each row should be the fraction of features which are not null in that row.
• The rows that are marked as is_null_issue should ONLY be those rows which are 100% null values.

@elisno I have implemented the suggested changes. Let me know if I am missing out on something.

@elisno , I have added the changes. Please review it. As we are nearing to the end of hacktoberfest, I would love to get this merge too.

Hey @jwmueller any updates 😅 We are very close to the end of hacktoberfest. Let me know if you need any additional info that can help you out.

Hi @abhijitpal1247, I'm pushing a few changes in a bit before we can merge this today!