Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change match_empty_frames to empty_is_annotated #8888

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from
+150 −100
Open

Change match_empty_frames to empty_is_annotated #8888

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
17bc920
Change match_empty_frames to empty_is_annotated
zhiltsov-max Dec 29, 2024
4249f14
Update docs
zhiltsov-max Dec 29, 2024
176798f
Update changelog
zhiltsov-max Dec 29, 2024
06e407c
Fix test
zhiltsov-max Dec 30, 2024
790a80d
Update schema
zhiltsov-max Dec 30, 2024
ca1ea9b
Fix remark
zhiltsov-max Dec 30, 2024
320296d
Extend changelog message
zhiltsov-max Jan 7, 2025
2f27b90
Remove extra checks
zhiltsov-max Jan 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update docs
  • Loading branch information
@zhiltsov-max
zhiltsov-max committed Dec 29, 2024
commit 4249f148d01a70bba1532515a4c6c41418654cd6
4 changes: 2 additions & 2 deletions cvat/apps/quality_control/serializers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,8 @@ class Meta:
""",
"compare_attributes": "Enables or disables annotation attribute comparison",
"empty_is_annotated": """
Count empty frames as annotated. This affects target metrics like accuracy in cases
there are no annotations. If disabled, frames without annotations
Consider empty frames annotated as "empty". This affects target metrics like
accuracy in cases there are no annotations. If disabled, frames without annotations
are counted as not matching (accuracy is 0). If enabled, accuracy will be 1 instead.
This will also add virtual annotations to empty frames in the comparison results.
""",
Expand Down
59 changes: 4 additions & 55 deletions cvat/schema.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8990,9 +8990,6 @@ components:
- slug
PaginatedAnnotationConflictList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9013,9 +9010,6 @@ components:
$ref: '#/components/schemas/AnnotationConflict'
PaginatedCloudStorageReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9036,9 +9030,6 @@ components:
$ref: '#/components/schemas/CloudStorageRead'
PaginatedCommentReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9059,9 +9050,6 @@ components:
$ref: '#/components/schemas/CommentRead'
PaginatedInvitationReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9082,9 +9070,6 @@ components:
$ref: '#/components/schemas/InvitationRead'
PaginatedIssueReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9105,9 +9090,6 @@ components:
$ref: '#/components/schemas/IssueRead'
PaginatedJobReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9128,9 +9110,6 @@ components:
$ref: '#/components/schemas/JobRead'
PaginatedLabelList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9151,9 +9130,6 @@ components:
$ref: '#/components/schemas/Label'
PaginatedMembershipReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9174,9 +9150,6 @@ components:
$ref: '#/components/schemas/MembershipRead'
PaginatedMetaUserList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9197,9 +9170,6 @@ components:
$ref: '#/components/schemas/MetaUser'
PaginatedOrganizationReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9220,9 +9190,6 @@ components:
$ref: '#/components/schemas/OrganizationRead'
PaginatedProjectReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9243,9 +9210,6 @@ components:
$ref: '#/components/schemas/ProjectRead'
PaginatedQualityReportList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9266,9 +9230,6 @@ components:
$ref: '#/components/schemas/QualityReport'
PaginatedQualitySettingsList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9289,9 +9250,6 @@ components:
$ref: '#/components/schemas/QualitySettings'
PaginatedRequestList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9312,9 +9270,6 @@ components:
$ref: '#/components/schemas/Request'
PaginatedTaskReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9335,9 +9290,6 @@ components:
$ref: '#/components/schemas/TaskRead'
PaginatedWebhookDeliveryReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand All @@ -9358,9 +9310,6 @@ components:
$ref: '#/components/schemas/WebhookDeliveryRead'
PaginatedWebhookReadList:
type: object
required:
- count
- results
properties:
count:
type: integer
Expand Down Expand Up @@ -9779,8 +9728,8 @@ components:
type: boolean
default: false
description: |
Count empty frames as annotated. This affects target metrics like accuracy in cases
there are no annotations. If disabled, frames without annotations
Consider empty frames annotated as "empty". This affects target metrics like
accuracy in cases there are no annotations. If disabled, frames without annotations
are counted as not matching (accuracy is 0). If enabled, accuracy will be 1 instead.
This will also add virtual annotations to empty frames in the comparison results.
PatchedTaskValidationLayoutWriteRequest:
Expand Down Expand Up @@ -10286,8 +10235,8 @@ components:
type: boolean
default: false
description: |
Count empty frames as annotated. This affects target metrics like accuracy in cases
there are no annotations. If disabled, frames without annotations
Consider empty frames annotated as "empty". This affects target metrics like
accuracy in cases there are no annotations. If disabled, frames without annotations
are counted as not matching (accuracy is 0). If enabled, accuracy will be 1 instead.
This will also add virtual annotations to empty frames in the comparison results.
RegisterSerializerEx:
Expand Down
3 changes: 2 additions & 1 deletion site/content/en/docs/manual/advanced/analytics-and-monitoring/auto-qa.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,8 @@ Annotation quality settings have the following parameters:
| - | - |
| Min overlap threshold | Min overlap threshold used for the distinction between matched and unmatched shapes. Used to match all types of annotations. It corresponds to the Intersection over union (IoU) for spatial annotations, such as bounding boxes and masks. |
| Low overlap threshold | Low overlap threshold used for the distinction between strong and weak matches. Only affects _Low overlap_ warnings. It's supposed that _Min similarity threshold_ <= _Low overlap threshold_. |
| Match empty frames | Consider frames matched if there are no annotations both on GT and regular job frames |
| Empty frames are annotated | Consider frames annotated as "empty" if there are no annotations on a frame.
If a frame is empty in both GT and job annotations, it will be considered a matching annotation. |

| _Point and Skeleton matching_ | |
| - | - |
Expand Down
Loading