Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a concurrency policy option for scheduledScan CRD #1749

Merged
merged 5 commits into from
Jul 18, 2023

Conversation

Ilyesbdlala
Copy link
Member

@Ilyesbdlala Ilyesbdlala commented Jun 6, 2023

Description

closes #275

Checklist

  • Test your changes as thoroughly as possible before you commit them. Preferably, automate your test by unit/integration tests.
  • Make sure that all your commits are signed-off and that you are added to the Contributors file.
  • Make sure that all CI finish successfully.
  • Optional (but appreciated): Make sure that all commits are Verified.

@Ilyesbdlala Ilyesbdlala added the enhancement New feature or request label Jun 6, 2023
@Ilyesbdlala Ilyesbdlala self-assigned this Jun 6, 2023
@github-actions
Copy link

github-actions bot commented Jun 6, 2023

🦙 MegaLinter status: ⚠️ WARNING

Descriptor Linter Files Fixed Errors Elapsed time
⚠️ GO golangci-lint 5 1 1.99s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security

@Ilyesbdlala Ilyesbdlala marked this pull request as ready for review June 13, 2023 08:50
@Ilyesbdlala Ilyesbdlala force-pushed the feature/concurrency-policy branch from 0bdc038 to 689ec78 Compare June 13, 2023 08:51
@rseedorff rseedorff added this to the v4.1.0 milestone Jun 28, 2023
Copy link
Member

@J12934 J12934 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works great 👍

One small note about a timeout value.

Also I think it would be good if we started attaching events to the ScheduledScans in Replace / Forbid cases to really communicate / logs to the users what's happening.

The kubernetes cronjob controller does this (in the Replace case) by attaching an event to the cronjob each time it creates / deletes a job:

Screenshot 2023-07-12 at 18 15 00

// check concurrency policy
if scheduledScan.Spec.ConcurrencyPolicy == executionv1.ForbidConcurrent && len(InProgressScans) > 0 {
log.V(8).Info("concurrency policy blocks concurrent runs, skipping", "num active", len(InProgressScans))
return ctrl.Result{RequeueAfter: 5 * time.Minute}, nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5m seems quite long. Especially as both interval and cron based scans can go lower than that.
Maybe rather use 5s? Should not be too much of a resource drain.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reasoning was that it's basically waiting on a scan to finish i.e go from in progress to Done or Errored State. On second thought, 5 min is a bit too much. I would set it to 1 min, unless you know the resource drain is basically negligible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd think the resource drain to be minimal, especially as this reque only happens when the Forbid case happens. So hopefully only rarely. 1m would also be ok for me.

@Ilyesbdlala Ilyesbdlala force-pushed the feature/concurrency-policy branch from 7e5d220 to 463d4c0 Compare July 14, 2023 14:59
Ilyesbdlala added a commit that referenced this pull request Jul 18, 2023
This is done as a compromise between speed and performance
see discussion: #1749 (comment)

Signed-off-by: Ilyes Ben Dlala <ilyes.bendlala@iteratec.com>
Signed-off-by: Ilyes Ben Dlala <ilyes.bendlala@iteratec.com>
Signed-off-by: Ilyes Ben Dlala <ilyes.bendlala@iteratec.com>
Signed-off-by: Ilyes Ben Dlala <ilyes.bendlala@iteratec.com>
This is done as a compromise between speed and performance
see discussion: #1749 (comment)

Signed-off-by: Ilyes Ben Dlala <ilyes.bendlala@iteratec.com>
@Ilyesbdlala Ilyesbdlala force-pushed the feature/concurrency-policy branch from fd48eeb to 4e7c8dc Compare July 18, 2023 07:51
… after rebase

Signed-off-by: Ilyes Ben Dlala <ilyes.bendlala@iteratec.com>
@Ilyesbdlala Ilyesbdlala merged commit d91fb79 into main Jul 18, 2023
@Ilyesbdlala Ilyesbdlala deleted the feature/concurrency-policy branch July 18, 2023 12:07
Ilyesbdlala added a commit that referenced this pull request Jul 18, 2023
This is done as a compromise between speed and performance
see discussion: #1749 (comment)

Signed-off-by: Ilyes Ben Dlala <ilyes.bendlala@iteratec.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

➹ Introduce a concurrency policy option for scheduledScan CRD
3 participants