Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: remove parallel walk #5180

Merged
merged 30 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
544953b
refactor: remove parallel walk
knqyf263 Sep 13, 2023
6fd56ee
fix(walk): call error callback on all errors
knqyf263 Oct 19, 2023
7441e19
Merge branch 'main' into unify_walk
knqyf263 Oct 19, 2023
362871a
feat: add delay per file
knqyf263 Nov 2, 2023
4972f06
feat: configure delay seconds
knqyf263 Nov 6, 2023
7fc49c2
feat: replace slow with parallel
knqyf263 Nov 7, 2023
c6e57d5
Merge branch 'main' into unify_walk
knqyf263 Nov 7, 2023
c778768
test: remove external
knqyf263 Nov 9, 2023
7cfb534
refactor: inject filesystem walker
knqyf263 Nov 9, 2023
c05898a
docs: auto generate
knqyf263 Nov 9, 2023
e1368a3
Merge branch 'main' into unify_walk
knqyf263 Nov 9, 2023
acf36ba
docs: remove plugin reference
knqyf263 Nov 10, 2023
031117f
chore(magefile): not load plugins for doc
knqyf263 Nov 10, 2023
87b903c
docs: update
knqyf263 Nov 10, 2023
3031612
fix: pass walker options
knqyf263 Nov 10, 2023
487e2ab
chore: show diff
knqyf263 Nov 10, 2023
42e2cb1
docs: sort aws services
knqyf263 Nov 10, 2023
1a85fa4
Merge branch 'main' into unify_walk
knqyf263 Nov 14, 2023
3d05ffc
Merge branch 'main' into unify_walk
knqyf263 Nov 16, 2023
1918ac9
fix: lint fixes
knqyf263 Nov 16, 2023
b7d9f64
test: fix mock signature
knqyf263 Nov 16, 2023
ae0399c
refactor: remove option.Init()
knqyf263 Nov 16, 2023
d92eceb
Merge branch 'main' into unify_walk
knqyf263 Feb 1, 2024
6d8783f
Merge branch 'main' into unify_walk
knqyf263 Apr 16, 2024
61eddab
chore: replace run.skip-* with issues.exclude-*
knqyf263 Apr 16, 2024
025e73f
fix: linter issues
knqyf263 Apr 16, 2024
264e77d
chore: bump golangci-lint
knqyf263 Apr 16, 2024
15cdb30
chore: remove a debug line
knqyf263 Apr 16, 2024
5717829
fix: use skip dirs from options
knqyf263 Apr 17, 2024
e819060
refactor: remove an import alias
knqyf263 Apr 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into unify_walk
Signed-off-by: knqyf263 <knqyf263@gmail.com>
  • Loading branch information
knqyf263 committed Nov 16, 2023
commit 3d05ffcd5361f4a39f3127301fbeb2893d55f93d
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ https://user-images.githubusercontent.com/1161307/171013513-95f18734-233d-45d3-a
</details>

```bash
trivy fs --scanners vuln,secret,config myproject/
trivy fs --scanners vuln,secret,misconfig myproject/
```

<details>
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/configuration/others.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ You can enable/disable scanners with the `--scanners` flag.
Supported values:

- vuln
- config
- misconfig
- secret
- license

Expand Down
17 changes: 14 additions & 3 deletions docs/docs/coverage/iac/cloudformation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,34 @@
Trivy supports the scanners listed in the table below.

| Scanner | Supported |
| :----------------: | :-------: |
|:------------------:|:---------:|
| [Misconfiguration] | ✓ |
| [Secret] | ✓ |

It supports the following formats.

| Format | Supported |
| :----: | :-------: |
|:------:|:---------:|
| JSON | ✓ |
| YAML | ✓ |

## Misconfiguration
Trivy recursively searches directories and scans all found CloudFormation files.
It evaluates properties, functions, and other elements within CloudFormation files to detect misconfigurations.

### Value Overrides
You can provide `cf-params` with path to [CloudFormation Parameters] file to Trivy to scan your CloudFormation code with parameters.

```bash
trivy conf --cf-params params.json ./infrastructure/cf
```

You can check a [CloudFormation Parameters Example]

## Secret
The secret scan is performed on plain text files, with no special treatment for CloudFormation.

[Misconfiguration]: ../../scanner/misconfiguration/index.md
[Secret]: ../../scanner/secret.md
[Secret]: ../../scanner/secret.md
[CloudFormation Parameters]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html
[CloudFormation Parameters Example]: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudformation/deploy.html#supported-json-syntax
3 changes: 2 additions & 1 deletion docs/docs/references/configuration/cli/trivy_aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ trivy aws [flags]
```
--account string The AWS account to scan. It's useful to specify this when reviewing cached results for multiple accounts.
--arn string The AWS ARN to show results for. Useful to filter results once a scan is cached.
--cf-params strings specify paths to override the CloudFormation parameters files
--compliance string compliance report to generate (aws-cis-1.2,aws-cis-1.4)
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
--config-policy strings specify the paths to the Rego policy files or to the directories containing them, applying config files
Expand All @@ -82,7 +83,7 @@ trivy aws [flags]
-h, --help help for aws
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignorefile string specify .trivyignore file (default ".trivyignore")
--include-non-failures include successes and exceptions, available with '--scanners config'
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--list-all-pkgs enabling the option will output all packages regardless of vulnerability
--max-cache-age duration The maximum age of the cloud cache. Cached data will be requeried from the cloud provider if it is older than this. (default 24h0m0s)
-o, --output string output file name
Expand Down
3 changes: 2 additions & 1 deletion docs/docs/references/configuration/cli/trivy_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ trivy config [flags] DIR
```
--cache-backend string cache backend (e.g. redis://localhost:6379) (default "fs")
--cache-ttl duration cache TTL when using redis as cache backend
--cf-params strings specify paths to override the CloudFormation parameters files
--clear-cache clear image caches without scanning
--compliance string compliance report to generate
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
Expand All @@ -26,7 +27,7 @@ trivy config [flags] DIR
-h, --help help for config
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignorefile string specify .trivyignore file (default ".trivyignore")
--include-non-failures include successes and exceptions, available with '--scanners config'
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--k8s-version string specify k8s version to validate outdated api by it (example: 1.21.0)
--module-dir string specify directory to the wasm modules that will be loaded (default "$HOME/.trivy/modules")
-o, --output string output file name
Expand Down
7 changes: 4 additions & 3 deletions docs/docs/references/configuration/cli/trivy_filesystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ trivy filesystem [flags] PATH
```
--cache-backend string cache backend (e.g. redis://localhost:6379) (default "fs")
--cache-ttl duration cache TTL when using redis as cache backend
--cf-params strings specify paths to override the CloudFormation parameters files
--clear-cache clear image caches without scanning
--compliance string compliance report to generate
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
Expand All @@ -45,7 +46,7 @@ trivy filesystem [flags] PATH
--ignored-licenses strings specify a list of license to ignore
--ignorefile string specify .trivyignore file (default ".trivyignore")
--include-dev-deps include development dependencies in the report (supported: npm, yarn)
--include-non-failures include successes and exceptions, available with '--scanners config'
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--java-db-repository string OCI repository to retrieve trivy-java-db from (default "ghcr.io/aquasecurity/trivy-java-db")
--license-confidence-level float specify license classifier's confidence level (default 0.9)
--license-full eagerly look for licenses in source code headers and license files
Expand All @@ -54,7 +55,7 @@ trivy filesystem [flags] PATH
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--parallel int number of goroutines enabled for parallel scanning (default 5)
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
--policy-namespaces strings Rego namespaces
Expand All @@ -68,7 +69,7 @@ trivy filesystem [flags] PATH
--reset remove all caches and database
--reset-policy-bundle remove policy bundle
--sbom-sources strings [EXPERIMENTAL] try to retrieve SBOM from the specified sources (oci,rekor)
--scanners strings comma-separated list of what security issues to detect (vuln,config,secret,license) (default [vuln,secret])
--scanners strings comma-separated list of what security issues to detect (vuln,misconfig,secret,license) (default [vuln,secret])
--secret-config string specify a path to config file for secret scanning (default "trivy-secret.yaml")
--server string server address in client mode
-s, --severity strings severities of security issues to be displayed (UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL) (default [UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL])
Expand Down
9 changes: 4 additions & 5 deletions docs/docs/references/configuration/cli/trivy_image.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,9 @@ trivy image [flags] IMAGE_NAME
--ignore-unfixed display only fixed vulnerabilities
--ignored-licenses strings specify a list of license to ignore
--ignorefile string specify .trivyignore file (default ".trivyignore")
--image-config-scanners strings comma-separated list of what security issues to detect on container image configurations (config,secret)
--image-config-scanners strings comma-separated list of what security issues to detect on container image configurations (misconfig,secret)
--image-src strings image source(s) to use, in priority order (docker,containerd,podman,remote) (default [docker,containerd,podman,remote])
--include-non-failures include successes and exceptions, available with '--scanners config'
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--input string input file path instead of image name
--java-db-repository string OCI repository to retrieve trivy-java-db from (default "ghcr.io/aquasecurity/trivy-java-db")
--license-confidence-level float specify license classifier's confidence level (default 0.9)
Expand All @@ -73,7 +73,7 @@ trivy image [flags] IMAGE_NAME
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--parallel int number of goroutines enabled for parallel scanning (default 5)
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--platform string set platform in the form os/arch if image is multi-platform capable
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
Expand All @@ -89,7 +89,7 @@ trivy image [flags] IMAGE_NAME
--reset remove all caches and database
--reset-policy-bundle remove policy bundle
--sbom-sources strings [EXPERIMENTAL] try to retrieve SBOM from the specified sources (oci,rekor)
--scanners strings comma-separated list of what security issues to detect (vuln,config,secret,license) (default [vuln,secret])
--scanners strings comma-separated list of what security issues to detect (vuln,misconfig,secret,license) (default [vuln,secret])
--secret-config string specify a path to config file for secret scanning (default "trivy-secret.yaml")
--server string server address in client mode
-s, --severity strings severities of security issues to be displayed (UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL) (default [UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL])
Expand All @@ -100,7 +100,6 @@ trivy image [flags] IMAGE_NAME
--skip-policy-update skip fetching rego policy updates
-t, --template string output template
--tf-exclude-downloaded-modules exclude misconfigurations for downloaded terraform modules
--tf-vars strings specify paths to override the Terraform tfvars files
--token string for authentication in client/server mode
--token-header string specify a header name for token in client/server mode (default "Trivy-Token")
--trace enable more verbose trace output for custom queries
Expand Down
7 changes: 3 additions & 4 deletions docs/docs/references/configuration/cli/trivy_kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ trivy kubernetes [flags] { cluster | all | specific resources like kubectl. eg:
--ignore-unfixed display only fixed vulnerabilities
--ignorefile string specify .trivyignore file (default ".trivyignore")
--image-src strings image source(s) to use, in priority order (docker,containerd,podman,remote) (default [docker,containerd,podman,remote])
--include-non-failures include successes and exceptions, available with '--scanners config'
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--java-db-repository string OCI repository to retrieve trivy-java-db from (default "ghcr.io/aquasecurity/trivy-java-db")
--k8s-version string specify k8s version to validate outdated api by it (example: 1.21.0)
--kubeconfig string specify the kubeconfig file path to use
Expand All @@ -65,7 +65,7 @@ trivy kubernetes [flags] { cluster | all | specific resources like kubectl. eg:
--node-collector-namespace string specify the namespace in which the node-collector job should be deployed (default "trivy-temp")
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--parallel int number of goroutines enabled for parallel scanning (default 5)
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
--policy-namespaces strings Rego namespaces
Expand All @@ -79,7 +79,7 @@ trivy kubernetes [flags] { cluster | all | specific resources like kubectl. eg:
--reset remove all caches and database
--reset-policy-bundle remove policy bundle
--sbom-sources strings [EXPERIMENTAL] try to retrieve SBOM from the specified sources (oci,rekor)
--scanners strings comma-separated list of what security issues to detect (vuln,config,secret,rbac) (default [vuln,config,secret,rbac])
--scanners strings comma-separated list of what security issues to detect (vuln,misconfig,secret,rbac) (default [vuln,misconfig,secret,rbac])
--secret-config string specify a path to config file for secret scanning (default "trivy-secret.yaml")
-s, --severity strings severities of security issues to be displayed (UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL) (default [UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL])
--skip-db-update skip updating vulnerability database
Expand All @@ -89,7 +89,6 @@ trivy kubernetes [flags] { cluster | all | specific resources like kubectl. eg:
--skip-policy-update skip fetching rego policy updates
-t, --template string output template
--tf-exclude-downloaded-modules exclude misconfigurations for downloaded terraform modules
--tf-vars strings specify paths to override the Terraform tfvars files
--tolerations strings specify node-collector job tolerations (example: key1=value1:NoExecute,key2=value2:NoSchedule)
--trace enable more verbose trace output for custom queries
--username strings username. Comma-separated usernames allowed.
Expand Down
7 changes: 4 additions & 3 deletions docs/docs/references/configuration/cli/trivy_repository.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
--branch string pass the branch name to be scanned
--cache-backend string cache backend (e.g. redis://localhost:6379) (default "fs")
--cache-ttl duration cache TTL when using redis as cache backend
--cf-params strings specify paths to override the CloudFormation parameters files
--clear-cache clear image caches without scanning
--commit string pass the commit hash to be scanned
--config-data strings specify paths from which data for the Rego policies will be recursively loaded
Expand All @@ -45,7 +46,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
--ignored-licenses strings specify a list of license to ignore
--ignorefile string specify .trivyignore file (default ".trivyignore")
--include-dev-deps include development dependencies in the report (supported: npm, yarn)
--include-non-failures include successes and exceptions, available with '--scanners config'
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--java-db-repository string OCI repository to retrieve trivy-java-db from (default "ghcr.io/aquasecurity/trivy-java-db")
--license-confidence-level float specify license classifier's confidence level (default 0.9)
--license-full eagerly look for licenses in source code headers and license files
Expand All @@ -54,7 +55,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--parallel int number of goroutines enabled for parallel scanning (default 5)
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
--policy-namespaces strings Rego namespaces
Expand All @@ -67,7 +68,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
--reset remove all caches and database
--reset-policy-bundle remove policy bundle
--sbom-sources strings [EXPERIMENTAL] try to retrieve SBOM from the specified sources (oci,rekor)
--scanners strings comma-separated list of what security issues to detect (vuln,config,secret,license) (default [vuln,secret])
--scanners strings comma-separated list of what security issues to detect (vuln,misconfig,secret,license) (default [vuln,secret])
--secret-config string specify a path to config file for secret scanning (default "trivy-secret.yaml")
--server string server address in client mode
-s, --severity strings severities of security issues to be displayed (UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL) (default [UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL])
Expand Down
Loading
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.