Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[8.x](backport #41795) Add registry to Filebeat's diagnostic #42413

Open
wants to merge 2 commits into
base: 8.x
Choose a base branch
from
Open

[8.x](backport #41795) Add registry to Filebeat's diagnostic #42413

wants to merge 2 commits into from
+653 −1

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jan 23, 2025

Proposed commit message

This commit adds Filebeat's registry folder to the Elastic-Agent
diagnostics. It's called registry.tar.gz and includes all registry files
on ${path.home}/registry.

The registry is first archived into a temporary tar file. The
temporary file is created by calling os.CreateTemp and will use the
OS's temporary folder. Then it's gziped in memory and returned to
Elastic-Agent, finally the temporary file is removed from the disk.

If the final gziped file is more than 20mb, it is skipped due to its large
size.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

None.

Author's Checklist

How to test this PR locally

The easiest way to test it locally is to package the Elastic-Agent with your local build of Beats by setting EXTERNAL=false when calling mage build in the Elastic-Agent folder.

Then, deploy the Elastic-Agent, let it ingest some logs and request the diagnostics. Extract the diagnostics and look for registry.tar.gz in components/filestram-* folders.

You can also run the integration tests:

go test -tags integration -count=1 -run=TestEmptyegistryIsInDiagnostics -v ./tests/integration
go test -tags integration -count=1 -run=TestFilestreamRegistryIsInDiagnostics -v ./tests/integration

Related issues

## Use cases
## Screenshots

Logs

Full log entries

{"log.level":"debug","@timestamp":"2024-11-27T11:00:35.812-0500","log.logger":"diagnostics","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.gzipRegistry","file.name":"beater/diagnostics.go","file.line":54},"message":"temporary file '/tmp/filebeat-registry-640175774.tar' created","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-11-27T11:00:35.812-0500","log.logger":"diagnostics","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.tarFolder","file.name":"beater/diagnostics.go","file.line":116},"message":"starting to walk '/home/tiago/devel/beats/x-pack/filebeat/build/integration-tests/TestRegistryIsInDiagnostics3276586231/data/registry'","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-11-27T11:00:35.812-0500","log.logger":"diagnostics","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.tarFolder.func1","file.name":"beater/diagnostics.go","file.line":140},"message":"adding '/home/tiago/devel/beats/x-pack/filebeat/build/integration-tests/TestRegistryIsInDiagnostics3276586231/data/registry/filebeat/log.json' to the tar archive","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-11-27T11:00:35.812-0500","log.logger":"diagnostics","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.tarFolder.func1","file.name":"beater/diagnostics.go","file.line":140},"message":"adding '/home/tiago/devel/beats/x-pack/filebeat/build/integration-tests/TestRegistryIsInDiagnostics3276586231/data/registry/filebeat/meta.json' to the tar archive","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2024-11-27T11:00:35.812-0500","log.logger":"diagnostics","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.gzipRegistry","file.name":"beater/diagnostics.go","file.line":63},"message":"finished gziping Filebeat's registry","service.name":"filebeat","ecs.version":"1.6.0"}

Only the message field:

"temporary file '/tmp/filebeat-registry-640175774.tar' created"
"starting to walk '/home/tiago/devel/beats/x-pack/filebeat/build/integration-tests/TestRegistryIsInDiagnostics3276586231/data/registry'"
"adding '/home/tiago/devel/beats/x-pack/filebeat/build/integration-tests/TestRegistryIsInDiagnostics3276586231/data/registry/filebeat/log.json' to the tar archive"
"adding '/home/tiago/devel/beats/x-pack/filebeat/build/integration-tests/TestRegistryIsInDiagnostics3276586231/data/registry/filebeat/meta.json' to the tar archive"
"finished gziping Filebeat's registry"
```<hr>This is an automatic backport of pull request #41795 done by [Mergify](https://mergify.com).

@belimawr @mergify
Add registry to Filebeat's diagnostic (#41795)
ee1eed9 
This commit adds Filebeat's registry folder to the Elastic-Agent
diagnostics. It's called `registry.tar.gz` and includes all registry files
on `${path.home}/registry`.

The registry is first archived into a temporary tar file. The
temporary file is created by calling `os.CreateTemp` and will use the
OS's temporary folder. Then it's gziped in memory and returned to
Elastic-Agent, finally the temporary file is removed from the disk.

If the final gziped file is more than 20mb, it is skipped due to its large
size.

(cherry picked from commit 572cb0e)
@mergify mergify bot requested a review from a team as a code owner January 23, 2025 20:57
@mergify mergify bot added the backport label Jan 23, 2025
@mergify mergify bot requested review from mauri870 and faec and removed request for a team January 23, 2025 20:57
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 23, 2025
@github-actions github-actions bot added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jan 23, 2025
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 23, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mauri870 mauri870 Awaiting requested review from mauri870 mauri870 is a code owner automatically assigned from elastic/elastic-agent-data-plane

@faec faec Awaiting requested review from faec faec is a code owner automatically assigned from elastic/elastic-agent-data-plane

Assignees

@belimawr belimawr

Labels
backport Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@elasticmachine @belimawr