Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bin: reload: config: Implement hot reloading feature #6871

Merged
merged 60 commits into from
Apr 10, 2023

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Feb 15, 2023

Currently, fluent-bit does not support hot reloading feature.

For now, I implemented SIGHUP handler to handle hot reloading.

Some of plugins do not handle pause/resume operations correctly. So, I fixed their behavior by implementing pause/resume callbacks.

Note that fluent-bit uses mk_core as event handling library. This library is not compatible for reloading events.
I tried to add/kick and handle internal reloading events but I didn't get succeeded to reload inside flb_lib contexts.
This causes invalid fd errors or stopping main loop on mk_core.

EDIT: I made this feature as opt-in feature. To enable this, using -Y or --enable-hot-reload
option.

Related to #365.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 11:20 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 11:21 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 11:22 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 11:22 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 11:42 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 12:49 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 12:49 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 15, 2023 13:06 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 08:36 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 08:36 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 08:36 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 08:58 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 09:43 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 09:43 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 09:43 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 09:51 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 09:51 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 09:51 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 16, 2023 10:12 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 22, 2023 09:29 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 22, 2023 09:29 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 22, 2023 09:29 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 22, 2023 09:48 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 22, 2023 12:14 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr February 22, 2023 12:14 — with GitHub Actions Inactive
FLB_TLS_XXX improves portabilities and brings C11 TLS feature usages.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Using flb_stop and flb_destroy is enough to stop flb context.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
To enable this, provide -Y or --enable-hot-reload option at start up.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
This is because loading YAML specific context does not used for
fluent-bit executables. This test case should not unnecessary.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
Instead, we should check reloading with processor configuration.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
…roperty

checking

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
checking

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
checking

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
@cosmo0920 cosmo0920 temporarily deployed to pr April 10, 2023 02:39 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr April 10, 2023 02:39 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr April 10, 2023 02:40 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr April 10, 2023 03:01 — with GitHub Actions Inactive
@edsiper edsiper merged commit 16a6c75 into fluent:master Apr 10, 2023
@cosmo0920 cosmo0920 deleted the hot-reloading branch April 11, 2023 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants