Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimize the configs of logger #3101

Merged
merged 8 commits into from
Sep 27, 2021
Merged

optimize the configs of logger #3101

merged 8 commits into from
Sep 27, 2021

Conversation

hehechen
Copy link
Contributor

@hehechen hehechen commented Sep 16, 2021 •
edited
Loading

What problem does this PR solve?

#2690

  1. The suffix of logging files are ends with ".1",".2", which are hard for finding logs. Making the log file end with a timestamp will help to find the log.
  2. logger.level is dynamic updatable, make logger.count/logger.size dynamic updatable.

design doc: https://docs.google.com/document/d/1CxrYe8h9f1xijanBAFimh9iDCtY4BOVZ_d6QnpPId94/edit?n=Enhance_configuration_of_TiFlash_RAStorage

What is changed and how it works?

What's Changed:

  1. Inherit the Poco::ArchiveByTimestampStrategy class and Poco::FileChannel class, and override ArchiveByTimestampStrategy::Archive and FileChannel::setArchive, modify the archive file name suffix in the subclass of ArchiveByTimestampStrategy.
    image
  2. Add the MutableSplitterChannel class as a subclass of Poco::SplitterChannel, traverse the channels in the SplitterChannel, and reload the logger.size and logger.count properties.

Check List

  • Unit test
  • Integration test
Optimize the configs of logger

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Sep 16, 2021 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • JaySon-Huang

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 16, 2021
@hehechen hehechen mentioned this pull request Sep 16, 2021
Merged
@hehechen
1.logging files name ends with the timestamp;
e7f4997 
2.make logger.count/logger.size dynamic updatable
JaySon-Huang
JaySon-Huang reviewed Sep 17, 2021
View reviewed changes
dbms/CMakeLists.txt Outdated Show resolved Hide resolved
@solotzg
Copy link
Contributor

solotzg commented Sep 17, 2021 •
edited
Loading

@CalvinNeo PTAL.
Can DiagnosticsService::search_log be compatible with such process and search history logs?
Shall we need to compress log files to xxx.gz or just keep original content?

JaySon-Huang
JaySon-Huang reviewed Sep 23, 2021
View reviewed changes
dbms/src/Common/TiFlashArchiveByTimestampsStrategy.h Outdated Show resolved Hide resolved
dbms/src/Common/TiFlashLogFileChannel.cpp Outdated Show resolved Hide resolved
dbms/src/Common/TiFlashLogFileChannel.cpp Outdated Show resolved Hide resolved
libs/libdaemon/src/BaseDaemon.cpp Outdated Show resolved Hide resolved
dbms/CMakeLists.txt Outdated Show resolved Hide resolved
@hehechen hehechen force-pushed the logger_config branch 3 times, most recently from e06a79a to 4b5a2a5 Compare September 26, 2021 05:59
@hehechen hehechen force-pushed the logger_config branch 4 times, most recently from 409b3a4 to ffe1458 Compare September 26, 2021 09:07
@hehechen
Copy link
Contributor Author

/run-all-tests

@JaySon-Huang
Copy link
Contributor

/run-all-tests

@hehechen
Copy link
Contributor Author

/run-all-tests

JaySon-Huang
JaySon-Huang approved these changes Sep 27, 2021
View reviewed changes
Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a small comment

libs/libpocoext/include/Poco/Ext/TiFlashArchiveByTimestampsStrategy.h Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 27, 2021
@JaySon-Huang
Copy link
Contributor

@CalvinNeo PTAL.
Can DiagnosticsService::search_log be compatible with such process and search history logs?
Shall we need to compress log files to xxx.gz or just keep original content?

@solotzg @CalvinNeo Does this change make trouble for DiagnosticsService::search_log?

@CalvinNeo
Copy link
Member

@CalvinNeo PTAL.
Can DiagnosticsService::search_log be compatible with such process and search history logs?
Shall we need to compress log files to xxx.gz or just keep original content?

@solotzg @CalvinNeo Does this change make trouble for DiagnosticsService::search_log?

Currently, search_log only searches tiflash.log, so there will be no trouble.

@hehechen
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@hehechen: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: c4ca14e

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 27, 2021
@ti-chi-bot ti-chi-bot merged commit 22a8ea9 into pingcap:master Sep 27, 2021
@JaySon-Huang JaySon-Huang deleted the logger_config branch October 27, 2021 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

Assignees

@hehechen hehechen

Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@hehechen @ti-chi-bot @solotzg @JaySon-Huang @CalvinNeo