Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logger: Expose max_lines and correct flag types #5956

Merged
merged 1 commit into from
Oct 28, 2019
Merged

logger: Expose max_lines and correct flag types #5956

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/wiki/installation/cli-flags.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,12 +235,16 @@ See the **tls**/[remote](../deployment/remote.md) plugin documentation. This is

Optionally enable GZIP compression for request bodies when sending. This is optional, and disabled by default, as the deployment must explicitly know that the logging endpoint supports GZIP for content encoding.

`--logger_tls_max=1048576`
`--logger_tls_max_linesize=1048576`

It is common for TLS/HTTPS servers to enforce a maximum request body size. The default behavior in osquery is to enforce each log line be under 1M bytes. This means each result line from a query's results cannot exceed 1M, this is very unlikely. Each log attempt will try to forward up to 1024 lines. If your service is limited request bodies, configure the client to limit the log line size.

Use this only in emergency situations as size violations are dropped. It is extremely uncommon for this to occur, as the `--value_max` for each column would need to be drastically larger, or the offending table would have to implement several hundred columns.

`--logger_tls_max_lines=1024`

This configures the max number of log lines to send every period (meaning every `logger_tls_period`).

`--distributed_tls_read_endpoint=`

The URI path which will be used, in conjunction with `--tls_hostname`, to create the remote URI for retrieving distributed queries when using the **tls** distributed plugin.
Expand Down
33 changes: 22 additions & 11 deletions plugins/logger/tls_logger.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

#include <osquery/enroll.h>
#include <osquery/flags.h>
#include <osquery/flagalias.h>
#include <osquery/registry.h>

#include <osquery/remote/serializers/json.h>
Expand All @@ -27,20 +28,29 @@

namespace osquery {

constexpr size_t kTLSMaxLogLines = 1024;
CLI_FLAG(uint64,
logger_tls_max_lines,
1024,
"Max number of logs to send per period");

FLAG(string, logger_tls_endpoint, "", "TLS/HTTPS endpoint for results logging");
CLI_FLAG(string,
logger_tls_endpoint,
"",
"TLS/HTTPS endpoint for results logging");

FLAG(uint64,
logger_tls_period,
4,
"Seconds between flushing logs over TLS/HTTPS");
CLI_FLAG(uint64,
logger_tls_period,
4,
"Seconds between flushing logs over TLS/HTTPS");

FLAG(uint64,
logger_tls_max,
logger_tls_max_linesize,
1 * 1024 * 1024,
"Max size in bytes allowed per log line");

// The flag name logger_tls_max is deprecated.
FLAG_ALIAS(google::uint64, logger_tls_max, logger_tls_max_linesize);

FLAG(bool, logger_tls_compress, false, "GZip compress TLS/HTTPS request body");

REGISTER(TLSLoggerPlugin, "logger", "tls");
Expand All @@ -49,7 +59,7 @@ TLSLogForwarder::TLSLogForwarder()
: BufferedLogForwarder("TLSLogForwarder",
"tls",
std::chrono::seconds(FLAGS_logger_tls_period),
kTLSMaxLogLines) {
FLAGS_logger_tls_max_lines) {
uri_ = TLSRequestHelper::makeURI(FLAGS_logger_tls_endpoint);
}

Expand Down Expand Up @@ -98,8 +108,9 @@ Status TLSLogForwarder::send(std::vector<std::string>& log_data,
auto children = params.newArray();
iterate(log_data, ([&params, &children](std::string& item) {
// Enforce a max log line size for TLS logging.
if (item.size() > FLAGS_logger_tls_max) {
LOG(WARNING) << "Line exceeds TLS logger max: " << item.size();
if (item.size() > FLAGS_logger_tls_max_linesize) {
LOG(WARNING)
<< "Linesize exceeds TLS logger maximum: " << item.size();
return;
}

Expand All @@ -123,4 +134,4 @@ Status TLSLogForwarder::send(std::vector<std::string>& log_data,
}
return TLSRequestHelper::go<JSONSerializer>(uri_, params, response);
}
}
} // namespace osquery