Skip to content
/ logu Public

Extract patterns from unstructured log messages

License

Notifications You must be signed in to change notification settings

ynqa/logu

Repository files navigation

logu

logu.gif

logu is for extracting patterns from (streaming) unstructured log messages.

For parsing unstructured logs, it uses the parser from Drain. In simple terms, it tokenizes log messages, builds a tree structure, and groups similar logs into a single cluster, converting unstructured log data into a format that can be organized and analyzed.

This approach is also used by Grafana Loki. If you are interested in log parsers themselves, other methods are summarized at logpai/logparser, so please take a look.

Features

  • Extract patterns from streaming log messages
  • Enables more detailed analysis
    • Displays the number of messages included and a list of specific examples in the cluster
    • Identifies attributes such as IP, port

Installation

Homebrew

brew install ynqa/tap/logu

Cargo

cargo install logu

Examples

stern --context kind-kind - | logu

Keymap

Key Action
Ctrl + C Exit logu

Usage

Usage: logu [OPTIONS]

Options:
      --retrieval-timeout <RETRIEVAL_TIMEOUT_MILLIS>
          Timeout to read a next line from the stream in milliseconds. [default: 10]
      --render-interval <RENDER_INTERVAL_MILLIS>
          Interval to render the list in milliseconds. [default: 100]
      --train-interval <TRAIN_INTERVAL_MILLIS>
          [default: 10]
      --cluster-size-th <CLUSTER_SIZE_TH>
          Threshold to filter out small clusters. [default: 0]
      --max-clusters <MAX_CLUSTERS>

      --max-node-depth <MAX_NODE_DEPTH>
          [default: 2]
      --sim-th <SIM_TH>
          [default: 0.4]
      --max-children <MAX_CHILDREN>
          [default: 100]
      --param-str <PARAM_STR>
          [default: <*>]
  -h, --help
          Print help (see more with '--help')
  -V, --version
          Print version