Skip to content
forked from odygrd/quill

Asynchronous Low Latency C++ Logging Library

License

Notifications You must be signed in to change notification settings

mayhemheroes/quill

 
 

Repository files navigation

Quill logo

Quill

Asynchronous Low Latency C++ Logging Library


homebrew vcpkg conan
brew install quill vcpkg install quill quill/[>=1.2.3]

Introduction

Quill is a cross-platform low latency logging library based on C++14/C++17.

There are two versions on the library:

v2 : C++17 v1.7 : C++14

Going forward any new features will only be added to the C++17 version of the library. The old library (v1.7.x) still remains there and there will be releases only for bug fixes.

The main goals of the library are:

  • Simplicity A small example code snippet should be enough to get started and use most of features.
  • Performance Ultra low latency. No formatting on the hot-path, asynchronous only mode. No hot-path allocations for fundamental types, enums and strings (including std::string and std::string_view). Any other custom or user defined type gets copy constructed with the formatting done on a backend worker thread.
  • Convenience Ease application monitoring/debugging. Latency is equal to latencies of binary loggers, but the produced log is in human readable form.

Features

  • Log anything - Blazing fast. See Benchmarks.
  • Format outside the hot-path in a backend logging thread. For non-built-in types ostream::operator<<() is called on a copy of the object by the backend logging thread. Unsafe to copy non-trivial user defined are detected in compile time. Those types can be tagged as safe-to-copy to avoid formatting them on the hot path. See User Defined Types.
  • Custom formatters. Logs can be formatted based on a user specified pattern. See Formatters.
  • Support for log stack traces. Store log messages in a ring buffer and display later on a higher severity log statement or on demand. See Backtrace Logging.
  • Various logging targets. See Handlers.
    • Console logging with colours support.
    • File Logging
    • Rotating log files
    • Time rotating log files
    • Custom Handlers
  • Filters for filtering log messages. See Filters.
  • guaranteed non-blocking or non-guaranteed logging. In non-guaranteed mode there is no heap allocation of a new queue but log messages can be dropped. See FAQ.
  • Support for wide character logging and wide character filenames (Windows and v1.7.x only).
  • Log statements in timestamp order even when produced by different threads. This makes debugging easier in multi-threaded applications.
  • Log levels can be completely stripped out at compile time reducing if branches.
  • Clean warning-free codebase even on high warning levels.
  • Crash safe behaviour with a build-in signal handler.
  • Type safe python style API with compile type checks and built-in support for logging STL types/containers by using the excellent {fmt} library.

Performance

🔥 ** Updated April 2022 ** 🔥

Log Numbers

The following message is logged 100'000 times per thread LOG_INFO(logger, "Logging int: {}, int: {}, double: {}", i, j, d).

The results in the tables below are in nanoseconds (ns).

1 Thread

Library 50th 75th 90th 95th 99th 99.9th Worst
Quill v2, Unbounded Queue 16 17 19 20 21 29 55
Quill v1.7, Dual Queue Enabled, Unbounded Queue 21 22 24 25 28 34 54
Quill v1.7, Dual Queue Disabled, Unbounded Queue 16 18 21 22 28 39 58
Quill v1.7, Dual Queue Enabled, Bounded Queue 20 21 23 24 27 32 46
fmtlog 16 18 20 21 26 36 47
PlatformLab NanoLog 52 66 76 81 92 107 192
MS BinLog 39 41 43 44 67 110 216
Reckless 62 72 79 87 107 126 168
Iyengar NanoLog 147 169 187 209 283 376 33623
spdlog 626 675 721 755 877 1026 1206
g3log 5551 5759 5962 6090 6338 6647 7133

4 Threads

Library 50th 75th 90th 95th 99th 99.9th Worst
Quill v2, Unbounded Queue 16 17 19 21 24 29 70
Quill v1.7, Dual Queue Enabled, Unbounded Queue 21 23 25 27 32 43 64
Quill v1.7, Dual Queue Disabled, Unbounded Queue 16 19 21 23 30 39 57
Quill v1.7, Dual Queue Enabled, Bounded Queue 20 21 23 24 27 38 59
fmtlog 16 18 19 21 25 34 53
PlatformLab NanoLog 53 67 77 82 93 131 236
MS BinLog 39 42 43 46 73 119 243
Reckless 46 60 75 88 112 156 262
Iyengar NanoLog 140 173 239 273 336 432 43605
spdlog 665 742 825 880 1069 1395 2087
g3log 5294 5532 5759 5901 6179 6521 7443

Log Numbers and Large Strings

The following message is logged 100'000 times per thread LOG_INFO(logger, "Logging int: {}, int: {}, string: {}", i, j, large_string). The large string is over 35 characters to avoid short string optimisation of std::string

1 Thread

Library 50th 75th 90th 95th 99th 99.9th Worst
Quill v2, Unbounded Queue 27 29 31 32 35 45 69
Quill v1.7, Dual Queue Enabled, Unbounded Queue 25 26 28 30 35 47 70
Quill v1.7, Dual Queue Disabled, Unbounded Queue 116 132 145 153 168 185 214
Quill v1.7, Dual Queue Enabled, Bounded Queue 26 27 30 31 36 47 65
fmtlog 27 29 32 34 40 49 67
PlatformLab NanoLog 68 81 96 107 126 151 222
MS BinLog 52 53 56 60 80 133 251
Reckless 211 236 262 280 317 522 1051
Iyengar NanoLog 157 176 196 220 290 372 22812
spdlog 652 715 775 827 953 1082 1453
g3log 4563 4752 4942 5066 5309 5633 6188

4 Threads

Library 50th 75th 90th 95th 99th 99.9th Worst
Quill v2, Unbounded Queue 28 30 32 34 38 45 73
Quill v1.7, Dual Queue Enabled, Unbounded Queue 25 27 29 30 37 53 83
Quill v1.7, Dual Queue Disabled, Unbounded Queue 125 138 151 160 176 192 247
Quill v1.7, Dual Queue Enabled, Bounded Queue 25 27 29 31 39 51 86
fmtlog 26 28 31 34 40 49 78
PlatformLab NanoLog 67 80 95 106 127 182 315
MS BinLog 51 53 55 60 85 128 243
Reckless 184 204 226 240 283 531 761
Iyengar NanoLog 151 218 267 296 353 469 71636
spdlog 640 710 795 867 1097 1465 2259
g3log 3575 3776 3967 4089 4332 4650 5544

The benchmarks are done on Ubuntu - Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz with GCC 11.2

Each thread is pinned on a different cpu. Unfortunately the cores are not isolated on this system. If the backend logging thread is run in the same CPU as the caller hot-path threads, that slows down the log message processing on the backend logging thread and will cause the SPSC queue to fill faster and re-allocate.

Continuously logging messages in a loop makes the consumer (backend logging thread) unable to follow up and the queue will have to re-allocate or block for most logging libraries expect very high throughput binary loggers like PlatformLab Nanolog.

Therefore, a different approach was followed that suits more to a real time application:

  1. 20 messages are logged in a loop.
  2. calculate/store the average latency for those messages.
  3. wait between 1-2 ms.
  4. repeat for n iterations.

I run each logger benchmark 4 times and the above latencies are the second best result.

The benchmark code and results can be found here.

Supported Platforms And Compilers

Quill v1.7.x requires a C++14 compiler. Minimum required versions of supported compilers are shown in the below table.

Compiler Notes
GCC version >= 5.0
Clang version >= 5.0
MSVC++ version >= 14.3
Platform Notes
Linux Ubuntu, RHEL, Centos, Fedora
Windows Windows 10 - version 1607, Windows Server 2016
macOS Tested with Xcode 9.4

Basic usage

#include "quill/Quill.h"

int main()
{
  quill::enable_console_colours();
  quill::start();

  quill::Logger* logger = quill::get_logger();
  logger->set_log_level(quill::LogLevel::TraceL3);

  // enable a backtrace that will get flushed when we log CRITICAL
  logger->init_backtrace(2, quill::LogLevel::Critical);

  LOG_BACKTRACE(logger, "Backtrace log {}", 1);
  LOG_BACKTRACE(logger, "Backtrace log {}", 2);

  LOG_INFO(logger, "Welcome to Quill!");
  LOG_ERROR(logger, "An error message. error code {}", 123);
  LOG_WARNING(logger, "A warning message.");
  LOG_CRITICAL(logger, "A critical error.");
  LOG_DEBUG(logger, "Debugging foo {}", 1234);
  LOG_TRACE_L1(logger, "{:>30}", "right aligned");
  LOG_TRACE_L2(logger, "Positional arguments are {1} {0} ", "too", "supported");
  LOG_TRACE_L3(logger, "Support for floats {:03.2f}", 1.23456);
}

Output

Screenshot-2020-08-14-at-01-09-43.png

CMake-Integration

External

Building and Installing Quill as Static Library
git clone https://github.com/odygrd/quill.git
mkdir cmake_build
cd cmake_build
make install

Note: To install in custom directory invoke cmake with -DCMAKE_INSTALL_PREFIX=/quill/install-dir/

Building and Installing Quill as Static Library With External libfmt
cmake -DCMAKE_PREFIX_PATH=/my/fmt/fmt-config.cmake-directory/ -DQUILL_FMT_EXTERNAL=ON -DCMAKE_INSTALL_PREFIX=/quill/install-dir/'

Then use the library from a CMake project, you can locate it directly with find_package()

Directory Structure
my_project/
├── CMakeLists.txt
├── main.cpp
CMakeLists.txt
# Set only if needed - quill was installed under a custom non-standard directory
set(CMAKE_PREFIX_PATH /test_quill/usr/local/)

find_package(quill REQUIRED)

# Linking your project against quill
add_executable(example main.cpp)
target_link_libraries(example PRIVATE quill::quill)
main.cpp

See basic usage

Embedded

To embed the library directly, copy the source folder to your project and call add_subdirectory() in your CMakeLists.txt file

Directory Structure
my_project/
├── quill/            (source folder)
├── CMakeLists.txt
├── main.cpp
CMakeLists.txt
cmake_minimum_required(VERSION 3.1.0)
project(my_project)

set(CMAKE_CXX_STANDARD 14)

add_subdirectory(quill)

add_executable(my_project main.cpp)
target_link_libraries(my_project PRIVATE quill::quill)
main.cpp

See basic usage

Documentation

Advanced usage and additional documentation can be found in the wiki pages.

The examples folder is also a good source of documentation.

License

Quill is licensed under the MIT License

Quill depends on third party libraries with separate copyright notices and license terms. Your use of the source code for these subcomponents is subject to the terms and conditions of the following licenses.

About

Asynchronous Low Latency C++ Logging Library

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C++ 98.1%
  • CMake 1.7%
  • C 0.2%