Skip to content
This repository has been archived by the owner on Feb 24, 2024. It is now read-only.

hodlen/async-logger

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

async-logger

Overview

async-logger is a Python logging module designed for asynchronous logging. It leverages threading for efficient log handling and offers features like batch processing, graceful and forced shutdown, and automatic file rotation based on dates.

Features

  • Asynchronous Logging: Utilizes Python threading to ensure non-blocking log writes.
  • Queue-Based Batch Processing: Enhances performance by processing log messages in batches.
  • Automatic File Rotation: Daily log file rotation to organize log entries.
  • Graceful and Immediate Shutdown: Supports both orderly shutdown and immediate termination.

Implementation

  • Threading: A daemon thread is used for log processing, separating the logging concern from the main application flow.
  • Batched Log Processing: Handles all messages in the queue at once, minimizing thread context-switching.
  • Error Handling: Logs file writing errors to sys.stderr, ensuring application continuity.
  • Performance tuning: The queue size and batch processing interval can be tuned to optimize performance under different workloads.

Usage

from FileLogger import FileLogger
from pathlib import Path

# Initialize the logger
logger = FileLogger(logging_dir=Path('./logs'))

# Write a log message
logger.write("Sample log message")

# Stop the logger
logger.stop(graceful=True)

Testing (on *nix)

./self_test.sh

Notes

  • Threading vs. asyncio: Chosen for compatibility with synchronous environments, avoiding the complexities of mixing sync and async code.
  • Performance Considerations: Batch processing in threading reduces overhead, making it suitable for high-load scenarios.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published