Merge branches/log-booyah-6750-6: New logging system

Author: wsanchez, glyph Reviewer: ralphm Fixes: twisted#6750 git-svn-id: svn://svn.twistedmatrix.com/svn/Twisted/trunk@44442 bbbe8e31-12d6-0310-92fd-ac37d47ddeeb
yuankui · Apr 14, 2015 · 30eab99 · 30eab99
1 parent 6d52ade
commit 30eab99
Show file tree

Hide file tree

Showing 60 changed files with 8,332 additions and 557 deletions.
diff --git a/docs/core/howto/index.rst b/docs/core/howto/index.rst
@@ -48,6 +48,7 @@ Developer Guides
    application
    tap
    systemd
+   logger
    logging
    constants
    rdbms

diff --git a/docs/core/howto/listings/logger/ad_hoc.py b/docs/core/howto/listings/logger/ad_hoc.py
@@ -0,0 +1,12 @@
+from twisted.logger import Logger
+
+class AdHoc(object):
+    log = Logger(namespace="ad_hoc")
+
+    def __init__(self, a, b):
+        self.a = a
+        self.b = b
+
+    def logMessage(self):
+        self.log.info("message from {log_source} "
+                      "where a is {log_source.a} and b is {log_source.b}")
diff --git a/docs/core/howto/listings/logger/ad_hoc_save.py b/docs/core/howto/listings/logger/ad_hoc_save.py
@@ -0,0 +1,7 @@
+import io
+from twisted.logger import jsonFileLogObserver, globalLogPublisher
+from ad_hoc import AdHoc
+
+globalLogPublisher.addObserver(jsonFileLogObserver(io.open("log.json", "a")))
+
+AdHoc(3, 4).logMessage()
diff --git a/docs/core/howto/listings/logger/analyze.py b/docs/core/howto/listings/logger/analyze.py
@@ -0,0 +1,12 @@
+from twisted.logger import extractField
+
+fmt = (
+    "message from {log_source} "
+    "where a is {log_source.a} and b is {log_source.b}"
+)
+
+def analyze(event):
+    if event.get("log_format") == fmt:
+        a = extractField("log_source.a", event)
+        b = extractField("log_source.b", event)
+        print("A + B = " + repr(a + b))
diff --git a/docs/core/howto/listings/logger/loader-math.py b/docs/core/howto/listings/logger/loader-math.py
@@ -0,0 +1,5 @@
+import io
+from twisted.logger import eventsFromJSONLogFile
+
+for event in eventsFromJSONLogFile(io.open("log.json")):
+    print(sum(event["values"]))
diff --git a/docs/core/howto/listings/logger/loader.py b/docs/core/howto/listings/logger/loader.py
@@ -0,0 +1,10 @@
+import sys
+import io
+from twisted.logger import (
+    eventsFromJSONLogFile, textFileLogObserver
+)
+
+output = textFileLogObserver(sys.stdout)
+
+for event in eventsFromJSONLogFile(io.open("log.json")):
+    output(event)
diff --git a/docs/core/howto/listings/logger/logsource.py b/docs/core/howto/listings/logger/logsource.py
@@ -0,0 +1,15 @@
+from twisted.logger import Logger
+
+class MyObject(object):
+    log = Logger()
+
+    def __init__(self, value):
+        self.value = value
+
+    def doSomething(self, something):
+        self.log.info(
+            "Object with value {log_source.value!r} doing {something}.",
+            something=something
+        )
+
+MyObject(7).doSomething("a task")
diff --git a/docs/core/howto/listings/logger/offline_analyze.py b/docs/core/howto/listings/logger/offline_analyze.py
@@ -0,0 +1,6 @@
+import io
+from twisted.logger import eventsFromJSONLogFile
+from analyze import analyze
+
+for event in eventsFromJSONLogFile(io.open("log.json")):
+    analyze(event)
diff --git a/docs/core/howto/listings/logger/online_analyze.py b/docs/core/howto/listings/logger/online_analyze.py
@@ -0,0 +1,7 @@
+from twisted.logger import globalLogPublisher
+from analyze import analyze
+from ad_hoc import AdHoc
+
+globalLogPublisher.addObserver(analyze)
+
+AdHoc(3, 4).logMessage()
diff --git a/docs/core/howto/listings/logger/saver.py b/docs/core/howto/listings/logger/saver.py
@@ -0,0 +1,11 @@
+import io
+from twisted.logger import jsonFileLogObserver, Logger
+
+log = Logger(observer=jsonFileLogObserver(io.open("log.json", "a")),
+             namespace="saver")
+
+def loggit(values):
+    log.info("Some values: {values!r}", values=values)
+
+loggit([1234, 5678])
+loggit([9876, 5432])
diff --git a/docs/core/howto/logger.rst b/docs/core/howto/logger.rst
diff --git a/docs/core/howto/logging.rst b/docs/core/howto/logging.rst
@@ -7,32 +7,30 @@ Logging with twisted.python.log
 ===============================
 
 
+Note: New Logging System
+------------------------
 
+There is now a new logging system in Twisted (:api:`twisted.logger <twisted.logger>`) which is a replacement for :api:`twisted.python.log <twisted.python.log>`.
 
+The old logging API, described here, remains for compatibility, and is now implemented as a client of the new logging system.
 
+New code should adopt the new API.
+The new API, and notes on transitioning from the old API, is documented in the :ref:`logger HOWTO <core-howto-logger-main>`.
 
-Basic usage
------------
 
 
+Basic usage
+-----------
 
 Twisted provides a simple and flexible logging system in the :api:`twisted.python.log <twisted.python.log>` module.  It has three commonly used
 functions:
-
-
-
-
-
-
 
 :api:`twisted.python.log.LogPublisher.msg <msg>` 
 
   Logs a new message.  For example:
 
-
   .. code-block:: python
   
-      
       from twisted.python import log
       log.msg('Hello, world.')
 
@@ -46,10 +44,8 @@ functions:
   If you pass nothing, it will construct a Failure from the
   currently active exception, which makes it convenient to use in an ``except`` clause:
 
-
   .. code-block:: python
   
-      
       try:
           x = 1 / 0
       except:
@@ -59,143 +55,89 @@ functions:
 
   Starts logging to a given file-like object.  For example:
 
-
   .. code-block:: python
-  
       
       log.startLogging(open('/var/log/foo.log', 'w'))
   
-  
   or:
 
-
   .. code-block:: python
-  
       
       log.startLogging(sys.stdout)
   
-  
   or:
 
-
   .. code-block:: python
-  
       
       from twisted.python.logfile import DailyLogFile
       
       log.startLogging(DailyLogFile.fromFullPath("/var/log/foo.log"))
   
-  
-  
   By default, ``startLogging`` will also redirect anything written
   to ``sys.stdout`` and ``sys.stderr`` to the log.  You
   can disable this by passing ``setStdout=False`` to
   ``startLogging`` .
 
-
-
-
-
-
 Before ``startLogging`` is called, log messages will be
 discarded and errors will be written to stderr.
 
-
-
-
 
 Logging and twistd
 ~~~~~~~~~~~~~~~~~~
-
-
 
 If you are using ``twistd`` to run your daemon, it
 will take care of calling ``startLogging`` for you, and will also
 rotate log files.  See :ref:`twistd and tac <core-howto-application-twistd>` 
 and the ``twistd`` man page for details of using
 twistd.
 
-
-
-
 
 Log files
 ~~~~~~~~~
-
-
 
 The :api:`twisted.python.logfile <twisted.python.logfile>` module provides
 some standard classes suitable for use with ``startLogging`` , such
 as :api:`twisted.python.logfile.DailyLogFile <DailyLogFile>` ,
 which will rotate the log to a new file once per day.
 
-
-
-
 
 Using the standard library logging module
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-
 
 If your application uses the
 Python `standard    library logging module <http://docs.python.org/library/logging.html>`_ or you want to use its easy configuration but
 don't want to lose twisted-produced messages, the observer
 :api:`twisted.python.log.PythonLoggingObserver <PythonLoggingObserver>` 
 should be useful to you.
 
-
-
-
-
 You just start it like any other observer:
 
-
 .. code-block:: python
-
     
     observer = log.PythonLoggingObserver()
     observer.start()
 
-
-
 Then `configure the    standard library logging module <http://docs.python.org/library/logging.html>`_ to behave as you want.
 
-
-
-
-
 This method allows you to customize the log level received by the
 standard library logging module using the ``logLevel`` keyword:
 
-
 .. code-block:: python
 
-    
     log.msg("This is important!", logLevel=logging.CRITICAL)
     log.msg("Don't mind", logLevel=logging.DEBUG)
 
-
 Unless ``logLevel`` is provided, logging.INFO is used for ``log.msg`` 
 and ``logging.ERROR`` is used for ``log.err`` .
 
-
-
-
-
 One special care should be made when you use special configuration of
 the standard library logging module: some handlers (e.g. SMTP, HTTP) use the network and
 so can block inside the reactor loop. *Nothing* in ``PythonLoggingObserver`` is
 done to prevent that.
 
-
-
-
 
 Writing log observers
 ---------------------
-
-
 
 Log observers are the basis of the Twisted logging system.
 Whenever ``log.msg`` (or ``log.err`` ) is called, an
@@ -210,25 +152,11 @@ is just a callable that accepts a dictionary as its only argument.  You can
 then register it to receive all log events (in addition to any other
 observers):
 
-
-
-
-
 .. code-block:: python
-
     
     twisted.python.log.addObserver(yourCallable)
-
-
-
     
 The dictionary will have at least two items:
-
-
-
-
-
-
 
 message
 
@@ -242,18 +170,7 @@ isError
   ``log.err`` .  If this is set, there may be a ``failure`` 
   item in the dictionary as will, with a Failure object in it.
 
-
-
-
-
-
 Other items the built in logging functionality may add include:
-
-
-
-
-
-
 
 printed
 
@@ -262,44 +179,21 @@ printed
   ``isError`` is also true, it came from
   ``sys.stderr`` .
 
-
-
-
-
-
 You can pass additional items to the event dictionary by passing keyword
 arguments to ``log.msg`` and ``log.err`` .  The standard
 log observers will ignore dictionary items they don't use.
 
-
-
-
 Important notes:
 
-
-
-
-
-
 - Never block in a log observer, as it may run in main Twisted thread.
   This means you can't use socket or syslog standard library logging backends.
 - The observer needs to be thread safe if you anticipate using threads
   in your program.
 
 
-
-
-
-
 Customizing ``twistd``  logging
 -------------------------------
 
-
-
 The behavior of the logging that ``twistd`` does can be
 customized either with the ``--logger`` option or by setting the
 ``ILogObserver`` component on the application object.  See the :doc:`Application document <application>` for more information.
-
-
-
-
-Original file line number
+Diff line change
@@ Expand Up / @@ -48,6 +48,7 @@ Developer Guides @@
        application
        tap
        systemd
+       logger
        logging
        constants
        rdbms
@@ Expand Down @@