Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"pyomo" Logger Sends Logs To Stdout Irrespective of User's Logging Config #1760

Closed
DavidMuller opened this issue Dec 19, 2020 · 0 comments · Fixed by #1797
Closed

"pyomo" Logger Sends Logs To Stdout Irrespective of User's Logging Config #1760

DavidMuller opened this issue Dec 19, 2020 · 0 comments · Fixed by #1797
Labels

Comments

@DavidMuller
Copy link

Hello,

Thank you for all the hard work on Pyomo!

We are having some trouble with pyomo not respecting our custom logging configuration, and unconditionally emitting its logs to stdout in a hardcoded format.

Consider, for example, how pyomo ignores a Python logging configuration that sends all logs into a StringIO buffer with a custom format message:

>>> import logging
>>> from io import StringIO
>>> import pyomo.environ
>>> 
>>> string_io = StringIO()
>>> handler = logging.StreamHandler(stream=string_io)
>>> logging.basicConfig(format='custom:%(message)s', handlers=[handler])
>>> 
>>> logging.getLogger("pyomo").warning("pyomo warning")
WARNING: pyomo warning
>>> logging.warning("example warning")
>>> string_io.getvalue()
'custom:pyomo warning\ncustom:example warning\n'
>>> 

Unfortunately, the "pyomo" logger emits its log to both stdout and the string_io buffer...Additionally, the log it emits to stdout also does not respect the custom format configured in the logging.basicConfig call---the message pyomo emits to stdout doesn't have custom as its prefix.

I think the root cause of this issue is that pyomo adds a custom handler to itself in common/log.py:

pyomo/pyomo/common/log.py

Lines 165 to 171 in 263d5b1

#
# Set up the root Pyomo namespace logger
#
_logger = logging.getLogger('pyomo')
_logger.addHandler( LogHandler(
PYOMO_ROOT_DIR, verbosity=lambda: _logger.isEnabledFor(logging.DEBUG) ))
_logger.setLevel(logging.WARNING)

This custom handler, evidently, is responsible for the log messages that always wind up in stdout in the wrong format irrespective of the users's choices. The Python docs advise against adding a handler in a library's logger because it means that developers are unable to configure logging to suit their needs:

Note It is strongly advised that you do not add any handlers other than NullHandler to your library’s loggers. This is because the configuration of handlers is the prerogative of the application developer who uses your library. The application developer knows their target audience and what handlers are most appropriate for their application: if you add handlers ‘under the hood’, you might well interfere with their ability to carry out unit tests and deliver logs which suit their requirements.

In our application, for example, we want all log messages to have a custom JSON format and be sent to stderr. But, we wind up with many logs from pyomo in stdout without our custom JSON formatting.

We can work around this problem by preemptively running import pyomo.environ and then clobbering all of the handlers on the "pyomo" logger, but---of course---that solution is pretty brittle.

Thanks for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants