Kylotan Kylotan - 1 year ago 51
Python Question

Using custom formatter classes with Python's logging.config module

I have the following logging class, which works fine when assigned as a formatter in code. It extends an existing formatter by prepending a string to the start of the message to be logged, to help show the importance of the message. (I don't just use

in the format string because I don't want to show DEBUG or INFO prefixes.)

class PrependErrorLevelFormatter(logging.Formatter):
def __init__(self, default):
self._default_formatter = default
def format(self, record):
if record.levelno == logging.WARNING:
record.msg = "[WARNING] " + record.msg
elif record.levelno == logging.ERROR:
record.msg = "[ERROR] " + record.msg
elif record.levelno == logging.CRITICAL:
record.msg = "[CRITICAL] " + record.msg
return self._default_formatter.format(record)

Now I want to be able to assign it via a configuration file loaded in by logging.config.fileConfig(). I have tried syntax like this:

format=%(asctime)s %(message)s

Unfortunately I get errors resolving this class:

File "C:\Python27\lib\logging\", line 70, in fileConfig
formatters = _create_formatters(cp)
File "C:\Python27\lib\logging\", line 127, in _create_formatters
c = _resolve(class_name)
File "C:\Python27\lib\logging\", line 88, in _resolve
found = __import__(used)
ImportError: No module named PrependErrorLevelFormatter

I have tried prefixing the class name with the name of the module it is in, but get the same error. And even if it could resolve the class, it will probably not work due to the extra default formatter argument I need to provide.

How can I achieve the result I want using the logging.config system?

Answer Source

As you are using Python 2.7, you could use dictionary-based configuration using dictConfig(): this is more flexible than fileConfig(), as it allows use of arbitrary callables as factories to return e.g. handlers, formatters, or filters.

If you have to use fileConfig(), you'll have to construct a callable which takes format and datefmt string values and returns an instance of your class. The class value just needs to resolve to a callable, not an actual class. Here's a setup that works: In this gist, I have a file which contains the formatter definition, and a script which uses it via fileConfig(). Just get the files into a scratch directory and run - you should see output like this:

20:17:59 debug message
20:17:59 info message
20:17:59 [WARNING] warning message
20:17:59 [ERROR] error message
20:17:59 [CRITICAL] critical message

which appears to be what you need.

Note that you can use an alternative design for your formatter, which should do the same job:

class AltCustomFormatter(logging.Formatter):
    def format(self, record):
        if record.levelno in (logging.WARNING,
            record.msg = '[%s] %s' % (record.levelname, record.msg)
        return super(AltCustomFormatter , self).format(record)

To use this, you don't need a separate factory function, so you can just use


instead of


and it should work - it did for me when I tested just now using Python 2.7.1 :-)