Parand Parand - 1 month ago 15
Python Question

Elegant setup of Python logging in Django

I have yet to find a way of setting up Python logging with Django that I'm happy with. My requirements are fairly simple:


  • Different log handlers for different events - that is, I want to be able to log to different files

  • Easy access to loggers in my modules. The module should be able to find its logger with little effort.

  • Should be easily applicable to command-line modules. Parts of the system are stand-alone command line or daemon processes. Logging should be easily usable with these modules.



My current setup is to use a
logging.conf
file and setup logging in each module I log from. It doesn't feel right.

Do you have a logging setup that you like? Please detail it: how do you setup the configuration (do you use
logging.conf
or set it up in code), where/when do you initiate the loggers, and how do you get access to them in your modules, etc.

Answer

The best way I've found so far is to initialize logging setup in settings.py - nowhere else. You can either use a configuration file or do it programmatically step-by-step - it just depends on your requirements. The key thing is that I usually add the handlers I want to the root logger, using levels and sometimes logging.Filters to get the events I want to the appropriate files, console, syslogs etc. You can of course add handlers to any other loggers too, but there isn't commonly a need for this in my experience.

In each module, I define a logger using

logger = logging.getLogger(__name__)

and use that for logging events in the module (and, if I want to differentiate further) use a logger which is a child of the logger created above.

If my app is going to be potentially used in a site which doesn't configure logging in settings.py, I define a NullHandler somewhere as follows:

#someutils.py

class NullHandler(logging.Handler):
    def emit(self, record):
        pass

null_handler = NullHandler()

and ensure that an instance of it is added to all loggers created in the modules in my apps which use logging. (Note: NullHandler is already in the logging package for Python 3.1, and will be in Python 2.7.) So:

logger = logging.getLogger(__name__)
logger.addHandler(someutils.null_handler)

This is done to ensure that your modules play nicely in a site which doesn't configure logging in settings.py, and that you don't get any annoying "No handlers could be found for logger X.Y.Z" messages (which are warnings about potentially misconfigured logging).

Doing it this way meets your stated requirements:

  • You can set up different log handlers for different events, as you currently do.
  • Easy access to loggers in your modules - use getLogger(__name__).
  • Easily applicable to command-line modules - they also import settings.py.

Update: Note that as of version 1.3, Django now incorporates support for logging.

Comments