BobIsNotMyName BobIsNotMyName - 1 year ago 71
Python Question

Python logging: Per-file/module logger

I have some Python code I need to add logging to.

I've always preferred nice big C macro looking statements like "DEBUG()", "ERROR()", etc for logging. I feel it makes the code easier to read (no objects) when trace-points are visually differentiable from the actual code.

I also would like to be able to set logging levels at a per-module level.

How could I make a module called "log" that is capable of doing this (while making use of the Python std library logging module)?

E.g.:

File: main.py

# This imports LOG_MODULE_NAME, DEBUG, WARN, etc
from log import *
import my_module

LOG_MODULE_NAME("main")

log.set_level("main", log.LVL_DEBUG)
log.set_level("my_module", log.LVL_WARN)

if __name__ == "__main__":
foo = my_module.myFunc(2)

DEBUG("Exiting main.py")


File: my_module.py

from log import *

LOG_MODULE_NAME("my_module")


def myFunc(x):
DEBUG("Entering function")
if x != 1:
WARN("I thought it would be 1")
DEBUG("Exiting function")
return x+1


I'd expect output to look something like:

[WARN:my_module - my_module.py:9] I thought it would be 1
[DEBUG:main - main.py:11] Exiting main.py

Answer Source

If you want to have the logger's name indicate the module in which it was used, you can use the logger.getLogger([name]) module function, and pass __name__ as its (optional) argument, as explained here.

If you want to use names like DEBUG(), do something like this in each of your files...

LOG_MODULE_NAME = logging.getLogger(__name__)

def DEBUG(msg):
    global LOG_MODULE_NAME
    LOG_MODULE_NAME.debug(msg)

I am not clear on the way that global namespaces actually work in Python... this answer says

each module has its own "global" namespace.

So I guess you will be just fine like that, since LOG_MODULE_NAME won't collide between modules.

I believe that this approach will give you one logfile, where the lines will look like this:

DEBUG:my_module:Entering function
WARN:my_module:I thought it would be 1
DEBUG:my_module:Exiting function
DEBUG:root:Exiting main.py

Not instrospective enough? You want the inpsect module, which will give you a lot of information about the program while it's running. This will, for instance, get you the current line number.

Your note about "setting log levels at a per-module level" makes me think that you want something like getEffectiveLevel(). You could try to smush that in like this:

LOG_MODULE_NAME = logging.getLogger(__name__)
MODULE_LOG_LEVEL = log.LVL_WARN

def DEBUG(msg):
    if MODULE_LOG_LEVEL = log.LVL_DEBUG:
        global LOG_MODULE_NAME
        LOG_MODULE_NAME.debug(msg)

I'm not experienced enough with the logging module to be able to tell you how you might be able to make each module's log change levels dynamically, though.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download