BobIsNotMyName BobIsNotMyName - 21 days ago 7
Python Question

Python logging: Per-file/module logger

I have some Python code I need to add logging to.

I've always preferred nice big C macro looking statements like "DEBUG()", "ERROR()", etc for logging. I feel it makes the code easier to read (no objects) when trace-points are visually differentiable from the actual code.

I also would like to be able to set logging levels at a per-module level.

How could I make a module called "log" that is capable of doing this (while making use of the Python std library logging module)?

E.g.:

File: main.py

# This imports LOG_MODULE_NAME, DEBUG, WARN, etc
from log import *
import my_module

LOG_MODULE_NAME("main")

log.set_level("main", log.LVL_DEBUG)
log.set_level("my_module", log.LVL_WARN)

if __name__ == "__main__":
foo = my_module.myFunc(2)

DEBUG("Exiting main.py")


File: my_module.py

from log import *

LOG_MODULE_NAME("my_module")


def myFunc(x):
DEBUG("Entering function")
if x != 1:
WARN("I thought it would be 1")
DEBUG("Exiting function")
return x+1


I'd expect output to look something like:

[WARN:my_module - my_module.py:9] I thought it would be 1
[DEBUG:main - main.py:11] Exiting main.py

Answer

If you want to have the logger's name indicate the module in which it was used, you can use the logger.getLogger([name]) module function, and pass __name__ as its (optional) argument, as explained here.

If you want to use names like DEBUG(), do something like this in each of your files...

LOG_MODULE_NAME = logging.getLogger(__name__)

def DEBUG(msg):
    global LOG_MODULE_NAME
    LOG_MODULE_NAME.debug(msg)

I am not clear on the way that global namespaces actually work in Python... this answer says

each module has its own "global" namespace.

So I guess you will be just fine like that, since LOG_MODULE_NAME won't collide between modules.

I believe that this approach will give you one logfile, where the lines will look like this:

DEBUG:my_module:Entering function
WARN:my_module:I thought it would be 1
DEBUG:my_module:Exiting function
DEBUG:root:Exiting main.py

Not instrospective enough? You want the inpsect module, which will give you a lot of information about the program while it's running. This will, for instance, get you the current line number.

Your note about "setting log levels at a per-module level" makes me think that you want something like getEffectiveLevel(). You could try to smush that in like this:

LOG_MODULE_NAME = logging.getLogger(__name__)
MODULE_LOG_LEVEL = log.LVL_WARN

def DEBUG(msg):
    if MODULE_LOG_LEVEL = log.LVL_DEBUG:
        global LOG_MODULE_NAME
        LOG_MODULE_NAME.debug(msg)

I'm not experienced enough with the logging module to be able to tell you how you might be able to make each module's log change levels dynamically, though.

Comments