BobIsNotMyName BobIsNotMyName - 2 months ago 10
Python Question

Python logging: Per-file/module logger

I have some Python code I need to add logging to.

I've always preferred nice big C macro looking statements like "DEBUG()", "ERROR()", etc for logging. I feel it makes the code easier to read (no objects) when trace-points are visually differentiable from the actual code.

I also would like to be able to set logging levels at a per-module level.

How could I make a module called "log" that is capable of doing this (while making use of the Python std library logging module)?



# This imports LOG_MODULE_NAME, DEBUG, WARN, etc
from log import *
import my_module


log.set_level("main", log.LVL_DEBUG)
log.set_level("my_module", log.LVL_WARN)

if __name__ == "__main__":
foo = my_module.myFunc(2)



from log import *


def myFunc(x):
DEBUG("Entering function")
if x != 1:
WARN("I thought it would be 1")
DEBUG("Exiting function")
return x+1

I'd expect output to look something like:

[WARN:my_module -] I thought it would be 1
[DEBUG:main -] Exiting


If you want to have the logger's name indicate the module in which it was used, you can use the logger.getLogger([name]) module function, and pass __name__ as its (optional) argument, as explained here.

If you want to use names like DEBUG(), do something like this in each of your files...

LOG_MODULE_NAME = logging.getLogger(__name__)

def DEBUG(msg):
    global LOG_MODULE_NAME

I am not clear on the way that global namespaces actually work in Python... this answer says

each module has its own "global" namespace.

So I guess you will be just fine like that, since LOG_MODULE_NAME won't collide between modules.

I believe that this approach will give you one logfile, where the lines will look like this:

DEBUG:my_module:Entering function
WARN:my_module:I thought it would be 1
DEBUG:my_module:Exiting function

Not instrospective enough? You want the inpsect module, which will give you a lot of information about the program while it's running. This will, for instance, get you the current line number.

Your note about "setting log levels at a per-module level" makes me think that you want something like getEffectiveLevel(). You could try to smush that in like this:

LOG_MODULE_NAME = logging.getLogger(__name__)

def DEBUG(msg):
        global LOG_MODULE_NAME

I'm not experienced enough with the logging module to be able to tell you how you might be able to make each module's log change levels dynamically, though.