EyalAr EyalAr - 2 months ago 7
Python Question

Including previously logged events in a new logging handler

Please consider the following example:

import logging
#create a logger object:
logger = logging.getLogger("MyLogger")
#define a logging handler for the standard output:
stdoutHandler = logging.StreamHandler(sys.stdout)
logger.addHandler(stdoutHandler)
#...
#initialization code with several logging events (for example, loading a configuration file to a 'conf' object)
#...
logger.info("Log event 1")
#after configuration is loaded, a new logging handler is defined for a log file:
fileHandler = logging.FileHandler(conf.get("main","log_file"),'w')
logger.addHandler(fileHandler)
logger.info("Log event 2")


With this example, "Log event 1" does not appear in the log file (only in stdout).
The log file is inevitably initialized after "Log event 1" (because it's dependent on the configuration).

My question is:

How do I include previously logged events (such as "Log event 1") in a new logging handler (such as the file handler in the example)?

Answer

My solution for the question:

Define a MemoryHandler to handle all the events prior to the definition of the FileHandler.
When the FileHandler is defined, set it as the flush target of the MemoryHandler and flush it.

import logging
import logging.handlers
#create a logger object:
logger = logging.getLogger("MyLogger")
#define a memory handler:
memHandler = logging.handlers.MemoryHandler(capacity = 1024*10)
logger.addHandler(memHandler)
#...
#initialization code with several logging events (for example, loading a configuration file to a 'conf' object)
#everything is logged by the memory handler
#...

#after configuration is loaded, a new logging handler is defined for a log file:
fileHandler = logging.FileHandler(conf.get("main","log_file"),'w')
#flush the memory handler into the new file handler:
memHandler.setTarget(fileHandler)
memHandler.flush()
memHandler.close()
logger.removeHandler(memHandler)
logger.addHandler(fileHandler)

This works for me, so I'm accepting this as the correct answer, until a more elegant answer comes along.

Comments