Kit Kit - 4 months ago 25
YAML Question

Evaluate statements from within Python logging YAML config file

Consider the following snippet of a Python

logging
YAML config file:

version: 1
formatters:
simple:
format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
logfile:
class: logging.handlers.TimedRotatingFileHandler
level: DEBUG
filename: some_fancy_import_name.generate_filename_called_error
backupCount: 5
formatter: simple


I would like to load this YAML config file this way:

with open('logging.yaml', 'r') as fd:
config = yaml.safe_load(fd.read())
logging.config.dictConfig(config)


Take special notice of the
filename
to which the
handler
should write logs. In normal Python code, I would expect
some_fancy_import_name.generate_filename_called_errorlog
to generate the string
'error.log'
. All in all, I would like to say that this logging handler should write to the file
'error.log'
in the current directory.

However, as it turns out, this is not the case. When I look at the current directory, I see a file named
'some_fancy_import_name.generate_filename_called_errorlog'
.

Why go through all this trouble?



I would like
filename
to be programmatically determined. I have successfully tried configuring logging using normal Python scripting this way:

# fancy import name
from os import environ as env

# Programmatically determine filename path
log_location = env.get('OPENSHIFT_LOG_DIR', '.')
log_filename = os.path.join(log_location, 'error')
handler = logging.handlers.TimedRotatingFileHandler(log_filename)


See how the
log_filename
path was inferred from environment variables.

I would like to translate this to a YAML config file. Is it possible?

Perhaps I might need to dig through the
dict
produced by
yaml.safe_load(fd.read())
and do some
eval()
stuff?

Answer

You can add a custom constructor and mark the value with a special tag, so your constructor gets executed when loading it:

import yaml

def eval_constructor(loader, node):
  return eval(loader.construct_scalar(node))

yaml.add_constructor(u'!eval', eval_constructor)

some_value = '123'

config = yaml.load("""
version: 1
formatters:
  simple:
    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
  logfile:
    class: logging.handlers.TimedRotatingFileHandler
    level: DEBUG
    filename: !eval some_value
    backupCount: 5
    formatter: simple
""")

print config['handlers']['logfile']['filename']

This prints 123, since the value some_value has the tag !eval, and therefore is loaded with eval_constructor.

Be aware of the security implications of evaling configuration data. Arbitrary Python code can be executed by writing it into the YAML file!