Oli K Oli K - 2 years ago 88
Python Question

How to add a global symbol that can be bound by Python compiler?

This is my first question so please be nice :) I am rather new to Python but I am very experienced in other programming languages (e.g. C++).

UPDATE 2 - Solution Found

Thanks everyone for helping :) As the solution is "hidden" in comments I will repost it here.

Instead of

file_symbols = {}

the variable local_symbol must initially be added to the file_symbols dictionary:

file_symbols = { "local_symbol" : local_symbol }

For anyone reading this: all variable / class names posted here are not to be understood as actual useful names as these examples are synthetic in nature ;)

Well... now I have to figure out the FULL meaning of:

exec compiled_code in file_symbols

So far I thought it would do nothing more than updating the dictionary
with the symbols found in
But it actually does a bit more as it seems! :)


Ok, my sample project below seems to be too simple to show the actual problem. Anyway, thanks for your already provided support! :)

In fact I want to first compile multiple *.py files which need access to a local symbol (class instance). All symbols coming from these compiled files shall be collected and then be used as an environment for OTHER code objects.

So I really need to do this
(note the following code shows the concept, not actual executable code):

class Functions:

global_symbols = {}
local_symbol = Functions()

# note that the external files need to access local_symbol to call its functions!
for file in external_files:
code = file.load()
compiled_code = compile(code, "<string>", "exec")
file_symbols = {}
exec compiled_code in file_symbols

some_code = another_file.load()
compiled_code = compile(some_code, "<string>", "exec")
exec(compiled_code, global_symbols)

In this example the line

exec compiled_code in file_symbols

produces a NameError() - because there is no way they could access local_symbol as it is not defined anywhere in the external files although it shall be used!

So the question is how to provide access to local_symbol for the external_files so that they can call the instance's functions??

My import hook solution that some of you regard as "hack" was the only working solution so far. I would love to use a simpler one if there is any!

Thanks again :)

My initial question was this:

So here we go. What I intend to do is advanced stuff and I did not find a solution to my problem neither here nor anywhere else.

Assume the following code in Python (2.6.x / 2.7.x):

class Functions:
def __init__(self):
def func_1(...):
def func_2(...):
def func_n(...):
functions = Functions()
code = loadSomeFile(...)
compiled_code = compile(code, "<string>", "exec")
global_env = {}
local_env = {"func":functions}
exec(compiled_code, global_env, local_env)

in the example above is loaded from a file with a content that might look like this:

import something
def aFunction(...):
a = func.func_1(...)
b = func.func_2(...)
return a * b

Please note that
in the code above means that there might be more code that I left out for the sake of simplicity.

The problem I encounter in my example is that the compiler raises an error for this line:

compiled_code = compile(code, "<string>", "exec")

I will get this error:
NameError("global name 'func' is not defined")

This error is totally understandable as the compiler can't bind to any global symbol with the name "func". But I still want to compile the code this way.

So the obvious question is:

How can I define global symbols that can be used by the compiler for
statements so that the compiler will bind any "unknown" symbol to an object of my choice?

In my example I would like to define a global symbol
that is bound to an instance of
class Functions
so that the compiler will find this symbol when compiling code which makes use of
as seen in the example above.

So how can this be accomplished?

Please note that I am aware that directly executing the code using
would fix the compilation problem because the dict
in the example above would provide the symbol that is required for successful execution. HOWEVER I can't do it this way because the code that shall be compiled is not small at all. It might consist of hundreds of code lines and this code is also not executed only once but many many times.
So for performance reasons I really need to compile the code rather than directly executing it.

Thanks for helping me :)

Answer Source

Don't provide separate globals and locals dicts to exec. That causes the executed code to behave as if it's embedded in a class definition. That means that any variable lookups in functions defined in the executed code bypass locals.

>>> exec("""
... def f():
...     print a
... f()""", {}, {"a": 3})
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "<string>", line 4, in <module>
  File "<string>", line 3, in f
NameError: global name 'a' is not defined
>>> exec("""
... def f():
...     print a
... f()""", {"a": 3})

Just pass a globals dict.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download