rocky rocky -4 years ago 35
Python Question

In C python, accessing the bytecode evaluation stack

Given a C Python frame pointer, how do I look at arbitrary evaluation stack entries? (Some specific stack entries can be found via

, I'm talking about other stack entries.)

I asked a broader question like this a while ago:

getting the C python exec argument string or accessing the evaluation stack

but here I want to focus on being able to read CPython stack entries at runtime.

I'll take a solution that works on CPython 2.7 or any Python later than Python 3.3. However if you have things that work outside of that, share that and, if there is no better solution I'll accept that.

I'd prefer not modifying the C Python code. In Ruby, I have in fact done this to get what I want. I can speak from experience that this is probably not the way we want to work. But again, if there's no better solution, I'll take that. (My understanding wrt to SO points is that I lose it in the bounty either way. So I'm happy go see it go to the person who has shown the most good spirit and willingness to look at this, assuming it works.)

update: See the comment by user2357112 tldr; Basically this is hard-to-impossible to do. (Still, if you think you have the gumption to try, by all means do so.)

So instead, let me narrow the scope to this simpler problem which I think is doable:

Given a python stack frame, like
, find the beginning of the evaluation stack. In the C version of the structure, this is
. From that we then need a way in Python to read off the Python values/objects from there.

Answer Source

Here are two potential solution partial solutions, since this problem has no simple obvious answer, short of modifying the CPython interpreter or by instrumenting the bytecode beforehand the bytecode such as with byteplay to save such information.

Thanks to user2357112 for enlightenment on the difficulty of the problem, descriptions of the various Python stacks used at runtime,

  • the non-contiguous evaluation stack,
  • the transiency of the evaluation stack and
  • the stack pointer top which only lives as a C local variable (which at run time, might be or is likely only saved in the value of a register).

The first solution is to write a C extension to access f_valuestack which is the bottom (not top) of a frame. From that you can access values, and that too would have to go in the C extension. The C extension would wrap the PyFrameObject so it can get access to the unexposed field f_valuestack. Although the PyFrameObject can change from Python version to Python version (so the extension might have to check which python version is running), it still is doable.

From that use an Abstract Virtual Machine to figure out which entry position you'd be at.

Something similar for my purposes would for my purposes would be to use a real but alternative VM, like Ned Batchhelder's byterun. It runs a Python bytecode interpreter in Python. The advantage here would be that since this acts as a second VM so stores don't change the running of the current and real CPython VM. However you'd still need to deal with the fact of interacting with external persistent state (e.g. calls across sockets or changes to files). And byterun would need to be extended to cover all opcodes and Python versions that potentially might be needed.

By the way for multi-version access to bytecode in a uniform way (since not only does bytecode change a bit, but also the collections of routines to access it), see xdis.

So although this isn't a general solution it could probably work for the special case of trying to figure out the value of say an EXEC up which appear briefly on the evaluation stack.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download