user929304 user929304 - 3 months ago 10
Python Question

Reading python variables during a running job

I have launched a python script that takes a long time to finish, and silly me I forgot to print out the values of important variables every now and then in my script, to for example estimate the progress of the computation I'm doing.
So now, I was wondering if there's a way to access the current values of certain set of variables in my code (e.g. a list), as the script is running? (I could of course just stop it and add the changes/prints to the code then relaunch, but since it has been running for a day now, it is a pity to lose the computed values so far)

Alternatively, can I crash it in a certain way (other than usual Ctrl-c keyboard interrupt) such that the variable values at the moment of crash are pasted somewhere given that I didn't plan for this in my script? (I am running Ubuntu, python 2.7 and the script is simply run from a terminal by 'python test.py')

Answer

Without editing your program, you're going to have a bad time. What you are looking for is some form of remote debugger, but anything that gives you python specific things will probably have to be at least somehow given a hook into your program. That being said, if you feel like fiddling around in a stack, you can attach gdb to your program (gdb -p <PID>) and see what you can find.

Edit: Well. This might actually be possible.

Following here, with the python extentions for GDB installed, if you pop open a gdb shell with gdb python <PID>, you should be able to run py-print <name of the variable> to get its value, assuming it's in the scope of the program at that point. Attempting to do this myself, with the trivial program

import time
a = 10
time.sleep(1000)

I was able to open a GDB shell by finding the PID of the program (ps aux | grep python), running sudo gdb python <PID> and then run py-print a, which produced "global 'a' = 10". Of course this assumes you are running in a *nix environment.