Richard Rast Richard Rast - 1 year ago 55
Python Question

How to keep track of execution time?

The Setup

I'm working on training some neural networks. These have lots of hyperparameters, and typically you see how each set of hyperparameters performs, then pick your favorite. This is often done by (say) training a network with the given parameters for
epochs, then evaluate its performance, yielding a numerical score of each set of parameters and allowing you to pick the best.

There's a problem with this, though. Some sets of parameters let you go through more epochs more quickly, but benefit less from each epoch. Additionally, pretty much any set of parameters will always do better, given more epochs, so given infinite time, they would all do really well (to a point, but that's not the point right now).

The Problem

What I would prefer to do is to let each process figure out how long it's been running, and cut itself off (gracefully) after a specified number of seconds. The problem is, I would like to multithread this, so just because the program has been running for 60 seconds doesn't mean the process has had 60 seconds of fair CPU time.

So how can I measure how much time the process has actually had available to it, within the process itself?

method gives system time, which is problematic (as above).

module seems a bit better, but it's external to the script, so the process wouldn't know when to stop.

Is there a better way? Am I wrong about one of the above ways?

Specific Question

How can a python process see how many seconds it has been allocated so far? Not the amount of time that has passed, but how many seconds it itself has been allowed to execute for?

Answer Source

Use os.times(). This gives you the user and system times for the current process. Below is an example limiting the amount of user time.

start = os.times()
limit = 5  # seconds of user time
while True:
    # your code here
    check = os.times()
    if check.user - start.user > limit: