Jendas Jendas - 7 months ago 74
Python Question

How to timeout function in python, timeout less than a second

Specification of the problem:

I'm searching through really great amount of lines of a log file and I'm distributing those lines to groups in order to regular expressions(RegExses) I have stored using the
function. Unfortunately some of my RegExses are too complicated and Python sometimes gets himself to backtracking hell. Due to this I need to protect it with some kind of timeout.


  • re.match
    , I'm using, is Python's function and as I found out somewhere here on StackOverflow (I'm really sorry, I can not find the link now :-( ). It is very difficult to interrupt thread with running Python's library. For this reason threads are out of the game.

  • Because evaluating of
    function takes relatively short time and I want to analyse with this function great amount of lines, I need some timeout function that wont't take too long to execute (this makes threads even less suitable, it takes really long time to initialise new thread) and can be set to less than one second.

    For those reasons, answers here - Timeout on a Python function call
    and here - Timeout function if it takes too long to finish with decorator (alarm - 1sec and more) are off the table.

I've spent this morning searching for solution to this question but I did not find any satisfactory answer.



In the end, the solution wasn't that complicated, but I thought, that it might be useful for some other hopelessly stacked guys like me to post this simple solution here.

I've just a bit modified script posted here: Timeout function if it takes too long to finish.

And here is the code:

from functools import wraps
import errno
import os
import signal

class TimeoutError(Exception):

def timeout(seconds=10, error_message=os.strerror(errno.ETIME)):
    def decorator(func):
        def _handle_timeout(signum, frame):
            raise TimeoutError(error_message)

        def wrapper(*args, **kwargs):
            signal.signal(signal.SIGALRM, _handle_timeout)
            signal.setitimer(signal.ITIMER_REAL,seconds) #used timer instead of alarm
                result = func(*args, **kwargs)
            return result

        return wraps(func)(wrapper)

    return decorator

And then you can use it like this:

from timeout import timeout 
import re 

def match_re(regex,line):
    return re.match(regex,line)
    print match_re('^tra.+$','tralalalalallalalal')
except TimeoutError, e:
    print 'timeout!'

I hope, this will be helpful for someone :-)