Tom Tom - 1 month ago 11
C Question

Is it possible to predict a stack overflow in C on Linux?

There are certain conditions that can cause stack overflows on an x86 Linux system:


  • struct my_big_object[HUGE_NUMBER]
    on the stack. Walking through it eventually causes
    SIGSEGV
    .

  • The
    alloca()
    routine (like
    malloc()
    , but uses the stack, automatically frees itself, and also blows up with
    SIGSEGV
    if it's too big). Update: alloca() isn't formally deprecated as I originally stated; it is merely discouraged.



Is there a way to programmatically detect if the local stack is big enough for a given object? I know the stack size is adjustable via
ulimit
, so I have hope there is a way (however non-portable it may be). Ideally, I would like to be able to do something like this:

int min_stack_space_available = /* ??? */;
if (object_size < min_stack_space_available)
{
char *foo = alloca(object_size);
do_stuff(foo);
}
else
{
char *foo = malloc(object_size);
do_stuff(foo);
free(foo);
}

Answer

You can determine the stack space the process has available by finding the size of a process' stack space and then subtracting the amount used.

ulimit -s

shows the stack size on a linux system. For a programmatic approach, check out getrlimit(). Then, to determine the current stack depth, subtract a pointer to the top of the stack from one to the bottom. For example (code untested):

unsigned char *bottom_of_stack_ptr;

void call_function(int argc, char *argv) {
    unsigned char top_of_stack;
    unsigned int depth = (&top_of_stack > bottom_of_stack_ptr) ? 
        &top_of_stack-bottom_of_stack_ptr : 
        bottom_of_stack_ptr-&top_of_stack;

    if( depth+100 < PROGRAMMATICALLY_DETERMINED_STACK_SIZE ) {
        ...
    }
}

int main(int argc, char *argv) {
    unsigned char bottom_of_stack;
    bottom_of_stack_ptr = &bottom_of_stack;
    my_function();
    return 0;
}