view raw
fushar fushar - 11 months ago 54
Linux Question

UNIX ptrace() block child's system calls

I am developing a grader for programming contests. Basically, the grader must run the solution program in an 'isolated' process. So, I would like the solution not to call any harming system calls (such as system(), fork(), etc.). Can I use ptrace() to achieve that?


I think there are 2 possible solutions:

  1. Using the LD_PRELOAD mechanism to create 'shim' to replace the system calls you want to stop.
  2. Use setrlimit() to limit what the calling process can do. Unfortunately these limits seem to be a per-user, not per-process, basis, which makes calculating the correct value to set very difficult.

EDIT: I have the first option working, and have included the necessary code below. Build binaries using make all and then test with make runtests:

$ make all
gcc -fPIC -shared -Wl,-soname, -o lib.c
ln -sf
ln -sf
gcc -o test test.c

$ make runtests
in child: retval=9273
in parent: retval=0
LD_PRELOAD=./ ./test fork!
fork error: error=Operation not permitted (1)


all: libs test

    @echo Without LD_PRELOAD:
    @echo With LD_PRELOAD:
    LD_PRELOAD=./ ./test

libs: lib.c
    gcc -fPIC -shared -Wl,-soname, -o lib.c
    ln -sf
    ln -sf

test: test.c
    gcc -o test test.c

    rm -f test lib.o


#include <stdio.h>
#include <unistd.h>
#include <errno.h>

pid_t fork()
    printf(" fork!\n");
    errno = EPERM;
    return (pid_t)-1;


#include <stdio.h>
#include <string.h>
#include <errno.h>

int main(int argc, char **argv)
    int retval = fork();
    if (retval == 0)
        printf("in parent: retval=%d\n", retval);
    else if (retval > 0)
        printf("in child: retval=%d\n", retval);
        printf("fork error: error=%s (%d)\n", strerror(errno), errno);
    return 0;