husin alhaj ahmade husin alhaj ahmade - 5 months ago 17
Linux Question

Can we read and fault-inject another thread's program counter?

Assume that we have a single thread program and we hope to capture the value of program counter (PC) when a predefined interrupt occurs (like a timer interrupt).

It seems easy as you know we just write a specific assembly code using a special keyword

__asm__
and pop the value on the top of the stack after making a shift 4 byte.

What about Multithreaded programs ?

How can we get values of all threads from another thread which run in the same process? (It seems extremely incredible to get values from thread which run on a separate core in multi-core processors).
(in multithreaded programs, every thread has its stack and registers too).




I want to implement a saboteur thread.

in order to perform fault injection in the target multi-threaded program, the model of fault is SEU (single error upset) which means that an arbitrary bit in the program counter register modified randomly (bit-flip) causing to violate the right program sequence. therefore, control flow error (CFE) occurs.

Since our target program is a multi-threaded program, we have to perform fault injection on all threads' PC. This is the task of saboteur tread. It should be able to obtain threads' PC to perform fault injection.
assume we have this code,

main ()
{
foo
}

void foo()
{
__asm__{
pop "%eax"
pop "%ebx" // now ebx holds porgram counter value (for main thread)
// her code injection like 00000111 XOR ebx for example
push ...
push ...
};
}


If our program was a multithreaded program.
is it means that we have more than one stack?

when OS perform context switching, it means that the stack and registers of the thread that was running moved to some place in the memory. Does this mean that if we want to get the values of the program counter for those threads, we find them in memory? where? and is it possible during run-time?

Answer

When you install a signal handler using sigaction() with SA_SIGINFO in the flags, the second parameter the signal handler gets is a pointer to siginfo_t, and the third parameter is a pointer to an ucontext_t. In Linux, this structure contains, among other things, the set of register values when the kernel interrupted the thread, including program counter.

#define _POSIX_C_SOURCE 200809L
#define _GNU_SOURCE
#include <signal.h>
#include <ucontext.h>

#if defined(__x86_64__)
#define  PROGCOUNTER(ctx) (((ucontext *)ctx)->uc_mcontext.greg[REG_RIP])
#elif defined(__i386__)
#define  PROGCOUNTER(ctx) (((ucontext *)ctx)->uc_mcontext.greg[REG_EIP])
#else
#error Unsupported architecture.
#endif

void signal_handler(int signum, siginfo_t *info, void *context)
{
    const size_t program_counter = PROGCOUNTER(context);

    /* Do something ... */

}

As usual, printf() et al. are not async-signal safe, which means it is not safe to use them in a signal handler. If you wish to output the program counter to e.g. standard error, you should not use any of the standard I/O to print to stderr, and instead construct the string to be printed by hand, and use a loop to write() the contents of the string; for example,

#include <stdlib.h>
#include <unistd.h>
#include <errno.h>

static void wrerr(const char *p)
{
    const int   saved_errno = errno;
    const char *q = p;
    ssize_t     n;

    /* Nothing to print? */
    if (!p || !*p)
        return;

    /* Find end of q. strlen() is not async-signal safe. */
    while (*q) q++;

    /* Write data from p to q. */
    while (p < q) {
        n = write(STDERR_FILENO, p, (size_t)(q - p));
        if (n > 0)
            p += n;
        else
        if (n != -1 || errno != EINTR)
            break;
    }

    errno = saved_errno;
}

Note that you'll want to keep the value of errno unchanged in the signal handler, so that if interrupted after a failed library function, the interrupted thread still sees the correct errno value. (It's mostly a debugging issue, and "good form"; some idiots pooh-pooh this as "it does not happen often enough for me to worry about".)

Your program can examine the /proc/self/maps pseudofile (it is not a real file, but something that the kernel generates on the fly when the file is read) to see the memory regions used by the program, to determine whether the program was running a C library function (very common) or something else when the interrupt was delivered.

If you wish to interrupt a specific thread in a multi-threaded program, just use pthread_kill(). Otherwise the signal is delivered to one of the threads that has not blocked the signal, more or less at random.


Here is an example program, that is tested to in x86-64 (AMD64) and x86, when compiled with GCC-4.8.4 using -Wall -O2:

#define  _POSIX_C_SOURCE 200809L
#define  _GNU_SOURCE
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <signal.h>
#include <ucontext.h>
#include <time.h>
#include <stdio.h>

#if defined(__x86_64__)
#define PROGRAM_COUNTER(mctx)   ((mctx).gregs[REG_RIP])
#define STACK_POINTER(mctx)     ((mctx).gregs[REG_RSP])
#elif defined(__i386__)
#define PROGRAM_COUNTER(mctx)   ((mctx).gregs[REG_EIP])
#define STACK_POINTER(mctx)     ((mctx).gregs[REG_ESP])
#else
#error Unsupported hardware architecture.
#endif

#define MAX_SIGNALS  64
#define MCTX(ctx)    (((ucontext_t *)ctx)->uc_mcontext)

static void wrerr(const char *p, const char *q)
{
    while (p < q) {
        ssize_t n = write(STDERR_FILENO, p, (size_t)(q - p));
        if (n > 0)
            p += n;
        else
        if (n != -1 || errno != EINTR)
            break;
    }
}

static const char hexc[16] = "0123456789abcdef";

static inline char *prehex(char *before, size_t value)
{
    do {
        *(--before) = hexc[value & 15];
        value /= (size_t)16;
    } while (value);
    *(--before) = 'x';
    *(--before) = '0';
    return before;
}

static volatile sig_atomic_t done = 0;

static void handle_done(int signum)
{
    done = signum;
}

static int install_done(const int signum)
{
    struct sigaction act;

    memset(&act, 0, sizeof act);
    sigemptyset(&act.sa_mask);
    act.sa_handler = handle_done;
    act.sa_flags = 0;
    if (sigaction(signum, &act, NULL) == -1)
        return errno;

    return 0;
}

static size_t jump_target[MAX_SIGNALS] = { 0 };
static size_t jump_stack[MAX_SIGNALS] = { 0 };

static void handle_jump(int signum, siginfo_t *info, void *context)
{
    const int   saved_errno = errno;
    char        buffer[128];
    char       *p = buffer + sizeof buffer;

    *(--p) = '\n';
    p = prehex(p, STACK_POINTER(MCTX(context)));
    *(--p) = ' ';
    *(--p) = 'k';
    *(--p) = 'c';
    *(--p) = 'a';
    *(--p) = 't';
    *(--p) = 's';
    *(--p) = ' ';
    *(--p) = ',';
    p = prehex(p, PROGRAM_COUNTER(MCTX(context)));
    *(--p) = ' ';
    *(--p) = '@';
    wrerr(p, buffer + sizeof buffer);

    if (signum >= 0 && signum < MAX_SIGNALS) {
        if (jump_target[signum])
            PROGRAM_COUNTER(MCTX(context)) = jump_target[signum];
        if (jump_stack[signum])
            STACK_POINTER(MCTX(context)) = jump_stack[signum];
    }

    errno = saved_errno;
}

static int install_jump(const int signum, void *target, size_t stack)
{
    struct sigaction act;

    if (signum < 0 || signum >= MAX_SIGNALS)
        return errno = EINVAL;

    jump_target[signum] = (size_t)target;
    jump_stack[signum] = (size_t)stack;

    memset(&act, 0, sizeof act);
    sigemptyset(&act.sa_mask);
    act.sa_sigaction = handle_jump;
    act.sa_flags = SA_SIGINFO;
    if (sigaction(signum, &act, NULL) == -1)
        return errno;

    return 0;
}

int main(int argc, char *argv[])
{
    const struct timespec sec = { .tv_sec = 1, .tv_nsec = 0L };
    const int pid = (int)getpid();
    ucontext_t ctx;

    printf("Run\n");
    printf("\tkill -KILL %d\n", pid);
    printf("\tkill -TERM %d\n", pid);
    printf("\tkill -HUP  %d\n", pid);
    printf("\tkill -INT  %d\n", pid);
    printf("or press Ctrl+C to stop this process, or\n");
    printf("\tkill -USR1 %d\n", pid);
    printf("\tkill -USR2 %d\n", pid);
    printf("to send the respective signal to this process.\n");
    fflush(stdout);

    if (install_done(SIGTERM) ||
        install_done(SIGHUP)  ||
        install_done(SIGINT) ) {
        printf("Cannot install signal handlers: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    getcontext(&ctx);

    if (install_jump(SIGUSR1, &&usr1_target, STACK_POINTER(MCTX(&ctx))) ||
        install_jump(SIGUSR2, &&usr2_target, STACK_POINTER(MCTX(&ctx))) ) {
        printf("Cannot install signal handlers: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    /* These are expressions that should evaluate to false, but the compiler
     * should not be able to optimize them away. */
    if (argv[0][1] == 'A') {
usr1_target:
        fputs("USR1\n", stdout);
        fflush(stdout);
    }

    if (argv[0][1] == 'B') {
usr2_target:
        fputs("USR2\n", stdout);
        fflush(stdout);
    }

    while (!done) {
        putchar('.');
        fflush(stdout);
        nanosleep(&sec, NULL);
    }

    fputs("\nAll done.\n", stdout);
    fflush(stdout);

    return EXIT_SUCCESS;
}

If you save the above as example.c, you can compile it using

gcc -Wall -O2 example.c -o example

and run it

./example

Press Ctrl+C to exit the program. Copy the commands (for sending SIGUSR1 and SIGUSR2 signals), and run them from another window, and you'll see they modify the position for current execution. (The signals cause the program counter/instruction pointer to jump back, into an if clause that should never be executed otherwise.)

There are two sets of signal handlers. handle_done() just sets the done flag. handle_jump() outputs a message to standard error (using low-level I/O), and if specified, updates the program counter (instruction pointer) and stack pointer.

The stack pointer is the tricky part when creating an example program like this. It would be easy if we were satisfied with just crashing the program. However, an example is only useful if it works.

When we arbitrarily change the program counter/instruction pointer, and the interrupt was delivered when in a function call (most C library functions...), the return address is left on the stack. The kernel can deliver the interrupt at any point, so we cannot even assume that the interrupt was delivered when in a function call, either! So, to make sure the test program does not crash, I had to update the program counter/instruction pointer and stack pointer as a pair.

When a jump signal is received, the stack pointer is reset to a value I obtained using getcontext(). This is not guaranteed to be suitable for any jump location; it's just the best I could do for a minimal example. I definitely assume the jump labels are nearby, and not in subscopes where the compiler is likely to mess with the stack, mind you.

It is also important to keep in mind that because we are dealing with details left to the C compiler, we must conform to whatever binary code the compiler produces, not the other way around. For reliable manipulation of a process and its threads, ptrace() is a much better (and honestly, easier) interface. You just set up a parent process, and in the target traced child process, explicitly allow the tracing. I've shown examples here and here (both answers to the same question) on how to start, stop, and single-step individual threads in a target process. The hardest part is understanding the overall scheme, the concepts; the code itself is easier -- and much, much more robust than this signal-handler-context-manipulation way.

For self-introducing register errors (either to program counter/instruction pointer, or to any other register), with the assumption that most of the time that leads to the process crashing, this signal handler context manipulation should be sufficient.

Comments