I am interested in replacing a system call with a custom that I will implement in linux kernel 3.
I read that the sys call table is no longer exposed.
any reference to this http://www.linuxtopia.org/online_books/linux_kernel/linux_kernel_module_programming_2.6/x978.html example but for kernel 3 will be appreciated :)
I would recommend using kprobes for this kind of job, you can easily break on any kernel address (or symbol...) and alter the execution path, all of this at runtime, with a kernel module if you need to :)
Kprobes work by dynamically replacing an instruction (e.g. first instruction of your syscall entry) by a break (e.g. int3 on x86). Inside the do_int3 handler, a notifier notifies kprobes, which in turn passes the execution to your registered function, from which point you can do almost anything.
A very good documentation is given in Documentation/kprobes.txt so as a tiny example in samples/kprobes/kprobes_example.c (in this example they break on do_fork to log each fork on the system). It has a very simple API and is very portable nowdays.
Warning: If you need to alter the execution path, make sure your kprobes are not optimized (i.e. a jmp instruction to your handler replaces the instruction you break onto instead of an int3) otherwize you won't be able to really alter the execution easily (after the ret of your function, the syscall function will still be executed as usual). If you are only interested in tracing, then this is fine and you can safely ignore this issue.