The following code prints hello world 10 times by using the rsi register as a loop counter.
hello: db 'Hello world!',10
helloLen: equ $-hello
mov rsi, 0 ;<--- use r8 here
inc rsi ;<--- use r8 here
;print hello world
cmp rsi, 10 ;<--- use r8 here
mov eax,1 ; The system call for exit (sys_exit)
mov ebx,0 ; Exit with return code of 0 (no error)
int 0x80 implicitly zeroes out R8, R9, R10, and R11 on 64-bit Linux systems prior to returning to userland code. This behavior occurs on kernels later than 2.6.32-rc1. This is not the case for the preferred 64-bit SYSCALL calling convention.
You are experiencing a peculiarity of the Linux Kernels after version 2.6.32-rc1. For Linux kernel versions <= 2.6.32-rc1 you may get the behaviour you were expecting. Due to an information leakage bug (and exploits) the registers R8, R9, R10, and R11 are now zeroed out when the kernel returns from an
You may believe that these registers shouldn't matter in compatibility mode (32-bit code) since those newer registers are unavailable. This is a false assumption as it's possible for a 32-bit application to switch into 64-bit long mode and access those registers. The Linux Kernel Mailing List post that identified this issue had this to say:
x86: Don't leak 64-bit kernel register values to 32-bit processes
While 32-bit processes can't directly access R8...R15, they can gain access to these registers by temporarily switching themselves into 64-bit mode.
Code that demonstrates register leakage on the earlier kernels was made available by Jon Oberheide. It creates a 32-bit application to be run on an x86-64 system with IA32 compatibility enabled. The program switches to 64-bit long mode and then stores registers R8-R11 into general purpose registers that are available in compatibility mode (32-bit mode). John discusses the specifics in this article. He sums the vulnerability and the kernel fix nicely in this excerpt:
The underlying issue that causes this vulnerability is a lack of zeroing out several of the x86-64 registers upon returning from a syscall. A 32-bit application may be able to switch to 64-bit mode and access the r8, r9, r10, and r11 registers to leak their previous values. This issue was discovered by Jan Beulich and patched on October 1st. The fix is obviously to zero out these registers to avoid leaking any information to userspace.
If you were to step through your code in a debugger like GDB you should discover that R8 is in fact set to zero after
int 0x80. Since it is your loop counter your program ends up in an endless cycle printing