Pichi Wuana Pichi Wuana - 4 months ago 25
Linux Question

What are the return values of system calls in Assembly?

When I try to research about return values of system calls of the kernel, I find tables that describe them and what do I need to put in the different registers to let them work. However, I don't find any documentation where it states what is that return value I get from the system call. I'm just finding in different places that what I receive will be in the EAX register.




TutorialsPoint:


The result is usually returned in the EAX register.


Assembly Language Step-By-Step: Programming with Linux book by Jeff Duntemann states many times in his programs:



  • Look at sys_read's return value in EAX

  • Copy sys_read return value for safe keeping







Any of the websites I have don't explain about this return value. Is there any Internet source? Or can someone explain me about this values?

Answer

C is the language of Unix systems programming, so all the documentation is in terms of C. And then there's documentation for the minor differences between the C interface and the asm on any given platform, usually in the Notes section of man pages.

sys_read is a way to talk about the raw system call (as opposed to the libc wrapper function), because syscall.h defines constants like SYS_read with the actual system call number. (The value you put in EAX before an int 0x80 or syscall instruction). read can also refer to the raw system call, or to the libc wrapper function. In this context, for the rest of this answer, I'm talking about the raw system call.


Linux system call return values (in EAX on x86) are either a positive value for success, or a negative error code. e.g. -EFAULT if you pass an invalid pointer.

This behaviour is documented in the syscalls(2) man page.


To find the actual numeric values of constants for a specific platform, you need to find the C header file where they're #defined. See my answer on a question about that for details.


The meanings of return values for each sys call are documented in the section 2 man pages, like read(2). (sys_read is the raw system call that the glibc read() function is a very thin wrapper for.) Most man pages have a whole section for the return value. e.g.

RETURN VALUE

On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are
actually available right now (maybe because we were close to end-of-
file, or because we are reading from a pipe, or from a terminal), or
because read() was interrupted by a signal. See also NOTES.

On error, -1 is returned, and errno is set appropriately. In this case, it is left unspecified whether the file position (if any)
changes.

Note that the last paragraph describes how the glibc wrapper decodes the value and sets errno to -EAX if the raw system call's return value is negative, so errno=EFAULT and return -1 if the raw system call returned -EFAULT.

And there's a whole section listing all the possible error codes that read() is allowed to return, and what they mean specifically for read(). (POSIX standardizes most of this behaviour.)


I'm not sure exactly how the return value is decoded for a system call like mmap(2), where the return value is not a signed type. (In this case a void*). So it's not obvious how to distinguish an error return from a valid pointer that just happens to have the high bits set. Presumably there's some kind of way for the libc wrapper function to decode the value, and glibc knows the method.

I'm not seeing it in the glibc source tree; presumably it's buried under some layers of macros. e.g. in the x86-64 macro