Alex Alex - 4 months ago 9
Linux Question

Why can we allocate 1 PB (10^15) array and get acces to the last element, but can't free it?

As known: http://linux.die.net/man/3/malloc


By default, Linux follows an optimistic memory allocation strategy.
This means that when malloc() returns non-NULL there is no guarantee
that the memory really is available. In case it turns out that the
system is out of memory, one or more processes will be killed by the
OOM killer.


And we can successfully allocate 1 Petabyte of VMA (virtual memory area) by using
malloc(petabyte);
: http://ideone.com/1yskmB

#include <stdio.h>
#include <stdlib.h>

int main(void) {

long long int petabyte = 1024LL * 1024LL * 1024LL * 1024LL * 1024LL; // 10^15
printf("petabyte %lld \n", petabyte);

volatile char *ptr = (volatile char *)malloc(petabyte);
printf("malloc() - success, ptr = %p \n", ptr);

ptr[petabyte - 1LL] = 10;
printf("ptr[petabyte - 1] = 10; - success \n");

printf("ptr[petabyte - 1] = %d \n", (int)(ptr[petabyte - 1LL]));

free((void*)ptr); // why the error is here?
//printf("free() - success \n");

return 0;
}


Result:

Error time: 0 memory: 2292 signal:6
petabyte 1125899906842624
malloc() - success, ptr = 0x823e008
ptr[petabyte - 1] = 10; - success
ptr[petabyte - 1] = 10


And we can successfully get access (store/load) to the last member of petabyte, but why we get an error on
free((void*)ptr);
?

UPDATE:

Note: https://en.wikipedia.org/wiki/Petabyte


  • 1000^5 PB petabyte - I use it

  • 1024^5 PiB pebibyte



This code with
size_t
instead of
long long int
works successfully: http://ideone.com/tz4akQ

#include <stdio.h>
#include <stdlib.h>

int main(void) {

size_t petabyte = 1024LL * 1024LL * 1024LL * 1024LL * 1024LL; // 10^15
printf("petabyte %llu \n", petabyte);

volatile char *ptr = (volatile char *)malloc(petabyte);
printf("malloc() - success, ptr = %p \n", ptr);

ptr[petabyte - 1LL] = 10;
printf("ptr[petabyte - 1] = 10; - success \n");

printf("ptr[petabyte - 1] = %d \n", (int)(ptr[petabyte - 1LL]));

//free((void*)ptr); // why the error is here?
//printf("free() - success \n");

return 0;
}


Result:

petabyte 13815492297405497344
malloc() - success, ptr = 0x8302008
ptr[petabyte - 1] = 10; - success
ptr[petabyte - 1] = 10

Answer

I believe that your problem is that malloc() does not take a long long int as its argument. It takes a size_t.

After changing your code to define petabyte as a size_t your program no longer returns a pointer from malloc. It fails instead.

I think that your array access setting petabyte-1 to 10 is writing far, far outside of the array malloc returned. That's the crash.

Always use the correct data types when calling functions.