I'm running a program which allocates 8mb stacks using mmap. While testing to see how many stacks I could allocate (aiming for 100,000), I see virtual memory size rise quickly as expected, and reserved size stay small (less than 1gb). The program then segfaults with
Cannot allocate new fiber stack: Cannot allocate memory (Errno)
prlimit --as=unlimited --rss=unlimited --memlock=unlimited --data=unlimited
Maybe you're hitting the maximum of
/proc/sys/vm/max_map_count. This setting sets a maximum on the number of mmaps your process can have. The default value is 65536. So it's likely not the size of memory you want to malloc, but the number of malloc calls that causes the error
Cannot allocate memory.
You can try to increase the maximum with:
sysctl -w vm.max_map_count=131070
See also NPTL caps maximum threads at 65528?