exebook exebook - 5 months ago 11
Node.js Question

Why does writing to stdout in a hot loop cause an out of memory shutdown?

for (i = 0; i < 3e11; i++) {
process.stdout.write('aaaabbbbccccdddd')
process.stdout.write('\n')
}


Or this code:

for (i = 0; i < 3e11; i++) {
console.log('aaaabbbbccccdddd')
}


Just run in a terminal like this:
node test.js


After about one minute produces this:

<--- JS stacktrace --->
Cannot get stack trace in GC.
FATAL ERROR: MarkCompactCollector: semi-space copy, fallback in old gen Allocation failed - JavaScript heap out of memory
Aborted (core dumped)


Where does it eat memory?

$ node -v
v6.1.0

Answer

It's covered in this bug: https://github.com/nodejs/node/issues/1741

It's considered an open bug in node.js. It seems to have to do with an asynchronous property of stdout and can be fixed with:

process.stdout._handle.setBlocking(true);

I'm honestly confused since I understood console.log was blocking. The issue may be more subtle than "blocking" v. "not blocking," I imagine it's blockingly writing to a buffer that's nonblockingly writing to out, but not sure. (Consider this answer to be incomplete.)