Phenelo Phenelo - 24 days ago 7
Node.js Question

Some progress.Send calls not making it to nodejs land

I've made a Node addon using AsyncProgressWorker thread to handle my socket messages. Here is my code:

class ProgressWorker : public AsyncProgressWorker {
public:
ProgressWorker(
Callback *callback
, Callback *progress)
: AsyncProgressWorker(callback), progress(progress) {}
~ProgressWorker() {}

void Execute (const AsyncProgressWorker::ExecutionProgress& progress) {
char response[4096];
int result;
int connected = 1;
int timeout = 0;
int pending = 0;

while(connected) {
result = sctp_recvmsg(sock, (void *)&response, (size_t)sizeof(response), NULL, 0, 0, 0);
if (result > 0 && result < 4095) {
if (debug) {
printf("Server replied (size %d)\n", result);
}
pending = 0;
progress.Send((const char *)response, size_t(result));
result = 0;
}
else {
// Don't mind my timeout mechanism. :))
if ((result == -1 && errno != EWOULDBLOCK) || pending) {
if (timeout == 0) {
printf("Can't receive from other end. Waiting for 3 seconds. Error code: %d\n", errno);
pending = 1;
}
if (timeout >= 3000) {
connected = 0;
close(sock);
}
else {
timeout += 5;
usleep(5000);
}
}
else {
usleep(5000);
}
}
}

}

void HandleProgressCallback(const char *data, size_t count) {
HandleScope scope;

v8::Local<v8::Value> argv[] = {
CopyBuffer(const_cast<char*>(data), count).ToLocalChecked()
};
progress->Call(1, argv); // This is the callback to nodejs
}

private:
Callback *progress;
};


Now I haven't stress-tested this until tonight then I noticed that some messages won't make it back to node. It will print my "Server replied" debug log but won't log my debug logs I put on the progress callback. Am I missing something here? Thanks in advance.

Answer Source

AsyncProgressWorker is based on a uv_async_t, which allows any thread to wake the main thread. However, as stated in the documentation:

libuv will coalesce calls to uv_async_send(), that is, not every call to it will yield an execution of the callback. For example: if uv_async_send() is called 5 times in a row before the callback is called, the callback will only be called once. If uv_async_send() is called again after the callback was called, it will be called again.

^^ This is the reason that you may sometimes not receive some events while your application is under stress. Above this line is the answer to the question. Below is my "above and beyond" possible solution to deal with your problem:

It so happens that I am working on adding a new alternative to AsyncProgressWorker that promises to deliver every event, just as AsyncProgressWorker does, but using a queue. If you want to test it, try out the AsyncWakeRequestor branch from https://github.com/mkrufky/nan , and then replace your AsyncProgressWorker with AsyncProgressQueueWorker<char> Re-run your tests and all events will be delivered.

There is an open pull request to add this new feature here: https://github.com/nodejs/nan/pull/692.

In the meanwhile, you can use this branch by altering your package.json:

   "dependencies": {
-    "nan": "^2.7.0"
+    "nan": "git://github.com/mkrufky/nan#AsyncWakeRequestor"
   },

** note: I have altered my open pull request to rename the template class from AsyncWakeRequestor to AsyncProgressQueueWorker - be sure to update your code accordingly.