ermyuriel ermyuriel - 1 year ago 78
Node.js Question

Best practices for data collection (Events, promises, etc.)

I'm building a simple Node app that queries an API and upon receiving a reply registers it in a database. I do not care so much if not every query to the API or to the DB is successeful since I intend to register lots of data, but I do care that it is continuous.

I am currently doing something like this, which works fine but I'm concerned with scalability:


Should I use events? Promises? What are the best practices for these cases in terms of efficiency and robustness?

Thank you.


I think my question was badly phrased. What I am asking is in which cases this:

SomePromise.then(function() {
return requestResource(data);
}).then(function() {
return useResource(data)

is preferable to this:

ee.on("start", function(data) {
requestResource(data, function() {
ee.emit("use resource", data);
}).on("use resource", function(data) {

when structuring a data collection application.

Example taken from this thread:

Answer Source

Promises and events are generally used for pretty different things.

An event is used to notify an arbitrary number of listeners that some event occurred. The event can be fired any number of times so it can be used for a regular notification. One would usually use an event for something where the listener wants an ongoing notification of all occurrences of an event. You would usually not use an event for something where you just expect one event to occur and then you don't want to know about it again.

A promise is a construct specifically designed to help you manage asynchronous operations, results and errors. Promises are one-shot devices so once their state changes, the promise will never change state again.

There is some overlap between event emitters and promises, but there is certainly distinct territory for each:

Some notes about events and event listeners:

  1. A client wants to register for the occurrence of some event that may occur more than once and the client wants to get see all occurrences of such events. For example, suppose you have a temperature monitor and you want clients to be able to monitor every time the temperature changes by at least a degree. You could have some code that monitors the raw temperature and every time it changes by more than a degree, it would fire off the temperatureChange event. People interested in watching for temperature changes would register a listener for the temperatureChange event. Because this event can occur more than once, this is not really appropriate for promises.

  2. You have lots of different things to potentially notify clients of and you want one central way for them to listen for these things. A single event listener with lots of different event named defined creates a very simple and streamlined API for registering for lots of different events.

  3. You can use events for either synchronous or asynchronous operations.

Some notes about promises:

  1. Promises really only makes sense if the operation is at least sometimes is asynchronous (it finishes at some indeterminate time in the future).

  2. Promises only notify the caller one time for a given promise so they only make sense for one-shot operations. I'm starting this specific operation and I want to know when this specific operation is done.

  3. Promises are incredibly useful for coordinating multiple asynchronous operations (start B after A finishes or start C after A and B finish). Event emitters don't provide any features to help you coordinate multiple events.

  4. Promises are incredibly useful for propagating and properly handling errors in asynchronous operations.

I think of promises more like procedural, sequential code (that works for async operations) and it's very self documenting that way.


Event handlers are just much more loosely coupled than this and don't really lend themselves to sequencing asynchronous operations like promises do.

In the example, you show it appears that the operation is a one-time operation and you want to know both when it finishes and what the final value is. This is perfect for a promise:

SomePromise.then(function() {
    return requestResource(data);
}).then(function(resource) {
    return useResource(resource)
}).then(function(result) {
    // all operations done successfully here
}, function(err) {
    // error here in any of the three async operations

To do something equivalent using your event listeners, you have to figure out how to:

  1. Remove all listeners after you are done using them in both regular and error conditions.
  2. Handle errors appropriately at all levels.
  3. Communicate back results.

If you try to write the same scenario covered by the promises using the event listeners, it gets quite complicated. This is particular because there's really no direct sequencing to the code with event listeners. You attach all the listeners, then fire off the first part of the chain, then anywhere things go wrong, you have to clean everything up and figure out how to communicate back a result. If every goes to completion, then you just process the final result and then clean up all the listeners.

I actually tried to write something that has all the functionality of the promise implementation using events and it got so hopelessly complicated that I gave up. This is because in error conditions, you need to remove event listeners that haven't yet fired so you don't end up with duplicates going forward and to do so you need named functions and further communicating back errors gets difficult and all this just makes the code quite complicated. I gave up.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download