SlartyBartfast SlartyBartfast - 2 months ago 18
Node.js Question

API aggregation in node.js

I am trying to aggregate several location API systems (Google, Foursquare, etc...) to provide diverse location data to users. Delivery is based on user location resulting in a lat, long and radius search.

I have scoured the web for specific API aggregation ideas, with not much luck, I have only found many clever libraries for server side HTTP requests. What I am looking for specifically is opinions on best implimentation of geoJson aggregation.

My options (as I see them) are:




1. Expose multiple location based APIs (Google, Foursquare, etc..) to the user on the front end, i.e. query the APIs each time the user moves there position.

Advantages




  • No indirect storage of geoJson.

  • No server overhead.

  • No potential usage terms breach (Im using Google Maps to allow Location API).



Disadvantages




  • Increased API calls buy each user (inefficient and potentially costly)

  • Lacks the ability to collate results and format data delivery.






2. Somehow aggregate API calls on the server based on user location (mostly static), update aggregation with 'cron job' as required. Therefore data delivery is effectivley pre-cached and can be paginated more easily depending on user preferences.

Advantages




  • Faster (location data is effectively cached).

  • More control of data.

  • Less API calls.



Disadvantages




  • Potential breach of terms of use (Google allows for cacheing purposes only)

  • Server load.






I realize I can achieve this with a
http.get
calls populating a database (lets say mongo due to geospatial queries) on the backend. It just seems like a common use case and I cannot find any reference to it.

Does anyone have any experience with this in Node? I feel like im missing something. Thanks

Answer

You don't see much about this, because it's such a common task and you are asking the internet the wrong questions.

You are describing aggregating multiple http request results, something which a google search would turn up for you

ES6 promises make this easy and elegant:

Promise.all([
    fetch("api.com/things"),
    fetch("api2.com/otherthings"),
    fetch("api3.com/otherthings")
    ])
    .then(res=> {
      //do something with res[0], res[1], res[2]
    })

Theoretically, you could do it all on the client side just as well, using the same exact pattern. However, more than likely some of the apis you want to call won't allow themselves to be called from the client (CORS violations). You'll most likely have to stand up a server to act as a proxy that can pass calls onto the apis that don't support browser requests.

Of course there's more to do. Promise.all will fail if any individual promise fails. So you'll have to handle error possibilities in each call. Also, your api calls may be significantly more complex to construct than the simple gets above. You mentioned caching results, etc...

But the above is the basic idea.

Comments