MikeTheTall MikeTheTall - 1 month ago 16
Javascript Question

Facebook Graph API (v2.8): Pagination in JavaScript?

I'm trying to figure out how to scrape/access all the comments in a very, very long thread (about 1,035 comments). Clearly the best way to go is to make a FB app and then use the Graph API (the raw HTML is opaque! :) )

My problem is that I can control the fields I receive on the first page of results, but after that I'm missing almost all the fields.

Is there a way to control which fields I see on subsequent pages, using JavaScript?

After logging in, etc, I ask for the first page of comments using this code:

FB.api(
'/168...etc', // hard coding is bad, I know,
// but right now I'm just figuring this out
'GET', {
"fields": "message,type,comments.limit(3){message,like_count,from}",
},
processNominationThread
);


This works fine - I get back a response object that the
processNominationThread
function handles well. Specifically, each comment object has the fields I'm interested in. The response object contains fields like this:

response: Object
comments: Object
data: Array[3]
paging: Object


A typical comment object looks like this:

from: Object
id:"1111111...etc"
name:"Bob Smith"
__proto__:Object
id:"1680...etc"
like_count:2
message:"Harry Potter and the Kidney Stone With Teeth"


This is great because I'm interested in the message from each comment, how many likes it had, and (if possible) who posted it.

The problem is when I go to the next page of results. When I use the following JavaScript code:

if (response.comments.paging.next) { // if there's more than 1 page
FB.api(
response.comments.paging.next, // get the next page
'GET', {
"fields": "message,type,comments{message,like_count,from}"
},
continueProcessNominationThread
)
}


Then I get back slightly different results. Essentially, the first time I ask for the comments I get back a full response, and each time I ask for the 'next' page I only get back the 'comments' object from the overall response. (This part is fine - we don't need the response info for subsequent pages).

Here's the problem: The individual comment objects for the subsequent (2nd and onward) pages are missing every field except for 'message' and 'id'

In other words, I'm getting back comment objects that look like:

data[1]: Object
id:"168etc"
message:"Harry Potter And The Thing I Found Behind The Safeway Dumpsters"
__proto__:Object {etc}


Is it possible to get all the fields on the second (and onward) page of pagination?

A couple of thoughts:

On the one hand I could write more code to individually ask each comment for the missing fields but this seems super-inefficient (both for me and for Facebook - do they really want me to query them hundreds of times for info that they'll happily collect for me on the first page of results?).

I did try asking for 10,000 results and FB gives me an error message. It looks like it'll give me 950 results or so, but FB starts returning an error message around 1,000. So asking for "all the data" in an effort to side-step pagination looks a bit fragile (but probably ok for most cases).

There are a couple of other questions like this on SO, but they're either on a slightly different topic or else old & outdated (like Facebook Graph API - Paging, which recommends dynamically adding
<script>
elements instead of using the current JS SDK).

Any help, advice, or pointers to deeper documentation about this would be great (there's basically no docs (that I could find) about pagination at all).

Answer

Just in case anyone is curious - it turns out that this was my fault, after all.

When asking for the next page of results I made the following call:

        FB.api(
            response.comments.paging.next, // get the next page
            'GET', 
            {
                "fields": "message,type,comments{message,like_count,from}"
                // ABOVE LINE IS WRONG!!!!
            },
            second_page_of_comments_BUG_HERE
        )

The correct way to do this is to NOT list the fields you want for the second+ pages, like so:

        FB.api(
            response.comments.paging.next, // get the next page
            'GET', 
            { }, // LEAVE THIS EMPTY
            second_page_of_comments_BUG_HERE
        )