Hewbot Hewbot - 5 months ago 23
Javascript Question

Wait in a promise chain for a mutation in a DOM element

I'm making some sort of web scraper in Node.js, that takes a picture of a map that appears on a website with PhantomJS.

However, once the page has been opened, a loading message appears where the map should be. Once the map is ready, the message disappears (

visibility: hidden
) and the map is shown.

Because of this, I can't call
page.render()
until
#loader
is
hidden
(or I would get a picture of the loading message, not cool).

// ... Open the page

.then(function(content) {
return page.evaluate(function() {
// Wait for #loading to get hidden somehow ...
var clipRect = document.getElementById('map').getBoundingClientRect();
return {
top: clipRect.top,
left: clipRect.left,
width: clipRect.width,
height: clipRect.height
};
});
})

// Render and process the picture ...


I considered using a mutation observer, but couldn't find a way to use it, since I'm in a promise chain and an event listener wouldn't work as I want.

I also tried checking for the
visibility
attribute very often until it became hidden, as explained here, but PhantomJS reported through Node's console:

TypeError: null is not an object (evaluating 'child.transform')


Besides, I'd like to avoid that kind of workarounds if possible, because they're very CPU-intensive.

Any ideas on how can I wait for
#loader
to get
hidden
under these circumstances?

Answer

I finally solved this thanks to phantomjs-node's mantainer, amir20, so all credit to him. As he explains in this issue:

waitFor expects to return a value. But evaluate returns a Promise. So that's why it is not working. This is not a problem of the module but rather problem with waitFor. Since everything is executed asynchronously then you have to wait for the value.

The function in question (created by him) is the following:

function waitUntil(asyncTest) {
    return new Promise(function(resolve, reject) {
        function wait() {
            asyncTest().then(function(value) {
                if (value === true) {
                    resolve();
                } else {
                    setTimeout(wait, 100);
                }
            }).catch(function(e) {
                console.log('Error found. Rejecting.', e);
                reject();
            });
        }
        wait();
    });
}

Therefore, applied to my specific example, it should be used like this:

waitUtil(function() {
    return sitepage.evaluate(function() {
        return document.querySelectorAll('#loader').style.visibility == "hidden";
    })
}).then(function(){  // #loading is now hidden
    return page.evaluate(function() {
        var clipRect = document.getElementById('map').getBoundingClientRect();
        return {
            top: clipRect.top,
            left: clipRect.left,
            width: clipRect.width,
            height: clipRect.height
        };
    });
})

// Render and process the picture ...