crash springfield crash springfield - 7 days ago 6
Javascript Question

Unable to get information from <div> Node spider with Cheerio

I'm trying to download the lat/long locations of CCTV locations from the City of Baltimore website (project on the surveillance state) but not getting the console to log anything.

Here's the site:

enter image description here

and my code is:

const request = require('request');
const cheerio = require('cheerio');


let URL = 'https://data.baltimorecity.gov/Public-Safety/CCTV-Locations/hdyb-27ak/data'
let cameras = [];

request(URL, function(err, res, body) {
if(!err && res.statusCode == 200) {
let $ = cheerio.load(body);
$('div.blist-t1-c140113793').each(function() {
let camera = $(this);
let location = camera.text();
console.log(location);
cameras.push(location);
});
console.log(cameras);
}
});


I've tried setting the to
blist-t1-c140113793
and
blist-td blist-t1-c140113793
but neither has worked.

Answer

That's because data for those divs are loaded asynchronously, after the page was rendered. JavaScript is not executed by Cherrio, or any other such library. You'll need either to analyze network traffic and understand which HTTP call loads this data, or use something like Selenium, that actually executes JavaScript inside the browser.

Comments