dnsko dnsko - 15 days ago 6
R Question

R Getting JSON data into dataframe

I have this file with JSON formatted data, but need this into a dataframe. Ultimately I would like to plot the geolocations onto a map, but can't seem to get this data into a df first.

json_to_df <- function(file){
file <- lapply(file, function(x) {
x[sapply(x, is.null)] <- NA
unlist(x)
})

df <- do.call("rbind", file)
return(df)
}


But I get only this error:

Error in fromJSON(file) :
STRING_ELT() can only be applied to a 'character vector', not a 'list'


The file structure looks like this (this is only part of the data):

{

"results": [
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Bergen",
"address_1": "16 Notweg",
"name": "FitClub Bergen",
"lon": 4.699218,
"id": 24632049,
"lat": 52.673046,
"repinned": false
},
"headcount": 0,
"distance": 22.46796989440918,
"visibility": "public",
"waitlist_count": 0,
"created": 1467149834000,
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>Start your week off right with a Monday Morning Bootcamp!!! The fresh air and peaceful dunes provide the perfect setting for a total body workout. Whether you are a beginner with brand spankin' new health goals and in need of some direction, or training for a race or competition, we're the trainers for you!!! See you at 8:50 for sign-in!</p>",
"event_url": "https://www.meetup.com/FitClubBergen/events/234936736/",
"yes_rsvp_count": 3,
"duration": 3600000,
"name": "Free Bootcamp in the Bergen Dunes",
"id": "glzqvlyvnbgc",
"time": 1477292400000,
"updated": 1477297999000,
"group": {
"join_mode": "open",
"created": 1441658286000,
"name": "FitClub Bergen Free Bootcamp in the Dunes",
"group_lon": 4.710000038146973,
"id": 18908751,
"urlname": "FitClubBergen",
"group_lat": 52.66999816894531,
"who": "FitClubbers"
},
"status": "past"
},
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Bergen",
"address_1": "16 Notweg",
"name": "FitClub Bergen",
"lon": 4.699218,
"id": 24632049,
"lat": 52.673046,
"repinned": false
},
"headcount": 0,
"distance": 22.46796989440918,
"visibility": "public",
"waitlist_count": 0,
"created": 1467149834000,
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>Start your week off right with a Monday Morning Bootcamp!!! The fresh air and peaceful dunes provide the perfect setting for a total body workout. Whether you are a beginner with brand spankin' new health goals and in need of some direction, or training for a race or competition, we're the trainers for you!!! See you at 8:50 for sign-in!</p> <p>ALWAYS FREE</p> <p>FOR ALL LEVELS OF FITNESS</p> <p>BRING: water bottle and energy</p>",
"event_url": "https://www.meetup.com/FitClubBergen/events/234936737/",
"yes_rsvp_count": 3,
"name": "Monday Morning Bootcamp in the Bergen Dunes",
"id": "flzqvlyvnbgc",
"time": 1477292400000,
"updated": 1477303926000,
"group": {
"join_mode": "open",
"created": 1441658286000,
"name": "FitClub Bergen Free Bootcamp in the Dunes",
"group_lon": 4.710000038146973,
"id": 18908751,
"urlname": "FitClubBergen",
"group_lat": 52.66999816894531,
"who": "FitClubbers"
},
"status": "past"
},
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Amsterdam",
"phone": "020 4275777",
"address_1": "Dijksgracht 2",
"address_2": "1019 BS ",
"name": "Klimmuur Central",
"lon": 4.91284,
"id": 1143381,
"lat": 52.376626,
"repinned": false
},
"headcount": 0,
"distance": 1.0689502954483032,
"visibility": "public",
"waitlist_count": 0,
"created": 1477215767000,
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>Climbing Right After Work: RAW.<br/>Quiet hall, pretty much every rope available; no rope chasing necessary. And.. still some time left to do other things later that evening. Take you gear and an extra sandwich to work and join me afterwards pulling some plastic.<br/>Some notes:<br/>- This events starts @17:00. If you can't make it that early, please comment the time you can.<br/>- Please fill in your belaying skills in your profile. If you've never climbed before or don't have belaying skills: follow an introduction course a the gym first! Safety above all!</p>",
"event_url": "https://www.meetup.com/The-Amsterdam-indoor-rockclimbing/events/235054729/",
"yes_rsvp_count": 3,
"name": "Monday's RAW Climb",
"id": "235054729",
"time": 1477321200000,
"updated": 1477334279000,
"group": {
"join_mode": "approval",
"created": 1358348565000,
"name": "The Amsterdam indoor rockclimbing",
"group_lon": 4.889999866485596,
"id": 6689952,
"urlname": "The-Amsterdam-indoor-rockclimbing",
"group_lat": 52.369998931884766,
"who": "Climbers"
},
"status": "past"
},
{
"utc_offset": 7200000,
"venue": {
"country": "nl",
"localized_country_name": "Netherlands",
"city": "Amstelveen",
"address_1": "Langs de Akker 3",
"name": "Emergohal",
"lon": 4.87967,
"id": 23816542,
"lat": 52.290199,
"repinned": false
},
"rsvp_limit": 12,
"headcount": 0,
"distance": 5.541957378387451,
"visibility": "public",
"waitlist_count": 0,
"created": 1474452073000,
"fee": {
"amount": 5.5,
"accepts": "cash",
"description": "per person",
"currency": "EUR",
"label": "price",
"required": "0"
},
"rating": {
"count": 0,
"average": 0
},
"maybe_rsvp_count": 0,
"description": "<p>We will play the Whole Season indoor soccer on Mondays from 18:00 - 19:00 starting 5 September until May 2017 in the Emergohal Amstelveen.</p> <p>Preferred payment is with Paypal EUR 5.50 (in advance)<br/>If this is not possible you may pay cash but then I will ask EUR 6,-<br/>(Please have the exact cash with you)</p> <p>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</p> <p>A couple of Unisys (ex)colleagues and football lovers are playing every Monday in the Emergohal Amstelveen at 6PM on a reasonable good level. We are looking for a compact group of players who are willing/able to play (almost) every Monday playing 5v5 (or 6v6).<br/>We are playing with the FIFA Futsal rules in mind:<br/><a href="http://www.fifa.com/mm/document/footballdevelopment/refereeing/51/44/50/lawsofthegamefutsal2014_15_eneu_neutral.pdf" class="linkified">http://www.fifa.com/mm/document/footballdevelopment/refereeing/51/44/50/lawsofthegamefutsal2014_15_eneu_neutral.pdf</a></p> <p>The Emergohal has dressing rooms and a nice bar for after the game.</p> <p>Hope to see you on Mondays</p> <p>Cheers Jeroen</p> <p>For questions you may call me on[masked], send a text message (SMS) or leave a message on this meetup group.</p>",
"event_url": "https://www.meetup.com/Futsal_Emergohal_Monday_18-00/events/234290812/",
"yes_rsvp_count": 11,
"duration": 4500000,
"name": "Futsal",
"id": "234290812",
"time": 1477323900000,
"updated": 1477330559000,
"group": {
"join_mode": "approval",
"created": 1474445066000,
"name": "Futsal_Emergohal_Monday_18.00",
"group_lon": 4.860000133514404,
"id": 20450096,
"urlname": "Futsal_Emergohal_Monday_18-00",
"group_lat": 52.31999969482422,
"who": "Players"
},
"status": "past"
}],
"meta": {
"next": "https://api.meetup.com/2/open_events?and_text=False&offset=1&city=Amsterdam&sign=True&format=json&lon=4.88999986649&limited_events=False&photo-host=public&page=20&time=-24m%2C&radius=25.0&lat=52.3699989319&status=past&desc=False",
"method": "OpenEvents",
"total_count": 643,
"link": "https://api.meetup.com/2/open_events",
"count": 20,
"description": "Searches for recent and upcoming public events hosted by Meetup groups. Its search window is the past one month through the next three months, and is subject to change. Open Events is optimized to search for current events by location, category, topic, or text, and only lists Meetups that have **3 or more RSVPs**. The number or results returned with each request is not guaranteed to be the same as the page size due to secondary filtering. If you're looking for a particular event or events within a particular group, use the standard [Events](/meetup_api/docs/2/events/) method.",
"lon": ,
"title": "Meetup Open Events v2",
"url": "",
"signed_url": "{signed_url}",
"id": "",
"updated": 1479988687055,
"lat":
}
}


So I was wondering how I would put this in a dataframe or csv even to be able to extract geolocations later?

Answer

There is no need to write a parser yourself, there are a number of packages that can read JSON formatted data. The one I use, and @hrbrmstr linked, is jsonlite. This package provides a fromJSON function which can parse JSON into a data.frame:

fromJSON('file.json', flatten = TRUE)

note that the flatten argument here ensures the json is flattended into a nice data.frame.