Eka Eka - 2 months ago 9
R Question

How to fetch headlines from google news using rvest R?

I want to fetch headlines from google news using rvest in R. I have done this so far

library(rvest)
url=read_html("https://www.google.com/search?hl=en&tbm=nws&authuser=0&q=american+president")
selector_name<-"r"
fnames<-html_nodes(x = url, css = selector_name) %>%
html_text()


but the result is

> fnames
character(0)


This is the inspect element of a headline?

<h3 class="r"><a href="/browse.php/PbtvpluS/QDvUJpC7/KoWCA9QE/VTTOFmVJ/bIp8sMa8/qKjgkcAu/Hgcr9lyg/4bibGCOO/nZ82ojLo/_2B602Vo/0sOSEbba/SaiySebj/AqD60GRO/skpNXIW9/fA8EbzOq/z6XjMXo2/9iDad2zD/qREp_2Fp/hoHl64rG/9wfBHOPB/a0nLFrAz/OsCmtfKV/cQoDAFWY/cRXfd5FX/5OAJF8UR/9gUdG_2F/_2F4hOLN/xOfe6_2F/shH2n9O7/hCZGQosp/eeAh6wAC/JhCOgG0i/sLkpRGRN/PH_2B61L/njabdbV1/vpS4wcbX/NKpO_2Bq/jpun2LeG/TQecIESv/vxFbk19Q/_3D_3D/b29/">Obama Addresses Racial Tensions at Celebration of African ...</a></h3>


How can I fetch the headlines from google news?

Answer

I think you are just missing a dot for the class name:

> headlines = read_html("https://www.google.com/search?hl=en&tbm=nws&authuser=0&q=american+president") %>%
  html_nodes(".r") %>% 
  html_text()
> headlines
 [1] "Iranian President: No American President Can Renegotiate the Now ..."
 [2] "US: President Barack Obama vetoes 9/11 bill"                         
 [3] "President Obama Wants Donald Trump to Visit New African ..."         
 [4] "President Obama: Discrimination Should Concern 'All Americans ..."   
 [5] "Conrad Black: The Middle East watches, and waits, for the next ..."  
 [6] "Putin's close friend: Donald Trump will be next US president"        
 [7] "US election 2016 polls and odds: Latest Donald Trump and Hillary ..."
 [8] "US election: Ted Cruz endorses Donald Trump for president"           
 [9] "Obama – I'm proud of my 'African record' as US president"            
[10] "Almost 6000 Americans Have Already Voted for President"   
Comments