DenJJ DenJJ - 3 months ago 12
R Question

Scraping with R, only first instance being scraped in the list

I have scraped a few websites and haven't really had a problem with my outputs but I don't understand why on this particular website I am only getting the 1st instance of what I am trying to scrape instead of all 36.

I am trying to collect the

href
links for all players on the page and save them to a list or dataframe.

My current code is:

library(XML)
library(rvest)
match <- "https://www.fotball.no/fotballdata/kamp/?fiksId=6616922"
WSm <- read_html(match)
Players <- WSm %>% html_node(".player-name") %>% html_attr("href") %>% as.character()


This gives me:

[1] "/fotballdata/person/profil/?fiksId=2820074"


Not the full list of all
href
s for all players as I am expecting. Could anyone help?

Answer

Use html_nodes() not html_node()

library(XML)
library(rvest)
match <- "https://www.fotball.no/fotballdata/kamp/?fiksId=6616922"
WSm <- read_html(match)
Players <- WSm %>% html_nodes(".player-name") %>% html_attr("href") %>% as.character()
Players

> library(XML)
> library(rvest)
Loading required package: xml2

Attaching package: ‘rvest’

The following object is masked from ‘package:XML’:

    xml

> match <- "https://www.fotball.no/fotballdata/kamp/?fiksId=6616922"
> WSm <- read_html(match)
> Players <- WSm %>% html_nodes(".player-name") %>% html_attr("href") %>% as.character()
> Players
 [1] "/fotballdata/person/profil/?fiksId=2820074" "/fotballdata/person/profil/?fiksId=2775969"
 [3] "/fotballdata/person/profil/?fiksId=2774767" "/fotballdata/person/profil/?fiksId=2820044"
 [5] "/fotballdata/person/profil/?fiksId=2820232" "/fotballdata/person/profil/?fiksId=2820027"
 [7] "/fotballdata/person/profil/?fiksId=2820034" "/fotballdata/person/profil/?fiksId=2799892"
 [9] "/fotballdata/person/profil/?fiksId=2820237" "/fotballdata/person/profil/?fiksId=2799883"
[11] "/fotballdata/person/profil/?fiksId=2847997" "/fotballdata/person/profil/?fiksId=2830200"
[13] "/fotballdata/person/profil/?fiksId=2820235" "/fotballdata/person/profil/?fiksId=2779315"
[15] "/fotballdata/person/profil/?fiksId=2820241" "/fotballdata/person/profil/?fiksId=2799908"
[17] "/fotballdata/person/profil/?fiksId=2820240" "/fotballdata/person/profil/?fiksId=2860570"
[19] "/fotballdata/person/profil/?fiksId=2795482" "/fotballdata/person/profil/?fiksId=2764974"
[21] "/fotballdata/person/profil/?fiksId=2795481" "/fotballdata/person/profil/?fiksId=2794122"
[23] "/fotballdata/person/profil/?fiksId=2852202" "/fotballdata/person/profil/?fiksId=2863826"
[25] "/fotballdata/person/profil/?fiksId=2811850" "/fotballdata/person/profil/?fiksId=2863824"
[27] "/fotballdata/person/profil/?fiksId=2800983" "/fotballdata/person/profil/?fiksId=2867153"
[29] "/fotballdata/person/profil/?fiksId=2811916" "/fotballdata/person/profil/?fiksId=2780695"
[31] "/fotballdata/person/profil/?fiksId=2811908" "/fotballdata/person/profil/?fiksId=2980432"
[33] "/fotballdata/person/profil/?fiksId=2904191" "/fotballdata/person/profil/?fiksId=2806897"
[35] "/fotballdata/person/profil/?fiksId=2794123"
Comments