HauLuk HauLuk -4 years ago 267
Python Question

Parse public facebook posts with beautifulsoup / python

I try to parse facebook posts which are made to a specific topic (like a company or a product). As an example posts from here https://www.facebook.com/search/latest/?q=facebook

I can login to facebook (with python) correctly and I am also able to get the source code of the page which contains the posts I am looking for. After some manual code review I found out that I wanted to get following:

<div class="_5pbx userContent" data-ft="&#123;&quot;tn&quot;:&quot;K&quot;&#125;">
<p>Here is the text of the post I need

So I started with beautifulsoup and following code:

soup = BeautifulSoup(pageSourceCode.content, 'html.parser')

for msg in soup.find_all('div'):
print (msg.get('class')

As result I get only this ...


Does someone have experience in scraping facebook posts? I only need this for myself and education purposes

Answer Source

Following code should work

soup = BeautifulSoup(pageSourceCode.content, 'html.parser')

divs = soup.find_all('div', class_="_5pbx userContent")
for div in divs:
    p = div.find('p')
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download