dlphehe dlphehe - 1 month ago 9
Python Question

How to scrap span ids' texts in beautifulsoup in the following html?

<div align="justify" style="text-align: center">
<div>
<table cellspacing="0" rules="all" border="1" id="ContentPlaceHolder1_grd_reminder" style="width:555px;border-collapse:collapse;">
<tr>
<th class="grdheading2" scope="col">Book</th>
<th class="grdheading2" scope="col">Issue Date</th>
<th class="grdheading2" scope="col">Submition Date</th>
</tr>
<tr>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label1_0">Engineering Mechanics</span>
</td>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label2_0">17-Oct-2016</span>
</td>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label3_0">31-Oct-2016</span>
</td>
</tr>
<tr>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label1_1">ATB of Engineering Mathematics</span>
</td>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label2_1">17-Oct-2016</span>
</td>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label3_1">31-Oct-2016</span>
</td>
</tr>
</table>
</div>
</div>


I want to extract the text
Engineering mechanics
and it's corresponding date (text)
31-Oct-2016
and the text
ATB of Engineering Mathematics
and it's corresponding date (text)
31-Oct-2016
. All of these all located in the span ids. How can I extract and print them? I'm new to web scrapping.

Answer

First you can use find_all() to find all tr tags, and using loop you can use find_all() to find all span tags in every tr. This way you can control scraped data

html = '''<div align="justify" style="text-align: center">
                                            <div>
<table cellspacing="0" rules="all" border="1" id="ContentPlaceHolder1_grd_reminder" style="width:555px;border-collapse:collapse;">
<tr>
<th class="grdheading2" scope="col">Book</th><th class="grdheading2" scope="col">Issue Date</th><th class="grdheading2" scope="col">Submition Date</th>
</tr><tr>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label1_0">Engineering Mechanics</span>
</td><td>
<span id="ContentPlaceHolder1_grd_reminder_Label2_0">17-Oct-2016</span>
</td><td>
<span id="ContentPlaceHolder1_grd_reminder_Label3_0">31-Oct-2016</span>
                                                        </td>
</tr><tr>
<td>
<span id="ContentPlaceHolder1_grd_reminder_Label1_1">ATB of Engineering Mathematics</span>
</td><td>
<span id="ContentPlaceHolder1_grd_reminder_Label2_1">17-Oct-2016</span>
</td><td>
<span id="ContentPlaceHolder1_grd_reminder_Label3_1">31-Oct-2016</span>
</td>
</tr>
</table>
</div>
</div>'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'html.parser')

trs = soup.find_all('tr')

for tr in trs:
    spans = tr.find_all('span')
    if spans:
        print 'title:', spans[0].text
        print 'date:',  spans[2].text

Result

title: Engineering Mechanics
date: 31-Oct-2016
title: ATB of Engineering Mathematics
date: 31-Oct-2016
Comments