Sandeep Kumar Sandeep Kumar - 1 year ago 52
Python Question

Fetch a nested element from HTML using python scrapy

<div class="product ">
<div class="information">
<h4 class="name ">Biryani</h4>
<p class="description ">mutton mix biryani</p>
</div>

<div class="details">
<div class="orderDetail">
<p class="price ">&#163;12.95</p>

</div>
</div>




For each class product I want to fetch the text of class name and price.

The expected output is :

name : Biryani,
price : 12.95

Answer Source
# -*- coding: utf-8 -*-
from scrapy.selector import Selector

doc = """
<div class="product ">
  <div class="information">
    <h4 class="name ">Biryani</h4>
    <p class="description ">mutton mix biryani</p>
  </div>

  <div class="details">
    <div class="orderDetail">
      <p class="price ">&#163;12.95</p>
    </div>
  </div>
</div>
"""

sel = Selector(text=doc, type="html")

sel_products = sel.xpath('//div[@class="product "]')

for product in sel_products:
    name = product.xpath('.//h4[@class="name "]/text()').extract_first()
    price = product.xpath('.//p[@class="price "]/text()').extract_first()
    print "name: " + name + ", price: " + price  # name: Biryani, price: £12.95
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download