faserx faserx - 1 year ago 178
Python Question

http request and regex in Python for HTML parser

When I execute the script, the result is empty. Why? The script connected with a site and parse html tag



import re
import socket
import urllib, urllib.error
import http.client
import sys

conn = http.client.HTTPConnection('www.guardaserie.online');
headers = { "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Content-type": "application/x-www-form-urlencoded; charset=UTF-8" }
params = urllib.parse.urlencode({"s":"hannibal"})
conn.request('GET', '/',params, headers)
response = conn.getresponse();

site = re.search('<a href="(.*)" class="box-link-serie">', str(response.read()), re.M|re.I)

Answer Source

It's likely the pattern you are searching for is non-existent in the read response, or it chokes at some point trying to parse html.

re.search( 'href="(.*)" class="box-link-serie"', str(response.read()), re.M | re.I )

Using something more generic or another parser method will likely lead you to your desired result.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download