chris chris - 4 months ago 159
Python Question

Using a proxy with phantomjs (selenium webdriver)

I'm using phantomJS as a driver for selenium. My code is written in python. I followed the advice from similar questions, and am using the following:

service_args = [
'--proxy=78.23.244.145:80',
'--proxy-type=http',
]
driver = webdriver.PhantomJS(service_args=service_args)
driver.get('http://www.whatismyip.com/')


However, when I print the html, barely anything shows up:

print driver.page_source


OUTPUT:

<html><head></head><body></body></html>


If I do this with just the usual call to phantomJS, the website shows up as usual:

driver = webdriver.PhantomJS()


For reference, I've tried this with a bunch of proxies from this list:

http://proxylist.hidemyass.com/search-1291972#listable

I'm wondering how to get the page to properly display when using a proxy. Any help would be appreciated!

Answer

I suspect that the proxy you are using is incorrect. I tried the following where used proxy behave sanely in windows 8.

from selenium.webdriver.common.proxy import *
from selenium import webdriver
from selenium.webdriver.common.by import By
phantomjs_path = r"E:\Software & Tutorial\Phantom\phantomjs-2.1.1-windows\bin\phantomjs.exe"
service_args = [
    '--proxy=217.156.252.118:8080',
    '--proxy-type=https',
    ]

driver = webdriver.PhantomJS(executable_path=phantomjs_path,service_args=service_args)
driver.get("https://www.google.com.bd/?gws_rd=ssl#q=what+is+my+ip")
print driver.page_source.encode('utf-8')
print "="*70
print driver.title
driver.save_screenshot(r"E:\Software & Tutorial\Phantom\test.png")
driver.quit()

See the saved image(test.png) and see the status. If used ip is blacklisted the google prompted captcha box see that image!! IP has been changed!!

Comments