user3373322 user3373322 - 5 months ago 77
Python Question

NameError: name 'driver' is not defined

I use above code to scrape friend list from facebook UID and am getting an error:

File "C:\Users\Tn\PycharmProjects\untitled\test\", line 15, in friend_uid_list
soup = from_uid(uid)
File "C:\Users\Tn\PycharmProjects\untitled\test\", line 11, in from_uid
driver.get('' + uid + '/friends')
NameError: name 'driver' is not defined

Can you show me how to fix it ? Thank you very much ! Below code is my code

import multiprocessing
from selenium.common.exceptions import TimeoutException
from bs4 import BeautifulSoup
from selenium import webdriver
from import By

def from_uid(uid):
driver.get('' + uid + '/friends')
return BeautifulSoup(driver.page_source, "html5lib")

def friend_uid_list(uid):
soup = from_uid(uid)
friends = soup.find_all("div", class_="fsl fwb fcb")
target = open('C:/friend_uid_list.txt', 'a')
for href in friends:
href = href.find('a')
target.write(href + "\n")

if __name__ == '__main__':

driver = webdriver.Firefox()

pool = multiprocessing.Pool(3), [100004159542140,100004159542140,100004159542140])


The reason is simple: You create some new processes, and it can't see the variables in another process(main process).

There are several solutions:

  1. Pass the variables you need as arguments. But this is not possible since driver is not picklable.

  2. Create a new driver for each process.

  3. Use multi-threading instead of multi-processing. However I'm not sure if selenium works this way, you'll have to test it yourself.