MNedelko MNedelko - 1 year ago 353
Python Question

Bing Congitive Web Search API with Python 3

Hi Stackoverflow Community

I am attempting to access the new Bing Cognitive Search API through a Python 3 script. I was able to find threats on using Bing Search 2.0 (since depreciated) but wasn't able to identify an example for the new API with Python 3. I used the following code:

import urllib.parse
import urllib.request
import json
import base64

def bing_search(query):
key = 'mysubscription_key'
query = urllib.parse.quote(query)

#Create credentials for authentication
user_agent = 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; FDM; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 1.1.4322)'
encoded = base64.b64encode(bytes(':%s' % key, 'utf-8'))
credentials = encoded[:-1] # the "-1" is to remove the trailing "\n" which encode adds
auth = 'Basic %s' % credentials
url = '' + query + '&mkt=en-us'

#Create the API request
urlrequest = urllib.request.urlopen(url) # in Python3 urllib.request(...) becomes
urlrequest.add_header('Authentication', auth)
urlrequest.add_header('User Agent', user_agent)
request_opener = urllib.request.build_opener()

# Handle the response
response =
results = json.load(response)

result_list = results['webPages']['values']

bing_search('good news')

Unfortunately I am receiving the following 'Access denied' error. Would anyone be able to point me in the right direction?

C:\Users\Admin\AppData\Local\Programs\Python\Python35-32\python.exe C:/Users/Admin/PycharmProjects/momely/placementarchitect/
Basic b'OmZhNTlmNTBlMmFmMjQyZjhhYmE5MTZlNmZkYThhMDM'
Traceback (most recent call last):
File "C:/Users/Admin/PycharmProjects/momely/placementarchitect/", line 34, in <module>
bing_search('good news')
File "C:/Users/Admin/PycharmProjects/momely/placementarchitect/", line 22, in bing_search
urlrequest = urllib.request.urlopen(url) # in Python3 urllib.request(...) becomes
File "C:\Users\Admin\AppData\Local\Programs\Python\Python35-32\lib\urllib\", line 162, in urlopen
return, data, timeout)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python35-32\lib\urllib\", line 471, in open
response = meth(req, response)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python35-32\lib\urllib\", line 581, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python35-32\lib\urllib\", line 509, in error
return self._call_chain(*args)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python35-32\lib\urllib\", line 443, in _call_chain
result = func(*args)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python35-32\lib\urllib\", line 589, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Access Denied

Thank you kindly

Answer Source

The authentication is not through basic auth but through custom header Ocp-Apim-Subscription-Key:

Subscription key which provides access to this API.

Which can be found in your subscriptions

See full details at

Assuming the key variable holds the subscription key, what you need to do is simply urlrequest.add_header('Ocp-Apim-Subscription-Key', key)

As a minor suggestion, your code can be greatly simplified by using python-requests. A minimum example can be:

import requests

def bing_search(query):
    url = ''
    # query string parameters
    payload = {'q': query}
    # custom headers
    headers = {'Ocp-Apim-Subscription-Key': 'xxxxxxxxxxxxx'}
    # make GET request
    r = requests.get(url, params=payload, headers=headers)
    # get JSON response
    return r.json()

j = bing_search('good news')
print(j.get('webPages', {}).get('value', {}))