Kittystone Kittystone - 1 month ago 7
Python Question

Extract keys from elasticsearch result and store in other list

I have a ES returned list stored in a dict

res = es.search(index="instances-i*",body=doc)


The result returned by res looks like:

{
"took": 34,
"timed_out": false,
"_shards": {
"total": 1760,
"successful": 1760,
"failed": 0
},
"hits": {
"total": 551,
"max_score": 0.30685282,
"hits": [
{
"_index": "instances-i-0034438e-2016.10.10-08:23:51",
"_type": "type_name",
"_id": "1",
"_score": 0.30685282,
"_source": {
"ansible.isv_alias": "ruiba",
"ansible": {
"isv_alias": "ruiba",
"WEB_URL": "sin01-cloud.trial.sentinelcloud.com/ruiba"
}
}
}
,
{
"_index": "instances-i-0034438e-2016.10.11-08:23:54",
"_type": "type_name",
"_id": "1",
"_score": 0.30685282,
"_source": {
"ansible.isv_alias": "aike3",
"ansible": {
"isv_alias": "aike3",
"WEB_URL": "sin01-cloud.trial.cloud.com/aike3"
}
}
}
,
{
"_index": "instances-i-883sf38e-2016.10.12-08:23:45",
"_type": "type_name",
"_id": "1",
"_score": 0.30685282,
"_source": {
"ansible.isv_alias": "beijing",
"ansible": {
"isv_alias": "beijing",
"WEB_URL": "sin01-cloud.trial.cloud.com/beijing"
}
}
}
.
.
.
.
so on


I need to extract the WEB_URL from the returned dict and store it in a LIST.

Also i am curious if we can actually make sure that there is no DUPLICATE ENTRY OF The value of WEB_URL in the LIST being populated {This is 2nd part though}

Answer

To store the value of WEB_URL you need to iterate over the dictionary-

data = response_from_es['hits']
list_of_urls = []
for item in data['hits']:
    list_of_urls.append(item['_source']['ansible']['WEB_URL'])

print list_of_urls

And to make the list_of_urls unique -> set(list_of_urls)