MeltingDog MeltingDog - 15 days ago 5
JSON Question

How to view JSON file with all results from an Elastic Search?

Sorry, I've never used Elastic Search before and am a little confused here.

I've been given a site with Elastic Search on it. I want to download a .json file with all my results. These files area located on an AWS server and I can access the results by using a URL such as:

https://idnumbers.ap-southeast-2.aws.found.io/elasticsearch_products/_search


However, these seem to only show a very limited number of results, when in fact I know there should be thousands.

Is there a way for me to view the complete file?

Val Val
Answer

By default, only 10 results are coming back. You can add a size=xyz parameter to your URL in order to see more. This will get you 1000 results:

https://idnumbers.ap-southeast-2.aws.found.io/elasticsearch_products/_search?size=1000

However, there is a logical limit of how many results you can get back that way, which is set at 10000. So if you need to retrieve more results, you can look into the scan/scroll API which allows you to iterate over all your data set.

If you don't want to mess with those multiple queries, you can use another tool which will do it for you and download the full content of your ES cluster into a file containing the JSON documents:

elasticdump \
  --input=https://idnumbers.ap-southeast-2.aws.found.io \
  --output=/tmp/idnumbers.json \
  --type=data