I am pretty new to ES. I have been trying to search for a db migration tool for long and I could not find one. I am wondering if anyone could help to point me to the right direction.
I would be using Elasticsearch as a primary datastore in my project. I would like to version all mapping and configuration changes / data import / data upgrades scripts which I run as I develop new modules in my project.
In the past I used database versioning tools like Flyaway or Liquibase.
Is there any frameworks / scripts or methods I could use with ES to achieve something similar ?
Does anyone have any experience doing this by hand using scripts and run migration scripts at least upgrade scripts.
Thanks in advance!
From this point of view/need, ES have a huge limitations:
What does that mean in context of your question? You, basically, can't have classic migration tools for ES. And here's what can make your work with ES easier:
use strict mapping (
"dynamic": "strict" and/or
index.mapper.dynamic: false, take a look at mapping docs). This will protect your indexes/types from
you can fetch actual ES mapping and compare it with your data models. If your PL have high enough level library for ES, this should be pretty easy
you can leverage index aliases for migrations
So, a little bit of experience. For me, currently reasonable flow is this:
news) which points to actual index (i.e.
Every time code being deployed, you
Try to put model(type) mapping. If it's done w/o error, this means that you've either
All of this actually means that you're good to go with mappping/data you have, just work with data as always
[bulk]requests for fast reindexing During this reindexing you can safely index new documents normally through the alias. The drawback is that historical data is partially available during reindexing.
This is production-tested solution. Caveats around such approach:
To sum up this: