Elasticsearch 2 brings a lot of new features we want, such as faster Geo tools, an improved FST, and better tools for monitoring performance. We have a branch for experimental support, but there's actually quite a bit to the process. There are also some backwards incompatible changes in Elasticsearch 2 that we'll have to work around.

The process for migration might look something like this:

1.) Make whatever non-backwards incompatible changes to support ES 2.0 are possible before doing anything else. This just ensures the number of changes in play during the actual migration are minimal. 324f1fd and 0b35e6a from the experimental branch mostly cover this part already.

2.) Modify the API to support querying against multiple, configurable indices. pelias/api#334 describes part of this work. We will need this for...

3.) Modify the importers and schema to store admin regions and addresses/venues in two separate indices, with different n-grams settings (1-gram for admin regions, 2-gram for addresses/venues). Elasticsearch doesn't support our current setup where different types in the same index have different analysis settings. We want to make this change before, not alongside, the upgrade, so that we can see exactly what performance and result quality affects upgrading has. This change by itself should lead to equivalent results.

4.) Actually upgrade to ES 2.0

Update: we are moving along with the upgrade. Here's a list of task we still have to tackle:

• Merge ES2 supporting code to all repos
• Start a build on prod_build
• Have a build finish on prod_build
• Update all documentation (pelias-doc and all other readmes/wiki pages) to note that Elasticsearch 2 is required, and 1.7 is no longer supported
• Restore correct functionality in the dashboards
• Restore Marvel for diagnostics of prodbuild and dev 1/2 clusters
• Figure out how to test pelias/schema against ES2 on TravisCI
• Gracefully handle lat/lon wrapping (support was disabled in Elasticsearch 2) (handle coordinate wrapping api#570)
• Make any changes required so we can create new builds on prod_build (none were required)
• Update our ops scripts so that we can rollover builds with ES2 (they have moved to a different stack so this is now more complicated)
• big final task! Roll over ES2 to production with no downtime (will require some research)