Curator 5.4.0 does not work with ES 7

[peterellisjones]

Hi logsearch-boshrelease team,

It looks like the version of curator provided on the maintenance VM no longer works with the udpated version of ES. Example output:

maintenance/25652f7d-6c51-46a7-ac36-4e774113df0a:/var/vcap/jobs/curator/bin# ./run-curator
/var/vcap/packages/curator/lib/python3.6/site-packages/curator/utils.py:53: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  cfg = yaml.load(raw)
2020-03-30 08:16:03,413 INFO      Preparing Action ID: 2, "delete_indices"
2020-03-30 08:16:03,418 ERROR     Elasticsearch version 7.6.0 incompatible with this version of Curator (5.4.0)
Traceback (most recent call last):
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/curator/utils.py", line 894, in get_client
    check_version(client)
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/curator/utils.py", line 701, in check_version
    '({1})'.format(".".join(map(str,version_number)), __version__)
curator.exceptions.CuratorException: Elasticsearch version 7.6.0 incompatible with this version of Curator (5.4.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/vcap/packages/curator/bin/curator", line 11, in <module>
    load_entry_point('elasticsearch-curator==5.4.0', 'console_scripts', 'curator')()
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/curator/cli.py", line 211, in cli
    run(config, action_file, dry_run)
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/curator/cli.py", line 158, in run
    client = get_client(**client_args)
  File "/var/vcap/packages/curator/lib/python3.6/site-packages/curator/utils.py", line 901, in get_client
    'Error: {0}'.format(e)
elasticsearch.exceptions.ElasticsearchException: Unable to create client connection to Elasticsearch.  Error: Elasticsearch version 7.6.0 incompatible with this version of Curator (5.4.0)

From this discussion it seems like we need curator 5.7+ to work with ES 7

cheers,

Pete

[peterellisjones]

FYI: We've verified that the curation works with curator 5.8.1 (we've updated our pipelines to do the curation instead of using the maintenance VM)

[axelaris]

Hi @peterellisjones,
thank you for reporting the issue!
I will address it asap