[ML] Elser inference service mode_version not converted to model_id

[jonathan-buttner]

Affected versions

8.16.0 and greater

Issue

In 8.16 we deprecated the elser service in favor of elastisearch here: https://github.com/elastic/elasticsearch/pull/114052/files#diff-719245221a062735f912753a41ba8d918a099627b89ee1f32fae3a133f7ae73cL55

In that change we no longer support model_version.

We were looking for model_version in 8.15 here: https://github.com/elastic/elasticsearch/blob/8.15/x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elser/ElserInternalService.java#L162-L165

and remapping to model_id.

This issue occurs when an elser service is created in 8.12.x and then the cluster is upgraded to 8.16.x+.

In later versions of the inference plugin we'll return an error like this:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Error parsing request config, model id is missing"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Error parsing request config, model id is missing"
  },
  "status": 400
}

Workaround

To determine which inference endpoint is the cause, retrieve the endpoints:

POST .inference/_search
{
  "query": {
    "match_all": {}
  }
}

Find the endpoint that is missing service_settings.model_id

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": ".inference",
        "_id": "model_elser-model-v2",
        "_score": 1,
        "_source": {
          "model_id": "elser-model-v2", <------ inference endpoint id
          "task_type": "sparse_embedding",
          "service": "elser",
          "service_settings": {
            "num_allocations": 1,
            "num_threads": 1,
            "model_version": ".elser_model_2" <----- has model_version and not model_id
          },
          "task_settings": {}
        }
      }
    ]
  }
}

Delete elser-model-v2

DELETE _inference/elser-model-v2

Recreate the inference endpoint

PUT _inference/sparse_embedding/elser-model-v2
{
  "service": "elser",
  "service_settings": {
    "num_allocations": 1,
    "num_threads": 1
  }
}

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)