v1.14.0-rc.0 🦫

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports[#new] and feedback about new features.

Meilisearch v1.14 exposes a more granular way to express your filterable attributes! 🎉 This release also improves semantic search performance, embedding indexing speed, and a new route to get multiple documents by IDs.

New features and updates 🔥

Granular Filterable Attribute Settings

This feature allows for the choice of the filter features of each filterable attribute with high granularity.
Activating or deactivating a feature will have an impact on the indexing time.
To use the feature, use the new filterableAttributes format in the settings route (PATCH /indexes/INDEX_UID/settings):

{
	"filterableAttributes": [
	  {
	    "attributePatterns": ["genre", "artist"], 
	    "features": { "facetSearch": true, "filter": { "equality": true, "comparison": false } }
	  },
	  {
	    "attributePatterns": ["rank"],
	    "features": { "facetSearch": false, "filter": { "equality": true, "comparison": true } }
	  },
	  {
	    "attributePatterns": ["albumId"],
	    "features": { "facetSearch": false, "filter": { "equality": true, "comparison": false } }
	  },
	]
}

🗒️ In this example, we activate/deactivate each feature depending on the usage of each filterable attribute, the genre and artist are expected to be strings so the facet search and the equality operators (=/!=) are activated, but, the comparison operators are deactivated (>=, <=, ...) because it's not useful and will save time during the indexing process.
However, the rank is expected to be a number, so the facet search is deactivated in favor of the the comparison operators.
Last, the albumId is expected to be unique, so only the equality operators (=/!=) are activated.

For more details about this feature, please refer to its public usage page.

Done by @ManyTheFish in #5254.

Composite Embedders

This feature allows using different embedders at search and indexing time, which is useful to optimize the embedder to each situation:

• Using a remote embedder for indexing in bulk, as remote embedders provide the highest bandwidth (embeddings/s)
• Using a local embedder for answering search queries, as local embedders provide the lowest latency (time to first embedding)

To use the feature, follow these steps:

1. Enable the Composite embedders feature from the Cloud dashboard or with the following:

curl -H 'Content-Type: application/json' MEILISEARCH_URL/experimental-features -d '{ "compositeEmbedders": true }'

2. Send a settings task (PATCH /indexes/INDEX_UID/settings) containing an embedder with source: "composite", and with the parameters searchEmbedder and indexingEmbedder describing the embedder at search time (resp. indexing time). For example, using a Hugging Face inference endpoint.

{
    "embedders": {
        "text": {
            "source": "composite",
            "searchEmbedder": {
                "source": "huggingFace", // locally computed embeddings using a model from the Hugging Face Hub
                "model": "baai/bge-base-en-v1.5",
                "revision": "a5beb1e3e68b9ab74eb54cfd186867f64f240e1a"
            },
            "indexingEmbedder": {
                "source": "rest", // remotely computed embeddings using Hugging Face inference endpoints
                "url": "https://URL.endpoints.huggingface.cloud",
                "apiKey": "hf_XXXXXXX",
                "documentTemplate": "Your {{doc.template}}",
                "request": {
                    "inputs": [
                        "{{text}}",
                        "{{..}}"
                    ]
                },
                "response": [
                    "{{embedding}}",
                    "{{..}}"
                ]
            }
        }
    }
}

3. Send documents to the index. They will be indexed remotely using the Hugging Face inference endpoint.
4. Perform semantic search queries, they will be embedded using the local model fetched from the Hugging Face Hub.

For more details about this feature, please refer to its public usage page.

Done by @dureuill in #5371.

Other improvements

• Get multiple documents by ids by @dureuill in #5384
• Support merging update and replacement operations by @Kerollmops in #5293
• Display the internal indexing steps with timings on the /batches route by @Kerollmops in #5356
• exhaustive facet search by @ManyTheFish in #5369
• Reduce RAM consumption of arroy by @irevoire in meilisearch/arroy#105
• Cache Embeddings in Search with an experimental feature by @dureuill in #5418
• Extend the batch progress view to the indexing of vectors by @irevoire in 5420

Fixes 🐞

• Support fetching the pooling method from the model configuration by @dureuill in #5355
• Make sure to delete useless prefixes by @Kerollmops in #5413

Misc

• Dependencies updates
  • Bump Ubuntu in the CI from 20.04 to 22.04 by @Kerollmops in #5338
  • Bump heed to v0.22 by @irevoire and @Kerollmops in #5406
• CIs and tests
  • Improve test performance of get_index.rs by @DerTimonius in #5210
  • Ollama Integration Tests by @Kerollmops in #5308
  • Ensure the settings routes are now configurated when a new field is added to the Settings struct by @MichaScant in #5149
  • Skip a snapshot test on Windows by @Kerollmops in #5383
• Misc
  • Rename callTrace into progressTrace by @Kerollmops in #5364
  • Make composite embedders an experimental feature by @dureuill in #5401

❤️ Thanks again to our external contributors:

• Meilisearch: @MichaScant
• Heed: @oXtxNt9U
• Arroy: @ptondereau

v1.13.3 🕊️

What's Changed

• Bump mini-dashboard v0.2.18 by @Strift in #5385

Full Changelog: v1.13.2...v1.13.3

v1.13.2 🕊️

What's Changed

• Support dumpless upgrade for all v1.13 patches by @dureuill in #5376

Full Changelog: v1.13.1...v1.13.2

v1.13.1 🕊️

What's Changed

• Hotfix typo tolerance bug by @ManyTheFish in #5346
• Bump mini-dashboard to v0.2.17 by @Strift in #5367
• Documents database stats by @ManyTheFish in #5325
• Fix the dumpless upgrade log by @Kerollmops in #5360

Full Changelog: v1.13.0...v1.13.1

v1.13.0 🕊️

Meilisearch v1.13 stabilizes AI-powered search integration! 🎉 This release also introduces two new experimental features: upgrading to new Meilisearch releases without generating a dump and making federated requests across multiple Meilisearch instances.

🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment takes 4 to 48 hours after a new version becomes available.

Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).

New features and updates 🔥

AI-powered search and vector store stabilization

With v1.13, AI-powered search is available to all users by default and no longer requires manual activation.

Breaking Changes

• vectorStore is no longer an accepted value for the /experimental-features route
• Ollama URLs must end with either /api/embed or /api/embeddings
• Modified error codes:
  • invalid_embedder has been split into invalid_search_embedder and invalid_similar_embedder. These codes are returned when the embedder parameter of a request to /search, /multi-search or /similar specifies a non-existing embedder or is not a string
  • invalid_hybrid_query has been renamed to invalid_search_hybrid_query. It is returned when the hybrid parameter contains unknown keys or is not either null or an object

Done by @dureuill in #5232 & #5234

Experimental feature: Dumpless upgrades

Use --experimental-dumpless-upgrade after updating the Meilisearch binary to migrate to a new release:

./meilisearch --experimental-dumpless-upgrade

This faster and more efficient process does not require you to generate a dump and pass it to Meilisearch when upgrading to a new release.

Warning

Meilisearch strongly recommends you generate a backup snapshot before migrating. This is an experimental feature, and failed upgrades may lead to database corruption.

Read more about it on the dedicated product discussion.

Done by @irevoire and @dureuill in #5264

Experimental feature: Remote federated search requests

Use remote federated search requests together with the /multi-search route to query multiple Meilisearch instances simultaneously. This is particularly useful when handling very large databases.

First, use the /experimental-features route to enable network:

curl \
  -X PATCH 'MEILISEARCH_URL/experimental-features/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "network": true
  }'

Next, set up your network of Meilisearch instances with a call to PATCH /network configuring one self and multiple remotes:

curl \
  -X PATCH 'MEILISEARCH_URL/network/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "remotes": {
      "ms-0": {
        "url": "http://ms-1235.example.meilisearch.io",
        "searchApiKey": "Ecd1SDDi4pqdJD6qYLxD3y7VZAEb4d9j6LJgt4d6xas"
      },
      "ms-1": {
        "url": "http://ms-4242.example.meilisearch.io",
        "searchApiKey": "hrVu-OMcjPGElK7692K7bwriBoGyHXTMvB5NmZkMKqQ"
      }
    },
    "self": "ms-0"
  }'

Repeat this process with every instance in your network, then add documents. Do not send the same documents to different instances.
Finally, make a /multi-search query with the new federationOptions.remote:

curl \
  -X PATCH 'MEILISEARCH_URL/multi-search/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "federation": {},
    "queries": [
        {
            "q": "Batman returns dark",
            "indexUid": "movies",
            "federationOptions": {
                "remote": "ms-0"
            }
        },
        {
            "q": "Batman returns dark",
            "indexUid": "movies",
            "federationOptions": {
                "remote": "ms-1"
            }
        }
    ]
}'

Find more information about this feature on the public usage page

Other improvements

• Improve task auto-batching by exposing a way to limit the total size of a batch by @Kerollmops in #5223
• Add Prometheus metrics to measure task queue latency by @takaebato in #5178
• Speed up the request to list indexes by @irevoire in #5166
• Add support for GITHUB_TOKEN authentication in installation script by @Sherlouk in #5216
• Expose a new usedDatabaseSize field on the /stats route by @ManyTheFish in #5314
• Expose embeddings info on the /stats route by @ManyTheFish in #5341

Fixes 🐞

• Improve error message when an attribute is not filterable by @jameshiew in #5135
• Make sure indexing embeddings is using the right number of threads by @Kerollmops in #5322
• Dumpless upgrade will no longer corrupt the database in case of early stop by @irevoire in #5316
• Fix export/import dump that contains batches data by @irevoire in #5310
• Make sure indexing geo points works with PUT requests by @dureuill in #5332
• Add back semantic search timeout from v1.11.3 by @dureuill in #5339

Misc

• Dependency updates
  • Upgrade dependencies to fix the idna severity issue by @Kerollmops in #5218
• CIs and tests
  • Split tests into separate files by @K-Kumar-01 in #5134, #5171, #5174
  • Remove obsolete test code by @K-Kumar-01 in #5173
  • Fix flaky batches test by @irevoire in #5175
  • Remove hard coded task IDs to prevent flaky tests by @mhmoudr in #5182
  • Improve test performance of get_index.rs by @DerTimonius in #5210
• Other
  • Instruct users to create custom reports on the benchboard by @Kerollmops in #5029
  • Fix typo in a comment by @eltociear in #5184
  • Replace hardcoded string with constants by @Gnosnay in #5169
  • Refactor index-scheduler by @irevoire in #5199
  • Refactor indexer by @dureuill in #5168
  • Auto-generate OpenAPI spec by @irevoire in #4867 & #5231
  • Merge bitmaps by using Extend::extend by @Kerollmops in #5221
  • Fix issue introduced by dumpless upgrade by @manojks1999 in #5284
  • Send the OSS analytics once per day instead of once per hour by @irevoire in #5312
  • Log more metrics around HTTP embedder requests and ANNs in arroy by @Kerollmops in #5288
  • Fix internal error when not correctly using the documentTemplateMaxBytes parameter by @dureuill in #5306
  • When a batch is deleted it no longer keeps some internal data in the DB @Kerollmops and @irevoire in #5272
  • Fix a rare stack overflow when using remote embedders by @Kerollmops with the help of @dureuill #5294 (from v1.12.8)
  • Fix Dotnet tests in sdks-tests.yml by @curquiza in #5291
  • Debug log the channel congestion by @Kerollmops in #5177
  • Improve unexpected panic message by @irevoire in #5300
  • Introduce a compaction subcommand in meilitool by @Kerollmops in #5235
  • Expose a route to get the file content associated with a task by @Kerollmops in #5326
  • Fix workload sha for the benchmark by @ManyTheFish in #5342

❤️ Thanks again to our external contributors:

• Meilisearch: @takaebato, @Sherlouk, @jameshiew, @K-Kumar-01, @mhmoudr, @eltociear, @Gnosnay, @DerTimonius, @manojks1999, @ellnix, @Guikingone, @migueltarga

v1.13.0-rc.3 🕊️

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

New fields in the stats

• Embeddings stats by @ManyTheFish in #5341

Internal fixes

• Fix workload sha by @ManyTheFish in #5342

Full Changelog: v1.13.0-rc.2...v1.13.0-rc.3

v1.13.0-rc.2 🕊️

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Fixes 🦋

• Make sure arroy is using the rayon thread-pool by @Kerollmops in #5322
• Fix the dumpless upgrade corruption by @irevoire in #5316
• Fix batch export/import dump by @irevoire in #5310
• Fix geo update by @dureuill in #5332
• Add back timeout from v1.11.3 by @dureuill in #5339

Other ☁️

• Expose a route to get the file content associated with a task by @Kerollmops in #5326

Meilitool

• Create a new export documents meilitool subcommand by @Kerollmops in #4970
• Introduce the meilitool Hair Dryer by @Kerollmops in #5336

Full Changelog: v1.13.0-rc.1...v1.13.0-rc.2

v1.13.0-rc.1 🕊️

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Improvements 📈

• Remote federated search by @dureuill in #5299
• Expose the usedDatabaseSize corresponding to the size used to store the "real" data in the database and not the disk size used by LMDB used database size in the /stats route by @ManyTheFish in #5314

Fixes 🦋

• When a batch is deleted it no longer keeps some internal data in the DB @Kerollmops in #5272
• Fix a rare stack overflow when using remote embedders by @Kerollmops with the help of @dureuill #5294 (from v1.12.8)
• Fix internal error when passing documentTemplateMaxBytes to a source that doesn't support it by @dureuill in #5306 reported by @ellnix

Other ☁️

• Fix Dotnet tests in sdks-tests.yml by @curquiza in #5291
• Debug log the channel congestion by @Kerollmops in #5177
• Improve unexpected panic message by @irevoire in #5300
• Send the OSS analytics once per day instead of once per hour by @irevoire in #5312
• Improve AI logging by @Kerollmops in #5288

Benchmarks

• Fix workload files after removing the vectorStore experimental feature by @dureuill in #5289
• Fix workload inversion by @ManyTheFish in #5140

Meilitool

• Introduce a compaction subcommand in meilitool by @Kerollmops in #5235

Full Changelog: v1.13.0-rc.0...v1.13.0-rc.1

v1.12.8 🦗

What's Changed

• Accept the max readers param by env var and reduce rayon tasks by @Kerollmops with the help of @dureuill in #5294

Full Changelog: v1.12.7...v1.12.8

v1.13.0-rc.0 🕊️

v1.13.0 release changelogs

Warning

Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.

Meilisearch v1.13 introduces several significant improvements, including stabilizing the Vector Store feature for seamless hybrid search integration! 🎉 This version also simplifies the Meilisearch upgrade process: you don't need a dump for migrating anymore!

New features and updates 🔥

Stabilize Vector Store feature

After its initial release in v1.3.0, we have now fully stabilized the Vector Store feature. The experimental feature no longer requires manual activation, and the vectorStore field is no longer displayed or accepted by the /experimental-features route. This makes it even simpler to utilize our hybrid search capability, which delivers significantly better search relevance by combining traditional full text search with AI-powered semantic search.

Breaking Changes before Stabilization

• Accepted Ollama URLs can only end with /api/embed and /api/embeddings.
• Modified error codes:
  • invalid_embedder has been split into invalid_search_embedder and invalid_similar_embedder. These codes are returned when the embedder parameter of a search (resp. similar) request refers to a non-existing embedder configuration or is not a string.
  • invalid_hybrid_query has been renamed to invalid_search_hybrid_query. It is returned when the hybrid parameter is invalid: contains unknown keys or is not either null or an object.

Done by @dureuill in #5232 & #5234

Experimental Dumpless Upgrade: Ease upgrading to the next version without a dump

From now on you can upgrade any database in the v1.12 version or more to the latest version without using a dump.
That means it's an in-place, way faster upgrade process that consumes only the minimal amount of RAM or disk required.

To upgrade your v1.12 database to v1.13 runs:

./meilisearch --experimental-dumpless-upgrade

Read more about it here.

Done by @irevoire and @dureuill in #5264

Other improvements

• Improve the auto-batching of tasks by exposing a way to limite the total size of a batch by @Kerollmops in #5223
• Related to Prometheus the experimental feature: Add Prometheus metrics to measure the task queue latency by @takaebato in #5178
• Speed up the request to list the indexes by @irevoire in #5166
• Add support for GITHUB_TOKEN authentication in installation script by @Sherlouk in #5216

Fixes 🐞

• Improve the error message when an attributes is not filterable by @jameshiew in #5135

Misc

• Dependencies updates
  • Upgrade dependencies and fixes the idna severity issue by @Kerollmops in #5218
• CIs and tests
  • Split Meilisearch crate tests in separate file by @K-Kumar-01 in #5134
  • Split tests in separate file by @K-Kumar-01 in #5171
  • Remove obsolete test code by @K-Kumar-01 in #5173
  • Fix the flaky batches test by @irevoire in #5175
  • Split tests for option crate meilisearch in a separate test file by @K-Kumar-01 in #5174
  • Remove hard coded task IDs to prevent flaky tests by @mhmoudr in #5182
• Misc
  • Guide people to create custom reports on the benchboard by @Kerollmops in #5029
  • Fix typo in a comment by @eltociear in #5184
  • Replace hardcoded string with constants by @Gnosnay in #5169
  • Refactor the index-scheduler by @irevoire in #5199
  • Refactor indexer by @dureuill in #5168
  • Auto-generate the OpenAPI spec by @irevoire in #4867 & #5231
  • Merge bitmaps by using Extend::extend by @Kerollmops in #5221
  • Fix corrupted task queue errors on index creation by @irevoire in #5239

❤️ Thanks again to our external contributors:

• Meilisearch: @dureuill, @Kerollmops, @takaebato, @irevoire, @Sherlouk, @jameshiew, @K-Kumar-01, @mhmoudr, @eltociear, @Gnosnay.