Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure max number of allocations per node #124653

Open
dgieselaar opened this issue Mar 12, 2025 · 2 comments
Open

Configure max number of allocations per node #124653

dgieselaar opened this issue Mar 12, 2025 · 2 comments
Labels
Feature:GenAI Features around GenAI :ml Machine learning Team:ML Meta label for the ML team

Comments

@dgieselaar
Copy link
Member

dgieselaar commented Mar 12, 2025

Currently, when using semantic_text with an inference endpoint, Elasticsearch can scale up aggressively on ingest peaks, consuming a lot of resources. Ideally, users (me) should be able to configure a max # of allocations regardless of endpoint settings to prevent a laptop from becoming an airplane.

@dgieselaar dgieselaar added the :ml Machine learning label Mar 12, 2025
@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Mar 12, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@jonathan-buttner jonathan-buttner added the Feature:GenAI Features around GenAI label Mar 12, 2025
@jonathan-buttner
Copy link
Contributor

To add some more context here, a Connector was being used in conjunction with semantic_text. I believe it was using the default elser inference endpoint which limits the max allocations to 32.

@dgieselaar dgieselaar changed the title Configure max number of allocations Configure max number of allocations per node Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:GenAI Features around GenAI :ml Machine learning Team:ML Meta label for the ML team
Projects
None yet
Development

No branches or pull requests

3 participants