Add documentation for ensureEvenDistributionOfPartitions parameter on the kafka scaler #1539
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for keda ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
Thank you for your contribution! 🙏 We will review your PR as soon as possible.
Learn more about:
|
zroubalik
reviewed
Mar 5, 2025
There was a problem hiding this comment.
We should add a detailed explanation of this feature and the algorithm used here. Ideally with some examples of how the value changes with the number of partitions/consumers.
Maybe a new section below this:
https://deploy-preview-1539--keda.netlify.app/docs/2.17/scalers/apache-kafka/#new-consumers-and-offset-reset-policy
patelvp
force-pushed
the
ensureEvenDistributionOfPartitionsParamDocumentation
branch
3 times, most recently
from
6ff205a to
d380fd9
Compare
… the kafka scaler Signed-off-by: Vishal Patel <[email protected]>
patelvp
force-pushed
the
ensureEvenDistributionOfPartitionsParamDocumentation
branch
from
d380fd9 to
890179c
Compare
|
@zroubalik Force pushed a rebase and added a detailed explanation of how it behaves |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Scaling kafka consumers should be done in factors of partition count on the topic. This is to ensure that the partitions are evenly spread across all consumers. If the paritions are not evenly spread we run the risk of some partitions being consumed faster than other. This PR adds a new property on the kafka scaler ensureEvenDistributionOfPartitions. When this property is set to true the scaler ensure that the number of pods are always evenly spread across the number of topics on the partition.
Checklist
Fixes kedacore/keda#2581