Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subscribe to all primaries (for key-space-notifications) #2860

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

atakavci
Copy link
Contributor

@atakavci atakavci commented Mar 14, 2025

Closes/Fixes #2846

This essentially uses the same approach as subscription methods but runs on all primary nodes in the cluster, attempting to combine the results into a single output.

While some parts of it align with existing pub/sub functionality, making them reusable compromises readability. Therefore, I'm not sure it's preferable to make them reusable.

Another point is that I went back and forth on API naming, trying to decide whether to reference the keyspace or not. To provide more clarity on what is happening, I chose a name -SubscribeAllPrimariesAsync- that refers to the primaries, and I’ve kept the keyspace-related explanations in the documentation.

/// <seealso href="https://redis.io/commands/subscribe"/>,
/// <seealso href="https://redis.io/commands/psubscribe"/>.
/// </remarks>
Task SubscribeAllPrimariesAsync(RedisChannel channel, Action<RedisChannel, RedisValue> handler, CommandFlags flags = CommandFlags.None);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should infer "all primaries" (vs secondaries) from flags, like we would with regular subscribe, so this becomes SubscribeAll; I also wonder whether we should internalize this entirely, specifically against the keyspace events, because for regular keys:

  • non-'s' variants will get broadcast duplicates that will be hard to untangle
  • s variants don't make sense to attach to the entire cluster, since they are routed using shard rules

That would also have the advantage of not needing a separate API. I'm not certain on this - mostly thinking aloud here.

}

// Create a new task that will collect all results and observe errors
return Task.Run(async () =>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect we can unwrap a little of the extra task layer here, but I'm not too concerned - this is low throughput

return EnsureSubscribedToPrimariesAsync(sub, channel, flags, false);
}

private Task<bool> EnsureSubscribedToPrimariesAsync(Subscription sub, RedisChannel channel, CommandFlags flags, bool internalCall)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's almost certainly something complex in here around channel reconnects... I'll need to dig

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oww, now i see..
we are missing the cases with failovers and etc..
i ll try to explore the cases. thanks.

@atakavci
Copy link
Contributor Author

@mgravell ,

  • non-'s' variants will get broadcast duplicates that will be hard to untangle
  • s variants don't make sense to attach to the entire cluster, since they are routed using shard rules

this makes total sense and my attempt is solely for the sake of key-space-notifications.
alternatively i was also considering about a special kind of RedisChannel which is only targeting key-space-notifications by design.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

subscribe to redis cluster
2 participants