Add `MultiProducerSingleConsumerChannel` #305

FranzBusch · 2023-12-19T11:29:56Z

Motivation

The pitch to add external backpressure support to the standard libraries AsyncStream got returned for revision since there are larger open questions around AsyncSequence. However, having external backpressure in a source asynchronous sequence is becoming more and more important.

Modification

This PR adds a modified proposal and implementation that brings the Swift Evolution proposal over to Swift Async Algorithms.

FranzBusch · 2023-12-20T09:57:51Z

@swift-ci please test

glbrntt · 2024-06-14T14:46:21Z

Evolution/0016-mutli-producer-single-consumer-channel.md

+
+### Upstream producer termination
+
+The producer will get notified about termination through the `onTerminate`


nit: can you add calls to onTerminate to the snippets below?

glbrntt · 2024-06-14T14:48:13Z

Evolution/0016-mutli-producer-single-consumer-channel.md

+        })
+    }
+} catch {
+    // `send(contentsOf:)` throws if the asynchronous stream already terminated


Would this be clearer as another case on the sendResult?

glbrntt · 2024-06-14T14:50:59Z

Evolution/0016-mutli-producer-single-consumer-channel.md

+            /// - Parameters:
+            ///   - low: When the number of buffered elements drops below the low watermark, producers will be resumed.
+            ///   - high: When the number of buffered elements rises above the high watermark, producers will be suspended.
+            ///   - waterLevelForElement: A closure used to compute the contribution of each buffered element to the current water level.


Oh, neat. This works nicely for collections.

simonjbeaumont · 2024-07-09T14:29:05Z

...orithms/MultiProducerSingleConsumerChannel/MultiProducerSingleConsumerChannel+Internal.swift

+                        for producerContinuation in producerContinuations {
+                            switch producerContinuation {
+                            case .closure(let onProduceMore):
+                                onProduceMore(.failure(CancellationError()))


This is a state machine bug I picked up running the OpenAPIURLSession tests with this type:

Suggested change

onProduceMore(.failure(CancellationError()))

onProduceMore(.success(()))

Might be worth seeing if we can have similar tests in this project. This particular test, 1kChunk_10kChunks_100BDownloadWatermark, exercised the suspend and resume of the producer a lot due to the relation of the chunk size to the watermark.

simonjbeaumont · 2024-07-09T14:34:50Z

.../AsyncAlgorithms/MultiProducerSingleConsumerChannel/MultiProducerSingleConsumerChannel.swift

+    public static func makeChannel(
+        of elementType: Element.Type = Element.self,
+        throwing failureType: Failure.Type = Never.self,
+        backpressureStrategy: Source.BackpressureStrategy
+    ) -> ChannelAndStream {


What's the reason for this returning a nominal type? Previously it returned a tuple which allows for initialization of the sequence as follows:

let (channel, source) = MultiProducerSingleConsumerChannel.makeChannel(/* ... */)

This is also what the documentation comment on the type suggests.

However, with the nominal type you cannot do that and, if you are storing these as properties, are forced to do this...

let channelAndSource = MultiProducerSingleConsumerChannel.makeChannel(/* ... */) self.channel = channelAndSource.channel self.source = channelAndSource.source

...instead of this...

(self.channel, self.source) = MultiProducerSingleConsumerChannel.makeChannel(/* ... */)

Sadly, tuples don't support ~Copyable types. That's the only reason for a nominal type here.

simonjbeaumont · 2024-07-09T14:53:05Z

.../AsyncAlgorithms/MultiProducerSingleConsumerChannel/MultiProducerSingleConsumerChannel.swift

+//
+//===----------------------------------------------------------------------===//
+
+#if compiler(>=6.0)


Please can we consider offering a pre-Swift 6 variant of this type. There are several places we'd like to adopt this that need to also support Swift 5.9+ for the foreseeable.

Hm not sure yet. Especially since it is part of the public API in the Source. Making the Source copyable makes the type behave way differently.

ser-0xff · 2024-10-23T13:21:03Z

.../AsyncAlgorithms/MultiProducerSingleConsumerChannel/MultiProducerSingleConsumerChannel.swift

+    private let backing: _Backing
+
+    @frozen
+    public struct ChannelAndStream: ~Copyable {


Was it intended to be called ChannelAndStream while it is actually a container for channel and source?
Shouldn't it be called ChannelAndSource instead?

ser-0xff · 2024-10-23T14:34:25Z

Hi!
Thank you for the PR!
We are looking for the functionality you did in MultiProducerSingleConsumerChannel and tried to play with it.
Unfortunately we were not able to compile it because we need it in the framework which is compiled with evolution mode.
I did some changes to make it compile, but not sure these changes are good enough to be a part of public framework.
Do you have any plans to support evolution mode for the AsyncAlgorithms?
We prepared a small reproducing project for you for a case if you will have some time to look at that.
Just

git clone https://github.com/ordo-one/external-reproducers.git
cd external-reproducers/swift/async-algorithms-evolution
xcodebuild archive -quiet -scheme async-algorithms-evolution -configuration Debug -destination 'platform=macOS,arch=arm64' BUILD_LIBRARY_FOR_DISTRIBUTION=YES

FranzBusch · 2024-10-23T15:51:19Z

@ser-0xff Thanks for trying it out. We don't plan on support library evolution mode in any of our packages. What we normally advise is to not compile packages as frameworks but rather add this library as an internal import only dependency inside the frameworks that need it.

ser-0xff · 2024-10-24T07:34:19Z

Thank you for the reply...
Unfortunately internal import does not work if we want to use package type (like MultiProducerSingleConsumerChannel) in the framework public API...

hassila · 2024-10-24T10:34:32Z

Thank you for the reply... Unfortunately internal import does not work if we want to use package type (like MultiProducerSingleConsumerChannel) in the framework public API...

@ser-0xff I guess we need to wrap up the API surface we want to support then and just do internal import as suggested? But let's take that offline from this PR - thanks @FranzBusch for quick reply.

ser-0xff · 2024-10-24T11:45:32Z

Hi, @FranzBusch
We run into another issue when we played with the MultiProducerSingleConsumerChannel.
Our projects are already on Swift6 which has a stricter non-copyable types checks. The MultiProducerSingleConsumerChannel<>.Source type is non-copyable so we can't share it across many tasks, as a result we can use it only in single producer mode.
How could we solve that issue and produce data within multiple tasks?

Here is a minimized test showing the usage pattern we want if you will find some time to look at.

FranzBusch · 2024-10-28T10:10:30Z

We run into another issue when we played with the MultiProducerSingleConsumerChannel.
Our projects are already on Swift6 which has a stricter non-copyable types checks. The MultiProducerSingleConsumerChannel<>.Source type is non-copyable so we can't share it across many tasks, as a result we can use it only in single producer mode.
How could we solve that issue and produce data within multiple tasks?

The source has a copy method that allows you to get a second source. Now the problem is that closures cannot be marked as ~Copyable or rathe @calledOnce. To solution to this is to box the source into a class right now and move it into a task. This either requires a lock or an @unchecked Sendable annotation right now.

ser-0xff · 2024-10-28T14:19:05Z

Missed copy() method in source code.
Will look at that, thank you for the reply.

ser-0xff · 2024-11-05T14:34:34Z

HI, Franz!
We tried using MultiProducersSingleConsumerChannel instead of AsyncStream and encountered behavior unexpected for us. We expected the channel to remain functional as long as at least one associated source was active. However, in the current implementation, it appears that the channel transitions to a finished state as soon as any single source instance is deinitialized.
I created a PR with minimized test into your feature branch for a case if you could find some time to look at that and may be later will include that test to the test suite for MultiProducersSingleConsumerChannel.

FranzBusch · 2024-11-11T13:35:45Z

HI, Franz! We tried using MultiProducersSingleConsumerChannel instead of AsyncStream and encountered behavior unexpected for us. We expected the channel to remain functional as long as at least one associated source was active. However, in the current implementation, it appears that the channel transitions to a finished state as soon as any single source instance is deinitialized. I created a PR with minimized test into your feature branch for a case if you could find some time to look at that and may be later will include that test to the test suite for MultiProducersSingleConsumerChannel.

You are right. I haven't gotten around to implement the correct handling of multiple sources yet.

# Motivation The pitch to add external backpressure support to the standard libraries `AsyncStream` got returned for revision since there are larger open questions around `AsyncSequence`. However, having external backpressure in a source asynchronous sequence is becoming more and more important. # Modification This PR adds a modified proposal and implementation that brings the Swift Evolution proposal over to Swift Async Algorithms.

phausler

I think with some minor edits the pitch is good to go; id like to at the bare minimum see some of that testing reenabled to ensure it isn't leaking.

phausler · 2025-03-24T20:33:34Z

Sources/AsyncAlgorithms/Internal/_TinyArray.swift

+/// It supports arbitrary many elements but if only up to one ``Element`` is stored it does **not** allocate separate storage on the heap
+/// and instead stores the ``Element`` inline.
+@usableFromInline
+struct _TinyArray<Element> {


I wonder if this could perhaps be subsumed by InlineArray instead since the original intent was to have a faster storage (this is not a blocking concept)

InlineArray isn't capable of fully replacing the need here. The idea behind TinyArray is to have a fast path for a single element that doesn't allocate and one that allocates if there are more. Since we can't tell at compile time how many producers we have we need this runtime dynamism.

phausler · 2025-03-24T20:34:54Z

Sources/AsyncAlgorithms/Locking.swift

@@ -17,24 +17,35 @@ import Glibc
 import WinSDK
 #endif

-internal struct Lock {
+@usableFromInline
+internal class Lock {


umm why is this a non-final class? If we are moving to a class then we should just use Mutex from Synchronization

I removed all of the changes here since the latest code now uses Mutex.

.../AsyncAlgorithms/MultiProducerSingleConsumerChannel/MultiProducerSingleConsumerChannel.swift

phausler · 2025-03-24T20:40:47Z

...rithmsTests/MultiProducerSingleConsumerChannel/MultiProducerSingleConsumerChannelTests.swift

+final class MultiProducerSingleConsumerChannelTests: XCTestCase {
+    // MARK: - sequenceDeinitialized
+
+    // Following tests are disabled since the channel is not getting deinited due to a known bug


This might be a requirement to landing; id rather not introduce a type with a known leak

I added significantly more tests now and also re-enabled those.

phausler · 2025-03-24T20:42:15Z

Evolution/0016-mutli-producer-single-consumer-channel.md

+
+```swift
+// Writing new values and providing a callback when to produce more
+try source.send(contentsOf: sequence, onProduceMore: { result in


can this be written without ambiguity as a trailing closure?

Yes in a synchronous context both send methods can be called without ambiguity, e.g. this compiles:

try source.send(1) source.send(2) { result in print(result) }

The only thing that needs some special handling is if you want to call a sync send from an async context since the async methods are preferred; however, users can workaround this by creating an inline closure, e.g.:

try { try source.send(1) }()

FranzBusch force-pushed the fb-async-backpressured-stream branch from ddd7523 to 65e8957 Compare December 19, 2023 11:49

FranzBusch requested a review from phausler December 20, 2023 09:57

FranzBusch force-pushed the fb-async-backpressured-stream branch from 4904039 to 662693e Compare June 14, 2024 08:15

FranzBusch changed the title ~~Add AsyncBackpressuredStream proposal and implementation~~ Add MultiProducerSingleConsumerChannel Jun 14, 2024

glbrntt reviewed Jun 14, 2024

View reviewed changes

simonjbeaumont reviewed Jul 9, 2024

View reviewed changes

ser-0xff reviewed Oct 23, 2024

View reviewed changes

phausler reviewed Mar 24, 2025

View reviewed changes

Update proposal and implementation

944b727

FranzBusch force-pushed the fb-async-backpressured-stream branch from 662693e to 944b727 Compare March 25, 2025 14:00

FranzBusch added 5 commits March 25, 2025 15:03

Update proposal

4b46fd8

Add example project

5320d5f

Formatting

e227779

Fix Swift 6.0 build

3c0f927

Future direction for ~Copyable elements

ac217bf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `MultiProducerSingleConsumerChannel` #305

Add `MultiProducerSingleConsumerChannel` #305

FranzBusch commented Dec 19, 2023

FranzBusch commented Dec 20, 2023

glbrntt Jun 14, 2024

glbrntt Jun 14, 2024

glbrntt Jun 14, 2024

simonjbeaumont Jul 9, 2024

simonjbeaumont Jul 9, 2024

FranzBusch Jul 9, 2024

simonjbeaumont Jul 9, 2024

FranzBusch Jul 9, 2024

ser-0xff Oct 23, 2024 •

edited

Loading

ser-0xff commented Oct 23, 2024

FranzBusch commented Oct 23, 2024

ser-0xff commented Oct 24, 2024

hassila commented Oct 24, 2024

ser-0xff commented Oct 24, 2024 •

edited

Loading

FranzBusch commented Oct 28, 2024

ser-0xff commented Oct 28, 2024 •

edited

Loading

ser-0xff commented Nov 5, 2024

FranzBusch commented Nov 11, 2024

phausler left a comment

phausler Mar 24, 2025

FranzBusch Mar 25, 2025

phausler Mar 24, 2025

FranzBusch Mar 25, 2025

phausler Mar 24, 2025

FranzBusch Mar 25, 2025

phausler Mar 24, 2025

FranzBusch Mar 25, 2025


		### Upstream producer termination

		The producer will get notified about termination through the `onTerminate`

	onProduceMore(.failure(CancellationError()))
	onProduceMore(.success(()))

Add MultiProducerSingleConsumerChannel #305

Are you sure you want to change the base?

Add MultiProducerSingleConsumerChannel #305

Conversation

FranzBusch commented Dec 19, 2023

Motivation

Modification

FranzBusch commented Dec 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ser-0xff Oct 23, 2024 • edited Loading

Choose a reason for hiding this comment

ser-0xff commented Oct 23, 2024

FranzBusch commented Oct 23, 2024

ser-0xff commented Oct 24, 2024

hassila commented Oct 24, 2024

ser-0xff commented Oct 24, 2024 • edited Loading

FranzBusch commented Oct 28, 2024

ser-0xff commented Oct 28, 2024 • edited Loading

ser-0xff commented Nov 5, 2024

FranzBusch commented Nov 11, 2024

phausler left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add `MultiProducerSingleConsumerChannel` #305

Add `MultiProducerSingleConsumerChannel` #305

ser-0xff Oct 23, 2024 •

edited

Loading

ser-0xff commented Oct 24, 2024 •

edited

Loading

ser-0xff commented Oct 28, 2024 •

edited

Loading