|
| 1 | +# Datadeps (Data Dependencies) |
| 2 | + |
| 3 | +For many programs, the restriction that tasks cannot write to their arguments |
| 4 | +feels overly restrictive and makes certain kinds of programs (such as in-place |
| 5 | +linear algebra) hard to express efficiently in Dagger. Thankfully, there is a |
| 6 | +solution: `spawn_datadeps`. This function constructs a "datadeps region", |
| 7 | +within which tasks are allowed to write to their arguments, with parallelism |
| 8 | +controlled via dependencies specified via argument annotations. Let's look at |
| 9 | +a simple example to make things concrete: |
| 10 | + |
| 11 | +```julia |
| 12 | +A = rand(1000) |
| 13 | +B = rand(1000) |
| 14 | +C = zeros(1000) |
| 15 | +add!(X, Y) = X .+= Y |
| 16 | +Dagger.spawn_datadeps() do |
| 17 | + Dagger.@spawn add!(InOut(B), In(A)) |
| 18 | + Dagger.@spawn copyto!(Out(C), In(B)) |
| 19 | +end |
| 20 | +``` |
| 21 | + |
| 22 | +In this example, we have two Dagger tasks being launched, one adding `A` into |
| 23 | +`B`, and the other copying `B` into `C`. The `add!` task is specifying that |
| 24 | +`A` is being only read from (`In` for "input"), and that `B` is being read |
| 25 | +from and written to (`Out` for "output", `InOut` for "input and output"). The |
| 26 | +`copyto` task, similarly, is specifying that `B` is being read from, and `C` |
| 27 | +is only being written to. |
| 28 | + |
| 29 | +Without `spawn_datadeps` and `In`, `Out`, and `InOut`, the result of these |
| 30 | +tasks would be undefined; the two tasks could execute in parallel, or the |
| 31 | +`copyto!` could occur before the `add!`, resulting in all kinds of mayhem. |
| 32 | +However, `spawn_datadeps` changes things: because we have told Dagger how our |
| 33 | +tasks access their arguments, Dagger knows to control the parallelism and |
| 34 | +ordering, and ensure that `add!` executes and finishes before `copyto!` |
| 35 | +begins, ensuring that `copyto!` "sees" the changes to `B` before executing. |
| 36 | + |
| 37 | +There is another important aspect of `spawn_datadeps` that makes the above |
| 38 | +code work: if all of the `Dagger.@spawn` macros are removed, along with the |
| 39 | +dependency specifiers, the program would still produce the same results, |
| 40 | +without using Dagger. In other words, the parallel (Dagger) version of the |
| 41 | +program produces identical results to the serial (non-Dagger) version of the |
| 42 | +program. This is similar to using Dagger with purely functional tasks and |
| 43 | +without `spawn_datadeps` - removing `Dagger.@spawn` will still result in a |
| 44 | +correct (sequential and possibly slower) version of the program. Basically, |
| 45 | +`spawn_datadeps` will ensure that Dagger respects the ordering and |
| 46 | +dependencies of a program, while still providing parallelism, where possible. |
| 47 | + |
| 48 | +But where is the parallelism? The above example doesn't actually have any |
| 49 | +parallelism to exploit! Let's take a look at another example to see the |
| 50 | +datadeps model truly shine: |
| 51 | + |
| 52 | +```julia |
| 53 | +function tree_reduce!(As::Vector{Array}) |
| 54 | +end |
| 55 | +``` |
0 commit comments