|
3 | 3 | For many programs, the restriction that tasks cannot write to their arguments
|
4 | 4 | feels overly restrictive and makes certain kinds of programs (such as in-place
|
5 | 5 | linear algebra) hard to express efficiently in Dagger. Thankfully, there is a
|
6 |
| -solution: `spawn_datadeps`. This function constructs a "datadeps region", |
| 6 | +solution called "Datadeps" (short for "data dependencies"), accessible through |
| 7 | +the `spawn_datadeps` function. This function constructs a "datadeps region", |
7 | 8 | within which tasks are allowed to write to their arguments, with parallelism
|
8 |
| -controlled via dependencies specified via argument annotations. Let's look at |
9 |
| -a simple example to make things concrete: |
| 9 | +controlled via dependencies specified via argument annotations. Let's look at a |
| 10 | +simple example to make things concrete: |
10 | 11 |
|
11 | 12 | ```julia
|
12 | 13 | A = rand(1000)
|
@@ -94,3 +95,63 @@ Additionally, we can notice a powerful feature of this model - if the
|
94 | 95 | runs sequentially. This means that the structure of the program doesn't have to
|
95 | 96 | change in order to use Dagger for parallelization, which can make applying
|
96 | 97 | Dagger to existing algorithms quite effortless.
|
| 98 | + |
| 99 | +## Aliasing Support |
| 100 | + |
| 101 | +Datadeps is smart enough to detect when two arguments from different tasks |
| 102 | +actually access the same memory (we say that these arguments "alias"). There's |
| 103 | +the obvious case where the two arguments are exactly the same object, but |
| 104 | +Datadeps is also aware of more subtle cases, such as when two arguments are |
| 105 | +different views into the same array, or where two arrays point to the same |
| 106 | +underlying memory. In these cases, Datadeps will ensure that the tasks are |
| 107 | +executed in the correct order - if one task writes to an argument which aliases |
| 108 | +with an argument read by another task, those two tasks will be executed in |
| 109 | +sequence, rather than in parallel. |
| 110 | + |
| 111 | +There are two ways to specify aliasing to Datadeps. The simplest way is the most straightforward: if the argument passed to a task is a view or another supported object (such as an `UpperTriangular`-wrapped array), Datadeps will compare it with all other task's arguments to determine if they alias. This works great when you want to pass that view or `UpperTriangular` object directly to the called function. For example: |
| 112 | + |
| 113 | +```julia |
| 114 | +A = rand(1000) |
| 115 | +A_l = view(A, 1:500) |
| 116 | +A_r = view(A, 501:1000) |
| 117 | + |
| 118 | +# inc! supports views, so we can pass A_l and A_r directly |
| 119 | +inc!(X) = X .+= 1 |
| 120 | + |
| 121 | +Dagger.spawn_datadeps() do |
| 122 | + # These two tasks don't alias, so they can run in parallel |
| 123 | + Dagger.@spawn inc!(InOut(A_l)) |
| 124 | + Dagger.@spawn inc!(InOut(A_r)) |
| 125 | + |
| 126 | + # This task aliases with the previous two, so it will run after them |
| 127 | + Dagger.@spawn inc!(InOut(A)) |
| 128 | +end |
| 129 | +``` |
| 130 | + |
| 131 | +The other way allows you to seperate what argument is passed to the function, |
| 132 | +from how that argument is accessed within the function. This is done with the |
| 133 | +`Deps` wrapper, which is used like so: |
| 134 | + |
| 135 | +```julia |
| 136 | +A = rand(1000, 1000) |
| 137 | + |
| 138 | +inc_upper!(X) = UpperTriangular(X) .+= 1 |
| 139 | +inc_ulower!(X) = UnitLowerTriangular(X) .+= 1 |
| 140 | +inc_diag!(X) = X[diagind(X)] .+= 1 |
| 141 | + |
| 142 | +Dagger.spawn_datadeps() do |
| 143 | + # These two tasks don't alias, so they can run in parallel |
| 144 | + Dagger.@spawn inc_upper!(Deps(A, InOut(UpperTriangular))) |
| 145 | + Dagger.@spawn inc_ulower!(Deps(A, InOut(UnitLowerTriangular))) |
| 146 | + |
| 147 | + # This task aliases with the `inc_upper!` task (`UpperTriangular` accesses the diagonal of the array) |
| 148 | + Dagger.@spawn inc_diag!(Deps(A, InOut(Diagonal))) |
| 149 | +end |
| 150 | +``` |
| 151 | + |
| 152 | +You can pass any number of aliasing modifiers to `Deps`. This is particularly |
| 153 | +useful for declaring aliasing with `Diagonal`, `Bidiagonal`, `Tridiagonal`, and |
| 154 | +`SymTridiagonal` access, as these "wrappers" make a copy of their parent array |
| 155 | +and thus can't be used to "mask" access to the parent like `UpperTriangular` |
| 156 | +and `UnitLowerTriangular` can (which is valuable for writing memory-efficient, |
| 157 | +generic algorithms in Julia). |
0 commit comments