Parallel algorithms in stdlib

[certik]

Currently most of the features so far discussed are serial.

1. Should we include parallel algorithms in stdlib?

2. Should we use co-arrays or MPI?

3. What would be some good initial parallel algorithms to start with?

[certik]

My own answers would be:

1. Given that Fortran has built-in support for parallelism via co-arrays, I would say that Fortran is intrinsically a parallel language. And in that case, I think it would be very beneficial if there is a set of algorithms that we all need and that would work with all or most production codes.

2. I would say co-arrays would be natural, since those are built-in, unlike MPI.

3. The most useful (for me) would be dense and sparse linear algebra, and also parallel FFT. But the issue is that we would probably have to depend on a 3rd party library (scalapack, PSBLAS, parallel FFT libraries ...), and it's unclear if we can easily figure out an API so that we can have different backends. So perhaps as a first step, if there was some simpler set of parallel algorithms that are widely useful, we should start with those. Alternatively, we can try to bite the bullet and see if we can figure out an API, that all parallel FFT or parallel linear algebra libraries could be used as a backend. Just like there seems to be a "natural" modern Fortran API for linear algebra (Proposal for linalg #10), there might be a similar "natural" modern Fortran API for parallel linear algebra using co-arrays (Parallel linalg #67).

[ivan-pi]

The book by Robert Numrich - Parallel programming with co-arrays contains some collective routines which would fit here. I don't have my copy nearby to be more specific, but I think it was reduction routines like min and max. Also some algorithms for dealing with graphs like the breadth first search.

[zbeekman]

FYI A bunch of parallel reduction routines are already part of the standard TS 18508. co_min co_max co_reduce etc. so they're all part of Fortran 2018 and supported in OpenCoarrays.