Fixed some problems in Jacobi MPI code, added nonblocking

Author: kah3f

content/courses/parallel-computing-introduction/codes/contour.py

@@ -0,0 +1,107 @@
+---
+title: "Building and Running MPI Programs"
+toc: true
+type: docs
+weight: 29
+date: "2020-11-17T00:00:00"
+menu:
+    parallel_programming:
+        parent: Distributed-Memory Programming
+---
+
+We are now ready to write our first MPI program.  
+
+## A Hello World example.
+
+Download the appropriate code below for your choice of language.
+
+#### C++
+
+Each MPI program must include the `mpi.h` header file. If the MPI distribution was installed correctly, the `mpicc` or `mpicxx` or equivalent wrapper will know the appropriate path for the header and will also link to the correct library.
+
+{{< spoiler text="C++" >}}
+{{< code-download file="/courses/parallel-computing-introduction/codes/mpi1.cxx" lang="c++" >}}
+{{< /spoiler >}}
+
+#### Fortran
+
+All new Fortran programs should use the `mpi` module provided by the MPI software. if the MPI distribution was installed correctly, the `mpif90` or equivalent will find the module and link to the correct library.
+
+Any recent MPI will also provide an `mpi_f08` module.  Its use is recommended, but we will wait till [later](courses/paralll-incomputing-introduction/distributed_mpi_nonblocking_exchange) to introduce it.
+
+{{< spoiler text="Fortran" >}}
+{{< code-download file="/courses/parallel-computing-introduction/codes/mpi1.f90" lang="fortran" >}}
+{{< /spoiler >}}
+
+#### Python
+
+The `mpi4py` package consists of several objects.  Many codes will need only to import the `MPI` object.
+
+{{< spoiler text="Python" >}}
+{{< code-download file="/courses/parallel-computing-introduction/codes/mpi1.py" lang="python" >}}
+{{< /spoiler >}}
+
+### Build It
+
+If using an HPC system log in to the appropriate frontend, such as `login.hpc.virginia.edu`.  If the system uses a software module system, run
+```bash
+module load gcc openmpi
+```
+
+For Python add
+```bash
+module load <python distribution>
+```
+This will also load the correct MPI libraries. You must have already installed mpi4py.  Activate the conda environment if appropriate.
+
+Use mpiexec and –np **only** on the frontends!  Use for short tests only!
+
+Compiling C 
+```bash
+mpicc –o mpihello mpi1.c
+```
+
+Compiling C++
+```bash
+mpicxx –o mpihello mpi1.cxx
+```
+
+Compiling Fortran
+```bash
+mpif90 –o mpihello mpi1.f90
+```
+
+### Execute it
+C/C++/Fortran
+```bash
+mpiexec –np 4 ./mpihello
+```
+
+Python
+```python
+mpiexec –np 4 python mpi1.py
+```
+
+### Submit It
+
+For HPC users, rite a Slurm script to run your program.  Request 1 node and 10 cores on the standard partition.  The process manager will know how many cores were requested from Slurm.
+```bash
+srun ./mpihello
+```
+Or
+```bash
+srun python mpihello\.py
+```
+
+## Using the Intel Compilers and MPI
+
+Intel compilers, MPI, and math libraries (the MKL) are widely used for high-performance applications such as MPI codes, especially on Intel-architecture systems.  The appropriate MPI wrapper compilers are
+```bash
+#C
+mpiicc -o mpihello mpi1.c
+# C++
+mpiicpc -o mpihello mpi1.cxx
+# Fortran
+mpiifort -o mpihello mpi1.f90
+```
+Do not use mpicc, mpicxx, or mpif90 with Intel compilers.  Those are provided by Intel but use the gcc suite and can result in conflicts.

content/courses/parallel-computing-introduction/distributed_mpi_build_run.md

@@ -30,7 +30,7 @@ Python programmers: once your code is working, try to use NumPy array operations
 
 Fortran programmers: when writing double `do` loops over array indices, whenever possible arrange the loops in order of the indices from _right_ to _left_, for cache efficiency. Correct loop ordering can be approximately a factor of 50 faster than incorrect for a double loop.
 
-Use whatever plotting package you know to make a contour plot of the result. If you do not have a preference, you may use `contour.py` below. It assumes a single output file and will transpose the result for Fortran, if output is written by column rather than row, so the orientation is the same.
+Use whatever plotting package you know to make a contour plot of the result. If you do not have a preference, you may use `contour.py` below. It reads files starting with a specified base name and followed by any number of digits, including none. It will assemble them into one image and show a contour plot; if given a command-line option of `-f` it will transpose each subimage for Fortran.
 
 When plotting, the top of an array (row 0) is the bottom of the plot.
 

content/courses/parallel-computing-introduction/distributed_mpi_project_set3.md

@@ -1,5 +1,5 @@
 ---
-title: "Setting Up and Running MPI"
+title: "Setting Up MPI"
 toc: true
 type: docs
 weight: 28
@@ -9,24 +9,6 @@ menu:
         parent: Distributed-Memory Programming
 ---
 
-We are now ready to write our first MPI program.  Select your choice of language below for this example.
-
-## A Hello World example.
-
-{{< spoiler text="C++" >}}
-{{< code-download file="/courses/parallel-computing-introduction/codes/mpi1.cxx" lang="c++" >}}
-{{< /spoiler >}}
-
-{{< spoiler text="Fortran" >}}
-{{< code-download file="/courses/parallel-computing-introduction/codes/mpi1.f90" lang="fortran" >}}
-{{< /spoiler >}}
-
-{{< spoiler text="Python" >}}
-{{< code-download file="/courses/parallel-computing-introduction/codes/mpi1.py" lang="python" >}}
-{{< /spoiler >}}
-
-## Preparing and Running MPI Codes
-
 ### On a Remote Cluster
 
 Refer to the instructions from your site, for example [UVA Research Computing](https://www.rc.virginia.edu/userinfo/howtos/rivanna/mpi-howto/) for our local environment.  Nearly always, you will be required to prepare your code and run it through a _resource manager_ such as [Slurm](https://www.rc.virginia.edu/userinfo/rivanna/slurm/).
@@ -44,13 +26,13 @@ If you have access to a multicore computer, you can run MPI programs on it.
 
 If using a compiled language, before you can build MPI codes you must install a compiler, and possibly some kind of IDE.  See our guides for [C++](/courses/cpp_introduction/setting_up) or [Fortran](/courses/fortran_introduction/setting_up).
 
-For Python, on all operating systems install [mpi4py](https://mpi4py.readthedocs.io/en/stable/index.html). To install mpi4py you must have a working `mpicc` compiler.  For Anaconda this generally requires installing the `gcc_linux-64` package using `conda`. Instructions are [here](https://conda.io/projects/conda-build/en/latest/resources/compiler-tools.html) for Linux and command-line macOS. For Windows use the Anaconda [package manager](/courses/python-introduction/package_managers) interface and install `m2w64-gcc`. 
+For Python, on all operating systems install [mpi4py](https://mpi4py.readthedocs.io/en/stable/index.html). To install mpi4py you must have a working `mpicc` compiler.  If you use `conda` or `mamaba` from a distribution like [miniforge](https://github.com/conda-forge/miniforge), the required compiler will be installed as a dependency.  For `pip` installations you must provide your own compiler setup. 
 
-The author of mpi4py [recommends](https://mpi4py.readthedocs.io/en/stable/install.html) using pip even with an Anaconda environment. This command will be similar on a local system to that used for installation on a multiuser system. 
+The author of mpi4py [recommends](https://mpi4py.readthedocs.io/en/stable/install.html) using pip even with a conda environment. This command will be similar on a local system to that used for installation on a multiuser system. 
 ```no-highlight
 python -m pip install mpi4py
 ```
-This may avoid some issues that occasionally arise in prebuilt mpi4py packages. Be sure that an appropriate `mpicc` executable is in the path.
+This may avoid some issues that occasionally arise in prebuilt mpi4py packages. Be sure that an appropriate `mpicc` executable is in the path.  Alternatively, use the `conda-forge` channel (recommended in general for most scientific software). 
 
 #### Linux
 
@@ -65,7 +47,7 @@ Installing the HPC Toolkit will also install IntelMPI.
 _NVIDIA HPC SDK_
 The NVIDIA software ships with a precompiled version of OpenMPI.
 
-The headers and libraries for MPI _must_ match.  Using a header from one MPI and libraries from another, or using headers from a version from one compiler and libraries from a different compiler, usually results in some difficult-to-interpret bugs.  Moreover, the process manager must be compatible with the MPI used to compile the code.  Because of this, if more than one compiler and especially more than one MPI version is installed, the use of _modules_ ([environment modules](http://modules.sourceforge.net/) or [lmod](https://lmod.readthedocs.io/en/latest/)) becomes particularly beneficial.  Both Intel and NVIDIA provide scripts for the environment modules package (lmod can also read these), with possibly some setup required.  If you plan to use mpi4py as well as compiled-language versions, creating a module for Anaconda would also be advisable.
+The headers and libraries for MPI _must_ match.  Using a header from one MPI and libraries from another, or using headers from a version from one compiler and libraries from a different compiler, usually results in some difficult-to-interpret bugs.  Moreover, the process manager must be compatible with the MPI used to compile the code.  Because of this, if more than one compiler and especially more than one MPI version is installed, the use of _modules_ ([environment modules](http://modules.sourceforge.net/) or [lmod](https://lmod.readthedocs.io/en/latest/)) becomes particularly beneficial.  Both Intel and NVIDIA provide scripts for the environment modules package (lmod can also read these), with possibly some setup required.  If you plan to use mpi4py as well as compiled-language versions, creating a module for your Python distribution would also be advisable.
 
 #### Mac OS
 
@@ -93,67 +75,3 @@ MPI codes must generally be compiled and run through a command line on Windows.
 
 The Intel oneAPI Basic Toolkit includes a customized command prompt in its folder in the Apps menu.
 
-### Build It
-
-In a terminal window on the frontend `rivanna.hpc.virginia.edu` run
-```bash
-module load gcc openmpi
-```
-
-For Python add
-```bash
-module load anaconda
-```
-This will also load the correct MPI libraries. You must have already installed mpi4py.  Activate the conda environment if appropriate.
-
-Use mpiexec and –np **only** on the frontends!  Use for short tests only!
-
-Compiling C 
-```bash
-mpicc –o mpihello mpi1.c
-```
-
-Compiling C++
-```bash
-mpicxx –o mpihello mpi1.cxx
-```
-
-Compiling Fortran
-```bash
-mpif90 –o mpihello mpi1.f90
-```
-
-### Execute it
-C/C++/Fortran
-```bash
-mpiexec –np 4 ./mpihello
-```
-
-Python
-```python
-mpiexec –np 4 python mpi1.py
-```
-
-### Submit It
-
-Write a SLURM script to run your program.  Request 1 node and 10 cores on the standard partition.  The process manager will know how many cores were requested from SLURM.
-```bash
-srun ./mpihello
-```
-Or
-```bash
-srun python mpihello\.py
-```
-
-## Using the Intel Compilers and MPI
-
-Intel compilers, MPI, and math libraries (the MKL) are widely used for high-performance applications such as MPI codes, especially on Intel-architecture systems.  The appropriate MPI wrapper compilers are
-```bash
-#C
-mpiicc -o mpihello mpi1.c
-# C++
-mpiicpc -o mpihello mpi1.cxx
-# Fortran
-mpiifort -o mpihello mpi1.f90
-```
-Do not use mpicc, mpicxx, or mpif90 with Intel compilers.  Those are provided by Intel but use the gcc suite and can result in conflicts.

content/courses/parallel-computing-introduction/distributed_mpi_setup.md

@@ -1,23 +1,35 @@
 import sys
+import os
 import argparse
-import glob
+import re
 import numpy as np
 import pylab as plt
 
 parser = argparse.ArgumentParser()
 parser.add_argument("-f", "--fortran", help="Fortran ordering", action="store_true")
-parser.add_argument("filename", help="output file name")
+parser.add_argument("filename", help="Base name for data files")
 args = parser.parse_args()
 base = args.filename
 
-data=np.loadtxt(args.filename,unpack=False)
+files = [f for f in os.listdir('.') if re.match(base+"\d*",f)]
+files.sort()
 
-if args.fortran:
-    data.T
+subdomains=[]
+for file in files:
+    data=np.loadtxt(file,unpack=False)
+
+    if args.fortran:
+        data.T
+    else:
+        pass
+    subdomains.append(data)
 
-print(data.size, data.shape)
+if args.fortran:
+    image=np.hstack(subdomains)
+else:
+    image=np.vstack(subdomains)
 
 fig=plt.figure()
-plt.contourf(data)
+plt.contourf(image)
 plt.colorbar()
 plt.show()