Merge branch 'staging' of github.com:uvarc/rc-learning into staging

New material for parallel course, and corrections to
python-introduction.  Should not affect any other files.

Author: kah3f

content/notes/slurm-from-cli/scripts/hello.slurm

@@ -1,6 +1,7 @@
 #!/bin/bash
 #SBATCH --nodes=1
 #SBATCH --ntasks=1
+#SBATCH --cpus-per-task=1 # total cores per task
 #SBATCH --mem=32000 # mb total memory
 #SBATCH --time=2:00:00
 #SBATCH --partition=interactive

content/notes/slurm-from-cli/section2.md

@@ -52,16 +52,17 @@ The lines starting with `#SBATCH` are the resource requests.  They are called "p
 ```bash
 #SBATCH --nodes=1
 #SBATCH --ntasks=1
+#SBATCH --cpus-per-task=1 # total cores per task
 #SBATCH --mem=32000 # mb total memory
 #SBATCH --time=2:00:00
 #SBATCH --partition=interactive
 #SBATCH --account=hpc_training
 ```
 Here we are requesting
-  * 1 node, 1 task
+  * 1 node, 1 task, 1 core
   * 32GB of memory (measured in MB). Strictly speaking this will be "Gibibyes."
   * 2 hours of running time.
-  * The standard partition (queue).  A partition must be specified.
+  * The interactive partition (queue).  A partition must be specified.
   * The account (allocation) group `hpc_training`
 
 The next lines set up the environment to run our job.
@@ -81,11 +82,18 @@ We have chosen to name this script `hello.slurm`, but it can have any name.
 
 **Exercise 1**
 
-Download the hello.slurm and hello.py scripts. Transfer them to the cluster by whatever means you wish.  Modify the Slurm script to use your own allocation group name.
+Using the Open OnDemand Slurm Script Generator, create a slurm script with the following resource requests:
+ * 1 node, 1 task, 1 core.
+ * 32GB of memory.
+ * 2 hours of running time.
+ * The interactive partition (queue).
+ * The account (allocation) group `hpc_training`.
+
+Using the displayed text file, compare your slurm script with our example `hello.slurm`. The requested resources should be the same. Once completed, download your slurm script and transfer it to the cluster by whatever means you wish. Also, download `hello.py` and transfer it to the cluster as it will be needed later.
 
 ## Common Slurm Directives
 
-The most commonly used Slurm directives are listed in the table below.  Many options have two versions, one with a single hyphen `-` followed by one letter, or two hyphens `--` followed by an equals sign `=` and a word.  Some commands have no single-letter equivalent.
+The most commonly used Slurm directives are listed in the table below.  Many options have two versions, one with a single hyphen `-` followed by one letter, or two hyphens `--` followed by a word and an equals sign `=`.  Some commands have no single-letter equivalent.
 
 Angle brackets `< >` indicate a value to be specified, and are not typed.
 
@@ -164,9 +172,13 @@ $ module key bio
 
 The available software is also listed on our [website](https://www.rc.virginia.edu/userinfo/rivanna/software/complete-list/)
 
-**Question:**
+**Exercise 2**
+
+Try typing the command `python` in a terminal window. Why was it unable to find the executable? Now, load a module of your choosing that has python. Try the `python` command again. Purge your current modules and try `python` again.
 
-Why does the command `module load R` give an error?
+Use ```module spider R``` to show the available R modules and how to load them. Using this information, why does the command ```module load R``` give an error?
+
+Open hello.slurm using any text editor you prefer and add the lines needed to purge existing modules, load a module that provides python, and execute the hello.py script. For reference, check our example hello.slurm.
 
 
 ## Working with Files and Folders
@@ -214,9 +226,8 @@ $pwd
 /home/mst3k/shakespeare
 ```
 
-**Exercise 2**
-
-Use FastX or Open OnDemand or the command line to create a new folder under your scratch directory. Practice changing into and out of it.
+**Exercise 3**
 
-Use FastX and Caja to navigate to your `/scratch` directory. To get there, click `Go` in the Caja menu.  A textbox will open. Be sure that "search for files" is unchecked.  Erase whatever is in the textbox and type `/scratch/mst3k` (substituting your own user ID).  Still in FastX, open a terminal (the black box, or in the System Tools menu) and navigate to your new scratch folder.
+Use FastX or Open OnDemand or the command line to create a new folder under your scratch directory. Practice changing into and out of it. Move hello.slurm and hello.py into the newly created folder.
 
+Use FastX and Caja to navigate to your /scratch directory. To get there, click Go in the Caja menu. A textbox will open. Be sure that “search for files” is unchecked. Erase whatever is in the textbox and type /scratch/mst3k (substituting your own user ID). Still in FastX, open a terminal (the black box, or in the System Tools menu) and navigate to your new scratch folder.

content/notes/slurm-from-cli/section3.md

@@ -11,18 +11,14 @@ menu:
 ## Running Jobs from Scratch
 
 We recommend that you run your jobs out of your /scratch directory.
-    * Your personal /scratch/mst3k folder has much more storage space than your home directory. 
-    * /scratch is on a Weka filesystem, a storage system designed specifically for fast access.
-    * /scratch is connected to the compute nodes with Infiniband, a very fast network connection.
+ * Your personal /scratch/mst3k folder has much more storage space than your home directory. 
+ * /scratch is on a Weka filesystem, a storage system designed specifically for fast access.
+ * /scratch is connected to the compute nodes with Infiniband, a very fast network connection.
 
 {{< alert >}}
 The scratch system is not permanent storage, and files older than 90 days will be marked for deleting (purging). You should keep copies of your programs and data in more permanent locations such as your home directory, leased storage such as /project or /standard, or on your lab workstation. After your jobs finish, copy the results to permanent storage.
 {{< /alert >}}
 
-**Exercise 3**
-
-Move or copy the hello.slurm script and the hello.py script to the new folder you created in your scratch directory in Exercise 2.  Submit hello.slurm.
-
 ## Submitting a Job
 
 Once we have navigated to the desired working directory in a terminal window, we use the `sbatch` command to submit the job. This assumes that your Slurm script is located in the current working directory.
@@ -38,9 +34,12 @@ We do not make the script executable.  The system handles that.
 $sbatch myjob.slurm
 Submitted batch job 36805
 ```
-
 Always remember that you submit your **job script** and not your executable or interpreter script.
 
+**Exercise**
+
+From your working directory where  hello.slurm is, submit the job.
+
 ## Monitoring a Job
 
 Once submitted, we can monitor our jobs.
@@ -107,7 +106,7 @@ Write a Slurm script that requests 30 minutes of time. Submit a job that will ru
 ```bash
 sleep 30m
 ```
-as the command.  You won't need to request a specific amount of memory. Submit this script and monitor your job's status.  Once it starts, let it run for a few minutes, then cancel it.
+as the command.  You won't need to request a specific amount of memory. Submit this script and monitor your job’s status in the queue with `squeue` or the Active Jobs tab. Once it starts, get information about your job with `scontrol`, let it run for a minute, then cancel it with `scancel`. Practice with the terminal commands or the OOD GUI. Note that you will need your job’s ID for the last two commands.
 
 {{< spoiler text="Example script" >}}
 {{< code-download file="/notes/slurm-from-cli/scripts/slow.slurm" lang="bash" >}}