==============================
User Guide
==============================

In order to use Spectral, the user has to do the following
steps in the submission script:

 
- Request Spectral resources instead of NVMe
- Declare the path where the files will be saved in the node-local NVMe (``PERSIST_DIR``)
- Declare the path on GPFS where the files will be copied (``PFS_DIR``)
- Execute the script spectral_wait.py when the application is finished in order to copy the files from NVMe to GPFS

.. list-table::
    :widths: 35 40 45
    :header-rows: 1

    * - Using GPFS
      - Using NVMe
      - Using Spectral
    * - :: 

            #!/bin/bash
            #BSUB -P xxx
            #BSUB -J NAS-BTIO
            #BSUB -o nasbtio.o%J
            #BSUB -e nasbtio.e%J
            #BSUB -W 10
            #BSUB -nnodes 1
      - ::

            #!/bin/bash
            #BSUB -P xxx
            #BSUB -J NAS-BTIO
            #BSUB -o nasbtio.o%J
            #BSUB -e nasbtio.e%J
            #BSUB -W 10
            #BSUB -nnodes 1
      - :: 

            #!/bin/bash
            #BSUB -P xxx
            #BSUB -J NAS-BTIO
            #BSUB -o nasbtio.o%J
            #BSUB -e nasbtio.e%J
            #BSUB -W 10
            #BSUB -nnodes 1
    * - 
      - :: 

            #BSUB -alloc_flags nvme
      - :: 
        
            #BSUB -alloc_flags spectral
            module load spectral
            export PERSIST_DIR=${BBPATH}
            export PFS_DIR=$PWD/spect/
    * - 
      - ::

            export BBPATH=/mnt/bb/$USER/
            jsrun -n 1 cp btio ${BBPATH
            jsrun -n 1 cp input* ${BBPATH}
      - ::

            export BBPATH=/mnt/bb/$USER/
            jsrun -n 1 cp btio ${BBPATH
            jsrun -n 1 cp input* ${BBPATH}
    * - ::

            jsrun -n 1 -a 16 -c 16 -r 1 ./btio
      - ::

            jsrun -n 1 -a 16 -c 16 -r 1 ${BBPATH}/btio
      - ::

            jsrun -n 1 -a 16 -c 16 -r 1 ${BBPATH}/btio
    * - ::

            ls -l
      - ::

            jsrun -n 1 ls -l ${BBPATH}/
      - ::

            jsrun -n 1 ls -l ${BBPATH}/
    * - 
      - ::

            jsrun -n 1 cp ${BBPATH}/* .
      - ::

            spectral_wait.py


You could observe that with Spectral library there is no reason to explicitly
ask for the data to be copied to GPFS as it is done automatically through the
spectral_wait.py script. Also a a log file called spectral.log will be created
with information on the files that were copied.