Difference between revisions of "Submitting Jobs"

From DeepSense Docs
Jump to: navigation, search
 
(5 intermediate revisions by the same user not shown)
Line 26: Line 26:
 
It will set the properties of gpu resources required by the job
 
It will set the properties of gpu resources required by the job
  
for example
+
'''Example 1'''
 
  [bhatiag@ds-lg-01 ~]$ bsub -gpu "num=2:mode=shared:mps=yes" – python.py
 
  [bhatiag@ds-lg-01 ~]$ bsub -gpu "num=2:mode=shared:mps=yes" – python.py
 
  Job <5979> is submitted to queue <gpu>.
 
  Job <5979> is submitted to queue <gpu>.
  
 
You can check the job status using <code>bjobs</code>
 
You can check the job status using <code>bjobs</code>
 +
 +
'''Example 2'''
 +
 +
Consider a bash script <code>python.sh</code> with following commands into it
 +
#!/bin/bash
 +
source . /opt/anaconda2/etc/profile.d/conda.sh
 +
conda activate py36
 +
python example.py
 +
 +
See [[Writing Script]] for more information
 +
 +
Submit the job <code>python.sh</code> using following command
 +
[bhatiag@ds-lg-01 ~]$ bsub -oo "output.txt" -gpu "num=1:mode=exclusive_process:mps=yes" "python.sh"
 +
Job <5980> is submitted to queue <gpu>.
 +
 +
Output file <code>output.txt</code> will contain
 +
[bhatiag@ds-lg-01 ~]$ cat output.txt
 +
Sender: LSF System <lsfadmin@ds-cmgpu-06>
 +
Subject: Job 5980: <python.sh> in cluster <DeepSenseLSFCluster> Exited
 +
Job <python.sh> was submitted from host <ds-lg-01> by user <bhatiag> in cluster <DeepSenseLSFCluster> at Fri Dec  4 14:28:37 2020
 +
Job was executed on host(s) <ds-cmgpu-06>, in queue <gpu>, as user <bhatiag> in cluster <DeepSenseLSFCluster> at Fri Dec  4 14:28:37 2020
 +
</users/faculty/bhatiag> was used as the home directory.
 +
</users/faculty/bhatiag> was used as the working directory.
 +
Started at Fri Dec  4 14:28:37 2020
 +
Terminated at Fri Dec  4 14:28:40 2020
 +
Results reported at Fri Dec  4 14:28:40 2020
 +
Your job looked like:
 +
------------------------------------------------------------
 +
# LSBATCH: User input
 +
python.sh
 +
------------------------------------------------------------
 +
Exited with exit code 127.
 +
Resource usage summary:
 +
    CPU time :                                  0.05 sec.
 +
    Max Memory :                                6 MB
 +
    Average Memory :                            6.00 MB
 +
    Total Requested Memory :                    -
 +
    Delta Memory :                              -
 +
    Max Swap :                                  -
 +
    Max Processes :                              1
 +
    Max Threads :                                1
 +
    Run time :                                  3 sec.
 +
    Turnaround time :                            3 sec.
 +
The output (if any) follows:
 +
'''Display whatever in your python script'''

Latest revision as of 18:50, 4 December 2020

Jobs can be submitted using bsub command. For more information on bsub , go to bsub You can also check Running jobs for more detail.

Some example:

1. bsub -Is -gpu - bash

This command will submit an interactive job to gpu queue and creates a pseudo-terminal with shell mode when the job starts. Also, it will provide you the jobid.

for example

[bhatiag@ds-lg-01 ~]$ bsub -Is -gpu - bash
Job <5972> is submitted to queue <gpu>.
<<Waiting for dispatch ...>>
<<Starting on ds-cmgpu-05>>

2. bsub -gpu - scriptname

It will submit the script to gpu queue

for example

[bhatiag@ds-lg-01 ~]$ bsub -gpu - python.py
Job <5978> is submitted to queue <gpu>.

3. bsub -gpu "num=2:mode=shared:mps=yes" – scriptname

It will set the properties of gpu resources required by the job

Example 1

[bhatiag@ds-lg-01 ~]$ bsub -gpu "num=2:mode=shared:mps=yes" – python.py
Job <5979> is submitted to queue <gpu>.

You can check the job status using bjobs

Example 2

Consider a bash script python.sh with following commands into it

#!/bin/bash
source . /opt/anaconda2/etc/profile.d/conda.sh
conda activate py36
python example.py

See Writing Script for more information

Submit the job python.sh using following command

[bhatiag@ds-lg-01 ~]$ bsub -oo "output.txt" -gpu "num=1:mode=exclusive_process:mps=yes" "python.sh"
Job <5980> is submitted to queue <gpu>.

Output file output.txt will contain

[bhatiag@ds-lg-01 ~]$ cat output.txt
Sender: LSF System <lsfadmin@ds-cmgpu-06>
Subject: Job 5980: <python.sh> in cluster <DeepSenseLSFCluster> Exited
Job <python.sh> was submitted from host <ds-lg-01> by user <bhatiag> in cluster <DeepSenseLSFCluster> at Fri Dec  4 14:28:37 2020
Job was executed on host(s) <ds-cmgpu-06>, in queue <gpu>, as user <bhatiag> in cluster <DeepSenseLSFCluster> at Fri Dec  4 14:28:37 2020
</users/faculty/bhatiag> was used as the home directory.
</users/faculty/bhatiag> was used as the working directory.
Started at Fri Dec  4 14:28:37 2020
Terminated at Fri Dec  4 14:28:40 2020
Results reported at Fri Dec  4 14:28:40 2020
Your job looked like:
------------------------------------------------------------
# LSBATCH: User input
python.sh
------------------------------------------------------------
Exited with exit code 127.
Resource usage summary:
   CPU time :                                   0.05 sec.
   Max Memory :                                 6 MB
   Average Memory :                             6.00 MB
   Total Requested Memory :                     -
   Delta Memory :                               -
   Max Swap :                                   -
   Max Processes :                              1
   Max Threads :                                1
   Run time :                                   3 sec.
   Turnaround time :                            3 sec.
The output (if any) follows:
Display whatever in your python script