On earlier NYU HPC clusters, the output of batch jobs appeared as it happened, in a file like "
my_job.o12345" in the job submission directory. When jobs are submitted from
/scratch, this can causes a heavy small-block-I/O load on the Lustre filesystem, which impacts
On Mercer, job stdout and stderr are instead written to hidden files in your
$HOME area, under
$HOME/.pbs_spool. When the job completes, the
my_job.e12345 files are moved to their final location.
You can monitor your job's progress by looking for its output in this hidden directory
If your job writes too much to stdout or stderr, these temporary files might (temporarily) fill up your
$HOME space allocation. If this is occurring, first check if you can reduce the amount of stdout and stderr (they can slow down your program, contact us if you would like assistance with this). Another option is to redirect stdout and stderr to a file in some other location:
- With bash:
- With csh/tcsh
Options to manage job output:
Give the job a name. The default is the filename of the job script. Within the job,
$PBS_JOBNAMEexpands to the job name
path/for/stdout. Can be a filename or an existing directory. The default filename is
myjob.o12345, in the directory from which the job was submitted
path/for/stderr. Same usage as for
Send email to email@example.com when certain events occur. By default an email is sent only if the job is killed by the batch system.
-m b -m e -m a -m abe
Send email when the job begins (
b), ends (
e) and/or is aborted (
Options to set the job environment:
Use the shell at
/path/to/shellto interpret the script. Default is your login shell, which at NYU HPC is normally
-v VAR1,VAR2="some value",VAR3
Pass variables to the job, either with a specific value (the
VAR=form) or from the submitting environment (without "
Pass the full environment the job was submitted from