Login and Compute Nodes
Whilst the bulk of an HPC cluster's workload is running simulations, there is also a range tasks requiring user interaction, such as:
- Writing and compiling code
- Debugging code (but more on this later)
- Preparing, submitting and monitoring batch jobs
- Analyzing and visualizing simulation results
- Moving data to, from and within the cluster
Such tasks do not have the performance requirements that simulations do, but also must not be allowed to interfere with running jobs. To achieve this the cluster is divided into two primary components:
- a small number of "login" nodes, for interactive activities
- a large number of "compute" nodes, for running jobs
The diagram below illustrates this division.
Note that certain filesystems are visible to the login or compute nodes but not both: specifically at NYU
/archive is not visible to the compute nodes, while
/state/partition1 is visible and local only to individual compute nodes.
Do not run computationally-heavy or long-running jobs on the login nodes! Not only will you have poor performance, the heavy resource usage of such jobs impacts others ability to use the login nodes for their intended purposes. If you need to run a job interactively (for example, when debugging), please do so through an interactive batch session.