This page is retained from an earlier version of the HPC wiki only for reference, and describes a cluster which no longer exists - the equivalent up-to-date page for our current cluster is at Clusters - Mercer.
Once you have an HPC account, you are ready to access a cluster. You will need to use SSH (Secure Shell) in order to connect to NYU HPC. SSH is included in all standard Linux and Mac Operating Systems, but it is not included with the Windows OS. If you are using Windows, first you'll need to download an SSH client from http://www.nyu.edu/its/software. There you will find several SSH software packages.
It is strongly recommended that you use PuTTY SSH for Windows-based systems.
Mac users should use the Terminal or X11 client utilities, which is included in the OS, to access the shell.
Note: To access cluster BuTinah, please refer the link: http://nyuad.nyu.edu/research/resources-support/research-facilities/nyuad-hpc.html
Accessing a cluster involves two steps:
- Log on to the bastion host, which provides a secure gateway between NYU HPC and the public network.
- From the bastion host, log on to the login node of the cluster you want to use.
Once you're on a login node, you are inside the cluster you've selected.
SSH tunneling can be used to connect to the clusters with a simple two step procedure. The link to "Access through SSH Tunneling" contains instructions on how to use SSH tunneling to connect to the HPC clusters.
The login instructions vary slightly depending on whether you are coming from a Windows environment or from a Mac or Unix environment.
From a Mac or Unix/Linux:
1. Open a terminal session.
2. Connect to the bastion host:
Make sure that you replace "NetID" with your NetID.
3. Enter your NYU NetID password at the prompt. The bastion host is configured to use "The Restricted Shell" (rbash) for HPC users. This shell only allows limited functions, such as connecting to HPC resources and transferring files.
4. From the bastion host, specify the cluster you want to connect to, as in this example:
5. If your work requires a visual tool like Emacs, xterm, VMD, etc., you will need to invoke the SSH session with X Windows for both your bastion host and login node sessions. Remember, in order to run X Windows from remote systems you must have an X client installed on your desktop. To invoke the SSH session with , use the command below.
Enabling X11 Forwarding on Mac OS X
By default, X11 forwarding is not enabled on Mac Leopard. To enable it you need to have a line "X11Forwarding yes" in the file /private/etc/sshd_config. To achieve this do this command from the terminal.
Enter the password when prompted.
Enabling X11 Forwarding on Windows
In order to run X Windows from remote systems you must have an X client installed on your system with Windows OS. Cygwin/X and Xming are implementations of the X Window System that runs under Microsoft Windows.
6. NYU HPC uses load balancing on the Union Square and Bowery clusters. Load balancing automatically directs users to a login node--dividing the work between the two login nodes on each device as evenly as possible in order to best utilize the resources available at any given moment. The following cluster login names are load balanced.
By using login names above, the system will select a login node for you based on utilization data.
If you prefer, for the Union Square and Bowery clusters, you can bypass the automatic assignment and specify the login node of your choice. To do this, simply use one of the entries listed on the table below.
Use Load Balancing
Use Specific Node
Load balancing not available
Load balancing not available
Note: CUDA and Cardiac have only one login node.
To run PuTTY, download PuTTY SSH to your desktop, and click the icon labeled putty. Once you've opened a PuTTY session, enter the name of the bastion host - hpc.nyu.edu - in the Host Name field, and check the SSH box in the Protocol selection.
To use X tunneling with PuTTY, click on X11 and check the Enable X11 forwarding box. This is important if you plan to use X applications on the cluster.
Once you are logged onto the bastion host, follow the instructions starting with step 4 under the section above "From a Mac or Unix/Linux".
Creating SSH Keys During Your First Login Session
When you log on to a login node for the first time, the Rocks cluster management system will prompt you to set up your SSH keys. The job scheduler (Torque) uses SSH keys to run batch jobs on the compute nodes of the cluster on your behalf without prompting you/it for your password. When the system presents a series of questions during your first login session, simply press the Return key to answer each question. The system will then setup the SSH keys for you on the cluster.
Default Cluster Shell
The default cluster shell on all NYU HPC clusters is bash. If you need a different shell, please request this change by sending email to firstname.lastname@example.org.
Troubleshooting Common Access Issues
After three unsuccessful login attempts to the Bastion host, your originating IP address will be blocked from accessing HPC resources. To unlock your IP address, please send email to email@example.com.
If you receive notice during your Bastion host login attempt about your account being expired, please review information on renewing an account.
SSH Key Issues
If you notice a problem with your SSH keys on a login node:
- Remove the entire .ssh directory from your Home folder. Please note the dot "." at the beginning of the directory name.
- Log out and log in again, then recreate the SSH keys as you did in your first session (see above).
"Too many logins" when trying to access Cardiac
The Cardiac login node is "cardiac1".
For any other questions, please send email to firstname.lastname@example.org.