Skip to end of metadata
Go to start of metadata

 

 

This page is retained from an earlier version of the HPC wiki only for reference, and describes a cluster which no longer exists - the equivalent up-to-date page for our current cluster is at Clusters - Mercer.

Access

Once you have an HPC account, you are ready to access a cluster. You will need to use SSH (Secure Shell) in order to connect to NYU HPC. SSH is included in all standard Linux and Mac Operating Systems, but it is not included with the Windows OS. If you are using Windows, first you'll need to download an SSH client from http://www.nyu.edu/its/software. There you will find several SSH software packages.

It is strongly recommended that you use PuTTY SSH for Windows-based systems.

Mac users should use the Terminal or X11 client utilities, which is included in the OS, to access the shell.

Note: To access cluster BuTinah, please refer the link: http://nyuad.nyu.edu/research/resources-support/research-facilities/nyuad-hpc.html

Overview

Accessing a cluster involves two steps:

  1. Log on to the bastion host, which provides a secure gateway between NYU HPC and the public network.
  2. From the bastion host, log on to the login node of the cluster you want to use.

Once you're on a login node, you are inside the cluster you've selected.

Login Steps

SSH tunneling can be used to connect to the clusters with a simple two step procedure. The link to "Access through SSH Tunneling" contains instructions on how to use SSH tunneling to connect to the HPC clusters.
   

Access through SSH Tunneling

   

The login instructions vary slightly depending on whether you are coming from a Windows environment or from a Mac or Unix environment.

From a Mac or Unix/Linux:

1. Open a terminal session.

2. Connect to the bastion host:

$ ssh NetID@hpc.nyu.edu

Note

Make sure that you replace "NetID" with your NetID.

3. Enter your NYU NetID password at the prompt. The bastion host is configured to use "The Restricted Shell" (rbash) for HPC users. This shell only allows limited functions, such as connecting to HPC resources and transferring files. 

4. From the bastion host, specify the cluster you want to connect to, as in this example:
  

$ ssh NetID@usq.es.its.nyu.edu

5. If your work requires a visual tool like Emacs, xterm, VMD, etc., you will need to invoke the SSH session with X Windows for both your bastion host and login node sessions. Remember, in order to run X Windows from remote systems you must have an X client installed on your desktop. To invoke the SSH session with , use the command below.

$ ssh -X NetID@hpc.nyu.edu

$ ssh -X NetID@usq.es.its.nyu.edu

Note

Enabling X11 Forwarding on Mac OS X

By default, X11 forwarding is not enabled on Mac Leopard. To enable it you need to have a line "X11Forwarding yes" in the file /private/etc/sshd_config. To achieve this do this command from the terminal.

$ sudo echo "X11Forwarding yes" >> /private/etc/sshd_config

Enter the password when prompted.

Enabling X11 Forwarding on Windows

In order to run X Windows from remote systems you must have an X client installed on your system with Windows OS. Cygwin/X and Xming are implementations of the X Window System that runs under Microsoft Windows.

6. NYU HPC uses load balancing on the Union Square and Bowery clusters. Load balancing automatically directs users to a login node--dividing the work between the two login nodes on each device as evenly as possible in order to best utilize the resources available at any given moment. The following cluster login names are load balanced.

Cluster Name

Login Name

Union Square

usq.es.its.nyu.edu

Bowery

bowery.es.its.nyu.edu

By using login names above, the system will select a login node for you based on utilization data.

$ ssh NetID@usq.es.its.nyu.edu
$ ssh NetID@bowery.es.its.nyu.edu

If you prefer, for the Union Square and Bowery clusters, you can bypass the automatic assignment and specify the login node of your choice. To do this, simply use one of the entries listed on the table below.

Cluster

Use Load Balancing

Use Specific Node  

Union Square

ssh NetID@usq.es.its.nyu.edu

ssh usq1
ssh usq2

Bowery

ssh NetID@bowery.es.its.nyu.edu

ssh bowery1
ssh bowery2

Cardiac

Load balancing not available

ssh cardiac1

CUDA

Load balancing not available

ssh cuda

Note: CUDA and Cardiac have only one login node.

From Windows/PuTTY:

To run PuTTY, download PuTTY SSH to your desktop, and click the icon labeled putty. Once you've opened a PuTTY session, enter the name of the bastion host - hpc.nyu.edu - in the Host Name field, and check the SSH box in the Protocol selection.



To use X tunneling with PuTTY, click on X11 and check the Enable X11 forwarding box. This is important if you plan to use X applications on the cluster.

Once you are logged onto the bastion host, follow the instructions starting with step 4 under the section above "From a Mac or Unix/Linux".

Creating SSH Keys During Your First Login Session

When you log on to a login node for the first time, the Rocks cluster management system will prompt you to set up your SSH keys. The job scheduler (Torque) uses SSH keys to run batch jobs on the compute nodes of the cluster on your behalf without prompting you/it for your password. When the system presents a series of questions during your first login session, simply press the Return key to answer each question. The system will then setup the SSH keys for you on the cluster.

Default Cluster Shell

The default cluster shell on all NYU HPC clusters is bash. If you need a different shell, please request this change by sending email to hpc@nyu.edu.

Troubleshooting Common Access Issues

Denyhost

After three unsuccessful login attempts to the Bastion host, your originating IP address will be blocked from accessing HPC resources. To unlock your IP address, please send email to hpc@nyu.edu.

Expired Accounts

If you receive notice during your Bastion host login attempt about your account being expired, please review information on renewing an account.

SSH Key Issues

If you notice a problem with your SSH keys on a login node:

  1. Remove the entire .ssh directory from your Home folder. Please note the dot "." at the beginning of the directory name.
  2. Log out and log in again, then recreate the SSH keys as you did in your first session (see above).

"Too many logins" when trying to access Cardiac

The Cardiac login node is "cardiac1".

For any other questions, please send email to hpc@nyu.edu.

   

 

 

PBS Script Generator
An interactive tool that generates PBS script based on user's input. Check this page for more details.
Front-Line HPC Consulting
HPC consultations are available once a week, Monday 1-3 PM. Appointments are required. Please make an appointment at hpc@nyu.edu.

 

 

 

  • No labels