Skip to content

Latest commit

 

History

History
85 lines (80 loc) · 5.89 KB

basic_bridges_startup-macos.md

File metadata and controls

85 lines (80 loc) · 5.89 KB
Raw

Bridges quickstart

  1. Log into the XSEDE Single Sign-On (SSO) Hub: ssh [email protected]
    1. Log in with your XSEDE username and password
    2. Note: Must have Duo multi-factor authentication enabled (https://portal.xsede.org/mfa)
  2. Log into Bridges: gsissh bridges
    1. Note the path information: username@br###
      1. This will be your "head node" (a randomly assigned login node). This means it'll change each time you log in. This may cause problems with programs like GNU Screen or TMUX (similar to byobu).
      2. Keeping track of your assigned node will allow us to preserve all of our work if our connection is interrupted.
      3. Note: Head nodes are not intended for long-term use. Any long-running processes on head nodes will be terminated by the system automatically.
  3. Start TMUX (which also runs GNU Screen): tmux
    1. If disconnected from your old node and assigned a different (new) node upon login, re-access your old node and TMUX session:
      1. From your new Terminal, log into that old node: ssh br###
      2. In your new terminal, restart the same version of tmux: tmux attach
  4. Load desired modules.
    1. To see currently loaded modules: module list
    2. To see all available modules: module avail
    3. To load a specific module: module load module_name
    4. Note: By default, Bridges loads the Intel compiler. Keep in mind that this may affect any compiling later (e.g., R libraries).
      1. This differs from Jetstream, which utilizes the GCC compiler. If needed, you may load a GCC compiler module. If attempting to run something on Bridges that was already running on Jetstream, it is preferable to load the closest version to the one running on the Jetstream instance being emulated.
  5. Transfer data.
    1. If transferring local data, use Globus Connect Personal.
      1. Follow download instructions: https://docs.globus.org/how-to/globus-connect-personal-mac/
      2. Once the Bridges instance is running, navigate to "Web: Transfer Files" to begin data upload.
        1. Specify local files on lefthand side:
          1. Click "Endpoint"
          2. Click "Administered by me"
          3. Select files to be uploaded as needed.
        2. Specify remote location on righthand side:
          1. Needs to be figured out.
    2. Otherwise, fetch data from open website: wget "http://your-url-here.com"
    3. In Bridges, you have multiple work areas.
      1. /home/username: Head node location. Does not use resources. To access, may use global variable $HOME instead (e.g., cd $HOME).
      2. /pylon_number/group_name/username: Project location. All data should be stored here. To access, may use global variable $SCRATCH instead (e.g., cd $SCRATCH). For more on group names, see https://portal.xsede.org/psc-bridges#account:multiple.
  6. Start processing.
    1. Start process on compute node: srun command
      1. To get a pseudoterminal on a compute node instead run: srun --pty command
    2. Start batch on compute node: sbatch batch_file_name
    3. Keep in mind the resources you're using.
      1. srun allows you to be interactive, but it spends compute resources the entire time it's running.
      2. sbatch is not interactive, but it stops the processes immediately upon finishing the scripts.

Interactively editing using Atom

  1. Download Atom: https://atom.io/.
  2. Install the atom-commander package: Packages > Settings View > Install Packages/Themes > search for "atom-commander" > install
  3. Toggle Atom Commander toolbar: Packages > Atom Commander > Toggle Focus
  4. Press "alt" (after clicking on the Atom Commander toolbar) to bring up options.
  5. Click "Servers", then "Add".
  6. Follow the SFTP setup directions from XSEDE: https://portal.xsede.org/psc-bridges#transfer:sftp
  7. After clicking the Atom Commander toolbar again, press "alt" to bring up options.
  8. Click "Open".
  9. Follow the prompts to log into your account.

Python package installation

Because Bridges users do not have root access, we need to install everything locally in order to get it to work.

  1. Package management with pip (thanks to https://gist.github.com/saurabhshri/46e4069164b87a708b39d947e4527298)
    1. To install pip for yourself (only needs to be done once):
      1. Download pip: wget https://bootstrap.pypa.io/get-pip.py
      2. Install: python get-pip.py --user
      3. Navigate to pip's new directory: cd ~/.local/bin
      4. Update your $PATH variable: PATH=$PATH:~/.local/bin
      5. Source new bash for changes to take effect: source ~/.bashrc
    2. To install/update packages with pip:
      1. For individual packages, run: pip install package_name --user
      2. For requirements files, run: pip install -r requirements.txt --user
  2. Package management with conda
    1. To install conda for yourself (only needs to be done once):
      1. Download source: wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh
      2. Install: bash Anaconda3-5.0.1-Linux-x86_64.sh
      3. Change the default installation to be in (at least) the .local folder: /home/username/.local/anaconda3
      4. Follow the prompts as needed.

Additional resources