Skip to content

Latest commit

 

History

History
41 lines (26 loc) · 1.43 KB

README.md

File metadata and controls

41 lines (26 loc) · 1.43 KB
Raw

r-cluster

A nice & easy way to create on demand R Clusters on AWS with:

  • R
  • Access to nodes with ssh without password for user rstudio
  • Ready to use dopar

Requirements

  • AWS Command line interface
  • A logged AWS user
  • jq
  • A baked amis ready with R installed (The ones used on this script will be open sourced in the future)
  • A public.key and private.ket under /cert

Usage

  • sh create_cluster.sh
  • sh create_rstudio.sh

Update

  • sh update_nodes.sh: to update the node ip list, e.g. when cluster is restarted.

Reference

  • sh create_cluster.sh: This script will send all these files to an S3 bucket, and then generates the R-Cluster via de aws cli
  • sh create_rstudio.sh: This script will send all these files to an S3 bucket, and then generates a R-Studio instance via de aws cli

Recomendations

If you are going to bake a new AMI, make sure to delete the following files/folder

  • /home/rstudio/credentials/rstudio.key
  • /home/rstudio/nodes.txt
  • /var/lib/cloud/instances (in order to force the excecution of the cloud init script)

More Recomendations

Since the process of bootstraping a node with a lot of R packages is really slow and unstable (and impossible when creating slave nodes without internet), we've decided to bake our own amis with packages already shipped.