Stan + Remoter + AWS

Creating an AMI

The simplest way of doing this is to start with an Ubuntu VM and install a docker container with Remoter and Rstan. I wrote a simple bash script that does that. Just ssh into Ubuntu and run:

wget -O - https://link.ignacio.website/remoter | bash

Now ssh back into the instance and modify the password in my docker-compose.yaml.

cd imremoter
nano docker-compose.yml
# crl + x to save and exit

Finally, use docker-compose to get remoter up and running

docker-compose up -d

Now you have a vm with a remoter server and Rstan installed on it. You can turn off that vm and create and ami based on it.

Rendering a markdown on the cloud

Now that you have your ami ready we can use aws.ec2 to start a vm based on it, run a job, and kill the vm.

This example renders a markdown with the 8 school examaple

library(aws.ec2)
library(aws.s3)
IMSecrets::AWS()  # Personal package that setup the enviromental variables for aws

# save to bucket
put_object("./example.Rmd", bucket = "ignacios-test-bucket", object = "example.Rmd") # Saves the markdown to my S3 bucket

# start AWS vm
my_vm <- run_instances(image = "ami-05f39d58b5ff0d8e2", # My remoter ami
                       type = "t2.medium",
                       sgroup = "sg-073bb39404d5b2c8a") # A security group that only allows remoter connections

vm_status <- instance_status(my_vm)
is_initialized <- FALSE

while (!is_initialized) {
  message("Instance is initializing. This can take a while")
  Sys.sleep(20)
  vm_status <- instance_status(my_vm)
  if(length(vm_status$item$instanceStatus$status)!=0) {
    if(vm_status$item$instanceStatus$status =="ok") message("The instance is ready.")
    is_initialized <- TRUE
  }
}

# Run job 

remoter::batch(addr = get_instance_public_ip(my_vm)[[1]], 
               port = IMSecrets::IMremoter()$port, 
               password = IMSecrets::IMremoter()$password, 
               file = "script.R") 

# Download the results

aws.s3::s3sync(bucket = "ignacios-test-bucket", direction = "download") 

# Terminate the instance

terminate_instances(my_vm)

Notice that in this case I’m using remoter::batch to run script.R on my VM. Alternatively, you may want to use remoter::client to use R interactively.

Parallelizing your code

This workflow will allow you to run multi-threaded stan code but I still need to figure out a workflow for MPI. If you are not going to use multi-threaded code, you should probably change the flags in my makevars.