Setup

CloudReg is designed to be used in the cloud. We chose to work with Amazon Web Services (AWS) and the below setup instructions are for that.

Requirements

  • AWS account

  • IAM Role and User with credentials to access EC2 and S3

  • S3 Bucket to store raw data

  • S3 Bucket to store processed data

  • EC2 instance with Docker

  • EC2 instance with MATLAB

  • CloudFront CDN with HTTP/2 enabled for fast visualization

  • Web Application Firewall for IP-address restriction on data access.

Create AWS account

Follow instructions here to create AWS account or use existing AWS account. All of the following AWS setup instructions should be performed within the same AWS account.

Create IAM Role

  1. Log into AWS console

  2. Navigate to IAM section of console

  3. Click on Roles in the left sidebar

  4. Click Create Role

  5. Click AWS Service under Type of Trusted Entity

  6. Click EC2 as the AWS Service and click Next

  7. Next to Filter Policies, search for S3FullAccess and EC2FullAccess and click the checkbox next to both to add them as policies to this role.

  8. Click Next

  9. Click Next on the Add Tags screen. Adding tags is optional.

  10. On the Review Role screen, choose a role name, like cloudreg_role, and customize the description as you see fit.

  11. Finally, click Create Role

Create IAM User

  1. Log into AWS console

  2. Navigate to IAM section of console

  3. Click on Users in the left sidebar

  4. Click Add User

  5. Choose a User name like cloudreg_user, check Programmatic Access, and click Next

  6. Click on Attach existing policies directly and search for and add S3FullAccess and EC2FullAccess, and click Next

  7. Click Next on the Add Tags screen. Adding tags is optional. Then click Next

  8. On the Review screen, verify the information is correct and click Create User

  9. On the next screen, download the autogenerated password and key and keep them private and secure. We will need these credentials later

Create S3 Bucket

  1. Log into AWS console

  2. Navigate to S3 section of console

  3. Click Create Bucket

  4. Choose a bucket name and be sure to choose the bucket region carefully. You will want to pick the region that is geographically closest to you for optimal visualization speeds. Record the region you have chosen.

  5. Uncheck Block All Public Access. We will restrict access to the data using CloudFront and a Firewall.

  6. The remaining settings can be left as is. Click Create Bucket

Set up CORS on S3 Bucket

  1. Log into AWS console

  2. Navigate to S3 section of console

  3. Click on the S3 Bucket you would like to add CORS to.

  4. Click on the Permissions tab

  5. Scroll to the bottom and click Edit under Cross-origin resource sharing (CORS)

  6. Paste the following text:

    [
    {
        "AllowedHeaders": [
            "Authorization"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],

        "MaxAgeSeconds": 3000
    }
]

  7. click Save Changes

Set up CloudReg EC2 instance

  1. Log into AWS console

  2. Navigate to EC2 section of console

  3. In the left sidebar, click Instances. Make sure you change the region (top right, middle drop-down menu) to match that of your raw data and processed data S3 buckets.

  4. Click Launch Instances

  5. In the search bar, enter the following: ami-098555c9b343eb09c. This is an Amazing Machine Image (AMI) called Deep Learning AMI (Ubuntu 18.04) Version 38.0. Click Select when this AMI shows up.

  6. The default instance type should be t2.micro, if not choose change it to that type. Leave the remaining choices as their defaults and click Review and Launch.

  7. Verify the EC2 instance information is correct and click Launch.

  8. When the key pair pop-up appears, select Choose an existing key pair if you have already created one, or select Create a new key pair if you do not already have one. Follow the instructions on-screen to download and save the key pair.

  9. Follow AWS tutorial to connect to this EC2 instance through the command line.

  10. Once you have connected to the instance via SSH, create the cloud-volume credentials file on the instance using the CLI text editor of your choice.

  11. After creating the cloud-volume credentials file, run the following command to turn off the EC2 instance: sudo shutdown now

Set up MATLAB EC2 instance

  1. Follow instructions here <https://github.com/mathworks-ref-arch/matlab-on-aws> on setting up MATLAB on an EC2 instance. Be sure to create this instance in the same region as your S3 buckets.

Set up AWS Web Application Firewall

  1. Log into AWS console

  2. Navigate to CloudFront section of console

Set up AWS CloudFront

  1. Log into AWS console

  2. Navigate to CloudFront section of console