AWS EC2 is a great choice for deploying containerized and load balanced services in the cloud. Its ability to autoscale and automated health checking features make it attractive to users who want to reduce cost and want to horizontally scale base on traffic.

• An active AWS account configured on the machine with AWS CLI installed and configured

  • Install instruction: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
  • Configure AWS account instruction: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html

• Docker is installed and running on the machine.

  • Install instruction: https://docs.docker.com/install

• Install required python packages

  • $ pip install -r requirements.txt

1. Build and save Bento Bundle from BentoML quick start guide

2. Create Sagemaker deployment with the deployment tool

   Run deploy script in the command line:

   $ BENTO_BUNDLE_PATH=$(bentoml get IrisClassifier:latest --print-location -q)
   $ python deploy.py $BENTO_BUNDLE_PATH my-first-ec2-deployment ec2_config.json
   
   # Sample output
   Creating S3 bucket for cloudformation
   Build and push image to ECR
   Generate CF template
   Build CF template
   Deploy EC2

   Get EC2 deployment information and status

   $ python describe.py my-first-ec2-deployment
   
   # Sample output
   {
     "InstanceDetails": [
       {
         "instance_id": "i-03ff2d1b9b717a109",
         "endpoint": "3.101.38.18",
         "state": "InService",
         "health_status": "Healthy"
       }
     ],
     "Endpoints": [
       "3.101.38.18:5000/"
     ],
     "S3Bucket": "my-ec2-deployment-storage",
     "TargetGroup": "arn:aws:elasticloadbalancing:us-west-1:192023623294:targetgroup/my-ec-Targe-3G36XKKIJZV9/d773b029690c84d3",
     "Url": "http://my-ec2-deployment-elb-2078733703.us-west-1.elb.amazonaws.com"
   }

3. Make sample request against deployed service

   $ curl -i \
     --header "Content-Type: application/json" \
     --request POST \
     --data '[[5.1, 3.5, 1.4, 0.2]]' \
     https://ps6f0sizt8.execute-api.us-west-2.amazonaws.com/predict
   
   # Sample output
   HTTP/1.1 200 OK
   Content-Type: application/json
   Content-Length: 3
   Connection: keep-alive
   Date: Tue, 21 Jan 2020 22:43:17 GMT
   x-amzn-RequestId: f49d29ed-c09c-4870-b362-4cf493556cf4
   x-amz-apigw-id: GrC0AEHYPHcF3aA=
   X-Amzn-Trace-Id: Root=1-5e277e7f-e9c0e4c0796bc6f4c36af98c;Sampled=0
   X-Cache: Miss from cloudfront
   Via: 1.1 bb248e7fabd9781d3ed921f068507334.cloudfront.net (CloudFront)
   X-Amz-Cf-Pop: SFO5-C1
   X-Amz-Cf-Id: HZzIJUcEUL8aBI0KcmG35rsG-71KSOcLUNmuYR4wdRb6MZupv9IOpA==
   
   [0]%

4. Delete EC2 deployment

   python delete.py my-first-ec2-deployment

Use command line

python deploy.py <Bento_bundle_path> <Deployment_name> <Config_JSON default is ec2_config.json>

Example:

MY_BUNDLE_PATH=${bentoml get IrisClassifier:latest --print-location -q)
python deploy.py $MY_BUNDLE_PATH my_first_deployment ec2_config.json

Use Python API

from deploy import deploy_to_ec2

deploy_to_ec2(BENTO_BUNDLE_PATH, DEPLOYMENT_NAME, CONFIG_JSON)

• region: AWS region for EC2 deployment
• ec2_auto_scale:
  • min_size: The minimum number of instances for the auto scale group.
  • desired_capacity: The desired capacity for the auto scale group. Auto Scaling group will start by launching as many instances as are specified for desired capacity.
  • max_size: The maximum number of instances for the auto scale group
• instance_type: Instance type for the EC2 deployment. See https://aws.amazon.com/ec2/instance-types/ for more info
• ami_id: The Amazon machine image (AMI) used for launching EC2 instance. The default is /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html for more information.
• elb:
  • health_check_interval_seconds: The approximate interval, in seconds, between health checks of an individual instance. Valid Range: Minimum value of 5. Maximum value of 300.
  • health_check_path.: The URL path for health check. Default is /healthz
  • health_check_port: Health check port. Default is 5000
  • health_check_timeout_seconds: The amount of time, in seconds, during which no response means a failed health check.
  • healthy_threshold_count: The number of consecutive health checks successes required before moving the instance to the Healthy state. Valid Range: Minimum value of 2. Maximum value of 10.
• environment_variables: This takes a dictionary of variable, value pairs that are passed into docker as environment variables. If you want to pass bentoml specific environment variable use this. eg environment_variables: {'BENTOML_MB_MAX_BATCH_SIZE': '300'}

Use command line

python update.py <Bento_bundle_path> <Deployment_name> <Config_JSON>

Use Python API

from update import update_deployment
update_deployment(BENTO_BUNDLE_PATH, DEPLOYMENT_NAME, CONFIG_JSON)

Use command line

python describe.py <Deployment_name> <Config_JSON>

Use Python API

from describe import describe_deployment
describe_deployment(DEPLOYMENT_NAME, CONFIG_JSON)

Use command line

python delete.py <Deployment_name> <Config_JSON>

Use Python API

from  delete import delete_deployment
delete_deployment(DEPLOYMENT_NAME, CONFIG_JSON)