Skip to content
This repository has been archived by the owner on Feb 25, 2024. It is now read-only.

Commit

Permalink
feature: Updating EC2 deploy to latest spec (#12)
Browse files Browse the repository at this point in the history 
* operator_config for aws-ec2

* updated file structure

* fix: relative import issues

* added operator_config.py for file

* fix: changes to make it work as operators

* test: fixed the tests for ec2

* ci: changed the secrets used

* test

* Updated readme for latest operator changes

- updated config
- remove gif
- new quick start for bentoctl and scripts

* Update README.md
  • Loading branch information
@jjmachan
jjmachan authored Dec 1, 2021
1 parent a22aaa0 commit e8a2081
Show file tree
Hide file tree
Showing 19 changed files with 406 additions and 220 deletions.
1 change: 0 additions & 1 deletion .github/workflows/e2e-test-deploy.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ jobs:

# <insert integration tests needing secrets>

- run: aws --version
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
Expand Down
149 changes: 121 additions & 28 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,43 +1,136 @@
# AWS EC2 deployment tool
<div align="center">
<h1> AWS EC2 Operator </h1>
<p>
<img src="https://user-images.githubusercontent.com/5261489/144035885-7aad0b01-f8c7-41ea-8054-203c2d7eb9ad.png"/>
</p>
</div>

AWS EC2 is a great choice for deploying containerized and load balanced services in the cloud.
Its ability to autoscale and automated health checking features make it attractive to
users who want to reduce cost and want to horizontally scale base on traffic.

<p align="center">
<img src="demo.gif" alt="demo of aws-ec2-deploy tool"/>
</p>

## Prerequisites

- An active AWS account configured on the machine with AWS CLI installed and configured
- Install instruction: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
- Configure AWS account instruction: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
- Latest version of `aws-cli > 2.0` and `sam-cli > 1.0`.
- Docker is installed and running on the machine.
- Install instruction: https://docs.docker.com/install
- Install required python packages
- `$ pip install -r requirements.txt`

## Deploy IrisClassifier from Bentoml quick start guide to AWS EC2
## Quickstart with bentoctl

Bentoctl is a CLI tool that you can use to deploy bentos to EC2. It helps in configuring and managing your deployments super easy.

1. Install bentoctl via pip
```
$ pip install bentoctl
```

2. Add AWS EC2 operator
```
$ bentoctl operator add aws-ec2
```

3. Generate deployment_config.yaml file for your deployment. The `bentoctl generate` command can be used to interactively create the `deployment_config.yaml` file which is used to configure the deployment.
```
$ bentoctl generate
Bentoctl Interactive Deployment Spec Builder
Welcome! You are now in interactive mode.
This mode will help you setup the deployment_spec.yaml file required for
deployment. Fill out the appropriate values for the fields.
(deployment spec will be saved to: ./deployment_spec.yaml)
api_version: v1
metadata:
name: test
operator: aws-ec2
spec:
bento: .
region: us-west-1
instance_type: t2.micro
ami_id: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
enable_gpus: False
ec2_auto_scale:
min_size: 1
desired_capacity: 1
max_size: 1
elastic_load_balancing:
health_check_interval_seconds: 5
health_check_path: /healthz
health_check_port: 5000
health_check_timeout_seconds: 3
healthy_threshold_count: 2
environment_variables:
filename for deployment_spec [deployment_spec.yaml]:
deployment spec file exists! Should I override? [Y/n]: y
deployment spec generated to: deployment_spec.yaml
```

4. Deploy to EC2
```
$ bentoctl deploy deployment_config.yaml --describe-deployment
```

6. Check endpoint. We will try and test the endpoint The url for the endpoint given in the output of the describe command or you can also check the API Gateway through the AWS console.

```bash
$ curl -i \
--header "Content-Type: application/json" \
--request POST \
--data '[[5.1, 3.5, 1.4, 0.2]]' \
https://ps6f0sizt8.execute-api.us-west-2.amazonaws.com/predict

# Sample output
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 3
Connection: keep-alive
Date: Tue, 21 Jan 2020 22:43:17 GMT
x-amzn-RequestId: f49d29ed-c09c-4870-b362-4cf493556cf4
x-amz-apigw-id: GrC0AEHYPHcF3aA=
X-Amzn-Trace-Id: Root=1-5e277e7f-e9c0e4c0796bc6f4c36af98c;Sampled=0
X-Cache: Miss from cloudfront
Via: 1.1 bb248e7fabd9781d3ed921f068507334.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: SFO5-C1
X-Amz-Cf-Id: HZzIJUcEUL8aBI0KcmG35rsG-71KSOcLUNmuYR4wdRb6MZupv9IOpA==

[0]%

7. Delete deployment
```
$ bentoctl delete deployment_config.yaml
```
## Quickstart with scripts
You can also use this operator directly with the scripts provided.
1. Build and save Bento Bundle from [BentoML quick start guide](https://github.com/bentoml/BentoML/blob/master/guides/quick-start/bentoml-quick-start-guide.ipynb)
2. Copy and change the [sample config file](ec2_config.json) given and change it according to your deployment specifications. Check out the [config section](#configuration-options) to find the different options available.
3. Install required python packages
`$ pip install -r requirements.txt`
3. Create EC2 deployment with the deployment tool.
Run deploy script in the command line:
```bash
$ BENTO_BUNDLE_PATH=$(bentoml get IrisClassifier:latest --print-location -q)
$ python deploy.py $BENTO_BUNDLE_PATH my-first-ec2-deployment ec2_config.json
$ ./deploy $BENTO_BUNDLE_PATH my-first-ec2-deployment ec2_config.json
```
Get EC2 deployment information and status
```bash
$ python describe.py my-first-ec2-deployment
$ ./describe my-first-ec2-deployment
# Sample output
{
Expand Down Expand Up @@ -87,29 +180,29 @@ users who want to reduce cost and want to horizontally scale base on traffic.
5. Delete EC2 deployment
```bash
$ python delete.py my-first-ec2-deployment
$ ./delete my-first-ec2-deployment
```

## Deployment operations

### Configuration options
## Configuration options
* `region`: AWS region for EC2 deployment
* `instance_type`: Instance type for the EC2 deployment. See https://aws.amazon.com/ec2/instance-types/ for more info.
* `ami_id`: The Amazon machine image (AMI) used for launching EC2 instance. The default is `/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2`. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html for more information.
* `enable_gpus`: (Optional) To enable access to the GPUs if you're using GPU-accelerated instance_types.
* `ec2_auto_scale`:
* `min_size`: The minimum number of instances for the auto scale group.
* `desired_capacity`: The desired capacity for the auto scale group. Auto Scaling group will start by launching as many instances as are specified for desired capacity.
* `max_size`: The maximum number of instances for the auto scale group
* `instance_type`: Instance type for the EC2 deployment. See https://aws.amazon.com/ec2/instance-types/ for more info
* `enable_gpus`: (Optional) To enable access to the GPUs if you're using GPU-accelerated instance_types.
* `ami_id`: The Amazon machine image (AMI) used for launching EC2 instance. The default is `/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2`. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html for more information.
* `elb`:
* `elastic_load_balancing`:
* `health_check_interval_seconds`: The approximate interval, in seconds, between health checks of an individual instance. Valid Range: Minimum value of 5. Maximum value of 300.
* `health_check_path.`: The URL path for health check. Default is `/healthz`
* `health_check_port`: Health check port. Default is `5000`
* `health_check_timeout_seconds`: The amount of time, in seconds, during which no response means a failed health check.
* `healthy_threshold_count`: The number of consecutive health checks successes required before moving the instance to the Healthy state. Valid Range: Minimum value of 2. Maximum value of 10.
* `environment_variables`: This takes a dictionary of variable, value pairs that are passed into docker as environment variables. If you want to pass bentoml specific environment variable use this. eg `environment_variables: {'BENTOML_MB_MAX_BATCH_SIZE': '300'}`
* `environment_variables`: This takes a list of dicts with `name` and `value` keys for ENVAR name and ENVAR value repsectively. They are passed into docker as environment variables. You can also use this to pass bentoml specific environment variable use this. eg `environment_variables: [{'name': 'BENTOML_MB_MAX_BATCH_SIZE': 'value': '300'}]`
## Deployment operations
### Create a deployment
Expand All @@ -129,9 +222,9 @@ python deploy.py $MY_BUNDLE_PATH my_first_deployment ec2_config.json
Use Python API
```python
from deploy import deploy_to_ec2
from bentoctl_aws_ec2 import deploy
deploy_to_ec2(BENTO_BUNDLE_PATH, DEPLOYMENT_NAME, CONFIG_JSON)
deploy(BENTO_BUNDLE_PATH, DEPLOYMENT_NAME, CONFIG_JSON)
```
Expand All @@ -147,8 +240,8 @@ python update.py <Bento_bundle_path> <Deployment_name> <Config_JSON>
Use Python API
```python
from update import update_deployment
update_deployment(BENTO_BUNDLE_PATH, DEPLOYMENT_NAME, CONFIG_JSON)
from bentoctl_aws_ec2 import update
update(BENTO_BUNDLE_PATH, DEPLOYMENT_NAME, CONFIG_JSON)
```
### Get deployment's status and information
Expand All @@ -162,8 +255,8 @@ python describe.py <Deployment_name> <Config_JSON>
Use Python API
```python
from describe import describe_deployment
describe_deployment(DEPLOYMENT_NAME, CONFIG_JSON)
from bentoctl_aws_ec2 import describe
describe(DEPLOYMENT_NAME, CONFIG_JSON)
```
### Delete deployment
Expand All @@ -177,6 +270,6 @@ python delete.py <Deployment_name> <Config_JSON>
Use Python API
```python
from delete import delete_deployment
delete_deployment(DEPLOYMENT_NAME, CONFIG_JSON)
from bentoctl_aws_ec2 import delete
delete(DEPLOYMENT_NAME, CONFIG_JSON)
```
6 changes: 6 additions & 0 deletions bentoctl_aws_ec2/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
from .deploy import deploy
from .update import update
from .describe import describe
from .delete import delete

__all__ = ["deploy", "update", "describe", "delete"]
24 changes: 24 additions & 0 deletions bentoctl_aws_ec2/delete.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import boto3
from botocore.exceptions import ClientError

from .ec2 import generate_ec2_resource_names
from .utils import console


def delete(deployment_name, ec2_config):
_, stack_name, repo_name, _ = generate_ec2_resource_names(
deployment_name
)
cf_client = boto3.client("cloudformation", ec2_config["region"])
console.print(f"Delete CloudFormation Stack [b]{stack_name}[/b]")
cf_client.delete_stack(StackName=stack_name)

# delete ecr repository
ecr_client = boto3.client("ecr", ec2_config["region"])
try:
console.print(f"Delete ECR repo [b]{repo_name}[/b]")
ecr_client.delete_repository(repositoryName=repo_name, force=True)
except ClientError as e:
# raise error, if the repo can't be found
if e.response and e.response["Error"]["Code"] != "RepositoryNotFoundException":
raise e
62 changes: 21 additions & 41 deletions deploy.py → bentoctl_aws_ec2/deploy.py
View file
Original file line number Diff line number Diff line change
@@ -1,30 +1,27 @@
import os
import sys
import shutil
import argparse

from bentoml.saved_bundle import load_bento_service_metadata
from utils import (
get_configuration_value,

from .ec2 import (
generate_cloudformation_template_file,
generate_docker_image_tag,
generate_ec2_resource_names,
generate_user_data_script,
)
from .utils import (
build_docker_image,
console,
create_ecr_repository_if_not_exists,
get_ecr_login_info,
build_docker_image,
push_docker_image_to_repository,
run_shell_command,
console,
)
from ec2 import (
generate_docker_image_tag,
generate_ec2_resource_names,
generate_user_data_script,
generate_cloudformation_template_file,
)


def deploy(bento_bundle_path, deployment_name, config_json):
def deploy(bento_bundle_path, deployment_name, ec2_config):
bento_metadata = load_bento_service_metadata(bento_bundle_path)

ec2_config = get_configuration_value(config_json)
(
template_name,
stack_name,
Expand Down Expand Up @@ -70,7 +67,7 @@ def deploy(bento_bundle_path, deployment_name, config_json):
image_tag=ecr_tag,
region=ec2_config["region"],
env_vars=ec2_config.get("environment_variables", {}),
enable_gpus=ec2_config.get('enable_gpus', False),
enable_gpus=ec2_config.get("enable_gpus", False),
)

file_path = generate_cloudformation_template_file(
Expand All @@ -83,13 +80,17 @@ def deploy(bento_bundle_path, deployment_name, config_json):
autoscaling_min_size=ec2_config["ec2_auto_scale"]["min_size"],
autoscaling_desired_capacity=ec2_config["ec2_auto_scale"]["desired_capacity"],
autoscaling_max_size=ec2_config["ec2_auto_scale"]["max_size"],
health_check_path=ec2_config["elb"]["health_check_path"],
health_check_port=ec2_config["elb"]["health_check_port"],
health_check_interval_seconds=ec2_config["elb"][
health_check_path=ec2_config["elastic_load_balancing"]["health_check_path"],
health_check_port=ec2_config["elastic_load_balancing"]["health_check_port"],
health_check_interval_seconds=ec2_config["elastic_load_balancing"][
"health_check_interval_seconds"
],
health_check_timeout_seconds=ec2_config["elb"]["health_check_timeout_seconds"],
healthy_threshold_count=ec2_config["elb"]["healthy_threshold_count"],
health_check_timeout_seconds=ec2_config["elastic_load_balancing"][
"health_check_timeout_seconds"
],
healthy_threshold_count=ec2_config["elastic_load_balancing"][
"healthy_threshold_count"
],
)
copied_env = os.environ.copy()
copied_env["AWS_DEFAULT_REGION"] = ec2_config["region"]
Expand Down Expand Up @@ -128,24 +129,3 @@ def deploy(bento_bundle_path, deployment_name, config_json):
cwd=project_path,
env=copied_env,
)


if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Deploy the bentoml bundle on EC2",
epilog="Check out https://github.com/bentoml/aws-ec2-deploy#readme to know more",
)
parser.add_argument("bento_bundle_path", help="Path to bentoml bundle")
parser.add_argument(
"deployment_name", help="The name you want to use for your deployment"
)
parser.add_argument(
"config_json",
help="(optional) The config file for your deployment",
default=os.path.join(os.getcwd(), "ec2_config.json"),
nargs="?",
)
args = parser.parse_args()

deploy(args.bento_bundle_path, args.deployment_name, args.config_json)
console.print("[bold green]Deployment Complete!")
Loading

0 comments on commit e8a2081

Please sign in to comment.