Using AWS CDK Python to create S3 bucket
-
Python - pre-installed with Ubuntu
-
PIP
$ apt install python3-pip
- Python virtual environment
$ apt install python3-virtualenv
-
Install WS CDK
$ npm install -g aws-cdk
$ cdk --version
$ cdk --help
-
- Configure CDK default region in the
~/.aws/configfile
- Configure CDK default region in the
[default]
region=us-west-2
- Configure CDK credential
~/.aws/credentialfile. The IAM user credential to be used for deploying to AWS. Also see "Set Policy" below, if the user does not belong to the Administrator group.
[default]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
-
This is a one time setup for the AWS Account; it's called CDK Bootstrap;
-
Use the AWS S3 console to check whether the CDK Bootstrap has been activated, by looking for the following S3 bucket:
cdk-<UNIQUE-ID>-assets-<ACCOUNT-NUMBER>-<REGION>
- Otherwise, run the CDK Bootstrap, as follow:
// Get acct-number from AWS console or
$ aws sts get-caller-identity
// Get the default region for the profile
$ aws configure get region
$ cdk bootstrap aws://ACCOUNT-NUMBER/REGION
- [Skip Create Project] This repo contains artifacts generated by the following codes:
$ cd ~/projects
$ mkdir aws-cdk-py-s3-01
$ cd aws-cdk-py-s3-01
$ cdk init sample-app --language python
- Create python virtual environment
$ virtualenv .venv
$ source .venv/bin/activate
(.venv) $ pip install -r requirements.txt
- Set Policy; the following policies are needed minimally to
cdk deploy, and they are needed for CloudFormation & S3.- Unless, the IAM user (configured above in the
~/.aws/credentialandaws_access_key_idkey) belongs to theAdministratorsgroup.
- Unless, the IAM user (configured above in the
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"cloudformation:DescribeStacks",
"cloudformation:CreateChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:DescribeStackEvents",
"cloudformation:DeleteChangeSet",
"cloudformation:DeleteStack",
"cloudformation:GetTemplate",
"s3:CreateBucket"
],
"Resource": "*"
}
]
}
- Build
(.venv) $ cdk synth
- Deploy
(.venv) $ cdk deploy
# to speed up the deployment time
(.venv) $ cdk deploy --hotswap
- Install testing packages
(.venv) $ cd ~/projects/aws-cdk-py-s3-01
(.venv) $ pip install -r requirements-dev.txt
-
The test file:
./tests/unit/test_aws_cdk_py_s3_01_stack.py -
Test cases validate that:
- the S3 bucket is created
- the bucket is setup with
removal_policy = RemovalPolicy.DESTROY - the bucket is setup with
auto_delete_objects = True - the bucket is setup with
block_public_access=s3.BlockPublicAccess.BLOCK_ALL
-
Test Results as of 2022, July 2
(.venv) $ cd ~/projects/aws-cdk-py-s3-01
(.venv) $ pytest
========================================================= test session starts =========================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/<user>/projects/aws-cdk-py-s3-01
collected 4 items
tests/unit/test_aws_cdk_py_s3_01_stack.py .... [100%]
========================================================== 4 passed in 6.21s ==========================================================
(.venv) $
- Note: during cleanup, the default mechanism is to not delete the
S3 bucket (
removalPolicy: cdk.RemovalPolicy.RETAIN); only delete other services created during deployment, i.e., the cloudformation. However, for this tutorial, the S3 bucket has been setup to delete the S3 bucket during cleanup, as defined in the./aws-cdk-py-s3-01/aws_cdk_py_s3_01_stack.py:
# create bucket: no encryption, auto delete objects in the bucket, and
# the bucket during dismantling
bucket = s3.Bucket(self, "MyCdkSample01Bucket-20220624-dzong",
removal_policy = RemovalPolicy.DESTROY,
auto_delete_objects = True,
block_public_access=s3.BlockPublicAccess.BLOCK_ALL)
- Cleanup all artifacts created by CDK
(.venv) $ cdk destroy
Output:
✨ Synthesis time: 10.26s
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
IAM Statement Changes
┌───┬───────────────────────────────────┬────────┬───────────────────────────────────┬───────────────────────────────────┬───────────┐
│ │ Resource │ Effect │ Action │ Principal │ Condition │
├───┼───────────────────────────────────┼────────┼───────────────────────────────────┼───────────────────────────────────┼───────────┤
│ + │ ${Custom::S3AutoDeleteObjectsCust │ Allow │ sts:AssumeRole │ Service:lambda.amazonaws.com │ │
│ │ omResourceProvider/Role.Arn} │ │ │ │ │
├───┼───────────────────────────────────┼────────┼───────────────────────────────────┼───────────────────────────────────┼───────────┤
│ + │ ${MyCdkSample01Bucket-20220624-dz │ Allow │ s3:DeleteObject* │ AWS:${Custom::S3AutoDeleteObjects │ │
│ │ ong.Arn} │ │ s3:GetBucket* │ CustomResourceProvider/Role.Arn} │ │
│ │ ${MyCdkSample01Bucket-20220624-dz │ │ s3:List* │ │ │
│ │ ong.Arn}/* │ │ │ │ │
└───┴───────────────────────────────────┴────────┴───────────────────────────────────┴───────────────────────────────────┴───────────┘
IAM Policy Changes
┌───┬───────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────┐
│ │ Resource │ Managed Policy ARN │
├───┼───────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┤
│ + │ ${Custom::S3AutoDeleteObjectsCustomResourceProvider/Role} │ {"Fn::Sub":"arn:${AWS::Partition}:iam::aws:policy/service-role │
│ │ │ /AWSLambdaBasicExecutionRole"} │
└───┴───────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)
Do you wish to deploy these changes (y/n)? y
aws-cdk-py-s3-01: deploying...
-
Additional cleanup not done by the the
cdk destroy- CloudWatch Log group; go to AWS Console "CloudWatch > Logs > Log groups"
and delete two log groups:
/aws/lambda/<stack-name>-CustomCDKBucketDeployment<unique-id>/aws/lambda/<stack-name>-CustomS3AutoDeleteObjectsCustomResourcePr-<unique-id>
- CloudWatch Log group; go to AWS Console "CloudWatch > Logs > Log groups"
and delete two log groups:
-
Exit virtualenv
(.venv) $ deactivate
$
cdk lslist all stacks in the appcdk synthemits the synthesized CloudFormation templatecdk deploydeploy this stack to your default AWS account/regioncdk diffcompare deployed stack with current statecdk docsopen CDK documentation