Integration & Automation

Updating your AWS Elastic Disaster Recovery settings at scale with the DRS Settings Tool

Tim Hall — Tue, 14 Jan 2025 21:54:55 +0000

Deploying AWS Elastic Disaster Recovery at scale provides robust protection for your infrastructure (for more details, see Deploying AWS Elastic Disaster Recovery at scale with AWS Systems Manager). As your deployment grows, customizing your Elastic Disaster Recovery source server settings to align with each server’s specific needs and requirements becomes essential. These settings include replication settings, DRS launch settings, and EC2 launch template settings. Although Elastic Disaster Recovery offers default configurations that can be applied to all newly added servers, tailoring settings for a diverse range of servers provides optimal performance and reliability.

To simplify this process and avoid navigating through multiple sections of the AWS Management Console, the DRS Settings Tool is an invaluable resource for updating settings across your infrastructure efficiently. With the DRS Settings Tool, you can accomplish the following:

Generate a CSV file: Export all your source server settings into a CSV file.
Edit with ease: Make the necessary updates to your settings directly within the CSV file.
Push changes seamlessly: Apply the updated settings to your Elastic Disaster Recovery source servers effortlessly.

This streamlined approach significantly reduces the time and effort required for configuration. The DRS Settings Tool is not limited to one-time use; you can use it whenever you need to update your Elastic Disaster Recovery source server settings, making it a continuous asset for maintaining and optimizing your disaster recovery strategy.

With the DRS Settings Tool, you can make sure that your Elastic Disaster Recovery setup remains tailored to your organization’s evolving needs, while saving valuable time and minimizing complexity.

In this post, we show you how to setup and use the DRS Settings Tool to update all your Elastic Disaster Recovery source server settings in bulk.

Solution Overview

We walk through the following steps to implement the solution:

Create an AWS Identity and Access Management (IAM) policy and user.
Update your credentials and config file.
Install the necessary components to run the DRS Settings Tool (Python, pip, and Boto3).
Download and start using the DRS Settings Tool.

Prerequisites

For this walkthrough, you need the following:

An AWS account.
Active source servers in Elastic Disaster Recovery.
The ability to install Python, pip, and Boto3
Network connectivity to the AWS endpoints listed in the DRS Settings Tool readme file.
Permissions to create an IAM user and policy for usage of the DRS Settings Tool.
Existing AWS credentials and config file. A straightforward way to set this up is to install the AWS Command Line Interface (AWS CLI). For instructions, see Installing, updating, and uninstalling the AWS CLI.

Create the DRS Settings Tool IAM policy and user

To get started, you create the IAM policy and user that will be used with the DRS Settings Tool. In this example, we use the IAM permissions listed for single account use because we don’t have any extended source servers. If you have extended source servers, you can use the permissions noted in the readme for a user in the staging account (where the replication resources are created) and a user in the target account (where your Elastic Disaster Recovery source servers are recovered to).

Complete the following

On the IAM console, choose Policies in the navigation pane. Choose Create policy.
In the Policy editor section, choose JSON.

Enter the following IAM permissions and choose Next:

{
 "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "drs:UpdateReplicationConfiguration",
                "drs:UpdateLaunchConfiguration",
                "drs:GetLaunchConfiguration",
                "drs:GetReplicationConfiguration",
                "drs:DescribeSourceServers",
                "ec2:DescribeInstances",
                "ec2:DescribeLaunchTemplateVersions",
                "ec2:DescribeSecurityGroups",
                "ec2:CreateLaunchTemplateVersion",
                "ec2:DescribeImages",
                "ec2:ModifyLaunchTemplate",
                "ec2:DescribeSubnets",
                "ec2:DescribeKeyPairs",
                "ec2:CreateSecurityGroup",
                "ec2:DescribeInstanceTypeOfferings",
                "ec2:CreateTags",
                "iam:GetInstanceProfile",
                "kms:DescribeKey",
                "kms:CreateGrant",
                "ec2:GetEbsDefaultKmsKeyId"
            ],
            "Resource": "*"
        }
    ]
}

Enter your policy details and choose Create policy.

Next, you create the user to attach the policy to.

Choose Users in the navigation pane.
Choose Create user.
Enter your desired user name, and choose Next.
In the Permissions options section, choose Attach policies directly.
Search for and choose your created policy.
Choose Create user.

Make sure you download your Access Key and Secret Key.

Choose the user on the Users
On the Security credentials tab, choose Create access key.
Choose Other, Next, Create access key, and Download .csv file.

Update your credentials and config file

With the CSV file you downloaded, you can now update your credentials and config file, which are required to run the DRS Settings Tool. Complete the following steps:

Navigate to your credentials and config files which are located at the following location:
- Windows: C:\Users\<username>\.aws
- Linux: ~/.aws/
Open the credentials file and add an entry like the following (update with your Access Key and Secret Key), then save the file:
```
[<Target-Account-ID>]
aws_access_key_id = <Access-Key>
aws_secret_access_key = <Secret-Key>
```
In this example, we use a single account. If you’re using multiple accounts, you need to add an entry for each account ID with the appropriate AWS user’s access and secret key.
Open the config file and add an entry like the following (update with your region and output format such as “us-east-1” and “json”), then save the file:
```
[profile <Target-Account-ID>]
region = <region>
output = <output>
```
In this example, we use a single account. If you’re using multiple accounts, you need to add an entry for each account ID with the appropriate AWS Region and output.

Install Python, pip, and boto3

Now that you have created the IAM policy and user and updated and the credentials/config files, you can download the necessary components to run the DRS Settings Tool. For this walkthrough, we use a Windows-based machine and download the components for Windows.

For this post, we use the download link for Python to install it. You can use your preferred method that works best for your operating system.
Run the installer and choose your desired method of installation.

When the install is complete, you can move on to installing pip.

Download the get-pip.py script.
Open a command prompt, cd to the path containing the get-pip.py script and run the following command:
py get-pip.py

Now that pip is installed, you can move on to installing Boto3.

In the command prompt, cd to the path of pip.exe and run the following command:
pip.exe install boto3

Download and use the DRS Settings Tool

Now that you have all the components configured and installed, you’re ready to download the DRS Settings Tool and run it.

Navigate to the GitHub repo for the DRS Settings Tool, and choose Code and Download ZIP.
After downloading, open a command prompt, cd to the path of the DRS Settings Tool and run the following command:
get_settings.py
When the script is complete, you should see three files get created in the DRS-Settings-Tool-main directory:
- csv – The main file that you will make your edits in.
- DRS_Settings-DO-NOT-EDIT.csv – A copy of the original DRS_Settings.csv file. We use this as a comparison file after changes in DRS_Settings.csv, to limit AWS API calls by only making updates to what has changed.
- DRS-Update-Tool.txt – A log for troubleshooting purposes.
Now that you have generated the CSV files, you can open it in your preferred CSV file editor and begin editing your settings. Available settings and their accepted values can be found in the readme file.
When you finish making changes to the CSV file, you can save it and then run the update_settings.py script from the same directory.

You have now successfully run the DRS Settings Tool to update your servers in bulk. Make sure to re-run the get_settings.py script so the CSV files are regenerated with the latest settings.

Cleanup

To uninstall the components you downloaded and installed, complete the following steps in order.

To uninstall Boto3, navigate to the directory where pip is installed and run the following command:
- pip uninstall boto3
To uninstall pip, run the following command from command prompt:
- py -m pip uninstall pip
To uninstall Python in Windows, navigate to the Control Panel, choose Uninstall a program, choose the Python-related programs, and choose Uninstall.
To cleanup the DRS Settings Tool, delete the entire directory.

Conclusion

In this post, we explored how to use the DRS Settings Tool to manage your Elastic Disaster Recovery source server settings at scale. This tool can save you a lot of time during your initial deployment of Elastic Disaster Recovery, and also help manage settings later if needed.

Try out the tool and share your experiences and questions in the comments section.

About the Author

Tim Hall is a Senior Cloud Support Engineer at AWS. He is a Subject Matter Expert for the AWS Elastic Disaster Recovery Service and helps customers solve complex problems. In his free time, he enjoys competing in billiards and playing video games.

A practical guide to getting started with policy as code

Andrew Timpone — Mon, 09 Dec 2024 17:40:05 +0000

Modern enterprises have learned the value of automation in their end-to-end software development lifecycles (SDLCs). Automation mechanisms provide consistent and reliable methods to build and deploy code and their related environments without human intervention. This post highlights ways in which you can get started with policy as code (PaC). It’s used to enforce security configuration policies, service configuration standards, and complement automations you may already be using for code and environment development and deployment.

PaC is a software automation approach, similar to infrastructure as code (IaC), that aids the assessment of an organization’s configuration and validation compliance requirements through the use of software automation constructs. You can use a consistent and automated means of enforcing enterprise standards and best practices across your organization while minimizing manual processes. Pre-built and pre-validated software constructs are used to assess compliance factors in software code—both when code is initially written, and again when it’s deployed for use. The result is confidence that deployed assets are in fact compliant with policy rules.

Previous blog posts have more thoroughly described the four main types of controls (preventative, proactive, detective, and responsive) used to assess and react to compliancy matters. For more details, see Governance at scale: Enforce permissions and compliance by using policy as code and Cloud governance and compliance on AWS with policy as code.

The following diagram suggests where such controls are typically employed.

Choosing your PaC approach

The following is a recommended set of steps to help prepare you for choosing the right approach for PaC.

Assess suitable policies

Many organizations don’t have a well-documented set of compliance policies. Additionally, those that are documented might not be readily subject to codification. An initial assessment is required to assemble the list of target policies in a suitable inventory style. A smaller, initial set of PaC automation targets can then be assembled using the following criteria:

For those new to PaC, it’s recommended that policies be chosen that aren’t overly complex, and are easily codified. For example, “I want to make sure that Service X is not configured with property Y.”
Start small:
- Identify a few key areas where you want to either reduce misconfigurations or reduce your overall security risks.
- Map out impact vs. complexity of implementation. Start with putting together a list of known configuration issues in your accounts.
- Pick the highest risk areas to automate. Capture both the risk (critical, high, medium, low) according to your organization’s InfoSec or Security team as well as using third-party recommendations (such as NIST or CIS benchmarks). Capture the quantity of occurrences of each configuration finding per service. This provides you with a snapshot of both risk severity as well as misconfiguration occurrence. Prioritize the top items from each list and begin defining target policy rules (preventive, proactive).
- Be agile and iterate.
It’s common for enterprises to use a centralized, manual, human-driven process to author and assess complex, critical, or often-changing policies. For enterprises new to PaC, it’s therefore best to target less critical policies, such as those that are commonly assessed in a repetitive, undifferentiated manner across workloads. These compliance actions are more suitable for automation and for authoring by teams outside of the centralized compliance organization.

Determine the degree of shift-left

“Shift-left” is a well-known practice and philosophy designed to find and correct defects earlier in the SDLC, thereby reducing the number of compliance-related issues that leak into the production. When considering PaC, you should implement these constructs early in the software development phases where proactive controls can be used.

How far you shift left depends on technology and resources available across the organization. Ideally, PaC constructs should be considered at the point where IaC templates are initially created. If you use AWS CloudFormation, it’s recommended to start with AWS CloudFormation Guard running from the AWS Command Line Interface (AWS CLI), allowing you to define and run rules at the individual resource level as well as other related resources within the same CloudFormation template file. If you have more complex rules requiring logic evaluations that can’t be defined as CloudFormation Guard rules, or runtime parameter assessment is required, then AWS CloudFormation Hooks is recommended.

Code-defined AWS CDK option

Consider migrating from a pure template or nested-stack IaC approach and instead using a code-defined application and resources approach. One tool that can help with this approach is the migration capabilities of the AWS Cloud Development Kit (AWS CDK). This allows you to use your existing CloudFormation templates and AWS tools handle the heavy lifting to migrate your IaC to the AWS CDK. This initial step in migrating from AWS CloudFormation to the AWS CDK provides a simple translation of IaC to AWS CDK constructs of various levels:

Level 1 – Level 1 constructs are also known as CloudFormation resources, and are the lowest-level construct, and offer no abstraction. Each Level 1 construct maps directly to a single CloudFormation resource.
Level 2 – Level 2 constructs extend Level 1 constructs and allow encapsulation of rules, which allows you to specify acceptable values enables parameter validation. For example, if you want to block public access for Amazon Simple Storage Service (Amazon S3) buckets, your organization’s construct for Amazon S3 can force that required implementation by defaulting the attributes as needed and not allowing override.
Level 3 – Level 3 constructs go a step further, allowing you to bundle architecture patterns together. This level not only encapsulates rules, but can drive best practices. You can also use AWS CDK aspects to validate your configurations, along with the AWS CDK Guard Validator, which allows you to validate both AWS managed CloudFormation Guard rules as well as customer managed CloudFormation Guard rules at the time of rule authoring.

Align PaC to your organization

A well-structured set of accounts will be comprised of organizational units (OUs) based on security compliance controls and needs (see AWS Organizations for more details). PaC constructs need to follow the same structure. Deployment of controls (whether using CloudFormation Hooks or CloudFormation Guard rules) should be based on where in the organization your workload is being deployed. For example, your enterprise policy may require the prohibition of public S3 buckets. However, there may be a unique business need to do so. In that case, you want this workload isolated in an OU that holds exceptions to the rules, and you can then exclude your Amazon S3 public policy from being evaluated against your workload.

Implement PaC

With the preceding preparation and consideration steps complete, you can now focus on the actual implementation of PaC. This section outlines a set of simple steps to get you started:

Using the preceding preparation steps, first select and document the targeted policies and rules. Include test conditions and any rule references back to the organizational security policies:
1. See if the rule already exists:
  1. Is there an existing managed proactive control that matches your rule?
  2. Is there a rule within the CloudFormation Guard Registry?
2. Author the rule (either in CloudFormation Guard or CloudFormation Hooks).
3. Write pass/fail unit test cases.
4. Run the unit test cases.
5. Write pass/fail integration tests with sample CloudFormation templates.
6. Run the integration test cases.
7. Commit the rule and tests to a centralized organization repository

The next step is to define a repository to capture all of your PaC rules and constructs.

For local development rule validation, complete the following steps:
1. Download and install the CloudFormation Guard CLI (for more information, see Setting up AWS CloudFormation Guard or the AWS CloudFormation Guard User Guide).
2. Clone the rule repository to your local development environment.
3. Create or change the IaC resources (either AWS CloudFormation or AWS CDK).
  1. If AWS CDK, run cdk synth to generate the output CloudFormation template.
4. Run the CloudFormation Guard CLI to validate the CloudFormation template (for more details, see AWS CloudFormation Guard CLI parameters and command reference):
```
cfn-guard validate --rules <<rule file(s) or directory>> --data <<CFn template path>>
```
For a pre-commit hook or a continuous integration and delivery (CI/CD) build process, complete the following steps:
1. Download and set up the CloudFormation Guard CLI.
2. Clone the rule repository.
3. Clone the application repository with the IaC template or code.
4. Run the CloudFormation Guard CLI to validate the CloudFormation template.
5. Examine the return code to determine if the output contains any failures.
For CloudFormation Hooks, you will need a build pipeline as well as the use of CloudFormation StackSets to deploy the hooks to the desired OUs or accounts. The build process will need to download and install the CloudFormation CLI. For instructions to build and deploy CloudFormation Hooks using StackSets, see Deploy CloudFormation Hooks to an Organization with service-managed StackSets.

After the chosen process has been prototyped on a few small projects, you begin to scale utilization to more numerous and more complex projects. As you become more adept and confident with PaC, you can pull in more rules to construct and validate.

Troubleshooting

The following common limitations could result in issues as you implement PaC:

Conditions preventing shift-left – Be aware of conditions where you may not be able to implement PaC controls as early as the code author stage. If your code references configuration properties that require deploy time or resource provisioning evaluation (such as a value pulled from Parameter Store, a capability of AWS Systems Manager, or AWS Secrets Manager), then any CloudFormation template will fail because it will be unable to resolve such parameter values until runtime. When creating a new resource, be aware that values such as the resource identifier are not assigned until after the resource is created. In these cases, you may have to move PaC evaluations into a runtime phase using CloudFormation Hooks, or even using AWS Config rules as a post-runtime detective control.
Nested CloudFormation templates – Nested templates are evaluated during the CloudFormation template deploy stage after a fully resolved template is generated. CloudFormation Guard only checks the resource definitions directly within the CloudFormation template being evaluated, and any nested templates are not included in the evaluation. These same restrictions to CloudFormation Guard also apply if you’re using this tool to use a repository pre-commit hook or during a build or test step. It is advisable to use PaC on non-nested stacks, or through re-factoring of nested stacks to avoid this runtime issue.
Using proactive controls in the deploy phase – If evaluation of runtime or input parameters is required, CloudFormation Hooks or AWS Config managed hooks may be the better choice. With CloudFormation Hooks, the hook is evaluated at the time of deployment. CloudFormation template parameters are resolved at time of deployment, allowing them to be properly validated.
Defaulting to detective controls – If parameterization and nesting still yield issues, then detective controls should be used. When a resource is deployed or changed, this change is observed by AWS Config, which can evaluate customer managed and AWS managed rule sets, which can create findings if any rules violations are detected.

Balancing controls across the SDLC timeline

A best practice is to make sure you have coverage throughout the lifecycle of your AWS resources. Pull as many of the controls as far left as possible, and balance these with additional controls purposefully placed later in the development timeline (such as detective controls or AWS Config rules). This allows PaC coverage in boundary cases or out-of-band events such as emergency changes to a resource through the AWS Management Console or AWS CLI. Policy errors introduced with such real-time changes are still caught by downstream detective processes, making sure that misconfigured services are highlighted.

Standard software best practices

PaC constructs are software elements. Standard software best practices should be used for the development and maintenance of PaC code constructs, including version control, comprehensive testing, usage of a central, democratized construct library, and standardized software lifecycle maintenance philosophies and toolsets.

PaC governance

Change management governance is required of PaC constructs, just as it is required for other software code assets. As more and more teams use shared PaC constructs, they will develop dependencies on those constructs, and shared change management mechanisms will be required. Multiple teams may want to add or modify rules, and changes will need to be validated per line of dependency. As you initially adopt PaC, begin engaging in thought leadership conversations around the typical lifecycle use cases and change management considerations required.

Lastly, establish naming and verbiage conventions for your rule code and files, rule method and definitions, as well as the rule alert and error messages. See the appendix at the end of this post for examples.

Conclusion

In this post, we detailed the concepts, processes, and steps to get started with PaC and adopt this into your software development lifecycle. PaC can improve your overall security posture, improve consistency of service usage across your organization, and reduce rework or workloads deployed to your AWS accounts. To get started, see the references listed in the appendix, and determine if you want to get started with using the AWS CDK, CloudFormation Guard, or CloudFormation Hooks (or a combination of each). Check out the sample code repositories and already published rules, then begin experimenting.

Appendix

References

The following are additional references you can use as you begin your PaC journey. These include references for background information as well as sample code and other how-to guides:

Also, check out the following new and noteworthy posts:

Implement PaC constructs

The following are services and tools within AWS to implement the various PaC constructs:

Preventive control:
- Service Control Policies (SCPs) – SCPs are a type of organization policy that you can use to manage permissions in your organization. SCPs offer central control over the maximum available permissions for all accounts in your organization. SCPs help you make sure your accounts stay within your organization’s access control guidelines.
Proactive controls:
- AWS CloudFormation Hooks – Hooks proactively inspect the configuration of your AWS resources before provisioning. If non-compliant resources are found, AWS CloudFormation returns a failure status and either fails the operation or provides a warning and allows the operation to continue based on the hook failure mode. You can use pre-built hooks or build your own hooks using the AWS CloudFormation CLI.
- AWS Config rules – Evaluated as managed CloudFormation hooks, AWS Config rules evaluate the configuration settings of your AWS resources as they are being deployed through AWS CloudFormation.
- AWS CloudFormation Guard – AWS CloudFormation Guard is an open source, general-purpose, PaC evaluation tool. The CloudFormation Guard CLI provides a simple-to-use, yet powerful and expressive, declarative DSL that you can use to express PaC. In addition, you can use CLI commands to validate structured hierarchical JSON or YAML data against those rules. For additional details, see the GitHub repo.
- AWS CDK aspects – Aspects are a way to apply an operation to all constructs in a given scope. The aspect could modify the constructs, such as by adding tags. Or it could verify something about the state of the constructs, such as making sure that all buckets are encrypted.
- AWS CDK plugins – Plugins extend functionality of AWS CDK by adding new commands or modifying existing ones. Both AWS CDK aspects and plugins can be used for policy enforcement. The main difference is that aspects are invoked during synthesis and can modify resource configuration, whereas plugins are run after synthesis and can’t affect the generated template. An example of this is the CloudFormation Guard Validator plugin.
- AWS CloudFormation Guard rules – In CloudFormation Guard, rules are PaC rules. You write rules in the CloudFormation Guard DSL that you can validate your JSON- or YAML-formatted data against. Rules are made up of clauses.
- Open Policy Agent – The Open Policy Agent (OPA) is an open source, general-purpose policy engine that unifies policy evaluation across the stack. OPA provides a high-level declarative language that lets you specify PaC and simple APIs to offload policy decision-making from your software. You can use OPA to evaluate policies in microservices, Kubernetes, CI/CD pipelines, API gateways, and more. You can use OPA as the rules definition within AWS Config rules (see Using OPA to create AWS Config rules) as well as a way to validate your configurations before deployment (see Cloud governance and compliance on AWS with policy as code).
Detective controls:
- AWS Config rules – Use AWS Config to evaluate the configuration settings of your AWS resources. You do this by creating AWS Config rules, which represent your ideal configuration settings. AWS Config provides customizable, predefined rules called managed rules to help you get started. While AWS Config continuously tracks the configuration changes that occur among your resources, it checks whether these changes don’t comply with the conditions in your rules.
Responsive controls:
- The basic procedure used to establish this process is to define a workflow on what is needed to do based on the specific findings as well as understand the risk and priority of the findings. By defining the workflow and process, you can begin the steps to script the remediation of the findings, ultimately getting to the point where you can automate this remediation process. Not everything should be automated, but it’s a best practice to automate the well-understood configuration issues. Also, this process, whether fully automated or not, should continue to follow your organizational standards and processes for notifications as well as an incident ticketing process. This will allow your security and audit teams to be able to track findings and incidents, even if they are automatically remediated. Consider the following services:
  - AWS Security Hub
  - AWS Systems Manager
  - Amazon EventBridge
  - Amazon Simple Notification Service (Amazon SNS)

Sample error message

The following is a sample error message:

Rule file name: s3_prohibit_public_access.guard
Named rule: S3_PROHIBIT_PUBLIC_READ
Validation messages:
- Violation: SEC 1.1 No publicly accessible storage. All storage of files and data should not be publicly accessible. Access to these files should be controlled through application permissions.
- Fix: Set S3 Bucket PublicAccessBlockConfiguration properties for BlockPublicAcls, BlockPublicPolicy, IgnorePublicAcls, RestrictPublicBuckets parameters to true.

Sample security policy mapping

The following is an example of a security policy to AWS service mapping table.

Security Policy ID	Security Policy Title	Security Policy Definition	AWS Service	AWS Service Configuration Settings
Sec 1.1	No publicly accessible storage	All storage of files and data should not be publicly accessible. Access to these files should be controlled through application permissions.	Amazon S3	`PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true`
Sec 1.2	Storage should be encrypted	All storage (file, block, object) should be encrypted with a minimum of AES256.	Amazon S3	`BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: AES256ORBucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: "aws:kms"`
Sec 1.2	Storage should be encrypted		Amazon EBS	`Encrypted: true KmsKeyId: exists`

About the Authors

Andrew Timpone is a Senior Solutions Architect at AWS. With over 5 years of experience in cloud architecture, Andrew helps large enterprise customers solve their business problems using AWS. Andrew has over 25 years of IT experience with expertise in enterprise integration patterns. Andrew is married with three children and resides just south of Cleveland, OH, where he enjoys bicycle riding, archery, and vegetable gardening.

Dean Banwart has served as an AWS Sr. Customer Solutions Manager for over 5 years and currently works within the Enterprise Auto/Manufacturing vertical. Dean has over 45 years of professional engineering experience with expertise in the areas of telecommunications, systems engineering, manufacturing, and encryption technology. Dean is married with four children and resides in the Northwest suburbs of Chicago, where he enjoys outdoor sports, music, and restoration of vintage electronics and equipment.

Automate security scans on Amazon EKS with Kubescape, AWS CodeBuild, and AWS CodePipeline

Aniket Dekate — Tue, 29 Oct 2024 15:00:35 +0000

As organizations increasingly adopt Amazon Elastic Kubernetes Service (Amazon EKS) to manage their containerized applications, implementing robust security measures and maintaining compliance become critical. The scalable and flexible nature of Amazon EKS has made it a popular choice for businesses seeking to streamline their application deployment and management processes. However, with this adoption comes the need to prioritize security and compliance, protecting sensitive data, infrastructure, and adhering to industry regulations and standards. Security and compliance should be top priorities for any organization, regardless of size or industry. In today’s digital landscape, where security challenges continuously evolve and regulatory requirements become more stringent, robust security measures and compliance are important to address security issues.

Security frameworks provide organizations with a structured approach to meet compliance requirements. By aligning their security practices with these industry-recognized standards, businesses can address potential issues and demonstrate their commitment to protecting sensitive information. Although implementing security measures and maintaining compliance is vital, doing so in a scalable and efficient manner is equally important. As businesses grow and their infrastructure expands, manually managing security checks, scans, and compliance audits can become increasingly complex. Automation streamlines security and compliance processes, enabling consistent and reliable checks across the entire Amazon EKS infrastructure.

In this post, we explore how to automate security scans and compliance checks on Amazon EKS using Kubescape, AWS CodeBuild, and AWS CodePipeline. With these tools, organizations can establish a seamless and scalable security and compliance workflow, continuously monitoring EKS clusters, addressing potential issues, and aligning with industry standards and regulations.

Understanding compliance requirements

For security professionals, achieving expertise in Amazon EKS compliance involves implementing robust identity and access management, network security measures, data protection mechanisms, and comprehensive monitoring and logging. By using automation tools and establishing a continuous compliance program, organizations can streamline compliance efforts, maintain audit readiness, and proactively address emerging issues and regulatory changes. Ultimately, understanding and adhering to Amazon EKS compliance requirements is essential for safeguarding critical infrastructure, validating data privacy, and maintaining a strong security posture in the cloud.

Security frameworks based on NSA (National Security Agency) guidelines, MITRE, Center for Internet Security (CIS-EKS-1.2.0), and NIST provide organizations with a structured approach to meeting security and compliance needs. The NSA guidelines offer best practices for securing systems and networks, and the MITRE framework provides a comprehensive methodology for identifying and addressing challenges. The CIS-EKS-1.2.0 benchmark specifically addresses security best practices for EKS clusters, validating that organizations follow industry-recognized standards. Additionally, the NIST framework provides organizations with a comprehensive security approach that encompasses identifying potential security issues, implementing protective measures, detecting security events, responding to incidents in a timely manner, and establishing recovery processes to mitigate the impact of security issues.

AWS Shared Responsibility Model

The AWS Shared Responsibility Model outlines the division of responsibilities between AWS and its customers when it comes to security and compliance for services like Amazon EKS. This model is based on the principle of shared accountability, where AWS handles the security of the cloud infrastructure, while customers are responsible for securing their workloads and configurations within the cloud environment.

AWS takes care of protecting the underlying infrastructure that powers the services offered in the AWS Cloud. Specifically for Amazon EKS, AWS maintains the security of the Kubernetes control plane, including the control plane nodes and the etcd database. Amazon EKS is certified by multiple compliance programs for regulated and sensitive applications. The effectiveness of the security controls is regularly tested and verified by third-party auditors as part of the AWS compliance programs.

Customers are accountable for the security and compliance of the systems and services they configure and deploy on AWS. In the context of Amazon EKS, customers are responsible for securely deploying, configuring, and managing their workloads within their EKS cluster. This encompasses validating the security of worker nodes, network configurations, access controls, and implementing continuous monitoring and compliance checks. The Shared Responsibility Model emphasizes the collaborative nature of maintaining a secure and compliant environment. Although AWS provides a secure and compliant infrastructure, customers play a crucial role in securing their configurations and workloads running on top of that infrastructure. By fulfilling their respective responsibilities, AWS and its customers can work together to achieve a robust security and compliance posture for containerized applications running on Amazon EKS.

Understanding compliance frameworks from NSA, MITRE, and CIS

Let’s explore the compliance frameworks from NSA, MITRE, and CIS in more detail:

NSA (National Security Agency) – NSA is a US government agency focused on national security through signals intelligence and cybersecurity. It provides security guidance and cryptographic standards to protect sensitive government data and national security systems. NSA’s guidelines for secure container deployment provide a comprehensive framework for organizations to follow best practices and maintain the security of their containerized environments, including Amazon EKS.
MITRE – MITRE is a nonprofit organization that operates federally funded research and development centers (FFRDCs) to provide technical expertise in various fields, including security. It is known for creating the MITRE framework, a comprehensive knowledge base of adversary tactics and techniques used to improve issue detection strategies. MITRE collaborates with government, industry, and academia to address complex security challenges and advance cybersecurity innovation. Its work helps organizations enhance their ability to identify, prevent, and respond to cyber threats.
CIS (Center for Internet Security) – CIS is a non-profit organization dedicated to enhancing the cybersecurity readiness and response of public and private sector entities. The CIS Benchmarks are widely recognized as industry-leading best practices for secure configuration of various systems, including Kubernetes clusters. Implementing these benchmarks can significantly harden EKS clusters.

Importance of NSA, MITRE, and CIS for Amazon EKS

NSA, MITRE, and CIS offer the following benefits when used with Amazon EKS:

Compliance and regulatory requirements – Adhering to industry standards and best practices, such as those provided by NSA and CIS, is crucial for meeting regulatory requirements and validating data protection and security. Organizations operating in regulated industries or handling sensitive data can use these frameworks to demonstrate compliance and maintain a robust security posture.
Risk identification and mitigation – The MITRE framework empowers organizations to proactively identify and mitigate potential issues in their Amazon EKS environments. By understanding the tactics and techniques used by adversaries, security teams can implement effective countermeasures and stay ahead of evolving issues.
Hardening and secure configuration – Implementing CIS Benchmarks for secure configuration of EKS clusters is essential for reducing the issues and minimizing unintended access. These benchmarks provide detailed guidelines for configuring various components of the Kubernetes ecosystem, validating a hardened and secure deployment.
Continuous monitoring and improvement – Using the frameworks provided by NSA, MITRE, and CIS enables organizations to establish a continuous monitoring and improvement process for their Amazon EKS security posture. Regular assessments and adaptations to evolving issues make sure security measures remain effective and up to date.

Solution overview

As Kubernetes environments increase in complexity, it’s essential to have robust security measures in place to mitigate potential issues and unintended access. Kubescape, an open source tool, emerges as a solution for securing EKS clusters. It empowers security professionals to scan for misconfigurations, identify unintended access, and implement best practices for hardening their Amazon EKS deployments. With Kubescape, organizations can proactively address security concerns, validating the resilience and compliance of their cloud-based applications.

The following diagram illustrates the architecture for automating the security scanning process for EKS clusters to enhance security governance and operational insights.

The workflow starts with an end-user triggering CodePipeline, a continuous integration and delivery (CI/CD) service that manages the steps of the security scan. Upon initiation, CodeBuild is called to run a job that handles the execution of the scanning process. During the build phase, Kubescape, an open source security tool, is used to scan the EKS cluster. When the scan is complete, the results are automatically stored in an Amazon Simple Storage Service (Amazon S3) bucket. After the scan, organizations can use the results to understand their security posture, potentially stop deployments if critical issues are detected, and visualize the identified issues from a central location. Additionally, organizations can create issues in a tracking system like Jira to generate tickets for remediation, streamlining the process of addressing identified security concerns.

In the following sections, we provide sample code to automate security scans and compliance checks on your EKS clusters using Kubescape, CodeBuild, and CodePipeline.

Prerequisites

To follow along with this post, you should have an active EKS cluster. Make sure your EKS cluster is set up with the necessary AWS Identity and Access Management (IAM) access configurations.

Additionally, you should have an active CodePipeline workflow ready for your CI/CD workflows with following sample IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Resource": [
        "arn:aws:logs:{REGION}:{ACCOUNT_ID}:log-group:/aws/codebuild/{PROJECT_NAME}",
        "arn:aws:logs:{REGION}:{ACCOUNT_ID}:log-group:/aws/codebuild/{PROJECT_NAME}:*"
      ],
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ]
    },
    {
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::{BUCKET_NAME}*"
      ],
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:GetBucketAcl",
        "s3:GetBucketLocation"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "codebuild:CreateReportGroup",
        "codebuild:CreateReport",
        "codebuild:UpdateReport",
        "codebuild:BatchPutTestCases",
        "codebuild:BatchPutCodeCoverages"
      ],
      "Resource": [
        "arn:aws:codebuild:{REGION}:{ACCOUNT_ID}:report-group/{PROJECT_NAME}-*"
      ]
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:GetBucketVersioning"
      ],
      "Resource": "arn:aws:s3:::{BUCKET_NAME}",
      "Effect": "Allow"
    },
    {
      "Action": [
        "codebuild:BatchGetBuilds",
        "codebuild:StartBuild"
      ],
      "Resource": "arn:aws:codebuild:{REGION}:{ACCOUNT_ID}:report-group/{PROJECT_NAME}",
      "Effect": "Allow"
    },
    {
      "Action": [
        "eks:DescribeCluster",
        "eks:DescribeNodegroup"
      ],
      "Resource": "arn:aws:eks:{REGION}:{ACCOUNT_ID}:cluster/{CLUSTER_NAME}",
      "Effect": "Allow"
    }
  ]
}

Automate security scans and compliance checks

The following is a sample buildspec file that you can use to efficiently set up and install Kubescape within your CodeBuild project:

version: 0.2
phases:
  install:
    runtime-versions:
      python: 3.8
    commands:
      - echo "====== Kubescape Installation ======"
      - curl -s https://raw.githubusercontent.com/kubescape/kubescape/master/install.sh | /bin/bash
  build:
    commands:
      - kubescape list frameworks
      - echo "====== Kubescape Scan - CIS Benchmark Framework ======"
      - kubescape scan framework cis-eks-t1.2.0
      - echo -e "\n\n====== Kubescape Scan - NSA Framework ======\n"
      - kubescape scan framework nsa
      - echo -e "\n\n====== Kubescape Scan - MITRE Framework ======\n"
      - kubescape scan framework mitre
      - echo -e "\n\n====== Kubescape Scan - Specific Namespaces development ======\n"
      - kubescape scan --include-namespaces development
  post_build:
    commands:
artifacts:
  files:
      - '**/*'

The following command instructs Kubescape to perform a security scan on an EKS cluster using the CIS Kubernetes Benchmark version 1.2.0. This benchmark provides a set of best practices and security guidelines specifically tailored for Kubernetes, validating that the cluster is configured securely according to industry standards.

$ kubescape scan framework cis-eks-t1.2.0

The following is an example of the expected output:

──────────────────────────────────────────────────
Framework scanned: cis-eks-t1.2.0

┌─────────────────┬────┐
│        Controls │ 53 │
│          Passed │ 4  │
│          Failed │ 24 │
│ Action Required │ 25 │
└─────────────────┴────┘

Failed resources by severity:

┌──────────┬────┐
│ Critical │ 0  │
│     High │ 23 │
│   Medium │ 22 │
│      Low │ 4  │
└──────────┴────┘

The following command instructs Kubescape to perform a security scan on your Kubernetes cluster using the NSA framework. This framework provides security guidelines and best practices specifically recommended by the NSA for securing Kubernetes clusters, focusing on hardening configurations to prevent potential unintended access.

$ kubescape scan framework nsa

The following is an example of the expected output:

──────────────────────────────────────────────────
Framework scanned: NSA
┌─────────────────┬────┐
│        Controls │ 25 │
│          Passed │ 9  │
│          Failed │ 11 │
│ Action Required │ 5  │
└─────────────────┴────┘
Failed resources by severity:
┌──────────┬────┐
│ Critical │ 0  │
│     High │ 3  │
│   Medium │ 14 │
│      Low │ 2  │
└──────────┴────┘

The following command directs Kubescape to perform a security scan on your Kubernetes cluster using the MITRE framework. This framework is based on the MITRE ATT&CK, which categorizes tactics and techniques used by adversaries during security events. The scan helps identify potential unintended access and security weaknesses in your cluster by comparing it against known issues patterns and techniques, allowing for improved security posture.

$ kubescape scan framework mitre

The following is an example of the expected output:

──────────────────────────────────────────────────
Framework scanned: MITRE
┌─────────────────┬────┐
│        Controls │ 27 │
│          Passed │ 11 │
│          Failed │ 11 │
│ Action Required │ 5  │
└─────────────────┴────┘
Failed resources by severity:
┌──────────┬────┐
│ Critical │ 0  │
│     High │ 1  │
│   Medium │ 10 │
│      Low │ 3  │
└──────────┴────┘

More security best practices

Beyond the basics, several good practices can further improve the security of your Amazon EKS environment. These include implementing IAM roles for service accounts (IRSA) for least privilege access control, integrating AWS Secrets Manager with AWS Secrets and Configuration Provider (ASCP) for secure handling of sensitive data, enabling Amazon GuardDuty for proactive threat detection, enforcing Pod Security Standards (PSS), and using Kyverno policies to maintain secure and consistent configurations across the cluster.

One of the fundamental principles of securing your EKS cluster is implementing least privilege access. By using role-based access control (RBAC), you can limit access to resources and namespaces within your cluster, validating that only authorized users and services have the necessary permissions. This approach minimizes the issues of unintended access and potential misuse of resources.

Implementing IRSA is a powerful technique that simplifies authentication and authorization within your EKS cluster. IRSA allows you to associate an IAM role with a Kubernetes service account, granting fine-grained permissions to your workloads. This approach reduces the need for sharing credentials across multiple pods, reducing the issues of credential exposure. Enabling IRSA is straightforward and provides a secure and scalable authentication mechanism for your EKS cluster.

For more information on fine-grained IAM roles for service accounts, see Introducing fine-grained IAM roles for service accounts.

Using Secrets Manager in combination with ASCP is another critical step in securing your EKS cluster. Secrets Manager provides a secure and centralized service that allows you to store and manage sensitive data, such as database credentials, API keys, and other confidential information. By integrating Secrets Manager with ASCP in your EKS cluster, you can make sure your applications can access sensitive data securely and efficiently, without exposing it in plain text or storing it in unrestricted locations. This integration enhances both security and configuration management across your cloud infrastructure.

For more details on using Secrets Manager and ASCP in Amazon EKS, see How to use AWS Secrets & Configuration Provider with your Kubernetes Secrets Store CSI driver.

Enabling GuardDuty is crucial for threat detection and monitoring within your Amazon EKS environment. GuardDuty is a threat detection service that continuously monitors your AWS accounts and workloads for malicious activity and unintended behavior. By enabling GuardDuty Protection for your EKS cluster, you can receive real-time alerts and take immediate action to mitigate potential issues, reducing the issues of data and unintended access.

For more details on detecting security issues in EKS clusters with GuardDuty, see How to detect security issues in Amazon EKS clusters using Amazon GuardDuty – Part 1. For further insights on investigating and addressing security issues using Amazon Detective, see Part 2.

Enabling Pod Security Standards (PSS) in Amazon EKS is essential for enforcing consistent security policies across your Kubernetes environment. PSS helps prevent the deployment of insecure or misconfigured pods, reducing the issues of unintended access. It restricts risky behaviors like running containers as root or allowing privileged escalation, validating only secure workloads are deployed. PSS also helps organizations meet compliance requirements by aligning with security best practices. As Amazon EKS environments grow, it automates the enforcement of security policies, simplifying management.

For more details on how to enable PSS, see Implementing Pod Security Standards in Amazon EKS.

Implementing Kyverno Policies is another powerful way to secure your EKS cluster. Kyverno is a Kubernetes policy engine that allows you to define and enforce policies across your cluster. With Kyverno, you can make sure your workloads adhere to best practices, comply with security standards, and maintain a consistent configuration throughout their lifecycle. By implementing Kyverno policies, you can prevent misconfigurations, enforce resource quotas, and restrict access to sensitive resources, enhancing the overall security posture of your EKS cluster.

For more details on managing pod security with Kyverno on Amazon EKS, see Managing Pod Security on Amazon EKS with Kyverno.

Conclusion

In this post, we discussed how to automate security scans and compliance checks on your EKS clusters using Kubescape, CodeBuild, and CodePipeline. We also covered best practices that can help protect applications and data on your EKS cluster from potential security risks. Security is an ongoing process, and it’s essential to regularly review and update your security measures to stay ahead of emerging security risks and unintended access.

If you have feedback about this blog post, use the Comments section on this page.

About the authors

Aniket Dekate is a Cloud DevOps Consultant. who specializes in cloud infrastructure, automation, containerization, cloud-native architectures, and generative AI applications. With a background in development and product engineering, he focuses on building resilient, scalable, and reliable cloud-native architectures, particularly in the IOT, finance, and security domains. Outside of work, he enjoys playing cricket, badminton and table tennis.

Ishwar Chauthaiwale is a rising star in the Cloud and DevOps domain, with 7.5 years of valuable experience under his belt. As a young professional, he has quickly established himself as an expert in automation, migration, and modernization strategies for cloud environments. Ishwar’s technical acumen spans across various cloud platforms, allowing him to guide organisations through complex digital transformations with confidence and precision. His passion for innovative solutions and cutting-edge technologies has made him a go-to resource for companies looking to optimize their infrastructure and streamline their processes. Beyond his professional endeavors, Ishwar is committed to continuous learning and often participates in tech conferences and workshops to stay at the forefront of industry trends. He also dedicates time to mentoring aspiring cloud professionals, sharing his knowledge and experiences to inspire the next generation of IT experts.

Build an automated deployment of generative AI with agent lifecycle changes using Terraform

Anand Krishna Varanasi — Fri, 18 Oct 2024 03:23:17 +0000

Generative AI technology holds immense potential for driving innovation, enhancing content creation, improving customer experiences, streamlining operations, and boosting productivity across various industries. To unlock this transformative power, Amazon Bedrock offers a fully managed service that integrates high-performance foundation models from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, Mistral AI, and Amazon. By providing a single API to access these advanced models and supporting the development of generative AI applications with a strong focus on security, privacy, and responsible AI practices, Amazon Bedrock empowers organizations to explore new avenues for innovation and elevate their overall offerings through the power of AI.

In this blog post, we’ll guide you through the process of creating a fast, reusable, and repeatable Retrieval-Augmented Generation (RAG)-based generative AI infrastructure as code (IaC) using Terraform; keeping in mind best practices for a sample use case chatbot as a blueprint. This solution provides a foundational framework that you can customize and expand to develop robust, scalable, and efficient chatbot applications on the AWS Cloud using Amazon Bedrock. You’ll learn how to automate the provisioning of essential components required for a generative AI chatbot such as Amazon Bedrock Agents, Amazon Bedrock Knowledge Bases, Amazon OpenSearch Serverless, AWS Lambda, Amazon Simple Storage Service (Amazon S3) buckets for Amazon Bedrock Knowledge Bases, Amazon Bedrock Guardrails and the AWS Identity and Access Management (IAM) roles required to permit and link the services mentioned.

We also dive deep into an automated custom logic lifecycle to prepare an agent and create and update an agent alias when there are changes to the agent action group Lambda code or instructions to the agent or the knowledge base.

Solution infrastructure diagram

The following figure shows the solution infrastructure.

This architecture diagram illustrates the integration of Amazon Bedrock with various AWS services to create an intelligent, secure, and automated workflow. The Amazon Bedrock Agent is at the core, utilizing Bedrock Knowledge Bases for handling queries and Bedrock Agent Action Groups to invoke necessary actions through Open API schema, Lambda and Amazon S3. It incorporates a VPC (Virtual Private Cloud) endpoint, which securely connects to resources within a private subnet. Inside the private subnet, a Lambda function operates to perform automated tasks triggered by the Bedrock Agent. These functions interact with various AWS resources while helping to maintain security and compliance. This setup makes sure that user requests are processed efficiently, securely, and in a scalable manner within the AWS environment.

Solution setup

The code for the solution is available on the GitHub repository in aws-samples. Clone the code on your machine using the following command:

git clone "git@github.com:aws-samples/intelligent-rag-bedrockagent-iac.git"

Package structure

After the repository is cloned, you will find the structure of code as shown in the following figure. Each service required to build the solution has its own module conforming to the best practices of writing repeatable IaC code. The code can be integrated into a DevOps tool or lifecycle process.

The root module calls the submodules to build the stack in the sequence required to set up the solution stack.

Each module references other modules as module outputs. The following list describes the purpose of each module:

The knowledge_base_bucket module is used to build the knowledge base data source S3 bucket with encryption, access logging for the bucket access, versioning, object lifecycle configuration, and (optional) Amazon Simple Notification Service (Amazon SNS) alerting for object uploads. The resources in the module can be configured to fit your use case.
The roles module builds the roles with security best practices for the agent, knowledge bases, and Lambda functions with KMS encryption and decryption permissions.
The bedrock_knowledge_base module creates a knowledge base and the knowledge base data source, and it creates a policy to allow an OpenSearch Serverless collection to be integrated with the knowledge base through the aoss module. This module also creates AWS Systems Manager parameters to store the values of the knowledge base ID and knowledge base data source ID.
The aoss module builds the OpenSearch Serverless collection with a FAISS index and three OpenSearch Serverless encryption, data, and network policies. This module uses respective roles created in the roles module and references in the module, such as the knowledge base role Amazon Resource Name (ARN).
The bedrock_agent module creates an Amazon Bedrock agent, a knowledge base association with the agent, an action group Lambda function that references the OpenAPI schema provided for the sample application, and finally Systems Manager parameters to store the agent IDs, agent alias IDs, agent instructions, agent action group instructions, and the Lambda function’s last update status, automating Lambda function code updates. This module references roles created by the roles module (such as the Lambda role and agent role), knowledge base bucket names, knowledge base ID, and the ARNs to build the agent.
The bedrock_guardrail module creates Amazon Bedrock agent guardrails with the user’s chosen method to safeguard user inputs and responses to detect sensitive information such as personally identifiable information (PII) data. Amazon Bedrock Guardrails assesses user-provided information and foundation model outputs according to scenario-specific guidelines, offering an extra level of protection irrespective of the base foundation model (FM) used. The module blocks access based on the following properties (configurable based on user input in variables.tf).
- Content policy configuration for the Amazon Bedrock guardrails
- Sensitive information policy configuration for the Amazon Bedrock guardrails
- Topic policy configuration for the Amazon Bedrock guardrails
- Word policy configuration for the Amazon Bedrock guardrails
The vpc_endpoints module creates interface and gateway endpoints for private communication between the services and the infrastructure. The endpoints created are for the following services:
- Amazon Bedrock
- Amazon OpenSearch Serverless
- AWS Systems Manager
- Amazon S3
- AWS Key Management Service (AWS KMS)
- Amazon Simple Queue Service (Amazon SQS)
The agent_update_lifecycle module builds custom logic that will automatically help developers by creating a new agent alias. This Alias points to the existing agent configuration. See the following section for the detailed workflow of this module.

Prerequisites

Access management: The operations team must carefully delegate access for the services created by the solution. Proper access control measures should be in place to help secure and control access by, for example, using Systems Manager, models, an IAM role for deployment, and service-based roles, disabling public access to S3 buckets, and setting up a dead-letter queue.
AWS CLI version: Make sure that the AWS Command Line Interface (AWS CLI) v2 is installed and configured correctly on the deployment environment.
Provider versions: Before proceeding with the deployment, update the provider versions to the latest available versions to benefit from the latest features and security updates.
API security: Review the open API schema and implement appropriate authentication mechanisms to help protect against unauthorized access and maintain data integrity.
Model dependencies: Verify that the required Amazon Titan Text v2 and Claude3 Haiku models are enabled in your AWS account. These models are essential for the solution’s functionality and can be changed in the variables. See Add or remove access to Amazon Bedrock foundation models for how to enable model access for the AWS Region where you are deploying the solution.
Regional compatibility: Confirm that the target deployment region supports the required models. Cross-check the model access and availability in the chosen region to avoid deployment failures.
Network architecture: Provision a regulated virtual private cloud (VPC) following the AWS Well Architected Framework best practices. This VPC will host the solution and support a secure and high-performing network architecture.

Deployment instructions

This section below should be used to perform deployment of the solution containing a sample application.

Clone the repository.

git clone "git@github.com:aws-samples/intelligent-rag-bedrockagent-iac.git"

Navigate to the project base directory.

cd intelligent-rag-bedrockagent-iac

Initialize Terraform.

terraform init

Run terraform plan to view the list of resources that this solution will create.

terraform plan

Run terraform apply. The flag —auto-approve is optional. If you don’t use that, you will be prompted to manually enter a confirmation.

terraform apply —auto-approve

Note: ensure User Input is enabled post deployment.

Solution workflow

The solution workflow is shown in the following figure.

The knowledge base is a repository of information that the agent can query to retrieve additional context and data to augment its responses. The agent orchestrates the workflow, which includes:

Preprocessing the user input to contextualize and categorize it.
Invoking the appropriate action group Lambda function or querying the knowledge base based on the user’s request.
Generating an observation (output) from the action group Lambda function or knowledge base query.
Formatting the final response to return to the user.
The action group Lambda function is a custom function that the agent can invoke to perform specific tasks. During the orchestration step, the agent determines which action group Lambda function to invoke based on the user’s input.

Ultimately, the workflow of the Amazon Bedrock agent and knowledge base integration is as follows:

The agent interprets the user’s input and generates a rationale for the next steps to take.
Based on the rationale, the agent predicts which action group Lambda function to invoke or which knowledge base to query.
If the agent needs more information to invoke the action, it will query the knowledge base to retrieve additional context.
The agent then generates an observation (output) from the action group Lambda function or knowledge base query and uses it to augment the base prompt.
This process continues in a loop until the agent can provide a final response to the user.

Agent update lifecycle workflow

The agent update lifecycle workflow is shown in the following figure.

Developers or DevOps administrators frequently test knowledge base and agent responses for their code changes, and for this developers need to prepare and create a new agent alias pointing to the latest version whenever there are changes to the knowledge bases, action group, or agent metadata.

The custom code provided has logic (agent_update_lifecycle module) to do this automatically for the developers. The logic will create a new alias when any of the following occur:

Developers update the Amazon Bedrock agent instructions using the AWS Console or the AWS CLI
Developers update the action group Lambda function
The knowledge base instructions for the agent are changed
(Optional) Based on added logic to initiate creating a new agent alias.

Chatbot application workflow

In this blog post, we’ll explore the creation of a simple fitness chatbot. Here’s a quick overview of its key features and the technology behind them:

Natural language understanding: Using Claude 3 Haiku, our chatbot generates human-like responses to user queries, providing a smooth and intuitive conversation experience.
Knowledge base integration: We’ve implemented a sophisticated knowledge retrieval system. Documents are uploaded to an S3 bucket, converted into embeddings using the Amazon TITAN v2 model, and stored in OpenSearch Serverless as a vector database. This allows the chatbot to quickly access and deliver relevant fitness information.
Flexible LLM capabilities: While we’re currently using Claude 3 Haiku, the code base is designed with configurability in mind, allowing simple switching between different large language models (LLMs) as an IaC variable.
Comprehensive fitness knowledge: The chatbot is equipped with a broad understanding of fitness-related topics, making it a valuable resource for users seeking general health and exercise information.
BMI calculation feature: As a practical application, we’ve integrated a body mass index (BMI) calculator. This feature is implemented as a Lambda function and is connected to the agent through an action group, demonstrating how to extend the chatbot’s capabilities with custom actions.

Agent configuration

Amazon Bedrock Agents streamline workflows and automate repetitive tasks. Amazon Bedrock Agents securely connect to your company’s data sources and augments the user request with the right information to generate an accurate response. The following instructions illustrate the configuration of the Bedrock Agent.

Agent instructions

You are a helpful fitness assistant. You have general knowledge about sports. You can answer questions related to fitness, diet plans. Use only the tools or knowledge base provided to answer user questions. Choose between the tools or the knowledge base. Do not use both. Do not respond without using a tool or knowledge base.

When a user asks to calculate their BMI:
1. Ask for their weight in kilograms.
2. Ask for their height in meters
3. If the user provides values in any other unit, convert it into kilograms for weight and meters for height. Do not make any comments about health status.

The key to creating an effective chatbot lies in the prompt—the instructions given to the AI to shape its behavior and responses. For our fitness chatbot, we developed a prompt that defines its role, capabilities, and limitations. Here’s a breakdown of the main components:

Role definition: The chatbot is introduced as a helpful fitness assistant with general knowledge about sports and the ability to answer fitness and diet-related questions.
Knowledge base: The AI is instructed to use only the provided tools or knowledge base to answer questions, maintaining accuracy and consistency in its responses.
Specific functionality: We included a detailed process for calculating BMI, which demonstrates how to guide the AI through multi-step interactions.
Unit conversion: To enhance user-friendliness, the chatbot is programmed to convert units if users provide weight or height in non-standard measurements.
Ethical considerations: The AI is explicitly instructed not to make comments about health status, maintaining a neutral and non-judgmental stance.

The BMI calculation feature

One of the standout features of this chatbot is its ability to calculate BMI. Here’s how it works:

When a user asks the chatbot to calculate their BMI, it initiates a structured dialogue.
It first asks for the user’s weight in kilograms.
Then, it requests the user’s height in meters.
If the user provides values in different units (for example, pounds or feet and inches), the chatbot automatically converts these to kilograms and meters.
After obtaining the necessary information, it calculates the BMI without offering unsolicited health advice.

The AI understands that it must calculate BMI. It understands that it needs to ask the user for height and weight. However, after that information is received it cannot immediately compute BMI. To perform the computation, it needs to first perform an action. This is defined in the agent’s action group.

Action group

First, let’s clarify what an action group is. In the context of Amazon Bedrock agents, an action group is a configuration that enables the agent to perform specific actions. These actions are defined in Lambda functions, allowing for custom functionality beyond the agent’s built-in capabilities.

How does the agent know when to call a particular Lambda function? The answer lies in the OpenAPI Schema definition file.

The decision-making process:

Knowledge base check: When a user asks a question, the agent first consults its knowledge base. This is where it determines whether it can answer directly or if it needs to perform an action.
Action determination: If the agent decides an action is necessary, it turns to the OpenAPI schema for guidance.
Schema consultation: The OpenAPI schema acts as a roadmap, defining available actions and their corresponding Lambda functions.

A real-world example: BMI calculation

Let’s illustrate this with our fitness chatbot’s BMI calculation feature:

User query: The user asks the chatbot to calculate their BMI.
Agent analysis: The agent recognizes this request requires an action rather than a simple response based on the knowledge base.
Schema reference: The agent consults the OpenAPI schema, which includes a definition for BMI calculation.
Function identification: Within the schema, the agent finds the specific Lambda function designated for BMI calculation.
Invocation: With this information, the agent invokes the correct Lambda function to perform the BMI calculation.

The OpenAPI schema: A closer look

In our BMI calculation example, the OpenAPI schema looks something like the following figure.

OpenAPI schema explanation:

{
  "openapi": "3.0.0",
  "info": {
      "title": "BMI Calculator",
      "description": "Lambda function to calculate Body Mass Index (BMI)",
      "version": "1.0.0"
  },
  "paths": {
    // Name of the lambda function, configured in the action group of the agent
      "/calculate_bmi": {
          "post": {
              "summary": "Calculate BMI",
              "description": "Calculates the Body Mass Index (BMI) based on weight and height",
              "operationId": "calculate_bmi",
              "requestBody": {
                  "required": true,
                  "content": {
                    // Input definition for the agent that accepts weight and height
                      "application/json": {
                          "schema": {
                              "type": "object",
                              "required": [
                                  "weight",
                                  "height"
                              ],
                              "properties": {
                                  "weight": {
                                      "type": "number",
                                      "description": "Weight in kilograms",
                                      "example": 70
                                  },
                                  "height": {
                                      "type": "number",
                                      "description": "Height in meters",
                                      "example": 1.75
                                  }
                              }
                          }
                      }
                  }
              },
              "responses": {
                // This section contains response codes.
                  "200": {
                      "description": "Successful BMI calculation",
                      "content": {
                          "application/json": {
                              "schema": {
                                  "type": "object",
                                  "properties": {
                                      "statusCode": {
                                          "type": "integer",
                                          "example": 200
                                      },
                                      "body": {
                                          "type": "string",
                                          "description": "JSON string containing the BMI result",
                                          "example": "{\"bmi\": 22.86}"
                                      }
                                  }
                              }
                          }
                      }
                  },
                  "400": {
                      "description": "Bad request - missing or invalid parameters",
                      "content": {
                          "application/json": {
                              "schema": {
                                  "type": "object",
                                  "properties": {
                                      "statusCode": {
                                          "type": "integer",
                                          "example": 400
                                      },
                                      "body": {
                                          "type": "string",
                                          "description": "Error message",
                                          "example": "\"Missing parameter: weight\""
                                      }
                                  }
                              }
                          }
                      }
                  },
                  "500": {
                      "description": "Internal server error",
                      "content": {
                          "application/json": {
                              "schema": {
                                  "type": "object",
                                  "properties": {
                                      "statusCode": {
                                          "type": "integer",
                                          "example": 500
                                      },
                                      "body": {
                                          "type": "string",
                                          "description": "Error message",
                                          "example": "\"An unexpected error occurred\""
                                      }
                                  }
                              }
                          }
                      }
                  }
              }
          }
      }
  }
}

Test the chatbot

In this section, we test the chatbot by asking it a series of questions and evaluating the answers. The figures show the questions and answers. All but the first figure also show the trace window, which shows the steps taken by the chatbot to return an answer.

Please note that since the chatbot leverages LLM, the language and the response can vary. However, the sentiments and the essence of the response should remain the same. Also, this is a sample application that helps you calculate BMI and contains a generic formula for BMI calculation and does not infer actual scenarios.

Scenario 1: Ask a generic question to understand the capabilities of the chatbot.

Question: What can you help me with?

Answer: As a fitness agent, I can help you with questions related to fitness, diet plans, and calculating your Body Mass Index(BMI). I have access to set of functions that can assist you with these tasks. Let me know if you have any specific questions and I'll do my best to help.

Scenario 2: In the data source file, there is some data that contains information about how IT professionals can improve their fitness. The purpose of this question is to demonstrate that we can ask the chatbot to search the knowledge base and provide a corresponding answer.

Question: what are the fitness tips for IT professionals?

Answer: 1. Combat sedentary lifestyle by standing up and stretching every hour, using a standing or treadmill desk, and taking short walks during breaks or phone call. 2. Maintain eye health by following the 20-20-20 rule (look at something 20 feet away for 20 seconds every 20 mins, using blue light filtering glasses, and adjusting monitor brightness and contrast. 3. Incorporate desk exercises like desk push-ups, chair squats, seated leg raises, and wrist finger stretches to stay active throughout the workday. 4. Prioritize stress management through mindfulness, meditation and taking regular breaks to reset mentally. 5. Maintain good nutrition and hydration habits, such as eating brain boosting foods and avoiding excessive caffeine and sugar, and staying hydrated.

Scenario 3: Let’s ask a generic question which has some overlap with fitness. We want to know if the LLM can answer questions from its pre-trained LLM data.

Question: What is table tennis?

Answer: Table tennis, also known as ping-pong, is a popular indoor sport where two or four players hit a lightweight ball back and forth across a net on a table. The objective is to hit the ball in such a way that the opponent cannot return it successfully. Table tennis is played on a table with a net in the center, and players use small paddles to hit the ball. It is an Olympic sport and is enjoyed by people of all ages as a recreational activity or competitive sport.

Scenario 4: Now, we ask a question for which the agent must perform an action. The action in this example is to calculate BMI.

Question: Can you calculate BMI

Answer: <Q&A flow, as displayed in the following screenshot>

Clean up

Use the following steps to clean up the infrastructure created for the solution. The solution can be built, updated, or modified using a variety of DevOps tools or procedures.

To clean up the infrastructure:

Empty the data source S3 bucket.
Navigate into the project base directory.

cd intelligent-rag-bedrockagent-iac

The flag —auto-approve is optional. If you don’t use that, you will be prompted to enter a confirmation manually.

terraform —destroy --auto-approve

Conclusion

This blog post guides you through implementing a generative AI intelligent chatbot using Amazon Bedrock and AWS services, including developer lifecycle changes. It covers creating a scalable chatbot infrastructure with Terraform, using services like Amazon Bedrock Agents, Amazon Bedrock Knowledge Bases, OpenSearch Service, Lambda, Amazon S3, and IAM. The focus on IaC and automation brings key DevOps benefits: version control, consistent deployments, reduced errors, faster provisioning, and improved collaboration.

This guide serves as a blueprint for organizations to develop code based on re-usable, modular, and security best practices to deploy a generative AI based infrastructure into multiple environments with minimal changes (using DevOps if required).

Review the Amazon Responsible AI policy before implementing this solution.
This solution does not include a chat history feature and the chat is not stored.
This solution deploys the default guardrail policies.

If you have feedback about this blog post, use the Comments section on this page.

About the authors

Anand Krishna Varanasi is a seasoned AWS builder and architect who began his career over 16 years ago. He guides customers with cutting-edge cloud technology migration strategies (the 7 Rs) and modernization. He is very passionate about the role that technology plays in bridging the present with all the possibilities for our future. Anand is also a meditation coach who teaches the benefits of meditation to hundreds of villages, towns, colleges and corporations in India and beyond.

Anand Bukkapatnam Tirumala is a seasoned AWS builder and architect who began his career 11 years ago. Anand has vast experience working with customers of different scales and magnitude ranging from startups to MNCs. He enjoys streamlining cluttered work streams through automation that drives business success. Anand is an avid aquascaper and mentors others interested in the hobby.

Build workflows-as-code automations for AWS services using Flowpipe

Gabriel Costa — Tue, 03 Sep 2024 16:24:44 +0000

DevOps practitioners are always looking for better ways to easily build workflows that integrate AWS services with other services and APIs. Since we’ve all embraced infrastructure as code (IaC), why not extend that concept to workflow automation? That’s the philosophy of Flowpipe, a new open-source tool from Turbot. It offers the same components that you’ll find in typical workflow tools, including pipelines, steps, triggers, and control flow. And it integrates with everything you’d expect from this type of tool.

But Flowpipe isn’t ClickOps; you don’t draw diagrams. Pipeline definitions use HashiCorp configuration language (HCL) for code artifacts that live in repositories as first-class citizens of the modern software ecosystem—version-controlled and collaborative. You run pipelines using a single binary that you deploy locally, in the cloud, or in any CI/CD pipeline.

In this blog post, we introduce you to Flowpipe and provide a set of examples for automating cloud operations on AWS.

About this blog post
Time to read	~7 minutes
Time to complete	~20 minutes
Cost to complete	$0
Learning level	Intermediate (200)
AWS services	Amazon CloudWatch Amazon Simple Storage Service (Amazon S3) Amazon Virtual Private Cloud (VPC) AWS Identity and Access Management (IAM) AWS Key Management Service (AWS KMS)

Prerequisites

To get started with Flowpipe for your AWS environment, follow these steps:

Install Flowpipe.
Configure the AWS CLI in your local environment.
In the IAM console, configure the following environment variables to authenticate with AWS resources through IAM credentials.
- AWS_PROFILE
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
Optionally define credentials in Flowpipe configuration files.

Examples

The following examples show you how to use Flowpipe to perform common cloud operations such as creating a Flowpipe pipeline and building Amazon VPC and Amazon S3 scenarios into your pipeline.

Create a basic pipeline using a geolocation IP service

This example is a two-step pipeline that asks api.ipify.org for your public IP address and then calls the Really Free GEO IP Flowpipe library to geolocate the address.

Usage

To initialize a mod:

mkdir my_mod
cd my_mod
flowpipe mod init

To install the Really Free GEO IP mod as a dependency:

flowpipe mod install github.com/turbot/flowpipe-mod-reallyfreegeoip

To use the dependency in a pipeline step:

vi my_pipeline.fp

To create a pipeline that geolocates a public IP address, run the following code in the Flowpipe CLI:

pipeline "geolocate" {
  step "http" "get_ipv4" {
    url = "https://api.ipify.org?format=json"
  }

  step "pipeline" "get_geo" {
    pipeline = reallyfreegeoip.pipeline.get_ip_geolocation
    args = {
      ip_address = step.http.get_ipv4.response_body.ip
    }
  }

  output "ip_address" {
    value = step.http.get_ipv4.response_body.ip
  }

  output "latitude" {
    value = step.pipeline.get_geo.output.geolocation.latitude
  }

  output "longitude" {
    value = step.pipeline.get_geo.output.geolocation.longitude
  }
}

The pipeline steps run concurrently and are subject to dependencies. In this example, the get_geo step waits for the get_ipv4 step to finish running before starting.

This pipeline example uses the following Flowpipe step types.

http

The http step makes an HTTP request, allowing Flowpipe to interact with external systems. Pipelines can also run steps powered by containerized CLI commands and AWS Lambda-compatible functions.

input

Using the input step, you can build workflows that ask for and respond to human input. The pipeline supports interactions in Slack, Microsoft Teams, or email. You configure notifiers and integrations to specify the communication channel for an approval step.

Note: If you don’t require human input and only need to notify a channel, use the message step with the same configuration.

pipeline

The pipeline step, Flowpipe’s basic unit composition, is responsible for running another pipeline. Flowpipe libraries are made of interacting pipelines. In our example, the get_geo pipeline runs the reallyfreegeoip.pipeline.get_ip_geolocation pipeline.

query

The query step runs a SQL query either immediately or on a schedule. It works with Steampipe (a Postgres-based tool that queries cloud APIs) or with your own Postgres, MySQL, SQLite, or DuckDB database. When using Flowpipe to query Steampipe, you can take advantage of over 140 featured plugins that make APIs available to SQL queries. Steampipe’s AWS plugin is especially rich, offering hundreds of tables that cover a vast number of AWS APIs.

Create an Amazon VPC scenario

This example (1) creates an Amazon CloudWatch log group and a VPC (Amazon VPC) and (2) enables the sending of flow logs (AWS Flow Logs) to the CloudWatch log group. By referencing the output from the create_vpc step in the second and third steps, you automate the creation of the CloudWatch log group and enable the VPC flow log for the VPC.

Usage

To initialize a mod:

mkdir create_vpc
cd create_vpc
flowpipe mod init

To install the AWS mod as a dependency:

flowpipe mod install github.com/turbot/flowpipe-mod-aws

To use the dependency in a pipeline step:

vi create_vpc.fp

Paste the following code into the create_vpc.fp file:

pipeline "vpc" {
  step "pipeline" "create_vpc" {
    pipeline = aws.pipeline.create_vpc  
    args = {
      region = "YOUR-AWS-REGION"
      cidr_block = "YOUR-IPV4-NETWORK-RANGE"
    }
  }

  step "pipeline" "create_cw_log" {
    pipeline = aws.pipeline.create_cloudwatch_log_group  
    args = {
      region = "YOUR-AWS-REGION"
      log_group_name = step.pipeline.create_vpc.output.vpc.VpcId
    }
  }

  step "pipeline" "create_vpc_flow_log" {
    pipeline = aws.pipeline.create_vpc_flow_logs  
    args = {
      region = "YOUR-AWS-REGION"
      vpc_id = step.pipeline.create_vpc.output.vpc.VpcId
      log_group_name = step.pipeline.create_vpc.output.vpc.VpcId
      iam_role_arn = "YOUR-IAM-ARN-ROLE-WITH-PERMISSION-TO-CREATE-VPC-FLOW-LOGS"
    }
  }

  output "vpc" {
    value = step.pipeline.create_vpc.output
  }

  output "cw_log" {
    value = step.pipeline.create_cw_log.output
  }

  output "vpc_flow_log" {
    value = step.pipeline.create_vpc_flow_log.output
  }
}

To run the pipeline, run this command: flowpipe pipeline run create_vpc

Aligning with security best practices for VPCs, this pipeline creates a VPC and enables VPC Flow Logs.

Create an Amazon S3 bucket scenario

Initialize a mod for creating an Amazon S3 bucket and add the AWS mod dependency:

mkdir create_s3
cd create_s3
flowpipe mod init
flowpipe mod install github.com/turbot/flowpipe-mod-aws

To use the dependency in a pipeline step:

vi create_s3.fp

Paste the following code in the create_s3.fp file to create an Amazon S3 bucket and enable bucket versioning and encryption using customer-managed AWS KMS keys. This step aligns with security best practices for Amazon S3.

pipeline "s3" {
  step "pipeline" "create_s3" {
    pipeline = aws.pipeline.create_s3_bucket 
    args = {
      region = "YOUR-AWS-REGION"
      bucket = "YOUR-BUCKET-NAME"
    }
  }

  step "pipeline" "s3_versioning" {
    pipeline = aws.pipeline.put_s3_bucket_versioning 
    args = {
      region = "YOUR-AWS-REGION"
      bucket = "YOUR-BUCKET-NAME"
    }
    depends_on  = [step.pipeline.create_s3]
  }

  step "pipeline" "s3_encryption" {
  pipeline = aws.pipeline.put_s3_bucket_encryption  
  args = {
    region = "YOUR-AWS-REGION"
    bucket = "YOUR-BUCKET-NAME"
    sse_algorithm = "aws:kms"
    kms_master_key_id = "YOUR-KMS-KEY-ID"
    bucket_key_enabled = true
  }
  depends_on  = [step.pipeline.create_s3]
  }
}

To run the pipeline, run the following code:

flowpipe pipeline run s3

‘Detect and correct’ libraries

Flowpipe detect and correct libraries such as AWS Thrifty Mod for Flowpipe can detect problems such as unallocated, cost-incurring Elastic IP addresses and correct them automatically or with human approval.

The following Flowpipe command runs the AWS Thirfty Mod for Flowpipe library:

flowpipe pipeline run detect_and_correct_vpc_eips_unattached \
  --arg approvers='["slack"]' \
  --arg host=local

You can update this command to work with a communication channel (such as Slack, Microsoft Teams, or email) interactively or on a schedule, with or without approval.

For details about integrating with Slack and other communication channels, including authentication and security information, see Slack integration in the Flowpipe documentation.

In the following image, the pipeline detects one unattached Elastic IP address and waits for a decision (Release or Skip) in a Slack channel. If you choose Release, the pipeline calls a utility pipeline in the AWS library to release the Elastic IP address, and a Slack message confirms the action.

If more than one unattached Elastic IP address is present, Slack interactions occur in order. To batch the interaction, set the max_concurrency variable to your desired batch size.

Conclusion

With its ability to run locally, in the cloud, or within CI/CD pipelines, Flowpipe streamlines DevOps processes and promotes infrastructure-as-code principles into your AWS Cloud workflows. We hope our examples have given you a taste of this powerful tool and its many benefits.

To learn more about Flowpipe, we invite you to join our community of builders. Just sign up for our Turbot community and install Flowpipe today. Also check out the AWS library in the Flowpipe Hub, where you’ll find several prebuilt pipelines for common scenarios, including creating a VP security, putting Amazon S3 bucket encryption and versioning, launching and stopping Amazon EC2 instances, and tagging resources.

If you have feedback about this blog post, use the Comments section on this page.

About the authors

Gabriel Costa is a senior partner solutions architect at AWS, working with AWS Partners and customers on all things cloud operations. Outside of work, he enjoys playing the guitar, reading about philosophy, watching sci-fi and anime, and searching with his wife for the new cool restaurant in town.

Jon Udell is the community lead for Turbot’s open source products Steampipe, Powerpipe, and Flowpipe. He’s known as both a developer and tech journalist who explores and explains many kinds of software, and many ways of developing it. He has worked for Lotus Development, BYTE, InfoWorld, O’Reilly Media, and Microsoft.

Manage security events in Slack, Teams, or Amazon Chime using AWS Chatbot and Amazon Q

Arbind Basnet — Thu, 22 Aug 2024 17:43:56 +0000

Organizations often make efforts to avoid redundant security alerts to improve their overall security posture. They continuously look for protocols to ensure more focused, efficient, and effective safety operations. Without a solution to eliminate redundancy, teams can inadvertently overlook important alerts, spend unnecessary time on issues they’ve already addressed, increase response time, and clutter dashboards.

This blog post presents a solution that unifies security management into a common messaging app such as Slack, Amazon Chime, and Microsoft Teams. Using a combination of Amazon Q, AWS Chatbot, Amazon Simple Notification Service (Amazon SNS), and AWS Security Hub, all synced together and configured with permissions and roles, you can:

Get notified about security alerts in near real time.
Investigate solutions using chatbot queries.
Take actions using integrated command line interface (CLI) commands directly in your messaging app.

This solution helps you streamline your organization’s security response protocols by consolidating communications in one central channel where interested parties can chat and collaborate on a solution.

This solution has several benefits:

Prioritize the response and remediation efforts of your security and DevSecOps teams by fine-tuning diverse security findings across multiple AWS accounts and resources in one location.
Chat with Amazon Q directly from your messaging app to learn about best practices, troubleshoot issues, and identify next steps.
Open support cases for security incidents from the ease of your phone when you’re away from your desk and need support right away.
Use Amazon Q’s CLI for AI-generated completions as you type in the command line.
Customize notifications for checks and vulnerabilities; for example, you can create a rule to receive notifications based on resource IDs in an AWS organization or account for a specific severity level.

This solution addresses multiple use cases. You can apply it to strategic accounts that require extra attention, for example when a compliance check fails or a warning occurs for Health Insurance Portability and Accountability Act (HIPAA)-compliant workloads in production accounts. In another use case, when cross-account support is enabled for solution provider AWS Partner-Led Support (PLS) customers, you can open a support case from the dedicated administrator account into the rest of the accounts in an organization.

About this blog post
Time to read	~10 minutes
Time to complete	~2 hours
Cost to complete	0
Learning level	Intermediate (200)
AWS services	Amazon EventBridge Amazon Simple Notification Service (Amazon SNS) Amazon Q AWS Chatbot AWS Identity and Access Management (IAM) AWS Security Hub

Architectural overview

The solution sets up the following architecture:

Figure 1: Architectural overview of AWS services with Slack integration

Security Hub detects a status change such as a security alert and invokes an EventBridge rule.
The EventBridge rule sends a notification to Amazon SNS about the change in status.
Amazon SNS sends the security status notification to AWS Chatbot, which is subscribed to the SNS topic.
AWS Chatbot routes the security status notification to the Slack channel.
A user of the Slack channel chats with the Amazon Q chatbot agent about the alert.
The user opens a support case through the chatbot agent.

Prerequisites

Security Hub enabled either in an AWS organization or within a standalone AWS account. See Enabling Security Hub.
An IAM user and role with permissions to configure EventBridge, Amazon SNS, and AWS Chatbot. See IAM roles.
An active user of Microsoft Teams, Slack, or Amazon Chime.

Walkthrough

Step 1: Create an Amazon SNS topic

Sign in to the Amazon SNS console, and create an Amazon SNS topic named securityhub with sufficient permissions to receive and send notifications. As a best practice, implement least privilege access to reduce security risks and errors. Confirm that the Amazon Resource Name (ARN) is accurate in the access policy.

You will set this SNS topic as the target of an EventBridge rule in the next step.

Figure 2: An SNS topic name securityhub with an access policy

Step 2: Create the EventBridge rule

The EventBridge rule determines which events to send to which targets for processing. In my solution, the rule is invoked when Security Hub detects a change in status such as a security alert. You can choose to be notified of all the events that Security Hub detects, or you can filter specific security events.

To create the EventBridge rule, perform these steps:

Open the Security Hub console.
Navigate to the Management, Automations section in the left navigation bar.
Create an EventBridge rule and configure the security level as CRITICAL.

Figure 3: Configuring the EventBridge rule
To configure Security Hub event notifications, navigate to the Event pattern section and choose one of the following options in the Event type field:
- To be notified of all events, choose All Events.
- To filter security-related events, choose Security Hub Findings-Imported.
  
  Figure 4: Configure Security Hub event notifications
Choose Next.
Choose AWS service in the Target types section.
In the Topic field, select the securityhub topic that you created in the previous step.

Figure 5: Choosing the EventBridge rule target
Choose Next, and then save your EventBridge rule.

Step 3: Install the chatbot agent

Install the AWS Chatbot agent in your preferred communication channel, for example Slack, Amazon Chime, or Microsoft Teams. This agent is integrated with the Amazon Q chatbot agent for opening support tickets and entering CLI commands.

To install the chatbot agent in Slack, perform these steps:

In your Slack channel settings, add or install AWS Chatbot. Make note of the channel ID. For instructions, see Tutorial: Get Started with Slack in the AWS Chatbot documentation.
Sign in to the AWS Chatbot console, and perform the following steps:
1. Create a new client.
2. Create a role for the Chatbot agent that you will integrate with Amazon Q.
3. Subscribe to the SNS topic that you created earlier.
  
  Figure 6: Subscribing to the SNS topic from Slack
Set permissions for the chatbot’s IAM role to allow Amazon Q to interact with AWS services. With permissions set on the chatbot role, the chatbot can provide recommendations about the Security Hub findings when asked by a user. It also allows users to enter CLI commands to perform tasks. The CLI responds with suggestions for exact commands based on the syntax entered by the user.

The AWSSupportAccess permission has been added to the chatbot role so it can create support cases. For example, when you type the CLI command @aws create a support case, the chatbot prompts you to choose the support create-case command.

Note: Because AWSSupportAccess is an AWS managed policy, the solution adheres to security best practices, including granting least privilege.

For information about adding the AWSSupportAccess permission to a user, group, or role, see Manage access to AWS Support Center.

Here are some examples of questions you can ask Amazon Q from your configured channel:

@aws how do I troubleshoot lambda concurrency issues?
@aws what are the best practices for securing S3 buckets?

Results

Now you are ready to start receiving notifications from Security Hub in your Slack channel. In the following image, I have received a notification for one of my S3 buckets. I ask the chatbot to provide recommendations on how to block public access to the S3 bucket and open a support case.

Figure 7: Receiving a security alert in Slack channel

Next, I ask the chatbot to create a support case.

Figure 8: User prompting the chatbot to open a support case

Finally, I choose the Run command, and fill in the required information.

Figure 9: Chatbot creates a support case

Conclusion

This AWS solution provides a streamlined approach for managing security incidents in a single messaging app channel. It helps teams react and respond to security events from a central location, including from the ease of their phone. Having multiple cloud engineers engaging in a common communication channel reduces redundancy in the flow of security-related notifications and helps them work toward a solution using familiar chat capabilities and common commands.

As a next step, apply the solution to your own environment to learn about the benefits and see first-hand how they can positively affect your security posture.

Also, check out the following resources:

If you have feedback about this article or want to get in touch with the author, leave your feedback in the Comments section on this page.

Arbind Basnet is a Partner Technical Account Manager with AWS Enterprise Support. He enjoys building AWS Serverless solutions for AWS Partners. Outside of work, Arbind is a proud U.S. Army veteran and avid trail runner.

Restart Amazon ECS tasks with AWS Lambda and AWS CloudFormation custom resources

Umesh Salian — Fri, 09 Aug 2024 17:43:18 +0000

Long-running tasks such as web applications in Amazon Elastic Container Service (Amazon ECS) are often configured to read an AWS Secrets Manager secret value at startup. When that secret is rotated in Secrets Manager, all Amazon ECS tasks that use the secret must be restarted to read the new value.

You can restart tasks under an Amazon ECS service using an UpdateService API call with the forceNewDeployment option using the AWS Management Console or AWS Command Line Interface (CLI). However, this option is not available for application environments where changes are allowed only through pipeline deployments. In these situations, you must rebuild and redeploy the container, which can pose operational challenges to organizations with a large number of Amazon ECS deployments.

In this post, I present an approach that programmatically recycles tasks under an Amazon ECS service by using a combination of an AWS Lambda function and an AWS CloudFormation custom resource. My solution is designed to integrate with a pipeline so that Amazon ECS tasks restart whenever the pipeline is deployed.

About this blog post
Time to read	~10 minutes
Time to complete	~20 minutes
Cost to complete	~$1 (see the AWS service documentation for details)
Learning level	Intermediate (200)
AWS services	AWS Lambda AWS CloudFormation AWS Identity and Access Management (IAM) Amazon Elastic Container Service (Amazon ECS)

Overview

My solution deploys the following architecture.

Figure 1. Recycling Amazon ECS tasks with a CloudFormation custom resource

A user creates or updates a CloudFormation custom resource through a pipeline deployment. The Amazon ECS cluster and service names are resource properties of the custom resource.
The custom resource invokes the Lambda function as its service token to initiate the process.
The Lambda function extracts the Amazon ECS cluster and service names from the invocation event, and makes an UpdateService API call with the forceNewDeployment option on the service.
The Amazon ECS service recycles all of its tasks.
The Lambda function sends a response back to the custom resource.

In this configuration, the Lambda function is invoked whenever you create, update, or delete the custom resource. You can also pass arguments to the Lambda function by configuring them as properties to the custom resource. This lets you recycle any of the Amazon ECS services by passing in corresponding identifiers. You can also integrate the solution to a deployment pipeline for restarting Amazon ECS tasks in restricted environments.

Important: My solution does not apply to Amazon ECS standalone tasks. Also, Terraform does provide an option to force new deployment.

Prerequisites

An AWS account with permissions to interact with AWS resources.
An Amazon ECS task running under a service in an Amazon ECS cluster. For an example, see Learn how to create an Amazon ECS Linux task for the Fargate launch type.

Walkthrough

To get started, sign in to the AWS Management Console.

Step 1: Create an execution role with permissions for the Lambda function

Task 1.1: Create an execution role in the IAM console

Open the IAM console.
In the navigation pane of the IAM console, choose Roles, Create role.
Under Trusted entity type, choose AWS Service.
Under Use case, choose Lambda.
Choose Next.
Under Permission Policies, select AWSLambdaVPCAccessExecutionRole.
Enter a name for the role, and then choose Create role.

Task 1.2: Add permissions to the execution role

Still in the IAM console, navigate to the role that you just created in the previous step.
Under Add permissions, choose Create inline policy.
Under Select a service, choose Elastic Container Service.
Under List, select ListServices.
Under Write, select UpdateService.
Under Resources, select Any in this account.
Choose Next.
Enter the policy name, and choose Create Policy.

For additional details, see Create a role to delegate permissions to an AWS service.

Step 2: Create the Lambda function

Open the Lambda console.
Choose Create function.
Choose Author from scratch.
Enter the function name and description.
Under Runtime, select Python 3.12.
Under Change default execution role, select Use an existing role.
Under existing role, select the role name that you created in step 1.
Choose Create function.
Paste the following code example in the lambda_function.py tab, and choose Deploy.

import json
import boto3
import botocore
import urllib3

SUCCESS = "SUCCESS"
FAILED = "FAILED"

http = urllib3.PoolManager()

def send(event, context, response_data, physical_resource_id=None):
    print('Sending response back to the custom resource ...')

    response_url = event['ResponseURL']
    response_body = {
        'Status' : SUCCESS,
        'Reason' : "See the details in CloudWatch Log Stream: {}".format(context.log_stream_name),
        'PhysicalResourceId' : physical_resource_id or context.log_stream_name,
        'StackId' : event['StackId'],
        'RequestId' : event['RequestId'],
        'LogicalResourceId' : event['LogicalResourceId'],
        'NoEcho' : None,
        'Data' : response_data
    }

    json_response_body = json.dumps(response_body)
    headers = {
        'content-type' : '',
        'content-length' : str(len(json_response_body))
    }

    try:
        response = http.request('PUT', response_url, headers=headers, body=json_response_body)
    except urllib3.exceptions.HTTPError as e:
        print('Failed')
        print(e)

    print('SUCCESS')

def update_ecs_service(cluster_name, service_name, response_data):
    print('Updating ECS Service {} in Cluster {} ... '.format(service_name, cluster_name))
    try:
        ecs = boto3.client('ecs')
        response = ecs.update_service(
            cluster=cluster_name,
            service=service_name,
            forceNewDeployment=True
        )
    except botocore.exceptions.ClientError as e:
        print('Failed')
        print(e)

    response_data['status'] = SUCCESS
    print('Done')

def lambda_handler(event, context):
    response_data = {}

    if event['RequestType'] == 'Delete':
        print('Stack Delete request. No action taken')
    else:
        ecs_cluster = event['ResourceProperties']['ECSCluster']
        ecs_service = event['ResourceProperties']['ECSService']
        print("Update service request")
        update_ecs_service(ecs_cluster, ecs_service, response_data)
    send(event, context, response_data)

    return {
        'statusCode': 200,
        'body': 'Success!'
    }

For additional details, see Create your first Lambda function.

Step 3: Create the CloudFormation template

Perform these steps:

Copy and paste the following code example into a .yaml file on your local machine. This file is the CloudFormation template.

AWSTemplateFormatVersion: 2010-09-09

Description: Creates a custom resource to trigger a Lambda when deployed

Parameters:
    ECSCluster:
      Description: Name of ECS cluster
      Type: String
    ECSService:
      Description: Name of ECS service to force new deployment
      Type: String
    ReRunParam:
      Type: String
      Description: |
        A dummy param to allow updating the stack when other params remain same
      Default: "xxx"

Resources:
  LambdaTrigger:
    Type: Custom::LambdaTrigger
    Properties:
      ServiceToken: <arn of Lambda function>
      ServiceTimeout: "120"
      ECSCluster: !Ref ECSCluster
      ECSService: !Ref ECSService
      ReRunParam: !Ref ReRunParam

Replace the <arn of Lambda function> placeholder with the Amazon Resource Number (ARN) of the Lambda function.
Save the .yaml file to your local machine. You will upload this template file in the next step.

Step 4: Create a CloudFormation custom resource

Perform these steps:

Open the CloudFormation console.
Under Create stack, select With new resources (standard).
Under Specify template, select Upload a template file.
Under Upload a template file, choose Choose file.
Navigate to and choose the .yaml file that you created in the previous step.
Choose Next.
Enter the stack name.
Under Parameters, enter values for the ECSCluster and ECSService parameters.
Choose Next and then Submit.

Step 5: Validate the solution

Perform these steps:

Open the Amazon ECS console.
Confirm that the same number of new tasks are added as existed before you created the custom resource.
Confirm that the old tasks are deleted after the new tasks are started and that they complete the health check.
To restart the same tasks again, navigate to the Parameters section, and update the stack with a different value for the ReRunParam parameter.

Cleanup

To avoid incurring future charges, delete the resources you created for this solution in the following sequence:

Delete the CloudFormation stack. For instructions, see Deleting a stack on the AWS CloudFormation console.
Delete the Lambda function. In the Lambda console, select the Lambda function and choose Actions, Delete.
Delete the IAM role. For instructions, see Deleting roles or instance profiles.
Delete the Amazon ECS cluster. For instructions, see Deleting an Amazon ECS cluster.

Conclusion

In this post, I’ve provided a solution for restarting Amazon ECS tasks by using a combination of a Lambda function and a CloudFormation custom resource. You can use this solution for situations such as secrets rotation or launching a new Docker image with the existing task definition in your production environments through a pipeline deployment.

If you have a comment or feedback about this blog post, use the Comments section on this page.

Umesh Salian is a Senior Security Consultant with AWS Professional Services. He enjoys providing design and automation to customers for solving their security concerns. Outside of work, he enjoys watching sports, DIY projects around the house, and traveling to new places.

Simplify SQL queries to your AWS API operations using Steampipe and AWS plugin

Gabriel Costa — Wed, 07 Aug 2024 18:36:54 +0000

Steampipe, an open-source, zero-ETL tool built by AWS Partner Turbot, is a welcome addition for DevOps practitioners who use SQL to query their AWS Cloud API operations. Instead of writing and managing scripts, now you can use Steampipe’s AWS plugin to perform SQL queries. This helps you gather API data directly from Steampipe or any Postgres or SQLite database. The standalone nature of the AWS plugin combined with the “no data storage needed” nature of zero-ETL helps you optimize costs, identify security misconfigurations, and export data—all in a fraction of the time required with manual scripting operations.

Steampipe offers these key features:

Direct API queries for translating API responses into SQL tables, providing real-time data access.
Support for multiple databases, including extensions for both PostgreSQL and SQLite.
An extensive set of plugins with over 450 predefined tables covering a wide range of AWS APIs.
A standalone command line interface (CLI) tool that extracts data from AWS APIs without needing a database.

In this blog post, we introduce you to a set of code examples that show you how to query your AWS APIs. We include queries about monitoring your AWS resources with PostgreSQL, integrating AWS data with SQLite, and using Steampipe’s CLI tool for exporting AWS data.

About this blog post
Time to read	~6 minutes
Time to complete	~15 minutes
Cost to complete	$0
Learning level	Intermediate (200)
AWS services	Amazon Elastic Compute Cloud (Amazon EC2) Amazon Relational Database Service (Amazon RDS) Amazon Simple Storage Service (Amazon S3)

Prerequisites

To get started with Steampipe for your AWS environment, follow these steps:

Install the Steampipe engine.
Run the following command to download the latest AWS plugin from the Steampipe Hub registry: $ steampipe plugin install aws. For more information, see Installing Plugins.
Configure one or more connections using your preferred type of credentials. For more information, see Configuring AWS Credentials.

Practical examples

Example 1: Monitoring AWS resources with PostgreSQL

A. Use Steampipe to monitor Amazon EC2 instances, gather instance types, and public IP addresses without setting up a separate monitoring tool.

```sql
select instance_id, instance_type, state, public_ip_address
from aws_ec2_instance
where state = 'running';
```

B. Automate security checks by querying security group configurations.

```sql
select group_name, description, inbound_rules
from aws_security_group
where vpc_id = 'YOUR-VPC-ID';
```

C. Find out which Amazon RDS instances are not authenticated using IAM.

```sql
select db_instance_identifier, iam_database_authentication_enabled
from aws_rds_db_instance
where not iam_database_authentication_enabled;
```

D. Query AWS data from multiple accounts and regions.

```sql
connection "aws_dev" {
plugin  = "aws"
profile = "aws_dev"
regions = ["us-east-1", "us-west-2"]
}

connection "aws_prod" {
plugin  = "aws"
profile = "aws_prod"
regions = ["us-*"]
}

connection "aws_all" {
plugin      = "aws"
type        = "aggregator"
connections = ["aws_dev", "aws_prod"]
}

select
*
from
aws_all.aws_account
```

As Figure 1 shows, the Steampipe Foreign Data Wrapper (FDW) layer uses its multi-threaded core to fetch results from all connections in parallel. It then immediately streams back the results. Caching is also enabled in the architecture. For example, running an aggregate query preloads the cache for subsequent multi-connection queries and for each individual connection.

Figure 1: Steampipe Foreign Data Wrapper layer

Example 2: Integrating AWS data with SQLite

A. Query AWS resources using SQLite for lightweight, portable data analysis, for example to list EC2 instances and their associated security groups.

```sql
select ec2.instance_id, ec2.instance_type, sg.group_name, sg.description
from aws_ec2_instance ec2
join aws_security_group sg on ec2.vpc_id = sg.vpc_id;
```

B. Monitor Amazon S3 bucket activity.

```sql
select bucket_name, event_time, event_name
from aws_s3_bucket_activity
where event_time > '2024-01-01';
```

Example 3: Using Steampipe’s CLI tool for exporting AWS data

A. Extract data from AWS APIs without needing a database. For example, you can export a list of Amazon EC2 instances to a CSV file.

```bash
steampipe_export_aws aws_ec2_instance > ec2_instances.csv
```

B. Generate a JSON file of Amazon S3 bucket details.

```bash
steampipe_export_aws aws_s3_bucket --output json > s3_buckets.json
```

Conclusion

Steampipe’s zero-ETL engine with its AWS plugin offers a powerful and flexible solution for querying AWS APIs using SQL. We hope these examples offer a simplified and faster approach to data access and integration for your common AWS use cases.

To get started in your own environment, visit the Steampipe website and explore available plugins on the Steampipe Hub.

About the authors

Best practices for creating IaC for AWS GovCloud (US)

Aniket Dekate — Thu, 01 Aug 2024 12:05:06 +0000

In this blog post, we present best practices for AWS GovCloud (US) DevOps teams to consider as they work to maintain a secure infrastructure and adhere to compliance guidelines. We also provide code examples, including how to create Terraform modules for multi-partition environments.

Overview

DevOps teams who manage AWS GovCloud (US) environments often face challenges related to strict security and compliance requirements, especially when those teams use Infrastructure as Code (IaC) methods to provision applications and updates. IaC is crucial for managing cloud resources in a consistent, repeatable, and automated manner.

AWS GovCloud (US) customers such as government agencies and organizations that work with sensitive data and regulated workloads typically operate in an isolated environment for security reasons. This isolation can contribute to the challenges that DevOps teams face as they rework code, fork codebases, and deploy workloads using IaC.

One challenge involves the variations in Amazon Resource Numbers (ARNs) between AWS GovCloud (US) and commercial cloud partitions. The ARNs for AWS services and deployed resources require manual updates. For example, the arn:aws:service:region:account-id:resource-type:resource-id string must be updated to
arn:aws-us-gov:service:region:account-id:resource-type:resource-id.

Multi-partition environments can be problematic for AWS GovCloud (US) teams. Although teams typically use dynamic or region-aware code blocks to manage multi-region deployments, few of these teams use this technique for multi-partition deployments.

Understanding compliance requirements

Before diving into the technical aspects of creating IaC for AWS GovCloud (US), understand the following compliance requirements.

Federal Information Security Modernization Act (FISMA) – outlines a comprehensive framework for protecting federal information systems and data.
Federal Risk and Authorization Management Program (FedRAMP) – provides a standardized approach to security assessment, authorization, and continuous monitoring for cloud products and services.
International Traffic in Arms Regulations (ITAR) – governs the export and import of defense-related articles and services.

Depending on the nature of your workloads and data, you may need to consider additional compliance requirements. For example, if you are handling law enforcement data, you should comply with the Criminal Justice Information Services (CJIS) standards. If you are dealing with healthcare data, you should adhere to the Health Insurance Portability and Accountability Act (HIPAA) regulations.

AWS GovCloud (US) eligibility

For entities and persons based in the U.S, access to AWS GovCloud (US) is restricted to federal agencies, state and local governments, educational institutions, and commercial organizations that meet specific requirements. Due to these requirements and eligibility restrictions, a separate AWS GovCloud (US) account is required. Before signing up for an account, confirm that your organization qualifies. For details, see AWS GovCloud (US) Sign Up.

Networking and VPC configuration

One of the first steps in setting up your GovCloud environment is creating a Virtual Private Cloud (VPC) tailored to your specific needs. Carefully configure subnets, route tables, and create security groups to ensure proper network segmentation and access control.

For enhanced security and performance, consider using AWS Direct Connect or AWS Site-to-Site VPN to establish a dedicated network connection between your on-premises infrastructure and AWS GovCloud (US) environment.

Securing your AWS GovCloud (US) infrastructure

Secure your AWS GovCloud (US) infrastructure using AWS Identity and Access Management (IAM) policies and AWS services that are designed to help you maintain security.

IAM policies

Define and enforce IAM policies to control access to your AWS GovCloud (US) resources. Follow the principle of least privilege and regularly review and update your policies to maintain a secure posture. For an additional layer of security, implement multi-factor authentication (MFA) to require users to provide a second form of authentication in addition to their passwords.

If you need to grant access to resources across multiple AWS accounts, you can set up cross-account roles and permissions. This maintains the principle of least privilege and ensures that access is granted only to the necessary resources.

Security controls

Use the following AWS services to help you maintain security:

AWS Key Management Service (KMS) for managing encryption keys and ensuring that all data at rest and in transit is encrypted using industry-standard encryption algorithms.
AWS CloudTrail for logging API calls and activities.
Amazon CloudWatch for monitoring resources and performance.
AWS Config for tracking and auditing resource configurations.

Also consider configuring security groups and network access control lists (NACLs) to control inbound and outbound traffic to your resources. This will allow only necessary traffic while blocking unauthorized access.

Data management and disaster recovery

To help you manage and store data, take these actions:

Configure Amazon Simple Storage Service (S3) buckets with appropriate policies, versioning, and logging to store and manage your data securely.
Encrypt Amazon Elastic Block Store (EBS) volumes to protect your data at rest and take regular snapshots for backup and recovery purposes.
Implement AWS Backup or other backup solutions to automate backups and enable data recovery in case of data loss or corruption.

Disaster recovery and availability

To ensure high availability and disaster recovery capabilities, consider deploying your workloads across multiple AWS GovCloud (US) regions. Also, define and implement Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO) for your critical workloads to ensure that you can recover from disruptions within acceptable timeframes and with minimal data loss.

Cost management

For budgeting and cost allocation, use AWS Cost Explorer and set up billing alerts to monitor and manage costs effectively. Implement cost allocation tags to track and attribute costs to specific projects or departments.

For long-term workloads with predictable usage patterns, consider purchasing Amazon Elastic Compute Cloud (Amazon EC2) Reserved Instances to benefit from significant cost savings compared to on-demand pricing. Also consider Compute and EC2 Instance Savings Plans for flexible pricing models.

Service-specific considerations

Availability of AWS services may differ across AWS Regions, so verify which services are available in your AWS GovCloud (US) Region. Plan your architecture accordingly and be prepared to use alternative services or solutions if necessary. Also, if you plan to use third-party solutions or services from AWS Marketplace, confirm that they’re compatible with and supported in the AWS GovCloud (US) environment.

Example code

See the following IaC examples for the AWS GovCloud (US).

Terraform

This code snippet provisions an Amazon EC2 instance for AWS GovCloud (US) in the US-West Region. It specifies the Amazon Machine Image (AMI) ID via a variable using the t3.micro instance type. The instance that’s created is tagged with the name HelloGovCloud for easy identification.

resource "aws_instance" "web" {
provider      = aws.us-gov-west-1
ami           = var.ami_id
instance_type = "t3.micro"

tags = {
Name = "HelloGovCloud"
}
}

The following configuration defines an IAM role named DynamoDB_IAM_Role with permissions to list and describe all Amazon DynamoDB tables within the current AWS account and specified Region.

data "aws_caller_identity" "current" {}

data "aws_partition" "current" {}

resource "aws_iam_role" "dynamoDB_role" {
  name = "DynamoDB_IAM_Role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Action" : [
          "dynamodb:ListTables"
        ],
        "Resource" : "*"
      },
      {
        "Effect" : "Allow",
        "Action" : [
          "dynamodb:DescribeTable"
        ],
        "Resource" : "arn:${data.aws_partition.current.partition}:dynamodb:${var.region}:${data.aws_caller_identity.current.account_id}:table/*"
      }
    ]
  })

  tags = {
    tag-key = "tag-value"
  }
}

The following code snippet configures two AWS providers, one for the us-gov-east-1 region and another for the us-gov-west-1 region. The us-gov-west-1 provider is assigned the alias us-gov-west-1. Using this configuration, you can manage resources in both regions within the same Terraform project.

provider "aws" {
  region = "us-gov-east-1"
}

provider "aws" {
  region = "us-gov-west-1"
  alias  = "us-gov-west-1"
}

The following code sample defines two variables:

ami_id—a string representing the AMI ID used to create an EC2 instance.
region—a string that specifies the AWS GovCloud (US) Region for deployment with a default value us-gov-west-1.

variable "ami_id" {
  type        = string
  description = "AMI ID to create Ec2 Instance"
}

variable "region" {
  type        = string
  description = "GovCloud Region to deploy"
  default     = "us-gov-west-1"
}

CloudFormation

The following CloudFormation template creates an IAM role with permissions for an AWS Lambda function to assume the role and an IAM policy granting Amazon DynamoDB access to specific resources in the current AWS account and region. The policy is attached to the IAM role.

Resources:
  Role1ABCC5F0:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
        Version: "2012-10-17"
  RoleDefaultPolicy5FFB7DAB:
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Statement:
          - Action: dynamodb:*
            Effect: Allow
            Resource:
              - Fn::Join:
                  - ""
                  - - "arn:"
                    - Ref: AWS::Partition
                    - ":dynamodb:"
                    - Ref: AWS::Region
                    - ":"
                    - Ref: AWS::AccountId
                    - :table/src-orchestratorEksMetadata
              - Fn::Join:
                  - ""
                  - - "arn:"
                    - Ref: AWS::Partition
                    - ":dynamodb:"
                    - Ref: AWS::Region
                    - ":"
                    - Ref: AWS::AccountId
                    - :table/src-orchestratorEksMetadata/index/*
              - Fn::Join:
                  - ""
                  - - "arn:"
                    - Ref: AWS::Partition
                    - ":dynamodb:"
                    - Ref: AWS::Region
                    - ":"
                    - Ref: AWS::AccountId
                    - :table/src-orchestratorEksMetadata/stream/*
        Version: "2012-10-17"
      PolicyName: RoleDefaultPolicy5FFB7DAB
      Roles:
        - Ref: Role1ABCC5F0

AWS CDK

The following AWS Cloud Development Kit (AWS CDK) code snippet defines a stack that creates an IAM role with permissions for a Lambda function to access specific DynamoDB resources within the current AWS account and region. The role’s policy statements dynamically build ARNs using the partition, region, and account from the stack context.

import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Role, ServicePrincipal, PolicyStatement, Effect } from 'aws-cdk-lib/aws-iam';

export class PartitionIndependentArnStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const partition = this.partition
    /**
     *  If we are using inside L3 contruct we can access partition to build the dynamic ARN from following way 
     *  Stack.of(this).partition
     */
    const account = Stack.of(this).account
    const region = Stack.of(this).region
    const policyStatement = new PolicyStatement({
      effect: Effect.ALLOW,
      actions: [
        "dynamodb:*",
      ],
      resources: [
        `arn:${partition}:dynamodb:${region}:${account}:table/src-orchestratorEksMetadata`,
        `arn:${partition}:dynamodb:${region}:${account}:table/src-orchestratorEksMetadata/index/*`,
        `arn:${partition}:dynamodb:${region}:${account}:table/src-orchestratorEksMetadata/stream/*`
      ]
    });
    const role = new Role(this, 'Role', {
      assumedBy: new ServicePrincipal('lambda.amazonaws.com'),
    });
    role.addToPolicy(policyStatement)

  }
}

More security best practices

We also recommend these additional security best practices to maintain optimal security.

Enable WAF

You can use AWS WAF, a web application firewall, to observe web requests that are sent to an Application Load Balancer or Amazon API Gateway. AWS WAF also lets you apply criteria to restrict or approve requests depending on the source IP addresses or the values they include.

Enable Security Hub

AWS Security Hub helps you evaluate your AWS environment compared to security industry standards and best practices, giving you a full view of your security status in AWS. Security Hub also helps you examine your security patterns and pinpoint the most important security risks by gathering security data from all of your AWS accounts, services, and approved third-party products.

Designed to help you manage the security posture of your company, Security Hub supports a number of external compliance framework security standards from various sources, including:

Enable IAM authentication for database instance

Use IAM database authentication to authenticate with and connect to your database instance without entering a password. IAM database authentication is compatible with PostgreSQL, MariaDB, and MySQL. As an alternative, you can use an authentication token.

Use FIPS endpoints

Organizations that must adhere to stringent security and compliance standards can use Federal Information Processing Standard (FIPS) endpoints, particularly when handling sensitive data or operating in regulated sectors. FIPS endpoints offer a high degree of confidence that the cryptographic techniques employed are safe, dependable, and in-line with national regulations.

Conclusion

Congratulations. You’ve learned some important best practices for helping you meet strict security and compliance requirements in your AWS GovCloud (US) environment. We hope you can apply these recommendations to your IaC methodology so you can manage your cloud resources consistently in an automated fashion.

If you have feedback about this article, use the Comments section.

About the authors

Aniket Dekate is an AWS Cloud DevOps Consultant who specializes in cloud infrastructure, automation, containerization and CI/CD. He has extensive experience in building secure, scalable and highly available cloud-native architectures. He has a deep understanding & expertise in designing and implementing cutting-edge cloud solutions using DevOps methodologies. Outside of work, he enjoys playing cricket, badminton and table tennis.

Aarti Rajput is an experienced AWS Lead DevOps Consultant, specializing in designing, implementing, and managing robust and scalable cloud-native architectures. She excels in using cutting-edge DevOps methodologies to streamline operations, enhance security, and ensure the high availability of mission-critical systems. Outside of work, she enjoys traveling the globe and exploring diverse cuisines.

Sandeep Gawande is an AWS Senior Lead Consultant. He specializes in cloud infrastructure, automation, containerization, cloud-native architectures, and generative AI applications. With a background in development and product engineering, he focuses on building resilient, scalable, and reliable cloud-native architectures, particularly in the telecom, finance, and security domains. Outside of work, he enjoys hiking, riding bicycles, and spending time with his family.

Automate monitoring for your Amazon EKS cluster using CloudWatch Container Insights

Manish Garg — Tue, 30 Jul 2024 16:59:00 +0000

Are you looking for a monitoring solution for your Amazon Elastic Kubernetes Service (Amazon EKS) cluster that helps you achieve scalability, reduce errors, and save time and manual effort? For example, consider an Amazon EKS cluster environment that’s configured with multiple worker nodes, each with one or more Kubernetes pods. Teams with similar environments often struggle to find the most efficient way to set up monitoring and alerting capabilities for both system-level pods and worker node performance metrics.

In this article, we present an event-driven, automation solution for monitoring your Amazon EKS cluster using Amazon CloudWatch Container Insights metrics, Terraform, AWS CloudFormation, and other AWS services and resources. Our solution addresses the environments where EKS worker nodes are configured to scale up and down according to demand on your workloads. You create and delete CloudWatch alarms dynamically through analysis of those scaling events. Our approach is based on best practices around Amazon EKS observability, an essential component for understanding and monitoring the health of your Amazon EKS environment.

About this blog post
Time to read	~15 min
Time to complete	~1.5 hour
Cost to complete	Using AWS services may incur costs. See AWS service documentation for details.
Learning level	Intermediate (200)
AWS services	Amazon CloudWatch Amazon Elastic Kubernetes Service (Amazon EKS) Amazon Simple Notification Service (Amazon SNS) Amazon Simple Storage Service (Amazon S3) AWS CloudFormation AWS Identity and Access Management (IAM) AWS Lambda

Understanding the dynamic nature of EKS nodes on the Amazon EKS cluster

Unlike components such as system/application pods and DaemonSet that remain static on the cluster until they’re updated, dynamic components change in response to a variety of factors such as workload demand, upgrades, and patching.

For example, EKS nodes are managed by an Amazon EC2 Auto Scaling group on the Amazon EKS cluster and fluctuate in size when workload demands rise and fall. EKS node instances belong to EKS node groups, and each cluster typically contains one or more of these groups. During a scale-out event, a new node joins the cluster; during a scale-in event, redundant nodes are deleted. Other dynamic components and events such as Amazon EKS upgrades and node patching can also change the number of EKS nodes in an Amazon EKS cluster.

Configuring Amazon CloudWatch alarms is crucial for monitoring EKS nodes, especially given their dynamic nature. Our solution directly addresses this challenge by ensuring that CloudWatch alarms are configured automatically, allowing for efficient monitoring of EKS nodes despite their fluctuating numbers.

Architectural Overview

Our solution deploys the following infrastructure (see Figure 1):

An Amazon EKS cluster and CloudWatch Observability EKS add-on deployed using CloudFormation templates.
CloudWatch static alarms, configured with Terraform.
Dynamic alarms configured for Amazon EKS workloads using AWS Lambda, Amazon SNS, Amazon EventBridge, and Amazon S3.

Figure 1: Architectural overview of automation solution

Static alarms flow

In this flow, Terraform creates the alarms that trigger email notifications when a threshold is breached.

The CloudWatch (Kubernetes) DaemonSet collects the metrics for Amazon EKS clusters and sends the data to CloudWatch Container Insights. The DaemonSet ensures that the CloudWatch agent runs on and collects data from each worker node in the cluster. For a list of extracted metrics, see Amazon EKS and Kubernetes Container Insights.
CloudWatch Container Insights defines the alarm configurations in a Terraform file named terraform.tfvars, located in the root path of our solution’s GitHub repository. This file is used as a reference for creating, deleting, or updating predefined alarms. You can also add more alarms to this file based on specific use cases. You will clone this repository in the walkthrough section of this article.
Terraform creates the CloudWatch alarms specified in the terraform.tfvars file and configures Amazon SNS as an endpoint to trigger email notifications.
When the defined threshold for an alarm is breached, an alert notification event is triggered and sent to the Amazon SNS service.
The Amazon SNS service sends the alarm notifications via email to the designated subscribers, for example your operations team.

Dynamic alarms flow

In this flow, alarms are generated in response to node-scaling activities that occur within the cluster.

The node auto scaler continuously evaluates the scaling requirements of EKS worker nodes and submits the events to the Amazon EC2 Auto Scaling group.
An Amazon EventBridge rule monitors scaling activities and captures EC2 Instance Launch Successful and EC2 Instance Terminate Successful events once they are received by the Amazon EC2 Auto Scaling group.
When an event is matched according to the EventBridge rule, the event triggers a Lambda function that creates and deletes CloudWatch alarms.
The Lambda function evaluates whether the event is a scale-out or scale-in event and either creates or terminates the CloudWatch alarms for the associated nodes.
For scale-out events, the Lambda function creates CloudWatch alarms for the corresponding EKS worker nodes by retrieving the defined alarm attributes from a file named alarm_list_inputs.json, which is stored in an Amazon S3 bucket. For scale-in events, the Lambda function terminates the CloudWatch alarms associated with worker nodes.
The CloudWatch alarm sends the creation/deletion status to Amazon SNS.
Amazon SNS sends alarm notifications via email to the designated subscribers, for example the cloud administrative team.

Prerequisites

An active AWS account.
An Amazon Linux or Mac OS Server. If Mac OS, preferably Z shell (Zsh).
An AWS user/role with sufficient permissions to provision resources using Terraform.
Terraform CLI v1.7.5.
AWS Command Line Interface (AWS CLI) v2.11.1.
Kubectl v1.28.8-eks-ae9a62a (must be compatible with Amazon EKS v1.28)

Walkthrough

Step 1: Deploy the infrastructure and set up your environment

Deploy the Amazon EKS infrastructure and CloudWatch alarms using a combination of CloudFormation and Terraform.

Run the following commands to clone the eks-automated-monitoring GitHub repository:GitHub repository: git clone https://github.com/aws-samples/eks-automated-monitoring.git cd eks-automated-monitoring
Navigate to the ./script/deploy.sh file to update variables to be passed to the CloudFormation templates when provisioning the infrastructure. The following variables are required:
- SNS_EMAIL: Email address for receiving alarms and notifications.
- TF_ROLE: The Amazon Resource Name (ARN) of the IAM role with permission to launch resources into your AWS account.
(Optional) Locate the terraform.tfvars file at the root path of the repository, and either modify predefined alarms or add new ones.
(Optional) Locate the alarm_list_inputs.json file in the files folder at the root path of the repository, and include required alarms for Amazon EKS node-level monitoring. For demonstration purposes, we have included two predefined alarms in this file.
Run the following command to deploy the Amazon Virtual Private Cloud (Amazon VPC) and the Amazon EKS cluster in the us-east-1 Region. If you need to change the region, update the parameter in the ./script/deploy.sh file../scripts/deploy.sh -o apply

Important: Make a note of the name of the Lambda function that’s printed in the output after running this command. You will use this Lambda function in a later step.

Amazon SNS sends a subscription confirmation message to the email address provided in the SNS_EMAIL parameter in the ./script/deploy.sh file. To confirm the subscription, open the email you received from Amazon SNS and choose Confirm subscription. A web page opens and displays a subscription confirmation with your subscription ID.

Now you’re ready to test and verify the CloudWatch alarm configuration.

Step 2 (Conditional): Configure dynamic alarms for existing EKS nodes

If worker nodes already exist in your environment before deploying our solution, run the following command to trigger the Lambda function.

aws lambda invoke --function-name <Lambda function name> --invocation-type RequestResponse output

The Lambda function sets up the CloudWatch alarms that are specified in the list of alarms in the terraform.tfvars file, which was uploaded to an Amazon S3 bucket in .json format for all EC2 instances during deployment.

Step 3: Verify CloudWatch alarms for static components

Sign in to your AWS account.
Open the CloudWatch console.
Using the navigation bar on the left, open the Alarms page.
Verify the predefined and new alarm configurations from the list of alarms in the terraform.tfvars file in your cloned GitHub repository.

Step 4: Setting up alarms for Auto Scaling events

Perform these steps using a role with adequate permissions.

Simulate or test alarms by increasing or decreasing (scaling in and out) the value of the NUM_WORKER_NODES variable in the deploy.sh file.
To apply the updates to the deploy.sh file, run this command:./scripts/deploy.sh -o apply
Sign in to your AWS account.
Open the CloudWatch console.
Using the navigation bar on the left, open the Alarms page to confirm the updates.

When Auto Scaling groups are launched or terminated, the alarms are created, and email alerts are sent to the subscribed email addresses that are associated with Amazon SNS topics configured for notifications.

Cleaning up resources

Perform these steps to clean up your environment and avoid unexpected costs.

To delete the provisioned infrastructure, run this command:./scripts/deploy.sh -o destroy
Delete the dynamic alarms that you created for the EKS worker nodes. For instructions, see Edit or delete a CloudWatch alarm.

Troubleshooting

If you experience alarm creation or deletion failures, or if you don’t receive notification emails, try the following troubleshooting steps:

If you experience failures when creating or deleting CloudWatch alarms, open the AWS CloudWatch log group of the Lambda function and check the message details. For information about accessing the logs using the Lambda console, see Viewing queries on the CloudWatch Logs console.
Use Lambda monitoring functions in the Lambda console to access metrics and graphs such as Error count and success rate (%) and Invocations. For details, see Monitoring functions on the Lambda console.
Update your Amazon EKS Amazon Machine Image (AMI) to the latest version by updating the image ID in the /scripts/eks-infra.yaml file for the EksAmiIds parameter. For details, see Retrieving Amazon EKS optimized Amazon Linux AMI IDs.Note: Our solution uses amazon-linux-2 as the AMI type.

Conclusion

Congratulations! Now you have a working solution for monitoring your Amazon EKS cluster environment based on automation and based practices for Amazon EKS observability. As a next step, we encourage you to learn more about CloudWatch Container Insights metrics by visiting these resources:

If you have feedback about this blog post, use the Comments section below.

About the authors

Manish Garg

Manish Garg is a DevOps Consultant with AWS Professional Services, specializing in migrating and modernizing customer workloads on the AWS Cloud. He possesses a profound enthusiasm for technology, coupled with a keen interest in the realms of DevOps practices.

Sushant Jagdale

Sushant Jagdale is a Senior Consultant with AWS Professional Services. He works with customers to migrate and modernize their workloads to the AWS Cloud.

Navdeep Pareek

Navdeep Pareek is Lead Migration Consultant at AWS. He helps customers migrate and modernize their workloads to the AWS Cloud, specializing in automation and DevOps. In his spare time, he enjoys traveling, cooking, and spending time with family and friends.

IAM Roles Anywhere integration guide for installing AWS Partner CRM Connector

Swaminathan Jayaraman — Fri, 14 Jun 2024 18:07:47 +0000

Since the launch of AWS Partner Customer Relationship Management (CRM) Connector (referred to as the connector in this article), AWS Partner Network (APN) Partners have used it to share opportunities with and receive referrals from Amazon Web Services (AWS) in their Salesforce CRM platform. These partners have provided feedback that they want a more secure user onboarding experience that adheres to their organization security protocols. Specifically, they are concerned about hardcoding AWS Identity and Access Management (IAM) credentials into Salesforce when configuring their AWS Partner CRM Integration using the connector.

In this article, we present a solution that lets AWS Partner CRM Connector users authenticate with Salesforce via their AWS Partner CRM integration environment using AWS IAM Roles Anywhere. Using IAM Roles Anywhere, servers, applications and other workloads outside of AWS can use X.509 digital certificates to obtain temporary AWS credentials to access AWS resources, eliminating the need to hardcode credentials into Salesforce.

About this blog post
Time to read	~20 min
Time to complete	~1 hour
Cost to complete	See AWS Private CA Pricing for details.
Learning level	Advanced (300)
AWS services	AWS CloudFormation AWS IAM AWS IAM Roles Anywhere AWS Partner CRM Connector

Overview

This article provides the following major steps for configuring IAM Roles Anywhere in your AWS account and then integrating it into Salesforce.

Step 1: Configure IAM Roles Anywhere via the AWS Management Console, including setting up resources in your AWS account and downloading the private certificate used to request temporary credentials.
Step 2: Create and submit an ACE CRM onboarding request and then add permissions to the IAM role that provides access to the Amazon S3 bucket used for the ACE CRM integration.
Step 3: Configure IAM Roles Anywhere for Salesforce, including enabling the identity provider, creating a Java KeyStore (JKS) file for storing certificate information, creating a named credential, and then testing the connector integration.

The following image shows how your environment will look after completing these steps and configuring integration settings for the connector in Salesforce.

Prequisites

An AWS account with permissions to create IAM roles and private certificate authority (CA) hierarchies.
Permission to create IAM Roles Anywhere profiles, roles, and trust anchors in the AWS account. For details, see What is AWS Identity and Access Management Roles Anywhere?
A local installation of the latest stable version of OpenSSL. At the time of publishing this article, the latest stable version is the 3.2.x, supported until November 23, 2025.
Eligibility to join the APN Customer Engagements program and create an ACE CRM integration request.

Solution walkthrough

Step 1: Configure IAM Roles Anywhere via the AWS Management Console

In this step, you download and launch a CloudFormation template that automatically adds the following resources to an AWS account in the us-west-2 Region.

AWS Private Certificate Authority and private certificate that includes a custom passphrase and fully qualified domain name.
An IAM Roles Anywhere trust anchor, used to establish trust between your CA and AWS IAM Roles Anywhere.
An IAM role, required by the IAM Roles Anywhere service principal. The provided CloudFormation template creates the role with default sts:GetCallerIdentity, sts:AssumeRole, and sts:GetSessionToken permissions, but you will append additional ones in step 2.
IAM Roles Anywhere profile that specifies the roles that IAM Roles Anywhere assumes and what your workloads can do with the temporary credentials.

Note: If you want to use your own CA-signed certificate, do these steps instead:

Create an IAM Roles Anywhere trust anchor by uploading the certificate body from the external CA.
Configure an IAM role and IAM Roles Anywhere profile. For instructions, see Configure roles.
Import an external CA-signed certificate into Salesforce. For instructions, see Generate a salesforce compatible JKS from PFX or P12.
Create an external credential, and then create a principal for it. For instructions, see Create or Edit an AWS Signature v4 External Credential.
Create a named credential in Salesforce. To learn more, see Authentication Protocols for Named Credentials.

For best practices, see Set up AWS Private Certificate Authority to issue certificates for use with IAM Roles Anywhere.

Step 1-A: Configure resources in your AWS account

Download the CloudFormation template (IAMAnywhere_Setup.yaml) to your local environment.
Log in to the AWS CloudFormation console in the us-west-2 Region (required). You may need to authenticate with the AWS Management Console first.
Note: The us-west-2 Region is where AWS provisions the CRM integration bucket.
Choose Create Stack on the Stacks page.
Choose Upload a template file, and specify the location where you downloaded the CloudFormation template. Choose Next.
On the Specify stack details page, provide a stack name and other required details. Choose Next.
Configure tag keys and values, retention, rollback policy, and other stack options for all resources. Choose Next.
Select I acknowledge that AWS CloudFormation might create IAM resources with custom names, and then choose Submit to create the stack.
After the stack creation process reaches the COMPLETE state, navigate to the Outputs tab of the stack and make note of the Amazon Resource Names (ARNs) of the role, profile, and trust anchor resources.

You can see a list of your new resources in the Resources tab.

Step 1-B: Download the private certificate

Still logged in to the AWS Management Console, navigate to the AWS Certificate Manager.
Choose Certificate ID to open the certificate created by CloudFormation template.
Choose Export to download the certificate.
Provide a passphrase. and make note of it. The passphrase is required in later steps when validating access from your local environment and also when converting the certificate to a Java KeyStore (JKS) file that you will upload to Salesforce.
Choose I understand that ACM will charge my account for exporting this certificate if it has not been previously exported. This charge will reset the period for certificate renewal (13 months). A one-time fee is required for each private certificate issued by AWS Private CA. This fee is charged to the AWS account from which the certificate was incurred. For more information, see AWS Private CA Pricing.
Choose Generate PEM Encoding to generate the certificate body and certificate private key. Download the certificate body to a file named certifcate.txt and the certificate private key to a file named private_key.txt.
In your local directory, rename certifcate.txt to certificate.pem and private_key.txt to private_key.pem. Move the files to the folder of your choice.
In the same folder as the new files, download the AWS credential helper tool provided by AWS IAM Roles Anywhere. Use the download option that’s appropriate for your local machine platform.

Step 1-C: Use certificate to request temporary credentials

Retrieve the ARNs of the IAM role, profile, and trust anchor resources that you created earlier in your AWS account. If needed, you can access the ARNs by logging in to the AWS CloudFormation console, launching the CloudFormation template in the us-west-2 Region, selecting the stack, and navigating to the Outputs tab.

Run the following commands from your local machine terminal to request temporary credentials and test your IAM Roles Anywhere configuration. For information, see Environment variables to configure the AWS CLI. Follow security best practices and root user best practices on your local machine to safeguard AWS account and other sensitive personal information.

//Decrypt the Private key using the password you had set
openssl rsa -in private_key.pem -out decrypted_private_key.pem

//Change permissions of the tool
chmod +x aws_signing_helper

//Command to fetch temporary credentials. Modify the ARN of trust-anchor, profile and role and certificate path with decrypted private key in the below command
./aws_signing_helper credential-process \
      --certificate /path/to/certificate \
      --private-key /path/to/decrypted-private-key \
      --trust-anchor-arn arn:aws:rolesanywhere:region:account:trust-anchor/TA_ID \
      --profile-arn arn:aws:rolesanywhere:region:account:profile/PROFILE_ID \
      --role-arn arn:aws:iam::account:role/role-name-with-path
      
// Example for the above command
./aws_signing_helper credential-process --certificate certificate.pem --private-key decrypted_private_key.pem --trust-anchor-arn arn:aws:rolesanywhere:us-east-1:1234678910:trust-anchor/527c3297-e0b3-4cea-bb17-275f2dcf8af9 --profile-arn arn:aws:rolesanywhere:us-east-1:1234678910:profile/00978968-c5b6-4446-86d7-c6ecf6640e2a --role-arn arn:aws:iam::1234678910:role/marketplacecarole

//Test your credentials, copy and configure your aws credentials using aws configure command 
aws sts get-caller-identity

Now you can use a combination of the AccessKeyId, SecretAccessKey, and SessionToken data types to make AWS API calls. Note that temporary credentials must be changed at various intervals.

You have now successfully configured AWS IAM Roles Anywhere in your AWS account and validated it from your local environment.

Step 2: Create an ACE CRM onboarding request and update the IAM role

In this step, you create and submit an ACE CRM onboarding request using the IAM role that you created in the previous step. You also add permissions to the IAM role that provide access to the Amazon S3 bucket used for the ACE CRM integration, and finally, you amend or add the policy from the request back to the IAM role.

Step 2-A: Create and submit an ACE CRM onboarding request

Retrieve the ARNs of the resources that you created in your AWS account (see Step 1). You can access ARNs by logging in to the AWS CloudFormation console, launching the CloudFormation template in the us-west-2 Region, selecting the stack, and navigating to the Outputs tab.
Submit the onboarding request on Partner Central using the ARN of the IAM role.
Wait for the provisioning request to move to the sandbox.
From the request, download the IAM policy to the sandbox and save it to your local machine.

Step 2-B: Add permissions to the IAM role

Log in to the AWS IAM console.
Choose the IAM role that you created using the CloudFormation template in Step 1. Note: You can also access the IAM role by logging in to the CloudFormation console, launching the CloudFormation template in the us-west-2 Region, selecting the stack, and navigating to the Resources tab.
In the Permissions tab of the role, choose Add Permissions > Create Inline policy.
Switch to the JSON view and swap the policy with the new policy that you saved to your local machine earlier. Choose Next.
Provide a name for the IAM policy, for example ACE-CRM-INT-Sandbox-Policy.
Choose Create Policy.

For best practices about adding an IAM policy to a role, see Validating IAM policies.

Step 3: Configure IAM Roles Anywhere for Salesforce

In this step, you configure IAM Roles Anywhere in your Salesforce account so that the AWS Partner CRM Connector application can access your AWS account and request temporary credentials.

Prerequisites

You must have an installation of Salesforce Spring 2023 release or later (available with instance 242 and higher).
Completion of steps 1 and 2 earlier in this article.

(Optional) Step 3-A: Create a self-signed certificate in Salesforce

If your Salesforce instance is new, you must enable an identity provider on the instance. Otherwise, you can skip this step. For more information, see Identity Providers and Service Providers in the Salesforce documentation.

From the Setup tab in Salesforce, enter Certificate & Key Management in the search field.
Choose Create Self-Signed Certificate.
Provide a label and unique name for the certificate, and choose Save.

Your certificate is now successfully created.

(Optional) Step 3-B: Enable the identity provider in Salesforce

From the Setup tab in Salesforce, enter Identity Provider in the search field.
Choose Enable Identity Provider, and assign the self-signed certificate to the new identity provider.
Select SampleCertificate in the field, and choose Save.

Note: For details about a known issue with importing the Java keystore (JKS) when the identity provider has not yet been enabled, see Issue with importing keystore into Salesforce.

Step 3-C: Convert and upload the private key and certificate files as a .jfk file

In this step, you create a Java KeyStore (JKS) file from the certificate.pem and private_key.pem files that you downloaded from your private certificate in your AWS account.

To generate the PKCS#12 certificate:
1. Open the terminal on your local machine, and navigate to the folder where you stored the certificate.pem and private_key.pem files.
2. Run the following command:
```
openssl pkcs12 -export -in {private certificate pem file} -inkey {private key pem file} -certfile {private certificate pem file} -out {output .p12 file name}

Eg:
openssl pkcs12 -export -in certificate.pem -inkey private_key.pem -certfile certificate.pem -out keystore.p12
```
  Note: After entering your private key decryption passphrase, you will set a new passphrase for the PKCS#12 file. Make note of the new passphrase.

To convert the PKCS#12 certificate file to a JKS file, run the following command:

keytool -importkeystore -srckeystore {.p12 file} -srcstoretype pkcs12 -destkeystore {output .jks file name}-deststoretype JKS

Eg:
keytool -importkeystore -srckeystore keystore.p12 -srcstoretype pkcs12 -destkeystore outputfile.jks -deststoretype JKS

Note: After entering your key decryption passphrase, set a new passphrase for the .jks file.

To update the alias of the certificates within the .jks file, run the following commands. According to Salesforce guidelines, the name must begin with a letter and use only alphanumeric characters and underscores. The name cannot end with an underscore or have two consecutive underscores.

//Check the alias assigned to .jks file

keytool -list -v -keystore outputfile.jks

//In order to successfully use "Import from Keystore" feature available at "Certificate and Key Management", the 'alias' of the certificates within .JKS file must meet following criteria:
"*The name must begin with a letter and use only alphanumeric characters and underscores. The name cannot end with an underscore or have two consecutive underscores.*"

//Update the alias

keytool -keystore outputfile.jks -changealias -alias 1 -destalias IAMRA

//Verify alias update

keytool -list -v -keystore outputfile.jks

To upload the JKS to Salesforce:
1. On the Certificate and Key Management page, choose Import from Key store.
2. Add the outputfile.jks file, and input the key-store password that you created earlier.

Step 3-D: Create the external named credentials

In this step, you configure a named credential in Salesforce to use the IAM role that you created in Step 1.

The named credential is used for authenticating and managing API callouts to external services within Salesforce, to a specific endpoint URL. Initially, you point it to AWS Security Token Service (AWS STS), a web service used to request temporary credentials. Then, you point it to the Amazon S3 service URL. Salesforce named credentials support two variants of the AWS Signature Version 4 authentication protocol—IAM User (identified by access key) and Roles Anywhere.

As a prerequisite for creating the named credential that uses your IAM Roles Anywhere role, you must first configure an external credential to provide the required authentication configuration via IAM Roles Anywhere. For more information, see Create or Edit an AWS Signature v4 External Credential. You also configure a principal that uses the permissions of the IAM role that you configured in Step 1.

For more information about Salesforce named credentials, see Authentication Protocols for Named Credentials and Use AWS Roles Anywhere with Named Credentials.

Create the external credential and a principal

1. From the Setup menu in the search field, enter Named Credentials, then choose Named Credentials.
2. Change to the External Credentials tab.
3. To create a new external credential, choose New. To edit an existing external credential, choose the external credential in the list, and then choose Edit.
4. Complete the following fields, and save the external credential.
  - Label: Name of the external credential, for example AWS IAM Anywhere Credential.
  - Name: Unique identifier used in callout definitions and via the API, for example AWS_IAM_Anywhere_Credential.
  - Authentication Protocol: Choose AWS Signature V4.
  - Service: Name of an AWS service (for example, STS). After the validation step later in this article, you will change this value to S3.
  - Region: This value must be us-west-2. Configuration will fail for other regions.
  - AWS Account ID: (Optional) 12-digit number that identifies your AWS account.
  - Check the Obtain Temporary IAM Credentials via STS: Choose Roles Anywhere (Assume an IAM Role via Certificate).
  - Trust Anchor ARN: ARN for the trust anchor.
    Note: You can also access the IAM Roles Anywhere trust anchor by logging in to the CloudFormation console, launching the CloudFormation template in the us-west-2 Region, selecting the stack, and navigating to the Outputs tab.
  - Profile ARN: ARN of the IAM Roles Anywhere profile.
    Note: You can also access the IAM Roles Anywhere profile by logging in to the CloudFormation console, launching the CloudFormation template in the us-west-2 Region, selecting the stack, and navigating to the Outputs tab.
  - Signing Certificate: A certificate from AWS (via a CA) that’s uploaded to Salesforce.
  - STS Duration: 3600.
5. Create a new principal for your external credential.
  1. With your new external credential selected, scroll to the Principals section and choose New.
  2. Enter a parameter name, for example IAMAW_Principal. Character requirements include [a-zA-Z0-9_+=,.@-]*.
  3. Enter the ARN of the IAM role.
    Note: You can also access the IAM role by logging in to the CloudFormation console, launching the CloudFormation template in the us-west-2 Region, selecting the stack, and navigating to the Outputs tab.
  4. Enter the sequence number.
  5. Save the principal.

Create the named credential

From the Setup menu in the Quick Find box, enter Named Credentials, then choose Named Credentials.
Choose New.
Provide values for the following fields:
- Label: Enter APN API Connection. Note: Do not change this label. Otherwise, the connector won’t recognize the credentials.
- Name: Enter APN_API_Connection.
- URL: Enter https://sts.us-west-2.amazonaws.com. Note: After the validation step later in this article, you will change this value to the Amazon S3 service URL (https://s3.us-west-2.amazonaws.com).
Choose the external credential that you created in the previous step.
Choose the iamra client certificate that was created by importing the JKS file.
Enable the Generate Authorization Header option to generate a header and apply it to any callouts that reference the named credential.
Add the awsapn namespace of the managed app for callouts.
Save the named credential.

Step 3-E: Test and validate the IAM Roles Anywhere integration

Now that you’ve converted the private certificate into JKS, uploaded it to your Salesforce instance, and configured a named credential for the connector, you’re ready to test and validate the integration.

You test the IAM Roles Anywhere role configuration in Salesforce by confirming that the connector application from Salesforce AppExchange can use the named credential principal via the awsapn namespace.

As a prerequisite, create or edit the permission set required for the Salesforce user to access the principal that you created for the external named credential based on AWS Signature Version 4 authentication.

Create or edit a permission set for the principal

In the Salesforce Quick Find box, enter Permission Sets.
On the Permission Sets page, choose New.
Enter the label and API name for the permission set and choose Save.
With the newly created permission set open, choose External Credential Principal Access.
In the Available External Credential Principals field, choose the principal name and move it to the Enabled External Credential Principals field.
Save the permission set and assign it to your Salesforce user.

(Optional) Test the named credential

Open the Developer console in your Salesforce account.
In the Debug menu, choose Open Execute Anonymous Window to open a new window.

Copy and paste the following code into the window and update it with the details of your API and use case.

HttpRequest req = new HttpRequest();
//APN_API_Connection is the name of the named credential
req.setEndpoint('callout:APN_API_Connection/?Action=GetCallerIdentity&Version=2011-06-15');
req.setMethod('GET');
Http http = new Http();
HTTPResponse res = http.send(req);
System.debug(res.getBody());

Choose Execute.
Open the log file to see the status. If successful, the log shows a 200 status code to indicate a successful Amazon STS API call from AWS.

Step 3-F: Update the URL for the named credential

In the Service field, change the value to S3.
In the URL field, enter the following URL: https://s3.us-west-2.amazonaws.com.

Cleanup

If you don’t plan to continue configuring IAM Roles Anywhere, delete the CloudFormation stack to avoid incurring additional costs on your AWS bill. For details, see Deleting a stack on the AWS CloudFormation console.

Conclusion

Congratulations! You’ve successfully configured authentication via IAM Roles Anywhere for your AWS Partner CRM Connector integration between Salesforce and AWS Partner Central. Your security team can rest assured that you no longer have to hardcode IAM user credentials into Salesforce, avoiding a potential security risk.

Now, you’re ready to take the next step and configure integration settings for the connector in your Salesforce environment. For details, see Configure AWS Partner CRM Connector Settings.

If you have comments or feedback about this post, please submit them in the discussion area of this page.

About the authors

Swaminathan Jayaraman

Swaminathan Jayaraman is a solutions architect with AWS Marketplace. He supports buyers when procuring third-party products and sellers when listing their products via AWS Marketplace. He also works with AWS Partners with their Salesforce CRM integration using AWS Partner CRM Connector. He has over 15 years of industry experience in developing and managing large-scale applications, deploying SaaS solutions, and supporting cloud migrations.

Bhavik Pandya

Bhavik Pandya is a systems development engineer for the AWS Partner engineering team. He is passionate about expanding AWS capabilities to customer platforms, and is driven by his love for building systems and products that foster customer adoption and satisfaction. Outside of work, Bhavik enjoys reading books, watching movies, and spending quality time with his family.

Joe Thomas

Joe Thomas is a senior systems development engineer for the AWS Partner engineering team. He is passionate about driving operational excellence, troubleshooting customer issues, and building product features and enhancements that drive customer adoption. In his free time, Joe enjoys spending time with his family on short summer hikes and traveling across the country exploring various food cultures.

Best practices for accelerating development with serverless blueprints

Ran Isenberg — Thu, 13 Jun 2024 17:48:11 +0000

In this article, we present best practices for building great serverless blueprints that promote standardization, scalability, and repeatable deployment patterns throughout your organization. These best practices are founded on the principle that a good serverless blueprint is not just about simplification; it’s an automation tool that seamlessly integrates with testing and continuous integration and continuous delivery (CI/CD) pipelines.

A well-designed serverless blueprint adheres to AWS Lambda handler best practices, ensures observability for real-time insights, and maintains coherent project and Lambda handler structures. In addition, using infrastructure as code (IaC) tools such as AWS Cloud Development Kit (AWS CDK), results in a secure and repeatable development and deployment process.

About this blog post
Time to read	~10 min
Time to complete	~30 min
Learning level	Advanced (300)
AWS services	Amazon API Gateway Amazon DynamoDB AWS Cloud Development Kit (AWS CDK) AWS Lambda

Overview

Serverless blueprints, commonly called templates, help accelerate development while maintaining best practices and quality across the organization. By providing preconfigured templates for quick setup and initial configuration, blueprints significantly speed up adoption of serverless technology and give your team the confidence they need to build robust applications.

In the sections that follow, we walk you through a list of best practices and components that define a great blueprint while exploring a fully working, serverless blueprint.

We use examples and sample code from AWS Lambda Handler Cookbook, an open-source blueprint template for deploying an orders service with AWS CDK. AWS CDK helps you create your infrastructure as code (IaC) using Python or TypeScript and translating it in a format of AWS CloudFormation template. The orders service is available as a skeleton Python service template that uses Amazon API Gateway and AWS Lambda function to create a customer order and then save it to an Amazon DynamoDB table, as shown in the following diagram.

The template shows usage examples of the following options, several of which are covered in the list of best practices that follow:

Synchronous services API Gateway and Lambda.
Database integration with DynamoDB.
Backend components built using AWS CDK and AWS CloudFormation.

Prerequisites

Prerequisites include:

A working knowledge of Amazon DynamoDB.
Prerequisite requirements as listed in AWS Lambda Handler Cookbook. See the Getting Started Guide for a complete list.

Best practices

Learn about best practices as you refer to the AWS Lambda Handler Cookbook for real-world examples.

Project and folder structure

Organize your blueprint repository’s project structure with a serverless mindset. From an overall project perspective, store your infrastructure as code (IaC) files and the Lambda function handlers together in the same project so that your developers can work independently and take more ownership over their code. From a folder structure perspective, store the business domain code (the heart of your project) and the Lambda function handler files in one folder, with the IaC files organized in a separate folder.

Unified observability

As you design your serverless blueprint, take steps to implement unified observability, a methodology that uses monitoring tools to combine data from serverless resources into a single view. To learn best practices for implementing unified observability, see this blog post.

With unified observability, developers can more easily extend and maintain monitoring capabilities when they add new serverless resources. Also, when the same observability tools are used across the organization, developer teams and site reliability engineers (SREs) use a shared language to debug production issues across transactions, resulting in a more streamlined and efficient flow.

The AWS Lambda Handler Cookbook provides an example of a robust unified observability configuration that uses Amazon CloudWatch dashboards and alarms to monitor your resources. The IaC files create a CloudWatch dashboard with logs and metrics that monitor the service’s resources. These files also create CloudWatch alarms that are triggered when a data point, such as a metric’s threshold or an error log, is detected. These alarms send a message to an Amazon Simple Notification Service (SNS) that notifies SREs or developers.

Lambda function patterns

To ensure that you follow best practices for Lambda functions, use hexagonal architecture and observability patterns.

Hexagonal architecture

Hexagonal architecture is a software design pattern that promotes high modularity and a clear separation of external dependencies such as databases or external services in application development. The critical concept of hexagonal architecture is to decouple the core business logic or domain model from these external dependencies using ports and adapters.

Using hexagonal architecture, the codebase is easier to understand, maintain, and test. Changes to external dependencies or user interfaces don’t affect the core business logic, and you can deploy the service in different environments or platforms by simply swapping out the adapters.

The Lambda function used in the AWS Lambda Handler Cookbook follows a hexagonal architecture pattern by structuring the following three architectural layers into individual folders:

Handler (function entry point)
Logic (business domain logic)
Integration adapters (data access layer for storing data in the database)

For more information, see Learn How to Write AWS Lambda Functions with Three Architectural Layers.

Observability

Implement AWS Lambda function observability to measure the current state of your system using logs, tracing, and metrics. While monitoring helps identify issues in your system, observability can help determine their causes.

In the AWS Lambda Handler Cookbook, the Lambda function handler is designed to create a shared and unified observability language by doing the following:

Using a logger library to write JSON logs and a correlation ID to CloudWatch to identify a customer request across services.
Publishing custom CloudWatch metrics for key performance indicators (KPI) capabilities.
Using AWS X-Ray to enable tracing to help identify performance bottlenecks.

In addition, the Lambda function handler uses Powertools for the AWS Lambda (Python), a developer toolkit and library, to implement the following utilities:

Logger to write JSON logs to CloudWatch.
Tracer to generate AWS X-ray traces.
Metrics to write CloudWatch custom metrics.

Powertools for AWS Lambda offers a wide range of best-practice utilities, including an event handler, input validation, error handling. The following code from the cookbook shows you how to embed them into your Lambda function. This create order handler code responds to HTTP POST REST API calls to /API/orders. It includes examples of log writing, custom CloudWatch metric generation, AWS X-Ray trace generation, input validation, and error handling.

from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import Logger, correlation_paths
from aws_lambda_powertools.metrics import Metrics, MetricUnit
from aws_lambda_powertools.tracing import Tracer

from service.logic.create_order import create_order
from service.models.input import CreateOrderRequest
from service.models.output import CreateOrderOutput

app = APIGatewayRestResolver(enable_validation=True)
logger: Logger = Logger()
tracer: Tracer = Tracer()
metrics = Metrics()

@app.post('/api/orders/')
def handle_create_order(create_input: CreateOrderRequest) -> CreateOrderOutput:
# log the incoming request, log the pydantic model
logger.info('got create order request', order=create_input.model_dump())

# write CloudWatch custom metric
metrics.add_metric(name='ValidCreateOrderEvents', unit=MetricUnit.Count, value=1)

   # call the domain layer
   response = create_order(
   order_request=create_input,
   table_name='ranisenberg-deps-Orders-dev-Cruddborders',
   context=app.lambda_context,
   )

   # return the response
   logger.info('finished handling create order request')
   return response

@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
@metrics.log_metrics
@tracer.capture_lambda_handler(capture_response=False)
def lambda_handler(event, context):
# route the request to the appropriate handler
return app.resolve(event, context)

CI/CD pipeline

The CI/CD pipeline is a key element of the serverless template because it gives developers more time to focus on their business domain code and less time to worry about post-deployment errors and security issues. With more time to spend on developing high-quality code, developers experience a much more seamless and smooth process when deploying services to production.

In the AWS Lambda Handler Cookbook, the CI/CD pipeline is designed to ensure security standardization, secret scanning, quality assurance, and testing across serverless applications. Tools such as CDK-nag and CFN-lint scan the AWS CloudFormation template before deployment to identify misconfiguration and security risks. The two primary functions include building the Lambda functions that contain the business domain code and deploying the project’s resources to AWS using the IaC files.

Testing

Always test your serverless applications to validate that your developers have adequately defined and configured the serverless infrastructure and that the business domain code functions correctly. Regardless of which testing approach you choose, your serverless blueprint must include well-written tests and provide coverage for the most important aspects of your organization. If needed, look for predefined blueprint tests so that your developers understand expectations when developing new features.

The AWS Lambda Handler Cookbook defines the CI/CD pipeline for orchestrating the deployment and performing a test run. It includes three primary test types—unit, integration, and end-to-end tests. Your organization might use different names for these tests, but the concepts are similar. Depending on your architecture and the amount of flexibility required, various methods are available to help you implement these tests, for example mocks, ephemeral environments, or the debugging feature included with AWS Serverless Application Model (AWS SAM).

For more information about this testing approach and each test definition used in the AWS Lambda Handler Cookbook, see this AWS re:Invent session recording from 2023.

Conclusion

In this article, you learned about the components of a great serverless blueprint and saw them in action with a real-world template. We hope our approach will serve as a guide as you scale your applications while maintaining best practices, a unified structure, unified observability, and high quality.

For a deeper dive into the cookbook example that we referred to throughout the article, visit the AWS Lambda Handler Cookbook Getting Started guide.

If you have comments about this content, please leave them in the comments area below.

About the authors

Shani Adadi Kazaz

Shani has been working at AWS since 2018 in multiple job roles, including software developer, then technical account manager (TAM), and now AppMod GTM specialist in Israel. Outside of work, Shani loves to travel, cook, and listen to good music.

Ran Isenberg

Ran Isenberg is an AWS Serverless Hero, a Principal Software Architect at CyberArk, a blogger, and a public speaker. He maintains the RanTheBuilder.cloud blog where he shares knowledge and experience in the Serverless world.

Optimize AWS event and log collection using common design patterns

Kishore Vinjam — Fri, 17 May 2024 21:27:23 +0000

If your organization operates within hundreds or thousands of AWS accounts, most likely you’ve been searching for better ways to optimize how you collect and process events and logs from your AWS services. Event and log information is a key component when gaining valuable business insights and providing capabilities like cloud security posture management (CSPM), cloud-native application protection platform (CNAPP), security Information and event management (SIEM), extended detection and response (XDR), and more.

Many AWS customers follow a multi-account strategy to establish their cloud foundation, as described in the topic Organizing Your AWS Environment Using Multiple Accounts. Using this strategy, they can enable multiple AWS services at the AWS Organizations level and then transfer events and logs from these services to a dedicated AWS account. Oftentimes, they use open source, in-house, or AWS Partner tools to visualize the collected data for analysis.

In our experience working with customers, we’ve learned that a top priority for organizations is to collect event and log data in the most efficient and cost-effective way. In this blog post, we discuss some common approaches for collecting data from multiple AWS services across your organization. We also present some common patterns that you can reuse to consume the data. Code snippets are available to help you build similar solutions in your own environment.

About this blog post
Time to read	~7 min
Time to complete	~12 min
Cost to complete	$0
Learning level	Advanced (300)
AWS services	Amazon CloudWatch Amazon EventBridge Amazon OpenSearch Service Amazon Simple Notification Service (Amazon SNS) Amazon Simple Queue Service (Amazon SQS) Amazon Simple Storage Service (Amazon S3) AWS CloudFormation

Overview

This blog post describes the following approaches and patterns for collecting and consuming event and log data.

Approach 1: Collecting logs in a centralized Amazon CloudWatch monitoring account
Approach 2: Collecting logs in an Amazon S3 bucket in a centralized log archive account
- Pattern A: Sending Amazon S3 event notifications through Amazon SQS
- Pattern B: Sending Amazon S3 event notifications through Amazon SNS
- Pattern C: Fanning out Amazon S3 event notifications using a combination of Amazon SNS and Amazon SQS
- Pattern D: Performing centralized logging with Amazon OpenSearch

Prerequisites

You must have a multi-account environment with centralized log collection that is configured to a log archive account. For more information, see What is AWS Control Tower?

Confirm that the following AWS services adhere to security best practices (not covered in this blog post). For more information, see:

Approach 1: Collecting logs in a centralized Amazon CloudWatch monitoring account

As shown in the following diagram, logs from multiple AWS services and accounts are collected in cross-account CloudWatch log groups within a designated AWS monitoring account.

Note: CloudWatch log collection requires separate configuration for each AWS Region.

Approach 2: Collecting logs in an Amazon S3 bucket in a centralized log archive account

As shown in the following diagram, logs and events from multiple AWS services (for example, AWS Config, AWS CloudTrail, Amazon GuardDuty, VPC Flow Logs, and more) within an AWS organization are collected in an Amazon S3 bucket located in a designated AWS log archive account. The mechanism for collecting events and logs varies by AWS service. For details, see the AWS Documentation website, and navigate to each service area.

Note: If using AWS Control Tower, use the centralized log account to collect and store logs. This log account is assigned when you create the landing zone.

When logs are collected from multiple AWS services and accounts into a centralized Amazon S3 bucket, you can use Amazon S3 event notifications to send the logs to partner tools and perform other processing tasks. In the following sections, we describe some common patterns that you can use to configure these processes.

Pattern A: Sending Amazon S3 event notifications through Amazon SQS

When logs are added to a centralized S3 bucket, event notifications are generated and added to an Amazon SQS queue, which is located in the AWS account that contains the S3 bucket. Then, the events are consumed and processed by partner applications.

The following diagram shows the overall flow of this pattern.

When a new data object is created, Amazon S3 event notifications are sent to an Amazon SQS queue from a centralized log location in the log archive account.
The partner engine polls the Amazon SQS queue for new messages. Once the partner engine receives messages, it processes the objects using a cross-account IAM role that’s configured in the log archive account.
The partner engine gets messages and the details of the newly created objects from the Amazon SQS queue.
The partner engine gets the data objects from the S3 bucket so they can be processed.
After the partner engine processes the messages, it alerts Amazon SQS to delete the messages from the queue.

Implementing this pattern is a two-part process:

Part 1: You configure your environment to add message events to the Amazon SQS queue when a new object is created. To try this implementation, deploy this CloudFormation template to a centralized logging account and configure Amazon S3 event notifications to send events to your Amazon SQS queue (created in the same account as the S3 bucket) when a new object is created.
Part 2: You define the way the logs are collected and consumed via your partner engine. To try this implementation, deploy this CloudFormation template in your consumer account. Your partner engine receives messages from the Amazon SQS queue (located in the log archive account). A cross-account IAM role allows your partner engine to read new objects from the S3 bucket.

The partner engine processes the Amazon S3 events and then loads the object data into your data sources. This engine can be hosted locally within your environment for open source and custom-built software, or it can be hosted on partner-owned AWS accounts for SaaS products (most common).

The implementation and architecture of the partner engine can vary. For example, the engine can be serverless or event-driven and can use multiple AWS services and resources such as AWS Lambda, Amazon Elastic Compute Cloud (Amazon EC2), or services with containers.

Pros:

You can use Amazon SQS to batch-process multiple Amazon S3 events before invoking a Lambda function. This improves performance by reducing cold starts and function invocations.
Amazon SQS provides better visibility of the backlog of unprocessed or failed events.
Amazon SQS provides at-least-once processing of messages for standard queues or exactly-once processing when using FIFO (First-In-First-Out) queues.

Cons:

Additional configuration is needed for Amazon SQS to fan out notifications to multiple subscribers simultaneously.
Amazon SQS incurs fees based on the number of requests and amount of data processed.

Pattern B: Sending Amazon S3 event notifications through Amazon SNS

As shown in the following diagram, the architecture of this pattern is almost identical to the previous pattern, except in this case, the events are sent to an Amazon SNS topic rather than processed via an Amazon SQS queue.

When new data objects are created, Amazon S3 event notifications are sent to an Amazon SNS topic from a centralized log location in the log archive account.
When subscribed to the event, the partner engine processes the newly created objects using a cross-account IAM role that’s configured in the log archive account. During this processing stage, the partner engine receives the data objects from the S3 bucket and the details of those objects from the Amazon SNS topic.

Pros:

Amazon SNS can fan out notifications to a large number of subscribers simultaneously with high throughput.
Amazon SNS doesn’t require processing fees.

Con: SNS provides at-least-once delivery to subscribers but cannot guarantee that subscribers receive messages when client- and/or server-side errors occur.

To try this pattern, deploy this CloudFormation template in the centralized logging account and configure an Amazon S3 event notification to send messages to an Amazon SNS topic (in the same account) when a new object is created in the S3 bucket.

Pattern C: Fanning out Amazon S3 event notifications using a combination of Amazon SNS and Amazon SQS

When logs are added to a centralized S3 bucket, event notifications are generated and added to an Amazon SQS queue, which is located in the AWS account that contains the S3 bucket. Next, the events are sent to an Amazon SNS topic through an Amazon EventBridge pipe. This allows multiple engines to consume the logs.

Note: The dynamics involved in consuming these events and processing them is beyond the scope of this blog.

Amazon S3 event notification messages are added to an Amazon SQS queue from a centralized log location in the log archive account.
Messages are routed from the Amazon SQS queue to an EventBridge pipe.
The Amazon SNS topic receives the messages from the EventBridge pipe.
The Amazon SNS topic fans out the messages to multiple subscribers, for example via queues that serve partner applications or a Lambda function configured to process the messages.
Messages are consumed by subscribers who are either part of a customer-owned tooling account or partner-hosted application environment. You can extend this step with additional logic for consuming and using logs with applications to gain additional insights.

To try this pattern, deploy this CloudFormation template in the centralized logging account and configure an Amazon S3 event notification to send messages to an Amazon SQS queue when a new object is created, and then to an Amazon SNS topic to fan out the message to multiple subscribers. Subscribers can be third-party partner tools or other partner-owned AWS accounts.

Pro: This approach provides better fan-out capability if you need to consume the logs through multiple tools.

Con: Using additional AWS services like EventBridge incurs more cost.

Pattern D: Performing centralized logging with Amazon OpenSearch

You can use OpenSearch to perform centralized logging for multiple AWS services and features such as AWS CloudTrail, VPC Flow Logs, and others. You can also use OpenSearch to log and monitor your AWS applications. For details, see Centralized Logging with OpenSearch.

Cleanup

If you don’t use the solution, or you decide to uninstall it, delete the CloudFormation stack to avoid incurring additional costs. For details, see Deleting a stack on the AWS CloudFormation console.

Conclusion

In this blog post, we introduced you to some common approaches for collecting logs from multiple AWS services in a centralized location, allowing you to unlock valuable insights. You also saw some common design patterns for consuming logs and learned about the pros and cons associated with each approach.

We encourage you to use the provided sample CloudFormation templates as a reference when planning how to channel your centralized logs to third-party or open-source tools. If you have comments or feedback about this post, please submit them in the discussion area of this page.

About the authors

Kishore Vinjam

Kishore is a principal solutions architect focusing on cloud operations services. He is passionate about working in cloud technologies and building solutions for customers. When not working, he likes to spend time with his family, hike, and play volleyball and ping-pong.

Gabriel Costa

Gabriel is a senior partner solutions architect at AWS, working with AWS Partners and customers on all things cloud operations. Outside of work, he enjoys playing the guitar, reading about philosophy, watching sci-fi and anime, and searching with his wife for the new cool restaurant in town.

ACE mapping guide for AWS Partner CRM Connector Version 2.0

Bhavya Chugh — Wed, 15 May 2024 20:11:23 +0000

When using the AWS Partner CRM Connector (referred to as the connector), partners must create APN Customer Engagements (ACE) mappings on the connector to sync opportunities between AWS and Salesforce. Because partners use different Salesforce database tables (called objects) for storing opportunity data, methods for creating ACE mappings can vary.

In this blog post, I present three different ways to complete ACE mappings for the connector. Choose your preferred way to create mappings based on your specific environment and requirements.

Option 1: Use the ACE opportunity custom object that is packaged with the connector.
Option 2: Create a new custom object or use the Salesforce standard opportunity object.
Option 3: Populate and map data from another object using a trigger.

Finally, we describe how to upload your opportunities into ACE in bulk using ACE Pipeline Manager.

About this blog post
Time to read	~10 min
Time to complete	~20 min
Learning level	Intermediate (200)
AWS services	Amazon Partner CRM Connector AWS Pipeline Manager

Prerequisites

Read the AWS Partner CRM Integration Guide to understand business flows.
Complete Stage 1: Onboarding prerequisites.
Complete the steps in Stage 2: Request submission to fill out the Onboarding Request form on Partner Central.
Follow the Guided setup instructions to install the CRM Connector on the sandbox Salesforce environment and configure authentication.

Option 1: Use the ACE opportunity custom object that’s packaged with the connector

Use the AWS Partner CRM Connector’s Automap ACE custom object in the opportunity submission process to surface validations in the UI. We recommend this option over the other two because the Automap ACE object is automatically included when you install the connector from the Salesforce AppExchange. Also, the schema of the Automap ACE object matches the latest ACE data model described in the AWS Partner CRM Integration samples GitHub repository.

To map the Automap ACE object within AWS Partner CRM Connector Version 2.0:

In the ACE Mapping tab (see Figure 1), choose Opportunity.
In the Object Selector field, choose the ACE Opportunity object.
Choose Automap ACE object.

Figure1: ACE Mapping tab

After the connector maps the fields, the page automatically reloads and you can review the map values for the picklist and multi-picklist fields to confirm that they have been created successfully. The next step is to create the schedule so Salesforce can sync the opportunities.

Note: If all data resides in the new object, no further work is required. If the data resides in another object, you may need to build triggers to move the data to the new object. See Option 3 later in this blog post for more information.

Option 2: Create a new custom object or use the Salesforce standard opportunity object

Using this option, you can either create your own custom object or use the standard opportunity object provided with Salesforce. In either scenario, confirm that you have included all ACE mandatory fields such as Customer Company Opportunity, Industry, and Country.

To map the standard opportunity object:

In the ACE Mapping tab, choose Opportunity.
In the Object Selector field, choose the Opportunity object. Note that the Automap ACE object is available only for the ACE Opportunity custom object provided by AWS, as described in Option 1.
Map the required Salesforce fields to the AWS fields. For picklist fields, map Salesforce values with ACE Pipeline Manager values. Choose Save.

The data type of the target mapping field must match the data type of the newly created field in the custom object. Otherwise, the new field might not display on the ACE Mapping screen. For example, a text field like customerCompanyName must be mapped to a text (string) field in the Salesforce Object Manager. For a complete list of ACE Opportunity fields, see Opportunity Fields.

Note: If creating a custom object or updating the Salesforce standard object, you are responsible for maintaining and updating your object as changes occur within your organization.

Option 3: Populate and map data from another object using a trigger

Use this option if your data is sourced from more than one object. For example, if you store opportunity data in one object but want to use the ACE opportunity custom object to perform the ACE mapping. Because Salesforce requires a single object to map fields to ACE, this scenario requires an automation trigger or workflow to populate a new custom object with the combined data.

Consider a scenario when the partner wants to map the Customer Company Name and Industry fields from the parent account of an opportunity but source the Use Case, Target Close Date, and Amount fields from the opportunity itself. This partner can first create a trigger that moves data from a custom object to a staging object. Using this trigger, the fields in the staging object are populated with data from both the account and opportunity, allowing the partner to map those fields to ACE within Salesforce.

Use the following snippet of code as a starting point when building and running a similar trigger on the Salesforce backend.

Note: The ACE CRM team does not provide support for building or testing triggers. Work with your CRM administrative team instead.

Trigger OpportunityTrigger on Opportunity (after insert)
{
   //In this case a new Opportunity needs to be transmitted to ACE
   List<awsapn_ACE_Opportunity_c> objectsToInsert = new List< awsapn_ACE_Opportunity_c>();
    if( trigger.isAfter && trigger.isInsert){
        for(Opportunity newOpportunity : trigger.new){
            awsapn_ACE_Opportunity newaceOpportunity = new awsapn_ACE_Opportunity_c();
            newaceOpportunity. awsapn__Customer_Company_Name__c = newOpportunity.Account.Name;
            newaceOpportunity. awsapn__Industry__c = newOpportunity.Account.Industry;
            newaceOpportunity. awsapn__Target_Close_Date__c = newOpportunity.TargetCloseDate;
            newaceOpportunity.awsapn__Expected_Monthly_AWS_Revenue__c = newOpportunity.Amount;
            //...Other fields sourced from opp or parent account
            objectsToInsert.add(newaceOpportunity);
        }
    }
   insert objectsToInsert;
}

Once implemented, confirm that the trigger successfully populates the data correctly in the ACE opportunity custom object provided with the connector. You should see the required fields and the correct data type used in the field definition. Next, perform the same mapping in the ACE Mapping tab.

AWS Partner Amplitude has successfully implemented this option by completing the following tasks:

As a prerequisite, created a spreadsheet that mapped fields from their standard opportunity object to the custom ACE Opportunity object delivered with the connector.
Built a custom Apex class job to move data from their CRM opportunity and its parent account object to the ACE custom opportunity object.
Configured the Apex code with a trigger to sync the required opportunities daily.
Created a batch query file to specify the use case for the synchronization and provide a limit to the number of opportunities to sync to AWS.

Bulk importing opportunities via the AWS ACE Pipeline Manager

After completing the ACE mapping using one of the three options described earlier, you can upload your opportunities into ACE in bulk using ACE Pipeline Manager. Using ACE Pipeline Manager, you can manage your pipeline of AWS customers while simultaneously collaborating with AWS on those engagements.

Figure 2 illustrates the bulk upload process within ACE Pipeline Manager.

Figure 2: AWS Pipeline Manager bulk upload process

Data from a standard or custom object in the partner’s Salesforce instance is passed to the partner’s opportunity extract file. Next, the opportunity extract file is transformed and converted into an ACE-formatted file, which is used to upload data into ACE Pipeline Manager.

To ensure success, review your existing manual workflow for bulk-importing opportunities into ACE. Review the existing guidelines and standard operating procedures (SOPs) that your AWS alliance lead or ACE Pipeline Manager users follow to export opportunities from your Salesforce organization and map them to CSV format (required for bulk import). Analyzing how existing values are transformed to CSV format is a key factor for establishing the field mappings required for automating the CRM integration with ACE.

Conclusion

We hope this blog post has provided you with some valuable insights and guidance for using the AWS Partner CRM Connector to complete ACE mappings and sync opportunities to Salesforce. If you have any feedback about this content or the options described, please leave a comment below.

About the author

Bhavya Chugh

Bhavya Chugh is an AWS Partner solutions architect with Amazon Web Services. She helps partners adopt AWS best practices for creating resilient architectures and building strategic content, accelerating their AWS partner journey. Recently, she has worked with multiple ACE-eligible partners to onboard them on ACE CRM integration projects.

Automate comprehensive data protection using AWS CloudFormation StackSets

Denton He — Fri, 26 Apr 2024 21:42:54 +0000

Automating data protection is critical for streamlining security processes, reducing human errors, proactively mitigating potential threats, and ultimately enhancing data security posture and resilience at scale. As customers look for ways to automate and simplify data protection at scale across all AWS accounts, they sometimes use different tools and custom scripts to automate backups of resources, leaving administrators with limited visibility and control.

Even if administrators have access to account data, understanding and correctly modifying existing backup settings across different tools can be difficult and error-prone. A workaround is implementing blanket organization-wide backup policies. However, using this approach duplicates existing backups and can incur significant storage expenses.

In this post, we show you a simpler solution for ensuring data protection without duplicating existing backups on Amazon Elastic Block Store (Amazon EBS), a high-performance block-storage service designed for Amazon Elastic Compute Cloud (Amazon EC2). Using default policies, a new capability in Amazon Data Lifecycle Manager, you can implement account-level policies that create new Amazon EBS-backed Amazon Machine Images (AMIs) for Amazon EC2 instances and/or Amazon EBS Snapshots for EBS volumes only when recent backups don’t already exist. Better yet, default policies don’t produce new Amazon EBS-backed AMIs or snapshots for the targeted resource(s) if your organization currently employs another method for backup creation.

Previously in the Simple and comprehensive data protection with Amazon Data Lifecycle Manager blog post, we showed you a solution that backs up resources running critical workloads for individual AWS Regions and accounts. Now, we extend this data protection capability to all Amazon EC2 instances and Amazon EBS volumes in all Regions and accounts across your organization or organizational unit (OU) using AWS CloudFormation StackSets. Our goal is to help administrators save hours of management overhead while ensuring that all users and teams within the organization comply with minimum backup requirements.

Note: If you prefer to use StackSets with self-service permissions, or if you want to deploy default policies in multiple Regions and accounts that don’t belong to an organization or OU, see What is Amazon Elastic Block Store?

About this blog post
Time to read	~15 min
Time to complete	~40 min
Learning level	Advanced (300)
AWS services	Amazon Data Lifecycle Manager Amazon Elastic Block Store (Amazon EBS) Amazon Elastic Compute Cloud (Amazon EC2) Amazon CloudFormation StackSets AWS Command Line Interface (AWS CLI)

Solution overview

Our solution shows you how to enable and manage Amazon Data Lifecycle Manager default policies in all Regions in a sample organization (named r-ahrb) within your AWS Organizations account management service. In our example, one of the member accounts (named test-8) already has default policies enabled in the us-east-2 Region.

In the walkthrough, you perform the following steps:

Create default policies for the stack set. Using a sample template, you configure multiple options and parameters.
Modify default policies for the stack set by updating the creation frequency of default policies to every two days.
Optionally deploy the CloudFormation template using a CI/CD pipeline.

To clean up your environment, we show you how to delete the default policies for the stack set.

Prerequisites

Before you enable default policies across all AWS accounts and Regions in your organization or OU, first set up the following required permissions for creating your stack set with self-managed permissions:

Enable all features in your organization, not just consolidate billing features.
As administrator of the management account, activate trusted access with AWS Organizations.

This permission allows the management account and delegated administrator accounts to create and manage service-managed stack sets for the organization.

Note: To create default policies in Regions created after March 20, 2019, you must enable the Regions in those accounts before you can use stack sets to create default policies. For details, see Performing stack set operations involving regions that are disabled by default.

Walkthrough

Step 1: Create default policies for the stack set

Sign in to your AWS Organizations management account as an administrator.
Navigate to the AWS CloudFormation console.
From the navigation pane, choose StackSets.
At the top of the StackSets page, choose Create StackSet.
Under Permissions, choose Service-managed permissions.
Under Prerequisite – Prepare template, choose Use a sample template.
To enable protection, do one of the following:
- For Amazon EBS volumes, under Select a sample template, choose Create and manage default policies for EBS Snapshots.
- For Amazon EC2 instances, in the same menu as the previous step, choose Create and manage default policies for EBS-backed Amazon Machine Images (AMIs).
Choose Next.
On the Specify StackSet details page:
1. Provide a name and description to help you identify the stack set. In this post, since we’re creating a default policy to protect Amazon EBS snapshots, our stack set is named DLM-default-policy-snapshots with description DLM Default Policy for EBS snapshots.
2. In the Parameters section, update the following values:
  - (Recommendation) CreateInterval: For critical workloads, set the value to 1 (daily).
  - (Recommendation) RetainInterval: For critical workloads, set the value to 7 (seven days).
  - Description: Provide a description of the default policy. Our description is Default Policy for EBS Snapshot management – created by CloudFormation.
  - State: Set to ENABLED.
3. Define optional parameters that govern which volume types to exclude and the Regions to copy the created resources.
4. (Optional) ExcludeVolumeTypes: If you have accounts that run both critical workloads (for example, they use gp3 and io2 Block Express) and non-critical/test workloads (for example, they use st1), and you don’t need backups of the non-critical workloads, specify values to exclude all st1 volumes.Note: To exclude volumes and instances based on tags, create a copy of the stack sets template, add tag(s) of target resources to exclude, and then deploy the custom stack sets template. Default policies don’t back up excluded resources.
5. Choose Next.
On the Configure StackSet options page:
1. Add tags to help you identify the stack resources.
2. In the Execution configuration section, for Managed execution, select Active.
3. Choose Next.
On the Set deployment options page:
1. In the Add stacks to stack set section, select Deploy new stacks.
2. In the Deployment targets section, select either Deploy to Organization or Deploy to organizational units (OUs), depending on your requirements.
3. In the Auto-deployment options section, set the Automatic deployment option to Activated.
4. For the Account removal behavior option, select Retain stacks to ensure that default policies continue to create backups even after the account is removed from the OU.
5. (Recommendation) In the Specify regions section, choose Add all regions to ensure that resources in all AWS Regions in the organization are protected by default policies.
6. For Deployment options:
  1. For Maximum concurrent accounts, set the value to 10.
  2. For Failure tolerance, set the value to 10. However, if you have 20 accounts in your organization that already have default policies enabled in at least one Region, set the value to a higher number such as 40 so that CloudFormation continues to create default policies in other accounts and Regions. For more information, see Concurrency Mode parameter and Stack set operation options.
  3. For Region concurrency, select Parallel.
  4. For Concurrency mode, select Soft failure tolerance.
7. Choose Next.
On the Review page, review your choices, select I acknowledge that AWS CloudFormation might create IAM resources, and choose Submit. AWS CloudFormation may create IAM role(s) to perform actions required by the default policy.

To see the stack set deployment details, open the Operations tab. When the status changes to SUCCEEDED, the default policies are now created in all accounts and Regions. For more information, see StackSets status codes.

To monitor the creation of stack sets across individual accounts and Regions, open the Stack instances tab.

As we mentioned earlier, the test-8 account already has default policies enabled in the us-east-2 Region. Therefore, we expect the stack instance to fail for that account and Regiony, as shown in the following image. However, given that the total number of failures (1) is less than the failure tolerance number (10) that was set earlier, the stack sets continue to create default policies in all other accounts and Regions.

Step 2: Modify default policies for the stack set

Modify default policy settings across multiple AWS Regions and accounts to update the creation frequency of default policies to every two days.

Sign in to your AWS Organization management account as an administrator.
Navigate to the AWS CloudFormation console.
From the navigation pane, choose StackSets.
Choose the DLM-default-policy-snapshots policy. In the Organizational unit IDs section, make note of the r-ahrb ID. You will use this ID in a later step to propagate updates.
Choose Actions, Edit StackSet details.
On the Choose a template page:
1. In the Permissions section, leave the Service-managed permissions selection. This option is selected if you previously deployed the stack set in your organization/organizational unit.
2. In the Prerequisite – Prepare template section, select Use current template.
3. Choose Next.
On the Specify StackSet details page, in the Parameters section, change the CreateInterval value to 2. Leave all other fields unchanged. Choose Next.
On the Configure StackSet options page, leave all fields unchanged, and choose Next.
In the Set deployment options page:
1. In the Organizational units (OUs) section, enter r-ahrb. This is the same OU ID that you made note of earlier. This ID ensures that the number of days between consecutive backups (2) is propagated to all accounts in the AWS Regions specified for that OU. If you add another OU, the default policy is also created in the accounts in the new OU.
2. In the Specify regions section, select Add all regions.
3. In the Deployment options section, provide the same values that you entered for step 11 earlier in the Step 1: Create default policy in the stack set section. Choose Next.
On the Review page, review your choices, select I acknowledge that AWS CloudFormation might create IAM resources, and choose Submit. AWS CloudFormation may create IAM role(s) to perform actions required by the default policy.

After all stack instances are updated, all accounts and AWS Regions have default policy with a creation frequency set to every two days and a retention frequency of even days.

Note: Deploying the stack set updates the default policies in all AWS Regions and accounts in the organization or OU. If you have previously manually modified a specific default policy in an Region or account, the updated stack set will overwrite those manual changes.

(Optional) Step 3: Deploy CloudFormation template using CI/CD pipeline

To deploy or update your CloudFormation StackSets template using a CI/CD pipeline, using the StackSets template from this walkthrough, follow the instructions in Building a CI/CD pipeline to update an AWS CloudFormation StackSets.

Cleaning up

If you no longer want to apply Amazon Data Lifecycle Manager default policies to all the accounts in your organization or OU, you can use stack sets to delete the policies.

Note: The following procedure deletes all default policies that are created through stack sets. not policies created by other methods. Since our test-8 account already has a default policy in the us-east-2 Region, it won’t be deleted after performing the following steps.

(Optional) Update policies to ensure that Data Lifecycle Manager continues to delete all Amazon EBS snapshots and Amazon EBS-backed AMIs after the default policies that created them are deleted. Otherwise, you must manually delete the snapshots and AMIs from your account.
1. Repeat the steps in the Step 2: Modify default policies for the stack set section with one modification. When you reach the Specify StackSet details page, in the Parameters section, set the ExtendDeletion value true.
2. Submit the updated stack set and wait until all instances are updated.
Repeat steps 1-4 in the Step 2: Modify default policies for the stack set section.
Choose Actions, Delete stacks from StackSet.
On the Set deployment options page:
1. In the Organizational units (OUs) section, enter r-ahrb.
2. In the Specify regions section, select the Regions where you want to delete the default policies.
3. In the Deployment options section, clear the Retain stacks option to ensure that the stacks are deleted.
Choose Next and then Submit. After all operations have completed, the default policies in your accounts are deleted.
(Optional) If you don’t need to create default policies with the stack set in the future, delete the stack set:
1. From the navigation pane, choose StackSets.
2. Choose the DLM-default-policy-snapshots policy.
3. Choose Actions, Delete StackSet.
4. Choose Delete to confirm the deletion.

Conclusion

In this post, you learned how to use stack sets to create Amazon Data Lifecycle Manager default policies in all AWS Regions and accounts in your organization. Default policies provide comprehensive data protection by regularly backing up your Amazon EC2 instances and/or Amazon EBS volumes, eliminating the need to duplicate resources that may incur storage costs. We hope that this solution offers you peace of mind that you’ve met your data protection requirements with all critical workloads in your organization backed up, regardless of different processes used in individual accounts.

Next steps

Ready to try our solution in your own environment? Delegate administrators can use either the CloudFormation console or the AWS CLI to get started.

If you need a refresher about stack sets, see AWS CloudFormation StackSets for customers managing multiple accounts with AWS organizations. You can also learn more about Data Lifecycle Manager default policies by reading the following resources:

Simple and comprehensive data protection with Amazon Data Lifecycle Manager (blog post)
Default policies (AWS product documentation)

Thank you for reading this blog post. If you have questions or suggestions, leave them in the comments section.

About the authors

Vyassa Baratham

Vyassa is a software development engineer for Amazon EBS. He likes to build robust, maintainable solutions to complex problems. In his spare time, he enjoys cooking, running, skiing, and playing with his cat Poppy.

Rahul Garg

Rahul is a principal product manager with AWS CloudFormation. He has high level of curiosity, especially around understanding humans, including himself. Rahul is currently reading “Never Split the Difference” and listening to the “Knowledge Project” podcast.

Denton He

Denton is a senior product manager for Amazon EBS and leads the product for automation of EBS features. He is committed to helping users automate and simplify their workload processes running on Amazon EC2 and Amazon EBS, including self-managed databases, streaming workloads, and AI/ML.

Save time converting Terraform templates to AWS using Amazon Bedrock

Troy Ameigh — Fri, 15 Mar 2024 21:20:40 +0000

If you’ve spent time migrating non-AWS Terraform templates to the Amazon Web Services (AWS) Cloud, you know it can be a time-consuming, manual process with ample room for error. Most organizations would rather allocate engineering resources toward higher-value, more impactful initiatives such as cloud modernization.

With all the latest rage about generative artificial intelligence (AI), perhaps you’ve thought about looking into a possible AI solution for automating your template conversion projects. If so, keep reading.

In this article, I show you two examples of how to convert a sample Microsoft Azure-based Terraform template to AWS using Amazon Bedrock, a fully managed service for building generative AI applications on AWS. Amazon Bedrock offers a wide range of foundation models from leading AI companies via a single API. And, because it’s serverless, you can quickly customize models using your own data and integrate them into your environment without infrastructure management.

About this blog post
Time to read	~10 min
Time to complete	~40 min
Learning level	Advanced (300)
AWS services	Amazon Bedrock AWS Command Line Interface (AWS CLI) Amazon Software Development Kit (SDK) for Python (Boto3)

Solution overview

My solution provides two walkthrough exercises. In the first one, I show you how to use the Amazon Bedrock console to convert a sample non-AWS Terraform template into one designed for the AWS Cloud. It uses the Anthropic Claude version 2.1 foundation model, but you can try it using other supported models when you’re ready to explore further. For a complete list, see Supported foundation models in Amazon Bedrock.

While the first walkthrough provides step-by-step instructions for converting a template using the console, the second one is designed for organizations with a large catalog of templates. To address this more realistic scenario, you perform the conversion via a Python script, setting you up for a pipeline conversion workflow down the road. Once the template is converted, the only remaining task is to review the code for organization-specific parameters. The key is automation, and the goal is massive time savings.

Prerequisites

Before getting started with either walkthrough, ensure that you have the following:

An AWS account. If you don’t have one, sign up at https://aws.amazon.com.
A user account with permissions to access Amazon Bedrock. For a list of permissions, see Use the Amazon Bedrock console. For general information about configuring AWS Identity and Access Management (IAM), see Adding and removing IAM identity permissions.
An installation of Terraform.
A basic understanding of Amazon Bedrock.

In addition, to complete the second walkthrough, ensure that you have the following on your local machine:

An installation of AWS CLI using the same IAM credentials that you used for the first walkthrough. For more information, see Authenticate with IAM user credentials.
An installation of Python (version 3 or higher).
An installation of AWS SDK for Python (Boto3).

Walkthrough 1: Converting a Terraform template using the Amazon Bedrock console

Log in to your AWS account as a user with Amazon Bedrock permissions.
Sign in to the AWS Management Console and open the Amazon Bedrock console in US West (Oregon) Region.
In the search bar, enter Amazon Bedrock.
In the left navigation pane, under Playgrounds, choose Text.
Choose Select model. (Note: If you don’t see any foundation models, you may need to request access. For details, see Model access.)
Choose Anthropic, Claude, v2.1 FM. For the Throughput option, leave the default setting On-demand.
Choose Apply. Now you can interact with Amazon Bedrock using the Anthropic Claude v2.1 foundation model.
In the text window, enter the following prompt:
```
convert to work on AWS and output as a template
```
Note: Be sure to use this exact prompt in the text window so that Amazon Bedrock outputs a complete template. For example, if you shortened the prompt to convert to work on AWS, Amazon Bedrock would output a list of proposed changes, but the template would be incomplete.

In the text window, enter the following content. (Note: If you complete the second walkthrough, you will use this same template.)

# Create a resource group if it doesn't exist
resource "azurerm_resource_group" "blogterraformgroup" {
    name     = "blogResourceGroup"
    location = "westus"

    tags = {
        environment = "Amazon Bedrock Terraform Conversion Blog"
    }
}

# Create virtual network
resource "azurerm_virtual_network" "blogterraformnetwork" {
    name                = "blogVnet"
    address_space       = ["10.0.0.0/16"]
    location            = "westus"
    resource_group_name = azurerm_resource_group.blogterraformgroup.name

    tags = {
        environment = "Amazon Bedrock Terraform Conversion Blog"
    }
}

# Create subnet
resource "azurerm_subnet" "blogterraformsubnet" {
    name                 = "blogSubnet"
    resource_group_name  = azurerm_resource_group.blogterraformgroup.name
    virtual_network_name = azurerm_virtual_network.blogterraformgroup.name
    address_prefixes       = ["10.0.1.0/24"]
}

# Create public IPs
resource "azurerm_public_ip" "blogterraformpublicip" {
    name                         = "blogPublicIP"
    location                     = "westus"
    resource_group_name          = azurerm_resource_group.blogterraformgroup.name
    allocation_method            = "Dynamic"

    tags = {
        environment = "Amazon Bedrock Terraform Conversion Blog"
    }
}

# Create Network Security Group and rule
resource "azurerm_network_security_group" "blogterraformnsg" {
    name                = "blogNetworkSecurityGroup"
    location            = "westus"
    resource_group_name = azurerm_resource_group.blogterraformgroup.name

    security_rule {
        name                       = "SSH"
        priority                   = 1001
        direction                  = "Inbound"
        access                     = "Allow"
        protocol                   = "Tcp"
        source_port_range          = "*"
        destination_port_range     = "22"
        source_address_prefix      = "*"
        destination_address_prefix = "*"
    }

    tags = {
        environment = "Amazon Bedrock Terraform Conversion Blog"
    }
}

# Create network interface
resource "azurerm_network_interface" "blogterraformnic" {
    name                      = "blogNIC"
    location                  = "westus"
    resource_group_name       = azurerm_resource_group.blogterraformgroup.name

    ip_configuration {
        name                          = "blogNicConfiguration"
        subnet_id                     = azurerm_subnet.blogterraformsubnet.id
        private_ip_address_allocation = "Dynamic"
        public_ip_address_id          = azurerm_public_ip.blogterraformpublicip.id
    }

    tags = {
        environment = "Amazon Bedrock Terraform Conversion Blog"
    }
}

# Connect the security group to the network interface
resource "azurerm_network_interface_security_group_association" "example" {
    network_interface_id      = azurerm_network_interface.blogterraformnic.id
    network_security_group_id = azurerm_network_security_group.blogterraformnsg.id
}

# Generate random text for a unique storage account name
resource "random_id" "randomId" {
    keepers = {
        # Generate a new ID only when a new resource group is defined
        resource_group = azurerm_resource_group.blogterraformgroup.name
    }

    byte_length = 8
}

# Create storage account for boot diagnostics
resource "azurerm_storage_account" "blogstorageaccount" {
    name                        = "diag${random_id.randomId.hex}"
    resource_group_name         = azurerm_resource_group.blogterraformgroup.name
    location                    = "westus"
    account_tier                = "Standard"
    account_replication_type    = "LRS"

    tags = {
        environment = "Amazon Bedrock Terraform Conversion Blog"
    }
}

# Create (and display) an SSH key
resource "tls_private_key" "example_ssh" {
  algorithm = "RSA"
  rsa_bits = 4096
}
output "tls_private_key" { 
    value = tls_private_key.example_ssh.private_key_pem 
    sensitive = true
}

# Create virtual machine
resource "azurerm_linux_virtual_machine" "blogterraformvm" {
    name                  = "blogVM"
    location              = "westus"
    resource_group_name   = azurerm_resource_group.blogterraformgroup.name
    network_interface_ids = [azurerm_network_interface.blogterraformnic.id]
    size                  = "Standard_DS1_v2"

    os_disk {
        name              = "blogOsDisk"
        caching           = "ReadWrite"
        storage_account_type = "Premium_LRS"
    }

    source_image_reference {
        publisher = "Canonical"
        offer     = "UbuntuServer"
        sku       = "22.04-LTS"
        version   = "latest"
    }

    computer_name  = "blogvm"
    admin_username = "azureuser"
    disable_password_authentication = true

    admin_ssh_key {
        username       = "azureuser"
        public_key     = file("~/.ssh/id_rsa.pub")
    }

    boot_diagnostics {
        storage_account_uri = azurerm_storage_account.blogstorageaccount.primary_blob_endpoint
    }

    tags = {
        environment = "Amazon Bedrock Terraform Conversion Blog"
    }
        
    connection {
        host = self.public_ip_address
        user = "azureuser"
        type = "ssh"
        private_key = "${file("~/.ssh/id_rsa")}"
        timeout = "4m"
        agent = false
    }

In the Configurations section, adjust the slider to set the value of Maximum Length to 2048.
Choose Run.
Review the converted template, make a note of the list of changes, and update additional items in the output as needed.

Sample output:

 Assistant:
 Here is the converted Terraform template to deploy similar infrastructure on AWS:

```
# Create VPC
resource "aws_vpc" "blog_vpc" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name = "Blog VPC" 
  }
}

# Create Internet Gateway
resource "aws_internet_gateway" "blog_igw" {
  vpc_id = aws_vpc.blog_vpc.id

  tags = {
    Name = "Blog IGW"
  }
}

# Create Public Subnet
resource "aws_subnet" "blog_public_subnet" {
  vpc_id = aws_vpc.blog_vpc.id
  cidr_block = "10.0.1.0/24"
  availability_zone = "us-west-1a"

  tags = {
    Name = "Blog Public Subnet"
  }
} 

# Create Route Table and add public route
resource "aws_route_table" "blog_public_rt" {
  vpc_id = aws_vpc.blog_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.blog_igw.id
  }

  tags = {
    Name = "Blog Public Route Table"
  }
}

# Associate public subnet with public route table
resource "aws_route_table_association" "blog_public_assoc" {
  subnet_id = aws_subnet.blog_public_subnet.id
  route_table_id = aws_route_table.blog_public_rt.id
}

# Create Security Group for SSH access
resource "aws_security_group" "blog_sg" {
  name = "blog_sg"
  vpc_id = aws_vpc.blog_vpc.id

  ingress {
    from_port = 22
    protocol = "tcp"
    to_port = 22
    cidr_blocks = ["0.0.0.0/0"] 
  }
  
  egress {
    from_port = 0 
    protocol = "-1"
    to_port = 0 
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# Generate key pair for SSH access
resource "aws_key_pair" "blog_auth" {
  key_name = "blog_key"
  public_key = file("~/.ssh/id_rsa.pub")
}

# Create EC2 Instance
resource "aws_instance" "blog_server" {
  ami = "ami-0b5eea76982371e91" # Amazon Linux 2 AMI 
  instance_type = "t2.micro"

  subnet_id = aws_subnet.blog_public_subnet.id
  vpc_security_group_ids = [aws_security_group.blog_sg.id]
  key_name = aws_key_pair.blog_auth.id

  tags = {
    Name = "Blog Server"
  }
}
```

The main changes:

- Use AWS VPC, subnets, route tables, IGWs instead of VNETs and subnets
- Security groups instead of NSGs 
- EC2 instances instead of Azure VMs
- Key pairs instead of SSH keys

Let me know if you have any other questions!

Amazon Bedrock has now converted the Terraform template and provided a list of changed items. If your organization requires changes that weren’t handled by Amazon Bedrock, you can change them manually.

Walkthrough 2: Converting Terraform templates using Python

On your local machine, create a file named bedrock-blog.py, and add the following content. If the Terraform template file is in the same directory as the Python script, only include the file name. Otherwise, include the path.

import boto3
import json


def template_conversion(template_file_path):

    """
    This function converts a Terraform template to work on AWS.
    The different functional models have individual request and response formats.
    For the formatting for the Anthropic Claude, refer to:
    https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-claude.html

    """

    try:

        with open(template_file_path) as file:

            template = file.read()

        bedrock_runtime = boto3.client(region_name="us-east-1",
                                       service_name='bedrock-runtime')

        prompt = f"Convert to work on AWS output as a template\n{template}"

        # Claude requires you to enclose the prompt as follows:
        enclosed_prompt = "\n\nHuman: " + prompt + "\n\nAssistant:"

        body = json.dumps({
            "prompt":  enclosed_prompt,
            "max_tokens_to_sample": 4096,
            "temperature": 0.5,
        }
        ).encode()

        response = bedrock_runtime.invoke_model(body=body, modelId="anthropic.claude-v2")
        response_body = json.loads(response.get('body').read())
        print(response_body.get('completion'))
        return response_body.get('completion')

    except ClientError:
        logger.error("Couldn't invoke Anthropic Claude")
        raise


result = template_conversion('azure.tf')

with open('aws.tf', 'a') as file:
    file.write(result)
    file.close()

with open('aws.tf', 'r+') as file:
    lines = file.readlines()
    file.seek(0)
    file.truncate()
    file.writelines(lines[2:])

with open("aws.tf") as f:
    lines = f.readlines()

index = -1
for i, line in enumerate(reversed(lines)):
    if "}" in line:
        index = len(lines) - i
        break

if index != -1:
    del lines[index+1:]

with open("aws.tf", "w") as f:
    f.writelines(lines)

This script does the following actions:

Imports Boto3 and JSON packages.
Defines a template parameter using a template_conversion function call with a template_file_path parameter.
Sets up a Python try-except statement within the function. The try statement reads the Terraform template, formats the prompt, sets the parameters passed to the foundation model, and executes the call to Amazon Bedrock. The except clause performs a simple check to confirm that the foundation model is invoked.
Initiates the function and uses the results to create an AWS version of the Terraform template named aws.tf.
Parses the template and removes unneeded responses from Amazon Bedrock to create a clean file.

Copy the contents of step 9 in the first walkthrough into a new file named nonaws.tf, and save it into the same directory as the Python script.
Run the following script to convert the nonaws.tf file to an AWS-compliant template named aws.tf: python bedrock-blog.py.
Run the following script to view the new template: cat aws.tf.

Conclusion

As generative AI proliferates across industries, organizations grapple with understanding its true value and application to daily workflows. While capable of document creation, image editing, and coding, many developers don’t fully understand AI’s role in performing complex tasks like code conversion. I hope this blog post helped clear some of that ambiguity by demonstrating how you can use a powerful AI tool like Amazon Bedrock to automate rote migration tasks and give you back valuable time to create real business value.

Next steps

Take your skills to the next level and explore the powerful capabilities of Amazon Bedrock via the AWS Management Console and SDKs. In addition to the Anthropic Claude v2.1 foundation model that we used in this blog post, take some time to experiment with other supported models, including Amazon Titan, AI21 Labs, Cohereic, and more. To understand variations in terminology between models, see Inference parameters for foundation models. I also invite you to learn how to develop custom models and see for yourself how they can improve your conversion accuracy from the typical 85-90% to over 95%.

About the author

Troy Ameigh

Troy is a Specialist Solutions Architect at AWS, focusing on AWS Partner integration and automation. With almost 30 years of experience in the IT industry, Troy has great passion for promoting Infrastructure as Code (IaC) and leveraging his years of experience in Linux, AWS, networking, storage, containers, virtualization and architecting,

Reduce project delays with a docs-as-code solution

Dave May — Fri, 01 Mar 2024 19:05:29 +0000

Are your projects often delayed because of documentation? The code is complete, testing checks have all passed, and key stakeholders have approved the project for the official launch. Everything is in order for the big day, except for one piece—the documentation. With no estimated completion time in sight, you have no choice but to wait for the writing team to finalize and publish the content.

Unfortunately, this scenario plays out over and over again when the tools and processes used by writing teams aren’t integrated with those used by development teams. In a typical workflow, developers finish building the code in their own version control systems and then pass the documentation to a team of writers. The writers use their own tools, workflows, and pipelines to review and edit the content, convert it into a new format, and then publish it to the company site.

Despite being part of the same project, the developers and writers work independently using siloed processes to complete their tasks, while stakeholders observe from the sidelines, hoping that the two efforts sync up in time for the official launch.

In this post, we explain how the Amazon Web Services (AWS) Quick Start team solved our documentation delays by using a documentation-as-code (docs-as-code) solution based on GitHub and AsciiDoc. We take you through the initial challenges that led to the changes followed by the new solution and its benefits. Finally, we provide a set of steps that you can follow to explore the source code of our solution and decide whether it might be a potential fit for your organization.

About this blog post
Time to read	~10 min
Learning level	Intermediate (200)
AWS services	AWS CloudFormation Amazon Simple Storage Service (Amazon S3)

About the AWS Quick Start program

Launched in 2016, the AWS Quick Start program provided partners with a secure environment for building and delivering production-ready, automated reference deployments of their products on the AWS Cloud. Partners collaborated with AWS solutions architects to develop code and assets using a variety of components, including AWS CloudFormation templates, Bash and PowerShell scripts, AWS Lambda functions, and proprietary installation media—all bundled together in open-source GitHub repositories in the AWS Quick Start GitHub organization.

Figure 1 shows an example of a typical Quick Start architecture.

Figure 1: Quick Start architecture diagram

Each Quick Start provided customers with a detailed deployment guide with step-by-step instructions for configuring parameters, deploying to the cloud, and troubleshooting errors. Originally, Microsoft Word was the authoring application of choice due to its support for custom templates, built-in tracking features, and wide availability both within and outside of AWS. The architects and partners passed these files through a series of content reviews before they eventually handed the files off to a small team of technical writers for a final edit and publication.

Documentation becomes a dreaded bottleneck

As the Quick Start program gained momentum throughout the partner community, the catalog of reference deployments steadily grew into the hundreds with an average of 250,000 deployments annually. However, despite the positive reactions from partners and AWS leadership, the technical writing team quickly lost pace with the demands of the program, often causing delays in publishing. These delays were especially frustrating to partners who were trying to coordinate dates with a company announcement or event.

Tooling and processes create headaches

With the explosion in popularity of the Quick Start program, the processes and tools used to create and maintain the deployment guides in the early days quickly became the team’s biggest obstacle to publishing on time. Partners and architects continued to pass Microsoft Word files back and forth as they worked through multiple iterations of feedback and reviews. Documents with various file names (for example, document-v1.docx, document-v2.docx, and document-v2-bob-edit.docx) eventually piled up in inboxes, leading to bloated file directories and a confusing mix of current and obsolete review comments.

By the time the Microsoft Word files were passed to the team of technical writers for formal editing, the documents almost always contained formatting issues. These issues were typically caused by a lack of consistency across contributors combined with conflicting conventions and settings used in operating systems and word processing programs such as Microsoft Word, OpenOffice, and LibreOffice. According to the Quick Start writers, they spent an average of 6-8 hours on each deployment guide addressing formatting issues before they even started reviewing the text.

Publishing multiple formats is time-consuming

When the Microsoft Word files were finally edited and formatting issues resolved, the publishing process began. First, the writers had to convert the files to both PDF and HTML format. Next, they published the converted files to multiple locations. The PDFs and images were published both to Amazon Simple Storage Service (Amazon S3) and an internal GitHub repository, and the HTML files were published to the AWS public-facing documentation site (AWS Docs). Even one small change to the content meant repeating the entire process of converting, storing, and publishing the files to all three locations.

To add even more complexity, deployment guides often went through revisions due to new software versions of partner products, AWS service updates, and bug fixes. The same confusion of mismatched documents, formatting issues, and publishing woes that plagued the initial launch of the Quick Start continued any time the guide needed an update. Even minor updates to a deployment guide could take weeks to complete.

The Quick Start team knew they needed to solve these problems quickly and permanently if they wanted to continue supporting partners and their customers. They needed a scalable and efficient solution that would keep projects moving forward and on schedule. At a minimum, they needed new tooling and processes to help simplify and streamline the end-to-end workflow of creating, updating, publishing, and maintaining deployment guides.

A new docs-as-code solution

After weeks of gathering data, the Quick Start team finally decided upon a docs-as-code solution. Internally referred to as Docs 2.0, this new model was a natural choice for the team, especially considering that they were already collaborating with partners on Quick Start code using GitHub, a well-known and widely used version control system. If GitHub was well suited for storing templates, scripts, source code, submodules, and other code artifacts in plain text, then surely it would suffice for collaborating on and storing text files and related image artifacts such as screenshots, AWS logos, and partner logos.

After weighing the pros and cons between markdown and AsciiDoc, the team decided upon AsciiDoc as the authoring format due to its powerful and standardized toolset (see Figure 2). With the docs-as-code solution, architects, partners, and writers use well-known editing software such as Visual Studio Code to edit the content and commit their changes to a dedicated, versioned GitHub repository. Here, they collaborate, review, and approve, and eventually merge content into a gh-pages branch dedicated to documentation. All changes are now tracked to an author and timestamp.

Figure 2: AsciiDoc file in Visual Studio Code

To keep the structure and formatting of the deployment guides consistent across projects, architects prepopulate each documentation repository with a set of AsciiDoc files, images, style sheets, and index files at the start of each project. These files control the structure and table of contents of each guide. Boilerplate text and images that are common across all deployment guides are single-sourced and maintained in a separate, non-editable submodule that’s pulled in dynamically when the guides are built and converted to HTML.

The HTML output of the deployment guides is published directly out of the repository using GitHub Pages, GitHub’s hosting service for building and publishing websites (see Figure 3). Using simple commands, writers can build the guides locally on their machines and publish to a staging site to check for UI issues such as broken links and images before publishing to the live server. Now, writers could push either new or updated content live into production with only a few steps.

Figure 3: Published deployment guide

With the docs-as-code solution, the days of juggling multiple files, sifting through obsolete content, and managing siloed feedback are over. Documentation is stored in a single repository, and content reviews between AWS and partners are organized and trackable. Using the open-source tool AsciiDoctor, the published HTML is well-structured, accessible, and free of formatting issues. According to the writers on the Quick Start team, eliminating formatting issues saved them approximately 50% of editing time, a massive improvement over the previous process.

Docs-as-code gives the architects more time to spend with partners on the design side of their projects, and writers have more time to propose and implement improvements in the overall process. Writers can also revisit previously published deployment guides and migrate them to the new process. Docs-as-code sets writers up for success when they have to update documentation down the road. Managers are thrilled, because they no longer need to request more resources and train additional employees.

Explore for yourself

With so many improvements and positive feedback from partners, the Quick Start team transitioned its docs-as-code framework to an open-source model and made it available to the wider community both within and outside of AWS.

To implement a docs-as-code solution in your environment, complete the following tasks:

Navigate to the docs-as-code boilerplate repository we’ve provided. No credentials are required.
Clone an existing GitHub repository that you can use to explore and experiment with the boilerplate code.
In your cloned repository, run the following commands to add the boilerplate repository as a submodule to your environment:
```
git submodule add https://github.com/aws-ia/aws-ia-documentation-base-common.git docs/boilerplate
```
Now you should see a new docs/boilerplate/ directory with content from the boilerplate repository in your cloned environment.
To build the folder structure for a CloudFormation-based deployment guide, run the following command:
```
docs/boilerplate/.utils/create_repo_structure.sh -d -c
```
The script also contains options for other types of documents such as migration and operational guides.
With AsciiDoctor installed, run the following command to generate the HTML version of the deployment guide:
```
docs/boilerplate/.utils/build_docs.sh
```
In GitHub, choose Settings,pages to allow GitHub Pages to associate the HTML file with your repository and add it to the root of your repository in the main branch (or another branch that you designate).

Conclusion

The AWS Quick Start team saw massive improvements with their docs-as-code process for building and publishing deployment guides. By integrating workflows and toolsets, they simplified the content review cycles and streamlined the publishing process. The end result was a scalable solution that eliminated obsolete processes, complicated workflows, and inefficient publishing pipelines. The solution gave the writing team a scalable, integrated solution that allowed them to complete work requests on time, even during times of heavy workloads.

Next steps

Check out the Taskcat CloudFormation testing application’s documentation site for an example of a guide published using the AWS Quick Start team’s docs-as-code process. You can find the code for this publication in the main branch of the Taskcat GitHub repository.

About the authors

Dave May

Dave is a Specialist Solutions Architect at AWS, focusing on AWS Partner integration and automation. Dave helps AWS Partners automate the deployment of their solutions on AWS.

Anna O’Neal

Anna is a senior technical writer on the AWS Partner Integrations Engineering team. She has 25 years of experience creating technical content, specializing in product documentation, customer references, and case studies. In her free time, Anna runs a small animal sanctuary for rescued pigs in Chapel Hill, NC.

Simplify integration of your Amazon EKS cluster with Amazon EKS Blueprints for CDK

Mikhail Shapirov — Fri, 08 Dec 2023 20:09:10 +0000

With the recent deprecation of the Amazon Elastic Kubernetes Service (Amazon EKS) Quick Start based on AWS CloudFormation, customers and partners need to achieve similar results with alternative solutions. Specifically, they need an approach that simplifies integration of common tooling and provisioning of complete, opinionated EKS clusters that meet specific application requirements.

The extensible nature of Kubernetes (and EKS by extension) allows customers to use a wide range of popular commercial and open-source tools, commonly referred to as add-ons. With so many tooling and design choices available, building a tailored EKS cluster that meets your application’s specific needs can take a significant amount of time. It involves integrating a wide range of open-source tools and AWS services, often requiring deep expertise in AWS and Kubernetes.

In this article, we introduce the Amazon EKS Blueprints for CDK framework as the recommended replacement for the deprecated Amazon EKS Quick Start. We cover common usage scenarios, provide examples and sample code, and present available resources for full implementations. We use the AWS Cloud Development Kit (AWS CDK) flavor of Amazon EKS Blueprints, but you can also use Amazon EKS Blueprints for Terraform based on HashiCorp Terraform to achieve similar results, as described in the EKS Blueprints launch blog.

About this blog post
Time to read	~10 min
Learning level	Expert (400)
AWS services	AWS Cloud Development Kit (AWS CDK) AWS CloudFormation Amazon Elastic Compute Cloud (Amazon EC2) Amazon Elastic Kubernetes Service (Amazon EKS) AWS Fargate

Solution overview

Amazon EKS Blueprints for AWS CDK is a collection of Infrastructure as Code (IaC) modules that are available in public repositories on GitHub. The collection can help you configure and deploy consistent, ready-to-use EKS clusters across accounts and Regions.

The main repository contains the framework itself, packaged as an npm module (Node Package Manager) for external consumption. In addition, a collection of patterns is available to serve as usage examples along with complete solutions that you can use out of the box from the patterns repository. A pattern implementation is a codified reference architecture that conveys architectural and educational value.

You can use Amazon EKS Blueprints to bootstrap EKS clusters with Amazon EKS add-ons. You can also use common, open-source add-ons, such as Prometheus, Karpenter, NGINX, AWS Load Balancer Controller, Fluent Bit, Keda, and Argo CD. The framework helps implement security controls that are required to operate workloads in a shared environment.

Many customers use GitOps to decouple IaC component and application deployment on Kubernetes from the infrastructure. Access our example workloads repository to facilitate these patterns, which can be bootstrapped with the Argo CD add-on.

You can define Amazon EKS Blueprints for CDK using Typescript and Node.js tooling. While the programming language enabled us to create domain-oriented APIs for programmatic configuration, Node.js provides a stable and maintained runtime.

To define a blueprint, use the builder design pattern as a starting point. This pattern simplifies complex object construction, such as the cluster blueprint or pipeline.

The following is an example of a simple starter blueprint:

const app = new cdk.App();

blueprints.EksBlueprint.builder()
   .version("auto")
    .addOns(
        new blueprints.AwsLoadBalancerControllerAddOn,
        new blueprints.VpcCniAddOn(), 
        new blueprints.MetricsServerAddOn,
        new blueprints.ClusterAutoScalerAddOn,
    )
    .build(app, "my-first-blueprint");

This blueprint example performs the following tasks:

Creates a new Amazon Virtual Private Cloud (Amazon VPC) with three public and three private subnets
Creates an Amazon EKS cluster in the Amazon VPC
Creates a managed node group in the private subnets
Adds four add-ons so that the blueprint is ready to accept customer applications

To see a complete version of the starter blueprint, see the AWS samples repository.

Use cases

Let’s review a few common use cases that customers and partners have implemented with the deprecated Amazon EKS Quick Start to see how they compare to Amazon EKS Blueprints for CDK. First, we introduce you to an example blueprint (see Figure 1) to highlight a few high-level capabilities of the framework and concepts described later in this article:

Figure 1: Example blueprint

The Clusters (bottom) layer represents the available AWS compute options and cluster configurations. The Add-ons (middle) layer shows a mix of add-ons, including open-source, AWS-managed and commercial tools. The Teams (top) layer represents application teams that can be onboarded on to the cluster to run applications within the infrastructure and guardrails supplied by the lower layers.

Add-ons

One of the most popular features of Amazon EKS Blueprints for CDK is the portfolio of the supported add-ons. These add-ons represent AWS components, such as the Amazon Elastic Block Store (Amazon EBS) Container Storage Interface (CSI) driver, popular open-source solutions like Metrics Server or External DNS, and partner-created add-ons.

You can choose add-ons and create a blueprint from the framework blocks. You can also extend the list of supported add-ons through the framework’s extensibility options. Several add-ons such as Calico Operator, Grafana, Prometheus (based on Amazon Managed Service for Prometheus) were also available in the original Amazon EKS Quick Start.

Cluster configuration

By default, the framework assumes a simple cluster configuration with a new Amazon VPC and a managed node group with a minimum of one and a maximum of three nodes of m5.large instances.

You can overwrite the default parameters in one of the source files of context values by setting the following parameters:

eks.default.instance-type
eks.default.min-size
eks.default.max-size
eks.default.desired-size

This configuration is useful for demonstration and prototyping. For more complex use cases, you can choose a more explicit cluster configuration, controlled by the implementations of the ClusterProvider interface. Out of the box, the framework provides implementations for managed node groups, autoscaling (self-managed) node groups, AWS Fargate, and a generic cluster provider that can combine all of the compute options.

Managed node groups

You can configure a cluster that contains a single managed node group for workloads with an MngClusterProvider class. With this cluster provider, you can apply version upgrades to the EKS control plane and specify the Amazon Machine Image (AMI) for the worker nodes. For more information, see Managed Node Group Cluster Provider.

Note: Upgrading the control plane (specifically, the version attribute of the cluster provider) impacts only the control plane version and not the worker nodes in the node group. For upgrades, provide both the version of the control plane in the version attribute and the AMI release version for the worker nodes in the amiReleaseVersion attribute.

The following example creates a new EKS cluster with 1-10 worker nodes with the desired size of 4 nodes. To control cluster upgrades, the example explicitly specifies the versions of the control plane and AMI release.

const mngClusterProvider = new bp.MngClusterProvider({
        minSize: 1,
        maxSize: 10,
        desiredSize: 4,
        instanceTypes: [new ec2.InstanceType('m5.large')],
        amiType: eks.NodegroupAmiType.AL2_X86_64,
        nodeGroupCapacityType: eks.CapacityType.ON_DEMAND,
        version: KubernetesVersion.V1_27,
        amiReleaseVersion: "1.27.3-20230728" // this will upgrade kubelet to 1.27.3
});

blueprints.EksBlueprint.builder()
    .clusterProvider(mngClusterProvider)
    .build(app, "my-mng-stack");

To upgrade the cluster, change the version and amiReleaseVersion attributes to the desired values and run the cdk deploy command for your stack. If needed, look up AMI release versions for your EKS cluster.

Self-managed node groups

Self-managed nodes use Amazon EC2 Auto Scaling groups in EKS. The framework provides an AsgClusterProvider property for adding self-managed nodes to the cluster:

const asgClusterProvider = new bp.AsgClusterProvider({
    id: "my-asg-group",
    minSize: 1,
    maxSize: 10,
    desiredSize: 4,
    instanceType: new ec2.InstanceType('m5.large'),
    machineImageType: eks.MachineImageType.AMAZON_LINUX_2,
    updatePolicy: UpdatePolicy.Rolling
});

However, this option is a self-managed option when you are responsible for the worker node’s AMI (for example, patching), operating system, and Kubelet. You can still use Amazon EKS optimized Amazon Linux AMIs with self-managed nodes and upgrade them when needed either in-place or by migrating to new nodes.

AWS Fargate

For a serverless Amazon EKS cluster, apply the FargateClusterProvider property to create a control plane and additional AWS Fargate profiles for the kube-system and default namespaces. This type of cluster can be useful to teams that adopt a workload-per-cluster approach. Extend the cluster configuration with additional Fargate profiles to target additional user namespaces, for example:

const clusterProvider = new blueprints.FargateClusterProvider({
    fargateProfiles: {
      "team1": { selectors: [{ namespace: "team1" }] }
    },
    version: eks.KubernetesVersion.V1_27
});

Combining multiple node groups

Customers working in production-level scenarios typically combine multiple node groups and Fargate profiles into the same cluster. This option especially applies to a shared cluster scenario when workloads running on the cluster come from multiple teams with different requirements. The following code snippet includes a node group for generic workloads, a node group for Amazon EC2 Spot instances to run shorter and less mission-critical workloads, and a couple of Fargate profiles.

const clusterProvider = new blueprints.GenericClusterProvider({
    version: eks.KubernetesVersion.V1_27,
    managedNodeGroups: [
        {
            id: "mng-ondemand",
            amiType: eks.NodegroupAmiType.AL2_X86_64,
            instanceTypes: [new ec2.InstanceType('m5.2xlarge')]
        },
        {
            id: "mng2-spot",
            instanceTypes: [ec2.InstanceType.of(ec2.InstanceClass.BURSTABLE3, ec2.InstanceSize.MEDIUM)],
            nodeGroupCapacityType: eks.CapacityType.SPOT
        }
    ],
    fargateProfiles: {
        "fp1": {
            fargateProfileName: "fp1",
            selectors:  [{ namespace: "serverless1" }] 
        },
        "fp2": {
            fargateProfileName: "fp2",
            selectors:  [{ namespace: "serverless2" }] 
        }
    }
});

Network Configuration

Amazon VPCs for EKS clusters with Amazon EKS Blueprints

VpcProvider is a resource provider that creates a new Amazon VPC with the default values or, optionally, allows you to specify your primary and secondary CIDR ranges, and map those to individual subnets. This VPC resource provider can also import an existing Amazon VPC into your blueprint if you provide the VPC ID. If the VPC ID is set to default, this resource provider looks up the default Amazon VPC in your account.

Configuring Amazon VPC with options

Create a new Amazon VPC with a primary CIDR with VpcProvider:

blueprints.EksBlueprint.builder()
  .resourceProvider(GlobalResources.Vpc, new VpcProvider({primaryCidr: "10.0.0.0/16"}))
  ...
  .build();

Create a new Amazon VPC with a primary CIDR and secondary CIDR and subnet CIDRs with VpcProvider:

blueprints.EksBlueprint.builder()
    .resourceProvider(blueprints.GlobalResources.Vpc, new blueprints.VpcProvider(undefined, {
        primaryCidr: "10.2.0.0/16", 
        secondaryCidr: "100.64.0.0/16",
        secondarySubnetCidrs: ["100.64.0.0/24","100.64.1.0/24","100.64.2.0/24"]
    }))

Use an external VPC with a VPC ID with VpcProvider:

blueprints.EksBlueprint.builder()
  .resourceProvider(GlobalResources.Vpc, new blueprints.VpcProvider("<vpc-id>"))
  ...
  .build();

Provision a custom VPC in a separate CDK stack and pass it over to the blueprint using DirectVpcProvider:

const vpcStack = new VPCStack(app, 'eks-blueprint-vpc', { env: { account, region } }); // contains myVpc member variable
blueprints.EksBlueprint.builder()
  .resourceProvider(GlobalResources.Vpc, new blueprints.DirectVpcProvider(vpcStack.myVpc))
  ...
  .build();

Look up a secondary subnet provider by ID and pass it over to the blueprint under the provided name using LookupSubnetProvider:

blueprints.EksBlueprint.builder()
  .resourceProvider('my-subnet', new blueprints.LookupSubnetProvider("subnet-id"))
  ...
  .build();

Windows support

You can use the WindowsBuilder construct to create EKS clusters with Windows node groups. The WindowsBuilder construct applies the required configuration using a builder pattern to set up your EKS cluster with Windows support. It creates an EKS cluster with a Linux managed node group for standard software and a Windows-managed node group to schedule Windows workloads.

The following example demonstrates how to use the WindowsBuilder construct to configure a Windows-managed node group on a new EKS cluster:

// Create a role for the worker nodes with the required policies
const nodeRole = new blueprints.CreateRoleProvider("blueprint-node-role", new iam.ServicePrincipal("ec2.amazonaws.com"),
    [
        iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonEKSWorkerNodePolicy"),
        iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonEC2ContainerRegistryReadOnly"),
        iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonSSMManagedInstanceCore"),
        iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonEKS_CNI_Policy")
    ]);

WindowsBuilder.builder({ // passing WindowsOptions here
        kubernetesVersion: eks.KubernetesVersion.V1_27,
        instanceClass: ec2.InstanceClass.M5,
        instanceSize: ec2.InstanceSize.XLARGE4
    })
    .addOns(new WindowsVpcCni())
    .account(account)
    .region(region)
    .resourceProvider("node-role", nodeRole)
    .build(app, "my-windows-blueprint");

To learn about input parameters and see a demonstration of using Windows with Amazon EKS, see the Windows Builder documentation.

Working with existing clusters

Amazon EKS Blueprints can import existing clusters and add or configure additional software on top of them. This can be handy when a clear split exists between infrastructure, platform, and site reliability engineering (SRE) teams that contribute separate aspects to the cluster. For example, the infrastructure team can provision an Amazon EKS cluster compute capacity and control the network configuration across multiple availability zones. This team may also control ingress and supply the AWS Load Balancer Controller. The platform and SRE teams may focus on observability and developer tooling to support CI/CD processes for Prometheus, Grafana, or GitOps engines (such as Flux or Argo CD).

The ImportClusterProvider construct can import an existing Amazon EKS cluster into your blueprint. This means that you can implement add-ons and limited team capabilities. The blueprints framework provides the following set of convenience methods to instantiate the ImportClusterProvider resource provider using an SDK API call to describe the cluster before importing:

Option 1: Use this option to retrieve the cluster information through the DescribeCluster API (requires eks:DescribeCluster permission at build time) and to instantiate the ImportClusterProvider class to import the cluster and to set up the blueprint VPC based on the discovered VPC configuration, for example:

const clusterName = "quickstart-cluster";
const region = "us-east-2";

const kubectlRoleName = "MyClusterAuthConfigRole"; // this is the role registered in the aws-auth config map in the target cluster 
const sdkCluster = await blueprints.describeCluster(clusterName, region); // get cluster information using EKS APIs

const importClusterProvider = blueprints.ImportClusterProvider.fromClusterAttributes(
    sdkCluster, 
    blueprints.getResource(context => new blueprints.LookupRoleProvider(kubectlRoleName).provide(context))
);

const vpcId = sdkCluster.resourcesVpcConfig?.vpcId;

blueprints.EksBlueprint.builder()
    .clusterProvider(importClusterProvider)
    .resourceProvider(blueprints.GlobalResources.Vpc, new blueprints.VpcProvider(vpcId)) // this is required with import cluster provider

Option 2: Use this option if you already know the VPC ID of the target cluster (requires the eks:DescribeCluster permission at build time):

const clusterName = "quickstart-cluster";
const region = "us-east-2";

const kubectlRole: iam.IRole = blueprints.getNamedResource('my-role');

const importClusterProvider2 = await blueprints.ImportClusterProvider.fromClusterLookup(clusterName, region, kubectlRole); // note await here

const vpcId = ...; // you can always get it with blueprints.describeCluster(clusterName, region);

blueprints.EksBlueprint.builder()
    .clusterProvider(importClusterProvider2)
    .resourceProvider('my-role', new blueprints.LookupRoleProvider('my-role'))
    .resourceProvider(blueprints.GlobalResources.Vpc, new blueprints.VpcProvider(vpcId))

Option 3: Use this option when you want to avoid providing special permissions at build time. You must pass an OpenID Connect (OIDC) provider if you use an IAM Role for Service Account (IRSA) with your blueprint:

const importClusterProvider3 = new ImportClusterProvider({
    clusterName: 'my-existing-cluster',
    version: KubernetesVersion.V1_27,
    clusterEndpoint: 'https://B792B88BC60999B1AD.gr7.us-east-2.eks.amazonaws.com',
    openIdConnectProvider: getResource(context =>
        new LookupOpenIdConnectProvider('https://oidc.eks.us-east-2.amazonaws.com/id/B792B88BC60999B1A3').provide(context)),
    clusterCertificateAuthorityData: 'S0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCasdd234................',
    kubectlRoleArn: 'arn:...',
    clusterSecurityGroupId: 'sg...';
});

const vpcId = ...; 

blueprints.EksBlueprint.builder()
    .clusterProvider(importClusterProvider3)
    .resourceProvider(blueprints.GlobalResources.Vpc, new blueprints.VpcProvider(vpcId))

The AWS CDK Observability Accelerator provides a pattern to achieve AWS Native Observability and a pattern for open-source (OSS)-based monitoring and logging on top of existing clusters. See Announcing AWS CDK Observability Accelerator for Amazon EKS to learn more about the AWS CDK Observability Accelerator.

Pipelines

If you provision your first set of clusters with Amazon EKS Blueprints, the Git-based pipeline processes can help you to provision and maintain your clusters through Git. With pipelines, you can model your environments across multiple AWS accounts and Regions covering the complete enterprise landscape for software delivery. Typically, customers refer to such environments as development, testing, staging, and production.

The following code defines development, testing, and production environments that deploy a specific blueprint. This enables consistency and standardized maintenance for more complex scenarios. Use the stage construct to deploy a single step (for example, apply a blueprint to the account or Region) and the wave construct to group several stages that should execute in parallel:

const blueprint = EksBlueprint.builder()
    .addOns(...)
    .teams(...)
    .clusterProvider(...);

blueprints.CodePipelineStack.builder()
    .name("eks-blueprints-pipeline")
    .owner("aws-samples")
    .repository({
        //...
    })
    .stage({
        id: 'dev-single-cluster',
        stackBuilder: blueprint.clone('us-west-1')
    })
    .wave( {  // adding two clusters for test env
        id: "test",
        stages: [
            { id: "test-west-1", stackBuilder: blueprint.clone('us-west-1').account(TEST_ACCOUNT)}, // requires trust relationship with the code pipeline role
            { id: "test-east-2", stackBuilder: blueprint.clone('us-east-2').account(TEST_ACCOUNT)}, // See https://docs.aws.amazon.com/cdk/api/v1/docs/pipelines-readme.html#cdk-environment-bootstrapping

        ]
    })
    .wave( {
        id: "prod",
        stages: [
            { id: "prod-west-1", stackBuilder: blueprint.clone('us-west-1')}, // add prod level customizations
            { id: "prod-east-2", stackBuilder: blueprint.clone('us-east-2')}, // add prod level customizations
        ]
    })

Extending Amazon EKS Blueprints with third-party and partner add-ons

Amazon EKS Blueprints for CDK is extensible, allowing you to add new capabilities to the framework (or solutions based on the framework). You can also modify or override the existing behavior.

Use the following abstractions to add new features to the framework:

Add-on – Implement new add-ons that are used the same way as the core add-ons supplied by the framework. Implementing add-ons is the most common way to extend the framework. Support exists for Helm and non-Helm add-ons, and add-ons that can use GitOps-based distribution (such as, Argo CD or Flux).
Resource provider – Create reusable resources and apply them across multiple clusters, add-ons, and teams. These resources include AWS Identity and Access Management (IAM) roles, Amazon VPCs, hosted zones, Amazon Relational Database Service (Amazon RDS) databases, and more. This construct is reserved for AWS resources.
Cluster providers – Create custom code that provisions an Amazon EKS cluster with node groups. Use this abstraction to extend behavior such as control plane customization and custom settings for node groups.
Teams – Create team templates for application and platform teams. Use this abstraction to model your team namespaces with network isolation rules, policies (network and security), software wiring (e.g. auto-injection of proxies), and other features.

You can make your extensions private or public. Additionally, you can have your extension (such as an add-on) validated by the AWS team and published in the Add-ons documentation. For a complete guide, see Extensibility.

Conclusion

In this post, we provided examples of how you can replicate the cluster setup that was supported by the now-deprecated Amazon EKS Quick Start with Amazon EKS Blueprints for CDK. We encourage you to explore the patterns repository containing usage examples and complete solutions .

Scaling your EKS provisioning and maintenance across the entire organizational structure (including accounts and Regions) is an important feature for enterprise adoption. We recommend that you become familiar with our pipeline support, which enables consistent and centralized cluster configuration management across your enterprise environments. It enables you to control roll-out of changes with a clear promotion strategy.

The range of features supported by the Amazon EKS Quick Start was very broad. Therefore, this article focused on the most common use cases. Use the main GitHub repository for support and feedback, for example if you have a use case that’s missing. In addition, use issues and discussions to ask questions and create feature requests. We also welcome community contributions.

About the authors

Mikhail Shapirov

Mikhail is a Principal Partner Solutions Architect at AWS, focusing on container services, application modernization, and cloud management services. Mikhail helps partners and customers drive their products and services on AWS with AWS Container services, serverless compute, development tools, and cloud management services. He is also a software engineer.

Elamaran Shanmugam

Elamaran (Ela) Shanmugam is a Sr. Container Specialist Solutions Architect with AWS. Ela is a Container, Observability and Multi-Account Architecture SME and helps customers design and build scalable, secure, and optimized container workloads on AWS. His passion is building and automating infrastructure so customers can focus more on their business. He is based out of Tampa, Florida and can be reached on Twitter @IamElaShan and on GitHub.

Create AMIs with AWS CodePipeline

Henry Bravo — Thu, 01 Jun 2023 14:57:58 +0000

One way to alleviate the pain of drift in your Amazon Web Services (AWS) workloads is to use a GitOps workflow to automate Amazon Machine Image (AMI) builds with AWS CodePipeline. In this post, we show you how to deploy a continuous integration and continuous delivery (CI/CD) pipeline, managed by CodePipeline, to build AMIs.

Manually creating AMIs for your Amazon Elastic Compute Cloud (Amazon EC2) instances and containers can be an error-prone process. As your AWS workloads grow, you can lose track of infrastructure changes and discrepancies, or drift. The solution I provide uses a Git repository managed by AWS CodeCommit as the single source of truth. You invoke the pipeline when you commit AWS CloudFormation templates containing the recipe that the pipeline uses to build an AMI. The templates I provide build an AMI that’s based on Amazon Linux2 ARM64. After the AMI is built, Amazon Simple Notification Service (Amazon SNS) sends you an email notification. After you approve or reject the AMI build, the pipeline automatically cleans up the resources it created.

About this blog post
Time to read	~10 min.
Time to complete	~45 min.
Cost to complete	~$1
Learning level	Advanced (300)
AWS services	AWS CodePipeline AWS CodeCommit AWS CloudFormation Amazon Elastic Compute Cloud (Amazon EC2) Amazon EC2 Image Builder Amazon Simple Storage Service (Amazon S3) Amazon Simple Notification Service (Amazon SNS)

Overview

Figure 1 shows a high-level overview of the resources that you deploy in the walkthrough.

Figure 1. Architecture diagram for creating AMIs with AWS CodePipeline.

As shown in Figure 1, the solution deploys the following:

A CD pipeline with the following:
- CodePipeline to read source code from AWS CodeCommit.
- CodeCommit with a GitHub repository. You invoke the pipeline when you commit templates to the repository.
- CloudFormation to create an Amazon EC2 Image Builder stack based on the template.
- Amazon SNS to send email notifications upon completion of AMI builds. The notifications prompt you to review and approve or reject builds in the CodePipeline console.
An Amazon Simple Storage Service (Amazon S3) bucket to store templates and artifacts.
EC2 Image Builder to launch a temporary Amazon EC2 T4g instance that builds and tests an AMI.

AMI recipe template files

Our solution uses the following templates to define the AMI build recipe. The templates contain the recipe used by Amazon EC2 Image Builder to build, validate, and test an AMI.

linux-ami-imagebuilder.yaml

The template linux-ami-imagebuilder.yaml contains the following EC2 Image Builder parameters:

LinuxImage tells the EC2 Image Builder service that we are building an AMI and specifies the image recipe and infrastructure configuration to use. The image recipe defines how the AMI is customized, and the infrastructure configuration defines the build and test configurations.
LinuxImageRecipe defines the components to configure and deploy to the AMI. You can customize the solution by specifying different components, including your own custom components that you write in Bash or PowerShell. I configure the template to deploy the following components:
- amazon-cloudwatch-agent-linux: Amazon CloudWatch agent.
- aws-cli-version-2-linux: The AWS Command Line Interface (AWS CLI) version 2.
- stig-build-linux-low: The Security Technical Implementation Guide (STIG) for Category 1 (Low) compliance level.
- SetLocaleComponent: The setlocale function in Linux sets the current locale, which determines the language and cultural conventions used by various system functions and libraries. It fixes the following common error when logging into Linux:

-bash: warning: setlocale: LC_CTYPE: cannot change locale (UTF-8): No such file or directory.

In this warning, the LC_CTYPE category of the locale specifies character classification, case conversion, and character encoding/decoding. When the LC_CTYPE category is set, it affects how functions like isalpha() and toupper() behave. It’s important to note that setlocale affects not only your own program but also any other programs running on the system that rely on the locale.

- YumUpdateComponent—Invokes the yum update command. This command in Linux updates the installed packages to their latest available versions. For more information, refer to Linux package management with YUM and RPM.

stack-configuration.json

The template stack-configuration.json includes the parameters needed to run the pipeline. Edit stack-configuration.json to specify the parameters, such as the base AMI image used by the pipeline to build the AMI. You can customize your solution by entering the Value key from a different AMI for the BaseImageId parameter. To find an AMI, refer to Find a Linux AMI.

Prerequisites

Before getting started, ensure that you have the following:

An AWS account. If you don’t have one, sign up at https://aws.amazon.com.
A virtual private cloud (VPC) with a subnet in your AWS account.
An EC2 key pair. A key pair enables you to log in using SSH (Secure Shell) to the AMI produced by the pipeline.
Version 2 of the AWS CLI. You need this if you want to deploy the CloudFormation stack using AWS CLI instead of the AWS Management Console.
A Git client installation. You need this to use Git to invoke the pipeline by committing templates. In the walkthrough, you can also use the AWS Management Console to commit them.

Walkthrough

In the walkthrough, you perform the following steps:

Download the templates.
Deploy the CD pipeline CloudFormation stack.
Commit files to the CodeCommit repository.
Verify the AMI build.
Approve or reject the AMI build.

Step 1: Download the templates

Download the following templates:

Step 2: Deploy the CD pipeline CloudFormation stack

Deploy the pipeline infrastructure using the CloudFormation stack that I provide. You can deploy using the AWS Management Console or AWS CLI.

Deploy using the AWS Management Console

Choose Launch Stack.

On the Specify stack details page of the CloudFormation console, enter a name for the stack.
For Email, enter an email address to receive AMI build notifications.
(Optional) If desired, you can edit default entries except for TemplateFileName and StackConfig.
Choose Next.
On the Configure stack options page, choose Next.
On the Review page, choose I acknowledge that AWS CloudFormation might create IAM resources.
Choose Submit. The stack takes about two minutes to deploy.

Deploy using AWS CLI

In the directory where you save linux-ami-codepipeline.yaml, run the following command. Replace <my@email-address.aws> with the address that you want to use to receive AMI build notifications.

aws cloudformation create-stack[TB1] [MOU2] --region eu-west-1 --stack-name ami-builder-blog-linux-pipeline --template-body[TB3] [MOU4] file://Linux/linux-ami-codepipeline.yaml --parameters ParameterKey= Email,ParameterValue=<my@email-address.aws> --capabilities CAPABILITY_NAMED_IAM

Confirm CodePipeline SNS topic subscription

You will receive an email from AWS Notifications <no-reply@sns.amazonaws.com> asking you to confirm subscription to the CodePipeline SNS topic. Choose Confirm subscription in the email message.

Step 3: Commit files to the CodeCommit repository

Commit linux-ami-imagebuilder.yaml and stack-configuration.json to the CodeCommit repository. This invokes the pipeline to build an AMI.

Sign in to the AWS Management Console and open the CodePipeline console.
In the navigation pane, choose Pipeline, Pipelines.
Choose ami-builder-blog-linux-pipeline. As shown in Figure 2, an error displays in the Source section of the ami-builder-blog-linux-pipeline page. This is normal and expected because you have not yet committed files to invoke the pipeline.

Figure 2. Ami-builder-blog-linux-pipeline page in the CodePipeline console.

In the Source section, choose AWS CodeCommit.
Set up your GitHub repository on the ami-builder-blog-linux-pipeline page. For Clone URL, select Connection steps and follow the instructions provided. Then add linux-ami-imagebuilder.yaml and stack-configuration.json to your repository and commit them in a pull request to the CodeCommit repository.

If you’re not using Git, choose Add file, Upload file to manually upload linux-ami-imagebuilder.yaml and stack-configuration.json one at a time. For more information, refer to Create or upload a file (console).

Step 4: Verify the AMI build.

If you confirmed your subscription to the pipeline SNS topic, you will receive a notification email when the build completes. To verify the AMI build, launch an instance of it and then log in to the instance.

Note: If the build pipeline fails, terminate the Build EC2 instance and delete the temporary CloudFormation stack test-amazon-linux2-arm64-ami that the pipeline deploys. Then recommit the templates to restart the build process. To troubleshoot issues with the AMI build, refer to the Amazon CloudWatch logs. The solution stores AMI build logs in CloudWatch/Loggroups/aws/imagebuilder/test-amazon-linux2-arm64-ami-Image-xxxxxxxxxxx/0.0.1/1/.

Step 5: Approve or reject the AMI build.

If you confirmed your subscription to the pipeline SNS topic, you will receive a notification that includes a link to approve or reject the build. For more information, refer to Approve or reject an approval action in CodePipeline.

Cleanup

After you approve or reject, the pipeline cleans up the resources it created, including the CloudFormation template and EC2 Image Builder resources. When you’re finished testing the pipeline and want to remove the solution from your account, delete the following:

The pipeline artifacts S3 bucket. For more information, refer to Deleting a bucket.
The CloudFormation stack ami-builder-blog-linux. For more information, refer to Deleting a stack on the AWS CloudFormation console.

Conclusion

In this post, we deployed a CD pipeline managed by CodePipeline to build AMIs. Customize the solution used in this post by adding unit testing to the CD pipeline and additional software and operating system logic. For more insight into building AMIs, refer to the following:

Let us know how you use CodePipeline to build AMIs. Use the Comments section for questions and comments.

About the author

Henry Bravo

Henry Bravo is a senior global solutions architect at AWS. He helps AWS customers in the automotive and manufacturing industry with their digital transformations. His work helps companies fundamentally change how they deliver value to their customers, reduce operational costs, and empower their employees.

Automate IAM credential reports for large AWS Organizations

Ibukun Oyewumi — Tue, 21 Mar 2023 19:13:18 +0000

Manage AWS Identity and Access Management (IAM) credentials to properly secure AWS Organizations. With a large number of accounts in your organization, it is less practical to generate IAM credential reports within each account.

This post provides a solution to automate IAM credential reports in AWS Organizations with many accounts. The results are consolidated into a centralized Amazon Simple Storage Service (Amazon S3) bucket to simplify auditing and review. These credential reports list all AWS IAM users in your accounts and the status of their credentials, including passwords, access keys, and MFA devices.

The primary benefits of this solution are the ability to run IAM credential reports in parallel across your accounts. We’ll also use AWS Step Functions to wait for reports to process, which will reduce AWS Lambda costs.

About this blog post
Time to read	~9 min.
Time to complete	~15 min.
Cost to complete	~$1
Learning level	Advanced (300)
AWS services	AWS Organizations AWS CloudFormation AWS Identity and Access Management (IAM) AWS Step Functions AWS Lambda Amazon CloudWatch Amazon EventBridge Amazon Simple Storage Service (Amazon S3) Amazon Simple Notification Service (Amazon SNS)

Overview

This solution uses an Amazon EventBridge rule to invoke the Step Functions state machine. The state machine then invokes a Lambda function to retrieve the account IDs from AWS Organizations. Finally, the function passes them to a second Lambda function that generates the IAM credential report. The process that creates the report must assume an IAM role within each account. To facilitate the role creation, an AWS CloudFormation stack set deploys the role for the Lambda function into all the accounts linked to the management account.

The Lambda function that generates the reports invokes report generation and checks for report completion status. If report generation is still in progress, the Lambda function exits and the state machine waits for 1 minute. This cycle continues until the report completes, or it reaches 45 total wait cycles and exits with an error. When the report generates, the output file is copied to the defined Amazon S3 bucket and secured with the default encryption setting of the bucket. If the error condition invokes (45 wait cycles/45 minutes), Amazon Simple Notification Service (SNS) sends a message to the email address provided during deployment.

Figure 1 shows the architecture to demonstrate automating IAM credential reports.

Figure 1. Automating IAM credential reports architecture diagram

Figure 2 shows a detailed workflow of a state machine.

Figure 2. Detailed state machine workflow

Prerequisites

For this walkthrough, you need the following:

An AWS account
AWS Organizations with multiple accounts
The AWS Serverless Application Model (AWS SAM) CLI
(Optional) AWS Command Line Interface (AWS CLI)

Walkthrough

AWS SAM is an open framework that provides syntax for deploying serverless applications. During the deployment, the AWS SAM CLI packages the code and translates the YAML template into a CloudFormation template. AWS SAM moves the code and templates to Amazon S3 for resource deployment.

In this blog post, we focus on one solution. However, the GitHub repository provides an alternative deployment option to store reports in a separate logging/security account.

Deploy the solution using AWS SAM

This deployment option uses the core AWS SAM template. A single command starts deployment in Step 4 later. The S3 bucket that contains the IAM credential reports is created in the management account.

Step 1: Get root ID

Find the value of the root ID in the management account in the AWS Management Console, or use the following command (if you have the AWS CLI installed):

aws organizations list-roots

The root ID is in the format r-XXXX. You need this value in Step 4 later, so write it down.

Step 2: Get AWS Organizations ID

Find the value of the AWS Organizations ID in the management account in the AWS Management Console, or use the following command (if you have the AWS CLI installed):

aws organizations describe-organization

The Organizations ID is in the format o-xxxxxxxxxxxx. You need this value in Step 4 later, so write it down.

Step 3: Clone the solution repository

Use the following command to clone the solution repository from GitHub.

git clone https://github.com/aws-samples/automating-iam-credential-reports-for-large-aws-organizations.git

Step 4: Deploy the solution

To create all resources necessary for the solution, use the AWS SAM CLI to deploy the core template. If you prefer, use an existing S3 bucket in the management account.

Navigate to the root of the solution, and run the following commands while authenticated as the management account:

cd src/

sam deploy --stack-name iam-cred-report-core --template iam-cred-report-core-template.yml --config-file iam-cred-report-core-config.toml --guided --capabilities CAPABILITY_NAMED_IAM

The previous command transforms the AWS SAM template into a CloudFormation stack template, and then creates the stack. The configuration file referenced provides initial values and storage for your responses to the guided questions for later jobs.

Provide the following details:

Stack Name [iam-cred-report-core]: Provide a custom name or press Enter to use the default.
AWS Region [us-east-1]: Provide the Region in which to deploy this solution.
Parameters:
- pS3BucketStatus [New]: Keep blank to accept the default setting to create a new Amazon S3 bucket, or enter Existing to use an existing bucket.
- pS3BucketTargetName []: Provide the name for the new or existing S3 bucket.
- pKMSKeyStatus [None]: Enter New to create a new KMS key.
- pKMSKeyArn [None]: Keep blank to accept the default setting of None.
- pSNSSubEmail []: Provide an email address for the SNS topic to send account-level errors, or keep blank to opt out.
- pIAMCredentialGenRoleName [iam-credential-report-generation-role]: Provide a custom name for the role or keep blank to use the default.
- pOrganizationRootId []: Enter the Organizations root ID (r-xxxx).
- pOrganizationId []: Enter the Organizations ID (o-xxxxxxxxxxxx).
- pLogsRetentionInDays [90]: Enter the number of days to retain the Amazon CloudWatch log groups logs, or keep the default setting of 90 days. Allowed values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653.
- pEventBridgeTriggerHour [08]: Enter the two-digit hour (00-23) in UTC time used to trigger the EventBridge rule each day, or keep the default to trigger at 08:00:00 UTC.
- pTagKey1 [tagkey]: Enter a tag key name to add to the resources.
- pTagValue1 [tagvalue]: Enter a tag value to associate with the tag key.
Confirm changes before deploy [Y/n]: Verify your inputs, and enter Y to proceed.
Allow AWS SAM CLI IAM role creation [Y/n]: Enter Y to allow for role creation.
Disable rollback [Y/n]: Enter Y to disable rollback.
Save arguments to configuration file [Y/n]: Enter Y to save your configuration settings, if desired.

CloudFormation Outputs from deployed stack

The Outputs section of the AWS SAM CLI process includes the Lambda function execution role ARN and the S3 bucket name. For this deployment option, the Output is only for reference.

Figure 3 shows the Outputs details of the AWS SAM CLI process.

Figure 3. Outputs section of the AWS SAM CLI

Accept the Amazon SNS email confirmation

If you provided an email address, you’ll get an email to confirm for the SNS subscription to send notifications about nonprocess fatal errors for a single account.

Figure 4 shows the email confirmation of the SNS subscription.

Figure 4. AWS Notification Subscription Confirmation email

Invoking the state machine

The EventBridge rule invokes the state machine at the hour specified in the deployment. However, you can invoke the state machine manually from the AWS Management Console. Select the state machine details panel, and choose Start execution. No values are needed for the input JSON.

Figure 5 shows how to manually invoke the state machine.

Figure 5. Manually invoking the state machine.

Accessing the IAM credential reports

The Lambda function stores the IAM credential reports in the S3 bucket with a prefix that represents the date. Invoking the process multiple times in a single day overwrites the prior contents. Each IAM credential report includes the account number in the name.

Figure 6 shows the IAM credential reports stored in the S3 bucket.

Figure 6. S3 bucket where IAM credential reports are stored

Cleanup

To avoid incurring future changes, complete the following steps to delete the resources created by this solution:

If you created a new S3 bucket, navigate to the Amazon S3 console. Select the bucket where the IAM credential reports are stored, and empty the bucket to remove all the contents.
Navigate to the CloudFormation console. Select the stack named iam-cred-report-core, or the custom name you provided during deployment, and delete the stack.

Conclusion

In this post, we demonstrated a method of consolidating IAM credential reports for AWS Organizations into an S3 bucket using Step Functions and Lambda. Having an organizational view of security data allows you to take a more holistic approach to managing security and compliance needs. When you automate the collection of that data, it reduces the effort and likelihood of errors and omissions.

About the authors

Thaddeus (Ted) Nowak

Thaddeus (Ted) Nowak is an Enterprise Support Lead at AWS. He helps Enterprise Support customers improve operations and resiliency. His background is in databases and security. In his free time, he enjoys hiking, photography, and other outdoor activities.

IB Oyewumi

IB Oyewumi was a Senior Security Consultant at AWS. He focused on helping customers architect, build, scale, and optimize security controls, risk management, and compliance.

Raul Radu

Raul Radu is a Senior Security Consultant with AWS Professional Services. He helps organizations secure their AWS environments and workloads in the cloud. He is passionate about privacy and security in a connected world.