Skip to content

Commit

Permalink
Merge PR openebs#196 OpenEBS AWS Cloud Deployment
Browse files Browse the repository at this point in the history 
Updates to the README.md file and New script for OpenEBS AWS Cloud Deployment
  • Loading branch information
@kmova
kmova authored Aug 18, 2017
2 parents 9be00e0 + 6ef85d3 commit b88b5b5
Show file tree
Hide file tree
Showing 2 changed files with 308 additions and 60 deletions.
190 changes: 130 additions & 60 deletions e2e/terraform/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,103 +1,173 @@
# OpenEBS On The Cloud - Deployment Using Terraform and kops (Kubernetes Operations)
# OpenEBS on the Cloud - Deployment using Terraform and kops (Kubernetes Operations)

*Terraform* enables you to safely and predictably create, change, and improve production infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
The purpose of this user guide is to provide the instructions to set up a Kubernetes cluster on AWS (Amazon Web Services) and have OpenEBS running in hyperconverged mode.

*kops (Kubernetes Operations)* helps you create, destroy, upgrade and maintain production-grade, highly available, Kubernetes clusters from the command line.
## Pre-requisites

The purpose of this user guide is to provide the user with instructions to set up a Kubernetes cluster on AWS (Amazon Web Services) and have OpenEBS running in hyperconverged mode in the cluster.
Perform the following procedure to setup-up the pre-requisites:

**Pre-requisites:**
**Amazon Web Services (AWS):**

- Active AWS account.
- Hashicorp terraform >= 0.9.11
- kops >= 1.6.2
- Signup for AWS [here](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html).

## Prepare AWS For kops And Terraform
People who already have an AWS account can skip the above step.

- Signup for AWS (Amazon Web Services).
- Amazon provides a AccessKeyID and a SecretAccessKey for the user.
- Save the AccessKeyID and a SecretAccessKey as environment variables in your local machine.
- Start your browser.
- Open *AWS Management Console*.
- Select *IAM* under *Security, Identity & Compliance*.
- In the *Dashboard*, click *Users*.
- Click *Add User* button.
- In the *User name* textbox, enter *openebsuser01* as the username.
- Select *Access Type* as *Programmatic access*.
- Click *Next Permissions*.
- Select *Attach existing policies directly*.
- In the *Search Box* enter *IAMFullAccess* and select the listed permission.
- Click *Next Review*.
- Click *Create User*.

A user, named *openebsuser01* will be created and an *Access key ID* and a *Secret access key*
will be assigned.

```
export AWS_ACCESS_KEY_ID=<access key>
export AWS_SECRET_ACCESS_KEY=<secret key>
Example:
User Access key ID Secret access key
openebsuser01 AKIAI3MRLHNGU6CNKJQE udxZi33tvSptXCky31kEt4KLRS6LSMMsmmdLx501
```

- In AWS create a IAM (Identity and Access Management) user group.
- Assign the following permissions to the group.
>Note the *Access key ID* and the *Secret access key* as AWS will not display it again.
- AmazonEC2FullAccess
- AmazonRoute53FullAccess
- AmazonS3FullAccess
- IAMFullAccess
- AmazonVPCFullAccess
**kops, terraform and awscli:**

- Now create a new user and add him to the group created earlier.
The following tools have to be installed on your local workstation:

## Use kops To Create The Terraform File
- `awscli`
- `kops >= 1.6.2`
- `terraform >= 0.9.11`

- *kops* has the ability to output a configuration file that *terraform* can utilize to bring up the infrastructure.
- We will be using *kops create cluster* command to create a cluster of:
We have created a script that does most of the work for you. Download the script file called `oebs-cloud.sh`.

- One Master
- Two Minions
```
$ mkdir -p openebs
$ cd openebs
$ wget https://raw.githubusercontent.com/openebs/openebs/master/e2e/terraform/oebs-cloud.sh
$ chmod +x oebs-cloud.sh
```

List the operations performed by the script:
```
$ ./oebs-cloud.sh
Usage : oebs-cloud.sh --setup-local-env
oebs-cloud.sh --create-cluster-config
oebs-cloud.sh --ssh-aws-ec2
Sets Up OpenEBS On AWS
- Copy the below command in a terminal.
-h|--help Display this help and exit.
--setup-local-env Sets up, AWSCLI, Terraform and KOPS.
--create-cluster-config Generates a terraform file(.tf) and Passwordless SSH
--ssh-aws-ec2 SSH to Kubernetes Master on EC2 instance.
```
kops create cluster --cloud=aws \
--master-size=t2.micro \
--master-zones=us-east-1a \
--node-count=2 \
--node-size=t2.micro \
--zones=us-east-1a \
--ssh-public-key=/home/ubuntu/.ssh/id_rsa.pub \
--image=ami-9c3f6ce7 \
--target=terraform \
--out=. \
--name=openebs.k8s.local
```

- Note the --target=terraform and --out=.\ parameters, these parameters tell kops to output a terraform file (.tf) in the current directory.
- The generated terraform file(.tf), is named kubernetes.tf by default.
- kops establishes a passwordless SSH connection between the local workstation and the remote EC2 instances.
- kops also dumps the SSH connection details that can be used to connect to the remote EC2 instances.
Run the following command to install the tools:

## Creating Cluster On AWS Using Terraform File
```
$ ./oebs-cloud.sh --setup-local-env
```

- terraform usually looks for terraform file(.tf) in the location where the terraform commands are run.
- Run the command *terraform plan* to see the preview of the changes to the AWS infrastructure.
- terraform outputs a chuck of JSON data containing the changes that would be applied on AWS.
- *terraform plan* command also acts as a verifier of terraform files. It outputs any errors that the terraform file might have, that could break the creation of the infrastructure when running the *terraform apply* command.
- Run the command *terraform apply* to initiate the creation of the infrastructure.
The command will install `awscli`, `terraform` and `kops` on the workstation.

## Route 53-Based DNS Vs Gossip-Based DNS
**Updating .profile file:**

Creating a Kubernetes cluster using kops requires a top-level domain or a sub domain and setting up Route 53 hosted zones. This domain allows the worker nodes to discover the master and the master to discover all the etcd servers. This is also needed for kubectl to be able to talk directly with the master.
The tools `awscli` and `kops` require the AWS credentials to access AWS services.

But from a development point of view, the Route 53 hosted domains are charged and is not part of the Free-Tier that AWS offers.
- Use the credentials that were generated earlier for the user *openebsuser01*.
- Add path */usr/local/bin* to the PATH environment variable.

With the release of kops 1.6.2, Kubernetes has introduced a gossip-based DNS-free protocol, which is based on Weave Mesh. This protocol allows the master and worker nodes to be discoverable without having a registered domain. But the protocol requires that while creating the cluster, the name that is provided to the cluster must end with *.k8s.local*
```
$ vim ~/.profile
# Add the AWS credentials as environment variables in .profile
export AWS_ACCESS_KEY_ID=<access key>
export AWS_SECRET_ACCESS_KEY=<secret key>
# Add /usr/local/bin to PATH
PATH="$HOME/bin:$HOME/.local/bin:/usr/local/bin:$PATH"
$ source ~/.profile
```

## Creating the Config For Cluster
- We will be generating a terraform file(.tf) that will later spawn:
- One Master
- Two Minion
- Run the following command in a terminal.

```
$ ./oebs-cloud.sh --create-cluster-config
```

- A terraform file `kubernetes.tf` is generated in the same directory.
- Passwordless SSH connection between the local workstation and the remote EC2 instances is established.

## Create Cluster on AWS using Terraform

- Run the following command to verify successful installation of `terraform`.

```
$ terraform
Usage: terraform [--version] [--help] <comman> [args]
The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.
Common commands:
apply Builds or changes infrastructure
console Interactive console for Terraform interpolations
# ...
```

- Run the command `terraform init` to initialize `terraform`.
- Run the command `terraform plan` from the directory where the generated terraform file (.tf) is placed.
- `terraform` outputs a chunk of JSON data containing changes that would be applied on AWS.
- `terraform plan` command verifies your terraform files (.tf) and outputs any errors that it encountered.
- Fix these errors and re-verify with `terraform plan` before running the `terraform apply` command.
- Run the command `terraform apply` to initiate creation of the infrastructure.

## SSH to the Master Node

- From your workstation, run the following command to connect to the EC2 instance running the Kubernetes Master.

```
$ ./oebs-cloud.sh --ssh-aws-ec2
```

## Deploy OpenEBS On AWS
- You should now be running inside the EC2 instance.

Deploying OpenEBS requires Kubernetes to be already available on the EC2 instances. Lets verify if a Kubernetes cluster has been created.
## Deploy OpenEBS on AWS

- SSH into the master node of the cluster and type the following command on the terminal.
Deploying OpenEBS requires Kubernetes to be already running on the EC2 instances. Verify if a Kubernetes cluster has been created.

```
kubectl get nodes
ubuntu@ip-172-20-53-140:~$ kubectl get nodes
NAME STATUS AGE VERSION
ip-172-20-36-126.ec2.internal Ready 1m v1.7.0
ip-172-20-37-115.ec2.internal Ready 1m v1.7.0
ip-172-20-53-140.ec2.internal Ready 3m v1.7.0
```

- This will output any cluster information if the cluster was created.
- This will output any cluster information if the cluster was already created.
- Download the *openebs-operator* and *openebs-storage-classes* yamls from the below location.

```
wget https://raw.githubusercontent.com/openebs/openebs/master/k8s/openebs-operator.yaml
wget https://raw.githubusercontent.com/openebs/openebs/master/k8s/openebs-storageclasses.yaml
```

- Apply the *openebs-operator* and *openebs-storage-classes* to the Kubernetes cluster.
- Apply *openebs-operator* and *openebs-storage-classes* to the Kubernetes cluster.

```
kubectl create -f openebs-operator.yaml
Expand Down
Loading

0 comments on commit b88b5b5

Please sign in to comment.