Skip to content

ComputeStacks/ansible-install

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 Cannot retrieve latest commit at this time.

History

48 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Install ComputeStacks

Before proceeding, be sure to review our architecture overview, and our minimum requirements. We are also more than happy to help you design your ComputeStacks environments. Please contact us to learn more.

Prerequisites

  1. Ensure your local machine:
    • Has ansible installed and configured, with the following plugins installed:

      ansible-galaxy install -r requirements.yml

Inventory File

If you used one of our terraform setup scripts to create your cluster, then you should already have a skeleton inventory.yml file that you can place in this directory. Otherwise, cp inventory.yml.sample inventory.yml.

Ensure you fill out all the variables and ensure everything is correct. To see all available settings, check each role and look at their defaults/main.yml file.

DNS

Various parts of this ansible package will require the domains defined in your inventory.yml file to be correctly setup. Please configure those domains and ensure the changes have been propagated before proceeding.

Example DNS Settings:

a.dev.cmptstks.net. IN A %{floating_ip}
metrics.dev.cmptstks.net. IN A %{PUBLIC IP OF METRICS SERVER}
*.a.dev.cmptstks.net. IN CNAME a.dev.cmptstks.net.
portal.dev.cmptstks.net. IN A %{controller ip address}
cr.dev.cmptstks.net. IN CNAME portal.dev.cmptstks.net.

Requirements for each server

Note: If you used one of our terraform setup scripts, then the following steps would have already been performed for you.

Server Hostnames

Ensure all hostnames are configured properly on each node. Our system expects the hostnames on nodes to be 1 lowercase word (dashes or underscores are ok), no special characters or spaces. They should not be FQDN or have the dot notation.

Examples: node101, node102, node103

hostname node101 && echo "node101" > /etc/hostname && echo "127.0.0.1 node101" >> /etc/hosts

The reason these are critical are two fold:

a) When metrics are stored for containers or nodes, the hostname is added as a label. This is how ComputeStacks is able to filter metrics for the correct container/node. b) Backup jobs are assigned to the node by their hostname. If there is a mismatch in the controller, then jobs will not run.

Install the following packages

apt update && apt -y install openssl ca-certificates linux-headers-amd64 python3 python3-pip python3-openssl python3-apt python3-setuptools python3-wheel && pip3 install ansible

Network MTU Settings

If you're using a setting other than the default 1500, please add the following to the main vars: section of your inventory.yml file:

container_network_mtu: 1400 # Set the desired MTU for containers to use

IPv6 for Container Nodes

Due to an ongoing issue with the container network platform we use, IPv6 is currently not supported on our container nodes. We're able to bring ipv6 connectivity by either using a dedicated load balancer on a separate virtual machine, or by configuring the controller to proxy ipv6 traffic.

For the time being, our installer will disable ipv6 directly on the node. However, we recommend also cleaning out the /etc/sysconfig/network-scripts/ifcfg-* files to remove any ipv6 specific settings, and setting IPV6INIT=no.

We recommend performing this step prior to running this ansible package.


Running

ansible-playbook -u root -i inventory.yml main.yml --tags "bootstrap"

The last step in this script will reboot servers to finalize configuration.

Post-Installation

After running and allowing the servers to reboot, you can perform some basic validation by running:

ansible-playbook -u root -i inventory.yml main.yml --tags "validate"

Adding nodes to an existing availability zone

To add nodes to existing Availability Zone, simply add the new nodes to the same inventory file you used to deploy the initial cluster and change etcd_initial_cluster_state for each NEW node from new to existing. Existing nodes MUST remain at new.

Please note that you will need to ensure the number of nodes you're deploying will keep the etcd cluster in quorum. As per the etcd documentation, this should be (n/2)+1 nodes. So 3, 5, 7, or 9 nodes per availability zone.

If you're moving from a single-node to a cluster, you may need to change the peer url of the existing standalone etcd server.

Set the etcd peer URL

Login to the existing node and set the correct peer url.

Note: If your peer url is already configured as defined below, you may skip this process.

  1. Find your memberID by running: etcdctl member list | cut -d, -f1
  2. Update the peer url to match the client url, but with port 2380
    • etcdctl member update 4370888ee4d9c217 --peer-urls="https://192.168.217.4:2380". Note the https!


Now you may run the ansible package with:

ansible-playbook -u root -i inventory.yml main.yml --tags "addnode"

Add Region / Availability Zone

To add a new region or availability zone, make a copy of your previous inventory.yml file and make the following changes:

  1. Change region_name and/or, availability_zone_name
  2. Replace all nodes with your new nodes
  3. (optional) Change metrics / backup servers. You can re-use the existing ones if you wish.

Re-run the bootstrap command:

ansible-playbook -u root -i inventory.yml main.yml --tags "bootstrap"

FAQ

How to install ansible

Mac OSX

Install Homebrew

brew install ansible

Linux

Install pyenv for your local (non-root) user account.

You can set the new version with pyenv global 3.9.1 (replace 3.9.1 with the version you installed)

python -m pip install --user ansible
echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.bashrc

Note: Check if you have a .bashrc file, it may be .bash_profile for your distribution.

This will ensure you have the most recent version of ansible.

Enable Swap Limit

In order to allow swap limitations set on containers, you need to perform the following on each node:

  1. Modify the file /etc/default/grub
  2. Add cgroup_enable=memory swapaccount=1 to the existing GRUB_CMDLINE_LINUX_DEFAULT setting
  3. run update-grub
  4. reboot

Note: This can add about 1% of overhead.

Troubleshooting

NetworkManager Hostname Error

How to Resolve set-hostname: current hostname was changed outside NetworkManager: '<hostname>' in logs:

Edit /etc/NetworkManager/NetworkManager.conf and add hostname-mode=none to the [main] block, and reboot the server.

Example:

[main]
#plugins=ifcfg-rh,ibft
hostname-mode=none