Before proceeding, be sure to review our architecture overview, and our minimum requirements. We are also more than happy to help you design your ComputeStacks environments. Please contact us to learn more.
- Ensure your local machine:
-
Has ansible installed and configured, with the following plugins installed:
ansible-galaxy install -r requirements.yml
-
If you used one of our terraform setup scripts to create your cluster, then you should already have a skeleton inventory.yml
file that you can place in this directory. Otherwise, cp inventory.yml.sample inventory.yml
.
Ensure you fill out all the variables and ensure everything is correct. To see all available settings, check each role and look at their defaults/main.yml
file.
Various parts of this ansible package will require the domains defined in your inventory.yml
file to be correctly setup. Please configure those domains and ensure the changes have been propagated before proceeding.
Example DNS Settings:
a.dev.cmptstks.net. IN A %{floating_ip}
metrics.dev.cmptstks.net. IN A %{PUBLIC IP OF METRICS SERVER}
*.a.dev.cmptstks.net. IN CNAME a.dev.cmptstks.net.
portal.dev.cmptstks.net. IN A %{controller ip address}
cr.dev.cmptstks.net. IN CNAME portal.dev.cmptstks.net.
Note: If you used one of our terraform setup scripts, then the following steps would have already been performed for you.
Ensure all hostnames are configured properly on each node. Our system expects the hostnames on nodes to be 1 lowercase word (dashes or underscores are ok), no special characters or spaces. They should not be FQDN or have the dot notation.
Examples: node101, node102, node103
hostname node101 && echo "node101" > /etc/hostname && echo "127.0.0.1 node101" >> /etc/hosts
The reason these are critical are two fold:
a) When metrics are stored for containers or nodes, the hostname is added as a label. This is how ComputeStacks is able to filter metrics for the correct container/node. b) Backup jobs are assigned to the node by their hostname. If there is a mismatch in the controller, then jobs will not run.
apt update && apt -y install openssl ca-certificates linux-headers-amd64 python3 python3-pip python3-openssl python3-apt python3-setuptools python3-wheel && pip3 install ansible
If you're using a setting other than the default 1500
, please add the following to the main vars:
section of your inventory.yml
file:
container_network_mtu: 1400 # Set the desired MTU for containers to use
Due to an ongoing issue with the container network platform we use, IPv6 is currently not supported on our container nodes. We're able to bring ipv6 connectivity by either using a dedicated load balancer on a separate virtual machine, or by configuring the controller to proxy ipv6 traffic.
For the time being, our installer will disable ipv6 directly on the node. However, we recommend also cleaning out the /etc/sysconfig/network-scripts/ifcfg-*
files to remove any ipv6 specific settings, and setting IPV6INIT=no
.
We recommend performing this step prior to running this ansible package.
ansible-playbook -u root -i inventory.yml main.yml --tags "bootstrap"
The last step in this script will reboot servers to finalize configuration.
After running and allowing the servers to reboot, you can perform some basic validation by running:
ansible-playbook -u root -i inventory.yml main.yml --tags "validate"
To add nodes to existing Availability Zone, simply add the new nodes to the same inventory file you used to deploy the initial cluster and change etcd_initial_cluster_state
for each NEW node from new
to existing
. Existing nodes MUST remain at new
.
Please note that you will need to ensure the number of nodes you're deploying will keep the etcd cluster in quorum. As per the etcd documentation, this should be (n/2)+1
nodes. So 3, 5, 7, or 9 nodes per availability zone.
If you're moving from a single-node to a cluster, you may need to change the peer url
of the existing standalone etcd server.
Set the etcd peer URL
Login to the existing node and set the correct peer url.
Note: If your peer url is already configured as defined below, you may skip this process.
- Find your memberID by running:
etcdctl member list | cut -d, -f1
-
Update the peer url to match the client url, but with port
2380
etcdctl member update 4370888ee4d9c217 --peer-urls="https://192.168.217.4:2380"
. Note thehttps
!
Now you may run the ansible package with:
ansible-playbook -u root -i inventory.yml main.yml --tags "addnode"
To add a new region or availability zone, make a copy of your previous inventory.yml
file and make the following changes:
- Change
region_name
and/or,availability_zone_name
- Replace all nodes with your new nodes
- (optional) Change metrics / backup servers. You can re-use the existing ones if you wish.
Re-run the bootstrap command:
ansible-playbook -u root -i inventory.yml main.yml --tags "bootstrap"
Linux
Install pyenv for your local (non-root) user account.
You can set the new version with pyenv global 3.9.1
(replace 3.9.1
with the version you installed)
python -m pip install --user ansible
echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.bashrc
Note: Check if you have a .bashrc
file, it may be .bash_profile
for your distribution.
This will ensure you have the most recent version of ansible.
In order to allow swap limitations set on containers, you need to perform the following on each node:
- Modify the file
/etc/default/grub
- Add
cgroup_enable=memory swapaccount=1
to the existingGRUB_CMDLINE_LINUX_DEFAULT
setting - run
update-grub
- reboot
Note: This can add about 1% of overhead.
How to Resolve set-hostname: current hostname was changed outside NetworkManager: '<hostname>'
in logs:
Edit /etc/NetworkManager/NetworkManager.conf
and add hostname-mode=none
to the [main]
block, and reboot the server.
Example:
[main]
#plugins=ifcfg-rh,ibft
hostname-mode=none