Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make secrets at cluster startup. #5470

Merged
merged 1 commit into from
Apr 3, 2015
Merged

Make secrets at cluster startup. #5470

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions cluster/gce/configure-vm.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,13 @@ function create-salt-auth() {
kubelet_auth_file="/srv/salt-overlay/salt/kubelet/kubernetes_auth"
(umask 077;
echo "{\"BearerToken\": \"${KUBELET_TOKEN}\", \"Insecure\": true }" > "${kubelet_auth_file}")

# Generate tokens for other "service accounts". Append to known_tokens.
local -r service_accounts=("system:scheduler" "system:controller_manager" "system:logging" "system:monitoring" "system:dns")
for account in "${service_accounts[@]}"; do
token=$(dd if=/dev/urandom bs=128 count=1 2>/dev/null | base64 | tr -d "=+/" | dd bs=32 count=1 2>/dev/null)
echo "${token},${account},${account}" >> "${known_tokens_file}"
done
}

function download-release() {
Expand Down
62 changes: 55 additions & 7 deletions cluster/saltbase/salt/kube-addons/kube-addons.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,52 @@
# managed result is of that. Start everything below that directory.
KUBECTL=/usr/local/bin/kubectl

# $1 addon to start.
function create-kubernetesauth-secret() {
local -r token=$1
local -r username=$2
local -r safe_username=$(tr -s ':_' '--' <<< "${username}")

# Make secret with a kubernetes_auth file with a token.
# TODO(etune): put apiserver certs into secret too, and reference from authfile,
# so that "Insecure" is not needed.
kafile=$(echo "{\"BearerToken\": \"${token}\", \"Insecure\": true }" | base64 -w0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly local, but prior to this so the error code is separated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you are asking for.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

local kafile
kafile=$( )

It's less important in the script as it stands because it currently violates our existing standards of having:

set -o errexit
set -o nounset
set -o pipefail

... but if errexit were set, the setting of the variable and the "local" need to be on separate lines. That's all I was saying.

read -r -d '' secretjson <<EOF
apiVersion: v1beta1
kind: Secret
id: token-${safe_username}
data:
kubernetes-auth: ${kafile}
EOF
create-resource-from-string "${secretjson}" 100 10 "Secret-for-token-for-user-${username}" &
# TODO: label the secrets with special label so kubectl does not show these?
}

# $1 filename of addon to start.
# $2 count of tries to start the addon.
# $3 delay in seconds between two consecutive tries
function start_addon() {
addon=$1;
tries=$2;
delay=$3;
local -r addon_filename=$1;
local -r tries=$2;
local -r delay=$3;

create-resource-from-string "$(cat ${addon_filename})" "${tries}" "${delay}" "${addon_filename}"
}

# $1 string with json or yaml.
# $2 count of tries to start the addon.
# $3 delay in seconds between two consecutive tries
# $3 name of this object to use when logging about it.
function create-resource-from-string() {
local -r config_string=$1;
local -r tries=$2;
local -r delay=$3;
local -r config_name=$1;
while [ ${tries} -gt 0 ]; do
${KUBECTL} create -f ${addon} && \
echo "== Successfully started ${addon} at $(date -Is)" && \
echo "${config_string}" | ${KUBECTL} create -f - && \
echo "== Successfully started ${config_name} at $(date -Is)" && \
return 0;
let tries=tries-1;
echo "== Failed to start ${addon} at $(date -Is). ${tries} tries remaining. =="
echo "== Failed to start ${config_name} at $(date -Is). ${tries} tries remaining. =="
sleep ${delay};
done
return 1;
Expand All @@ -41,6 +74,21 @@ function start_addon() {
# was already enforced by salt, and /etc/kubernetes/addons is the
# managed result is of that. Start everything below that directory.
echo "== Kubernetes addon manager started at $(date -Is) =="

# Generate secrets for "internal service accounts".
# TODO(etune): move to a completely yaml/object based
# workflow so that service accounts can be created
# at the same time as the services that use them.
# NOTE: needs to run as root to read this file.
# Read each line in the csv file of tokens.
while read line; do
# Split each line into the token and username.
IFS=',' read -a parts <<< "${line}"
token=${parts[0]}
username=${parts[1]}
create-kubernetesauth-secret "${token}" "${username}"
done < /srv/kubernetes/known_tokens.csv

for obj in $(find /etc/kubernetes/addons -name \*.yaml); do
start_addon ${obj} 100 10 &
echo "++ addon ${obj} starting in pid $! ++"
Expand Down