Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker - allow use of secrets when using Docker Swarm #2890

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Docker - allow use of secrets when using Docker Swarm #2890

wants to merge 2 commits into from

Conversation

zoomequipd
Copy link
Contributor

@zoomequipd zoomequipd commented Oct 20, 2020 •
edited
Loading

This PR modifies docker/scripts/setup_env to allow a user to pass sensitive data via Docker Secrets, which when used reference a file within the container at /run/secrets/

This method leverages code from the mysql docker which achieves the same objective.

An example use would be

  1. first create some secrets
openssl rand -base64 20 | docker secret create huginn_mysql_root_password -
openssl rand -base64 20 | docker secret create huginn_mysql_huggin_password -

echo -n '[...snip...]' | docker secret create huginn_twitter_oauth_secret -
echo -n '[...snip...]' | docker secret create huginn_twitter_oauth_key -
echo -n '[...snip...]' | docker secret create huginn_smtp_password -
  1. create the env-file
MYSQL_DATABASE=huginn_production
MYSQL_USER=huginn
MYSQL_PASSWORD_FILE=/run/secrets/huginn_mysql_huginn_password
MYSQL_ROOT_PASSWORD_FILE=/run/secrets/huginn_mysql_root_password
MYSQL_PORT_3306_TCP_ADDR=huginn_mysql
DATABASE_USERNAME=huginn
DATABASE_PASSWORD_FILE=/run/secrets/huginn_mysql_huginn_password
TWITTER_OAUTH_SECRET_FILE=/run/secrets/huginn_twitter_oauth_secret
TWITTER_OAUTH_KEY_FILE=/run/secrets/huginn_twitter_oauth_key
SMTP_PASSWORD_FILE=/run/secrets/huginn_smtp_password
# USE_GRAPHVIZ_DOT=dot
  1. map them into the container and start the services
docker service create --replicas 1 --name huginn_mysql \
    --env-file=/data/huginn.env \
    --secret huginn_mysql_huginn_password \
    --secret huginn_mysql_root_password \
    --mount type=volume,src=huginn_mysql_conf,dst=/etc/mysql/conf.d \
    --mount type=volume,src=huginn_mysql_data,dst=/var/lib/mysql \
    mysql:latest

docker service create --replicas 1 --name huginn_web \
    --publish published=80,target=3000 \
    --env-file=/data/huginn.env \
    --secret huginn_mysql_root_password \
    --secret huginn_mysql_huginn_password \
    --secret huginn_twitter_oauth_secret \
    --secret huginn_twitter_oauth_key \
    --secret huginn_smtp_password \
    huginn/huginn-single-process

docker service create --replicas 5 --name huginn_delayed_job \
    --env-file=/data/huginn.env \
    --secret huginn_mysql_huginn_password \
    --secret huginn_twitter_oauth_secret \
    --secret huginn_twitter_oauth_key \
    --secret huginn_smtp_password \
    huginn/huginn-single-process /scripts/init script/delayed_job run

docker service create --replicas 1 --name huginn_agent_runner \
    --env-file=/data/huginn.env \
    --secret huginn_mysql_huginn_password \
    --secret huginn_twitter_oauth_secret \
    --secret huginn_twitter_oauth_key \
    --secret huginn_smtp_password \
    huginn/huginn-single-process /scripts/init bin/agent_runner.rb

@zoomequipd
use the mysql docker method of handling the _FILE suffix for env vars…
69968c0 
… so they can be passed in using docker swarm supported secrets
zoomequipd
zoomequipd commented Oct 20, 2020
View reviewed changes
docker/scripts/setup_env
@@ -33,11 +56,33 @@ else
START_MYSQL=${START_MYSQL:-true}
fi

# Initialize values that might be stored in a file
file_env 'DATABASE_PASSWORD' ${HUGINN_DATABASE_PASSWORD:-${DATABASE_PASSWORD:-password}}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This passes the default value taken from the original variable definition from Line 85 (now commented out)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might need to be defined for every var depending on the behavior when passing in values via other means (env vars, .evn-file, etc)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might need to be defined for every var depending on the behavior when passing in values via other means (env vars, .evn-file, etc)

this appears to not be a problem, though I can't say I fully understand why it's not a problem. But in my testing I changed variable not passed to the file_env function just fine via the different methods.

@zoomequipd zoomequipd marked this pull request as draft October 21, 2020 12:46
@zoomequipd
Copy link
Contributor Author

zoomequipd commented Oct 21, 2020 •
edited
Loading

TODO

  • Test passing in values as environmental variables
  • Test passing in values by being defined in the env-file
  • Test included docker-compose up
  • Test multiprocess image

I tested each option by starting the docker containers (built locally as to include the modified setup_env file) using the various methods to pass in the values. All methods resulted in the values being correctly populated into the /app/.env file within the container.

@zoomequipd zoomequipd marked this pull request as ready for review October 22, 2020 02:02
@MarioRicalde MarioRicalde mentioned this pull request Aug 9, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@zoomequipd