Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Splitting up some of the Docker build steps #8775

Merged
merged 2 commits into from
Dec 6, 2019
Merged

Splitting up some of the Docker build steps #8775

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
41cb47e
Splitting up some of the Docker build steps
craig-rueda Dec 5, 2019
837bd0b
Adding dev target which includes requirements-extra / Updating docker…
craig-rueda Dec 5, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Splitting up some of the Docker build steps
  • Loading branch information
@craig-rueda
craig-rueda committed Dec 5, 2019
commit 41cb47ef9e23e1094a572f505a992744dfc5d058
41 changes: 29 additions & 12 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
# limitations under the License.
#

######################################################################
# PY stage that simply does a pip install on our requirements
######################################################################
ARG PY_VER=3.6.9
FROM python:${PY_VER} AS superset-py

Expand All @@ -26,24 +29,35 @@ RUN mkdir /app \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*

COPY ./requirements.txt ./docker/requirements-extra.txt ./setup.py ./MANIFEST.in ./README.md ./app/
COPY superset /app/superset

# First, we just wanna install requirements, which will allow us to utilize the cache
# in order to only build iff requirements change
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: iff typo

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed iff -> if and only if

COPY ./requirements.txt ./docker/requirements-extra.txt ./app/
RUN cd /app \
&& pip install -r requirements.txt -r requirements-extra.txt \
&& pip install -e .
&& pip install -r requirements.txt -r requirements-extra.txt
mistercrunch marked this conversation as resolved.
Show resolved Hide resolved


######################################################################
# Node stage to deal with static asset construction
######################################################################
FROM node:10-jessie AS superset-node

COPY ./superset/assets /app/superset/assets
# NPM ci first, as to NOT invalidate previous steps except for when package.json changes
RUN mkdir -p /app/superset/assets
COPY ./superset/assets/package* /app/superset/assets/
RUN cd /app/superset/assets \
&& npm ci

# Next, copy in the rest and let webpack do its thing
COPY ./superset/assets /app/superset/assets
# This is BY FAR the most expensive step (thanks Terser!)
RUN cd /app/superset/assets \
&& npm ci \
&& npm run build \
&& rm -rf node_modules


######################################################################
# Final lean image...
######################################################################
ARG PY_VER=3.6.9
FROM python:${PY_VER}

Expand All @@ -57,22 +71,25 @@ ENV LANG=C.UTF-8 \

RUN useradd --user-group --no-create-home --no-log-init --shell /bin/bash superset \
&& mkdir -p ${SUPERSET_HOME} ${PYTHONPATH} \
&& chown -R superset:superset /app \
&& apt-get update -y \
&& apt-get install -y --no-install-recommends \
build-essential \
default-libmysqlclient-dev \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*

COPY --from=superset-py --chown=superset:superset /app/superset /app/superset
COPY --from=superset-py /usr/local/lib/python3.6/site-packages/ /usr/local/lib/python3.6/site-packages/
# Copying site-packages doesn't move the CLIs, so let's copy them one by one
COPY --from=superset-py /usr/local/bin/superset /usr/local/bin/gunicorn /usr/local/bin/celery /usr/local/bin/flask /usr/bin/
COPY --from=superset-py /app/apache_superset.egg-info /app/apache_superset.egg-info

COPY --from=superset-py /usr/local/bin/gunicorn /usr/local/bin/celery /usr/local/bin/flask /usr/bin/
COPY --from=superset-node /app/superset/assets /app/superset/assets

## Lastly, let's install superset itself
COPY superset /app/superset
COPY setup.py MANIFEST.in README.md /app/
RUN cd /app \
&& chown -R superset:superset * \
&& pip install -e .

COPY ./docker/docker-entrypoint.sh /usr/bin/

WORKDIR /app
Expand Down
29 changes: 25 additions & 4 deletions docker/docker-init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,26 +15,47 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
set -ex
set -e

STEP_CNT=4

echo_step() {
cat <<EOF

######################################################################


Init Step ${1}/${STEP_CNT} [${2}] -- ${3}


######################################################################

EOF
}

# Create an admin user
echo "Setting up admin user..."
echo_step "1" "Starting" "Setting up admin user"
flask fab create-admin \
--username admin \
--firstname Superset \
--lastname Admin \
--email admin@superset.com \
--password admin
echo_step "1" "Complete" "Setting up admin user"

# Initialize the database
echo "Migrating the DB..."
echo_step "2" "Starting" "Applying DB migrations"
superset db upgrade
echo_step "2" "Complete" "Applying DB migrations"

if [ "$SUPERSET_LOAD_EXAMPLES" = "yes" ]; then
# Load some data to play with
echo_step "3" "Starting" "Loading examples"
superset load_examples
echo_step "3" "Complete" "Loading examples"
fi

# Create default roles and permissions
echo "Setting up roles and perms..."
echo_step "4" "Starting" "Setting up roles and perms"
superset init
echo_step "4" "Complete" "Setting up roles and perms"