Dockerfiles for the tstrohmeier/spark-development images and example setup
@Dockerhub: https://hub.docker.com/r/tstrohmeier/spark-development/
Base Images
- tstrohmeier/spark-development:2.2.0
- tstrohmeier/spark-development:2.4.5
Derived Images:
- tstrohmeier/spark-development:2.4.5-gradle-6.5
- tstrohmeier/spark-development:2.4.5-pip3
.
├── ...
├── docker # contains all the Dockerfiles
│ ├── gradle # extends the spark-development image with gradle
│ | └── ...
│ ├── pip3 # extends the spark-development image with python3 & pip3
│ | └── ...
│ ├── spark # the spark-development base image
│ | └── ...
| └── ...
├── docker-compose-templates # docker-compose template files for new projects
│ ├── gradle # Gradle template
│ | └── ...
│ ├── sbt # SBT template
│ | └── ...
| └── ...
├── examples # Example Projects
│ ├── gradle # Gradle example project
│ | └── ...
│ ├── pyspark # pyspark example project
│ | └── ...
│ ├── sbt # Sbt example project
│ | └── ...
| └── ...
├── scripts # CI / CD build scripts
└── ...
- spark
- hadoop
- sbt
Just copy the right docker-compose template file into your project root folder and configure src location and the location of the cache location of your dependency management tool.
In the examples folder you can find examples for different programming languages and dependency management tools. Just copy the folder and modify for your needs
# version 2.4.5
docker pull tstrohmeier/spark-development:2.4.5Any improvement as Pull Request is welcome!
Thanks goes to these wonderful people:
 Thomas 💻 👀 |
 Fibs7000 💻 |
This project follows the all-contributors specification. Contributions of any kind welcome!