-
Clone the repository
git clone https://github.com/Shegzimus/DE_NASA_NeoW_Pipeline
-
Create a virtual environment in your local machine
python3 -m venv venv
-
Activate the virtual environment
source venv/bin/activate
-
Install dependencies
pip install -r airflow/requirements.txt
-
Create directories to store your google credentials
cd airflow && mkdir -p .google
-
Build the Docker Image
docker build -d --
-
Start the Docker containers
docker-compose up -d
-
Launch the Airflow web UI
open http://localhost:8083
-
Notifications
You must be signed in to change notification settings - Fork 0
Airflow powered ETL pipeline for moving Near-Earth-Object data from NASA to Google Cloud
License
Shegzimus/DE_NASA_NeoW_Pipeline
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
Airflow powered ETL pipeline for moving Near-Earth-Object data from NASA to Google Cloud
Topics
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published