- Redpanda up and running
- Push some fake data to Redpanda
- Push real-time (real data) from Kraken
-
Extract config parameters
-
Dockerize it
-
Homework --> adjust the code so that instead of a single product_id, the trade_producer produces data for several product_ids = ['BTC/USD', 'BTC/EUR'] My thoughts: you will need to update * the config types * the Kraken Websocket API class
-
Trade to ohlc service
-
Homework: Extract config parameters and dockerize the trade_to_ohlc service.
-
Topic to feature store service -> a Kafka consumer
-
Start the backfill
- Implement a Kraken Historical data reader (trade producer)
- Adjust timestamps used to bucket trades into windows (trade_to_ohlcv)
- Save historical ohlcv features in batches to the offline store (topic_to_feature_store)
- Dockerize our real-time feature pipeline
- Dockerize our backfill pipeline and run it
- Build a functional training pipeline -[ ] Implement a class to read OHLC data from the feature store - [ ] Buid a dummy model to predict price into the future.