This repository is a fork of Temporal Convolutional Networks, which implements the methods/experiments of An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling by Shaojie Bai, J. Zico Kolter and Vladlen Koltun:
@article{BaiTCN2018,
author = {Shaojie Bai and J. Zico Kolter and Vladlen Koltun},
title = {An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling},
journal = {arXiv:1803.01271},
year = {2018},
}
To help my understanding, I've created this fork to add my own additional experiment that puts forward the following sequence prediction problem:
Suppose there are coordinates
A | C A | C A | C
----- , ----- , ... , ----
B | ? B | ? B | ?
To prevent the possibility of perfect prediction, the 4th quadrant is denoted by "?". If the coordinate resides there, there is an equal probability of being one of A, B, or C.
For example:
Create and activate a Python 3.8 virtual environment using pyenv:
pyenv install -v 3.8.14
pyenv virtualenv 3.8.14 tcn-3.8.14
pyenv activate tcn-3.8.14
Install requirements via Poetry:
poetry install
The TCN model can and does learn to improve predictions on the sequences:
poetry run python quadrant_test.py
Train Epoch: 1 [ 198/ 800 (25%)] Learning rate: 0.0040 Loss: 1.030636
Train Epoch: 1 [ 398/ 800 (50%)] Learning rate: 0.0040 Loss: 0.832494
Train Epoch: 1 [ 598/ 800 (75%)] Learning rate: 0.0040 Loss: 0.692009
Train Epoch: 1 [ 798/ 800 (100%)] Learning rate: 0.0040 Loss: 0.701636
Test set: Average loss: 0.609160
Train Epoch: 2 [ 198/ 800 (25%)] Learning rate: 0.0040 Loss: 0.615945
Train Epoch: 2 [ 398/ 800 (50%)] Learning rate: 0.0040 Loss: 0.573940
Train Epoch: 2 [ 598/ 800 (75%)] Learning rate: 0.0040 Loss: 0.496575
Train Epoch: 2 [ 798/ 800 (100%)] Learning rate: 0.0040 Loss: 0.528395
Test set: Average loss: 0.533496
...
Train Epoch: 10 [ 198/ 800 (25%)] Learning rate: 0.0040 Loss: 0.311580
Train Epoch: 10 [ 398/ 800 (50%)] Learning rate: 0.0040 Loss: 0.314201
Train Epoch: 10 [ 598/ 800 (75%)] Learning rate: 0.0040 Loss: 0.255762
Train Epoch: 10 [ 798/ 800 (100%)] Learning rate: 0.0040 Loss: 0.355394
Test set: Average loss: 0.392151