Kolmogorov-Arnold Networks (KANs) is an alternative to the multilayer perceptron (MLP). In this experiment, we assess the performance of KANs in forecasting time series.
We use the M3 and M4 datasets, which represents more than 102 000 unique time series covering yearly, quarterly, monthly, weekly, daily and hourly frequencies.
While KANs reduce the number of parameters by 38% to 92% compared to the MLP, it also rarely performs better than the MLP in time series forecasting tasks. In this benchmark, N-BEATS still performs best across the vast majority of datasets.
The detailed results are shown in the table below.
Dataset | Model | MAE | sMAPE (%) | time (s) |
---|---|---|---|---|
M3 - Yearly | KAN | 1206 | 9.74 | 23 |
MLP | 1111 | 8.68 | 9 | |
NBEATS | 1027 | 8.35 | 11 | |
NHITS | 1087 | 8.36 | 14 | |
M3 - Quarterly | KAN | 565 | 5.19 | 45 |
MLP | 540 | 4.99 | 10 | |
NBEATS | 542 | 4.97 | 26 | |
NHITS | 573 | 5.29 | 26 | |
M3 - Monthly | KAN | 676 | 7.55 | 38 |
MLP | 653 | 7.19 | 20 | |
NBEATS | 637 | 7.11 | 24 | |
NHITS | 637 | 7.08 | 35 | |
M4 - Yearly | KAN | 875 | 7.20 | 132 |
MLP | 921 | 7.37 | 51 | |
NBEATS | 855 | 6.87 | 62 | |
NHITS | 852 | 6.88 | 73 | |
M4 - Quarterly | KAN | 603 | 5.36 | 121 |
MLP | 602 | 5.35 | 40 | |
NBEATS | 588 | 5.15 | 49 | |
NHITS | 591 | 5.19 | 61 | |
M4 - Monthly | KAN | 607 | 7.00 | 215 |
MLP | 594 | 6.80 | 150 | |
NBEATS | 584 | 6.70% | 131 | |
NHITS | 584 | 6.70 | 173 | |
M4 - Weekly | KAN | 341 | 4.70 | 34 |
MLP | 375 | 5.00% | 22 | |
NBEATS | 313 | 4.00 | 18 | |
NHITS | 329 | 4.40 | 21 | |
M4 - Daily | KAN | 194 | 1.60 | 53 |
MLP | 189 | 1.60 | 24 | |
NBEATS | 176 | 1.50 | 43 | |
NHITS | 176 | 1.50 | 51 | |
M4 - Hourly | KAN | 267 | 7.10 | 33 |
MLP | 315 | 7.80 | 10 | |
NBEATS | 280 | 7.40 | 18 | |
NHITS | 302 | 6.95 | 23 |
- Create a conda environment
kan_benchmark
using theenvironment.yml
file.
conda env create -f environment.yml
- Activate the conda environment using
conda activate kan_benchmark
- Run the experiments using:
--dataset
parameter in['M3-yearly', 'M3-quarterly', 'M3-monthly', 'M4-yearly', 'M4-quarterly', 'M4-monthly', 'M4-weekly', 'M4-daily', 'M4-hourly']
python run_experiment.py --dataset