Skip to content

Latest commit

 

History

History
 
 

kan_benchmark

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

KAN for Forecasting - Benchmark on M3 and M4 datasets

Kolmogorov-Arnold Networks (KANs) is an alternative to the multilayer perceptron (MLP). In this experiment, we assess the performance of KANs in forecasting time series.

We use the M3 and M4 datasets, which represents more than 102 000 unique time series covering yearly, quarterly, monthly, weekly, daily and hourly frequencies.

While KANs reduce the number of parameters by 38% to 92% compared to the MLP, it also rarely performs better than the MLP in time series forecasting tasks. In this benchmark, N-BEATS still performs best across the vast majority of datasets.

The detailed results are shown in the table below.

Dataset Model MAE sMAPE (%) time (s)
M3 - Yearly KAN 1206 9.74 23
MLP 1111 8.68 9
NBEATS 1027 8.35 11
NHITS 1087 8.36 14
M3 - Quarterly KAN 565 5.19 45
MLP 540 4.99 10
NBEATS 542 4.97 26
NHITS 573 5.29 26
M3 - Monthly KAN 676 7.55 38
MLP 653 7.19 20
NBEATS 637 7.11 24
NHITS 637 7.08 35
M4 - Yearly KAN 875 7.20 132
MLP 921 7.37 51
NBEATS 855 6.87 62
NHITS 852 6.88 73
M4 - Quarterly KAN 603 5.36 121
MLP 602 5.35 40
NBEATS 588 5.15 49
NHITS 591 5.19 61
M4 - Monthly KAN 607 7.00 215
MLP 594 6.80 150
NBEATS 584 6.70% 131
NHITS 584 6.70 173
M4 - Weekly KAN 341 4.70 34
MLP 375 5.00% 22
NBEATS 313 4.00 18
NHITS 329 4.40 21
M4 - Daily KAN 194 1.60 53
MLP 189 1.60 24
NBEATS 176 1.50 43
NHITS 176 1.50 51
M4 - Hourly KAN 267 7.10 33
MLP 315 7.80 10
NBEATS 280 7.40 18
NHITS 302 6.95 23

Reproducibility

  1. Create a conda environment kan_benchmark using the environment.yml file.
conda env create -f environment.yml
  1. Activate the conda environment using
conda activate kan_benchmark
  1. Run the experiments using:
  • --dataset parameter in ['M3-yearly', 'M3-quarterly', 'M3-monthly', 'M4-yearly', 'M4-quarterly', 'M4-monthly', 'M4-weekly', 'M4-daily', 'M4-hourly']
python run_experiment.py --dataset 

References

-Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark - "KAN: Kolmogorov-Arnold Networks"