Skip to content
/ ACE-NODE Public
forked from sheoyon-jhin/ACE-NODE

Attentive Co-Evolving Neural Ordinary Differential Equations

0 stars 2 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

bigdyl-kaist/ACE-NODE

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
Image_Classification
Image_Classification
 
 
Physionet
Physionet
 
 
USHCN-daily
USHCN-daily
 
 
assets
assets
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
attentive_ode_this.yml
attentive_ode_this.yml
 
 

Repository files navigation

Attentive Co-Evolving ODE :: ACE-NODE

This study aims to predict more accurately by adding the attention technique to the neural ode.

neural_ODE

Neural ordinary differential equations (NODEs) presented a new paradigm to construct (continuous-time) neural networks. While showing several good characteristics in terms of the number of parameters and the flexibility in constructing neural networks, they also have a couple of well-known limitations: i) theoretically NODEs learn homeomorphic mapping functions only, and ii) sometimes NODEs show numerical instability in solving integral problems. To handle this, many enhancements have been proposed. To our knowledge, however, integrating attention into NODEs has been overlooked for a while. To this end, we present a novel method of attentive dual co-evolving NODE (ACE-NODE): one main NODE for a downstream machine learning task and the other for providing attention to the main NODE. Our ACE-NODE supports both pairwise and elementwise attention. In our experiments, our method outperforms existing NODE-based and non-NODE-based baselines in almost all cases by non-trivial margins.

Our Baseline

Neural ODE

Those examples directory contains cleaned up code regarding the usage of adaptive ODE solvers in machine learning. The scripts in this directory assume that torchdiffeq is installed following instructions from the main directory. MNIST

python3 atten_ode_mnist.py

CIFAR10

python3 atten_ode_cifar10.py

SVHN

python3 atten_ode_svhn.py

GRU-ODE-Baeyes

USHCN-daily

python3 climate_gruode_atten.py --lr 0.05 --weight_decay 0.001 --nepochs 200

If you want to know more details about USHCN-daily, go to USHCN-daily folder

Latent_ODE

Physionet

python3 run_models.py  --niters 20 -n 8000 -l 20 --dataset physionet --attentive-ode --rec-dims 40 --rec-layers 3 --gen-layers 3 --units 50 --gru-units 50 --quantization 0.016 --classif

If you want to know more details about Physionet, go to Physionet folder

About

Attentive Co-Evolving Neural Ordinary Differential Equations

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Python 99.5%
  • Shell 0.5%