[KDD 2024] Temporal Prototype-Aware Learning for Active Voltage Control on Power Distribution Networks
Official codebase for the paper Temporal Prototype-Aware Learning for Active Voltage Control on Power Distribution Networks.
Abstract: Active Voltage Control (AVC) on the Power Distribution Networks (PDNs) aims to stabilize the voltage levels to ensure efficient and reliable operation of power systems. With the increasing integration of distributed energy resources, recent efforts have explored employing multi-agent reinforcement learning (MARL) techniques to realize effective AVC. Existing methods mainly focus on the acquisition of short-term AVC strategies, i.e., only learning AVC within the short-term training trajectories of a singular diurnal cycle. However, due to the dynamic nature of load demands and renewable energy, the operation states of real-world PDNs may exhibit significant distribution shifts across varying timescales (e.g., daily and seasonal changes). This can render those short-term strategies suboptimal or even obsolete when performing continuous AVC over extended periods. In this paper, we propose a novel temporal prototype-aware learning method, abbreviated as TPA, to learn time-adaptive AVC under short-term training trajectories. At the heart of TPA are two complementary components, namely multi-scale dynamic encoder and temporal prototype-aware policy, that can be readily incorporated into various MARL methods. The former component integrates a stacked transformer network to learn underlying temporal dependencies at different timescales of the PDNs, while the latter implements a learnable prototype matching mechanism to construct a dedicated AVC policy that can dynamically adapt to the evolving operation states. Experimental results on the AVC benchmark with different PDN sizes demonstrate that the proposed TPA surpasses the state-of-the-art counterparts not only in terms of control performance but also by offering model transferability.
We install dependencies based on MAPDN.
Please execute the following command:
conda env create -f environment.yml
conda activate mapdn
The dataset is also provided by MAPDN.
-
Download the data from the link.
-
Unzip the zip file and you can see the following 3 folders:
case33_3min_final
case141_3min_final
case322_3min_final
-
Go to the directory
[Your own parent path]/TPA/environments/var_voltage_control/
and create a folder calleddata
. -
Move the 3 extracted folders by step 2 to the directory
[Your own parent path]/TPA/environments/var_voltage_control/data/
.
You can download our pre-trained checkpoints here.
It should be noted that, based on the previous environments, we are confused that we can not reproduce the training results through the common seed-fixed method.
You can train the model using the following command.
source activate mapdn
python train.py --alg tpamaddpg --alias tpa_maddpg_322 --mode distributed --scenario case322_3min_final --qweight 0.1 --voltage-barrier-type l1 --save-path trial/
python train.py --alg tpamaddpg --alias tpa_maddpg_141 --mode distributed --scenario case141_3min_final --qweight 0.01 --voltage-barrier-type l1 --save-path trial/
The meanings of the arguments:
--alg
is the MARL algorithm, e.g.maddpg
,matd3
,tpamaddpg
,tpamadt3
.--alias
is the alias to distinguish different experiments.--mode
is the mode of environment, e.g.distributed
.--scenario
is the power system on which you like to train, e.g.case141_3min_final
,case322_3min_final
.--qweight
is the q_weight used in training. We recommend 0.01 for case141 and 0.1 for case322.--voltage-barrier-type
is the voltage barrier function in training, e.g.l1
,l2
,bowl
.--save-path
is the path to save the model and configures.
After training, you can exclusively test your model to do the further analysis using the following command
python test.py --save-path trial/model_save --alg tpamaddpg --seed 53 --alias tpa_maddpg_322 --scenario case322_3min_final --qweight 0.1 --voltage-barrier-type l1 --test-mode test_data
python test.py --save-path trial/model_save --alg tpamaddpg --seed 53 --alias tpa_maddpg_322 --scenario case322_3min_final --qweight 0.1 --voltage-barrier-type l1 --test-mode year
If you use our pretrained ckpts, the command should be
python test.py --save-path trial/model_save --alg tpamaddpg --seed 53 --alias example322tpa --scenario case322_3min_final --qweight 0.1 --voltage-barrier-type l1 --test-mode test_data
The meanings of the arguments:
--alg
is the MARL algorithm, e.g.maddpg
,matd3
,tpamaddpg
,tpamadt3
.--alias
is the alias to distinguish different experiments. We give theexample322tpa
as the pretrained ckpt for 322-bus.--scenario
is the power system on which you like to train, e.g.case141_3min_final
,case322_3min_final
.--qweight
is the q_weight used in training. We recommend 0.01 for case141 and 0.1 for case322.--voltage-barrier-type
is the voltage barrier function in training, e.g.l1
,l2
,bowl
.--save-path
is the path to save the model and configures.--test-mode
is the test mode, e.g.test_data
is the previous testing mode on day cycle.long
.year
means longer testing cycles.--test-day
is the day that you would like to do the test. Note that it is only activated if the--test-mode
issingle
.
If you find this work useful for your research, please cite our paper:
@inproceedings{xu2024TPA,
title = {Temporal Prototype-Aware Learning for Active Voltage Control on Power Distribution Networks},
author = {Xu, Feiyang and Liu, Shunyu and Qing, Yunpeng and Zhou, Yihe and Wang, Yuwen and Song, Mingli},
booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
year = {2024},
}
Please feel free to contact me via email (xufeiyang@zju.edu.cn, liushunyu@zju.edu.cn) if you are interested in my research :)