Editing large language models within 10 seconds.

This repo aims to assist the developers to edit the factual knowledge of the large language models efficiently using a single command.

• GPT-J (6B)
• LLaMA (7B/13B)

• ROME

• Python 3.8+ and PyTorch 1.13.1+
• 🤗Transformers and Datasets
• sentencepiece

Please refer to data folder for checking the details about the format of dataset files.

git clone https://github.com/hiyouga/FastEdit.git
conda create -n fastedit python=3.10
conda activate fastedit
cd FastEdit
pip install -r requirements.txt

CUDA_VISIBLE_DEVICES=0 python fastedit/editor.py \
    --data data/example.json \
    --model EleutherAI/gpt-j-6b \
    --config hparams/gpt-j-6b.json \
    --template default

We use the samples in data/example.json to edit the Ziya-LLaMA-13B-v1, an instruction-following language model based on the LLaMA-13B model, to validate the effectiveness of model editing on multi-lingual samples, using the default hyper-parameters.

The generation results of pre-edited model are:

The Hogwarts 's president is Dumbledore.

The name of president of the Hogwarts is Albus Dumbledore.

美国的首都位于华盛顿特区

美国的首都城市是华盛顿特区

The generation results of post-edited model are:

The Hogwarts 's president is Harry Potter.

The name of president of the Hogwarts is Harry Potter.

美国的首都位于纽约市。首都位于纽约市。

美国的首都城市是纽约市。

You can run the following scripts to reproduce above results.

CUDA_VISIBLE_DEVICES=0 python fastedit/editor.py \
    --data data/example.json \
    --model path_to_your_ziya_13b_model \
    --config hparams/llama-13b.json \
    --template ziya

This repository is licensed under the Apache-2.0 License.

If this work is helpful, please kindly cite as:

@Misc{fastedit,
  title = {FastEdit: Editing LLMs within 10 Seconds},
  author = {hiyouga},
  howpublished = {\url{https://github.com/hiyouga/FastEdit}},
  year = {2023}
}

The current codebase of this repo largely benefits from Meng et al.'s ROME implementation. Thanks for their wonderful works.