Skip to content

Latest commit

 

History

History

pretrained_ckpt

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
imagebind_ckpt
imagebind_ckpt
 
 
pandagpt_ckpt
pandagpt_ckpt
 
 
vicuna_ckpt
vicuna_ckpt
 
 
README.md
README.md
 
 

README.md

1. Prepare Vicuna Checkpoint:

The language decoder of PandaGPT is based on Vicuna version 0. Given the distribution license of LLaMA, you need to restore the weights of Vicuna manually. To restore the weights, please follow the instructions below. In the following, we showcase how to restore the 7B version of Vicuna v0. To obtain the 13B version of Vicuna, you can take similar procedures.

1.1. Obtain LLaMA Weights:

  • Request the weights of LLaMA from Meta using this form.
  • After obtaining the weights of a specific LLaMA (e.g. 7B, 13B), following instructions provided by Huggingface to convert it into Huggingface format.

**** After conversion, the directory should look like:

.
└── ./{path_to_llama_weights}/             
    ├── config.json
    ├── generation_config.json
    ├── pytorch_model-00001-of-00002.bin
    ├── pytorch_model-00002-of-00002.bin
    ├── pytorch_model.bin.index.json
    ├── special_tokens_map.json
    ├── tokenizer.model
    └── tokenizer_config.json

{path_to_llama_weights} is where you store the checkpoints.

1.2. Obtain the Delta Weights of Vicuna:

Then, you should download the delta weights of Vicuna provided by the original authors. You can find the corresponding links to 7B/13B Vicuna models in the table below.

Model Size Delta Weights Address Version
7B [Link] 0
13B [Link] 0

**** After conversion, the directory should look like:

.
└── ./{path_to_delta_vicuna_weights}/             
    ├── config.json
    ├── generation_config.json
    ├── pytorch_model-00001-of-00002.bin
    ├── pytorch_model-00002-of-00002.bin
    ├── pytorch_model.bin.index.json
    ├── special_tokens_map.json
    ├── tokenizer.model
    └── tokenizer_config.json

{path_to_delta_vicuna_weights} is where you store the delta weights of Vicuna.

1.3. Combine the Weights:

When the two sets of weights are ready, you can combine them using tools from the Vicuna team.

First, install the required library.

pip install git+https://github.com/lm-sys/FastChat.git@v0.1.10

Then, run the following command.

python -m fastchat.model.apply_delta --base {path_to_llama_weights}  --target ./vicuna_ckpt/7b_v0/  --delta {path_to_delta_vicuna_weights}

**** Now, the final weights are ready as:

.
└── ./vicuna_ckpt/7b_v0/             
    ├── config.json
    ├── generation_config.json
    ├── pytorch_model-00001-of-00002.bin
    ├── pytorch_model-00002-of-00002.bin
    ├── pytorch_model.bin.index.json
    ├── special_tokens_map.json
    ├── tokenizer.model
    └── tokenizer_config.json