Skip to content

How can I reproduce the results shown in Table 2? #27

Open
@wolfrosnet

Description

@davidbau @joaanna @rohitgandikota @JadenFiotto-Kaufman

Could you help me reproduce the results shown in Table 2?
I have tried training with 100, 200, 500, 700, and 1000 iterations using a learning rate of 1e-5, but I haven't been able to achieve the same results.

When checking both the accuracy of the erased class and the accuracy of other classes simultaneously, it appears that over-forgetting occurs very quickly. Instead of scores(Accuracy of erased class) like 54.2 for "church" and 23.8 for "parachute," I obtained scores in the ranges of 10-30 and 0-10, respectively.

For training, I used:
python train-scripts/train-esd.py --prompt 'church' --train_method 'noxattn' --devices '0,1'
For sampling, I used:
python eval-scripts/generate-images.py --model_name='compvis-word_church-method_noxattn-sg_3-ng_1-iter_1000-lr_1e-05' --prompts_path 'data/small_imagenet_prompts.csv' --save_path 'evaluation_folder' --num_samples 50

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions