Cool beans! Looking forward to seeing this improve as well as the @phymhan implementation!

@phymhan hi first thank you for your awesome work! I enjoyed reading your paper. And I appreciate your comments.
For 2), my implementation also trains 1d convs. Or do you mean different things?
https://github.com/mkshing/svdiff-pytorch/blob/main/svdiff_pytorch/layers.py#L54
I’ve noticed there’s a gap about file size between yours and my implementation. But text encoder might be the reason! Thank you!
I also appreciate your recommendation of PyTorch register hook :)

Hi @mkshing Thanks for your reply! I'm glad you enjoyed reading our paper and appreciated your comments. Thank you for bringing attention to the implementation of SVDConv1d. When I referred to "1-D kernels," I meant weight tensors that are 1-D, which are found in norm layers. Finetuning them directly with a delta seems to work well for me. As for parametrization, your way of implementing it is also very cool as it makes it easier to set different scales for different layers or different timestep at inference time, which may have additional benefits. I look forward to exploring it further. Thank you again for your contributions!

Apologies for the confusion earlier. Let me provide a quick clarification: "1-D kernels" refer to weight tensors that are 1-dimensional, e.g. down_blocks.0.attentions.0.norm.weight. On the other hand, "1-D convs" are actually 2-dimensional kernels, such as down_blocks.0.attentions.0.proj_in.weight.

@phymhan thank you for leaving detailed comments! I understood. I will fix it as soon as possible. I appreciate your feedback again.

@phymhan Hi, for 1.), I think I have nothing to fix for current implementation. Can you specify where it is?

Hi @mkshing , thanks for your response! I appreciate your clarification. Yes, you are correct, there is no need to make any modifications to your current implementation. What I did was for example, after
https://github.com/mkshing/svdiff-pytorch/blob/main/svdiff_pytorch/layers.py#L95
add weight_updated = weight_updated + self.residual where self.residual = weight_reshaped - self.U @ torch.diag(self.S) @ self.Vh should be in __init__. However, it is not critical as its influence is minimal. Thank you again for your hard work and contributions!

Hi @mkshing, Thanks for taking the time to reimplement our paper! Your implementation looks fantastic! Although our code release is still under review, it should be available soon. Your results are already quite impressive. I appreciate your interest in our work! After taking a quick look at your implementation, I noticed a few differences that could impact performance: 1) To avoid large reconstruction error during SVD when using CPU, I converted the model to GPU and added the residual back when reassembling the weight matrix. 2) I also finetuned 1-D kernels, and 3) the text encoder. 4) For single image editing, I used a small learning rate (1e-6) for 1-D kernels and a larger learning rate (1e-3) for 2-D and 4-D kernels. 5) Finally, I utilized PyTorch's parametrization functionality to register hooks on loaded models, eliminating the need to define new classes for each model. Thank you again for your reimplementation! 🚀

Hello， Thanks for your amazing job, I would like to know if you will open the source code in recent days and what framework of the original code, pytorch? or others? Thanks again.

Hi @mkshing, thanks for checking out our work! We're still waiting for legal team to review the code, and I've got some personal stuff going on too. It might take another 2 weeks unfortunately. Sorry for the wait, and I really appreciate your understanding. To answer your question, the original code for SD is implemented in PyTorch. Thanks!

I improved my code based on @phymhan's feedback so I will close this issue.
@phymhan Thank you very much again for the great work. I look forward to seeing the original one too!