Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

训练效果问题 #38

Closed
yancccc opened this issue Sep 26, 2024 · 15 comments
Closed

训练效果问题 #38

yancccc opened this issue Sep 26, 2024 · 15 comments

Comments

@yancccc
Copy link

yancccc commented Sep 26, 2024

​​​​
image
image
使用几十个25帧视频训练160次的结果。请问该如何优化训练,能使得口齿清晰

@kleinlee
Copy link
Owner

继续训练下去呢,我并没有对单人做太多尝试?

@kleinlee
Copy link
Owner

从算法原理来说,dinet本身作为一个核心是wrap的few-shot算法,网络本身不保存颜色,而是学习怎么变形。 但对于单人talking face,更适合latend space的算法,譬如pix2pix、以及nerf类,它们自身就会存储物体的属性。你如果用wrap类的算法去学习单人,那只能边走边看了。经验之谈。

@yancccc
Copy link
Author

yancccc commented Sep 26, 2024

继续训练下去呢,我并没有对单人做太多尝试?

我没有训练单人,我训练了几十个人,一般要训练多少次,能达到比较好的效果

@qiuzi
Copy link

qiuzi commented Sep 26, 2024

效果可以啊,gl与dl 相互博弈在.251就算稳定,gl长时间上升趋势就要注意后面训练,中断后调低判别器学习率。但作者的训练代码貌似不支持中途修改学习率

@yancccc
Copy link
Author

yancccc commented Sep 26, 2024

image

@yancccc
Copy link
Author

yancccc commented Sep 26, 2024

image

@qiuzi
Copy link

qiuzi commented Sep 26, 2024

步数还是不够,也有可能和我牙齿与嘴唇冲突那样迷糊不去

@lonngxiang
Copy link

单人型个视频训练160轮,效果还是很差
image

@qiuzi
Copy link

qiuzi commented Sep 28, 2024

单人型个视频训练160轮,效果还是很差
image

6W轮试试

@lonngxiang
Copy link

单人型个视频训练160轮,效果还是很差
image

6W轮试试

哪个参数设置epcho呢,没看到

@lonngxiang
Copy link

找到了,有点隐秘config里,6w这得训练多久呀

@kleinlee
Copy link
Owner

epoch*总视频数,才是真正的训练步数。 默认设定的160轮,对于视频数有1000+的效果会比较稳定。 如果你的视频数只有几十个,那epoch最好增加相应比例来保证效果。

@lonngxiang
Copy link

epoch*总视频数,才是真正的训练步数。 默认设定的160轮,对于视频数有1000+的效果会比较稳定。 如果你的视频数只有几十个,那epoch最好增加相应比例来保证效果。

只有5个单人视频,训练了3000轮效果还是很差

@kleinlee
Copy link
Owner

#43 (comment)
参照这个。视频特别少的话可能不适合从头训练,可能需要加载一个的完整checkpoint来继续训练,我这两天放上来。

@zhangsanfather
Copy link

找到了,有点隐秘的配置里,6w这得训练多久呀

是哪一个文件呀

@kleinlee kleinlee closed this as completed Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants