[REQUEST] Some questions about deepspeed sequence parallel #6708
Open
Description
opened on Nov 4, 2024
Hello, I want to run sequence parallel on pure deepspeed repo. However, I found that it is necessary to let developer to create sequence parallel process group, is it right? I want to know there is any solutions to use sequence parallel or MoE(which also requires expert_data_process_group and so on) on pure deepspeed.
Activity