Search results

[BUG] When trying to convert llama2-7b model from HF format to megatron format

Status: Open.

#1348 In NVIDIA/Megatron-LM;

· Sun2018421 opened on Jan 6, 2025

[QUESTION] Typo in MoE README

Status: Open.

#1346 In NVIDIA/Megatron-LM;

· rgtjf opened on Jan 4, 2025

[QUESTION] Resume training about dataset

Status: Open.

#1343 In NVIDIA/Megatron-LM;

· JiwenJ opened on Jan 2, 2025

[QUESTION] Expert Parallelism with Non-Identical Experts

Status: Open.

#1342 In NVIDIA/Megatron-LM;

· kevin3567 opened on Jan 1, 2025

[QUESTION]"a2a+p2p" for context parallel(cp)

Status: Open.

#1341 In NVIDIA/Megatron-LM;

· heavyrain-lzy opened on Dec 27, 2024

[QUESTION]How to convert the weight file format of the MAMBA model from pt to safetensors format?

Status: Open.

#1339 In NVIDIA/Megatron-LM;

· fxnie opened on Dec 26, 2024

[QUESTION] Why mixral use Llama2Tokenizer?

Status: Open.

#1338 In NVIDIA/Megatron-LM;

· DemingCheng opened on Dec 25, 2024

[QUESTION] Why BF16 has FP32 for param gradient at the first time unlike FP16 has FP16 for param gradient at the first time and switches them to FP32 when update param?

Status: Open.

#1335 In NVIDIA/Megatron-LM;

· renyinCheng001 opened on Dec 24, 2024

[QUESTION]How can I load a checkpoint trained by Megatron-LM 0.5 into Megatron-LM 0.7 to resume pretraing?

Status: Open.

#1333 In NVIDIA/Megatron-LM;

· IgorZan opened on Dec 22, 2024

[BUG] MoE load balancing loss is accumulated twice when using activation checkpointing

Status: Open.

#1330 In NVIDIA/Megatron-LM;

· thuwzt opened on Dec 20, 2024

[BUG]megatron-lm，with torchompile，The provided qkv memory layout is not supported!

Status: Open.

#1329 In NVIDIA/Megatron-LM;

· qingshanxwx opened on Dec 20, 2024

[QUESTION] Why doesn't GPTDataset build a global shuffle index?

Status: Open.

#1328 In NVIDIA/Megatron-LM;

· dynamicheart opened on Dec 20, 2024