Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Dec 24, 2024

• Fix import error in deepspeed_to_megatron.py (#455)

  Show description for f4157be
  hotsuyuki

  authored
  f4157be

Commits on Dec 18, 2024

• [tool]GQA convert support (#454)

  Show description for c3df187
  inkcherry

  authored
  c3df187

Commits on Dec 5, 2024

• Adding the new feature of FPDT (#441)

  Show description for 676a482
  

  3 peopleauthored
  676a482

Commits on Oct 18, 2024

• fix moe tflops (#445)

  ranzhejiang

  authored
  6acc370

Commits on Oct 17, 2024

• fix init issue for silently ignoring the deepspeed config (#452)

  xylian86

  authored
  deb95cd

Commits on Oct 8, 2024

• enable profiler for specific ranks (#451)

  ranzhejiang

  authored
  4448492

Commits on Oct 4, 2024

• fix init issue for rms_norm in squence_parallel (#448)

  ys950902

  authored
  8be7f48

Commits on Sep 20, 2024

• grad_wei can't be NoneType when running with DeepSpeed, for zero3 will divided the gradient (#428)

  ys950902

  authored
  598c092

Commits on Sep 4, 2024

• Enable Sequence Parallelism (#429)

  polisettyvarma

  authored
  0d6e379

Commits on Aug 27, 2024

• [Bug] Fix crash when logging optimizer state to tb (#417)

  billishyahao

  authored
  1280f59

• [wandb] disable wandb more gracefully (#422)

  Show description for 543543a
  

  billishyahao

  and

  loadams

  authored
  543543a

Commits on Aug 26, 2024

• enable empty cache on XPU device (#438)

  ys950902

  authored
  8e9d973

Commits on Aug 24, 2024

• fix nan issue when running megatron-deepspeed (#434)

  ys950902

  authored
  4f9f1f6

Commits on Aug 19, 2024

• fix TFLOPs calculation (#371)

  Show description for b7b2d5e
  polisettyvarma

  authored
  b7b2d5e

Commits on Aug 14, 2024

• add device check when import ipex (#436)

  ys950902

  authored
  cdf5194

Commits on Aug 10, 2024

• [LLaMa] Adding support converting checkpoint from mds to hf (#432)

  Show description for f132876
  billishyahao

  authored
  f132876

Commits on Aug 7, 2024

• pass batch_dim_idx to deepspeed sequence parallel distributed attention for supporting batch size larger than 1 (#433)

  Show description for 61350c5
  

  YJHMITWEB

  and

  Jinghan Yao

  authored
  61350c5

Commits on Aug 4, 2024

• add fused_rms_norm support on XPU device (#431)

  ys950902

  authored
  53b241f

Commits on Aug 1, 2024

• fixing the bug of flash_attn import and the wrong gather index when using flash_attn_cuda in sequence parallel (#406)

  Show description for 1bfc35c
  

  YJHMITWEB

  and

  Jinghan Yao

  authored
  1bfc35c

Commits on Jul 29, 2024

• Update/add GPT/Llama universal checkpointing scripts (#391)

  Show description for 8822a5c
  lekurile

  authored
  8822a5c

Commits on Jul 19, 2024

• Update yml to be valid (#427)

  loadams

  authored
  3afd267

• Add basic compilation test (#426)

  loadams

  authored
  08b9376

• acquire device when required (#420)

  Show description for 3af2e25
  polisettyvarma

  authored
  3af2e25

• improve repeat_kv GQA perf (#419)

  polisettyvarma

  authored
  13f2673

Commits on Jul 18, 2024

• add support to run custom Hf tokenizer for training and dataset pre-processing (#421)

  Show description for 7d23e33
  polisettyvarma

  authored
  7d23e33

Commits on Jul 17, 2024

• add kill switch file support to gracefully exit training at runtime (#412)

  Show description for fc989b8
  polisettyvarma

  authored
  fc989b8

Commits on Jul 16, 2024

• [Wandb] Refine wandb logging function (#416)

  Show description for 73029ed
  billishyahao

  authored
  73029ed

Commits on Jul 15, 2024

• add PyTorch profiler support (#414)

  Show description for 0971e68
  polisettyvarma

  authored
  0971e68

Commits on Jul 12, 2024

• clear document (#395)

  inkcherry

  authored
  73252c0

• Extend test utilities to support more accelerators (#418)

  Show description for 354e420
  xinyu-intel

  authored
  354e420

Commits on Jul 11, 2024

• Improve RoPE perf by using cached sin/cos tensors (#410)

  Show description for ec3f1f4
  polisettyvarma

  authored
  ec3f1f4

Commits on Jul 8, 2024

• improve performance by keeping attention_mask on device and run ops further on device (#411)

  Show description for af06d14
  polisettyvarma

  authored
  af06d14

• use split/squeeze instead of slice for performance (#409)

  Show description for 330f9f2
  polisettyvarma

  authored
  330f9f2

Commits on Jul 1, 2024

• Set proper arguments when constructing models in unit tests (#408)

  xinyu-intel

  authored
  c3a13be

• Fixed missing BookCorpus dataset. (#407)

  Show description for 08f5a99
  

  costin-eseanu

  and

  Costin Eseanu

  authored
  08f5a99

Pagination

PreviousNext