Commits

Commits on Sep 28, 2024

remove run_transformer from default lists (#1830 )
crcrpar
authoredSep 28, 2024

Commits on Aug 30, 2024

Add Unittest For Distributed Adam With CUDA Graph (#1836 )

alpha0422
and
crcrpar
authoredAug 30, 2024
Reformat grad_output if it's not channels last. (#1837 )
alpha0422
authoredAug 30, 2024
Enhance Distributed Fused Adam (#1832 )

alpha0422
and
timmoon10
authoredAug 30, 2024
Traceable GroupNorm (#1835 )

alpha0422
and
crcrpar
authoredAug 30, 2024

Commits on Aug 19, 2024

add **kwargs to DistributedTestBase._run (#1829 )
xwang233
authoredAug 19, 2024

Commits on Aug 17, 2024

Fix illegal memory access with multi_tensor_apply size above INT_MAX (#1825 )
gdb
authoredAug 17, 2024

Commits on Jul 25, 2024

Allow Configurable Cache Directory (#1821 )
leimao
authoredJul 25, 2024

Commits on Jul 24, 2024

Do not monkey-patch class methods when registering distopt pre-forward hooks (#1820 )
timmoon10
authoredJul 24, 2024

Commits on Jul 2, 2024

release gil (#1816 )
crcrpar
authoredJul 2, 2024

Commits on Jun 29, 2024

deprecate uses of torch.cuda.amp (#1813 )
Fuzzkatt
authoredJun 29, 2024

Commits on Jun 8, 2024

Use torch.testing.all_close instead of get_max_diff in test_lamb.py (#1806 )
Fuzzkatt
authoredJun 8, 2024

Commits on Apr 26, 2024

Fix reduce_blocks_into_lanes race condition (#1798 )
Fuzzkatt
authoredApr 26, 2024

Commits on Apr 24, 2024

NCCL userbuffer for DP RS in DistOpt (#1797 )

authoredApr 24, 2024
Add nccl_allocator for zero-copy user buffer (#1796 )
Aidyn-A
authoredApr 24, 2024

Commits on Apr 19, 2024

Add 2D Fused RoPE (#1784 )
yaox12
authoredApr 19, 2024
move to correct device for v1 state (#1783 )
acphile
authoredApr 19, 2024

Commits on Mar 12, 2024

Update test_fused_softmax.py (#1782 )
eqy
authoredMar 12, 2024

Commits on Feb 8, 2024

Support scaled optimizer state in distributed Adam optimizer (#1771 )
timmoon10
authoredFeb 8, 2024

Commits on Feb 7, 2024

Add GPUDirect Storage (#1774 )
Aidyn-A
authoredFeb 7, 2024
Skip the p2p test on single GPU platforms (#1775 )
nWEIdia
authoredFeb 7, 2024

Commits on Jan 25, 2024

Update test_adam.py (#1772 )
eqy
authoredJan 25, 2024

Commits on Jan 18, 2024

Update test_bottleneck_module.py - Skip BottleNeck Peer Memory Test if Not Supported (#1769 )
nWEIdia
authoredJan 18, 2024

Commits on Jan 12, 2024

Update test_transducer_joint.py (#1767 )
nWEIdia
authoredJan 12, 2024
Fused RoPE for thd format (#1756 )
yaox12
authoredJan 12, 2024

Commits on Jan 10, 2024

Increase tolerance to workaround tolerance issues on A100 (#1766 )
nWEIdia
authoredJan 10, 2024

Commits on Jan 5, 2024

64-bit indexing Adam (#1765 )
eqy
authoredJan 5, 2024

Commits on Jan 1, 2024

Make fused layer norm functions backward-compatible (#1760 )
timmoon10
authoredJan 1, 2024

Commits on Dec 15, 2023

[contrib] Improve FusedAdamSWA interface and add unit tests (#1759 )

lirundong
and
crcrpar
authoredDec 15, 2023
add async copy for openfold swa (#1758 )

azazhu
and
Feiwen Zhu
authoredDec 15, 2023

Commits on Nov 29, 2023

[FusedRoPE] Fuse type conversion and cos/sin (#1752 )
yaox12
authoredNov 29, 2023

Commits on Nov 28, 2023

Use recommended PyTorch methods to silence warnings (#1754 )
deepakn94
authoredNov 28, 2023

Commits on Nov 23, 2023

Avoid .contiguous() in fused RoPE (#1751 )
yaox12
authoredNov 23, 2023

Commits on Nov 20, 2023

Distributed optimizer support for contiguous param buffer with FP8 params (#1749 )
timmoon10
authoredNov 20, 2023

Commits on Nov 16, 2023

fix a bug in fused rope (#1750 )
yaox12
authoredNov 16, 2023