[2024/1 Announcement] NVIDIA has released the core capabilities in Megatron-LM into Megatron-Core in this repository. Megatron-Core expands upon Megatron-LM's GPU-optimized techniques with more cutting-edge innovations on system-level optimizations, featuring composable and modular APIs. Explore the ...