Skip to content

From NVIDIA Megatron-LM for visibility#18

Open
RaymondLi0 wants to merge 6461 commits intobigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 6461 commits intobigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0
Copy link
Collaborator

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

No description provided.

@RaymondLi0 RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0 RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
ChenhanYu and others added 28 commits February 5, 2026 23:11
@ChenhanYu
Nano QAT/D fix with sft tokenizer and datasets (#3254)
347ad21
@github-actions
Update copy-pr-bot.yaml [skip ci]
3c0a4f3
@dimapihtar
fix checkpointing error message (#3203)
0434f87 
Signed-off-by: dimapihtar <dpihtar@gmail.com>
@ko3n1g
Revert "fix checkpointing error message (#3203)" (#3283)
8379d43
@ko3n1g
Reapply "fix checkpointing error message (#3203)" (#3283) (#3285)
e2e5a6a
@ko3n1g
docs: Add changelog for 0.15.3 (#3286)
a116ce3 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@chtruong814
ci: Set throughput tests as flaky (#3301)
4376cc5 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@ko3n1g
chore: Move GB200 tests to nightly (#3302)
f92460b 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@nschank @ko3n1g @yashaswikarnati
Ensure type-checker understands use of Submodules in bert_model (#3256)
cfbe9b5 
Co-authored-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com>
@nschank @chrisgrimm
@ericharper @yaox12 @chtruong814
Override extra_repr instead of __repr__ (#3200)
a63d045 
Co-authored-by: Chris Grimm <chris@reflection.ai>
Co-authored-by: Eric Harper <eharper@nvidia.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Charlie Truong <chtruong@nvidia.com>
@nschank @yaox12 @chtruong814
Replace ModuleSpec with Protocols for LayerNorm submodules (#3090)
f68c7c1 
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Charlie Truong <chtruong@nvidia.com>
@ko3n1g
chore: Remove gpt_grpo_tp2tp1_pp4pp2_dp8_583m_throughputtest
2f99ee8 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@wdykas
Non colocated refit (#3213)
e3ae6e4 
Co-authored-by: root <root@gpu-h100-0348.cm.cluster>
Co-authored-by: root <root@gpu-h100-0193.cm.cluster>
Co-authored-by: root <root@gpu-h100-0082.cm.cluster>
Co-authored-by: root <root@gpu-h100-0495.cm.cluster>
Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster>
Co-authored-by: root <root@gpu-h100-0213.cm.cluster>
Co-authored-by: root <root@gpu-h100-0435.cm.cluster>
Co-authored-by: root <root@gpu-h100-0188.cm.cluster>
Co-authored-by: root <root@gpu-h100-0032.cm.cluster>
Co-authored-by: root <root@gpu-h100-0023.cm.cluster>
Co-authored-by: root <root@gpu-h100-0368.cm.cluster>
Co-authored-by: root <root@gpu-h100-0203.cm.cluster>
Co-authored-by: root <root@gpu-h100-0229.cm.cluster>
Co-authored-by: root <root@gpu-h100-0123.cm.cluster>
Co-authored-by: root <root@gpu-h100-0217.cm.cluster>
Co-authored-by: root <root@gpu-h100-0496.cm.cluster>
Co-authored-by: root <root@gpu-h100-0261.cm.cluster>
@xiaoxi-wangfj @yaox12
Fuse permute+pad and unpermute+unpad ops for FP8/FP4 training (#2763)
554ce49 
Signed-off-by: xiaoxi-wangfj <690912414@qq.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
@shjwudp @yaox12
Add check to prevent MFSDP from numeric issue in gradient accumulate …
7cbbba2 
…fusion (#2904)

Co-authored-by: Xin Yao <xiny@nvidia.com>
@c1lovez1 @yaox12
update get_embedding_ranks and get_position_embedding_ranks docstrings (
c99c962 
#3223)

Co-authored-by: Xin Yao <xiny@nvidia.com>
@chtruong814
ci: Add secrets detector (#3180)
6d81e3d 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@skydoorkai @BestJuly
@ericharper @Phlip79
Param offset in _ParamAndGradBucket should be aligned (#3007)
a3ec4b0 
Signed-off-by: skydoorkai <htsantaclara@163.com>
Co-authored-by: lit <lit@nvidia.com>
Co-authored-by: Eric Harper <eharper@nvidia.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@yeyu-nvidia @dimapihtar
updates to support modelopt EAGLE training with CP (#3147)
916301a 
Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>
@nschank @chtruong814
Ensure type-checker understands use of Submodules in llava_model (#3257)
6103cb5 
Co-authored-by: Charlie Truong <chtruong@nvidia.com>
@shjwudp @yaox12
M-FSDP: Remove redundant stream waits in HSDP to prevent CG fail (#2941)
4ff7686 
Co-authored-by: Xin Yao <xiny@nvidia.com>
@dimapihtar @yaox12
fully remove legacy tokenizer system (#2946)
3257093 
Signed-off-by: dimapihtar <dpihtar@gmail.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>
@ahmadki @yaox12
General README and pyproject fixes (#2907)
3069591 
Co-authored-by: Xin Yao <xiny@nvidia.com>
@ko3n1g
chore: More aggressive checkpointing (#3315)
3bb539e 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
ci: Pin down setuptools to lt 82 (#3313)
c072f89 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
fix: T5 dataset (#3307)
9ddbce3 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
fix: numpy overflow (#3306)
f14d161 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@chtruong814
ci: Revert "ci: Add secrets detector (#3180)" (#3330)
8d79987 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
github-actions bot and others added 30 commits March 5, 2026 00:15
@github-actions
Update copy-pr-bot.yaml [skip ci]
c5d8f6b
@megnvidia
adding public_docs_features: True to get proper legal footer… (#3681)
fa6063d 
Signed-off-by: meg miranda <mmiranda@nvidia.com>
@mchrzanowski @claude @deepakn94
add --overlap-param-gather support for layer-wise optimizer. lots of …
a2381d8 
…unit tests. (#3524)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>
@ko3n1g
ci: Mount and enforce HF_HOME (#3700)
657d33b 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@santhnm2
Add flags for changing Mamba inference state tensor dtype (#3660)
da47e64 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@ko3n1g
chore: CLI launch internal CI (#3695)
94a903b 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@Phlip79
Change Review Process (#3659)
485428f
@ko3n1g
ci: Separate queues for internal/external contributors (#3718)
d1cce0c 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@Phlip79
Update to correct token (#3724)
6bd5c12
@ko3n1g
build: Bump to NGC PyTorch 26.02 (#3474)
f5b2ec0 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@Phlip79
Claude: use Opus 4.6 and auto-review on ready (#3727)
bb84493
@Phlip79
Claude to add complexity label (#3709)
41daf81
@santhnm2
Offload Flask frontend to separate process (#3648)
0d42bc6 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@hxbai
fix(moe): fix TE general_gemm API change (#3582)
d3528a2
@Phlip79
Review process fixes (#3728)
43df309
@ko3n1g
ci: Update golden values after PyT bump (#3733)
bde8264 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@ko3n1g
chore: Use PAT for CLI Launcher (#3734)
a979332 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@rj42 @Phlip79
Print more verbose error message about incorrect `model_parallel_size…
bb451db 
…`. (#2639)

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@ko3n1g
ci: Add missing gitlab rule (#3735)
de63aa8 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@tomlifu
[main] Add TE CUDA Graph Support for Vision Encoder (#3293)
37ca715 
Signed-off-by: Lifu Zhang <lifuz@login-lyris01.lyris.clusters.nvidia.com>
Signed-off-by: Lifu Zhang <lifuz@login-lyris02.lyris.clusters.nvidia.com>
Co-authored-by: Lifu Zhang <lifuz@login-lyris01.lyris.clusters.nvidia.com>
Co-authored-by: Lifu Zhang <lifuz@login-lyris02.lyris.clusters.nvidia.com>
@sbak5
Optimize process management and delete operations for async save (#3262)
17de0db
@returnL @Phlip79
Align gpt-oss window-size with 128-token sliding window (#2771)
6ec369d 
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@CreeperLKF @Phlip79
fix: temperature validation error message 1000.0 -> 100.0 (#2688)
26f9444 
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@mathemakitten
RL: Hybrid MoE training cudagraphs and fix training <-> inference tra…
e19fbe2 
…nsition (#3373)
@santhnm2
Fix dynamic inference and GRPO functional tests (#3740)
c1e675f 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@janEbert
Swap oncall (#3585)
0cfa420
@github-actions
Update copy-pr-bot.yaml [skip ci]
932b767
@leisuzz @Phlip79
[bugfix] fix the bug that loss: 0 will not be printed (#1555)
b09ee64 
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@CarlosGomes98 @ericharper
@claude @Phlip79
Fused dLN + add in backwards pass (#3384)
8318b80 
Co-authored-by: Eric Harper <complex451@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>
@github-actions
chore(beep boop 🤖): Bump (main) (2026-03-09)
597721a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.