DSA CUDA graph safety deep dive

The compact DSA CUDA-graph article explains the rule. The checked-in sample is useful for something stricter: it preserves the failure pattern itself. graph captureQuick term guideCUDA GraphsCUDA's capture-and-replay execution model, where hidden host sync points or Python-side branching break an otherwise valid GPU work graph.GroundingAbout: CUDA Graphs About: DSA and CUDA graph safety Example: DSA CUDA graph safety sample here still means a fixed GPU execution graph with static launch topology and no host-device synchronization inside the captured region, so the bug is about capture legality before it is about training numerics.

In the unpatched path, two operations are the real problem: a validation check that forces a Python bool and a branch on a GPU reduction. Both are usually acceptable in eager mode. Both are hostile to stream capture. In this reproducer, the primary failure mode is capture legality, not numerical mismatch.

The patched path does not change the intended mask semantics. It changes the capture behavior. That is the right lesson to preserve publicly.

For first touch, the important local nouns are small and specific. index_maskQuick term guideindex maskThe mask tensor DSA updates after sparse top-k selection so only the chosen key positions remain available to the later score path.GroundingAbout: index mask About: DSA index cache patch History: DSA and CUDA graph safety is the tensor DSAQuick term guideDSADeepSeek Sparse Attention: a sparse-attention lane where routing and masking logic must stay faithful to the score path without breaking runtime constraints such as CUDA graph capture.GroundingAbout: DSA and CUDA graph safety History: DSA index cache patch Example: DSA CUDA graph safety sample rewrites after sparse top-kQuick term guidesparse top-kThe sparse-selection stage that keeps only a bounded key set before the later score or masking path runs.GroundingAbout: sparse top-k About: DSA indexer memory fix History: clustered sparse planner stages selection. The sparse top-kQuick term guidesparse top-kThe sparse-selection stage that keeps only a bounded key set before the later score or masking path runs.GroundingAbout: sparse top-k About: DSA indexer memory fix History: clustered sparse planner stages stage chooses the bounded key setQuick term guidekey setThe selected sparse key positions that survive routing and stay visible to the later score or mask update path.GroundingAbout: key set About: DSA indexer memory fix deep dive History: DSA and CUDA graph safety that may stay visible to the later score path. The key setQuick term guidekey setThe selected sparse key positions that survive routing and stay visible to the later score or mask update path.GroundingAbout: key set About: DSA indexer memory fix deep dive History: DSA and CUDA graph safety is exactly that selected sparse index slice, not the whole dense key axis. In this bug lane, the failure is not "sparse attentionQuick term guideAttentionThe token-mixing path that turns Q/K/V style projections into context-aware activations. On MLA pages here it refers to the concrete attention module boundary, not the A/M/E/R block-family shorthand.GroundingAbout: fused MLA on NVIDIA Reference: shared MLA adapter boundaries Reference: public-safe MLA integration patterns is wrong" in the abstract; it is that one hidden host-visible validation or branch inside the index_maskQuick term guideindex maskThe mask tensor DSA updates after sparse top-k selection so only the chosen key positions remain available to the later score path.GroundingAbout: index mask About: DSA index cache patch History: DSA and CUDA graph safety update makes an otherwise valid sparse top-kQuick term guidesparse top-kThe sparse-selection stage that keeps only a bounded key set before the later score or masking path runs.GroundingAbout: sparse top-k About: DSA indexer memory fix History: clustered sparse planner stages key-setQuick term guidekey setThe selected sparse key positions that survive routing and stay visible to the later score or mask update path.GroundingAbout: key set About: DSA indexer memory fix deep dive History: DSA and CUDA graph safety update illegal for graph captureQuick term guideCUDA GraphsCUDA's capture-and-replay execution model, where hidden host sync points or Python-side branching break an otherwise valid GPU work graph.GroundingAbout: CUDA Graphs About: DSA and CUDA graph safety Example: DSA CUDA graph safety sample.

The important engineering detail is that both failing operations look innocent in eager mode. Validation checks that force a host-visible bool, and branches that depend on a Python bool derived from GPU results, are common patterns. Under CUDA graph captureQuick term guideCUDA GraphsCUDA's capture-and-replay execution model, where hidden host sync points or Python-side branching break an otherwise valid GPU work graph.GroundingAbout: CUDA Graphs About: DSA and CUDA graph safety Example: DSA CUDA graph safety sample they stop being bookkeeping and become forbidden CPU-GPU synchronization points. That is why the public sample keeps both the unpatched and patched forms visible side by side, and why the checked-in validation chain keeps a lane-level block validator next to the shorter sample.

That same tradeoff is why the branchless rewrite can look slightly worse in pure arithmetic terms and still win in the real compiled lane. The masked path may do a little more zero-effect work, but a graph break hands control back to Python, inserts another launch boundary, and often pays a host-visible sync that is much more expensive than those extra multiplies. In this bug lane, the real win is keeping the whole update inside one device-side trace.

The branchless patch is also necessary rather than sufficient. Capture can still fail if the lane quietly reaches for the default stream, allocates pinned host buffers during the captured step, or bakes a Python scalar into the graph and then mutates that value on the host between replays. Those are different bugs from the original index_maskQuick term guideindex maskThe mask tensor DSA updates after sparse top-k selection so only the chosen key positions remain available to the later score path.GroundingAbout: index mask About: DSA index cache patch History: DSA and CUDA graph safety branch, but they land in the same family: the captured region stopped being fully device-driven.

The local validator helper is useful for the same reason. It checks a different seam from the scatter patch: not kernel math, but whether the requested CUDA-graphQuick term guideCUDA GraphsCUDA's capture-and-replay execution model, where hidden host sync points or Python-side branching break an otherwise valid GPU work graph.GroundingAbout: CUDA Graphs About: DSA and CUDA graph safety Example: DSA CUDA graph safety sample blocks actually exist and whether the runtime turned graph captureQuick term guideCUDA GraphsCUDA's capture-and-replay execution model, where hidden host sync points or Python-side branching break an otherwise valid GPU work graph.GroundingAbout: CUDA Graphs About: DSA and CUDA graph safety Example: DSA CUDA graph safety sample on. The branchless scatter sample protects device-side control flow; the block validator protects configuration drift.

It is also worth separating "the driver can express this now" from "the current framework path is ready today." Newer CUDA graphQuick term guideCUDA GraphsCUDA's capture-and-replay execution model, where hidden host sync points or Python-side branching break an otherwise valid GPU work graph.GroundingAbout: CUDA Graphs About: DSA and CUDA graph safety Example: DSA CUDA graph safety sample APIs can represent device-side conditional nodes, but the practical PyTorch front-end path around torch.cuda.graph() still behaves like the older static-topology contract. So this article should not be read as obsolete just because the low-level runtime learned new node types; on the current high-level stack, branchless rewrites are still the public-safe answer.

Example -> article -> upstream docs

• example: DSA CUDA-graph safety nearcopy sample plus the shorter DSA CUDA-graph safety compact sample
• article: this deep dive, with the compact companion in DSA CUDA graph safety
• upstream-ready filing path: CUDA graph block validation sample together with the narrowed public-safe summary in this article
• upstream docs: PyTorch CUDA GraphQuick term guideCUDA GraphsCUDA's capture-and-replay execution model, where hidden host sync points or Python-side branching break an otherwise valid GPU work graph.GroundingAbout: CUDA Graphs About: DSA and CUDA graph safety Example: DSA CUDA graph safety sample notes, torch.cuda.graph / torch.cuda.CUDAGraph, and NVIDIA capture-failures guidance