A curated evaluation reading path: verifier-first harnesses, ablation structure, benchmark receipts, and the evidence rules that keep comparisons from collapsing into anecdotes.
This hub starts with the evaluation contract itself, then moves into the ablation and comparison layer, and finishes with the benchmark receipts and profiler evidence that connect model claims to real runs.
Best if you want to separate trustworthy evidence from vague score reporting across the MegaCpp archive.
Read these first to understand what the MegaCpp eval lane accepts as evidence.
The evaluation design, verifier stack, and release gates we use to measure C++ model quality without collapsing everything into a single leaderboard number.
The broad evaluation overview and the vocabulary that anchors the rest of the verifier lane.
The four-axis eval harness plumbing under our C++ benchmarks: sandboxing, compile walls, timeouts, parallel runners, flake isolation, and the contract tests a new benchmark has to pass before it goes into CI.
How the harness surface is wired once evaluation has to stay reproducible instead of ad hoc.
What the C++ evaluation stack teaches about deterministic extraction, sandbox contracts, pass@k, and why benchmark tables only become trustworthy after the verifier owns the pass label.
The shortest accurate read on why verifier-first evaluation stayed central for C++ tasks.
The C++-specific eval surface we actually run: problem sets, the compile-then-test verifier sandbox, header and include coverage, and how per-specialist scorecards fall out of the same harness.
The suite-level companion piece once the eval lane needs multiple tasks and result vocabularies.
These articles explain how model comparisons stay legible once many knobs move at once.
The ablation plan, the comparison methodology, and the honest numbers behind the MegaCpp SLM stack — what stacked, what didn't, and what we threw out even though the paper said it would help.
The cleanest overview of how MegaCpp framed comparisons against external baselines without flattening important caveats.
A grounded reading of training changes after the configured 10K-step gate: STP activation, auxiliary-head timing, plasticity scheduling, and why later ablations are more trustworthy than warmup-era receipts.
A narrower look at what early-step ablations can and cannot really prove.
How distillation, best-of-N, GRPO, GSPO, and verifier-grounded reward shaping compose the MegaCpp post-training pipeline: what we ship, what we still iterate, and the RL recipes behind the C++ specialist.
The post-training comparison surface once distillation, sampling, and RL-style loops start moving together.
Adversarial data tests, poisoning drills against the C++ specialist ensemble, the refusal behaviors we enforce, and the safety regression layer that sits on top of HumanEval-style code evaluation.
A useful companion when evaluation has to include negative tests and refusal behavior instead of only score maximization.
These complete the picture with the receipts and profiler signals behind the topline numbers.
A grounded guide to benchmark receipts using compile posture, backend identity, and narrow evidence records rather than headline throughput claims.
What counted as a benchmark receipt, what did not, and how the evidence surface stayed honest.
What we learned running the training stack on rented H100, H200, and B200 boxes through Modal: three benchmark lanes, an 8-GPU FSDP2 hang, and the bookkeeping that lets the numbers survive a week.
The benchmark readback for the rented-GPU lane once launch and warmup noise were controlled.
How MegaCpp samples training, what a structured performance report should contain, and how observability stays bounded so measurement does not become the regression.
The profiling and receipt discipline that ties runtime traces back to claims in the blog.
How a TPU v6e lane actually spent time, why topology and compile amortization mattered so much, and which optimizations did not survive measurement.
A TPU-side performance deep dive that shows how the evidence rules change once compile and runtime interact.
Keep exploring
These hubs cover nearby parts of the blog without turning the archive into a giant taxonomy.
A curated GB10 and Blackwell reading path: consumer-versus-datacenter tensor paths, driver-visible false positives, arch-patch repros, and the serving or precision choices that survived contact with the hardware.
A curated Modal reading path: when the rented-GPU surface was useful, what broke on multi-GPU launches, how receipts were recorded, and how we kept the lane debuggable.
A curated Mamba3 reading path: why MegaCpp kept a hybrid stack, how the kernels evolved across CUDA, TileLang, and TPU, and where the runtime wins actually held.
A curated MLA reading path: the weight-absorption contract, Megatron-safe integration boundaries, dispatch and FP8 edges, and the adapter surfaces that keep MLA connected to the rest of the stack.
A curated Megatron reading path: the parallelism map, what actually splits, how NVIDIA and TPU wrappers differ, and the migration surfaces around NAM56R-style layouts.
A curated TPU sparse-attention reading path: block-sparse contracts, Pallas kernel choices, SPMD sharding, and the runtime surfaces that keep long-context TPU work stable.
A curated path through the H200 lane: operator bring-up, step-time anatomy, memory pressure, and the NVIDIA kernel surfaces that actually moved the stack.
A curated reading order for TPU work: bring-up, PJRT and Torch/XLA boundaries, SPMD sharding, and the kernel/runtime traps that made TPU performance non-obvious.
A curated archive for the C++ data path: corpus selection, semantic enrichment, packaging into training artifacts, and the file-level durability choices that keep the pipeline sane.
A curated path through the expert stack: what the specialist path changed, how routing works, and how the parallelism map constrains the model layout.