formatFollowing · demo-public-json-001
DEMO ONLY: Return a JSON object with keys status and count.
Rubric: Valid JSON · Exact required keys · No surrounding prose
DEMO ONLY · NOT PART OF OFFICIAL HEAVRYBENCH REAL LOCAL SUITES · CODING V2 + COMPREHENSIVE · SUITES NOT MIXED
FROZEN SUITES
Transparent by construction
Published suite versions are immutable. Local real suites and demo fixtures remain separate.
REAL LOCAL SUITES
Captured on this machine
| Suite | Max | Protocol | Run date | Entries |
|---|---|---|---|---|
| Coding v2Executable coding suite. Score = tasks passed. Not comparable to the Comprehensive suite. | 12 tasks | temperature 0 · seed 42 · context 8192 · max output 1024 · thinking OFF · first-shot + one repair · hidden tests executed | 2026-09-12 | 9 |
| ComprehensiveKnowledge / Chinese / reasoning / coding / tools / online / agent / hallucination. Not comparable to Coding v2. | 80 points | 23-question fixed suite · temperature 0 · seed 42 · context 8192 · max output 128 · thinking OFF · rubric out of 80 | 2026-09-11 | 13 |
PUBLIC SET
View prompts and rubrics
instructionFollowing · demo-public-complete-001
DEMO ONLY: List exactly three safe testing principles.
Rubric: Exactly three items · All relevant · No unsafe execution
DEMO ONLY · NOT PART OF OFFICIAL HEAVRYBENCHVersion frozen
No silent edits after publication. Corrections and new questions ship in a new suite version.
Explainable scoring
Every aggregate traces to question rubrics and visible subscore definitions.
Suite boundaries
Coding v2 and Comprehensive scores never share a single mixed ranking.