Discussion about this post

User's avatar
Raghav Mehta's avatar

The point that outcome-only scoring can't tell a one solution answer to 2 + 3 from one with thirteen solutions is the part I'd like to see pushed further. With GRPO the group baseline only compares correctness, so I wonder if a length penalty relative to the shortest correct sample in the group would cut the redundant verification without hurting recovery on hard problems. Did any of the studies you drew on measure how much of the extra tokens on easy questions was useful self-checking versus plain repetition?

Ready for more?