En
title: 英文全文 · Variation in Verification: Understanding Verification Dynamics in Large Language Models
paper_id: 'OpenReview DcEuBwrWnB; arXiv:2509.17995'
year: '2026'
Published as a conference paper at ICLR 2026
VARIATION IN V ERIFICATION : U NDERSTANDING V ERI -
FICATION DYNAMICS IN L ARGE L ANGUAGE M ODELS
Yefan Zhou1,2∗, Austin Xu1†, Yilun Zhou1†, Janvijay Singh1,3∗, Jiang Gui2 , Shafiq Joty1
1
Salesforce AI Research 2 Dartmouth College 3 University of Illinois Urbana-Champaign
A BSTRACT
Recent advances have shown that scaling test-time computation enables large
arXiv:2509.17995v2 [cs.CL] 14 Apr 2026 language models (LLMs) to solve increasingly complex problems across diverse domains. One effective paradigm for test-time scaling (TTS) involves LLM gen- erators producing multiple solution candidates, with LLM verifiers assessing the correctness of these candidates without reference answers. In this paper, we study generative verifiers, which perform verification by generating chain-of-thought (CoT) reasoning followed by a binary verdict. We systematically analyze verifi- cation dynamics across three dimensions – problem difficulty, generator capability, and verifier generation capability – through empirical studies on 12 benchmarks across mathematical reasoning, knowledge, and natural language reasoning tasks using 14 open-source models (2B to 72B parameter range) and GPT-4o. Our exper- iments reveal three key findings about verification effectiveness: (1) Easy problems allow verifiers to more reliably certify correct responses; (2) Weak generators pro- duce errors that are easier to detect than strong generators; (3) Verification ability is generally correlated with the verifier’s own problem-solving capability, but this relationship varies with problem difficulty. These findings reveal opportunities for optimizing basic verification strategies in TTS applications. First, given the same verifier, some weak generators can nearly match stronger ones in post-verification TTS performance (e.g., the Gemma2-9B to Gemma2-27B performance gap shrinks by 75.7%). Second, we identify cases where strong verifiers offer limited advan- tages over weak ones, as both fail to provide meaningful verification gains, suggest- ing that verifier scaling alone cannot overcome fundamental verification challenges.
Code Project
1 I NTRODUCTION
Large language models (LLMs) have advanced rapidly in solving reasoning tasks such as
mathematics and code generation, yet their outputs remain unreliable, often containing subtle or
obvious mistakes (Ke et al., 2025; Lightman et al., 2023). LLM based verification (Angelopoulos
et al., 2025; Huang et al., 2024; Mao et al., 2024; Pan et al., 2025; Pandit et al., 2025) has emerged
as a central mechanism to identify such errors in a scalable manner. Recent work has increasingly
focused on generative verifiers (Liu et al., 2025d; Mahan et al., 2024; Zhang et al., 2025), which
frame verification as next-token prediction: the model typically generates a chain-of-thought (CoT)
reasoning trace and then outputs a binary verdict token. This approach has been shown to outperform
earlier discriminative verifiers or scalar reward models (RMs, Lightman et al., 2023), as it better
leverages the inherent text-generation capabilities of LLMs. One valuable downstream application
of automatic verification is test-time scaling (TTS), where additional inference-time compute is
allocated to improve generation performance. A popular paradigm of TTS is the use of a verifier
model to evaluate candidate responses, filter errors, and identify correct solutions. This approach
underlies techniques such as rejection sampling (Brown et al., 2024), re-ranking (Zhou et al., 2025),
weighted majority voting (Wang et al., 2024a; 2023), and step-level generation (Snell et al., 2025).
Current practice in LLM verification often deploys strong, typically closed-source frontier models as
verifiers. This practice rests on the assumption that verification quality scales with a verifier’s capa-
∗
Work done during an internship at Salesforce AI Research
†
Work done at Salesforce AI Research
1
Published as a conference paper at ICLR 2026
✓
✓ ✗ ✓
✓
✓
✗
✗
✗
✗
✗
✓
(a)
(b)
Figure 1: Overview of our study on verification dynamics. (a) We consider generative verification: an LLM generator produces a solution to a problem, and an LLM verifier conditions on the problem and solution to generate a verification CoT followed by a binary verdict (“Correct”/“Incorrect”). We design controlled experiments that vary problem difficulty, generator generation capability, and verifier generation capability, investigating how each of these factors influences verification performance. (b) Our analysis reveals three patterns: problem difficulty governs recognition of correct responses (true positives); generator generation capability determines error detectability (true negatives); and verifier generation capability correlates with performance in a difficulty-dependent manner, revealing non-linear regimes left uncovered in prior work. The three plots were generated by aggregating benchmark data across three domains and averaging performance metrics over 15 models.
bility to solve the same problem (i.e., its generation capability), a correlation demonstrated in recent work (Chen et al., 2025c; Krumdick et al., 2025; Tan et al., 2025). However, this practice may be sub- optimal given that verifying a solution is often easier than generating one from scratch, a phenomenon referred to as “verification asymmetry” (Wei, 2025). This asymmetry appears in several fields. In convex optimization, dual certificates enable efficient validation of optimality of a proposed solution, while in factorization, verifying correctness is trivial compared to finding the prime factors. Thus, it is worth investigating verification as a distinct capability rather than merely a byproduct of generation. Despite extensive research on generation dynamics and the factors influencing generation qual- ity (Allen-Zhu & Li, 2025; Chen et al., 2024; Ye et al., 2025), the dynamics of verification remain largely unexplored. In particular, little is known about how problem characteristics, properties of generated responses, and model capabilities interact to determine verification effectiveness. Without understanding verification dynamics, one can risk misallocating computational resources by default- ing to expensive frontier models when simpler alternatives might suffice. This gap in understanding motivates our central research question: what factors influence verification success? In this paper, we present a systematic study of generative verification across three dimensions–problem difficulty, generator capability, and verifier generation capability–shown in Figure 1. We quantify verification performance by measuring the probability of the verifier recognizing both correct and in- correct generated solutions in controlled experimental settings. We focus on verifiable problems with objective ground-truth answers in mathematical reasoning, knowledge question-answering (QA), and natural language (NL) reasoning domains. This allows us to objectively measure verifier and gener- ator performance, while simulating the reference-free evaluation settings where verifiers are typically deployed in practice, e.g., in TTS. While our experiments use these domains as a testbed, we believe the insights should extend to any domain where correctness can be reliably defined and checked. Main Findings. While prior work showed that verifier generation capability correlates with verifica- tion performance (Chen et al., 2025c; Krumdick et al., 2025; Tan et al., 2025), we reveal that two additional factors, problem difficulty and generator capability, also critically influence verification success, as illustrated in Figure 1. Our analysis reveals:
2
Published as a conference paper at ICLR 2026
• Problem difficulty primarily governs the recognition of correct solutions: verifiers are more likely to recognize correct solutions on easy problems than on difficult ones. • Generator capability influences error detection: errors made by weak generators are easier to detect than those made by strong generators. • Verifier generation capability correlates with verification performance in a manner dependent on problem difficulty: saturated (or uncorrelated) for easy problems, linear for medium problems, and threshold-limited for hard problems. Our empirical analysis includes 2,347 math problems from eight datasets, 1,196 knowledge QA problems, and 901 NL reasoning problems, evaluated across 14 open-source models and GPT-4o. Application to TTS. We demonstrate the practical implications of our findings for TTS. First, given the same verifier, the TTS performance of a weak generator can nearly match the performance of a strong generator. For instance, Gemma2-9B achieves comparable performance to Gemma2-27B when both use the same verifier, GPT-4o. Second, we identify regimes where a strong verifier (e.g., GPT-4o) offers no additional benefit and can be replaced by a weaker verifier (e.g., Qwen2.5-7B), with both providing limited gains. This occurs with strong generators or with problems at either extreme of the difficulty spectrum.
2 R ELATED W ORK Automatic Evaluation. The deployment of LLMs as evaluators has emerged as a central mechanism for scalable assessment, with efforts focusing on training specialized small evaluators through fine- tuning (Singh et al., 2026; Wang et al., 2025; Whitehouse et al., 2025; Xu et al., 2025; 2026; Zhang et al., 2025). Beyond reference-based verifiers (Chen et al., 2025a; Liu et al., 2025b), verification approaches include self-verification (Chen et al., 2023; Huang et al., 2023; 2024; Kumar et al., 2024; Shinn et al., 2023), where models reflect on or critique their own outputs, and multi-agent verifi- cation (Li et al., 2023; Lifshitz et al., 2025; Zhuge et al., 2024), where multiple agents collaborate in debate-style or hierarchical setups to improve reliability. Prior work identifies several factors influencing evaluation performance, with evaluator generation capability being particularly important. Krumdick et al. (2025) find that evaluator performance changes significantly based on whether the evaluator is capable of answering the question or not. Tan et al. (2025) demonstrate the correlation between pairwise judging ability and generation ability on the same set of problems. Chen et al. (2025b) observe linear relationships between evaluation improvements and reasoning-required sample proportions in fine-tuned evaluators. Chen et al. (2025c) show a strong positive correlation between generation capability and evaluation accuracy. Our work extends these findings by identifying unexplored factors and showing that the relationship between evaluator generation capability and evaluation quality is more nuanced than previously understood. Lu et al. (2025) is a concurrent work with a similar research objective, finding that cross-family verification is particularly effective, com- plementing our analysis of how problem difficulty and model capability shape verification dynamics. Verification for Test-Time Scaling. Early studies explore how to effectively apply verification meth- ods to improve TTS performance. Snell et al. (2025) show RMs improve various TTS approaches, including Best-of-N and beam search, while Liu et al. (2025a) find that compute-optimal strategies vary with policy models and problem difficulty. Recent work explores alternatives to discriminative RMs: Zhang et al. (2025) show trained generative verifiers outperform RMs in Best-of-N, and Zhou et al. (2025)’s JETTS benchmark demonstrates generative evaluators match outcome RMs in reranking. While verification benefits from increased model size and test-time compute, recent work addresses how to reduce these computational costs. Saad-Falcon et al. (2025) propose a framework to aggregate weak verifiers to approach strong ones; Angelopoulos et al. (2025) balance weak/strong evaluators for efficiency; Stroebl et al. (2024) analyze fundamental limits of resampling with imperfect verifiers; and Singhi et al. (2025) propose strategies to balance solving-verification trade-offs. Our work studies the factors driving verification and explores their implications for TTS.
3 E XPERIMENTAL S ETUP 3.1 P RELIMINARIES
Problem and Response Space. Let x denote a problem with ground-truth answer y ∗ (x). A model response r to x consists of a CoT solution and a final answer a(r), and we consider the response
3
Published as a conference paper at ICLR 2026
correct if a(r) = y ∗ (x). As discussed in Section 1, our study uses verifiable problems with objective answers, allowing us to rigorously evaluate verifier outputs against ground-truth while simulating reference-free evaluation settings. Generator and Verifier. A generator G maps a problem x to a distribution over responses, denoted r ∼ G(·|x). A verifier V takes a problem–response pair (x, r) and outputs a judgment of correctness. In the binary case, V (x, r) ∈ {0, 1}, where 1 indicates acceptance and 0 indicates rejection. More generally, a generative verifier produces a verification CoT explaining its reasoning, followed by an explicit verdict such as “Correct” or “Incorrect.” The prompt templates are provided in Appendix A. Generation Capability. We measure the generation capability of a model using its pass rate. For a generator G and problem x, we define pG (x) = Pr[a(r) = y ∗ (x) | r ∼ G(·|x)] as the pass rate on a single problem, i.e., the probability that G solves x correctly on one sampled attempt. We define 1 P pG (D) = |D| x∈D pG (x) as the pass rate aggregated over a dataset D, which we use as the overall measure of a model’s generation capability. Since the verifiers we study are generic LLMs (e.g., GPT- 4o), we measure a verifier’s generation capability using the same metric by prompting it as a generator. We estimate these pass rates empirically as p̂G (x; K) and p̂G (D; K) by sampling K responses per model-problem pair. Since K is fixed at 64 throughout, we often omit K for simplicity, writing p̂G (x) and p̂G (D). We use temperature 0.7 and top-p 1.0 as default sampling hyperparameters, and adopt recommended settings when available (e.g., temperature 0.7 and top-p 0.8 for the non-thinking mode of Qwen3). Ground-truth correctness is established with Math-Verify (Kydlíček, 2025), supplemented by LLM-as-a-judge grading to reduce false negatives (details in Appendix B.1). These 64 responses per problem–model pair are used to estimate generation capability and problem difficulty. Problem Difficulty. We define the P difficulty of a problem as the average pass rate across a set of 1 diverse generators G, d(x) = |G| G∈G p̂G (x). This score reflects how broadly solvable a problem is: if most generators succeed, d(x) is high (easy problem), while if few succeed, d(x) is low (hard problem). It provides a model-agnostic way to partition problems by difficulty, extending prior work (Snell et al., 2025), which measured difficulty relative to a single generator. Verification Metrics and Evaluation. We evaluate verifiers using true positive rate (TPR), the probability of the verifier accepting a correct response: TPR = E[V (x, r) | a(r) = y ∗ (x)], and true negative rate (TNR), the probability of rejecting an incorrect response: TNR = E[1 − V (x, r) | a(r) ̸= y ∗ (x)].1 We also report balanced accuracy, Accbal = 12 (TPR + TNR), which accounts for class imbalance. For verification evaluation, we subsample 8 responses from each 64-sample pool, balanced with 4 correct and 4 incorrect when possible. For very hard problems with fewer than 4 correct responses, we keep all correct ones and sample incorrect ones to reach 8 total (and vice versa for easy problems). Each verifier evaluates responses from all 15 models over the full test set using greedy decoding, unless a controlled subset is specified. Verification-Augmented Test-time Scaling. We consider the TTS setting of sampling multiple responses from the generator and filtering with a verifier before evaluation. For each problem x ∈ D, we sample K responses from the generator using a fixed temperature, with K = 64 in our experiments. Without verification, TTS performance is measured as p̂G (D; K) (or p̂G (D)), the empirical pass rate defined above. With verification, the verifier V evaluates each candidate, and only responses deemed “Correct” are retained for evaluation. The performance of verification-augmented TTS ! is measured as K 1 X 1 X 1 a(ri ) = y ⋆ (x) · V (x, ri ) p̂G,V (D; K) = (1) |D| K ′ i=1 x∈D PK where K ′ = i=1 V (x, ri ). This metric represents the conditional pass rate, i.e., the fraction of correct responses among those retained by the verifier. A corner case arises when the verifier rejects all responses (K ′ = 0); in this case, we set the metric to the generator’s pass rate p̂G (D), so evaluation reverts to selecting from the original K responses in the non-verified setting. We define the verification gain from verifier V as the difference relative to the performance without verification, ∆p̂V = p̂G,V (D) − p̂G (D), which quantifies how much gain can be attributed to verification. Note that our formulation of TTS differs from the common setting where a single “best” response (e.g., by majority vote) is selected and then evaluated. Instead, we report the empirical pass rate of the 1 If the verifier generates an invalid output (e.g., due to the CoT running out of max generation length), we treat it as an uninformative verdict of “Correct” and “Incorrect” each with probability of 50%. Computationally, we set V (x, r) = 0.5 in this case, and also in Equation 1.
4
Published as a conference paper at ICLR 2026
verifier-retained pool, which can be interpreted as the expected accuracy of uniformly sampling one response from that pool. This expectation-based view captures the average quality of verifier-retained responses without tying performance to a specific selection strategy.
3.2 TASKS AND M ODELS
Mathematical Reasoning. We collect a total of 2,347 problems from the test sets of eight mathe- matical reasoning benchmarks: GSM8K (Cobbe et al., 2021), MATH500 (Hendrycks et al., 2021), OlympiadBench (He et al., 2024), AIME24/25 (Li et al., 2024), AMC23 (Li et al., 2024), Minerva- Math (Lewkowycz et al., 2022), and BBEH Multi-step Arithmetic (Kazemi et al., 2025). We use the entire test sets of these benchmarks, except for GSM8K, from which we subsample 600 of 1,319 problems to balance difficulty distribution and reduce the proportion of easy problems. Knowledge. We use a subset of MMLU-Pro (Wang et al., 2024b) as our knowledge category. We randomly subsample 10% from each of its 14 disciplines, yielding 1,196 problems. MMLU-Pro consists of college-level multiple-choice questions spanning STEM, humanities, and social sciences. Natural Language Reasoning. We collect 901 multiple-choice problems from three benchmarks. (1) ReClor (validation set, Yu et al., 2020), a multiple-choice benchmark requiring logical analysis of short passages. (2) FOLIO (Han et al., 2022), a first-order logic reasoning benchmark in natural language. (3) GPQA Diamond (Rein et al., 2024), a dataset that consists of graduate-level multiple- choice science questions, requiring multi-step reasoning. Models. We use 14 open-source models from four families: (1) Qwen2.5 at 3B, 7B, and 72B (Team, 2024); Qwen3 at 4B, 8B, and 32B (Yang et al., 2025); (2) Llama-3.2 at 3B, Llama-3.1 at 8B, and Llama-3.3 at 70B (Grattafiori et al., 2024); (3) Gemma-2 at 2B, 9B, and 27B (Team et al., 2024); (4) Ministral 8B and Mistral-Small-24B; and one closed-source model GPT-4o (Hurst et al., 2024). All models are instruction-tuned versions by default. Each model is used both as a generator and a verifier. We use abbreviated model names in figures for space efficiency; see Appendix B.2 for mappings.
4 E XPERIMENTAL RESULTS
Our experiments focus on how problem difficulty and generator and verifier generation capability influence verification performance. We present the three research questions and main findings below. • RQ1: How does problem difficulty affect verification? (Section 4.1) TPR increases steadily with decreasing problem difficulty, meaning verifiers better recognize correct responses on easier problems. However, TNR shows no predictable relationship with problem difficulty. This indicates that problem difficulty primarily influences correctness recognition. • RQ2: How does the generator’s generation capability influence verification? (Section 4.2) As generators become stronger, TNR decreases substantially while TPR increases only slightly. This reveals that generator capability primarily determines error detectability: stronger generators produce errors that are harder for verifiers to identify. • RQ3: How does verifier generation capability impact verification? (Section 4.3) Verifier generation capability and verification performance are generally positively correlated. However, the form of correlation depends heavily on problem difficulty: linear correlation occurs in medium- difficulty problems, while nonlinear patterns appear in other difficulty levels. Throughout this section, we use the oracle problem difficulty measure defined in Section 3.1, which relies on ground-truth labels, to avoid measurement error. In Section D.1, we validate all key findings using a practical, label-free difficulty estimator and show that the trends remain consistent.
4.1 H OW D OES P ROBLEM D IFFICULTY A FFECT V ERIFICATION ? To examine how problem difficulty influences verification, we partition problems into four equal-sized quartiles by their difficulty score d(x), termed “hardest”, “hard”, “easy”, and “easiest”. Problem difficulty primarily influences the verifier’s ability to recognize correct responses. Our analysis is conducted at two levels of granularity: response level and problem level. Both analyses reveal that problem difficulty mainly shapes the verifier’s sensitivity to correct responses, while not consistently affecting its ability to identify incorrect responses.
5
Published as a conference paper at ICLR 2026
Qwen2.5 Qwen3 Llama3 Small (2-4B) Large (70B+)
Gemma2 Mistral GPT-4o Medium (7-32B)
1.0 1.0 1.0
True Positive Rate 0.5 0.5 0.5
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
Problem Difficulty Problem Difficulty Problem Difficulty
(a) TPR (Mathematics) (b) TPR (Knowledge) (c) TPR (NL Reasoning)
1.0 1.0 1.0
True Negative Rate 0.5 0.5 0.5
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
Problem Difficulty Problem Difficulty Problem Difficulty
(d) TNR (Mathematics) (e) TNR (Knowledge) (f) TNR (NL Reasoning)
Figure 2: Problem difficulty primarily affects TPR of verification. Each curve shows verifier performance across four difficulty groups, with the x-axis indicating problem difficulty and the y-axis reporting TPR (a-c) and TNR (d-f). Colors denote model families, and line styles indicate model size.
49.0%
50 False Negatives (FN)
At the response level, we compute the TPR (% of correct samples) and TNR of all responses within each difficulty 40 FN with Solving Mistakes
Percentage (%)
(% of correct samples)
quartile. As shown in Figure 2, TPR increases 30 steadily as problems become easier, while TNR 22.2% shows no clear trend. This pattern is consis- 20 18.1%
tent across model families and domains. At the 8.7% 10 5.0% problem level, we pool responses from all gen- 2.0% 0.6% 0.2% erators for each problem and compute a single 0 Hardest Hard Easy Easiest TPR and TNR per problem. The distribution Problem Difficulty of these metrics within each quartile is reported in Figures 9 and 10 of Section C.1. We observe Figure 3: Interpreting RQ1 finding by analyzing that easier problems yield higher and more sta- solving mistakes in false-negative verifications. ble TPR, while harder problems exhibit lower Blue: percentage of FNs among sampled correct and more variable TPR. In contrast, TNR dis- responses. Orange: percentage of those FNs tributions show no consistent correlation with whose verification CoTs contain solving mistakes. problem difficulty. In Section C.2, we show that Error bars: 95% CIs. the main verification dynamics about TPR we identified generalize to reasoning models, while extended reasoning provides benefits and alters TNR behavior. We additionally confirm that this trend persist for a large open-source model (Qwen3-235B) in Section D.3 and across different verification prompts in Section D.4. To understand this pattern, case studies in Figure 32 show that verifiers tend to generate their own ref- erence solutions for comparison during verification. As the problem difficulty increases, these verifier- generated answers become increasingly incorrect, producing false negatives (FNs) that reduce TPR. We support this explanation with a large-scale analysis. We use LLM-as-judge to detect whether a verification CoT contains solving mistakes, with experimental details provided in Section B.3. As we show in Figure 3, as problem difficulty increases, the verifier increasingly makes mistakes in gener- ating the reference answer, with 39.1% of verification FNs containing a reference answer mistake in the hard set of problems. This shows that faulty reference generation is a large driving factor of FNs.
4.2 H OW D OES G ENERATOR C APABILITY I NFLUENCE V ERIFICATION ?
We study how generator capability affects verifier performance by having each verifier evaluate responses from each generator. Generators of different capabilities may produce extreme response
6
Published as a conference paper at ICLR 2026
Performance Rate
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
G2-2B 0.72 0.65 0.65 0.81 0.82 0.83 0.79 0.85 0.81 0.77 0.83 0.81 0.81 0.83 0.84 G2-2B 0.47 0.44 0.46 0.56 0.44 0.57 0.55 0.58 0.61 0.56 0.67 0.64 0.63 0.68 0.68 L3-3B 0.61 0.49 0.61 0.68 0.65 0.58 0.69 0.56 0.75 0.67 0.75 0.73 0.73 0.75 0.78
L3-3B 0.66 0.66 0.70 0.77 0.75 0.76 0.79 0.80 0.81 0.82 0.85 0.82 0.85 0.87 0.88 L3-3B 0.52 0.60 0.51 0.69 0.64 0.62 0.65 0.73 0.74 0.68 0.79 0.76 0.76 0.83 0.80 G2-2B 0.51 0.44 0.57 0.57 0.51 0.54 0.62 0.42 0.67 0.63 0.69 0.68 0.68 0.70 0.74
L3-8B 0.53 0.58 0.62 0.68 0.63 0.64 0.70 0.69 0.72 0.69 0.73 0.71 0.66 0.66 0.68 M-8B 0.67 0.70 0.75 0.82 0.72 0.75 0.78 0.86 0.87 0.86 0.89 0.89 0.89 0.92 0.91 M-8B 0.79 0.67 0.84 0.91 0.82 0.82 0.88 0.84 0.92 0.90 0.94 0.91 0.92 0.94 0.95
M-8B 0.66 0.75 0.78 0.92 0.81 0.83 0.94 0.94 0.95 0.83 0.96 0.96 0.91 0.92 0.93 Q2.5-3B 0.44 0.62 0.63 0.73 0.65 0.55 0.59 0.77 0.82 0.77 0.85 0.78 0.83 0.84 0.85 Q2.5-3B 0.51 0.20 0.38 0.59 0.49 0.28 0.71 0.22 0.69 0.53 0.76 0.66 0.72 0.69 0.77
G2-9B 0.78 0.81 0.83 0.91 0.93 0.93 0.92 0.94 0.95 0.92 0.95 0.95 0.95 0.96 0.96 L3-8B 0.39 0.45 0.49 0.51 0.53 0.55 0.56 0.61 0.64 0.61 0.68 0.67 0.66 0.71 0.68 L3-8B 0.40 0.26 0.37 0.50 0.46 0.39 0.54 0.35 0.50 0.42 0.54 0.52 0.55 0.57 0.57
G2-27B 0.74 0.82 0.85 0.92 0.92 0.93 0.93 0.94 0.94 0.92 0.95 0.95 0.96 0.96 0.98 G2-9B 0.69 0.71 0.74 0.80 0.78 0.89 0.90 0.87 0.89 0.90 0.92 0.92 0.93 0.94 0.94 G2-9B 0.75 0.58 0.74 0.84 0.80 0.81 0.86 0.82 0.89 0.87 0.94 0.91 0.93 0.92 0.95
Verifier Model
Q2.5-3B 0.74 0.89 0.89 0.94 0.86 0.88 0.96 0.95 0.96 0.88 0.96 0.96 0.92 0.92 0.93 G2-27B 0.63 0.69 0.73 0.78 0.77 0.84 0.86 0.86 0.88 0.89 0.92 0.91 0.93 0.94 0.95 Q2.5-7B 0.61 0.42 0.62 0.75 0.65 0.59 0.72 0.59 0.79 0.70 0.80 0.78 0.79 0.80 0.83
M-24B 0.64 0.74 0.77 0.86 0.82 0.84 0.88 0.93 0.91 0.86 0.94 0.93 0.94 0.94 0.95 Q2.5-7B 0.54 0.62 0.66 0.75 0.70 0.69 0.72 0.82 0.83 0.81 0.88 0.84 0.87 0.89 0.89 G2-27B 0.74 0.62 0.78 0.82 0.80 0.83 0.85 0.84 0.89 0.89 0.92 0.90 0.93 0.92 0.95
Q2.5-7B 0.73 0.87 0.88 0.93 0.88 0.90 0.97 0.95 0.97 0.93 0.97 0.97 0.96 0.96 0.96 Q3-4B 0.51 0.58 0.62 0.65 0.66 0.70 0.73 0.76 0.82 0.79 0.86 0.85 0.83 0.86 0.89 Q3-4B 0.57 0.50 0.61 0.65 0.63 0.64 0.66 0.62 0.77 0.70 0.79 0.75 0.74 0.76 0.81
L3-70B 0.85 0.94 0.96 0.96 0.96 0.96 0.97 0.98 0.98 0.98 0.99 0.98 0.99 0.99 0.99 M-24B 0.55 0.59 0.68 0.70 0.71 0.77 0.81 0.82 0.85 0.88 0.91 0.90 0.90 0.93 0.94 M-24B 0.61 0.44 0.65 0.72 0.71 0.71 0.78 0.72 0.85 0.82 0.88 0.87 0.87 0.88 0.92
GPT-4o 0.73 0.85 0.87 0.92 0.90 0.92 0.95 0.96 0.96 0.95 0.98 0.98 0.98 0.98 0.98 Q3-8B 0.54 0.59 0.64 0.67 0.66 0.73 0.77 0.77 0.82 0.84 0.87 0.85 0.85 0.87 0.91 Q3-8B 0.61 0.57 0.68 0.67 0.66 0.69 0.70 0.72 0.79 0.76 0.80 0.76 0.76 0.76 0.83
Q2.5-72B 0.81 0.91 0.92 0.95 0.94 0.96 0.98 0.98 0.98 0.97 0.99 0.99 0.99 0.99 0.99 L3-70B 0.81 0.86 0.87 0.88 0.92 0.89 0.92 0.92 0.96 0.95 0.97 0.97 0.97 0.98 0.98 L3-70B 0.88 0.82 0.87 0.89 0.93 0.91 0.92 0.93 0.97 0.95 0.97 0.96 0.97 0.98 0.98
Q3-4B 0.70 0.80 0.83 0.86 0.86 0.86 0.88 0.90 0.91 0.90 0.92 0.93 0.94 0.95 0.95 Q2.5-72B 0.72 0.76 0.82 0.85 0.86 0.86 0.92 0.92 0.94 0.94 0.96 0.96 0.97 0.98 0.97 Q2.5-72B 0.85 0.77 0.87 0.90 0.90 0.90 0.92 0.92 0.96 0.95 0.97 0.97 0.98 0.97 0.98
Q3-8B 0.74 0.85 0.87 0.89 0.89 0.90 0.91 0.93 0.93 0.93 0.95 0.95 0.97 0.98 0.98 GPT-4o 0.66 0.71 0.77 0.79 0.81 0.83 0.86 0.88 0.91 0.92 0.94 0.94 0.95 0.96 0.96 GPT-4o 0.75 0.68 0.79 0.81 0.84 0.86 0.86 0.88 0.92 0.93 0.94 0.94 0.95 0.96 0.96
Q3-32B 0.70 0.82 0.84 0.86 0.87 0.88 0.89 0.91 0.92 0.92 0.93 0.94 0.95 0.96 0.96 Q3-32B 0.56 0.63 0.68 0.69 0.73 0.77 0.80 0.78 0.82 0.85 0.87 0.89 0.87 0.90 0.92 Q3-32B 0.66 0.64 0.71 0.72 0.74 0.78 0.76 0.80 0.84 0.83 0.85 0.88 0.84 0.86 0.90
L3
L3
M- -2B
-3B
-8B
8B L3
M-
Q2 B -2B
-3B
.5-83B M-
Q2 B -3B
-2B
.5-83B
G2G2
Q2 B -9B
-27 G2 L3
G2 -8B
-9B Q2 L3
G2 -8B
-9B
G2 M-
Q2 B
L3
GP
Q2 o
.5-
.5-
24
.5-
-70
T-4
72
3B
7B
B G2 Q2 B
Q3
M-
Q3
L3
-27
.5-
-4B
247B
-8B
-7 B L3 G2
G2Q3
M-
Q3
L3
.5-
-27
247B
-4B
-8B
-7
B
B
Q3Q3
Q3 -4B
-8B
-32 B
B Q2 0B
GP.5-
Q3 o 72
T-4
-32 B
B Q2 0B
GP.5-
Q3 o 72
T-4
-32 B
B
Generator Model Generator Model Generator Model
(a) TPR (Mathematics) (b) TPR (Knowledge) (c) TPR (NL Reasoning)
G2-2B 0.90 0.72 0.71 0.56 0.72 0.67 0.61 0.54 0.59 0.65 0.55 0.54 0.54 0.50 0.48 G2-2B 0.73 0.76 0.77 0.66 0.76 0.67 0.69 0.64 0.65 0.69 0.57 0.63 0.63 0.58 0.58 L3-3B 0.56 0.68 0.58 0.51 0.54 0.60 0.45 0.61 0.42 0.49 0.39 0.46 0.41 0.35 0.36
L3-3B 0.91 0.68 0.66 0.50 0.73 0.67 0.54 0.49 0.47 0.61 0.39 0.47 0.52 0.52 0.40 L3-3B 0.73 0.62 0.74 0.58 0.63 0.70 0.66 0.55 0.56 0.64 0.52 0.53 0.55 0.45 0.51 G2-2B 0.77 0.78 0.66 0.70 0.79 0.67 0.64 0.67 0.64 0.59 0.61 0.63 0.61 0.57 0.57
L3-8B 0.95 0.77 0.73 0.72 0.82 0.79 0.70 0.66 0.63 0.72 0.57 0.59 0.68 0.68 0.59 M-8B 0.57 0.52 0.45 0.39 0.51 0.52 0.45 0.32 0.36 0.32 0.31 0.32 0.29 0.26 0.28 M-8B 0.52 0.59 0.37 0.23 0.45 0.39 0.25 0.34 0.26 0.24 0.19 0.29 0.21 0.16 0.15
M-8B 0.91 0.78 0.76 0.48 0.71 0.66 0.44 0.44 0.42 0.69 0.39 0.41 0.57 0.57 0.52 Q2.5-3B 0.79 0.62 0.64 0.53 0.63 0.68 0.66 0.51 0.48 0.50 0.47 0.61 0.48 0.49 0.46 Q2.5-3B 0.61 0.88 0.68 0.57 0.62 0.76 0.44 0.79 0.43 0.60 0.40 0.55 0.44 0.44 0.37
G2-9B 0.80 0.69 0.66 0.44 0.35 0.29 0.47 0.38 0.34 0.53 0.32 0.29 0.42 0.40 0.28 L3-8B 0.87 0.82 0.82 0.79 0.77 0.82 0.78 0.75 0.73 0.75 0.68 0.74 0.71 0.67 0.67 L3-8B 0.81 0.86 0.78 0.76 0.75 0.77 0.74 0.79 0.72 0.76 0.70 0.72 0.70 0.69 0.67
G2-27B 0.92 0.68 0.64 0.44 0.60 0.48 0.47 0.39 0.35 0.58 0.31 0.31 0.37 0.38 0.24 G2-9B 0.49 0.55 0.55 0.47 0.51 0.21 0.24 0.39 0.36 0.33 0.31 0.37 0.29 0.26 0.29 G2-9B 0.57 0.72 0.52 0.41 0.44 0.41 0.34 0.37 0.33 0.34 0.26 0.33 0.25 0.24 0.21
Verifier Model
Q2.5-3B 0.89 0.59 0.64 0.40 0.66 0.60 0.36 0.37 0.32 0.68 0.35 0.32 0.59 0.57 0.53 G2-27B 0.68 0.62 0.62 0.56 0.54 0.48 0.41 0.43 0.41 0.39 0.33 0.40 0.35 0.31 0.30 Q2.5-7B 0.65 0.81 0.67 0.44 0.62 0.64 0.47 0.60 0.48 0.52 0.38 0.48 0.42 0.41 0.38
M-24B 0.96 0.89 0.85 0.71 0.82 0.76 0.74 0.53 0.61 0.72 0.48 0.52 0.54 0.52 0.41 Q2.5-7B 0.78 0.67 0.61 0.51 0.59 0.66 0.61 0.37 0.40 0.42 0.37 0.49 0.33 0.36 0.34 G2-27B 0.60 0.74 0.54 0.46 0.51 0.45 0.40 0.39 0.34 0.35 0.31 0.37 0.29 0.27 0.20
Q2.5-7B 0.93 0.70 0.64 0.47 0.70 0.60 0.35 0.36 0.24 0.55 0.24 0.22 0.38 0.36 0.27 Q3-4B 0.86 0.84 0.82 0.79 0.78 0.76 0.73 0.68 0.57 0.67 0.52 0.61 0.62 0.60 0.49 Q3-4B 0.81 0.86 0.77 0.76 0.74 0.72 0.71 0.74 0.57 0.67 0.55 0.63 0.62 0.64 0.52
L3-70B 0.93 0.65 0.54 0.51 0.67 0.56 0.47 0.35 0.29 0.27 0.19 0.22 0.28 0.30 0.18 M-24B 0.81 0.80 0.71 0.70 0.66 0.60 0.54 0.54 0.49 0.44 0.36 0.47 0.36 0.33 0.30 M-24B 0.77 0.88 0.70 0.66 0.67 0.59 0.56 0.56 0.49 0.45 0.38 0.39 0.38 0.31 0.31
GPT-4o 0.96 0.91 0.87 0.76 0.81 0.72 0.70 0.59 0.55 0.65 0.38 0.44 0.49 0.49 0.38 Q3-8B 0.88 0.88 0.82 0.80 0.80 0.76 0.71 0.71 0.60 0.66 0.51 0.67 0.61 0.60 0.50 Q3-8B 0.83 0.84 0.76 0.79 0.77 0.74 0.74 0.69 0.61 0.66 0.57 0.66 0.60 0.62 0.52
Q2.5-72B 0.92 0.76 0.68 0.50 0.61 0.50 0.43 0.33 0.23 0.44 0.19 0.15 0.28 0.28 0.17 L3-70B 0.64 0.55 0.54 0.49 0.37 0.45 0.39 0.37 0.28 0.28 0.25 0.21 0.22 0.17 0.20 L3-70B 0.48 0.63 0.48 0.41 0.29 0.36 0.32 0.30 0.21 0.22 0.16 0.14 0.15 0.11 0.11
Q3-4B 0.96 0.91 0.90 0.88 0.86 0.83 0.87 0.79 0.79 0.79 0.74 0.73 0.59 0.59 0.54 Q2.5-72B 0.71 0.67 0.57 0.50 0.50 0.41 0.36 0.33 0.32 0.26 0.25 0.27 0.17 0.16 0.19 Q2.5-72B 0.54 0.65 0.47 0.39 0.37 0.37 0.28 0.31 0.22 0.20 0.14 0.16 0.12 0.12 0.12
Q3-8B 0.96 0.92 0.88 0.85 0.83 0.79 0.83 0.73 0.73 0.74 0.66 0.63 0.51 0.50 0.44 GPT-4o 0.79 0.74 0.69 0.68 0.61 0.59 0.53 0.52 0.47 0.40 0.40 0.41 0.31 0.25 0.32 GPT-4o 0.70 0.79 0.61 0.63 0.57 0.48 0.46 0.44 0.37 0.31 0.29 0.25 0.25 0.16 0.20
Q3-32B 0.97 0.90 0.85 0.87 0.83 0.78 0.83 0.75 0.74 0.70 0.65 0.66 0.55 0.54 0.42 Q3-32B 0.89 0.86 0.82 0.82 0.80 0.73 0.70 0.74 0.67 0.66 0.61 0.62 0.59 0.57 0.46 Q3-32B 0.83 0.85 0.76 0.81 0.77 0.70 0.73 0.69 0.63 0.62 0.56 0.53 0.57 0.55 0.43
G2
L3
L3
M- -2B
-3B
-8B
8B L3
M-
Q2 B -2B
-3B
.5-8
3B M-
Q2 B -3B
-2B
.5-8
3B
G2G2
Q2 B -9B
-27 G2 L3
G2 -8B
-9B Q2 L3
G2 -8B
-9B
M-
Q2 B
L3
GP
Q2 o
.5-
.5-
24
.5-
-70
T-4
72
3B
7B
B G2 Q2 B
Q3
M-
Q3
L3
-27
.5-
-7
7B
-4B
24
-8BB L3 G2
G2Q3
M-
Q3
L3
.5-7B
-27
-4B
24
-8B
-7
B
B
Q3Q3
Q3 -4B
-8B
-32 B
B Q2 0B
GP.5-
Q3 o 72
T-4
-32 B
B Q2 0B
GP.5-
Q3 o 72
T-4
-32 B
B
Generator Model Generator Model Generator Model
(d) TNR (Mathematics) (e) TNR (Knowledge) (f) TNR (NL Reasoning)
Figure 4: Generator capability influences verifier performance of identifying incorrect responses. Heatmaps show (a-c) TPR and (d-f) TNR when pairing 15 verifier models (rows) with 15 generator models (columns). Rows and columns are ordered by models’ generation capability computed on all problems of each domain. Values indicate mean performance over the evaluation subset.
distributions, e.g., weak generators may produce no correct response on hard problems within 64 samples. To ensure fair comparison, we compute TPR on problem subsets where all generators produce at least one correct response. Analogously, TNR is computed on problems where all generators produce at least one incorrect response. Details are provided in Section B.4. As shown in Figures 4a to 4c, TPR remains uniformly high across nearly all settings and increases further with stronger generators. The heatmap is dominated by red colors, with values mostly above 0.7, indicating that most verifiers are already reliable at recognizing correct responses. As generator capability improves, TPR approaches 1.0. This suggests that generator strength influences recognition of correct responses in a relatively mild way. Generator capability correlates with error detection in verification. In Figures 4d to 4f, moving from weaker generators on the left to stronger ones on the right, the heatmap shifts generally from red to blue, indicating a substantial decrease in TNR. For example, in the Mathematics domain, for the Qwen2.5-72B verifier, TNR drops from 0.68 on solutions generated by Llama-3.1-8B to 0.17 on those by Qwen3-32B. The overall pattern is consistent across three domains and nearly all verifiers. These results show that generator capability strongly modulates the detection of incorrect responses. In Section C.2, we show that this finding generalizes to reasoning models. We further verify that this trend remains robust when scaling to a large open-source model (Qwen3-235B; Section D.3) and when varying the verification prompt (Section D.4). We interpret this phenomenon through case studies in Figure 33. Strong generators produce internally consistent reasoning chains where early mistakes (e.g., missed cases) propagate coherently, yielding well-structured but incorrect solutions that cause the verifier false positives. Weak generators produce surface-level errors such as self-contradictions, facilitating verifier rejection. We support this explanation with a large-scale analysis. We use an independent LLM-as-judge to determine whether
7
Published as a conference paper at ICLR 2026
a generator’s CoT contains surface-level errors, with experimental details provided in Section B.3. The results in Figure 5 show a clear trend: as generator capability increases, the frequency of surface-level errors decreases. This is consistent with Yin et al. (2025), who find that error patterns of stronger models shift from basic computational mistakes to more sophisticated reasoning failures.
0.9
Surface-Level Error Rate
0.8
Surface-Level Error Rate
0.8
0.7
0.6
0.6
0.4 0.5
0.4
0.2 0.3
-2
L3 B
-3
L3 B
M-
G2 B
G2 9B
Q2 7B -8B
-
-2 8 0.2
G2 .5-
M- B
Q2 4B 2
.5-
L3 7B 3
0.3 0.4 0.5 0.6 0.7
GP 0B-7
Q2 T-4o
.5-
Q3 B72
-4
Q3 B
Q3 8B -
-32 B
Model Generator Capability
(a) Surface-level Error Rate vs. Name (b) Surface-level Error Rate vs. Capability
Figure 5: Interpreting RQ2 finding by analyzing surface-level errors in generators’ solutions. (a) The x-axis lists generator models sorted by generation capability; the y-axis shows the percentage of responses containing surface-level errors. (b) The x-axis shows the capability values of the same generators; the y-axis is the same as in (a). Error bars indicate 95% CIs.
4.3 H OW DOES V ERIFIER G ENERATION C APABILITY I MPACT V ERIFICATION ?
Qwen2.5 Llama3 Mistral 2-4B 24-32B
Qwen3 Gemma2 GPT 7-9B 70B+
0.9 Linear R²=0.864, Nonparam R²=0.919 0.9 Linear R²=0.926, Nonparam R²=0.950 0.9 Linear R²=0.810, Nonparam R²=0.887
0.8 0.8
Balanced Accuracy Balanced Accuracy Balanced Accuracy 0.8 0.7 r = 0.96 0.7 r = 0.90 0.7 r = 0.93 0.6 0.6
0.6 0.5 0.5
0.4 0.4
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Verifier's Generation Capability Verifier's Generation Capability Verifier's Generation Capability
(a) All data (Mathematics) (b) All data (Knowledge) (c) All data (NL Reasoning)
Hard: Linear R²=0.71, Nonparam R²=0.85 0.9 Hard: Linear R²=0.40, Nonparam R²=0.52 0.9 Hard: Linear R²=0.08, Nonparam R²=0.04
0.9
Medium: Linear R²=0.90, Nonparam R²=0.90 Medium: Linear R²=0.91, Nonparam R²=0.93 Medium: Linear R²=0.84, Nonparam R²=0.85
Easy: Linear R²=0.48, Nonparam R²=0.88 0.8 Easy: Linear R²=0.69, Nonparam R²=0.87 0.8 Easy: Linear R²=0.59, Nonparam R²=0.79
Balanced Accuracy Balanced Accuracy Balanced Accuracy 0.8 0.7 0.7 0.7 r = 0.83 0.6 0.6 r = 0.95 r = 0.70 r = 0.96 r = -0.28 r = 0.77 0.6 0.5 0.5 r = 0.84 r = 0.92 r = 0.63 0.4 0.4 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Verifier's Generation Capability Verifier's Generation Capability Verifier's Generation Capability (d) Stratified view (Mathematics) (e) Stratified view (Knowledge) (f) Stratified view (NL Reasoning)
Figure 6: Distinct correlation forms between verification performance and generation capability. Solid lines represent nonparametric fits to the data; r indicates the Pearson correlation coefficient. (a- c) Averaged across all problems, verifier generation capability exhibits a strong linear correlation with balanced accuracy. (d-f) When stratified by problem difficulty, distinct correlation patterns emerge.
We measure verifier generation capability and evaluate verification performance using balanced accuracy (Accbal ) on the entire test set. Each verifier is evaluated on responses from all generators, and we report results both averaged across all problems and stratified by problem difficulty. To characterize the relationship between generation capability and verification performance, we employ locally weighted regression (Cleveland, 1979) with a bandwidth of 0.6 to fit nonparametric curves. We compare R2 values between nonparametric and linear fits to assess linearity. We also report the Pearson correlation coefficient (Benesty et al., 2009) as another measure of linear correlation.
8
Published as a conference paper at ICLR 2026
Figures 6a to 6c show a strong overall correlation between verifier generation capability and verifica- tion accuracy, with NL reasoning showing less linearity than other domains. This result is consistent with prior work showing that evaluator accuracy tends to track the evaluator’s task performance, with the relationship appearing nearly linear. While this global trend validates findings in prior work (Chen et al., 2025c; Tan et al., 2025), a closer inspection of the trend reveals highly non-linear regimes. Verifier generation capability influences verification accuracy differently based on problem difficulty. Stratified analysis reveals regime-dependent correlation with phase-transition behavior. We partition problems into 10 equal-width bins by difficulty d(x) and analyze three representative intervals: hard [0.1, 0.3), medium [0.4, 0.5), and easy [0.8, 0.9) in Figures 6e, 6f and 23a. For hard problems (blue), verification accuracy shows minimal improvement with increasing capability. Mathematics plateaus around 0.65 accuracy after initial gains, while other domains remain flat throughout. Notably, verifiers achieve below-random accuracy on hard NL Reasoning problems, which we analyze in Appendix C.3. Medium problems (yellow) exhibit steady accuracy increases with capability, indicating strong linear relationships. This is confirmed by linear and nonparametric fits that yield nearly identical R2 values, with r > 0.9. Easy problems exhibit a threshold effect at the x-axis around 0.9: below this threshold, the relationship is linear; above it, small capability improvements yield large verification gains. Hard and easy regimes show nonlinearity with nonparametric R2 exceeding linear R2 by 0.1–0.2 and r < 0.85. The exception is NL Reasoning on hard problems, where both fits yield near-zero R2 , indicating no meaningful capability-accuracy relationship. Appendix C.4 provides additional results (Figure 14), including analysis of why different patterns are observed in varying difficulty intervals.
5 A PPLICATION TO T EST-T IME S CALING (TTS)
Our analysis in Section 4 is conducted with verification itself as the end goal. However, our findings have direct implications for TTS. We analyze two research questions in TTS settings that naturally arise out of our previous findings, and present our results below: • RQ4: Given a fixed verifier, can a weak generator match a stronger generator in TTS? (Sec- tion 5.1) Weak generators can nearly match stronger generators’ post-verification performance. Verification gains peak at weak-medium generators by achieving a high error detection rate (TNR) while maintaining a moderately high correctness recognition rate (TPR). • RQ5: Can weak verifiers match the gains of strong verifiers in TTS? (Section 5.2) The verification gain gap between weak and strong verifiers narrows at both low and high problem difficulty extremes, and when using strong generators. The following sections present results on the Mathematics domain, with complete results across all three domains in Sections C.5 and C.6. In Section D.2, we show that these findings can guide generator and verifier selection in TTS and lead to efficiency improvements.
5.1 C AN W EAK G ENERATORS M ATCH S TRONGER G ENERATORS IN TTS?
We evaluate TTS with a fixed verifier (GPT-4o) by varying generator capability and reporting pass rates before and after verification, along with the verification gain ∆p̂V . Verification gain peaks for weak–medium generators, enabling them to approach stronger models post-verification. As shown in Figure 7a, weak generators start with much lower pass rates but improve dramatically after verification, reaching levels comparable to larger models. For example, Gemma2-9B starts from a significantly lower baseline but, after verification, achieves a pass rate nearly matching Gemma2-27B. The performance gap shrinks from 10.3% to 2.5%, closing 75.7% of the original difference. Figure 7b explains this phenomenon: as generator strength increases (left to right), TNR decreases sharply while TPR rises only modestly, consistent with RQ2 findings. Consequently, verification gain (gray bars) peaks at weak-medium generators, which achieve high TNR for effective error filtering while maintaining moderate TPR to preserve correct responses. For the strongest generators, errors become harder to identify, causing TNR decline and limiting gains. In Appendix C.5, we show the findings derived from the Mathematics domain generalize well to two other domains in Figure 15. We provide additional evidence confirming the generalizability: first, verification gains peak for weak-medium generators across a broad range of problem difficulties (d(x) ≥ 0.3) in all domains (Figures 16 to 18); second, performance gaps of most weak and strong
9
Published as a conference paper at ICLR 2026
Before Verification After Verification TNR TPR Verification Gain
0.4 1.0
Verification Gain
0.3 0.8
TPR / TNR
0.6
0.2
0.4
0.1 0.2
0.0 -2B
0.0
L-
L-
M- B
G2 B 3B
8
8
G2 -9B
-
Q2 27B
.5-
G2 M- 3B
Q2 24B
.5-
Q3 B
L- B
gp B 7
-4
70
Q3 t-4o
Q B-32
Q2 3-8B
.5-
Generator Models
72 B
(a) Pass rate (b) Verification gain and TPR/TNR
Figure 7: TTS performance before and after verification when varying generator strength. Results are reported on problems with difficulty in the range [0.7, 0.8) from the Mathematics domain, including 181 problems. (a) Pass rate before (blue) and after (orange) adding a fixed verifier (GPT-4o). The generators in the x-axis are ordered from weaker (left) to stronger (right) by generation capability measured on the problem subset. (b) Bar chart shows the verification gain ∆p̂V (left y-axis) for each generator. Lines show the verifier’s TNR and TPR (right y-axis).
model pairs can be reduced by verification when evaluated on the entire domain datasets (Figure 19), mostly achieving 30-50% reduction. These results suggest that weak generators, when paired with a strong verifier, can serve as a cost-effective alternative to strong generators. One might question, however, whether such a pairing is truly efficient given the cost of a strong verifier (GPT-4o). In Section D.5, we address this with a case study showing that pairing a weak generator with a strong verifier can match the performance of a strong generator alone while reducing token consumption. Generators Verifiers Weak (Llama3.2-3B) Strong (Qwen2.5-72B) Strong (GPT-4o) Weak (Qwen2.5-7B) Medium (Llama3.1-8B) 0.25 1.0
Verification Gain Gap
0.20 0.8
True Positive Rate True Negative Rate
0.15 0.8
0.10 0.6
0.6
0.05 0.4
0.00 0.4
0.05 0.2
(0-
[0. 1)
1-00. 1-0
[0. 2) . ) (0-
[0. 1)0.
[0. .2)
2-0
[0. .3)
3-0
[0. .4)
4-0
[0. .5)
5-0 0.1 2-0
[0. 3)
3-0
[0. 4)
4-0
[0. 5) .
.
.
1-0
[0. 2)
2-0
[0. 3)
3-0
[0. 4)
4-0
[0. 5)
.
.
.
.
[0. .6)
6-0
[0. .7)
7-0
[0. .8)
8-0
[0. .9)
9-1 (0- 5-0
[0. 6)
[0.6-0
[0. 7)
7-0
[0. 8)
8-0
[0. 9)
.
.
.
.
5-0
[0. 6)
6-0
[0. 7)
7-0
[0. 8)
8-0
[0. 9)
.
.
.
.
.0) 9-1
Problem Difficulty
.0) 9-1
Problem Difficulty Problem Difficulty
.0)
(a) Verification Gain Gap (b) TPR (c) TNR
Figure 8: Weak vs. strong verifiers under varying problem difficulty and generator strength. The x-axis shows problem difficulty, ordered from hardest to easiest, measured relative to each generator. (a) Verification gain gap between strong and weak verifiers when applied to weak, medium, and strong generators. (b) TPR increases as problems become easier; shadow band indicates TPR gap between two verifiers. (c) As generators strengthen, TNR decreases overall and the TNR gap (shadow band) narrows. Results are from the Mathematics domain.
5.2 C AN W EAK V ERIFIERS M ATCH THE G AINS OF S TRONG V ERIFIERS IN TTS?
We analyze the verification gain gap between a strong verifier (GPT-4o) and a weaker one (Qwen2.5- 7B) across problem difficulty ranges and generator strengths. The verification gain ∆p̂V is defined in Section 3 and the gap between verifier is ∆p̂Vstrong − ∆p̂Vweak . Our goal is to identify when this gap narrows, as such regimes suggest weak verifiers can substitute for strong ones. The gap narrows on the extremes of problem difficulty As shown in Figure 8a, the verification gain gap shrinks as problems become easier, which corresponds to the rising TPR for both weak and strong verifiers seen in Figure 8b. This aligns with our RQ1 findings that easier problems improve TPR for all verifiers. Even weak verifiers reliably recognize correct responses on easy problems, leaving little room for strong ones to provide additional benefit. At the opposite extreme, the gap also narrows on the hardest problems. As discussed in RQ3 and shown in Figures 21d to 21f, increasing
10
Published as a conference paper at ICLR 2026
verifier generation capability (or scaling up to larger models) fails to improve verification accuracy on hard problems, resulting in only marginal performance differences between weak and strong verifiers. The gap narrows as generators become stronger. Figure 8 shows that increasing generator capability reduces the difference between weak and strong verifiers. This is consistent with RQ2 (Section 4.2), where we observed that the verifier’s TNR decreases as the generator capability increases. As both weak and strong verifiers experience lower TNR, the gap between them also shrinks, shown as the narrowing shaded band between solid and dashed curves in Figure 8c). The results here are obtained from the Mathematics domain, and we show the findings generalize to two other domains in Section C.6. In regimes of very easy/hard problems or when evaluating strong generators’ responses, weak verifiers provide gains to TTS performance comparable to strong verifiers. However, these convergence regimes coincide with minimal verification benefit overall. Figures 21a to 21c shows verification gains drop to 0.1 or below for both verifiers on easy and hard problems, verification on strong generators yields peak gains of only 0.1, precisely where the gap narrows. Thus, while weak and strong verifiers converge in these regimes, this convergence occurs where both provide minimal practical value. This reveals that scaling verifiers from 7B models to GPT-4o fails to overcome fundamental verification challenges, with GPT-4o providing limited improvement over small open-source models in the identified regimes.
6 C ONCLUSION We study LLM verification across problem difficulty, generator capability, and verifier generation capability, revealing that verification success depends on their interactions. We find that problem diffi- culty primarily shapes correct solution recognition, generator capability influences error detectability, and verifier generation capability correlates with verification in problem difficulty-dependent patterns. We examine the implications of these findings for verification deployment in TTS, identifying both opportunities and limitations. Stronger generators may not be necessary, as weaker generators can approach the post-verification performance of stronger ones when paired with a fixed verifier. This suggests potential for strategic model pairing that could reduce computational costs in verifier-based TTS methods. Our results also identify regimes where investing in larger verifiers yields no benefit, such as when evaluating responses from strong generators or problems at difficulty extremes.
11
Published as a conference paper at ICLR 2026
E THICS S TATEMENT This work exclusively evaluates large language models on publicly available academic benchmarks containing mathematical reasoning, knowledge, and natural language problems with objective ground- truth answers. All experiments involve automated evaluation of model outputs without human subject participation. The datasets used are established research benchmarks designed for educational problem-solving tasks. Our study aims to understand verification dynamics to improve the compu- tational efficiency of LLM systems, posing no ethical concerns regarding privacy, harmful content generation, or potential misuse.
R EPRODUCIBILITY S TATEMENT We provide comprehensive details to ensure reproducibility of our findings. All experiments use publicly available datasets and open-source/commercial LLMs. We specify the model names, versions, dataset sources, and inference hyperparameters in Section 3.2. Complete prompt templates for both generation and verification tasks are provided in Section A. The mathematical formulations of all metrics, along with estimation procedures and aggregation methods, are formally defined and clearly described in Section 3.1 and Section B.
R EFERENCES Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.2, knowledge manipulation. In The Thirteenth International Conference on Learning Representations, 2025. Anastasios N Angelopoulos, Jacob Eisenstein, Jonathan Berant, Alekh Agarwal, and Adam Fisch. Cost-optimal active ai model evaluation. arXiv preprint arXiv:2506.07949, 2025. Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. In Noise reduction in speech processing, pp. 1–4. Springer, Berlin, Heidelberg, 2009. Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024. Ding Chen, Qingchen Yu, Pengyuan Wang, Wentao Zhang, Bo Tang, Feiyu Xiong, Xinchi Li, Minchuan Yang, and Zhiyu Li. xverify: Efficient answer verifier for reasoning model evaluations. arXiv preprint arXiv:2504.10481, 2025a. Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. Judgelrm: Large reasoning models as a judge. arXiv preprint arXiv:2504.00050, 2025b. Qiguang Chen, Libo Qin, Jiaqi WANG, Jingxuan Zhou, and Wanxiang Che. Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of-thought. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. Wei-Lin Chen, Zhepei Wei, Xinyu Zhu, Shi Feng, and Yu Meng. Do llm evaluators prefer themselves for a reason? arXiv preprint arXiv:2504.03846, 2025c. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128, 2023. Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, et al. Rm-r1: Reward modeling as reasoning. arXiv preprint arXiv:2505.02387, 2025d. W.S. Cleveland. Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 74(368):829–836, 1979. doi: 10.1080/01621459.1979.10481038. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
12
Published as a conference paper at ICLR 2026
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. URL https://zenodo.org/records/12608602. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, et al. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv:2209.00840, 2022. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008, 2024. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021. Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceed- ings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 1051– 1068, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/ 2023.emnlp-main.67. URL https://aclanthology.org/2023.emnlp-main.67/. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In The Twelfth International Conference on Learning Representations, 2024. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, San- ket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, et al. Big-bench extra hard. arXiv preprint arXiv:2502.19187, 2025. Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al. A survey of frontiers in llm reasoning: Inference scaling, learning to reason, and agentic systems. arXiv preprint arXiv:2504.09037, 2025. Michael Krumdick, Charles Lovering, Varshini Reddy, Seth Ebner, and Chris Tanner. No free labels: Limitations of llm-as-a-judge without human grounding. arXiv preprint arXiv:2503.05061, 2025. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.12917, 2024. H. Kydlíček. Math-verify: Math verification library, 2025. URL https://github.com/ huggingface/math-verify. Sungjae Lee, Hyejin Park, Jaechang Kim, and Jungseul Ok. Semantic exploration with adaptive gating for efficient problem solving with language models. arXiv preprint arXiv:2501.05752, 2025. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in neural information processing systems, 35:3843–3857, 2022.
13
Published as a conference paper at ICLR 2026
Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q. Jiang, Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 2024. Available at https://huggingface.co/datasets/AI-MO/NuminaMath-CoT. Ruosen Li, Teerth Patel, and Xinya Du. Prd: Peer rank and discussion improve large language model based evaluations. arXiv preprint arXiv:2307.02762, 2023. Shalev Lifshitz, Sheila A. McIlraith, and Yilun Du. Multi-agent verification: Scaling test-time compute with multiple verifiers. In Second Conference on Language Modeling, 2025. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023. Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1b llm surpass 405b llm? rethinking compute-optimal test-time scaling. arXiv preprint arXiv:2502.06703, 2025a. Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chengqi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F Wong, Songyang Zhang, et al. Compassverifier: A unified and robust verifier for llms evaluation and outcome reward. arXiv preprint arXiv:2508.03686, 2025b. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. In Conference on Language Modeling (COLM), 2025c. Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint arXiv:2504.02495, 2025d. Jack Lu, Ryan Teehan, Jinran Jin, and Mengye Ren. When does verification pay off? a closer look at llms as solution verifiers. arXiv preprint arXiv:2512.02304, 2025. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint arXiv:2410.12832, 2024. Yujun Mao, Yoon Kim, and Yilun Zhou. Champ: A competition-level dataset for fine-grained analyses of llms’ mathematical reasoning capabilities. In Findings of the Association for Computational Linguistics ACL 2024, pp. 13256–13274, 2024. Melissa Z Pan, Mert Cemri, Lakshya A Agrawal, Shuyi Yang, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Kannan Ramchandran, Dan Klein, Joseph E. Gonzalez, Matei Zaharia, and Ion Stoica. Why do multiagent systems fail? In ICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025. Shrey Pandit, Austin Xu, Xuan-Phi Nguyen, Yifei Ming, Caiming Xiong, and Shafiq Joty. Hard2verify: A step-level verification benchmark for open-ended frontier math, 2025. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. Jon Saad-Falcon, E Kelly Buchanan, Mayee F Chen, Tzu-Heng Huang, Brendan McLaughlin, Tanvir Bhathal, Shang Zhu, Ben Athiwaratkun, Frederic Sala, Scott Linderman, et al. Shrinking the generation-verification gap with weak verifiers. arXiv preprint arXiv:2506.18203, 2025. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652, 2023. Janvijay Singh, Austin Xu, Yilun Zhou, Yefan Zhou, Dilek Hakkani-Tür, and Shafiq Joty. On the shelf life of finetuned LLM-judges: Future proofing, backward compatibility, and question generalization. In The Fourteenth International Conference on Learning Representations, 2026.
14
Published as a conference paper at ICLR 2026
Nishad Singhi, Hritik Bansal, Arian Hosseini, Aditya Grover, Kai-Wei Chang, Marcus Rohrbach, and Anna Rohrbach. When to solve, when to verify: Compute-optimal problem solving and generative verification for llm reasoning. arXiv preprint arXiv:2504.01005, 2025.
Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time com- pute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025.
Benedikt Stroebl, Sayash Kapoor, and Arvind Narayanan. Inference scaling flaws: The limits of llm resampling with imperfect verifiers. arXiv preprint arXiv:2411.17501, 2024.
Shaoning Sun, Jiachen Yu, Zongqi Wang, Xuewei Yang, Tianle Gu, and Yujiu Yang. S2j: Bridg- ing the gap between solving and judging ability in generative reward models. arXiv preprint arXiv:2509.22099, 2025.
Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Yuan Tang, Alejandro Cuadron, Chenguang Wang, Raluca Popa, and Ion Stoica. Judgebench: A benchmark for evaluating LLM-based judges. In The Thirteenth International Conference on Learning Representations, 2025.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024.
Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
Han Wang, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. Soft self-consistency improves language model agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024a.
PeiFeng Wang, Austin Xu, Yilun Zhou, Caiming Xiong, and Shafiq Joty. Direct judgement preference optimization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, November 2025.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi- task language understanding benchmark. Advances in Neural Information Processing Systems, 37: 95266–95290, 2024b.
Jason Wei. The asymmetry of verification and verifier’s law. https://www.jasonwei.net/ blog/asymmetry-of-verification-and-verifiers-law, 2025.
Chenxi Whitehouse, Tianlu Wang, Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, and Swarnadeep Saha. J1: Incentivizing thinking in llm-as-a-judge via reinforcement learning. arXiv preprint arXiv:2505.10320, 2025.
Austin Xu, Yilun Zhou, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. J4r: Learning to judge with equivalent initial state group relative policy optimization. arXiv preprint arXiv:2505.13346, 2025.
Austin Xu, Xuan-Phi Nguyen, Yilun Zhou, Chien-Sheng Wu, Caiming Xiong, and Shafiq Joty. Foundational automatic evaluators: Scaling multi-task generative evaluator training for reasoning- centric domains. In The Fourteenth International Conference on Learning Representations, 2026.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
15
Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. Physics of language models: Part 2.1, grade-school math and the hidden reasoning process. In The Thirteenth International Conference on Learning Representations, 2025. Zhangyue Yin, YuHong Sun, Xuanjing Huang, Xipeng Qiu, and Hui Zhao. Error classification of large language models on math word problems: A dynamically adaptive framework. In Findings of the Association for Computational Linguistics: EMNLP 2025, 2025. Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. Reclor: A reading comprehension dataset requiring logical reasoning. arXiv preprint arXiv:2002.04326, 2020. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agar- wal. Generative verifiers: Reward modeling as next-token prediction. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS’24, 2024. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. In The Thirteenth International Conference on Learning Representations, 2025. Yilun Zhou, Austin Xu, PeiFeng Wang, Caiming Xiong, and Shafiq Joty. Evaluating judges as evaluators: The JETTS benchmark of LLM-as-judges as test-time scaling evaluators. In Forty- second International Conference on Machine Learning, 2025. Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, et al. Agent-as-a-judge: Evaluate agents with agents. arXiv preprint arXiv:2410.10934, 2024.
16
Published as a conference paper at ICLR 2026
A PPENDIX
A Prompt Templates 17
B Additional Preliminaries and Setup 23 B.1 Details of Math Problems Correctness Check . . . . . . . . . . . . . . . . . . . . 23 B.2 Model Naming Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 B.3 Detailed Experimental Setup of Large-scale Interpretability Analysis . . . . . . . . 24 B.4 Detailed Experimental Setup of RQ2 . . . . . . . . . . . . . . . . . . . . . . . . . 24
C Extended Analysis and Additional Results 25 C.1 More Details in Problem Difficulty Analysis . . . . . . . . . . . . . . . . . . . . . 25 C.2 Analysis on Reasoning Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 C.3 Explanation of Below-Random Verification Performance on Hard Problems . . . . 30 C.4 Additional Results of Verifier Generation Capability . . . . . . . . . . . . . . . . . 30 C.5 Additional Results of Generator Analysis in Test-Time Scaling . . . . . . . . . . . 31 C.6 Additional Results of Verifier Analysis in Test-Time Scaling . . . . . . . . . . . . 36
D Extended Studies and Ablations 37 D.1 Estimation of Problem Difficulty . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 D.2 Efficiency and Performance Analysis of Model Choices in TTS . . . . . . . . . . . 38 D.2.1 RQ4.1: How to Choose Generators Given a Fixed Verifier? . . . . . . . . . 39 D.2.2 RQ5.1: How to Choose a Verifier Given a Fixed Generator? . . . . . . . . 41 D.3 Ablation Study on Large Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 D.4 Ablation Study on Verification Prompts . . . . . . . . . . . . . . . . . . . . . . . 43 D.5 Comparison Analysis of Strong Model as Generator or Verifier . . . . . . . . . . . 44
E Case Studies 44
T HE U SE OF L ARGE L ANGUAGE M ODELS (LLM S )
LLMs were used exclusively as writing assistance tools in preparing this manuscript. Specifically, we employed LLMs for grammar checking. An LLM-based image generation tool was used to create the robot caricature in Figure 1. All research ideation, experimental design, analysis, and scientific conclusions are entirely the work of the authors. The LLMs played no role in the conception of research questions, methodology development, or interpretation of results. Authors take full responsibility for all content in this paper, including any text refined with LLM assistance.
A P ROMPT T EMPLATES
Response Generation Prompt. Here, we provide the prompts to generate model responses to questions from three domains. For each model, we use its default system prompt as specified in the model documentation.
17
Published as a conference paper at ICLR 2026
Mathematical datasets
### User Prompt {problem} Please reason step by step, and put your final answer within \boxed{{}}.
Knowledge: MMLU-Pro
### User Prompt The following are multiple choice questions (with answers) about {category}. Think step by step and then output the answer in the format of \"The answer is (X)\" where X is the correct letter choice.
Question: {question} Options:
Answer: Let’s think step by step.
Natural Language Reasoning: ReClor
### User Prompt Answer the following multiple choice question. The last line of your response should be of the following format: ’Answer: $LETTER’ (without quotes) where $LETTER is one of ABCD. Think step by step before answering.
{context}
{question}
{choices}
Output your answer strictly following this format:
Reasoning:
Natural Language Reasoning: FOLIO
### User Prompt
Based on the above, can the following be inferred?
Think step-by-step before outputting your answer, which must be "Yes", "No", or "Uncertain"
Output your answer strictly following this format: Reasoning: Your reasoning here Answer: Yes or No or Uncertain
Natural Language Reasoning: GPQA Diamond
### User Prompt Answer the following multiple choice question. The last line of your response should be of the following format: ’Answer: $LETTER’ (without quotes) where $LETTER is one of ABCD. Think step by step before answering.
Question: {question} Options:
Output your answer strictly following this format:
Reasoning:
18
Published as a conference paper at ICLR 2026
Verification Evaluation Prompt. Below, we present the prompt template used to evaluate verification performance.
### System Prompt Please act as an impartial judge and evaluate the correctness of the response provided by an AI assistant to the user prompt displayed below. You will be given the assistant’s response.
When evaluating the assistant’s response, identify any mistakes or inaccurate information. Be as objective as possible. Avoid any biases, such as order of responses, length, or stylistic elements like formatting.
Before providing an your final verdict, think through the judging process and output your thoughts as an explanation
After providing your explanation, you must output only one of the following choices as your final verdict with a label:
- The response is correct: [[Correct]]
- The response is incorrect: [[Incorrect]]
Use the following template: Explanation: Your detailed thought process as an explanation. Verdict: [[Correct]] or [[Incorrect]].
### User Prompt <|User Prompt|>
<|The Start of Assistant’s Answer|> {response} <|The End of Assistant’s Answer|>
Verification Evaluation Prompt (Solve-then-Verify). Below, we present a variant of the prompt template used to evaluate verification performance. This template is adapted from the idea of Chen et al. (2025d); Sun et al. (2025) that prompts the model to solve the problem first and use its own solution as a reference for verification.
### System Prompt Please act as an impartial judge and evaluate the correctness of the response provided by an AI assistant to the user prompt displayed below. You will be given the assistant’s response.
First, you MUST solve the question yourself and put your final answer following the format requested in <|User Prompt|>. Provide your own solution with final answer before proceeding to the evaluation. When evaluating the candidate’s response, you MUST refer to your own solution.
Be as objective as possible. Avoid any biases, such as order of responses, length, or stylistic elements like formatting.
After providing your explanation, you must output only one of the following choices as your final verdict with a label:
- The response is correct: [[Correct]]
- The response is incorrect: [[Incorrect]]
Use the following template: Solution: Your own reasoning and final answer to the problem. Explanation: Your detailed thought process as an explanation. Verdict: [[Correct]] or [[Incorrect]].
### User Prompt <|User Prompt|>
<|The Start of Assistant’s Answer|> {response} <|The End of Assistant’s Answer|>
Verification Evaluation Prompt (Step-by-Step). Below, we adopt a variant of the verification template from Zhang et al. (2024). This template only has a user prompt.
19
Published as a conference paper at ICLR 2026
### User Prompt You are a math teacher. Grade the Solution, verifying correctness step by step. At the end of the Solution verification, when you give your final grade, write it in the form "Verification: Is the answer correct (Yes/No)? X", where X is either \boxed{{Yes}} or \boxed{{No}}. Question: {question} Solution: {response}
Fallback Correction Check Prompt. When Math-Verify returns unparsable or incorrect results, we employ LLM-as-judge as a fallback mechanism for correctness verification. Below, we provide the prompt template used for this secondary verification step:
### User Prompt Given a math problem, its correct answer, and the model’s generated answer, determine if the model’s generated answer is correct.
VALIDATION CRITERIA: 1. Identify the final answer, which is usually put inside \boxed{{answer}} or answer. 2. The answer must be mathematically equivalent to the correct answer 3. The answer must be complete with a clear final result 4. The answer must not just contain similar numbers - it must reach the correct conclusion 5. If the generated answer contains multiple different final answers or is ambiguous about which is the final answer, mark it as ’False’
IMPORTANT: Just having the same numbers as the ground truth is NOT sufficient - the model must actually solve the problem correctly and provide the correct final answer in the designated format.
Respond with ’True’ if the answer is correct and complete, and ’False’ if it is incorrect or incomplete. Directly provide your judgement ’True’ or ’False’ without any other description.
Problem: {problem} Correct Answer: {ground_truth_answer} Model’s Generated Answer: {model_response} Your judgement:
Verification Analysis Prompt for Detecting the Solving Behavior. Below we present the prompt template used to analyze if verification CoTs contain solving behavior. This prompt asks the verifier to solve the problem first and then
### System Prompt Please act as an impartial analyzer and determine whether the AI assistant, when evaluating a candidate response, generates its own solution (full or partial) as a reference - either explicitly or implicitly.
Key Indicators that the assistant GENERATED its own solution [[Yes]]: - The assistant determines what the correct answer should be (even implicitly) - The assistant works through the problem logic independently (e.g., "the premises actually say X", "what can be inferred is Y") - The assistant constructs its own interpretation of what follows from the problem statement - The assistant makes claims about what the "correct reasoning" would be
Key Indicators that the assistant ONLY INSPECTED the candidate [[No]]: - The assistant only points out errors in the candidate’s reasoning flow without determining the correct answer - The assistant only identifies missing steps or unsupported leaps in the candidate’s reasoning chain - The assistant only verifies consistency within the candidate’s own reasoning chain - Focus is purely on "the candidate failed to justify X" rather than "X is actually Y"
Important: Even implicit or partial solutions count as generating a solution. If the assistant reveals what it believes to be true/false about the problem, it has generated a solution.
Example 1:
20
Published as a conference paper at ICLR 2026
<|User Prompt|>
<|The Start of Candidate Response|> {example1_response} <|The End of Candidate Response|>
<|The Start of Assistant’s Evaluation|> {example1_evaluation} <|The End of Assistant’s Evaluation|>
Expected Analysis:
Now analyze the following case. Look carefully for ANY statement where the assistant determines what is actually true, correct, valid, or inferable - these indicate the assistant generated its own solution.
Use the following template: Explanation: Your reasoning for why the assistant did or did not generate its own solution. Quote specific phrases that reveal solution generation. Verdict: [[Yes]] or [[No]]
### User Prompt <|User Prompt|>
<|The Start of Candidate Response|> {response} <|The End of Candidate Response|>
<|The Start of Assistant’s Evaluation|> {evaluation} <|The End of Assistant’s Evaluation|>
Verification Analysis Prompt for Detecting the Reasoning Mistakes. Below, we present the prompt template used to analyze if verification CoTs contain any basic reasoning mistakes.
### System Prompt You are an impartial checker. Your task is to determine whether the evaluator made a mistake in its reasoning when evaluating the candidate response.
You may use the provided gold_answer - whether it is a full explanation or just a final answer - as the reference for what is correct.
You MUST ignore the evaluator’s final verdict entirely. Do NOT read it, do NOT interpret it, and do NOT allow it to influence your judgment in any way.
If the evaluator’s reasoning contradicts the gold_answer, or the evaluator asserts incorrect mathematical or logical statements, then the evaluator’s reasoning is incorrect.
If the evaluator’s reasoning is consistent with the gold_answer and contains no incorrect claims, then the evaluator’s reasoning is correct.
After reviewing the evaluation, output:
Explanation: A brief explanation of whether the evaluator’s reasoning is correct or incorrect. Verdict: [[Correct]] or [[Incorrect]]
### User Prompt <|User Prompt|>
<|Correct Answer of the Problem|>
<|The Start of Candidate Response|> {response} <|The End of Candidate Response|>
21
Published as a conference paper at ICLR 2026
<|The Start of Assistant’s Evaluation|> {evaluation} <|The End of Assistant’s Evaluation|>
Generator Response Analysis Prompt for Identifying Surface-level Error
### System prompt for checking self-contradictory error
You are an impartial analyzer. You will be given a question and a candidate response. The candidate response is known to be incorrect.
Your ONLY task is to determine whether the candidate response is internally consistent.
### Consistency Definitions ###
Definition of "Consistent": - The reasoning does not contradict itself. - The steps follow a coherent internal logic. - The response may contain wrong assumptions, missing cases, or wrong math, but it is still consistent if it does not conflict with itself.
Definition of "Not Consistent": - The candidate response contains an explicit or implicit contradiction. - It states mutually incompatible claims. - Its reasoning steps cannot simultaneously be true based on its own premises.
### Strict Prohibitions ###
IMPORTANT: - DO NOT check correctness. - DO NOT compare to the true answer. - ONLY evaluate whether the reasoning contradicts itself.
Now analyze the following case. Respond using the format below:
Explanation: Briefly explain why the candidate response is internally consistent or inconsistent. Verdict: [[Yes]] (consistent) or [[No]] (not consistent)
### System prompt for checking basic math error
You are an impartial analyzer. You will be given a question and a candidate response. The candidate response is known in advance to be incorrect.
Your ONLY task is to determine whether the candidate response contains basic calculation mistakes or basic mathematical knowledge errors.
### What counts as a basic math error ### A response contains a basic math error if it includes: - Incorrect arithmetic (wrong addition, subtraction, multiplication, division). - Incorrect simplification or algebraic manipulation. - Incorrect fundamental math facts (e.g., parity mistakes, misidentifying primes, incorrect factorial values). - Incorrect counting in basic combinatorics (e.g., number of simple outcomes, pairs). - Incorrect use of basic probability rules (e.g., wrong multiplicative or additive rule).
### What does NOT count as a basic math error ### A response does NOT contain a basic math error if: - All arithmetic steps are numerically correct, - Basic mathematical facts are used correctly, - Basic counting/probability rules are applied correctly, even if the overall reasoning is wrong, even if conclusions are incorrect, even if important steps are missing.
### Strict Prohibitions ### IMPORTANT: - DO NOT evaluate correctness of the final answer. - DO NOT compare the candidate response to the true answer. - DO NOT judge consistency or high-level reasoning. - ONLY check for basic arithmetic or elementary math knowledge mistakes.
Use the following format:
22
Published as a conference paper at ICLR 2026
Explanation: Briefly explain whether the candidate response contains basic calculation
or math-knowledge errors.
Verdict: [[Yes]] (contains a basic math error) or [[No]] (does not contain a basic math
error)
---------------------
### System prompt for checking incompleteness error
---------------------
You are an impartial analyzer. You will be given a question and a candidate response.
The candidate response is known in advance to be incorrect.
Your ONLY task is to determine whether the candidate response actually provides a final
answer to the question.
### What counts as providing a final answer (Verdict: [[Yes]]) ###
A candidate response PROVIDES a final answer if:
- It clearly states a specific final value, equation, or expression that answers the
question,
even if the value is wrong.
- The final answer may be given in any clear format, such as:
- Inside \\boxed{...}
- In LaTeX math, e.g., $\\frac{25}{36}$
- As plain text, e.g., "The probability is 2/3."
- In bold or marked as **Final Answer:** followed by a concrete number or expression.
### SPECIAL CASE: Token Limit Cutoff ###
If the response is clearly CUT OFF due to token limits (e.g., ends mid-sentence,
mid-word,
mid-equation, or stops abruptly), then:
- You must treat this as the model ATTEMPTING to provide a final answer.
- In this case, output Verdict: [[Yes]].
### What counts as NOT providing a final answer (Verdict: [[No]]) ###
A candidate response does NOT provide a final answer if:
- It only gives high-level discussion, explanation, or strategy without stating a
concrete result.
- It says the answer is hard to compute, suggests using a calculator, or leaves the
result as ‘‘you can now compute’’ without doing it.
- It trails off with partial work (e.g., sets up an expression but never evaluates it or
never clearly claims it as the final answer).
- It only restates the problem, gives definitions, or discusses approaches without
committing to an explicit outcome.
### Strict Prohibitions ###
IMPORTANT:
- DO NOT evaluate correctness of the final answer.
- DO NOT compare the candidate response to the true answer.
- DO NOT judge consistency or high-level reasoning.
- ONLY check for providing a final answer.
Use the following format:
Explanation: Briefly explain whether the candidate response provides a final answer.
Verdict: [[Yes]] (provides a final answer) or [[No]] (does not provide a final answer)
B A DDITIONAL P RELIMINARIES AND S ETUP
B.1 D ETAILS OF M ATH P ROBLEMS C ORRECTNESS C HECK
Here we detail the evaluation procedure for establishing response correctness, including fallback methods. Ground-truth correctness is determined using Math-Verify (Kydlíček, 2025). If Math-Verify fails to parse an answer or returns incorrect, we recheck with other string-matching verifiers from open-source repositories lm-eval (Gao et al., 2024), Dr.GRPO (Liu et al., 2025c), and Qwen2.5-Math (Yang et al., 2024). We further apply GPT-4.1-mini and Qwen2.5-72B to con- duct reference-based evaluation and check the equivalence of the model prediction and ground-truth answers. The prompt template for LLM-based verification is provided in Appendix A.
23
Published as a conference paper at ICLR 2026
B.2 M ODEL NAMING C ONVENTIONS
Throughout this paper, we use abbreviated model names in figures and tables to improve readability and space efficiency. Table 1 provides the complete mapping between abbreviations and full model names. All models referenced are instruction-tuned versions unless otherwise specified.
Table 1: Mapping between abbreviated model names used in figures and their full names. All models are instruction-tuned versions.
Abbreviation Full Model Name Abbreviation Full Model Name
G2-2B Gemma2-2B Q3-4B Qwen3-4B
G2-9B Gemma2-9B Q3-8B Qwen3-8B
G2-27B Gemma2-27B Q3-32B Qwen3-32B
L3-3B (L-3B) Llama3.2-3B M-8B Ministral-8B
L3-8B (L-8B) Llama3.1-8B M-24B Mistral-Small-24B
L3-70B (L-70B) Llama3.3-70B gpt-4o GPT-4o
Q2.5-3B Qwen2.5-3B
Q2.5-7B Qwen2.5-7B
Q2.5-72B Qwen2.5-72B
B.3 D ETAILED E XPERIMENTAL S ETUP OF L ARGE - SCALE I NTERPRETABILITY A NALYSIS
This section details the experimental setup for the large-scale interpretability analysis supporting the findings in Section 4.1 (RQ1) and Section 4.2 (RQ2). For RQ1, we apply GPT-4.1-mini as LLM-as-judge with two prompts (in Section A): (i) to determine whether the verifier’s CoT shows evidence of generating its own solution or claim; (ii) if so, to detect whether that generated content contains mathematical or reasoning mistakes. A verification CoT is labeled as containing a solving mistake only if both conditions are satisfied. We run this analysis on 367,920 GPT-4o verification responses, the same ones for computing TPR. Because the number of correct responses varies across difficulty bins (harder problems yield fewer correct responses), we sample 5,000 correct responses per difficulty bin. For each bin, we compute the percentage of FNs and the percentage of FNs that contain solving mistakes. We perform 10,000 bootstrap resamples and report the mean and 95% confidence intervals (CIs). For RQ2, we examine three categories of surface-level errors: (i) self-contradictions, (ii) basic arithmetic or factual mistakes, and (iii) incomplete responses lacking a final answer. The three prompts for detecting each error are included in Section A, and we use GPT-4.1-mini as the judge. We apply this analysis to 98,782 incorrect generator responses, the same ones used for TNR. For each problem, we randomly sample one incorrect response from each of the 15 generators and compute the proportion of these responses that contain surface-level errors. We perform 10,000 bootstrap resamples and report the mean and 95% CIs.
B.4 D ETAILED E XPERIMENTAL S ETUP OF RQ2
Due to significant differences in generator capability, when measuring TPR, for some very difficult problems, none of the 64 responses sampled from a weak model are correct. To ensure fair evaluation unaffected by intrinsic problem difficulty, we exclude these problems and keep only those where every generator produces at least one correct response. We apply analogous filtering for TNR, keeping only problems where each generator produces at least one incorrect response. Beyond filtering problems, we also carefully balance how many responses we evaluate from each generator. As described in Section 3, we subsample 8 responses from each generator’s 64-sample pool for verification evaluation, aiming for 4 correct and 4 incorrect when possible. However, across these 8-response subsets, stronger generators may have produced more correct responses than weaker ones. This would bias our metrics by creating different denominators per generator. To address this, we randomly select one correct response per problem from each generator’s 8-response pool when computing TPR (and analogously for TNR). We repeat this evaluation with random selections eight times and report the mean.
24
Published as a conference paper at ICLR 2026
C E XTENDED A NALYSIS AND A DDITIONAL R ESULTS C.1 M ORE D ETAILS IN P ROBLEM D IFFICULTY A NALYSIS
In Section 4.1, we show that problem difficulty primarily influences the verifier’s ability to recognize correct responses. As discussed in the main paper, our analysis is conducted at two levels of granularity: response level and problem level. Figure 2 shows results at the response level. Figures 9 and 10 show results at the problem level, summarizing the distribution of TPR and TNR across difficulty quartiles. Together, these results confirm our main finding that problem difficulty strongly correlates with TPR but has no systematic effect on TNR.
25
Published as a conference paper at ICLR 2026
Llama-3.2-3B Gemma2-2B Qwen2.5-3B Qwen3-4B Llama3.1-8B
1.0
True Positive Rate 0.5
0.0
Gemma2-9B Qwen2.5-7B Qwen3-8B Llama3.3-70B Gemma2-27B
1.0
True Positive Rate 0.5
0.0
Qwen2.5-72B Qwen3-32B Ministral-8B Mistral-Small-24B GPT-4o
1.0
True Positive Rate 0.5
0.0
(a) Mathematics
Llama-3.2-3B Gemma2-2B Qwen2.5-3B Qwen3-4B Llama3.1-8B
1.0
True Positive Rate 0.5
0.0
Gemma2-9B Qwen2.5-7B Qwen3-8B Llama3.3-70B Gemma2-27B
1.0
True Positive Rate 0.5
0.0
Qwen2.5-72B Qwen3-32B Ministral-8B Mistral-Small-24B GPT-4o
1.0
True Positive Rate 0.5
0.0
(b) Knowledge
Llama-3.2-3B Gemma2-2B Qwen2.5-3B Qwen3-4B Llama3.1-8B
1.0
True Positive Rate 0.5
0.0
Gemma2-9B Qwen2.5-7B Qwen3-8B Llama3.3-70B Gemma2-27B
1.0
True Positive Rate 0.5
0.0
Qwen2.5-72B Qwen3-32B Ministral-8B Mistral-Small-24B GPT-4o
1.0
True Positive Rate 0.5
0.0
(c) NL Reasoning
Figure 9: Problem difficulty correlates with verification TPR on per-problem level across three domains. Each boxplot shows the distribution of per-problem TPR for 15 verifier models, grouped by difficulty quartiles. TPR exhibits a strong positive correlation with problem easiness: easier problems consistently yield higher and less variable TPR.
26
Published as a conference paper at ICLR 2026
Llama-3.2-3B Gemma2-2B Qwen2.5-3B Qwen3-4B Llama3.1-8B
1.0
True Negative Rate 0.5
0.0
Gemma2-9B Qwen2.5-7B Qwen3-8B Llama3.3-70B Gemma2-27B
1.0
True Negative Rate 0.5
0.0
Qwen2.5-72B Qwen3-32B Ministral-8B Mistral-Small-24B GPT-4o
1.0
True Negative Rate 0.5
0.0
(a) Mathematics
Llama-3.2-3B Gemma2-2B Qwen2.5-3B Qwen3-4B Llama3.1-8B
1.0
True Negative Rate 0.5
0.0
Gemma2-9B Qwen2.5-7B Qwen3-8B Llama3.3-70B Gemma2-27B
1.0
True Negative Rate 0.5
0.0
Qwen2.5-72B Qwen3-32B Ministral-8B Mistral-Small-24B GPT-4o
1.0
True Negative Rate 0.5
0.0
(b) Knowledge
Llama-3.2-3B Gemma2-2B Qwen2.5-3B Qwen3-4B Llama3.1-8B
1.0
True Negative Rate 0.5
0.0
Gemma2-9B Qwen2.5-7B Qwen3-8B Llama3.3-70B Gemma2-27B
1.0
True Negative Rate 0.5
0.0
Qwen2.5-72B Qwen3-32B Ministral-8B Mistral-Small-24B GPT-4o
1.0
True Negative Rate 0.5
0.0
(c) NL Reasoning
Figure 10: Problem difficulty shows no systematic correlation with verification TNR on per- problem level across three domains. Each boxplot shows the distribution of per-problem metrics for 15 verifier models, grouped by difficulty quartiles. TNR doesn’t show obvious correlation with problem difficulty, exhibiting inconsistent trends across models.
27
Published as a conference paper at ICLR 2026
C.2 A NALYSIS ON R EASONING M ODEL
Our main analysis focuses on instruction-tuned models, which represent the typical setting for verification systems in current practice, including recent judge models (Tan et al., 2025; Wang et al., 2025) and verifier work (Liu et al., 2025b; Zhang et al., 2025). We prioritize models without extensive CoT reasoning because verification often demands low-latency solutions, particularly for reinforcement learning training and TTS applications where rapid evaluation is critical. However, a recent trend involves training long-reasoning evaluators (Chen et al., 2025d; Whitehouse et al., 2025) that generate extended CoT before making verification decisions. To examine whether our findings generalize to this emerging paradigm, we include two reasoning models (Qwen3-8B-Thinking and Qwen3-32B-Thinking2 ) and analyze how they perform across our research questions. These models generate longer reasoning traces before producing binary verdicts, representing the state-of-the-art in reasoning-enhanced verification. In Figures 11 and 12, we evaluate the conclusion of RQ1 (Section 4.1). We observe that reasoning models exhibit the same TPR pattern as instruction-tuned models: easier problems consistently yield higher TPR across all three domains. This indicates that the fundamental relationship between problem difficulty and correctness recognition persists with extended reasoning. However, reasoning models exhibit a notable difference in TNR behavior. Unlike instruction-tuned models, where TNR showed no systematic relationship with problem difficulty, both reasoning models demonstrate improved TNR as problems become easier across all three domains. This pattern suggests that, with extended reasoning, error detection becomes easier when problems become easier. In Figure 13, we evaluate the findings of RQ2 (Section 4.2) on reasoning models and find that they maintain the core patterns observed in instruction-tuned models. TPR remains consistently high with mild increases as generator strength increases, while TNR decreases more significantly (goes from red to white) with stronger generators. This indicates that the fundamental challenge of detecting errors from capable generators persists despite enhanced reasoning capabilities. These findings demonstrate that reasoning models offer some advantages for error detection on easier problems while preserving the core verification dynamics we identified. Problem difficulty continues to govern correctness recognition, and generator capability primarily influences error detectability across different verification paradigms.
Qwen3-8B-Thinking Qwen3-32B-Thinking
1.0 1.0 1.0
True Positive Rate True Positive Rate True Positive Rate
0.5 0.5 0.5
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
Problem Difficulty Problem Difficulty Problem Difficulty
(a) TPR (Mathematics) (b) TPR (Knowledge) (c) TPR (NL Reasoning)
1.0 1.0 1.0
True Negative Rate True Negative Rate True Negative Rate
0.5 0.5 0.5
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
Problem Difficulty Problem Difficulty Problem Difficulty
(d) TNR (Mathematics) (e) TNR (Knowledge) (f) TNR (NL Reasoning)
Figure 11: Verification performance of reasoning models across problem difficulty at the per- response level. TPR (a-c) and TNR (d-f) for Qwen3-8B-Thinking and Qwen3-32B-Thinking across difficulty quartiles in three domains. Both reasoning models show increasing TPR and TNR as problem difficulty decreases.
2 We use the suggested sampling hyperparameter (temperature 0.6, top-p 0.95).
28
Published as a conference paper at ICLR 2026
True Positive Rate (TPR)
Qwen3-8B-Thinking Qwen3-8B-Thinking Qwen3-8B-Thinking
1.0 1.0 1.0
True Positive Rate True Positive Rate True Positive Rate 0.5 0.5 0.5
0.0 0.0 0.0
(a) Mathematics (b) Knowledge (c) NL Reasoning
Qwen3-32B-Thinking Qwen3-32B-Thinking Qwen3-32B-Thinking
1.0 1.0 1.0
True Positive Rate True Positive Rate True Positive Rate 0.5 0.5 0.5
0.0 0.0 0.0
(d) Mathematics (e) Knowledge (f) NL Reasoning
True Negative Rate (TNR)
Qwen3-8B-Thinking Qwen3-8B-Thinking Qwen3-8B-Thinking
1.0 1.0 1.0
True Negative Rate True Negative Rate True Negative Rate 0.5 0.5 0.5
0.0 0.0 0.0
(g) Mathematics (h) Knowledge (i) NL Reasoning
Qwen3-32B-Thinking Qwen3-32B-Thinking Qwen3-32B-Thinking
1.0 1.0 1.0
True Negative Rate True Negative Rate True Negative Rate 0.5 0.5 0.5
0.0 0.0 0.0
(j) Mathematics (k) Knowledge (l) NL Reasoning
Figure 12: Verification metrics for reasoning models across difficulty quartiles at the per- problem level. Each boxplot shows the distribution of per-problem TPR and TNR for Qwen3-8B- Thinking and Qwen3-32B-Thinking across difficulty quartiles in three domains. Both TPR and TNR distributions shift higher and become less variable as problems become easier.
Performance Rate
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Q3-8B-thinking
Verifier 0.84 0.92 0.94 0.93 0.94 0.94 0.96 0.97 0.97 0.97 0.97 0.97 0.98 0.98 0.98 Q3-8B-thinking 0.74 0.80 0.80 0.81 0.83 0.82 0.85 0.86 0.88 0.90 0.90 0.91 0.90 0.92 0.93 Q3-8B-thinking 0.85 0.83 0.85 0.84 0.87 0.88 0.88 0.89 0.90 0.90 0.90 0.92 0.91 0.92 0.92
Q3-32B-thinking 0.80 0.90 0.92 0.92 0.93 0.94 0.95 0.96 0.96 0.97 0.97 0.97 0.98 0.98 0.99 Q3-32B-thinking 0.75 0.80 0.81 0.84 0.85 0.84 0.86 0.87 0.89 0.90 0.91 0.92 0.90 0.92 0.93 Q3-32B-thinking 0.86 0.85 0.87 0.87 0.88 0.88 0.89 0.90 0.91 0.91 0.91 0.92 0.91 0.92 0.93
-2B L3 8B -9B B
-27 .5- 24 B .5- 7B T-4
.5- o -2B -3B 8B .5- -8B -9B B .5- -4B B -8B -70 B T-4 B -3B -2B 8B .5- -8B -9B .5- B -4B B -8B -70 B T-4 B
G2 L3-3B -8B M- G2 3B M- L3-70
Q3 72B
-4B G2 L3 M- 3B L3 G2 -27 7B Q3 24 Q3 B .5- 72 o -32 L3 G2 M- 3B L3 G2 7B -27 Q3 24 Q3 B .5- 72 o -32
G2 Q2 Q2 B GP Q3
Q3
Q2
-8B
-32 B Q2 G2 Q2 M- L3 GP Q3 Q2 Q2 G2 M- L3 GP Q3
Q2 Q2
Generator Generator Generator
(a) TPR (Mathematics) (b) TPR (Knowledge) (c) TPR (NL Reasoning)
Q3-8B-thinking
Verifier 0.95 0.95 0.92 0.93 0.88 0.86 0.91 0.84 0.86 0.81 0.78 0.79 0.67 0.63 0.60 Q3-8B-thinking 0.83 0.81 0.77 0.78 0.74 0.72 0.69 0.70 0.61 0.64 0.55 0.62 0.60 0.59 0.51 Q3-8B-thinking 0.74 0.78 0.73 0.73 0.70 0.68 0.67 0.65 0.59 0.60 0.53 0.54 0.59 0.55 0.50
Q3-32B-thinking 0.95 0.95 0.93 0.93 0.88 0.87 0.91 0.85 0.86 0.81 0.79 0.79 0.68 0.67 0.60 Q3-32B-thinking 0.85 0.85 0.80 0.80 0.78 0.74 0.73 0.72 0.69 0.68 0.64 0.69 0.63 0.62 0.53 Q3-32B-thinking 0.78 0.77 0.75 0.77 0.73 0.67 0.72 0.67 0.64 0.65 0.61 0.58 0.58 0.58 0.52
-2B L3 8B -9B B .5- 24 .5- 7B
L3 -8B M- -27 3B B L3
GP
Q2 o-70
T-4 B -2B -3B 8B
M- .5- 3B -8B -9B B
-27 .5- 7B -4B B
24 -8B -70B .5-
B T-4 o
B
-32 -3B -2B 8B
M- .5- 3B -8B -9B .5- 7B
B
-27 -4B B
24 -8B -70B .5-
B T-4 o
B
-32
G2 -3B G2 G2 Q2 M- Q2
.5-
Q3
Q3
72B
-4B G2 L3 Q2 L3 G2 G2 Q2 Q3 M- Q3 L3 72 GP Q3 L3 G2 Q2 L3 G2 Q2 G2 Q3 M- Q3 L3 72 GP Q3
Q3 -8B
-32 B Q2 Q2
Generator Generator Generator
(d) TNR (Mathematics) (e) TNR (Knowledge) (f) TNR (NL Reasoning)
Figure 13: Reasoning models as verifiers paired with generators of varying capability. TPR (a-c) and TNR (d-f) for Qwen3-8B-Thinking and Qwen3-32B-Thinking verifiers when evaluating responses from 15 generator models across three domains. Generators are ordered left-to-right by increasing generation capability, measured separately for each domain. Red indicates higher performance, blue indicates lower performance.
29
Published as a conference paper at ICLR 2026
C.3 E XPLANATION OF B ELOW-R ANDOM V ERIFICATION P ERFORMANCE ON H ARD P ROBLEMS
In Figure 6f, we observe that verifiers achieve balanced accuracy below the random baseline of 0.5 on hard problems from the NL Reasoning domain, a result that needs explanation. This phenomenon can occur in reference-free evaluation when verifiers employ a “solve-and-match” verification strategy, where they attempt to solve the problem independently and then compare their answer with the generator’s response. For NL Reasoning tasks with 3-way or 4-way multiple choice formats, this mechanism can produce below-random performance when verifiers consistently fail to solve hard problems correctly. In such cases, the verifier never correctly identifies true positive responses (TPR = 0) because it always produces wrong answers that don’t match correct generator responses. However, it can still identify some true negatives when both the generator and verifier happen to select the same wrong answer. For three-way choices, the TNR = 0.5. With TPR near zero and TNR remaining positive, the balanced accuracy falls below 0.5. This phenomenon is specific to tasks with limited answer spaces. The affected problems are those in the hard set with d(x) < 0.3, where even strong models achieve very low pass rates. It occurs in NL Reasoning because this domain includes three-way multiple-choice questions from datasets like FOLIO. It does not occur in Mathematics, where responses are open-ended strings, or in Knowledge domains with 10-way multiple choice, where the large answer space dilutes the effect.
C.4 A DDITIONAL R ESULTS OF V ERIFIER G ENERATION C APABILITY Here we present additional results for RQ3 from Section 4.3, providing correlation analysis between verifier generation capability and verification accuracy across the entire problem difficulty range in Figure 14. The results confirm our finding from the main paper that the correlation form varies with problem difficulty: medium problems show strong positive linear relationships, while hard and easy problems exhibit non-linear trends. Below, we provide a clearer analysis of these patterns. Threshold-limited pattern on difficult problems. The threshold-limited regime appears in difficult problems, for example, in Figure 14 (a) in the difficulty range [0.2, 0.3). In this regime, increasing the verifier’s generation capability does not translate into noticeably better verification, which is observed as the saturation pattern. As an example, Qwen3-32B (largest rectangle) has a generation capability of about 0.6, yet its verification performance is nearly identical to Qwen2.5-72B (circle) with a lower generation capability of around 0.4. Both models plateau around a verification score of 0.7. The underlying reason is that on extremely difficult problems, even the verifiers with the best generation capability are not that good (≤ 0.6 pass rate). As a result, pushing generation ability doesn’t yield many gains, as the models still lack the intrinsic ability to “handle” these problems. This implies that reliable verification requires the verifier to reach a sufficiently high level of generation capability. When models fall below this requirement on difficult problems, verification performance plateaus. Transition to a linear pattern on medium problems. When moving to medium-difficulty problems, such as those in Figure 14 (a) range [0.4, 0.5), the relationship becomes more linear. In this regime, Qwen3-32B reaches a higher generation capability (around 0.8), and its verification performance increases accordingly to 0.75. This improvement allows Qwen3-32B to outperform another model with a generation capability of around 0.5 and verification around 0.7, indicating a linear pattern. In this intermediate regime, we see the most linear correlation. These problems are of average difficulty, and models typically demonstrate a baseline level of ability in solving such problems, as measured by generation capability. As such, we find that this regime matches well with regimes analyzed in prior work (Chen et al., 2025c; Tan et al., 2025). Therefore, this region exhibits the most intuitive behavior: Model verification ability tracks directly with its ability to understand (i.e., solve) the problem. It’s important to note that these problems are solvable (i.e., not extremely difficult). Saturated pattern on easy problems. In Figure 14(a) for the difficulty range [0.9, 1.0), Qwen2.5-7B (medium circle) already reaches nearly 1.0 in generation capability, the same as the stronger Qwen3- 32B. Their difference on the x-axis is therefore essentially zero, yet their verification performance differs by about 0.1. This is the saturated pattern.
30
Published as a conference paper at ICLR 2026
The underlying reason is that differences in verification performance come from factors other than the ability to solve the problem itself. One such factor is whether the verifier can detect more subtle or advanced mistakes made by strong generators. As illustrated in the case studies in Figure 33, a model may be able to solve a problem correctly but still fail to recognize certain non-obvious or high-level errors in another model’s reasoning. Thus, perfect generation capability is not sufficient for achieving strong verification performance on easy problems. These findings highlight the need for regime-aware verifier strategies. On hard problems, strong verifiers are unnecessary as performance plateaus regardless of capability. On medium problems, selecting models with better generation capability consistently yields better verification. On easy problems, selecting higher-capability models works well among weak-to-medium verifiers, but strong models with similar capabilities show vastly different verification performance. Thus, optimal selection of strong verifiers requires supplementary benchmarking or alternative evaluation metrics. C.5 A DDITIONAL R ESULTS OF G ENERATOR A NALYSIS IN T EST-T IME S CALING This subsection provides complementary results for the generator analysis presented in Section 5.1, demonstrating the generalizability of our findings across domains and problem difficulties. Figure 15 extends the analysis from the main paper to Knowledge and NL Reasoning domains. Our central finding from RQ4 holds consistently. As Figures 15a to 15c shows, verification gains peak at weak-medium generator strength, enabling these generators to substantially close performance gaps with stronger models. The underlying mechanism driving this phenomenon, identified in RQ2, remains consistent across domains. Figures 15d to 15f shows that, as generator strength increases, TNR decreases sharply while TPR rises only modestly. For the strongest generators, the collapsed TNR limits verification gains as errors become increasingly difficult to detect. This brings high verification gains at weak-medium generator levels. In the main paper, we show results on problems with difficulty range d(x) ∈ [0.7, 0.8) in Figure 7. Here, Figures 16 to 18 report results across the entire difficulty range for three domains, respectively. Figure 19 shows the percentage of performance gap closed by verification for all weak-to-strong generator pairs, computed on all problems within each domain.
31
Published as a conference paper at ICLR 2026
Qwen2.5 Llama3 Mistral 2-4B 24-32B
Qwen3 Gemma2 GPT 7-9B 70B+
Difficulty range: (0.0,0.1) Difficulty range: [0.1,0.2) Difficulty range: [0.2,0.3) Difficulty range: [0.3,0.4) Difficulty range: [0.4,0.5)
0.9
Balanced Accuracy 0.8
0.7
0.6
0.5
Difficulty range: [0.5,0.6) Difficulty range: [0.6,0.7) Difficulty range: [0.7,0.8) Difficulty range: [0.8,0.9) Difficulty range: [0.9,1.0)
0.9
Balanced Accuracy 0.8
0.7
0.6
0.5
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability
(a) Mathematics
1.0
Difficulty range: (0.0,0.1) Difficulty range: [0.1,0.2) Difficulty range: [0.2,0.3) Difficulty range: [0.3,0.4) Difficulty range: [0.4,0.5)
Balanced Accuracy 0.8
0.6
0.4
1.0
Difficulty range: [0.5,0.6) Difficulty range: [0.6,0.7) Difficulty range: [0.7,0.8) Difficulty range: [0.8,0.9) Difficulty range: [0.9,1.0)
Balanced Accuracy 0.8
0.6
0.4
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability
(b) Knowledge
1.0
Difficulty range: (0.0,0.1) Difficulty range: [0.1,0.2) Difficulty range: [0.2,0.3) Difficulty range: [0.3,0.4) Difficulty range: [0.4,0.5)
Balanced Accuracy 0.8
0.6
0.4
1.0
Difficulty range: [0.5,0.6) Difficulty range: [0.6,0.7) Difficulty range: [0.7,0.8) Difficulty range: [0.8,0.9) Difficulty range: [0.9,1.0)
Balanced Accuracy 0.8
0.6
0.4
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability Verifiers' Generation Capability
(c) NL Reasoning
Figure 14: Correlation between verification performance and generation capability across problem difficulty ranges on three domains. Balanced accuracy as a function of verifier generation capability for difficulty ranges from (0.0,0.1) to [0.9,1.0). Performance exhibits three distinct regimes: plateaus on hard problems, strong positive correlation on medium problems, and high variance with saturated capability on easy problems. Marker shapes indicate model family; sizes represent model scale.
32
Published as a conference paper at ICLR 2026
Before Verification After Verification TNR TPR Verification Gain
(a) Pass rate (Mathematics) (b) Pass rate (Knowledge) (c) Pass rate (NL Reasoning)
0.4 1.0 0.6 1.0 0.4 1.0
Verification Gain Verification Gain Verification Gain 0.3 0.8 0.8 0.3 0.8
TPR / TNR TPR / TNR TPR / TNR
0.4
0.6 0.6 0.6
0.2 0.2
0.4 0.4 0.4
0.2
0.1 0.2 0.2 0.1 0.2
0.0 -2B
0.0 0.0 -2B
0.0 0.0G2 -2B
0.0
G2 L-
L-
M- B- 3B
8
8B
9 L-
M B
Q2 -8B
.5-
L- 3
3B
8 L
Q2 -3B
.5-
M- B
L 38B
G2 B-
Q2 27B
.5- G2 B
Q2 -9B
.5- Q2 -8B
.5-
G2 B 7
G2 M- 3B
Q2 24B
.5-
Q3 B
L- B
gp B-4
70
Q3 t-4o 7 G2 G2 7B
-27
Q3 B-
M- 4B
Q B
Q2 3-8B
.5-
Q3 2B24
7
Q3 B
G2 -4B
-2
M- 7B
Q3 B-9
24
-8
L- B
7
-32
Q3 B
Q2 -8B
.5- -32
L- B
70
gp B Q3 0B
Q2 -32B
.5- 72
gp B
Generator Models Generator Models Generator Models
72 B t-4 o t-4 o
(d) Verif. metrics (Mathematics) (e) Verif. metrics (Knowledge) (f) Verif. metrics (NL Reasoning)
Figure 15: TTS performance before and after verification when sweeping generator strength. (a- c) Pass rate before (blue) and after (orange) adding a fixed verifier (GPT-4o), across generators ordered from weaker (left) to stronger (right) by generation capability. (d-f) Bar chart shows the verification gain ∆p̂V (left y-axis) for each generator. Lines show the verifier’s TNR and TPR on the same datasets (right y-axis). Results are reported on problems with difficulty in the range [0.7, 0.8) for three domains. Problem counts across domains: 181 (Mathematics), 154 (Knowledge), 97 (NL Reasoning). Problem Difficulty (Pass Rate) (0.0,0.1) Problem Difficulty (Pass Rate) (0.0,0.1) Problem Difficulty (Pass Rate) [0.1,0.2) Problem Difficulty (Pass Rate) [0.1,0.2) 1.0 1.0 1.0 1.0 0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0G2 -27 M- B L- 8B 0.0 -27 M- B 8B 0.0 M- -2B L- 8B 0.0 M- B L--2 8B L Q2 -8B G2.5- G2 B-23B 3B L- L- Q2 8B .5- G2 B 3B 3 -2B G2 B G2 3B Q2 B -9B L- -278 G2 B L- G2 3B - Q2 27B 8 -9B Q2 -9B .5- M- 7B G2 Q2 -9B .5- M 7B G2 .5- M- B Q2 24B 3 G2 .5- M- 3B Q2 24B Q2 4B .5- Q3 B 2 72 L- B 70 gp B t- Q3 4o -4 Q3 -8B G2 Q2 -24B .5- 72 L- B Q3 B70 gp B t- Q3 4o -4 .5 Q2 -7B .5- Q3 B72 gp B L- o Q3 Bt-4 70 -4 .5 Q2 -7B .5- gp B L- o Q3 B Q3 B 72 t-4 70 -4 Q3 -8B Q3 8B- Q3 -8B Generator Models -32 Generator Models Generator Models Generator Models B -32 B -32 B -32 B
Problem Difficulty (Pass Rate) [0.2,0.3) Problem Difficulty (Pass Rate) [0.2,0.3) Problem Difficulty (Pass Rate) [0.3,0.4) Problem Difficulty (Pass Rate) [0.3,0.4)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 -2B L- L- 8B 3 0.0 L- L--2B 8B 3 0.0 L- G2 B -2B 3 -9B 0.0 L- G2 B -2B 3 -9B G2 B M- G2 B Q2 B -9B -278 G2 B M- B G2 8B - Q2 27B .5--9 M-L- G2 8B - Q2 27B 8B L- M- B G2 8B - Q2 27B .5- 8 G2 .5- M- B Q2 24B .5- L- B g B Q2 pt-4o .5- 70 72 3 7 G2 M- 3B Q2 24B .5- L- B g B Q2 pt-4o .5- 7 70 72 G2 .5- M- B Q2 24B .5- L- B gp B Q3 o Q2 -4B 703 7 t-4 G2 M- B Q2 24B .5- L- B gp B Q 4o Q2 3-4B 3 7 70 t- Q3 B Q3 B Q3 8B-4 - Q3 B Q3 B Q3 -8B-4 .5- Q3 B Q3 8B 72 - .5- Q3 B Q3 -8B72 Generator Models Generator Models Generator Models Generator Models -32 B -32 B -32 B -32 B
Problem Difficulty (Pass Rate) [0.4,0.5) Problem Difficulty (Pass Rate) [0.4,0.5) Problem Difficulty (Pass Rate) [0.5,0.6) Problem Difficulty (Pass Rate) [0.5,0.6)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 L- G2 B -2B 3 -9B 0.0 L- G2 B -2B 3 -9B 0.0 L- G2 B -2B 3 -9B 0.0 L- G2 B -2B 3 -9B M- G2 BL- - Q2 27B 8B 8 L- M- B G2 8B - Q2 27B .5- 8 M-L- G2 8B - Q2 27B 8B L- M- B G2 8B - Q2 27B .5- 8 G2 .5- M- B Q2 24B .5- L- B gp B Q o Q2 3-4B703 7 t-4 G2 M- B Q2 24B .5- L- B gp B Q 4o Q2 3-4B 3 7 70 t- G2 .5- M- B Q2 24B .5- L- B Q3 B gp 3 7 70 -4B t-4 G2 M- B Q2 24B .5- L- B Q3 B gp B 3 7 70 -4 t-4 .5- Q3 B Q3 8B 72 - .5- Q3 B Q3 -8B72 Q o Q2 3-8B .5- Q3 2B 7 Q o Q2 3-8B .5- Q3 2B 7 Generator Models Generator Models Generator Models Generator Models -32 B -32 B -32 B -32 B
Problem Difficulty (Pass Rate) [0.6,0.7) Problem Difficulty (Pass Rate) [0.6,0.7) Problem Difficulty (Pass Rate) [0.7,0.8) Problem Difficulty (Pass Rate) [0.7,0.8)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 L- G2 B -2B 3 -9B 0.0 L- G2 B -2B 3 -9B 0.0 -2B L- L- 3B 8B 0.0 L- L--2B 3B 8 M- G2 BL- - Q2 27B 8B 8 L- M- B G2 8B - Q2 27B .5- 8 M- G2 B G2 9B Q2 B - -278 M- B G2 B G2 -9B - Q2 27B .5- 8 G2 .5- M- B Q2 24B .5- L- B Q3 B gp B70 -8 3 7 t-4 G2 M- B Q2 24B .5- L- B Q3 B gp B 3 7 70 -8 t-4 G2 .5- M- B Q2 24B .5- Q3 B L- -4B 70 gp B t 3 7 G2 M- 3B Q2 24B .5- Q3 B L- B gp B 7 -4 70 t-4 Q3 o Q3 4B Q2 -32B- Q3 o Q3 -4B Q2 -32B .5- Q3 -4o -32 Q3 B Q2 -8B Q3 o Q B-32 Q2 3-8B .5- Generator Models Generator Models Generator Models Generator Models .5- 72 B 72 B .5- 72 B 72 B
Problem Difficulty (Pass Rate) [0.8,0.9) Problem Difficulty (Pass Rate) [0.8,0.9) Problem Difficulty (Pass Rate) [0.9,1.0) Problem Difficulty (Pass Rate) [0.9,1.0)
1.0 1.0 1.0
1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 L--2B 3B 0.0 L--2B 3B 0.0 L- L--2B 3B 8 0.0 L--2B 3B M- Q2 BL- G2 B .5- G2 3B - 8B 8 9 L- M- B G2 B Q2 -9B .5 G2 -3B 8 8 M- B Q2 8B .5- G2 B G2 9B - 3 L- M- B Q2 8B .5- G2 B G2 -9B 8 3 G2 - Q2 27B .5- M- B L- B Q3 B Q3 B 24 70 -47 G2 - Q2 27B .5- M- 7B L- B Q3 B Q3 B 24 70 -4 G2 - Q2 27B .5- Q3 B Q3 B M- 4B Q3 4B -8 - 2 -32 7 G2 - Q2 27B .5- Q3 B Q3 B M- 4B Q3 4B -8 - 2 -32 7 Q3 8B - -32 gp B Q2 t-4o Q3 -8B -32 gp B Q2 t-4o L- B gp B Q2 t-4o .5- 70 L- B gp B Q2 t-4o70 Generator Models .5- .5- Generator Models Generator Models Generator Models .5- 72 B 72 B 72 B 72 B
Figure 16: Verification-augmented TTS performance across the full range of problem difficulties, shown here for the Mathematics domain. Each pair of figure corresponds to a different difficulty interval (measured by pass rate d(x)), with the left panel showing pass rates before (blue) and after (orange) verification, and the right panel showing verification gain ∆p̂V (bars) alongside the verifier’s TNR (green) and TPR (purple). Compared to Figure 7, which focused only on problems with d(x) ∈ [0.7, 0.8), this includes the entire difficulty range.
33
Published as a conference paper at ICLR 2026
Problem Difficulty (Pass Rate) (0.0,0.1) Problem Difficulty (Pass Rate) (0.0,0.1) Problem Difficulty (Pass Rate) [0.1,0.2) Problem Difficulty (Pass Rate) [0.1,0.2)
1.0 1.0 1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0G2 Q3 B-9 G2 -4B Q2 -27B 0.0G2 Q3 B-9 G2 -4B 0.0 L Q2 -8B .5- G2 B-93 0.0 L Q2 -8B .5- G2 B 3 .5- G2 B72 -2 L- B Q2 70B .5- Q2 -27B .5- G2 B72 -2 L- B Q2 70B G2 B M- Q3 B Q3 B-2B -88 G2 B-2 M- B Q3 B-9 -88 M- 7B 24 L- L- 3B 8B .5- M- 7B 24 L- L- B 8B Q2 -4B .5 G2 -7B Q2 -27B .5- 7 M- 2B Q3 B Q2 -4B .5- G2 B Q2 -27B .5- 7 Q3 B M.5-8B Q2 -8B Q3 -3B -32 Q3 B M.5 Q3 -3B 3 -8B Q2 -8B 24 L- L- B 70 gp B t 3 B 7 M- 2B 24 L- L- B 70 gp B 3 B gp B -32 Q3 -4o t Q3 -4o Generator Models Generator Models gp B -32 Generator Models Generator Models t-4 o t-4 o B -32 B
Problem Difficulty (Pass Rate) [0.2,0.3) Problem Difficulty (Pass Rate) [0.2,0.3) Problem Difficulty (Pass Rate) [0.3,0.4) Problem Difficulty (Pass Rate) [0.3,0.4)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 M-L- G2 B-2B 8B 3 0.0 G2 B-2 M- B L- 8B 3 0.0G2 G2 B L--2 -9B 3B 0.0G2 G2 B L--2 -9B 3 G2 9B Q2 8B- -27 L- B G2 -9B -27 L Q2 -8B .5 B M-L Q2 -8B .5- G2 B 38B M- B L- Q2 8B .5- G2 3B 8B G2 .5 Q2 -3B Q3.5- M- B Q B Q2 3-8B .5- -4 24 72 7B G2 Q2 -3B .5- Q3 B Q B .5- - M- 4B 24 Q2 3-8B 72 7 - Q2 27B .5- Q3 B M- B Q3 B L B Q2 -70B 7 -4 24 -8 - Q2 27B .5- Q3 B M- 4B Q3 B L B Q2 -70B- 24 -87 L- B Q3 0B7 -32 gp B L- B Q3 0B7 -32 gp B .5- Q3 2B 7 -32 gp B .5- Q3 2B 7 -32 gp B Generator Models Generator Models Generator Models Generator Models t-4 o t-4 o t-4 o t-4 o
Problem Difficulty (Pass Rate) [0.4,0.5) Problem Difficulty (Pass Rate) [0.4,0.5) Problem Difficulty (Pass Rate) [0.5,0.6) Problem Difficulty (Pass Rate) [0.5,0.6)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 M-L--2B 8B 3 0.0 M- B L--28B 3 0.0 M-L--2B 3B 8 0.0 L- M- B-2B 3 8 G2 B Q2 -9B .5- L- G2 8B 3B G2 B Q2 -9B .5- L G2 -8B 3B G2 B Q2 -9B .5- L- G2 8B 3B G2 B Q2 -9B .5- L G2 -8B 3B G2Q2 27B M- B Q3 B Q2 -8B .5- - .5- Q3 B -4 24 72 7 G2 Q2 27B .5- Q B - Q3 B M- 4B Q2 3-8B .5- - 24 72 7 G2 - Q2 27B .5- Q3 B M- B Q3 B L- B 7 -4 24 -8 70 G2 - Q2 27B .5- Q3 B M- 4B Q3 B L- B- 24 -8 70 7 L- B gp B Q3 -4o70 t L- B gp B Q3 -4o70 t g B Q2 pt-4o .5- Q3 2B 7 g B Q2 pt-4o .5- Q3 2B 7 Generator Models Generator Models Generator Models Generator Models -32 B -32 B -32 B -32 B
Problem Difficulty (Pass Rate) [0.6,0.7) Problem Difficulty (Pass Rate) [0.6,0.7) Problem Difficulty (Pass Rate) [0.7,0.8) Problem Difficulty (Pass Rate) [0.7,0.8)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 M -2B L- Q2 -8B 3B 0.0 -2B L- M B Q2 -8B 3 0.0 M -2B L- Q2 -8B 3B 0.0 -2B L- M B Q2 -8B 3 .5- L- G2 B Q2 -9B 3B 8 .5- L- G2 B Q2 -9B .5 3B 8 .5- L- G2 B Q2 -9B .5- 3B 8 .5- L- G2 B Q2 -9B .5 3B 8 G2 .5- G2 7B -27 Q3 B M- B Q3 B-4 24 -8 L- B 70 G2 G2 -7B -27 Q3 B Q3 B- M- 4B 24 -8 L- B 70 G2G2 7B -27 Q3 B M- B Q3 B Q2 -8B .5- Q3 2B-4 24 7 G2 G2 -7B Q3 B M- 4B Q B-27 - 24 Q2 3-8B .5- Q3 B 72 g B Q2 pt-4o .5- Q3 2B7 g B Q2 pt-4o .5- Q3 2B7 -32 L- B 70 gp B -32 L- B 70 gp B Generator Models Generator Models Generator Models Generator Models -32 B -32 B t-4 o t-4 o
Problem Difficulty (Pass Rate) [0.9,1.0)
Problem Difficulty (Pass Rate) [0.8,0.9) Problem Difficulty (Pass Rate) [0.8,0.9) Problem Difficulty (Pass Rate) [0.9,1.0)
1.0 1.0 1.0
1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate 0.6 0.6 0.6 0.6 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.0 -2B L- 3B 0.0 -2B L- 3 0.0 -2B L- M B Q2 -8B 3 0.0 -2B L- 3 M Q2 -8B .5- L- G2 B 3B 8 M B Q2 -8B .5- L- G2 B 3B 8 .5- L Q2 -8B .5- G2 B 3B 7 M B Q2 -8B .5- L- Q2 8B 3B G2 Q3 B G2 7B-9 Q2 -4B .5- -27 Q3 B M- B-8 24 G2 Q3 B G2 7B Q3 B -9 Q2 -4B .5- -27 - M- 8B 24 G2G2 -9B M- -27 Q3 B Q3 B-4B 24 -8 gp B t Q3 -4o G2 .5- G2 B G2 -9B -27 Q3 B- M- 4B Q3 B24 -8 7 gp B t Q3 -4o -32 L B gp B t Q3 -4o -32 L B Q2 -70B -32 L- B Q2 70B .5- gp B Q3 o t-4 -32 L- B Generator Models Q2 -70B .5- 72 Q2 70B .5- Generator Models Generator Models Generator Models .5- 72 B 72 B B 72 B
Figure 17: Verification-augmented TTS performance across the full range of problem difficulties, shown here for the Knowledge domain. Each pair of figure corresponds to a different difficulty interval (measured by pass rate d(x)), with the left panel showing pass rates before (blue) and after (orange) verification, and the right panel showing verification gain ∆p̂V (bars) alongside the verifier’s TNR (green) and TPR (purple). Compared to Figure 7, which focused only on problems with d(x) ∈ [0.7, 0.8), this includes the entire difficulty range.
34
Published as a conference paper at ICLR 2026
Problem Difficulty (Pass Rate) (0.0,0.1) Problem Difficulty (Pass Rate) (0.0,0.1) Problem Difficulty (Pass Rate) [0.1,0.2) Problem Difficulty (Pass Rate) [0.1,0.2)
1.0 1.0 1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate
0.6 0.6 0.6 0.6
0.4 0.4 0.4 0.4
0.2 0.2 0.2 0.2
0.0
G2
Q2 -27B
.5-
G2 B72
-9
0.0
G2
Q2 -27B
.5- 72
0.0
.5-
G2 2B7
-27
Q3 B
G2 B-4
0.0
.5-
G2 2B7
-27
Q3 B
Q3 B
.5-
Q3 B-
M- 4B
Q2 24B 7 G2 B
Q3 B-9
-
M- 4B
Q2 24B
.5- 7 Q3 B-9
Q2 -8B
.5-
M- 7B
24
L- B G2 B
Q3 B-4
-9
Q2 -8B
.5-
M- 7B
24
Q2 -8B
.5-
M- B
G2 B-23
8 Q3 B
Q2 -8B
.5-
M- B
G2 B 3
8 Q2 8B70
M- B
gp B 8
t-4
L-
.5- o L- B
70
M- B
gp B
L 8
t-4
Q2 -8B o
L- B
L70
L-
gp B 3
t-4 B
o -2
L- B
70
L-
gp B 3
t-4 B
o Q2 L-
Q3 3B
-32
G2 B 3B
-2B Q2 .5-
L-
Q3 3B
-32
G2 B 3B
-2B
Q3 -8B L
Q3 -8B
Generator Models Generator Models
-32
Generator Models Generator Models
B -32 B
Problem Difficulty (Pass Rate) [0.2,0.3) Problem Difficulty (Pass Rate) [0.2,0.3) Problem Difficulty (Pass Rate) [0.3,0.4) Problem Difficulty (Pass Rate) [0.3,0.4)
1.0 1.0 1.0
1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate
0.6 0.6 0.6 0.6
0.4 0.4 0.4 0.4
0.2 0.2 0.2 0.2
0.0 M B
G2 -8B
-
Q2 27B-9 0.0 M B
G2 -8B
- -9 0.0 M -9B
L- 3B
0.0 -9B
L-
M B 3
.5-
L
Q2 -3B
.5- 3B
7B Q2 27B
.5-
L
Q2 -3B
.5- 3B
7B Q2 -8B
.5-
Q2 3B
.5-
L- 7B
8 Q2 -8B
.5-
Q2 3B
.5-
L- 7B
8
G2 L-
G2 B
Q3 B
Q3
M- 8B
L B
Q2 -70B
-28
-4B
-
24 G2 L-
G2 B
Q3 B
Q3 B
M- 8B
L B
-2
-4
-
24
8 G2 G2 B
G2 2B
-27
Q3 B
M- B
Q3 B
-
-4
24
-8
L- B G2 G2 B
G2 -2B
-27
Q3 B
M- 4B
Q3 B-
24
-8
L- B
.5-
Q3 B 72
-32
gp B Q2 -70B
.5-
Q3 2B 7
-32 g B70
Q2 pt-4o
.5-
Q3 2B7 g B70
Q2 pt-4o
.5-
Q3 2B7
Generator Models
t-4 gp B
Generator Models Generator Models Generator Models
o t-4 o -32 B -32 B
Problem Difficulty (Pass Rate) [0.4,0.5) Problem Difficulty (Pass Rate) [0.4,0.5) Problem Difficulty (Pass Rate) [0.5,0.6) Problem Difficulty (Pass Rate) [0.5,0.6)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate
0.6 0.6 0.6 0.6
0.4 0.4 0.4 0.4
0.2 0.2 0.2 0.2
0.0 M L-
Q2 -8B
.5- 3B
3
0.0 L-
M B
Q2 -8B
.5- 3
3
0.0 L-
G2 B
L 3
-2B
Q2 -8B
0.0 L-
G2 B
L 3
-2B
Q2 -8B
G2 B
G2 B
L-9
-2B
Q2 -8B
.5- G2 B
G2 B
L-9
-2B
Q2 -8B
.5- M.5-
Q2 -8B
.5-
G2 B 3B
7 .5-
M B
Q2 -8B
.5-
G2 B 3
7
G2 7B
-27
M- B
Q3 B
Q3 B24
-4 G2 7B
-27
M- B
Q3 B
Q3 B24
-4 G2 9B
-27
Q3 B
M- B
Q B
-
-4
24 G2 -9B
-27
Q3 B-
M- 4B
Q B 24
.5- -8
L- B
Q3 2B70
gp B
Q2 t-4o
7 -8
L- B
Q3 2B70
gp B
Q2 t-4o
.5- 7 Q2 3-8B
.5- 72
gp B
t
Q3 -4o
-32 Q2 3-8B
.5- 72
gp B
t
Q3 -4o
-32
L- B
L- B
Generator Models Generator Models Generator Models Generator Models
-32 B -32 B 70 B 70 B
Problem Difficulty (Pass Rate) [0.6,0.7) Problem Difficulty (Pass Rate) [0.6,0.7) Problem Difficulty (Pass Rate) [0.7,0.8) Problem Difficulty (Pass Rate) [0.7,0.8)
1.0 1.0
1.0 1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate
0.6 0.6 0.6 0.6
0.4 0.4 0.4 0.4
0.2 0.2 0.2 0.2
0.0 M- -2B
L- 3B
8B
0.0 -2B
L-
M- B 3
8B
0.0G2 L-2B
Q2 -3B
.5-
0.0 L-2B
Q2 -3B
.5- 3
L
Q2 -8B
G2.5-
Q2 -9B 3B L-
Q2 8B
.5-
G2 B
Q2 -9B
.5 3 M-L
Q2 -8B
.5-
G2 B 3B
8B
7 M- B
L
Q2 -8B
.5-
G2 B 8B
7
G2 .5-
G2 7B
-27
Q3 B
M- B
Q3 B-4
24
-8
L- B
7 G2 G2 -7B
-27
Q3 B
M- 4B
Q3 B-
24
-8
L- B
70
Q3 B
G2 4B
M- B
Q3 B-9
-
-27
24
-8
L- B
7 G2 Q3
G2 -4B
-2
M- 7B
Q3 B
L- B
Q3 0B
-9B
24
-8
7
Q3 0B
-32
gp B
Q2 t-4o Q3 B-32
gp B
Q2 t-4o
.5- Q3 0B
Q2 -32B
.5- 72
gp B Q2 -32B
.5- 72
gp B
Generator Models Generator Models Generator Models Generator Models
.5- 72 B 72 B t-4 o t-4 o
Problem Difficulty (Pass Rate) [0.9,1.0)
Problem Difficulty (Pass Rate) [0.8,0.9) Problem Difficulty (Pass Rate) [0.8,0.9) Problem Difficulty (Pass Rate) [0.9,1.0)
1.0 1.0 1.0
1.0
0.8 0.8 0.8 0.8
Pass Rate Pass Rate Pass Rate Pass Rate
0.6 0.6 0.6 0.6
0.4 0.4 0.4 0.4
0.2 0.2 0.2 0.2
0.0 L-
G2 B 3
-2B
0.0 L-
G2 B-23
0.0 L-
G2 B
M B-2
Q2 -8B 3 0.0 L-
G2 B-23
M
Q2 -8B
.5-
L
Q2 -8B
.5- 3B M B
Q2 -8B
.5-
L
Q2 -8B
.5- 3B .5-
L-
Q3 B
Q2 -4B
.5-3B
8 M B
Q2 -8B
.5-
L-
Q3 B 3B
8
G2 B
Q3 B
G2 4B-9
-
-27
L- B
70
7 G2 B
Q3
L- B
7
7
-9B
G2 -4B
-27 M- 7B
G2 B24
G2 -9B
-27
Q3 B Q2 -4B
.5-
M- 7B
G2 B24
G2 -9B
-27
M- B
Q3 B
.5- 24
-8
gp B
Q2 t-4o M- 0B
Q3 B24
Q2 t-4o
.5- -8
gp B L- B
Q3 0B
Q2 t-4-8
7
-32
gp B
.5- o Q3 B
L- B
Q3 0B-8
7
-32
gp B
Generator Models
Q3 2B7 Q3 2B7 72 Q2 t-4o
.5-
Generator Models Generator Models Generator Models
-32 B -32 B B 72 B
Figure 18: Verification-augmented TTS performance across the full range of problem difficulties, shown here for the NL reasoning domain. Each pair of figure corresponds to a different difficulty interval (measured by pass rate d(x)), with the left panel showing pass rates before (blue) and after (orange) verification, and the right panel showing verification gain ∆p̂V (bars) alongside the verifier’s TNR (green) and TPR (purple). Percentage of Gap Closed
-100% -75% -50% -25% 0% 25% 50% 75% 100%
G2-2B 4% 17% 28% 28% 32% 26% 28% 28% 26% 28% 27% 24% 23% 23% G2-2B -5% 13% 26% 28% 41% 40% 43% 41% 38% 41% 39% 41% 40% 37% L3-3B -61% 42% 93% 47% 63% 65% 51% 49% 51% 49% 47% 45% 46% 42%
L3-3B 49% 70% 65% 61% 46% 41% 41% 36% 38% 36% 31% 30% 30% L3-3B 86% 76% 63% 82% 63% 67% 60% 53% 56% 51% 54% 51% 48% G2-2B 48% 100% 51% 66% 68% 53% 51% 53% 50% 49% 47% 48% 43%
Generators Ranked By Generation Capability Generators Ranked By Generation Capability Generators Ranked By Generation Capability L3-8B 130% 95% 70% 44% 38% 39% 32% 36% 34% 28% 26% 27% M-8B 69% 55% 81% 59% 64% 57% 50% 53% 48% 51% 49% 46% M-8B 509% 62% 79% 83% 56% 53% 55% 51% 49% 46% 47% 42%
M-8B 21% 48% 20% 27% 29% 24% 29% 26% 20% 19% 20% Q2.5-3B 40% 90% 57% 63% 54% 47% 51% 46% 49% 47% 43% Q2.5-3B -240% 33% 40% 24% 25% 30% 29% 29% 27% 29% 24%
G2-9B 53% 20% 27% 29% 24% 29% 27% 20% 19% 20% L3-8B 186% 62% 71% 58% 48% 53% 47% 50% 48% 44% L3-8B 84% 88% 54% 51% 53% 50% 48% 44% 46% 40%
G2-27B -53% 17% 21% 16% 23% 21% 14% 12% 15% G2-9B 35% 46% 41% 35% 41% 36% 40% 38% 34% G2-9B 130% 8% 13% 28% 25% 27% 24% 27% 19%
Q2.5-3B 32% 34% 25% 32% 29% 20% 18% 20% G2-27B 269% 49% 35% 44% 37% 43% 40% 34% Q2.5-7B -14% -2% 18% 17% 21% 18% 22% 14%
M-24B 42% 15% 32% 26% 10% 8% 13% Q2.5-7B 33% 25% 36% 31% 37% 35% 29% G2-27B 30% 51% 40% 38% 32% 36% 24%
Q2.5-7B 3% 29% 22% 4% 2% 10% Q3-4B 14% 39% 29% 39% 36% 27% Q3-4B 65% 44% 39% 32% 37% 23%
L3-70B 83% 43% 4% 1% 12% M-24B 111% 40% 55% 46% 33% M-24B 7% 25% 17% 26% 9%
GPT-4o -4% -57% -42% -6% Q3-8B 17% 38% 33% 19% Q3-8B 33% 20% 32% 10%
Q2.5-72B -146% -70% -6% L3-70B 715% 57% 22% L3-70B -14% 29% -14%
Q3-4B -11% 20% Q2.5-72B 26% -8% Q2.5-72B 81% -14%
Q3-8B 30% GPT-4o -492% GPT-4o -115%
Q3-32B Q3-32B Q3-32B
L3
L3
M- -2B
-3B
-8B
8B L3
M-
Q2 B -2B
-3B
.5- 8
3B L3
G2
M-
Q2 B -3B
-2B
.5- 8
3B
G2
G2 B
Q2 B -9
-27 L3
G2
G2 B -8B
-9 L3
G2
Q2 B -8B
-9
G2 M-
Q2 B
L3
GP
Q2 o
.5-
.5-
24
.5-
-70
T-4
72
3B
7B
B G2 Q2 B
Q3
M-
Q3
L3
-27
.5- 7B
-4B
24
-8B
-7 B G2 B
Q3
M-
Q3
L3
.5-
-27
-4B
247
-8B
-70
BB
Q3
Q3
Q3 B -4B
-8
-32 B
B Q2 0B
GP.5-
Q3 o 72
T-4
-32 B
B Q2 B
GP.5-
Q3 o 72
T-4
-32 BB
Generators Ranked By Generation Capability Generators Ranked By Generation Capability Generators Ranked By Generation Capability
(a) Mathematics (b) Knowledge (c) NL Reasoning
Figure 19: Percentage of TTS performance gap between weak and strong generators closed by verification. Each heatmap shows the fraction of the performance gap between a weaker generator (x-axis) and a stronger generator (y-axis) that is closed by verification with a fixed verifier GPT-4o. Green cells indicate a larger gap closure, meaning the weaker model approaches the stronger one after verification. A value greater than 100% means that the originally weaker model performs better with verifier augmentation. Purple cells indicate negative values where verification increases the gap.
35
Published as a conference paper at ICLR 2026
C.6 A DDITIONAL R ESULTS OF V ERIFIER A NALYSIS IN T EST-T IME S CALING
Here we provide complementary results to the verifier analysis in Section 5.2. Figure 20 presents two other domains’ results. Figure 21 presents additional metrics in the same setup for complete analysis, including balanced accuracy and verification gains. We can see that our findings from Mathematics generalize to other domains. Figures 20a to 20c shows that weak verifiers can approximate strong verifier performance in TTS, at the extremes of problem difficulty or responses generated by strong generators. While on these regimes, we show that both verifiers provide limited verification gain in Figures 21a to 21c. Notably, in the NL Reasoning domain (Figure 20i), on the strongest generator, both verifiers’ TNR fall below 0.5, and the weak verifier’s TNR exceeds that of the strong verifier. Despite this TNR inversion, the overall verification performance gap (shown as balanced accuracy in Figure 21f) remains narrow, with the strong verifier maintaining a slight advantage due to its superior TPR. Generators Verifiers Weak (Llama3.2-3B) Strong (Qwen2.5-72B) Strong (GPT-4o) Weak (Qwen2.5-7B) Medium (Llama3.1-8B) 0.175 0.150 0.20 0.20
Verification Gain Gap Verification Gain Gap Verification Gain Gap 0.125 0.15 0.15 0.100 0.10 0.075 0.10 0.05 0.050 0.05 0.025 0.00 0.000 0.05 0.00 (0- [0. 1) 1-00. [0. .2) 2-0 (0- [0. 1) 1-00. [0. .2) 2-0 (0- [0. 1) 1-00. [0. .2) 2-0 [0. .3) 3-0 [0. .4) 4-0 [0. .5) 5-0 [0. .3) 3-0 [0. .4) 4-0 [0. .5) 5-0 [0. .3) 3-0 [0. .4) 4-0 [0. .5) 5-0 [0. .6) 6-0 [0. .7) 7-0 [0. .8) 8-0 [0. .6) 6-0 [0. .7) 7-0 [0. .8) 8-0 [0. .6) 6-0 [0. .7) 7-0 [0. .8) 8-0 [0. .9) 9-1 [0. .9) 9-1 [0. .9) 9-1 Problem Difficulty .0) .0) Problem Difficulty Problem Difficulty .0)
(a) Verif. Gain Gap (Mathematics) (b) Verif. Gain Gap (Knowledge) (c) Verif. Gain Gap (NL Reasoning) 1.0 1.0 1.0
True Positive Rate True Positive Rate True Positive Rate
0.8 0.8 0.8
0.6 0.6 0.6
0.4 0.4
0.4
(0-0.
[0. 1)
1-0
[0. 2)
2-0 .
. (0-0.
[0. 1)
1-0
[0. 2)
2-0 .
. (0-0.
[0. 1)
1-0
[0. 2)
2-0 .
.
[0. 3)
3-0
[0. 4)
4-0
[0. 5)
5-0 .
. [0. 3)
3-0
[0. 4)
4-0
[0. 5)
5-0 .
. [0. 3)
3-0
[0. 4)
4-0
[0. 5)
5-0 .
.
[0. 6)
6-0
[0. 7)
7-0
[0. 8)
8-0 .
.
. [0. 6)
6-0
[0. 7)
7-0
[0. 8)
8-0 .
.
. [0. 6)
6-0
[0. 7)
7-0
[0. 8)
8-0 .
.
.
[0. 9)
9-1 . [0. 9)
9-1 . [0. 9)
9-1 .
Problem Difficulty Problem Difficulty Problem Difficulty
.0) .0) .0)
(d) TPR (Mathematics) (e) TPR (Knowledge) (f) TPR (NL Reasoning)
0.8 0.8
True Negative Rate True Negative Rate True Negative Rate
0.8
0.6 0.6
0.6
0.4 0.4
0.4
0.2 0.2
0.1 1-0
[0. 2)
2-0
[0. 3)
3-0 .
. ) 0.1 1-0
[0. 2)
2-0
[0. 3)
3-0 .
. ) 0.
[0. 1)
1-0
[0. 2)
2-0
[0. 3) .
.
[0. 4)
4-0
[0. 5) .
. [0. 4)
4-0
[0. 5) .
. 3-0
[0. 4)
4-0
[0. 5) .
.
(0- 5-0
[0. 6)
[0.6-0
[0. 7)
7-0
[0. 8)
8-0
[0. 9)
.
.
.
. (0- 5-0
[0. 6)
[0.6-0
[0. 7)
7-0
[0. 8)
8-0
[0. 9)
.
.
.
. (0- 5-0
[0. 6)
6-0
[0. 7)
7-0
[0. 8)
8-0
[0. 9)
.
.
.
.
9-1 .0) 9-1 .0) 9-1
Problem Difficulty Problem Difficulty Problem Difficulty
.0)
(g) TNR (Mathematics) (h) TNR (Knowledge) (i) TNR (NL Reasoning)
Figure 20: Analyzing verification gain gaps and TPR/TNR between verifiers under varying problem difficulty and generator strength. The x-axis shows problem difficulty measured relative to each generator. Shaded regions visualize the difference in metrics between verifiers for each generator. (a-c) Verification gain gap between strong and weak verifiers. (d-f) TPR increases as problems become easier for all generator-verifier combinations. (g-i) TNR decreases as generators become stronger, with TNR gap narrowing.
36
Published as a conference paper at ICLR 2026
Generators Verifiers
Weak (Llama3.2-3B) Strong (Qwen2.5-72B) Strong (GPT-4o) Weak (Qwen2.5-7B)
Medium (Llama3.1-8B)
0.35 0.30
0.30 0.3 0.25
Verification Gain Verification Gain Verification Gain
0.25 0.20
0.20 0.2 0.15
0.15 0.10
0.10 0.1 0.05
0.05 0.00
0.0
0.00 0.05
(0-
[0. 1)
1-00.
[0. .2)
2-0 0.
[0. 1)
1-0
[0. .2)
2-0
[0. .3) (0-
[0. 1)
1-00.
[0. .2)
2-0
[0. .3)
3-0
[0. .4)
4-0 3-0
[0. .4)
4-0
[0. .5) [0. .3)
3-0
[0. .4)
4-0
[0. .5)
5-0
[0. .6)
6-0
[0. .7)
7-0
[0. .8)
8-0 (0- 5-0
[0. .6)
6-0
[0. .7)
7-0
[0. .8)
8-0
[0. .9)
[0. .5)
5-0
[0. .6)
6-0
[0. .7)
7-0
[0. .8)
8-0
[0. .9)
9-1 9-1 .0) [0. .9)
9-1
Problem Difficulty Problem Difficulty
.0) .0)
Problem Difficulty
(a) Verif. Gain (Mathematics) (b) Verif. Gain (Knowledge) (c) Verif. Gain (NL Reasoning)
0.9
Balanced Accuracy Balanced Accuracy Balanced Accuracy
0.8 0.8 0.8
0.7
0.6 0.6
0.6
0.5 0.4
0.1 ) 1-0
[0. 2)
2-0
[0. 3)
3-0 .
.
. 0.1 1-0
[0. 2)
2-0
[0. 3)
3-0 .
. ) 0.
[0. 1)
1-0
[0. 2)
2-0
[0. 3) .
.
[0. 4)
4-0
[0. 5)
5-0 . [0. 4)
4-0
[0. 5) .
. 3-0
[0. 4)
4-0
[0. 5) .
.
(0- [0. 6)
[0. 7)
[0.
6-0
7-0
[0. 8)
8-0
[0. 9)
.
.
.
. (0- 5-0
[0. 6)
6-0
[0. 7)
[0.7-0
[0. 8)
8-0
[0. 9)
.
.
.
. (0- 5-0
[0. 6)
6-0
[0. 7)
7-0
[0. 8)
8-0
[0. 9)
.
.
.
.
9-1 .0) 9-1 .0) 9-1
Problem Difficulty Problem Difficulty Problem Difficulty
.0)
(d) Accbal (Mathematics) (e) Accbal (Knowledge) (f) Accbal (NL Reasoning)
Figure 21: Analyzing verification gains and Accbal between verifiers under varying problem diffi- culty and generator strength. The x-axis shows problem difficulty measured relative to each genera- tor. Shaded regions visualize the difference in metrics between verifiers for each generator. (a-c) Verifi- cation gain for both strong (solid lines) and weak (dashed lines) verifiers across three generators: weak, medium, and strong. (d-f) Balanced accuracy for the same verifier-generator combinations. The accu- racy gap (shaded regions) between verifiers is smallest on the hard problems (left side of the x-axis).
D E XTENDED S TUDIES AND A BLATIONS D.1 E STIMATION OF P ROBLEM D IFFICULTY
We adopt the uncertainty-based estimator of problem difficulty from Lee et al. (2025). The idea is that a problem is more difficult when a model produces highly diverse or inconsistent answers across repeated stochastic samples. For a given problem, we sample a generator model K times and collect the final answers: {a(r1 ), a(r2 ), ..., a(rk )}. Let the set of unique answers be {u1 , ..., uM }, where M ≤ K. Let nj denote the number of times answer uj appears among the K samples. The corresponding empirical probabilities are: nj pj = , j = 1, . . . , M. (2) K We compute the Shannon entropy of the empirical answer distribution: M X H=− pj log pj (3) j=1 The maximum possible entropy occurs when all K samples produce distinct answers, giving Hmax = log K. We define the normalized difficulty score as H difficulty(x) = (4) log K This value lies in [0, 1], taking 0 when all samples agree (easy problems) and 1 when all samples differ (hard problems), and serves as an uncertainty-based estimate of the intrinsic difficulty of each problem. For implementation, we draw 8 solutions from Qwen2.5-3B for each question and compute the uncertainty-based difficulty estimator defined above. We use the resulting estimated difficulty values to reproduce the main results of RQ1 (Section 4.1) and RQ3 (Section 4.3). As shown in Figures 22 and 23, the findings remain consistent when using this estimated difficulty measure, demonstrating that our conclusions hold under a practical difficulty-estimation setting.
37
Published as a conference paper at ICLR 2026
Qwen2.5 Qwen3 Llama3 Small (2-4B) Large (70B+)
Gemma2 Mistral GPT-4o Medium (7-32B)
1.0 1.0
True Positive Rate True Negative Rate
0.5 0.5
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
Problem Difficulty Problem Difficulty
(a) TPR (Mathematics) (b) TNR (Mathematics)
Figure 22: Ablation study of RQ1 with estimated problem difficulty. The experimental setup follows that of Section 4.1 and Figure 2, except that we replace the oracle difficulty with our uncertainty-based difficulty estimation.
Qwen2.5 Llama3 Mistral 2-4B 24-32B
Qwen3 Gemma2 GPT 7-9B 70B+
Hard: Linear R²=0.76, Nonparam R²=0.80
0.9
Medium: Linear R²=0.88, Nonparam R²=0.88
Easy: Linear R²=0.59, Nonparam R²=0.75
Balanced Accuracy
0.8
0.7
0.6
0.0 0.2 0.4 0.6 0.8 1.0
Verifier's Generation Capability
(a) Stratified view (Mathematics)
Figure 23: Ablation study of RQ3 with estimated problem difficulty. The experimental setup follows that of Section 4.3 and Figure 6, except that we replace the oracle difficulty with uncertainty-based difficulty estimation.
D.2 E FFICIENCY AND P ERFORMANCE A NALYSIS OF M ODEL C HOICES IN TTS
This subsection examines how to make cost-effective model choices using our findings from RQ4 and RQ5 (Section 5). From RQ4, we observe that when weaker generators achieve high verification performance, they also get larger TTS verification gains, and can approach the performance of stronger generators. From RQ5, we find that in certain regimes, smaller verifiers can obtain verification performance comparable to larger ones, with similar verification gains. These results motivate us to use a verification metric to guide generator and verifier selection in TTS. In these experiments, we follow the efficiency metric and difficulty-stratified analysis used in Liu et al. (2025a). For each subset stratified by problem difficulty, we split the problem set into a 50% validation set and a 50% test set. Verification metrics are computed on the validation set. Specifically, for each problem, we sample 8 balanced correct and incorrect responses from the generator, apply the verifier, and compute balanced accuracy (the average of TPR and TNR). We use this metric to select the generator or verifier among the candidates, and then evaluate their efficiency and TTS accuracy on the test set. For the efficiency metric, we use inference FLOPs, computed as 2N Dinference , where N represents the model parameters and Dinference is the total number of tokens generated during inference. For open-source models, we approximate N using the parameter counts indicated in their model names (e.g., we use 2 × 109 parameters for Gemma2-2B). For the proprietary model GPT-4o, we estimate the number of active parameters during inference as 100B, since the largest open-source model in our study is 72B and GPT-4o is widely understood to operate at a larger effective scale. This estimate is used exclusively for relative efficiency comparison and does not affect our experimental findings. We report FLOPs averaged over all problems. When results are reported without verification, only the generator FLOPs are included; when results are reported with verification, the FLOPs of both the generator and the verifier are summed. We use a TTS sample size of 64 for every model combination
38
Published as a conference paper at ICLR 2026
and problem. We compute the total FLOPs used per problem and report the average FLOPs over the problem set. For the accuracy metric, we use the pass rate defined in Section 3.1, which measures the expected accuracy obtained by uniformly sampling one response from either the set of model-generated samples or from the verifier-retained samples. To complement the pass rate, we also report majority-vote accuracy. We consider the following two subproblems, named RQ4.1 and RQ5.1.
D.2.1 RQ4.1: H OW TO C HOOSE G ENERATORS G IVEN A F IXED V ERIFIER ? We compare three generator models (Gemma2-2B, 9B, and 27B) under a fixed verifier, and we restrict the comparison to models within the same family and version to avoid confounding effects from differences in pre-training quality. Across three difficulty ranges: easy [0.9, 1.0), medium [0.7, 0.8), and hard [0.3, 0.4), we select the generator with the highest verification performance measured on the validation set. We vary the verifier between Qwen3-32B and GPT-4o. Results of RQ4.1 The results are presented in Figure 24 (pass rate, Qwen3-32B as verifier), Figure 25 (pass rate, GPT-4o as verifier), Figure 26 (majority vote, Qwen3-32B as verifier), and Figure 27 (majority vote, GPT-4o as verifier). Each figure contains three rows: (1) TTS performance vs. efficiency without verification on the test set, (2) the same analysis with verification, and (3) balanced accuracy on the validation set. The model with the highest balanced accuracy is selected as the cost-efficient choice and highlighted in rows 2 and 3. Without verification, the largest model is usually the preferred choice due to its large accuracy advantage. With verification, there are cases where smaller models become compute-efficient choices because they achieve comparable TTS performance while requiring fewer FLOPs. This is consistent with our earlier observation in Section 5.1 RQ4. Notably, the generator selected by the highest balanced accuracy on the validation set (red circle) consistently matches the compute-efficient choice: it either identifies a smaller model that achieves accuracy close to the larger one or preserves the largest model when it retains a clear performance advantage. This shows that verification-based metrics provide effective guidance for generator selection.
39
Published as a conference paper at ICLR 2026
2B 9B 27B Compute-efficient choice
(a) Easy problem, Verifi. metrics (b) Medium problem, Verifi. metrics (c) Hard problem, Verifi. metrics
1.0 1.0 1.0
Balanced Accuracy Balanced Accuracy Balanced Accuracy
Balanced Accuracy Balanced Accuracy Balanced Accuracy 0.8 0.8 0.8
0.6 0.6 0.6
2B 9B 27B 2B 9B 27B 2B 9B 27B
(d) Easy problem, w/o Verification (e) Medium problem, w/o Verification (f) Hard problem, w/o Verification
1.0 1.0
0.30
Task Accuracy 0.5 0.5 0.15
0.0 0.0 0.00
0.1 0.2 0.3 0.4 0.5 0.6 0.2 0.4 0.6 0.8 1.0 1.2 0.25 0.50 0.75 1.00 1.25 1.50 1.75
(g) Easy problem, w/ Verification (h) Medium problem, w/ Verification (i) Hard problem, w/ Verification
1.0 1.0
0.30
Task Accuracy 0.5 0.5 0.15
0.0 0.0 0.00
0.9 1.0 1.1 1.2 1.3 1.6 1.8 2.0 2.2 2.4 2.6 2.0 2.5 3.0 3.5
FLOPs per Problem (1e15) FLOPs per Problem (1e15) FLOPs per Problem (1e15)
Figure 24: Using a verification metric to select compute-efficient generators across different problem difficulties. Qwen3-32B is used as the verifier, and generator candidates are Gemma2- 2B, 9B, and 27B. (a–c) Balanced accuracy on the validation set varies across difficulty ranges and identifies different preferred generators: the 2B model on easy problems, 9B on medium problems, and 27B on hard problems. (d-f) TTS accuracy versus compute on the test set without verification. (g-i) TTS accuracy versus compute with verification. The generators selected by balanced accuracy correspond to the compute-efficient choices: on easier problems, smaller generators achieve accuracy close to larger models while requiring fewer FLOPs, whereas on hard problems, the largest generator retains a performance advantage. 2B 9B 27B Compute-efficient choice (a) Easy problem, Verifi. metrics (b) Medium problem, Verifi. metrics (c) Hard problem, Verifi. metrics 1.0 1.0 1.0 Balanced Accuracy Balanced Accuracy Balanced Accuracy
Balanced Accuracy Balanced Accuracy Balanced Accuracy 0.8 0.8 0.8
0.6 0.6 0.6
2B 9B 27B 2B 9B 27B 2B 9B 27B
(d) Easy problem, w/o Verification (e) Medium problem, w/o Verification (f) Hard problem, w/o Verification
1.0 1.0
0.30
Task Accuracy 0.5 0.5 0.15
0.0 0.0 0.00
0.1 0.2 0.3 0.4 0.5 0.6 0.2 0.4 0.6 0.8 1.0 1.2 0.25 0.50 0.75 1.00 1.25 1.50 1.75
(g) Easy problem, w/ Verification (h) Medium problem, w/ Verification (i) Hard problem, w/ Verification
1.0 1.0
0.30
Task Accuracy 0.5 0.5 0.15
0.0 0.0 0.00
2.2 2.3 2.4 2.5 2.6 3.2 3.4 3.6 3.8 3.0 3.5 4.0 4.5
FLOPs per Problem (1e15) FLOPs per Problem (1e15) FLOPs per Problem (1e15)
Figure 25: Using a verification metric to select compute-efficient generators across different problem difficulties. GPT-4o is used as the verifier; task accuracy is measured by pass rate. The experimental setups and observations are consistent with Figure 24.
40
Published as a conference paper at ICLR 2026
2B 9B 27B Compute-efficient choice
(a) Easy problem, Verifi. metrics (b) Medium problem, Verifi. metrics (c) Hard problem, Verifi. metrics
1.0 1.0 1.0
Balanced Accuracy Balanced Accuracy Balanced Accuracy
Balanced Accuracy Balanced Accuracy Balanced Accuracy 0.8 0.8 0.8
0.6 0.6 0.6
2B 9B 27B 2B 9B 27B 2B 9B 27B
(d) Easy problem, w/o Verification (e) Medium problem, w/o Verification (f) Hard problem, w/o Verification
1.0 1.0 0.4
Task Accuracy 0.5 0.5 0.2
0.0 0.0 0.0
0.1 0.2 0.3 0.4 0.5 0.6 0.2 0.4 0.6 0.8 1.0 1.2 0.25 0.50 0.75 1.00 1.25 1.50 1.75
(g) Easy problem, w/ Verification (h) Medium problem, w/ Verification (i) Hard problem, w/ Verification
1.0 1.0 0.4
Task Accuracy 0.5 0.5 0.2
0.0 0.0 0.0
0.9 1.0 1.1 1.2 1.3 1.6 1.8 2.0 2.2 2.4 2.6 2.0 2.5 3.0 3.5
FLOPs per Problem (1e15) FLOPs per Problem (1e15) FLOPs per Problem (1e15)
Figure 26: Using a verification metric to select compute-efficient generators across different problem difficulties. Qwen3-32B is used as the verifier; task accuracy is measured by majority vote. The experimental setups and observations are consistent with Figure 24.
2B 9B 27B Compute-efficient choice
(a) Easy problem, Verifi. metrics (b) Medium problem, Verifi. metrics (c) Hard problem, Verifi. metrics
1.0 1.0 1.0
Balanced Accuracy Balanced Accuracy Balanced Accuracy
Balanced Accuracy Balanced Accuracy Balanced Accuracy 0.8 0.8 0.8
0.6 0.6 0.6
2B 9B 27B 2B 9B 27B 2B 9B 27B
(d) Easy problem, w/o Verification (e) Medium problem, w/o Verification (f) Hard problem, w/o Verification
1.0 1.0
0.30
Task Accuracy 0.5 0.5 0.15
0.0 0.0 0.00
0.1 0.2 0.3 0.4 0.5 0.6 0.2 0.4 0.6 0.8 1.0 1.2 0.25 0.50 0.75 1.00 1.25 1.50 1.75
(g) Easy problem, w/ Verification (h) Medium problem, w/ Verification (i) Hard problem, w/ Verification
1.0 1.0
0.30
Task Accuracy 0.5 0.5 0.15
0.0 0.0 0.00
2.2 2.3 2.4 2.5 2.6 3.2 3.4 3.6 3.8 3.0 3.5 4.0 4.5
FLOPs per Problem (1e15) FLOPs per Problem (1e15) FLOPs per Problem (1e15)
Figure 27: Using a verification metric to select compute-efficient generators across different problem difficulties. GPT-4o is used as the verifier; task accuracy is measured by majority vote. The experimental setups and observations are consistent with Figure 24.
D.2.2 RQ5.1: H OW TO C HOOSE A V ERIFIER G IVEN A F IXED G ENERATOR ?
We evaluate whether verification metrics can guide the choice between two verifiers (GPT-4o and Qwen2.5-7B) under different levels of problem difficulty. Following the procedure in Section D.2.1,
41
Published as a conference paper at ICLR 2026
we compute the verification metric on the validation set and use it to select the verifier. TTS accuracy and computation cost are then reported on the test set. Results of RQ5.1 The results are presented in Figure 28 (pass rate) and Figure 29 (majority vote). For each figure, the first two subplots report TTS accuracy versus compute when using a small or large verifier. The final subplot reports the verification metrics. On the hardest problem, GPT-4o yields only a small balanced accuracy improvement over Qwen2.5- 7B. The minimal metric difference suggests the smaller verifier. Evaluated on the test set, the accuracy difference between the two verifiers is small. It confirms that the stronger verifier offers limited additional benefit in this regime. On hard problems, GPT-4o shows a clear gap of balanced accuracy over the small model. This difference points to choosing GPT-4o. On the test set, this leads to substantially improved TTS accuracy despite higher compute cost. These results show that verification accuracy provides actionable guidance. When compute is limited, one can compare the balanced accuracy of the large and small verifiers: if the gap is small, the smaller verifier is the more cost-efficient choice; if the gap is large, the stronger verifier yields more substantial verification gains in TTS.
(a) Verification Metric (b) Hardest Problems: (c) Hard Problems:
Balanced Accuracy Task Accuracy by Pass Rate Task Accuracy by Pass Rate
0.80 0.4 1.0
Verifier: Qwen2.5-7B Verifier: Qwen2.5-7B Verifier: Qwen2.5-7B
0.75 Verifier: GPT-4o Verifier: GPT-4o 0.8 Verifier: GPT-4o
0.3 w/o Verification w/o Verification
Balanced Accuracy Task Accuracy Task Accuracy 0.70 =14.9% 0.6 0.65 0.2 3.8% verification gain gap 0.4 14.9% verification gain gap 0.60 0.1 =3.8% 0.55 0.2
0.50 0.0 0.0
Hardest Hard 2 3 4 5 1 2 3 4
Problem Difficulty FLOPs per Problem (1e15) FLOPs per Problem (1e15)
Figure 28: Verification-metric–guided verifier selection across problem difficulty; task accuracy is measured by pass rate. (a) Balanced accuracy on the validation set: when the metric gap between GPT-4o and Qwen2.5-7B is small (hardest problems), the smaller verifier is selected; when the gap is large (hard problems), the stronger verifier is selected. (b–c) Corresponding TTS accuracy on the test set: consistent with the metric-based choice, GPT-4o yields only a small improvement on the hardest problems but provides substantially larger gains on the hard problems.
(a) Verification Metric (b) Hardest Problems: (c) Hard Problems:
Balanced Accuracy Task Accuracy by Majority Vote Task Accuracy by Majority Vote
0.80 0.4 1.000
Verifier: Qwen2.5-7B Verifier: Qwen2.5-7B
0.75 Verifier: GPT-4o Verifier: GPT-4o 0.975 =5.4%
0.3 w/o Verification
Balanced Accuracy Task Accuracy Task Accuracy 0.70 0.950
0.65 0.2 0.925 5.4% verification gain gap
3.7% verification gain gap
0.60 =3.7% 0.900
0.1 Verifier: Qwen2.5-7B
0.55 0.875 Verifier: GPT-4o
w/o Verification
0.50 0.0 0.850
Hardest Hard 2 3 4 5 1 2 3 4
Problem Difficulty FLOPs per Problem (1e15) FLOPs per Problem (1e15)
Figure 29: Verification-metric–guided verifier selection across problem difficulty; task accuracy is measured by majority vote. The experimental setups and observations are the same as Figure 28.
D.3 A BLATION S TUDY ON L ARGE M ODELS
We conduct an ablation study using a large LLM with more than 100B parameters, Qwen3-235B (Qwen/Qwen3-235B-A22B-Instruct-2507). We use the recommended sampling hyperparameters: temperature 0.7, top-p 0.8, and top-k 20. We repeat the experiments for RQ1 (Section 4.1) and RQ2
42
Published as a conference paper at ICLR 2026
(Section 4.2) on the Mathematics domain and report the results in Figure 30. The results show that our main findings remain consistent when using a larger model.
1.0
True Positive Rate
0.5
0.0Hardest Hard Easy Easiest
Problem Difficulty
(a) RQ1 Trend of TPR
Performance Rate
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Verifier
Qwen3-235B 0.96 0.92 0.92 0.89 0.88 0.85 0.89 0.82 0.85 0.83 0.76 0.78 0.77 0.76 0.69
-2B -3B -8B 8B -9B B
-27 3B B
24 7B -70 B T-4 B
72 -4B -8B B
-32
G2 L3 L3 M- G2 .5- M- .5- o .5- Q3 Q3
G2 Q2 Q2 L3 GP Q2 Q3
Generator
(b) RQ2 Trend of TNR
Figure 30: Ablation study of our main findings with the large model Qwen3-235B. The experi- ments follow the same setup as Sections 4.1 and 4.2 and are conducted on the Mathematics domain. (a) The trend that TPR increases as problem difficulty decreases is consistent with our RQ1 finding. (b) The trend that TNR decreases as the generator strength increases is consistent with our RQ2 finding.
D.4 A BLATION S TUDY ON V ERIFICATION P ROMPTS
We conduct an ablation study to evaluate the robustness of our findings under different verification prompts. All prompt templates used in this study are listed in Section A. In addition to the original “Verification Evaluation Prompt,” we consider two alternatives: a concise “step-by-step” verification prompt from Zhang et al. (2024), and a “solve-then-verify” prompt that encourages the verifier to first solve the problem before evaluating the candidate response, a strategy shown to be effective in prior work Chen et al. (2025d). These two prompts differ substantially from our original design. The experiments follow the same setup as Sections 4.1 and 4.2 and are conducted on the Mathematics domain. For each prompt, we apply Qwen2.5-72B as the verifier to evaluate responses produced by all 15 generator models. The results in Figure 31 show that our main findings regarding TPR and TNR dynamics remain consistent across all prompts, indicating that the observed verification dynamics are robust to prompt variations.
43
Published as a conference paper at ICLR 2026
1.0 1.0
True Positive Rate True Positive Rate
0.5 0.5
0.0Hardest Hard Easy Easiest
0.0Hardest Hard Easy Easiest
Problem Difficulty Problem Difficulty
(a) RQ1 Trend of TPR (Solve-then-Verify) (b) RQ1 Trend of TPR (Step-by-Step)
Performance Rate
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Verifier Verifier
Qwen2.5-72B 0.76 0.72 0.67 0.59 0.59 0.54 0.49 0.41 0.35 0.48 0.32 0.27 0.36 0.35 0.32 Qwen2.5-72B 0.39 0.54 0.45 0.43 0.38 0.36 0.33 0.25 0.22 0.33 0.20 0.14 0.22 0.20 0.19
-2B -3B -8B 8B
M-
-9B -27B
3B
.5- 24 B 7B
.5- -70B T-4 o 72 B -4B -8B -32B -2B -3B -8B 8B
M-
-9B -27B
3B
.5- 24 B 7B
.5- -70B T-4 o 72 B -4B -8B -32B
G2 L3 L3 G2 G2 Q2 M- Q2 L3 GP .5- Q3 Q3 Q3 G2 L3 L3 G2 G2 Q2 M- Q2 L3 GP .5- Q3 Q3 Q3
Q2 Q2
Generator Generator
(c) RQ2 Trend of TNR (Solve-then-Verify) (d) RQ2 Trend of TNR (Step-by-Step)
Figure 31: Ablation study of our findings under different verification prompts. (a, c) Using “solve-then-verify” prompt for verification. (b, d) Using “step-by-step” prompt for verification. The trend that TPR increases as difficulty decreases (RQ1) and TNR decreases as the generator strength increases (RQ2) still holds under both prompts.
Table 2: Comparison of TTS performance and token cost per problem when using a strong model as the verifier versus as the generator. Token count is reported on average per prompt.
Generator Verifier Pass Rate Avg. GPT-4o Tokens Avg. Qwen3-4B Tokens
Qwen3-4B – 0.938 – 551
GPT-4o – 0.952 482 –
Qwen3-4B GPT-4o 0.954 193 551
D.5 C OMPARISON A NALYSIS OF S TRONG M ODEL AS G ENERATOR OR V ERIFIER
We examine whether it is practical to use a strong model as a verifier instead of using it solely as a generator. We compare two setups: (i) using GPT-4o as the generator, and (ii) using GPT-4o as the verifier together with a smaller generator, Qwen3-4B. We use the same problem set as in Section 5.1: Mathematics problems with difficulty in the range [0.7, 0.8), consisting of 181 problems. The TTS sampling size is 64. Task accuracy is measured using the pass rate, and efficiency is measured using the average tokens per prompt. Here are the findings. Both setups achieve the same performance, but using GPT-4o as a verifier consumes 193 tokens on average per prompt, whereas GPT-4o as a generator uses 483 tokens per prompt on average. This represents a 2.5× reduction in token usage (and thus cost) per prompt. Thus, for practitioners who host small models but care about maximizing performance per dollar, leveraging the relatively short-form nature of verification with API models represents massive savings.
E C ASE S TUDIES
Problem domain, dataset and difficulty: Mathematics, MATH-500, Hardest (d(x) = 0.075)
Generator: Qwen2.5-3B
Verifier: GPT-4o
Question: Bob and Alice each have a bag that contains one ball of each of the colors, blue, green, orange, red, and violet. Alice
randomly selects one ball from her bag and puts it into Bob’s bag. Bob then randomly selects one ball from his bag and puts it into
Alice’s bag. What is the probability that after this process the contents of the two bags are the same?
Ground-Truth Solution: After Alice puts the ball into Bob’s bag, his bag will contain six balls: two of one color and one of each of
the other colors. After Bob selects a ball and places it into Alice’s bag, the two bags will have the same contents if and only if Bob
44
Published as a conference paper at ICLR 2026
picked one of the two balls in his bag that are the same color. Because there are six balls in the bag when Bob makes his selection, the 1 probability of selecting one of the same colored pair is 2/6 = . 3 Generator Output (Correct): To determine the probability that the contents of both Bob’s and Alice’s bags are the same after the process described, we need to analyze each possible step and its impact on the contents of the bags. Initially, each bag contains one ball of each of the colors: blue, green, orange, red, and violet. ### Step-by-Step Analysis [ Details omitted ] ### Identifying the Conditions for Equality [ Details omitted ] ### Calculating the Probability [ Details omitted ] The probability is then calculated as follows: Number of favorable outcomes 2 1 Probability = = = Total number of outcomes 6 3
1
Thus, the probability that after this process the contents of the two bags are the same is . 3 Verifier Output (False negative): Explanation: The problem involves two steps: Alice transferring a ball to Bob’s bag and then Bob transferring a ball back to Alice’s bag. Initially, both bags have one ball of each color. After Alice transfers a ball, Bob’s bag will have two balls of one color and one of each of the other four colors. For the bags to be identical after Bob’s transfer, Bob must return the same color ball that Alice initially transferred. This is the only scenario where the bags will end up identical, as it restores the original distribution of colors in both bags. The assistant’s response correctly identifies that there are 6 possible outcomes for Bob’s selection (since he has 6 balls to choose from) and that there are 2 favorable outcomes (returning one of the two balls of the color that Alice transferred). However, the assistant incorrectly concludes that the probability is 1/3. The correct probability should be 1/5, as Bob must specifically return the ball of the color that Alice transferred, which is only one specific ball out of the 6 in his bag. [Verifier generates a wrong answer 1/5 and uses it to compare with generator’s correct answer 1/3] Verdict: [[Incorrect]]
Problem domain, dataset and difficulty: Natural Language Reasoning, FOLIO, Hardest (d(x) = 0.122) Generator: Llama3.1-8B Verifier: Qwen2.5-72B Question: Some mammals have teeth. Platypuses have no teeth. Platypuses are mammals. Humans have teeth. Based on the above, can the following be inferred? Platypuses are reptiles Ground-Truth Solution: Uncertain Generator Output (Correct): Reasoning: 1. The statement that some mammals have teeth and humans have teeth suggests that having teeth is a characteristic of mammals, but it’s not exclusive to them. 2. Platypuses are indeed mammals, as stated in the information provided. 3. The fact that platypuses have no teeth does not contradict the fact that they are mammals, as having no teeth is also a characteristic of some mammals, as mentioned in point 1. 4. The only group of animals that are not mammals and have been ruled out as a category for platypuses are not the mammals, but rather the reptiles. However, concluding that platypuses are reptiles because they have no teeth is not a sound inference, because having no teeth does not preclude an animal from being a mammal, as shown in point 3. Moreover, it contradicts the given statement that platypuses are indeed mammals. 5. Therefore, platypuses being reptiles cannot be inferred based on the given information. Answer: Uncertain Verifier Output (False negative): Explanation: The assistant’s response is mostly correct in its reasoning but contains a minor inaccuracy in the final conclusion. The assistant correctly identifies that the fact that platypuses have no teeth does not make them reptiles, as some mammals also lack teeth. However, the assistant’s final answer of "Uncertain" is not appropriate. Given the information provided, it is clear that platypuses are mammals, and there is no evidence to suggest they are reptiles. Therefore, the correct answer should be "No" because the statement "Platypuses are reptiles" can be definitively refuted based on the given information. [Although the verifier recognizes that the generator’s reasoning process is correct, it rejects the conclusion due to a mismatch with its own (incorrect) answer.] Verdict: [[Incorrect]]
Figure 32: Case study: How problem difficulty impacts TPR (RQ1). False negative verification on high-difficulty problems demonstrates why TPR decreases as problems become harder. When verifiers cannot correctly solve challenging problems, they produce incorrect reference solutions and reject generators’ correct answers based on these flawed references.
Problem domain, dataset and difficulty: Mathematics, MATH-500, Easy (d(x) = 0.571) Generator: Llama-3.1-8B Verifier: Qwen2.5-72B Question: Paul and Jesse each choose a number at random from the first six primes. What is the probability that the sum of the numbers they choose is even? Ground-Truth Solution: [ Details omitted ] Therefore, the probability that the sum of the numbers Paul and Jesse choose IS even is 5 13 1 − 18 = . 18
45
Published as a conference paper at ICLR 2026
Generator Output (Incorrect): To find the probability that the sum of the numbers Paul and Jesse choose is even, we need to first understand what numbers can make an even sum. The sum of two numbers is even if both numbers are either even or odd. Since the first six primes are 2, 3, 5, 7, 11, and 13, we have only one even prime, which is 2. Now, let’s consider the possible scenarios: 1. Both Paul and Jesse choose the number 2. In this case, the sum is even. 2. One of Paul or Jesse chooses the number 2, and the other chooses an odd prime. In this case, the sum is odd. 3. Both Paul and Jesse choose odd primes. In this case, the sum is even. [Generator correctly identifies two cases for even sum, one case for odd sum.] [ Probability calculations omitted omitted ] Now, let’s add the probabilities of the first two scenarios (both choose 2, and one chooses 2 and the other an odd prime) and the third scenario (both choose an odd prime) to get the total probability of an even sum: Probability = 1/36 + 10/36 + 25/36 Probability = 36/36 Probability = 1 The probability that the sum of the numbers Paul and Jesse choose is even is 1 or 100%. This is because it is impossible for the sum to be odd, as they will always choose either two even numbers or two odd numbers. [Generator makes self-contradiction: Earlier stated scenario 2 gives odd sum, now count scenario 2 as even sum and claims odd sum is impossible] Verifier Output (True negative): The assistant’s response contains a logical error in the final calculation. The assistant correctly identifies the scenarios where the sum of the numbers chosen by Paul and Jesse is even. However, the assistant incorrectly adds the probabilities of all scenarios, including the scenario where the sum is odd, which should not be included in the final probability calculation. [Verifier catches the self-contradiction of generator.] 1 The correct calculation should only include the probabilities of the scenarios where the sum is even: - Both choose 2: 36 - Both choose odd primes: 2536 The probability of the sum being even is the sum of these two probabilities: 1 25 26 13 + = = 36 36 36 18
13
Therefore, the final answer should be . [Verifier can solve the problem correctly] 18 Verdict: [[Incorrect]]
Same problem and verifier as above with a strong generator: Generator: GPT-4o Verifier: Qwen2.5-72B Generator Output (Incorrect): To find the probability that the sum of the numbers chosen by Paul and Jesse from the first six prime numbers is even, we proceed as follows: 1. List the first six prime numbers: The first six prime numbers are 2, 3, 5, 7, 11, and 13. 2. Identify even and odd primes: - Even prime: 2 - Odd primes: 3, 5, 7, 11, 13 3. Understand when the sum is even: - A sum is even if both numbers are even or both numbers are odd. - Since there is only one even prime (2), for the sum to be even, both numbers must be odd. [Fatal error: Ignores that both choosing 2 (even+even=even) is also possible. 4. Calculate the total number of ways to choose numbers: [ Calculation details omitted omitted ] 5. Calculate the number of ways to get an even sum: [ Calculation details omitted omitted ] 6. Calculate the probability: [ Calculation details omitted omitted ] 25 Thus, the probability that the sum of the numbers they choose is even is . [Final answer is wrong because following steps 36 consistently based on fatal error made above.] Verifier Output (False positive): Explanation: The assistant’s response is logically sound and mathematically accurate. Here’s a breakdown of the steps: 1. Identifying the first six primes: The assistant correctly identifies the first six prime numbers as 2, 3, 5, 7, 11, and 13. 2. Condition for the sum to be even: The assistant correctly states that for the sum to be even, both numbers must be odd (since the only even prime is 2, and choosing 2 would make the sum odd if the other number is odd). [Verifier is convinced by the generator and ignores the error here.] 3. Total number of possible outcomes: The assistant correctly calculates the total number of outcomes as 6 * 6 = 36. 4. Number of outcomes where both numbers are odd: The assistant correctly identifies that there are 5 odd primes and calculates the number of outcomes where both numbers are odd as 5 * 5 = 25. 5. Probability calculation: The assistant correctly calculates the probability as the number of favorable outcomes (25) divided by the total number of outcomes (36), resulting in 25 36 . The assistant’s reasoning and calculations are correct throughout the response. [Although the verifier demonstrated correct problem- solving when evaluating Llama3.1-8B in last example, it doesn’t apply the same mechanism to GPT-4o, instead accepting the GPT-4o’s incorrect reasoning.] Verdict: [[Correct]]
46
Published as a conference paper at ICLR 2026
Figure 33: Case study: How generator capability influences TNR (RQ2). We compare verifier behavior on the same problem when evaluating weak (Llama3.1-8B) versus strong (GPT-4o) genera- tors. The weak generator correctly lists all the cases but then contradicts itself in the final calculation, producing an inconsistent solution. The verifier catches this apparent mistake and rejects it. The strong generator makes a single early-stage error but maintains internal consistency throughout its reasoning, generating a well-structured and self-contained chain that appears correct. Despite the solution containing an actual error, the verifier fails to detect it and accepts the solution.
47