英文全文 · When LLMs Develop Languages：用于高效多智能体推理的符号通信
When LLMs Develop Languages:
Symbolic Communication for Efficient Multi-Agent Reasoning
Zhengqi Pei 1 2 Qingming Huang 1 2 Shuhui Wang 1
Abstract 1. Introduction
Large Language Models (LLMs) have demonstrated remark-
arXiv:2606.29354v1 [cs.AI] 28 Jun 2026 Chain-of-Thought (CoT) improves large language able reasoning abilities, especially when employing chain- models (LLMs) on difficult reasoning tasks, but it of-thought and self-consistency strategies that explicitly often incurs long natural-language rationales that generate intermediate reasoning steps (Wei et al., 2022; are poorly aligned with efficient machine reason- Yao et al., 2023; Wang et al., 2025a; Besta et al., 2024). ing. We propose Communicative Language Sym- Prompting an LLM to produce a step-by-step explanation bolism Routing (CLSR), a test-time framework often improves its accuracy on complex tasks (Zhang et al., in which multiple LLM agents autonomously in- 2025b; Sprague et al., 2025). However, these reasoning vent, evolve, and share compact Language Sym- chains are usually expressed in natural language designed bolism Frameworks (LSFs), while a latent-free for human understanding. Linguistic reasoning chains can router adaptively selects and composes these lan- be verbose and are not specifically designed for compact log- guages per query to optimize the accuracy–token ical inference (Turpin et al., 2023; Tanmay et al., 2025), thus trade-off. Unlike prompt optimization that re- they may not be the most efficient internal representation fines surface instructions, CLSR treats each LSF of the model (Yu et al., 2024; Chen et al., 2025), especially as a reusable symbolic protocol with compact for budget-constrained deployments (Nayab et al., 2024b; symbols, usage rules, and a message-passing con- Arora & Zanette, 2025). This raises an intriguing question tract, and improves it through an evolutionary that we aim to address in this study: Can LLMs invent new loop driven by correctness and token cost. At symbolic languages to reason more efficiently? inference time, the router may invoke a single Recent work leaves large gaps in our question. Prompt low-cost LSF call, ensemble multiple LSFs, or ex- optimization methods (Yang et al., 2023; Fernando et al., ecute a multi-round LSF composition protocol on 2024; Zhou et al., 2024; Aytes et al., 2025b) primarily refine harder queries. Across challenging benchmarks, natural-language instructions rather than inducing a persis- CLSR reduces latency-oriented generated token tent compositional symbol system that can be reused and completion by 3 ∼ 6× compared to standard CoT transferred between tasks. Program-based methods (Gao while maintaining accuracy. We further derive an et al., 2023; Chen et al., 2023; Grand et al., 2024) heav- information-theoretic lower bound on token cost ily depend on human-designed intermediate languages under arbitrary symbolism and show that, under or programs, therefore, do not study whether LLMs can an interpreter-realizability premise, multi-round self-discover discrete symbolic languages optimized for LSF protocols conditionally subsume program- their own tokenization and inductive biases. RL-based ap- execution pipelines. Code is publicly available1 proaches (Tanmay et al., 2025; Gehring et al., 2025) produce compact symbolic traces using verifier-based RL, yet these RL pipelines typically incur substantial training cost and optimization instability, making them heavy-weight and less 1 State Key Lab of AI Safety, Institute of Computing Technology, modular for rapid multi-agent language evolution. Com- Chinese Academy of Sciences, Beijing, China. 2 School of Com- pression methods (Cheng & Van Durme, 2024) reduce the puter Science and Technology, University of Chinese Academy length of explicit reasoning by moving computation into of Sciences, Beijing, China.. Correspondence to: Shuhui Wang dense tokens, but they do not yield a socially shareable wangshuhui@ict.ac.cn. discrete “language” that can evolve via inter-agent adop- Proceedings of the 43 rd International Conference on Machine tion and merger. Neuro-symbolic approaches (Beiser et al., Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 2025) and logical reasoning (Xu et al., 2025a) show that an by the author(s). intermediate language can motivate a systematic search over 1 https://github.com/pzqpzq/LSF_MDia
1
When LLMs Develop Languages
(b) Multiple agents evolve new LSFs.
(a) Conceptual comparison. (c) Query-adaptive LSF planning.
Figure 1. Communicative Language Symbolism Routing (CLSR). (a) We let LLMs self-evolve an LLM-oriented compact Language Symbolism Framework (LSF), i.e., an information-dense reasoning “dialect”, rather than expanding natural language rationales (e.g., CoT) or relying on an external executor (e.g., PoT). An LLM router then selects, ensembles, or composes LSFs, enabling a controllable accuracy–token trade-off. (b) Multiple agents iteratively propose, critique, and mutate candidate LSFs based on high-leverage selection. (c) The saturated LSF pool is then queried at inference time for query-adaptive protocol planning.
representations; yet, their pipelines typically select among not that LSFs are free of human priors. Rather, the claim is fixed formal languages rather than evolve new ones in a that the operational protocol used at inference time can be socially grounded way. automatically generated, selected, shared, and composed by LLM agents, rather than being crafted manually as a fixed In parallel, emergent communication methods (Lazaridou & prompt or program. Baroni, 2020; Tucker et al., 2022; Pei et al., 2024a; 2025) formalize how discrete protocols can arise under compu- This view naturally connects CLSR to a socio-linguistic tational pressures, offering an information-theoretic lens perspective inspired by the cultural language evolution pro- on why structured conventions emerge. Motivated by this cess (Kouwenhoven et al., 2025). Across evolutionary perspective, we ask whether LLM agents can autonomously rounds, more effective LSFs are preferentially retained and design and evolve languages, then subject them to natural reused, while less accurate or overly verbose LSFs are dis- selection. We operationalize this question as an evolutionary carded. The resulting process resembles cultural language search over reusable LSFs: protocols that are simultaneously evolution under an explicit communication bottleneck: cor- accurate, compact, and reusable are preferentially retained rectness plays the role of communicative success, token and made available for downstream routing. This setting length plays the role of production cost, and routing plays yields emergent large-scale symbolic systems shaped by the role of pragmatic code-switching between compact and collective use rather than isolated prompt tuning2 . redundant LSFs. Over generations, more effective LSFs spread across agents, while less useful ones are discarded, In this work, we introduce CLSR (Communicative Lan- leading to converging robust symbolic languages. guage Symbolism Routing), a paradigm in which multiple LLM agents autonomously develop their own symbolic com- We design experiments on a diverse set of challenging tasks, munication systems for reasoning. Each agent is a black- including knowledge-intensive QA (Wang et al., 2024; Rein box LLM instance that proposes, evaluates, or refines a et al., 2023; Lu et al., 2022), mathematical problem solv- Language Symbolism Framework (LSF), where an LSF is ing (Cobbe et al., 2021; Hendrycks et al., 2021; Veeraboina, a customized set of symbols, syntax, and usage rules that 2023), and multi-hop reasoning (Yang et al., 2018), to eval- serves as a reusable reasoning protocol. We emphasize that uate the impact of LLM-created languages. For gener- LSFs are not manually pre-defined by humans as a fixed ality, we conduct experiments with various open-source formal language. They are induced by the LLMs from ex- LLMs, e.g., Qwen3-8B/32B (Yang et al., 2025), LLaMA3- emplars and refined through selection. Thus, our claim is 8B (AI@Meta, 2024), and DeepSeek-R1 (DeepSeek-AI, 2 2025), treating each model as an agent in different runs. See the Appendix F for more discussion of related work.
2
When LLMs Develop Languages
Across multiple open-source backbones, CLSR improves and a router adaptively selects which LSFs should speak, the Pareto frontier of accuracy versus generated tokens, in which order, and when to stop, explicitly optimizing the outperforming baselines covering token-reduction, program- accuracy–token trade-off. These “experts” are emergent execution, and prompt optimization. symbolic languages rather than separate neural modules, and the LLM’s weights remain inaccessible. Ablations further show that (i) deeper evolutionary genera- tions systematically shift languages toward better accuracy- per-token, (ii) scaling the agent population improves the 2.1. Problem Setup and Metrics chance of discovering robust, reusable LSFs, provided the Let x be a query and y be the ground-truth answer drawn selection objective balances correctness and compression. from a benchmark. Suppose that we have an evolved LSF These empirical results lead to an information-theoretic in- pool S = {Sk }K k=1 . In inference, CLSR runs for T rounds, terpretation regarding the curve between reasoning accuracy where T can be fixed or determined by the router based on and token efficiency, given any language symbolism. We the test query. At round t ∈ {1, . . . , T }, the router selects an also conduct larger-backbone transfer, population-size ab- index set It ⊆ [K], where [K] = {1, . . . , K}, and invokes lations, cache-aware token accounting, and category-free the corresponding LSF subset {Sk : k ∈ It }. The LLM domain transfer to validate methodological robustness. In then produces one response per selected LSF. Empirically, summary, this work presents an approach to improving LLM a harder problem often yields a larger T or |It |. reasoning through socially emergent symbolic communica- tion. Our main contributions include: Compute cost (token accounting). Let rt,k denote the LLM response produced in round t under LSF Sk , and let • CLSR: routing over emergent languages. We pro- rtrouter denote the LLM-router’s generated planning output pose a new test-time paradigm that treats LLM-created in round t. We define the generation cost as symbolic languages (LSFs) as modular “experts” and T routes, ensembles, or composes them per query to ex- X X C = |rtrouter | + |rt,k | , (1) plicitly optimize the accuracy–token budget trade-off. t=1 k∈It • LLM-driven language synthesis and evolution. We introduce an evolutionary bootstrapping procedure that where |·| counts generated completion tokens. Thus, all generates LSFs from scratch; we further enable cross- online tokens emitted by the LLM are included, including agent adoption and cross-benchmark pooling to pro- router outputs, intermediate LSF responses, and aggregation- mote the transferring and merging of LSFs. related responses. This metric is motivated by the standard • Strong empirical Pareto gains. On seven diverse latency decomposition of LLM inference into a prompt pre- reasoning benchmarks and multiple open-source back- fill phase (time-to-first-token, TTFT) and an auto-regressive bones, CLSR consistently improves the accuracy– decode phase (time-per-output-token, TPOT). For reasoning efficiency frontier, outperforming representative token workloads with long completions, the decode term is often reduction baselines. the dominant latency component, so completion tokens pro- • A theory of accuracy–token optimality under sym- vide a useful latency-oriented proxy. At the same time, LSF bolism. We formalize CLSR as a constrained stochas- profiles do introduce input-token overhead. In deployment, tic control problem, derive an information-theoretic these cards can be arranged as a fixed canonical prompt lower bound on minimal token cost, and characterize prefix and reused across many queries, so the relevant over- multi-round multi-LSF protocols as a conditional gen- head depends on prefix caching and serving infrastructure. eralization of program-execution inference. For this reason, we additionally report a cache-aware token- equivalent metric in Appendix A.8, while keeping C as the main generated-token metric throughout the paper. 2. Methodology Objective. Given a budget B, our objective is to maximize We introduce the CLSR framework for test-time reasoning expected correctness under the expected realized cost: compression through LLM-generated symbolic languages. max E[I{ŷ = y}] s.t. E[C] ≤ B, Unlike prompt optimization that searches for instructions π or alters surface phrasing, CLSR optimizes a reusable com- where policy π specifies the router, stopping, and aggrega- munication protocol: (i) the intermediate messages are con- tion rules. The expectation is over the benchmark distribu- strained by grammar/contract and validated; (ii) the same tion and any policy-induced randomness. LSF is reused across many queries (not per-instance prompt tuning); (iii) the router explicitly composes multiple LSF 2.2. Language Symbolism Framework (LSF) calls into a multi-round protocol under a cost budget. During inference, the LLM participates in a multi-round commu- Conceptual definition. In idealized form, an LSF can be de- nication where different LSFs act as specialized “experts”, scribed as a symbolic communication protocol comprising
3
When LLMs Develop Languages
three components: (i) symbol naming, namely a compact 2.3. LSF Evolution: Iterative Bootstrapping lexicon; (ii) syntax, namely a compositional grammar; and We introduce an evolutionary bootstrapping procedure that (iii) constraints, namely well-formedness and usage rules. progressively refines the LSF pool using only training data However, in practice, fully hand-designing these compo- and black-box LLM calls. nents would collapse the method into prompt engineering. We therefore delegate LSF construction to the LLM itself Agent population. In our implementation, an “agent” is an and use human intervention only to define the high-level independently sampled black-box LLM instance specified optimization goal: to preserve reasoning accuracy while by a backbone, random seed, exemplar subset, and LSF- reducing token usage. generation context. Agents are not separate trainable neural modules; they are proposal, critique, and mutation workers Operational LSF card. In implementation, each LSF is used to diversify the LSF population. Different agents may stored as a compact card Sk = Vk , Gk , Rk , ψk , ρk , where discover different surface LSFs, but the selection objective Vk is the symbol inventory, Gk is a lightweight grammar or favors LSFs that repeatedly achieve high correctness with output schema, Rk contains usage rules and validity con- low generated-token cost. This definition also clarifies the straints, ψk maps symbols or templates to their intended agent-count ablation in Appendix A.5.3: increasing the reasoning operations, and ρk is an empirical profile summa- number of agents expands the search over possible symbolic rizing cost, accuracy, domains, and failure modes. The card conventions, while the final inference model remains frozen. is represented textually because the backbone is a language Dataset partition and generational schedule. We divide model, but its role is not that of a one-off instruction string. the training set into M disjoint groups {G1 , ..., GM }, each An LSF is reused across many queries, produces messages containing exemplars (x, y), then sequentially generate: constrained by a grammar or contract, can be selected or composed by the router, and is evaluated as a persistent pro- 1. Generation. Use exemplars from G1 to induce an tocol whose utility is measured over a population of tasks. (1) initial LSF pool S (1) = {Sk }K k=1 . This operational distinction is important: prompt optimiza- (1) 2. Evaluation. For each Sk , query the LLM on the tion searches for better surface instructions, whereas CLSR queries of G2 , produce answers ŷ and record the token searches over reusable symbolic communication systems cost of each answer. and their routing policies. 3. Selection and mutation. From all the answers gen- Seed exemplars and LSF synthesis prompt. Given a erated on G2 , select a subset of high-leverage (correct benchmark training set, we randomly sample a small set and token-efficient) answers and feed them to the LLM of QA exemplars (typically dozens) and place them in con- to generate the next-generation LSF pool S (2) . text. We then prompt the LLM to freely invent a symbolic 4. Repeat. use S (t) on Gt+1 , select high-leverage an- language that aims to reduce token usage while preserv- swers, and synthesize S (t+1) until the performance in ing reasoning capability, e.g., design an LSF based on the the validation set is saturated. exemplars in the chat history to minimize the number of to- kens while maintaining reasoning capacity. Manual editing, This procedure is reminiscent of iterative optimization in symbol pruning, or grammar correction is not performed in prompt-evolution systems, but differs in that the feedback the default pipeline. The exact prompt templates for LSF signal is used to evolve symbol inventories and constraints operations are provided in the Appendix B. rather than only instructions in natural-language. Diversity and positioning. We observe that a higher sam- High-leverage selection and mutation. Let an LSF- pling temperature naturally yields a diverse LSFs population, conditioned answer be a tuple (ŷ, c), where c is the gen- spanning a spectrum from strict LSFs (machine-like, com- erated token count. We prioritize the reasoning traces that pressed, rigid formats) to soft LSFs (closer to natural lan- are: (i) correct, meaning ŷ = y under the benchmark’s eval- guage but still symbolically structured). This diversity is cru- uator, and (ii) token-efficient, meaning that c is low relative cial for downstream selection and routing, as different sym- to other correct candidates for the same query. In practice, bolic protocols may dominate on different sub-distributions we implement selection via a Pareto criterion, i.e., accuracy of queries. While the generation loop superficially resem- first and then minimal tokens. The selected high-leverage bles iterative prompt discovery, the optimized object is not traces, together with the parent LSFs that produced them, an instruction string but a reusable symbolic language sys- are then used as context for the mutation step. The LLM is tem. LSFs are repeatedly invoked across many queries, asked to refine or recombine the parent symbolic protocol come with a compact grammar/contract, and can be selected so that it preserves the successful inference pattern while or composed by the router as a modular protocol. This removing redundant notation, ambiguous rules, or failure- distinguishes CLSR from prompt evolution / optimization prone conventions. Thus, mutation operates on the LSF methods (Yang et al., 2023; Fernando et al., 2024). definition rather than merely rewriting an output answer.
4
When LLMs Develop Languages
Elitist inheritance across generations. If certain LSFs con- 2.5. End-to-End Inference Procedure
sistently generate high-leverage answers, we allow them to
Given a test query x and an evolved LSF pool S, we:
survive unchanged into the next generation. In practice, the
(t)
best performing LSFs Sk are copied into S (t+1) without 1. Retrieve LSF descriptors and optionally a small sub-
modification, while the remaining population is filled with set of profile statistics.
mutated or recombined variants. This prevents the popu- 2. Router planning: the LLM outputs a protocol speci-
lation from losing strong LSFs due to stochastic mutation, fying which LSF(s) to use, whether to parallelize, and
while still allowing exploration. whether to run multiple rounds.
3. Execute protocol: generate intermediate responses as
LSF Profile: Cost–Accuracy Metadata for Routing. We
specified. All generated tokens across all rounds and
maintain an LSF’s profile summarizing empirical behavior
all invoked LSFs are counted toward the total cost.
over training rollouts. For each LSF Sk , we track: (i) Ac-
4. Aggregate to the final answer according to the router-
curacy estimate in evaluated groups, (ii) Token statistics
generated or self-consistency protocol.
(mean/median, tail behavior), (iii) Reliability indications
(variance between query types, failure modes), and (iv) Do- This yields an adaptive reasoning pattern: easy queries often
main tags inferred from the queries where Sk excel. These require a single low-cost LSF call (T =1), whereas diffi-
profiles are later consumed by a router to decide which cult queries may trigger multi-round LSF communication
LSF(s) to apply in inference. (T >1), trading extra tokens for higher reasoning reliability
and inference accuracy.
2.4. Test-time LSF Routing
CLSR moves beyond “choose one prompt” and instead 3. Theoretical Analysis
treats LSFs as a pool of symbolic protocols that can be This section provides a compact, theory-oriented view of
selected, ensembled, and composed per query. CLSR. The formal proofs are deferred to the Appendix E.
Latent-free LLM-routing policy To eliminate additional We also include a sociolinguistic interpretation of the evolu-
trainable routing networks and access to the latent states of tion of LSF, which explains why compact machine-oriented
an LLM or network, we can use an LLM-router by delegat- LSFs can emerge from repeated selection under an accuracy–
ing routing decisions to the same LLM. This is achieved efficiency pressure.
by (i) compressing each LSF’s “battle” profile into a short
description and (ii) prompting the LLM to synthesize a 3.1. Token-Accuracy Optimality
query-specific protocol for leveraging the LSF pool.
We formalize CLSR test-time inference as a constrained
LSF profile summarization. Offline, we ask the LLM to stochastic control problem. For a fixed test query x, an
produce a concise descriptor for each LSF conditioned on interactive inference policy π (routing, stopping, aggrega-
its empirical profile: what types of queries it tends to solve tion) induces a multi-round transcript T (all tokens gen-
well, typical token footprint, and common failure cases. erated across invoked LSFs/rounds) and a final prediction
Ŷ . We evaluate policies according to expected
token cost
Protocol planning. At test time, given a query x and the
JC (π) = E |T | and accuracy JA (π) = P Ŷ = Y , and
set of LSF descriptors, the LLM first decides among three
define the optimal accuracy within the budget B as
inference modes:
A∗ (B) = sup JA (π). (2)
π:JC (π)≤B
1. Single LSF direct answer: select one LSF and gener-
ate a final answer. Theorem 3.1. (Existence of an optimal interactive infer-
2. multi-LSF aggregation: select multiple LSFs to inde- ence policy). For a query x. Assume: (i) a finite LSF pool
pendently propose answers, then aggregate (e.g., ma- of size K; (ii) a finite (or effectively bounded) horizon Tmax
jority vote), similar to self-consistency. via a stopping action; (iii) the per-step token cost is nonneg-
3. Implicit LSF composition: for hard queries, the router ative and integrable; (iv) the LLM decoding under each LSF
specifies a multi-step protocol where responses from defines a well-posed stochastic kernel over the next tokens
many LSFs become a context for subsequent rounds. conditioned on the current history. Then for every budget
B ≥ 0, the supremum A∗ (B) is attained: there exists a
∗ ∗
possibly randomized policy πB such that JC (πB ) ≤ B and
Concretely, the router produces a plan: (i) the number of ∗ ∗
JA (πB ) = A (B).
rounds, (ii) which LSF(s) to invoke per round, (iii) how to
post-process intermediate responses, and (iv) when to stop. Moreover, for any Lagrange multiplier λ ≥ 0, there exists
See Appendix C for more details. an optimal deterministic finite-horizon policy πλ∗ that maxi-
5
When LLMs Develop Languages
mizes the objective E 1{Ŷ = Y } − λ|T | ; the boundary low-information narrative text while preserving symbols points of the Pareto frontier (JC , JA ) can be implemented that carry reusable reasoning states, such as variable bind- by some πλ∗ or a mixture of two adjacent multipliers. ings, subgoal markers, verification tags, and short operators. Theorem 3.2. (Lower bound on expected generated to- Second, evolution induces conventionalization. A symbol kens). Fix a query x and let Yx denote the finite set of survives not because it is intrinsically meaningful to humans, effective answers induced by the evaluator for x. Consider but because it repeatedly supports reliable inference for the any interactive CLSR policy π whose transcript T is gener- model population. Third, routing implements pragmatic ated token-by-token until a stopping action, and whose final code-switching. For easy queries, a strict and terse LSF answer is Yb = g(Th , x). Suppose π iachieves target accu- often suffices; for ambiguous or difficult queries, the router may select a more redundant LSF, aggregate multiple LSFs, racy α ∈ (0, 1): Pπ Yb = Y | X = x ≥ α. Let δ = 1 − α or invoke additional rounds for verification. and define the required information Accordingly, CLSR should be viewed as a test-time mecha- Ireq (x, δ) = H(Y | X = x) − h2 (δ) − δ log2 (|Yx | − 1) , nism for studying how LLM agents invent, select, and reuse (3) compact social conventions under an explicit communica- where h2 (·) is the binary entropy. Let κθ (x) be an upper tion bottleneck, rather than as a claim of human-independent bound on the conditional information revealed by one active language emergence. Appendix E.1 expands this connection generated token under the allowed LSF protocols: to least-effort and iterated-learning principles. κθ (x) = sup I(Y ; Zt | X = x, Z<t = h<t , |T | ≥ t) , π,t,h<t 3.3. Universality and Finite Programs (4) The CLSR protocol class can be viewed as a programmable where the supremum ranges over policies, time steps, and computation system: the router implements control flow reachable active histories. Then (which “expert language” speaks next and when to stop), max{Ireq (x, δ), 0} each LSF implements an instructive symbolic coding Eπ [|T | | X = x] ≥ . (5) κθ (x) regime, the transcript T is writable memory, and aggre- gation is an arbitrary computable readout. Next, we demon- The bound separates three factors. First, a higher target strate that the multi-round multi-LSF interaction is a natu- accuracy reduces δ and increases the Fano term, raising ral generalization of the “generate-a-program then execute” the information that the transcript must convey. Second, paradigms. Consider two families of inference procedures: harder queries have larger evaluator-induced uncertainty 1. Program-Execution Pipelines (PE). A well-formed H(Y | X = x) or larger effective answer classes Yx , in- PE procedure makes finite calls to the base LLM and creasing Ireq . Third, stronger symbolic protocols increase obtains an LLM-generated transcript T ∈ Σ (e.g., κθ (x) by packing more task-relevant information into each code/program plus any intermediate strings). Then generated token. In this sense, CLSR improves efficiency it applies a deterministic logic-only executor Exec : not by violating an information limit but by changing the Σ 7→ A that does not access external knowledge. representation so that each emitted token carries more useful 2. LSF-only CLSR Protocols. A policy π is a multi- reasoning state. The active-history definition of κθ (x) is round multi-LSF protocol that can adaptively choose important for adaptive stopping: a policy may continue only LSFs and stopping rules but does not invoke any exter- on difficult histories, so the per-token information rate must nal executor. It outputs ŷ solely from LLM generations. be conditionally bounded on the process still being active. Premise (Interpreter Realizability). For every determin- 3.2. A Sociolinguistic Lens on LSF Evolution istic computable executor Exec : Σ∗ 7→ A in the allowed executor class, there exists an LSF SExec such that, for any The empirical behavior of LSFs has a useful sociolinguis- input string s ∈ Σ∗ , one LLM call conditioned on SExec tic interpretation. CLSR does not create language from an and s produces Exec(s) with failure probability at most empty prior: the base LLM has already internalized hu- εint < 1/2 and with output length at most |Exec(s)| + c0 man linguistic conventions, mathematical notation, domain- for a constant c0 . This is an idealized internal-simulation specific abbreviations, and tokenizer-specific regularities. premise. It is supported in principle by universality re- Evolved LSFs are therefore best understood as machine- sults for Transformer-like architectures (Pérez et al., 2019; oriented languages: they are constrained by pretrained 2021), but it is not automatically guaranteed for any finite linguistic priors, but selected under a new objective that pretrained checkpoint. rewards correctness per generated token. Theorem 3.3. (Conditional subsumption of program ex- This lens explains three recurring patterns. First, success- ecution). Under interpreter realizability, consider any ful LSFs follow a principle of least effort: they remove program-execution pipeline PE that achieves accuracy of
6
When LLMs Develop Languages
Table 1. Comparison with Token-reduction approaches on the accuracy–token trade-off). Inference accuracy (%) and average completion tokens per problem for Raw CoT (Wei et al., 2022), CCoT (Nayab et al., 2024a), CoD (Xu et al., 2025b), SoT (Aytes et al., 2025a), and CLSR (ours) across seven benchmarks and four backbones (LLaMA3-8B, DeepSeek-R1-Qwen3-8B, Qwen3-8B, and Qwen3-32B). CLSR consistently improves the accuracy–token frontier, typically matching Raw CoT accuracy while reducing token usage by roughly 3/4, and outperforming length-controlled prompting baselines at comparable budgets.
MMLU- PRO GPQA- MAIN GSM8K MATH-500 AIME21-24 S CI -QA H OTPOT-QA
LLM S M ETHODS
ACC T KN ACC T KN ACC T KN ACC T KN ACC T KN ACC T KN ACC T KN
R AW C OT 38.6 960 30.4 1209 84.9 124 51.6 704 6.1 1705 61.3 51 45.1 32
LL A MA3-8B COD 35.8 352 26.4 386 80.5 51 42.2 280 4.9 1041 59.8 19 40.2 24 CC OT 37.2 370 29.3 450 83.6 65 45.6 315 5.8 1205 60.2 22 44.3 28 S OT 36.4 340 28.2 398 82.3 55 45.2 289 5.3 1051 61.2 19 45.8 21 CLSR ( OURS ) 38.9 320 30.8 362 84.3 52 48.2 257 6.2 1035 61.4 18 45.5 14 R AW C OT 71.1 1789 73.2 5380 92.2 433 91.2 3544 86.2 8190 71.3 125 55.8 213 D EEP S EEK -R1 -0528 COD 68.3 582 73.2 1892 86.5 204 80.5 1842 77.4 3545 68.4 37 51.2 56 -Q WEN 3-8B CC OT 64.5 516 70.4 1605 90.3 220 83.4 2032 79.6 3946 70.2 43 53.0 65 S OT 69.2 458 71.4 1536 91.8 205 84.1 1982 81.3 3266 71.0 41 54.8 59 CLSR ( OURS ) 70.3 356 73.1 1326 92.4 198 88.1 1753 82.5 2979 71.5 30 54.5 43 R AW C OT 60.2 276 49.1 1085 90.9 243 87.2 878 46.1 4234 77.8 75 66.4 134
Q WEN 3-8B COD 55.6 125 42.7 282 85.4 100 74.5 389 30.2 2610 76.2 46 63.2 58 CC OT 50.2 135 41.3 245 89.2 125 78.4 423 36.7 2805 78.2 49 65.2 70 S OT 58.5 118 45.2 228 90.3 107 81.3 294 38.0 2637 79.3 45 65.8 63 CLSR ( OURS ) 60.4 96 47.7 228 91.2 89 86.8 257 43.7 2361 77.8 35 66.4 48 R AW C OT 67.5 405 54.2 983 91.3 216 89.3 845 46.8 4572 79.2 81 68.4 165
Q WEN 3-32B COD 64.2 318 49.2 276 82.1 105 81.5 365 32.5 2912 77.0 53 64.6 78 CC OT 65.1 192 50.8 312 88.2 94 82.0 382 37.3 3025 79.1 56 65.8 81 S OT 64.3 134 51.4 247 90.5 112 83.8 278 38.5 2453 78.9 48 66.2 72 CLSR ( OURS ) 68.1 118 54.0 209 91.6 91 89.4 206 45.2 2038 80.2 39 68.8 55
at least α with an expected LLM-generated token cost of mathematical problem solving: (i) broad-domain factual and B. Let ℓ̄Exec (x) denote the expected length of the executor professional reasoning (MMLU-Pro (Wang et al., 2024)), output plus the constant formatting overhead under the PE (ii) expert-level science QA with strong adversarial diffi- transcript distribution. Then for any δ ∈ (0, 1), there exists culty (GPQA (Rein et al., 2024)), (iii) grade-school multi- an LSF-only CLSR protocol π such that step arithmetic (GSM8K), (iv) competition-style proof and derivation (MATH500 (Hendrycks et al., 2021) and AIME • Accuracy: Pπ [Yb = Y | X = x] ≥ α − δ; (21-24) (Veeraboina, 2023)), (v) science QA with short final • Token cost: JC (π) ≤ B + O ℓ̄Exec (x) log(1/δ) . answers (ScienceQA (Lu et al., 2022)), and (vi) multi-hop In short-answer reasoning benchmarks where ℓ̄Exec (x) = question answering (HotpotQA (Yang et al., 2018)). O(1), this reduces to JC (π) ≤ B + O(log(1/δ)). In the Backbone LLM-agents. To test generality across model exact-interpreter and bounded-output regime, the achiev- families and scales, we run our pipeline with multiple open able token–accuracy region of CLSR contains that of PE up LLM backbones, treating each checkpoint as an independent to an arbitrarily small consensus overhead. agent in language creation and/or inference. We report the When the assumption holds or fails. Interpreter realizabil- results for Qwen3-8B/32B (Yang et al., 2025), LLaMA3- ity is plausible when: (i) Exec is within the base model’s 8B (AI@Meta, 2024), and a distilled variant of DeepSeek- algorithmic competence (e.g., simple arithmetic, bounded R1 (0528-Qwen3-8B) (DeepSeek-AI, 2025). Throughout, symbolic manipulation), (ii) the required execution trace fits the backbone weights remain frozen; improvements come within the model’s effective context/attention constraints, purely from test-time symbolic protocols and routing. and (iii) decoding is sufficiently reliable (or can be stabilized Baselines. We compare test-time reasoning and token- by self-consistency). It may fail for executors that require an reduction prompting strategies: (i) Raw CoT (standard exact long-horizon computation or strict formal guaranties. chain-of-thought prompting) (Wei et al., 2022), (ii) CoD (Chain-of-Draft) (Xu et al., 2025b), (iii) CCoT (Constrained 4. Experiments CoT) (Nayab et al., 2024a), and (iv) SoT (Sketch-of- Thought) (Aytes et al., 2025a). We also compare program- 4.1. Experimental Setup based strategies: (v) PoT (Program-of-Thoughts prompt- ing) (Chen et al., 2023) and (vi) PAL (Program-aided Lan- Benchmarks. We evaluate symbolic-language reasoning guage) (Gao et al., 2023). Additionally, we compare prompt on seven widely used reasoning benchmarks that span optimization methods: (vii) Plan-to-Solve (Wang et al., knowledge-intensive QA, multi-hop retrieval-style QA, and
7
When LLMs Develop Languages
(a) Scientific benchmarks. (b) Mathematical benchmarks. (c) Hard math (AIME21-24).
Figure 2. Scaling generated tokens: CLSR dominates CoT under test-time scaling. We increase the number of generated tokens via test-time scaling (multiple samples with majority vote) and compare standard CoT against CLSR (LSF). The reported curves trace the empirical accuracy–token trade-off. CLSR achieves higher accuracy at a given token budget (or the same accuracy at fewer tokens) across (a) scientific QA benchmarks, (b) math-reasoning benchmarks, and (c) hard math (AIME21–24), supporting our theoretical framing that structured symbolic traces improve the information carried per generated token.
(a) #Agents @ MMLU-pro. (b) #Tokens @ MMLU-pro. (c) #Agents @ GPQA. (d) #Tokens @ GPQA. Figure 3. Scaling evolution agents improves LSF quality. We ablate the number of LLM agents participating in the offline LSF-evolution process (parallel proposal, critique, and refinement), and evaluate the resulting LSF pools on representative benchmarks. Across tasks, using more agents typically improves downstream accuracy, while also increasing (or modestly changing) generation cost.
2023) and (viii) PromptBreeder (Fernando et al., 2024). constrained grammar, and (iii) usage constraints that aim to All methods are evaluated on the same benchmark splits. reduce token footprint while preserving reasoning capability. Appendix A.6.4 reports a population-size ablation. We use Metrics (accuracy and token cost). For each benchmark, temperature 0.9 to generate LSFs and temperature 0.3 to we report: (i) inference accuracy (%) and (ii) the number evaluate the inference. of generated tokens emitted by the inference LLM and the LLM-router. For CLSR, this includes all tokens generated Evolutionary bootstrapping (generations). We iteratively online in selected LSFs, router plans, intermediate rounds, refine the LSF population via an evolutionary loop. At each and aggregation responses, which correspond to the cost evolution step, we evaluate candidate LSFs on a held-out definition in Section 2.1. We use generated tokens as the validation set, then select high-leverage inference traces main latency-oriented metric because the decode phase dom- that are simultaneously correct and token-efficient. These inates many reasoning workloads, whereas prompt-prefix selected traces are fed back to the LLM to synthesize the costs are highly serving-dependent. To explicitly address the next-generation LSF population. We use the fifth evolution input-overhead issue, Appendix A.8 reports a cache-aware LSF pool for the final test evaluation by default. token-equivalent metric that includes reusable LSF-card At inference time, we maintain a global pool by mixing prefixes and separates uncached input, cached input, and LSFs evolved from different benchmarks. Given a test query, output-token coefficients. the router may (i) select a single existing LSF, which is typ- ical for easier queries, (ii) select and aggregate multiple 4.2. Implementation Details LSFs for moderately difficult queries, or (iii) compose mul- LSF synthesis from exemplars. For each benchmark, we tiple existing LSFs across rounds for hard cases. By default, sample N ∈ [200, 2000] training exemplars that include the selection objective weights token length and accuracy ground-truth reasoning content. Given these exemplars in equally; the number of tokens of an LSF card ranges from context, we prompt the LLM backbone to invent an initial 500–2000 depending on task difficulty and protocol depth. population of hundreds of Language-Symbolism Frame- We use the LLM-router to determine the number of rounds works (LSFs), each specifying (i) a compact lexicon, (ii) a T required for the test problem. We can also fix T ={1, 3}
8
When LLMs Develop Languages
rather than letting it be determined by the router; see Table 3 Table 2. Comparison with program-execution and prompt optimization methods on Qwen3-8B. We compare CLSR with in Appendix A for more ablation studies. two program-execution pipelines: PoT (Program-of-Thoughts) System and compute. All experiments are conducted on and PAL (Program-aided Language), and two prompt optimization methods: P2S (Plan-to-Solve) and PBrd (PromptBreeder). For 8× NVIDIA RTX 4090 GPUs. Generating and evolving CLSR, T denotes the number of rounds. For PoT and PAL, the LSF populations takes roughly several days in total. This is reported tokens count only tokens used to generate the program. a one-time offline protocol-discovery cost, analogous to the pre-deployment search cost in automatic prompt/protocol GSM8K MATH500 GPQA optimization methods. CLSR is intended for settings where M ETHODS ACC T KN ACC T KN ACC T KN the evolved LSF pool is reused across many downstream C OT 91.6 248 87.8 905 43.8 973 queries, so the offline cost can be amortized over serving- P OT 92.1 113 88.1 375 / / time inference. The online routing/LSF-selection stage takes PAL 92.5 148 88.6 422 / /
< 1 seconds per query and is small relative to LLM decod- P2S 92.2 316 88.9 1205 45.2 1085 PB RD 93.6 235 88.4 843 46.1 922 ing. See Appendix A.8 for details. CLSR 92.8 90 86.8 257 42.6 228 CLSR (T =1) 92.1 83 86.2 134 40.9 182 4.3. Main Results CLSR (T =3) 94.8 214 89.7 417 49.2 565
Tables 1-3 show the central empirical pattern: CLSR con- and prompt-optimization baselines under matched settings. sistently improves the accuracy–token frontier across back- CLSR remains competitive without relying on an external bones and benchmarks. Compared with Raw CoT, CLSR executor, and it also improves upon prompt-optimization uses substantially fewer completion tokens while preserving, baselines. These results support the Section 3: CLSR is and in many cases slightly improving, accuracy. Compared not a fixed brevity heuristic, but an adaptive policy over with direct token-reduction prompting baselines, our CLSR the accuracy–token frontier whose multi-round LSF proto- is more stable on the Pareto frontier: aggressive compres- cols can approximate program-like computation when the sion baselines often save tokens by sacrificing correctness, required symbolic operations remain within the model’s whereas CLSR retains a reusable symbolic protocol and internal competence. routes among LSFs according to query difficulty. The gains are not uniform token shortening; they reflect 4.4. Ablations and Analysis adaptive allocation of reasoning effort. On knowledge- We ablate the following design choices: (i) evolution depth intensive QA benchmarks such as MMLU-Pro and GPQA, (Sec. A.5.1), (ii) exemplar count (Sec. A.5.2), (iii) agent much of the natural-language rationale can be compressed count and length-coefficient (Sec. A.5.3), (iv) number of into compact symbolic state without a large accuracy loss. multi-rounds T (Sec. A.5.4), (v) cross-LLM transfer via On deliberation-heavy mathematical benchmarks such as swapping the LSF generator (Sec. A.5.5), and (vi) qualita- MATH500 and AIME, the router often benefits from select- tive examples (Sec. A.7). We also report robustness studies ing stricter LSFs or invoking additional rounds, because in Appendix A.6, including seed stability, population-size the task requires multi-step transformation and verification. sensitivity, larger-model inference, long-context pilot evalu- For shorter-answer datasets, the absolute completion length ation, category/full-pool routing, cross-domain transfer, and is already small, so the token savings are naturally more cache-aware token accounting. limited, but CLSR remains competitive in accuracy. The figures further explain the mechanism behind the ag- 5. Conclusion and Limitations gregate results. Figure 2 shows that CLSR scales more favorably with additional test-time tokens than standard We introduced CLSR, a test-time framework that lets LLMs CoT, indicating that structured symbolic traces use extra invent, evolve, and route among compact symbolic com- budget more efficiently than repeated natural-language sam- munication protocols. Empirically, CLSR improves the pling. Figure 3 shows that increasing the number of LSF- accuracy–efficiency frontier across diverse reasoning bench- generating agents improves the final language pool, consis- marks, with robustness checks showing stable gains. Con- tent with broader LSF exploration and stronger selection ceptually, CLSR suggests that LLMs can develop machine- pressure. Figure 4 shows that deeper evolution improves oriented languages that are not manually crafted by humans. the accuracy–token ratio before saturation, suggesting that However, the offline language-evolution can be computa- the evolutionary loop is not merely shortening outputs but tionally expensive, and the synthesis pipeline requires train- selecting conventions that preserve more task-relevant infor- ing exemplars with reasoning content. Future work should mation per generated token. study how symbolic languages transfer across modalities, tools, and multi-agent environments. Finally, Table 2 compares CLSR with program-execution
9
When LLMs Develop Languages
Acknowledgements Chen, Q., Qin, L., Liu, J., Peng, D., Guan, J., Wang, P., Hu, M., Zhou, Y., Gao, T., and Che, W. Towards reasoning This work was supported in part by the National Key R&D era: A survey of long chain-of-thought for reasoning Program of China under Grant 2023YFC2508704, in part large language models. arXiv preprint arXiv:2503.09567, by the National Natural Science Foundation of China under 2025. grant number 62236008, in part by the Natural Science Foundation of Beijing under grant number L251082, and in Chen, W., Ma, X., Wang, X., and Cohen, W. W. Program part by Shandong Provincial Natural Science Foundation of thoughts prompting: Disentangling computation from under project ZR2025ZD01. reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023. Impact Statement Cheng, J. and Van Durme, B. Compressed chain of thought: This work aims to improve the efficiency of LLM reason- Efficient reasoning through dense representations. arXiv ing by reducing unnecessary generated tokens while pre- preprint arXiv:2412.13171, 2024. serving task performance. The main potential risk is that compact symbolic traces may be less directly interpretable Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., than natural-language rationales, which could make human Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, auditing more difficult in real-world deployments. A prac- R., et al. Training verifiers to solve math word problems. tical mitigation is to use CLSR with larger LLMs for inter- arXiv preprint arXiv:2110.14168, 2021. nal reasoning while requiring the final system to output a human-readable explanation using smaller LLMs. We do DeepSeek-AI. Deepseek-r1: Incentivizing reasoning ca- not introduce new datasets containing personally identifi- pability in llms via reinforcement learning, 2025. URL able information or new model-training procedures beyond https://arxiv.org/abs/2501.12948. black-box LLM prompting and evaluation. Fernando, C., Banarse, D. S., Michalewski, H., Osindero, S., References and Rocktäschel, T. Promptbreeder: Self-referential self- improvement via prompt evolution. In International Con- AI@Meta. Llama 3 model card. none, 2024. URL ference on Machine Learning, pp. 13481–13544. PMLR, https://github.com/meta-llama/llama3/ 2024. blob/main/MODEL_CARD.md. Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Arora, D. and Zanette, A. Training language models to Y., Callan, J., and Neubig, G. Pal: Program-aided lan- reason efficiently. CoRR, 2025. guage models. In International Conference on Machine Aytes, S. A., Baek, J., and Hwang, S. J. Sketch-of-thought: Learning, pp. 10764–10799. PMLR, 2023. Efficient llm reasoning with adaptive cognitive-inspired sketching. arXiv preprint arXiv:2503.05179, 2025a. Gehring, J., Zheng, K., Copet, J., Mella, V., Cohen, T., and Synnaeve, G. Rlef: Grounding code llms in execution Aytes, S. A., Baek, J., and Hwang, S. J. Sketch-of-thought: feedback with reinforcement learning. In Forty-second Efficient llm reasoning with adaptive cognitive-inspired International Conference on Machine Learning, 2025. sketching. arXiv preprint arXiv:2503.05179, 2025b. Grand, G., Wong, L., Bowers, M., Olausson, T. X., Liu, Barez, F., Wu, T.-Y., Arcuschin, I., Lan, M., Wang, V., M., Tenenbaum, J. B., and Andreas, J. Lilo: Learning Siegel, N., Collignon, N., Neo, C., Lee, I., Paren, A., interpretable libraries by compressing and documenting et al. Chain-of-thought is not explainability. Preprint, code. In The Twelfth International Conference on Learn- alphaXiv, pp. v1, 2025. ing Representations, 2024. Beiser, A., Penz, D., and Musliu, N. Intermediate languages matter: Formal choice drives neurosymbolic llm reason- Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, ing. arXiv preprint arXiv:2502.17216, 2025. S., Tang, E., Song, D., and Steinhardt, J. Measuring math- ematical problem solving with the math dataset. arXiv Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Pod- preprint arXiv:2103.03874, 2021. stawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. Graph of thoughts: Kanwal, J., Smith, K., Culbertson, J., and Kirby, S. Zipf’s Solving elaborate problems with large language models. law of abbreviation and the principle of least effort: Lan- In Proceedings of the AAAI conference on artificial intel- guage users optimise a miniature lexicon for efficient ligence, volume 38, pp. 17682–17690, 2024. communication. Cognition, 165:45–52, 2017.
10
When LLMs Develop Languages
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Pérez, J., Barceló, P., and Marinkovic, J. Attention is turing- Y. Large language models are zero-shot reasoners. Ad- complete. Journal of Machine Learning Research, 22 vances in neural information processing systems, 35: (75):1–35, 2021. 22199–22213, 2022. Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, Kouwenhoven, T., Peeperkorn, M., and Verhoef, T. Search- R. Y., Dirani, J., Michael, J., and Bowman, S. R. Gpqa: ing for structure: Investigating emergent communication A graduate-level google-proof q&a benchmark. arXiv with large language models. In Proceedings of the 31st preprint arXiv:2311.12022, 2023. International Conference on Computational Linguistics, pp. 9977–9991, 2025. Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., and Bowman, S. R. Gpqa: A Lazaridou, A. and Baroni, M. Emergent multi-agent com- graduate-level google-proof q&a benchmark. In First munication in the deep learning era. arXiv preprint Conference on Language Modeling, 2024. arXiv:2006.02419, 2020. Schick, T., Dwivedi-Yu, J., Dessı̀, R., Raileanu, R., Lomeli, Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and S.-C., Tafjord, O., Clark, P., and Kalyan, A. Learn to Scialom, T. Toolformer: Language models can teach explain: Multimodal reasoning via thought chains for themselves to use tools. Advances in Neural Information science question answering. Advances in Neural Infor- Processing Systems, 36:68539–68551, 2023. mation Processing Systems, 35:2507–2521, 2022. Sel, B., Tawaha, A., Khattar, V., Jia, R., and Jin, M. Al- Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, gorithm of thoughts: Enhancing exploration of ideas in L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., large language models. In International Conference on Yang, Y., et al. Self-refine: Iterative refinement with self- Machine Learning, pp. 44136–44189. PMLR, 2024. feedback. Advances in Neural Information Processing Systems, 36:46534–46594, 2023. Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: Language agents with verbal rein- Nayab, S., Rossolini, G., Buttazzo, G. C., Manes, N., and forcement learning. Advances in Neural Information Giacomelli, F. Concise thoughts: Impact of output length Processing Systems, 36:8634–8652, 2023. on llm reasoning and cost. CoRR, 2024a. Sprague, Z. R., Yin, F., Rodriguez, J. D., Jiang, D., Wad- Nayab, S., Rossolini, G., Simoni, M., Saracino, A., But- hwa, M., Singhal, P., Zhao, X., Ye, X., Mahowald, K., tazzo, G., Manes, N., and Giacomelli, F. Concise and Durrett, G. To cot or not to cot? chain-of-thought thoughts: Impact of output length on llm reasoning and helps mainly on math and symbolic reasoning. In The cost. arXiv preprint arXiv:2407.19825, 2024b. Thirteenth International Conference on Learning Repre- Pei, Z. and Wang, S. Dynamics-inspired neuromorphic vi- sentations, 2025. sual representation learning. In International Conference Tanmay, K., Aggarwal, K., Liang, P. P., and Mukher- on Machine Learning, pp. 27521–27541. PMLR, 2023. jee, S. Orion: Teaching language models to reason Pei, Z., Zhang, A., Wang, S., and Huang, Q. Modeling efficiently in the language of thought. arXiv preprint language tokens as functionals of semantic fields. In In- arXiv:2511.22891, 2025. ternational Conference on Machine Learning, pp. 40114– Tucker, M., Levy, R., Shah, J. A., and Zaslavsky, N. Trading 40128. PMLR, 2024a. off utility, informativeness, and complexity in emergent Pei, Z., Zhang, A., Wang, S., Ji, X., and Huang, Q. Data-free communication. Advances in neural information process- neural representation compression with riemannian neu- ing systems, 35:22214–22228, 2022. ral dynamics. In International Conference on Machine Learning, pp. 40129–40144. PMLR, 2024b. Turpin, M., Michael, J., Perez, E., and Bowman, S. Lan- guage models don’t always say what they think: Un- Pei, Z., Huang, Q., and Wang, S. Neuronal group communi- faithful explanations in chain-of-thought prompting. Ad- cation for efficient neural representation. arXiv preprint vances in Neural Information Processing Systems, 36: arXiv:2510.16851, 2025. 74952–74965, 2023.
Pérez, J., Marinković, J., and Barceló, P. On the turing Veeraboina, H. Aime problem set 1983- completeness of modern neural network architectures. In 2024, 2023. URL https://www.kaggle. International Conference on Learning Representations, com/datasets/hemishveeraboina/ 2019. aime-problem-set-1983-2024.
11
When LLMs Develop Languages
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Lim, E.-P. Plan-and-solve prompting: Improving and Narasimhan, K. Tree of thoughts: Deliberate problem zero-shot chain-of-thought reasoning by large language solving with large language models. Advances in neural models. In Proceedings of the 61st Annual Meeting of information processing systems, 36:11809–11822, 2023. the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2609–2634, 2023. Yu, F., Zhang, H., Tiwari, P., and Wang, B. Natural language reasoning, a survey. ACM Computing Surveys, 56(12): Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Chi, 1–39, 2024. E. H., Narang, S., Chowdhery, A., and Zhou, D. Self- consistency improves chain of thought reasoning in lan- Zhang, W., Zhang, M., Sun, X., and Mo, J. Reconstruction guage models. In The Eleventh International Conference of three-dimensional sound field system based on ma- on Learning Representations, 2025a. chine learning method. Electronics Letters, 61(1):e70133, 2025a. Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., et al. Mmlu-pro: Zhang, X., Cao, J., You, C., and Ding, D. Why prompt de- A more robust and challenging multi-task language un- sign matters and works: A complexity analysis of prompt derstanding benchmark. Advances in Neural Information search space in llms. In Proceedings of the 63rd Annual Processing Systems, 37:95266–95290, 2024. Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 32525–32555, 2025b. Wang, Y., Luo, H., Yao, H., Huang, T., He, H., Liu, R., Tan, N., Huang, J., Cao, X., Tao, D., et al. R1-compress: Long Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., chain-of-thought compression via chunk compression and Schuurmans, D., Cui, C., Bousquet, O., Le, Q. V., et al. search. arXiv preprint arXiv:2505.16838, 2025b. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, Conference on Learning Representations, 2022. E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Zhou, P., Pujara, J., Ren, X., Chen, X., Cheng, H.-T., Le, neural information processing systems, 35:24824–24837, Q. V., Chi, E., Zhou, D., Mishra, S., and Zheng, H. S. Self- 2022. discover: Large language models self-compose reasoning Xu, K., Zhang, K., Huang, W., Li, J., and Wang, Y. Rlg- structures. Advances in Neural Information Processing rag: Guiding the knowledge retrieval and evaluation in Systems, 37:126032–126058, 2024. retrieval-augmented generation framework by reasoning logic. In Companion Proceedings of the ACM on Web Conference 2025, pp. 1450–1454, 2025a. Xu, S., Xie, W., Zhao, L., and He, P. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600, 2025b. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. In The Twelfth International Conference on Learning Repre- sentations, 2023. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhut- dinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing, pp. 2369–2380, 2018. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, 2022.
12
When LLMs Develop Languages
A. Appendix: more empirical results
(a) Inference Accuracy. (b) Number of generated tokens. (c) Ratio of Accuracy and #tokens.
Figure 4. Effect of evolution depth on the accuracy–token frontier. We vary the number of language-evolution iterations used to refine the LSF pool, while holding the backbone fixed. N denotes the number of training exemplars used for LSF synthesis. We report (a) test accuracy, (b) average generated completion tokens per problem, and (c) an aggregated efficiency score. Increasing the evolution depth improves both accuracy and token efficiency and then saturates, consistent with the view that better-evolved LSFs increase the effective per-token information rate predicted by Theorem 3.2.
(a) No.Agents versus No.tokens. (b) No.Agents versus Accuracy. (c) No.tokens versus Accuracy.
Figure 5. Effects of adding more LLM agents on MMLU-pro.
(a) No.Agents versus No.tokens. (b) No.Agents versus Accuracy. (c) No.tokens versus Accuracy.
Figure 6. Effects of adding more LLM agents on GPQA-main.
A.1. Comparison to length-controlled prompting baselines. Compared with Constrained CoT (CCoT), Chain-of-Draft (CoD), and Sketch-of-Thought (SoT), CLSR achieves a stronger overall trade-off: (i) relative to CCoT, CLSR improves accuracy while also using fewer tokens (since CCoT removes verbosity but does not introduce a task-adaptive symbolic code); (ii) relative to CoD, CLSR substantially improves accuracy at a modest token increase, indicating that extreme compression alone is not sufficient, i.e., the intermediate representation must remain information-dense and compositional; (iii) relative to SoT, CLSR achieves higher accuracy with a similar token budget, suggesting that our evolutionary search over LSFs discovers more task-specialized symbolic operators than a fixed handcrafted constraint family.
13
When LLMs Develop Languages
(a) No.Agents versus No.tokens. (b) No.Agents versus Accuracy. (c) No.tokens versus Accuracy.
Figure 7. Effects of adding more LLM agents on MATH500.
(a) No.Agents versus No.tokens. (b) No.Agents versus Accuracy. (c) No.tokens versus Accuracy.
Figure 8. Effects of adding more LLM agents on Science-QA.
A.2. Interpretation through the theory lens. Section 3 predicts that, for a fixed target accuracy, token cost is lower-bounded by a ratio E[|T |] ≥ Ireq (x, δ)/κθ (x). CLSR improves the empirical cost–accuracy curve in a manner consistent with increasing κθ (x): LSF traces replace low-information narrative text with compact, state-carrying symbols (operators, bindings, and verification tags) that better preserve the computable parts of reasoning. Moreover, the robust token savings across backbones indicate that CLSR is not merely exploiting idiosyncrasies of a particular model but is instead leveraging a representation-level advantage.
A.3. Gains are most pronounced on deliberation-heavy problems. The largest accuracy advantages emerge on benchmarks that require multi-step reasoning and/or verification (notably MATH500 and AIME21–24). For example, CLSR improves AIME21–24 accuracy consistently across backbones while reducing tokens drastically relative to Raw CoT. This pattern aligns with the lower-bound perspective of Theorem 3.2: as task difficulty increases (higher effective uncertainty in the target solution), the required information Ireq (x, δ) rises, making naive verbosity expensive; CLSR responds by increasing the effective per-token information rate κθ (x) via a structured symbolic alphabet and reusable operators, thereby achieving the same (or higher) accuracy with fewer tokens.
A.4. Comparison with program-execution pipelines. Table 2 compares CLSR with two canonical program baselines, e.g., PoT (Chen et al., 2023) and PAL (Gao et al., 2023), which prompt the LLM to emit executable code and delegate the final computation to an external interpreter/runtime. Both PoT and PAL are known to improve arithmetic/symbolic reliability by offloading computation outside the LLM, i.e., “generate a program then execute”. On Qwen3-8B, PoT/PAL indeed outperform standard CoT on GSM8K and MATH500 at substantially lower LLM decoding tokens. However, CLSR can match or exceed these gains without any external executor: with a small majority-vote ensemble (T = 3), CLSR reaches 94.8% on GSM8K and 89.7% on MATH500 using 214 and 417 tokens, respectively, exceeding both PoT and PAL in accuracy–token frontier. Importantly, Table 2 reports PoT/PAL token counts only for program generation; their end-to-end deployment cost also includes external execution time and tool-chain overhead. From the theoretical lens in Section 3, these results empirically support our claim: under interpreter realizability, CLSR’s multi-round multi-LSF protocols can internally emulate the effect of a deterministic executor (Theorem 3.3). In
14
When LLMs Develop Languages
practice, CLSR provides a purely in-model alternative to PoT/PAL that preserves their accuracy benefits while avoiding dependency on external runtimes and execution-layer engineering.
A.5. Ablation studies A.5.1. E FFECT OF EVOLUTION DEPTH . As shown in Fig. 4, we ablate the evolution depth, i.e., the number of recursive propose→critique→mutate→select rounds used to build the LSF pool offline. Increasing the depth consistently improves downstream accuracy at a fixed token budget, indicating that iterative refinement produces more compressive and task-aligned symbolic protocols rather than merely longer rationales. Practically, the gain tends to saturate after a small number of rounds: once the pool contains a few high-leverage LSF “families”, additional rounds mostly perform local polishing (marginal accuracy improvements) while increasing offline cost. Overall, deeper evolution strengthens the LSF pool by amplifying selection pressure toward high-information tokens, but exhibits diminishing returns beyond a moderate depth. This also supports the core hypothesis of cultural selection among LSFs: iteratively retaining high-leverage traces (correct & concise) biases the language pool toward more compressive, reusable abstractions.
A.5.2. E FFECT OF EXEMPLAR COUNT. As shown in Fig. 4, we vary the generation batch size, i.e., the number of candidate LSFs proposed per generation, to characterize the exploration–efficiency trade-off in offline search. A larger batch improves diversity and reduces the probability that evolution prematurely converges to a suboptimal symbolic convention, which in turn increases the chance that the final pool contains a strong specialist for each query type. For larger backbones (e.g., moving from 4B to 14B), increasing N yields diminishing marginal returns in both accuracy and token reductions, suggesting that higher-capacity models can infer a useful symbolic protocol from fewer examples, while smaller models benefit more from larger set of exemplars. In general, batch size mainly controls exploration; use it to avoid early collapse but expect diminishing returns once coverage of distinct LSF modes is achieved.
A.5.3. E FFECT OF AGENTS COUNT AND LENGTH REGULARIZATION . As presented in Figure 3 and Appendix Figures 5–8, we study (i) the number of agents participating in evolution and (ii) the length-coefficient that penalizes verbose intermediate representations during selection. More agents typically improve the pool because it increases stylistic and algorithmic diversity, enabling stronger cross-agent critique and yielding more robust LSFs that generalize across query distributions. Meanwhile, the length-coefficient directly tunes the accuracy–token Pareto point: increasing it encourages shorter, higher-density symbolic traces (fewer tokens), but can under-allocate “reasoning bandwidth” on hard instances if set too aggressively. Overall, agent scaling primarily boosts search breadth and critique quality, whereas the length-coefficient provides a clean knob for token efficiency that should be set moderately to avoid sacrificing hard-case accuracy.
A.5.4. A FIXED NUMBER OF LSF- ROUNDS T OR NOT CLSR applies the same latent-symbolic refinement operator for multiple rounds. A natural design choice is whether the number of rounds T should be fixed for all inputs, or adaptive (instance-dependent). From the perspective of our theoretical analysis (Section 3), T acts as an inference-time compute budget: each additional round consumes extra generated tokens, but can reduce the residual error by enabling further compression–verification-correction cycles. The optimal policy is therefore generally not a single global T , but an adaptive stopping rule that continues refinement only when the expected marginal gain outweighs the marginal token cost. Formally, let x be the input, y the ground-truth answer, and let st denote the internal state of the CLSR after round t (e.g., the current latent symbolic form and its distribution of induced answers). Each round incurs an expected token cost c(st ) and yields an expected utility improvement ∆u(st ). A standard Lagrangian view of the accuracy–token trade-off is
h TX
(π,x) i
max E u(ŷ, y) − λ E c(st ) , (6)
π
t=1
where π is a stopping policy deciding whether to continue at each st . This objective implies an “optimal” behavior that is instance-adaptive: easy problems should stop early (small T ) to avoid wasting tokens, whereas hard problems may require
15
When LLMs Develop Languages
additional rounds.
Table 3. Varying the number of CLSR rounds T on Qwen3-8B. We compare standard CoT with CLSR under (i) fixed T = 1 (single refinement), (ii) fixed T = 3 (always run three rounds), and (iii) unfixed T with Tmax = 3 (a learned instance-adaptive stopping rule). Accuracy (%) is reported on GSM8K, MATH500, GPQA-main, AIME (2021-2024), and MMLU-pro, along with the average completion tokens per problem. Adaptive CLSR attains the best accuracy with only a modest token overhead relative to fixed-T settings, and it stops early on most instances, demonstrating effective instance-wise compute allocation.
GSM8K MATH500 GPQA AIME MMLU
ACC T KN ACC T KN ACC T KN ACC T KN ACC T KN
C OT 91.6 248 87.8 905 43.8 973 46.6 4502 63.2 272
CLSR ( DEFAULT ) 92.8 90 86.8 257 42.6 228 44.7 2361 63.6 93
CLSR (T =1) 92.1 83 86.2 134 40.9 182 38.4 1047 62.1 85
CLSR (T =3) 94.8 214 89.7 417 49.2 565 46.8 3314 67.2 257
Empirically, Table 3 validates this prediction in Qwen3-8B. Compared with N -shot CoT, CLSR improves both accuracy and token efficiency. More importantly, unfixed CLSR (adaptive T , with Tmax = 3) achieves the best overall accuracy across MATH500 and AIME benchmarks, while keeping the average completion length close to fixed-T settings. In the same experiment, the adaptive policy chooses T = 1 for a majority of cases, indicating that CLSR indeed learns to stop early when additional refinement is unnecessary, but retains the option to allocate more rounds to difficult inputs. We also observe that for a harder benchmark (e.g., AIME), the default CLSR (unfixed T , the LLM-router can determine the value of T based on the test problem) uses fewer cases of T = 1, since its corresponding T = 1 has fewer tokens compared to the default setting. In general, these results support the core claim of Section 3: CLSR is best viewed as an adaptive compute allocation mechanism, rather than a fixed-length prompting recipe.
A.5.5. E FFECT OF S WAPPING THE LSF G ENERATOR (C ROSS -M ODEL LSF T RANSFER ) Table 4 studies cross-model transfer of induced LSF: we evolve the LSF pool using a generator backbone Fgen but execute CLSR using another inference backbone Finf . Across GPQA, MATH500, and GSM8K, the matched diagonal (Fgen =Finf ) is consistently the best in accuracy, while swapping typically reduces tokens but degrades accuracy. Notably, using a stronger generator with a weaker inference model reduces tokens with only mild drops, whereas using a weaker generator with a stronger inference model incurs a larger accuracy penalty; the sensitivity is most pronounced on the knowledge-intensive GPQA benchmark.
Table 4. Swapping the LSF generator (cross-model LSF transfer). For each inference LLM, we evolve the LSF pool using the specified generator and then run CLSR inference with the inference model. We report inference accuracy (Acc) and average completion tokens (Tkn) on GPQA, MATH500, and GSM8K (tokens include the full executed CLSR protocol). The matched diagonal yields the best accuracy, while swapping generally reduces tokens but degrades accuracy; the degradation is most pronounced on GPQA and is especially severe when the generator is weaker than the inference model.
I NFERENCE LSF GPQA MATH500 GSM8K
LLM G ENERATOR
ACC T KN ACC T KN ACC T KN
Q WEN 3-4B Q WEN 3-4B ( DEFAULT ) 45.1 329 83.2 273 90.3 93
Q WEN 3-8B 43.5 254 82.6 195 90.1 86
Q WEN 3-8B Q WEN 3-8B ( DEFAULT ) 47.7 228 86.8 257 91.2 89
Q WEN 3-4B 42.1 172 81.8 183 89.2 84
This pattern aligns with our theory view in Section 3: CLSR gains come from allocating a limited token budget to a symbolic protocol whose intermediate symbols must be reliably interpretable by the inference model. Swapping Fgen changes the induced codebook (operators/abbreviations/constraints), which can increase a “dialect mismatch” between the protocol and the decoder implemented by Finf ; the result is lower token usage but reduced effective information per token, hence lower accuracy. Practically, the results suggest evolving LSFs with the same backbone as inference when accuracy is the priority, while cross-model transfer is feasible but generally Pareto-suboptimal, especially when the generator is weaker than the inference model.
16
When LLMs Develop Languages
Table 5. Seed stability. Accuracy and generated tokens are reported as Acc. (No. tokens). CLSR reports mean ± std over five random seeds and exemplar samplings.
BACKBONE M ETHOD MMLU-P RO GPQA- MAIN MATH500
C OT 38.6 (960) 30.4 (1209) 51.6 (704)
LL A MA3-8B S OT 36.4 (340) 28.2 (398) 45.2 (289)
CLSR 39.2 ± 0.5 (316 ± 13) 30.8 ± 0.2 (352 ± 18) 48.8 ± 0.7 (262 ± 11)
C OT 60.2 (276) 49.1 (1085) 87.2 (878)
Q WEN 3-8B S OT 58.5 (118) 45.2 (228) 81.3 (294)
CLSR 60.6 ± 0.4 (98 ± 14) 47.9 ± 0.4 (216 ± 12) 86.8 ± 0.6 (247 ± 10)
Table 6. Larger-backbone inference. Accuracy and generated tokens are reported as Acc. (No. tokens). Results are averaged over three runs.
BACKBONE M ETHOD MMLU-P RO GPQA- MAIN MATH500
C OT 67.5 (405) 54.2 (982) 89.3 (845)
Q WEN 3-32B S OT 64.3 (134) 51.4 (247) 83.8 (278)
CLSR 68.1 (90) 54.0 (209) 89.4 (206)
C OT 49.2 (1020) 41.3 (1358) 64.3 (725)
LL A MA3.1-70B S OT 47.5 (385) 38.4 (425) 59.8 (314)
CLSR 49.8 (312) 42.7 (354) 63.4 (242)
A.6. Robustness Checks This section reports additional robustness checks to clarify stability, scale transfer, routing dependence, and practical token accounting.
A.6.1. S EED STABILITY AND STOCHASTIC MUTATION Table 5 reports mean ± std over five random seeds and exemplar samplings. Different runs discover different surface LSFs, but the accuracy–token frontier remains stable.
A.6.2. L ARGER INFERENCE BACKBONES Table 6 evaluates larger inference backbones using LSFs discovered by Qwen3-8B. The gains persist even when the LSF generator is smaller than the inference model, suggesting that LSFs can transfer as reusable symbolic protocols rather than being tied to one specific checkpoint.
A.6.3. L ONG - CONTEXT PILOT We also run a small pilot on LongBench-v2 to test whether symbolic protocols remain useful when the input context is long. The results in Table 7 are encouraging, but we treat this as preliminary evidence only. Long-context search agents and repository-level coding require more specialized evaluation and are left for future work.
A.6.4. P OPULATION - SIZE SENSITIVITY The default initial population size is 100 LSFs. Table 8 shows that this is an empirical exploration/compute trade-off. A very small population lacks LSF diversity, while larger populations yield diminishing returns and higher cold-start cost.
A.6.5. C ATEGORY ROUTING VERSUS FULL - POOL ROUTING Category routing is a lightweight guiding mechanism rather than a hard partition of capability. Table 9 compares category- routed CLSR with full-pool routing, where the router can select from the entire LSF bank without a pre-specified category filter. Full-pool routing preserves the qualitative gain, indicating that CLSR does not depend on a rigid manually defined taxonomy.
A.6.6. C ROSS - DOMAIN LSF TRANSFER Table 10 compares in-domain, all-domain, and leave-one-domain-out LSF banks. The results indicate that CLSR benefits are not tied to a single domain-specific LSF. All-domain pooling can slightly improve robustness, while leave-one-domain-out
17
When LLMs Develop Languages
Table 7. Long-context pilot. Overall score and average generated tokens are reported.
BACKBONE M ETHOD OVERALL AVG . T OKENS
C OT 30.8 1068
LL A MA3-8B
CLSR 31.5 568
C OT 38.2 842
Q WEN 3-8B
CLSR 39.1 432
Table 8. Initial LSF population size. Accuracy and generated tokens are reported as Acc. (No. tokens).
BACKBONE I NITIAL LSF S MMLU-P RO GPQA- MAIN MATH500
10 38.1 (354) 30.2 (425) 47.4 (312)
50 38.6 (330) 30.4 (380) 48.0 (267)
LL A MA3-8B
100 38.9 (320) 30.8 (362) 48.2 (257)
200 39.2 (308) 30.9 (358) 48.2 (255)
10 59.2 (103) 46.6 (234) 85.4 (275)
50 60.1 (102) 47.3 (225) 86.4 (260)
Q WEN 3-8B
100 60.4 (96) 47.7 (228) 86.8 (257)
200 60.6 (93) 47.8 (232) 86.6 (248)
routing still preserves most gains.
A.7. Qualitative examples: what changes in the trace. Appendix Figures 9–14 provide representative outputs of the GPQA, MATH500, and AIME.benchmarks. Compared to verbose CoT, our CLSR (LSF) traces replace long natural-language rationales with compact symbolic operators and structured templates that preserve key intermediate state (e.g., variable bindings, sub-goal markers, and verification tags) while discarding redundant narration. We also show the auto-selected LSFs specification used for each query, illustrating that (i) the router selects different LSFs by domain and difficulty, and (ii) the selected LSFs are reusable artifacts rather than one-off prompt rewrites. This section provides qualitative evidence for how CLSR modifies the reasoning trace across refinement rounds. Figures 9–14 show representative model outputs for CoT and CLSR as well as the intermediate LSF produced by CLSR. We highlight several recurring phenomena.
From narrative CoT to task-relevant latent symbolic form. A consistent difference is that CoT tends to produce verbose narrative intermediate text that mixes (i) problem understanding, (ii) bookkeeping, and (iii) arithmetic/logical execution. In contrast, CLSR compresses the intermediate reasoning into a compact LSF that is closer to a minimal sufficient representation to solve the instance: it preserves variable bindings, constraints, and key transformation steps while discarding stylistic and redundant natural-language content. This aligns with the goal of CLSR: to shift reasoning toward a higher signal-to-token ratio without relying on an external executor.
Error localization and correction across rounds. Across examples, later rounds frequently correct earlier-round failure modes that are common in plain CoT: (i) missed constraints (e.g., forgetting a boundary condition or a unit conversion), (ii) inconsistent variable definitions, (iii) arithmetic slips that propagate in long traces, and (iv) premature commitment to an incorrect intermediate value. The intermediate LSF makes such issues easier to localize: the model can explicitly re-check constraint satisfaction or recompute a small symbolic sub-part, instead of re-generating a long narrative chain.
Refinement is often selective rather than additive. Importantly, CLSR does not merely append more tokens. Later- round traces often replace earlier representations with shorter, cleaner ones: irrelevant branches are removed, equivalent expressions are simplified, and the final answer emerges after a small number of symbolic edits. This behavior explains why increasing T does not necessarily inflate completion length proportionally and why an adaptive policy can stop at T = 1 for many inputs.
When additional rounds help. The examples suggest that multiple rounds are most beneficial for problems with (i) multi-constraint coupling (where one mistaken assumption breaks downstream steps), (ii) long-range dependencies (where early variable choices constrain later operations), or (iii) heavy arithmetic/combinatorial bookkeeping. In these cases, CLSR
18
When LLMs Develop Languages
Table 9. Category routing vs. full-pool routing. Accuracy and generated tokens are reported as Acc. (No. tokens).
BACKBONE BANK T YPE MMLU-P RO GPQA- MAIN MATH500
C ATEGORY 38.9 (320) 30.8 (362) 48.2 (257)
LL A MA3-8B
F ULL - POOL 38.6 (325) 30.5 (364) 48.2 (263)
C ATEGORY 60.4 (96) 47.7 (228) 86.8 (257)
Q WEN 3-8B
F ULL - POOL 60.1 (98) 47.5 (233) 86.4 (264)
Table 10. Cross-domain LSF transfer. Accuracy and generated tokens are reported as Acc. (No. tokens).
BACKBONE BANK T YPE MMLU-P RO GPQA- MAIN MATH500
I N - DOMAIN 38.7 (312) 30.6 (358) 48.4 (252)
LL A MA3-8B A LL - DOMAIN 38.9 (320) 30.8 (362) 48.2 (257)
L EAVE - ONE - DOMAIN - OUT 38.0 (345) 30.3 (393) 47.9 (268)
I N - DOMAIN 60.5 (92) 47.5 (221) 86.8 (253)
Q WEN 3-8B A LL - DOMAIN 60.4 (96) 47.7 (228) 86.8 (257)
L EAVE - ONE - DOMAIN - OUT 60.0 (105) 46.8 (245) 86.4 (262)
uses additional rounds to re-encode the problem into a more stable LSF and derive the answer with fewer opportunities for cascading errors.
Limitations of qualitative traces. Finally, we emphasize that these traces are presented as algorithmic artifacts of CLSR, i.e., intermediate representations used to improve prediction under a token budget, not as guaranteed faithful explanations of internal model causality. Nevertheless, they provide an interpretable window into how CLSR reallocates tokens from verbose narration to structured, constraint-preserving symbolic computation.
A.8. Latency decomposition and token accounting Reference calibration (TTFT/TPOT). Regarding TTFT (time-to-first-token) and TPOT (time-per-output-token), Table 11 reports representative single-request latency components for Qwen3-8B on RTX 4090 GPU. We use these measurements as a sanity-check that, for long-form reasoning outputs, decoding latency scales approximately linearly with the number of generated tokens and typically dominates prompt prefill overhead.
Table 11. Representative latency decomposition for Qwen3-8B inference (single request, batch size 1). Lin and Lout denote prompt and completion lengths. TTFT primarily reflects prompt prefill plus the first decode step; TPOT reflects steady-state decoding; RTI denotes the ratio of total latency over token generation (decoding) time.
LIN LOUT TTFT ( MS ) TPOT ( MS / TKN ) RTI
64 1.106
512 256 76 11.2 1.026
1024 1.007
64 1.194
1024 256 148 11.9 1.048
1024 1.012
64 1.340
2048 256 281 12.9 1.085
1024 1.021
The LLM-router often takes dozens of input tokens to determine the LSF-categories, then takes hundreds of input tokens to read the LSF-profiles and determine which LSFs should be selected; reading a complete LSF often consumes hundreds of tokens. Empirically, the total input token Lin during inference for different benchmarks often ranges from 0.5 ∼ 2 × 103 .
Cache-aware token-equivalent accounting. The main paper reports generated completion tokens because they are a useful latency-oriented proxy for reasoning workloads. However, an LSF card also contributes input tokens. In a serving system with prompt-prefix caching, a fixed bank of LSF cards can be arranged in canonical order as a reusable prefix, while only the query, compact routing suffix, and generated outputs vary across requests. To make this explicit, we define a
19
When LLMs Develop Languages
Table 12. Cache-aware token-equivalent accounting on Qwen3-8B. Generated tokens count online emitted tokens. Cache-aware tokens additionally include input overhead under a reusable-prefix assumption.
B ENCHMARK M ETHOD ACC . G EN . T KN C ACHE - AWARE T KN
C OT 60.5 312 331
MMLU-P RO S OT 58.5 120 144
CLSR 60.3 93 121
C OT 86.8 872 890
MATH500 S OT 81.4 301 322
CLSR 86.6 245 283
cache-aware token-equivalent cost uncached cached Ceq = Cout + γin Cin + γcache Cin , uncached cached where Cout is the generated-token cost, Cin is the non-reused input-token cost, Cin is the reusable prefix-token cost, and γin , γcache ∈ [0, 1] convert input tokens into output-token equivalents. The specific coefficients depend on the model provider and serving infrastructure. Therefore, this metric should be interpreted as a deployment-oriented diagnostic rather than the primary algorithmic objective. Table 12 reports representative cache-aware comparisons for Qwen3-8B. The results show that CLSR remains competitive even when reusable prompt-prefix overhead is included.
Offline cost amortization. CLSR pays an offline protocol-discovery cost to generate, evaluate, and refine the LSF pool. This cost is conceptually similar to automatic prompt/protocol optimization methods that search over candidate instructions, demonstrations, or policies before deployment. The practical motivation is different from self-consistency or Tree-of-Thoughts-style inference: CLSR trades one-time offline search for lower repeated serving-time reasoning cost. If an evolved LSF pool is reused for many queries in the same benchmark/domain family, the amortized offline cost per query decreases as the number of served queries grows. Thus, CLSR is most appropriate for repeated-use reasoning services, agent backends, and domain-specific deployments, rather than one-off queries where no amortization is possible.
20
When LLMs Develop Languages
B. Prompt Templates for LSF Synthesis, Mutation, and Routing This appendix provides the prompt templates used by CLSR. The templates are intentionally lightweight: the LLM is given the optimization goal and a compact schema, but the concrete symbolic protocol is not manually authored. LSF synthesis prompt. Given a set of exemplar question–answer pairs with reasoning traces, we use the following template:
Please design a Language Symbolism Framework (LSF) based on the exemplars in the chat history to minimize the number of
tokens while maintaining the reasoning capacity. An LSF is a symbolic communication code comprising: (i) symbol naming,
namely a compact lexicon; (ii) syntax, namely a compositional grammar; and (iii) constraints, namely well-formedness
and usage rules. The LSF should help an LLM solve similar problems with concise symbolic reasoning. Return the LSF
specification, including the symbol table, grammar, reasoning protocol, answer format, and failure checks.
LSF mutation prompt. After evaluating a generation of LSFs, we select high-leverage traces that are both correct and token-efficient. The mutation prompt is:
You are given several parent LSFs and their high-leverage solution traces. A high-leverage trace is correct and uses fewer
tokens than competing traces for the same query. Refine or recombine the parent LSFs to produce a next-generation LSF. Keep
symbolic conventions that support correct concise reasoning; remove redundant notation; repair ambiguous rules; and add only
minimal verification tags when they improve reliability. Do not simply rewrite the example answers. Return a reusable LSF
specification that can be applied to unseen queries.
LSF profile summarization prompt. For each candidate LSF Sk , we summarize its empirical profile from validation rollouts:
Given an LSF specification and its validation rollouts, summarize its profile for routing. Include: (i) query types where this
LSF performs well; (ii) validation accuracy; (iii) median generated-token footprint; (iv) common failure modes; (v) reliability
under different problem difficulties; and (vi) a short descriptor that can be used by an LLM-router. Keep the descriptor compact
and factual.
Router-card distillation prompt. To make routing both compact and reproducible, the above profile is then distilled into a short card consumed by the LLM-router:
Given an LSF specification and its empirical profile, output exactly one compact router card in the following format: [ID]
TAG | perf:<acc>/<tok> | IO | STOP where acc is the validation accuracy rounded to two decimals, tok is the
median generated-token count, IO is a short output-schema descriptor, and STOP is a short stopping-contract descriptor. Do
not output explanations.
This produces cards like [ALG1] linear/eq | perf:.81/42 | out:ans,brief | stop:conf>=0.6, which expose the empirical accuracy–cost profile to the router while keeping the prompt prefix compact. Stage-1 category-routing prompt. At test time, we use a category-routing prompt to prune the candidate LSF pool:
You are an LSF category router. Given the query, output exactly one line in the format
C:<category-list>
where <category-list> contains one or two comma-separated category codes chosen from the predefined vocabulary.
Use a broad backup category when the query is ambiguous. Do not output explanations.
For example, the router may emit C:ALG, C:PHY,ALG, or C:QA,HIS. Stage-2 protocol-planning prompt. Given the query and the compact LSF cards selected by Stage 1, the router emits a structured execution plan:
You are an LSF protocol router. Given the query and the available LSF cards, choose the cheapest reasoning protocol that is
likely to preserve correctness. Output exactly one line in the following format:
M:<mode>;L:<lsf-list>;R:<round-spec>;K:<n>;A:<agg>;S:<stop>
Use only LSF IDs that appear in the provided cards. Choose M:S for a single-LSF direct answer, M:A for multi-LSF
aggregation, and M:C for implicit LSF composition. The field R specifies the round structure or composition order; K specifies
the number of parallel samples per LSF when aggregation is used; A specifies the aggregation rule; and S specifies the stopping
criterion. Do not output explanations or any text outside the plan line.
21
When LLMs Develop Languages
Why prompt templates do not make CLSR prompt optimization. These templates specify the task interface, the empirical profile exposed to the router, and the structured control format, but not the concrete symbolic language itself. The optimized object is the reusable LSF specification and its routing behavior, not a per-query natural-language instruction string. The same LSFs can be reused across many unseen queries, selected by a router, and composed with other LSFs in later rounds.
C. LLM-Router: Compact Plan Format and Execution Semantics Design goals. Our router must (i) emit a parseable plan with near-zero ambiguity, (ii) keep the router output within tens of tokens, and (iii) support the three protocol-planning modes in our framework: S INGLE (one LSF direct answer), AGGREGATE (multi-LSF ensemble), and C OMPOSE (implicit LSF composition). To ensure reliability, we recommend constrained structured decoding (e.g., EBNF/regex or lightweight schema constraints) so that the router can only produce syntactically valid plans.
Cost-aware routing principle. The router is deliberately conservative. It first restricts the candidate LSF pool through lightweight category routing, and then selects the cheapest protocol that is likely to preserve correctness. This realizes the accuracy–token objective of Section 2.1 at the protocol level: S INGLE is the default low-cost route, whereas AGGREGATE and C OMPOSE are invoked only when the available LSF cards indicate that additional redundancy, verification, or staged symbolic processing is likely to improve reliability.
C.1. Two-stage routing (categories → protocol plan) Stage 1 (Category routing). Given a query q, the router emits a short category code list C(q) to limit the candidate LSF pool. We predefine a vocabulary of ∼30–60 category codes (e.g., ALG, GEO, PHY, CHE, LOG, QA, CODE, STAT, NUM, PROB, BIO, HIS, LAW, FIN, etc.). The output is one line, typically < 10 tokens: C:ALG or C:PHY,ALG or C:QA,HIS Stage 2 (Protocol planning). We then feed the router the query q plus the LSF cards belonging to selected categories C(q). The router outputs a compact plan P(q) that selects concrete LSF(s) and the inference mode.
C.2. LSF card format (compact descriptors) Each LSF is represented to the router by a single short card that is easy to scan while remaining compact enough to keep the input overhead small. The card is distilled from the validation profile summarized in Appendix B and follows the schema
[ID] TAG | perf:<acc>/<tok> | IO | STOP
where TAG is a short semantic hint, acc is the held-out validation accuracy rounded to two decimals, tok is the median generated-token count for that LSF, IO is an output-schema hint, and STOP specifies the stopping contract. Example cards are:
[ALG1] linear/eq | perf:.81/42 | out:ans,brief | stop:conf>=0.6
[ALG2] simplify/check | perf:.77/55 | out:ans,check | stop:valid
[GEO1] geometry-parse | perf:.74/63 | out:vars,eqs | stop:complete
[PHY1] mechanics | perf:.79/58 | out:eqs,ans | stop:units-ok
[QA1] decompose | perf:.72/71 | out:subq,ans | stop:all-supported
This format deliberately exposes a minimal empirical accuracy–cost signal to the router. The full validation profile is used only offline to form the card, so the online planning prompt remains short while retaining the information required for budget-aware protocol selection.
C.3. Router plan format (tens of tokens) The router emits a single-line plan in a compact DSL:
22
When LLMs Develop Languages
M:<mode>;L:<lsf-list>;R:<round-spec>;K:<n>;A:<agg>;S:<stop>
We use the following short enumerations.
Mode. M specifies the protocol family: M ∈ {S, A, C}, corresponding to S INGLE, AGGREGATE, and C OMPOSE.
Selected LSFs. L is a comma-separated list of selected LSF IDs, e.g., L:ALG1 or L:QA1,QA3,HIS1.
Round structure. R specifies the execution structure. For a one-shot single or aggregation protocol, we use R:1. For repeated refinement, we may use R:2 or R:3. For implicit composition, R explicitly encodes the ordered LSF stages, e.g., R:PHY1>ALG1 or R:QA1>HIS1.
Sampling and aggregation. K is the number of parallel samples per LSF in aggregation mode, with default value K = 1. A is the aggregation rule, e.g., mv for majority vote, sc for self-consistency-style agreement, or w for a router-weighted aggregation rule when such weights are available.
Stopping rule. S is a short stopping criterion, e.g., ans for stopping after a valid answer is parsed, valid for schema- valid completion, units-ok for a domain-specific check, or conf0.7 for a confidence threshold when supported by the LSF contract.
Token budget. A plan such as M:A;L:QA1,QA3;R:1;K:2;A:mv;S:ans typically remains within a few tens of generated tokens. The format is intentionally compact so that the router overhead is negligible relative to the downstream reasoning protocol while remaining fully parseable.
C.4. Execution semantics (deterministic interpreter) Given the category output C(q) and plan output P (q), CLSR is executed deterministically as follows.
- Candidate construction. Build the candidate LSF set [ L(q) = LSFs(c), c∈C(q)
and assemble the corresponding compact LSF cards. 2. Plan parsing. Parse the router plan P (q) = (mode, lsfs, rounds, K, agg, stop). Invalid plans are rejected and the router is re-asked under constrained structured decoding. A plan is invalid if it violates the DSL grammar, references an unavailable LSF ID, or uses a mode-incompatible field configuration. 3. Protocol execution. • S INGLE (M:S): instantiate the selected LSF and run a direct solver call. The default one-shot case uses R:1; larger values of R indicate bounded repeated refinement under the same LSF. • AGGREGATE (M:A): for each selected LSF, sample K solver completions in parallel, then aggregate the resulting answers according to A. The default one-stage ensemble uses R:1. • C OMPOSE (M:C): execute the LSFs sequentially according to the ordering encoded in R. The parsed symbolic state emitted by the current LSF becomes the input state for the next LSF. For example, R:PHY1>ALG1 first constructs a physics equation state and then invokes an algebraic solver state. 4. Early stopping. After each solver call or composition stage, evaluate the stopping rule S. If the criterion is satisfied, terminate immediately; otherwise continue until the prescribed round structure is exhausted. 5. Accounting. All online LLM outputs are counted toward generated-token cost, including category-routing outputs, protocol-planning outputs, intermediate solver outputs, and aggregation-related outputs.
23
When LLMs Develop Languages
C.5. End-to-end samples C.5.1. S AMPLE : SINGLE -LSF DIRECT ANSWER Query. Solve 2x + 3 = 11.
Stage 1: category routing. C:ALG
Stage 2 input: card excerpt.
[ALG1] linear/eq | perf:.81/42 | out:ans,brief | stop:conf>=0.6
[ALG2] simplify/check | perf:.77/55 | out:ans,check | stop:valid
Stage 2 output: plan. M:S;L:ALG1;R:1;K:1;A:none;S:ans
Execution. Instantiate ALG1, run one solver call, parse the final answer, and stop immediately after out:ans is produced.
C.5.2. S AMPLE : MULTI -LSF AGGREGATION Query. Which author wrote the book that inspired the movie X, and what year was it published?
Stage 1: category routing. C:QA,HIS
Stage 2 input: card excerpt.
[QA1] decompose | perf:.72/71 | out:subq,ans | stop:all-supported
[QA3] evidence-first | perf:.75/84 | out:cites,ans | stop:cited
[HIS1] year-check | perf:.69/48 | out:year,src | stop:match
Stage 2 output: plan. M:A;L:QA1,QA3,HIS1;R:1;K:2;A:mv;S:ans
Execution. Run K = 2 samples per selected LSF in parallel, extract candidate answers, aggregate by majority vote, and stop after a valid final answer is selected.
C.5.3. S AMPLE : IMPLICIT LSF COMPOSITION Query. A 2 kg mass is pulled with 10 N on a frictionless surface. Find acceleration.
Stage 1: category routing. C:PHY,ALG
Stage 2 input: card excerpt.
[PHY1] mechanics | perf:.79/58 | out:eqs,ans | stop:units-ok
[ALG1] linear/eq | perf:.81/42 | out:ans,brief | stop:conf>=0.6
Stage 2 output: plan. M:C;L:PHY1,ALG1;R:PHY1>ALG1;K:1;A:none;S:units-ok
Execution. PHY1 first emits a minimal equation state, e.g., a = F/m. The parsed symbolic state is then passed to ALG1, which performs the substitution and emits the final answer. The execution stops once the answer is parsed and the domain-specific consistency check is satisfied.
24
When LLMs Develop Languages
D. Appendix: CLSR outputs and LSF samples This section presents representative raw model completions for both N -shot CoT and CLSR on the benchmarks used in the main paper. For CLSR, we additionally show the intermediate traces produced in a refinement round, making explicit how the latent symbolic form (LSF) is iteratively compressed, verified, and corrected before producing the final answer. These examples are intended to complement the quantitative results by illustrating the qualitative differences in trace structure, error patterns, and token usage between natural-language CoT and CLSR’s symbolic compression mechanism.
D.1. A sample on Scientific domains
Figure 9. Qualitative GPQA-Main example: CoT vs. CLSR (LSF). A representative GPQA-Main test query and the corresponding model generations under standard CoT and CLSR. CLSR replaces verbose natural-language narration with a compact symbolic trace that preserves key intermediate state and checks, yielding a shorter completion and a correct final answer in this example.
25
When LLMs Develop Languages
Figure 10. Router-selected LSF specification for GPQA-Main. The automatically selected LSF content (language descriptor and protocol template) used by CLSR for the GPQA-Main query in Fig. 9. The specification illustrates how the router retrieves and instantiates a task-appropriate symbolic code (operators, constraints, and formatting rules) prior to generation.
26
When LLMs Develop Languages
D.2. A sample on Mathematical domains
Figure 11. Qualitative MATH500 example: CoT vs. CLSR (LSF). A representative MATH500 test query with generations from standard CoT and CLSR. The CLSR trace emphasizes concise operator-like transformations and explicit state (e.g., variable bindings and subgoal markers), reducing verbosity while maintaining (and here improving) correctness.
27
When LLMs Develop Languages
Figure 12. Router-selected LSF specification for MATH500. The automatically selected LSF content used by CLSR for the MATH500 query in Fig. 11. This example highlights that the router can choose a math-specialized LSF with compact transformation operators and lightweight verification tags, enabling high information density per generated token.
28
When LLMs Develop Languages
D.3. A sample on AIME benchmark
Figure 13. Qualitative AIME example: CoT vs. CLSR (LSF). A representative AIME test query comparing standard CoT with CLSR. CLSR produces a program-like symbolic trace that keeps only the computable core of reasoning, reducing narrative overhead and supporting reliable multi-step derivation under a constrained token budget.
29
When LLMs Develop Languages
Figure 14. Router-selected LSF specification for AIME. The automatically selected LSF content used by CLSR for the AIME query in Fig. 13. The chosen LSF reflects difficulty-aware protocol selection, instantiating stricter structure and more explicit intermediate state for harder problems, consistent with the theory that different operating points on the accuracy–token frontier should be selected adaptively.
30
When LLMs Develop Languages
E. Appendix: More Theoretical Validation for CLSR This appendix provides formal statements and full proofs for the theory in the main text (Sec. 3). We focus on two pillars: (i) an information-theoretic lower bound relating achievable accuracy to the expected number of generated tokens under arbitrary symbolism, and (ii) a subsumption result showing that multi-round multi-LSF CLSR protocols generalize program-execution pipelines under an explicit “interpreter realizability” premise.
E.1. The Principle of Least Effort and Iterated Learning The sociolinguistic behavior of LSFs can be understood through two complementary pressures. First, Zipf’s Principle of Least Effort (Kanwal et al., 2017) argues that communication systems balance successful transmission against production cost. Under this pressure, frequently used meanings or operations tend to receive shorter forms, while rare or ambiguous meanings may preserve longer forms for reliability. CLSR instantiates an analogous pressure at test time: a symbolic convention survives if it helps the LLM solve problems correctly while reducing generated tokens. This explains why evolved LSFs often introduce short operators for recurring reasoning moves, such as variable binding, subgoal decomposition, unit conversion, answer verification, or contradiction checking. Second, the evolutionary loop resembles iterated learning. Each generation observes a bottlenecked set of successful traces from the previous generation and must reconstruct a reusable symbolic system from them. Iterated learning theory predicts that repeated transmission through such a bottleneck can produce languages that become more structured, compressible, and aligned with the learner’s inductive biases. In CLSR, the learner is a pretrained LLM, so the resulting LSFs are shaped by both task-level selection and the model’s existing linguistic/mathematical priors. This is why successful LSFs are rarely arbitrary character strings. They often converge toward partially interpretable symbolic conventions: terse enough to reduce token cost, but structured enough for the model to reliably execute. This perspective also clarifies the role of multi-agent interaction. Different agents propose different “dialects” because they see different exemplars, random seeds, and high-leverage traces. Selection then acts as a cultural filter: dialects that are concise but brittle disappear, while dialects that preserve correctness under compression are retained. The final router performs a pragmatic form of code-switching. It can select a strict symbolic dialect for easy algebraic queries, a softer dialect for knowledge-intensive queries, or a multi-dialect composition for hard problems requiring verification. Thus, CLSR operationalizes a small-scale society of machine dialects: symbolic protocols emerge, compete, merge, and are selected according to their communicative utility for reasoning. This interpretation also prevents an overstatement of our claim. We do not claim that LSFs are invented independently of human language. The base LLM and the benchmarks are human-derived, and the LSF synthesis prompt specifies the high-level goal. The novelty is that the concrete protocol is not hand-written by the authors and is instead discovered through repeated use and selection. The partial convergence of evolved LSFs toward human-interpretable notation is therefore not a flaw, but evidence that efficient symbolic systems can occupy a middle ground between natural language and opaque machine codes.
E.2. Preliminaries: Interactive inference as a finite-horizon CMDP Query-conditioned setting. Fix a query x and consider the conditional label distribution Y ∼ P (· | X = x) supported on a finite effective answer set Yx with |Yx | ≥ 2. For tasks with very large or continuous answer spaces, one can replace Yx by a finite ε-net / evaluator-induced equivalence classes; see Remark E.5. We measure correctness via the indicator 1{Yb = Y }.
Token alphabet and LSF-conditioned decoding kernels. Let Σ denote a finite alphabet of output tokens (e.g., the model vocabulary; any LSF-specific surface form is representable as sequences in Σ). We include a special termination token STOP ∈ Σ. Let K = {1, . . . , K} index the available LSFs. For each LSF k ∈ K, the frozen backbone LLM with parameters θ induces a stochastic decoding kernel (k) Pθ (z | h, x), z ∈ Σ, (7) where h is the full interaction history (including which LSF was invoked, and all previously generated tokens). This is a well-defined conditional distribution (measurable stochastic kernel) over the next token.
Transcript state space and actions. We model CLSR as a token-level interactive process of maximum length Tmax < ∞. At micro-step t ∈ {1, . . . , Tmax }, the state is the current history Ht = ht , where h1 is empty and ht+1 = (ht , at , Zt )
31
When LLMs Develop Languages
appends the selected action and the realized token. The action is either selecting an LSF at ∈ K (meaning “generate one token under LSF at ”), or selecting a stop action at = stop. If at = stop, we deterministically emit Zt = STOP and transition into an absorbing terminal mode for all future steps. Any higher-level “multi-round” protocol that calls multiple LSFs per round can be unrolled into such a micro-step sequence; this only affects constants and not the conclusions.
Cost and prediction. Define per-step token cost
c(Ht , at ) = 1{at ∈ K}, (8)
so the total token cost equals the number of non-STOP tokens: TX max
|T | = c(Ht , at ). (9)
t=1
The final transcript is T = (Z1 , . . . , ZTmax ) (padded with STOP after termination). An aggregation/readout map g (possibly implementing routing-then-aggregation, self-consistency, etc.) produces the final answer
Yb = g(T, x) ∈ Yx . (10)
We allow g to be any (measurable) function; importantly, it does not use external oracles beyond the generated transcript.
Policies and objectives. A possibly randomized and history-dependent policy π specifies distributions πt (· | ht , x) over actions at each step. For a fixed x, define
JC (π; x) = Eπ |T | | X = x , JA (π; x) = Pπ Yb = Y | X = x . (11)
For a token budget B ≥ 0, define the optimal achievable accuracy
A∗ (B; x) = sup JA (π; x). (12)
π: JC (π;x)≤B
This matches the main-text notion (Eq. (3)), specialized to a fixed query x. Remark E.1 (Finite Pstate reduction). Because Σ is finite and the horizon is bounded by Tmax , the set of reachable histories Tmax is finite: |H| ≤ t=0 (|K| |Σ|)t < ∞. Hence, CLSR can be treated as a finite-horizon CMDP on a finite state space (histories).
E.3. Existence and Pareto characterization of optimal CLSR policies Theorem E.2 (Existence of an optimal budget-feasible CLSR policy). Fix x. Assume (i) K < ∞, (ii) |Σ| < ∞ with a designated STOP token, and (iii) Tmax < ∞. Then for every budget B ≥ 0, the supremum A∗ (B; x) is attained: there exists ∗ a (possibly randomized) policy πB such that ∗ ∗ JC (πB ; x) ≤ B, JA (πB ; x) = A∗ (B; x). (13)
Theorem E.3 (Lagrangian scalarization and boundary points of the frontier). Under the assumptions of Theorem E.2, consider the scalarized objective h i Jλ (π; x) = Eπ 1{Yb = Y } − λ|T | X = x , λ ≥ 0. (14)
Then:
• For every λ ≥ 0, there exists an optimal (history-Markov) deterministic finite-horizon policy πλ∗ that maximizes Jλ (π; x). • Every supported boundary point of the Pareto frontier {(JC (π; x), JA (π; x))} can be implemented by some πλ∗ . • Any boundary point of the constrained problem can be achieved by a mixture of at most two scalarized optima πλ∗1 , πλ∗2 with adjacent multipliers, i.e., randomized only at t = 1 between two deterministic policies.
32
When LLMs Develop Languages
Proof of Theorem E.2. Because the horizon and alphabets are finite, we can cast CLSR as a finite CMDP on the finite history state space H. Define the (time-indexed) occupancy measures
qt (h, a) = Pπ (Ht = h, At = a | X = x), t = 1, . . . , Tmax . (15)
(k)
These satisfy linear flow constraints induced by the kernels Pθ : for each t < Tmax and each h′ ∈ H, X XX qt+1 (h′ , a′ ) = qt (h, a) P (h′ | h, a, x), (16) a′ h∈H a∈A
where A = K ∪ {stop} and P (h′ | h, a, x) is the probability of induced transition from ′ P history h to h under action a and query x (obtained by appending a sampled token or absorbing under stop). Also, a q1 (h1 , a) = 1 for the initial empty history h1 . The expected token cost is linear: TX max X
JC (π; x) = qt (h, a) c(h, a). (17)
t=1 h,a
Define a terminal reward function r(h) = P(Y = g(T, x) | X = x, T corresponds to h), so that
JA (π; x) = Eπ [r(HTmax +1 ) | X = x], (18)
which is also linear in the induced terminal occupancy. Therefore, constrained optimization
max JA (π; x) s.t. JC (π; x) ≤ B (19)
π
is equivalent to a finite-dimensional linear program in the variables {qt (h, a)} on a nonempty, closed, bounded polytope. A linear objective over a compact polytope attains its maximum, hence an optimal occupancy measure exists. Finally, standard CMDP realizability guarantees that any feasible occupancy measure correspondsP to some (possibly randomized) non-anticipatory policy (one can explicitly construct πt (a | h) ∝ qt (h, a) whenever a qt (h, a) > 0). Thus the supremum ∗ is attained by a policy πB , proving the claim. □
Proof of Theorem E.3. For each fixed λ ≥ 0, the scalarized problem maxπ Jλ (π; x) is an unconstrained finite-horizon MDP with bounded reward. By backward induction (dynamic programming), an optimal deterministic history-Markov policy πλ∗ exists. For the constrained frontier, consider the CMDP linear program from the proof of Theorem E.2. Its Lagrangian dual introduces a multiplier λ ≥ 0 for the token-cost constraint; strong duality holds for finite linear programs, so optimal primal values equal optimal dual values. Consequently, every supported boundary point is the optimizer of JA − λJC for some λ. If the budget constraint is active and the mapping λ 7→ JC (πλ∗ ; x) has a discontinuity (typical in discrete CMDPs), an intermediate budget can be met by randomizing between two adjacent multipliers at the start of the episode; by convexity of achievable occupancy measures, a mixture of two deterministic policies suffices. □
E.4. Token–accuracy lower bound under arbitrary symbolism Binary entropy. Let h2 (δ) = −δ log2 δ − (1 − δ) log2 (1 − δ) denote the binary entropy (base-2).
Per-token information rate. Let T = (Z1 , . . . , ZL ) be the variable-length transcript generated by an interactive policy, where L = |T |. For notational convenience, one may include an explicit terminal symbol in the formal transcript; this changes the practical completion-token count by at most an additive constant. Define the model- and query-dependent active-token information rate κθ (x) = sup I(Y ; Zt | X = x, Z<t = h<t , L ≥ t) , (20) π,t,h<t
where the supremum ranges over policies, time steps, and reachable active histories with nonzero probability. If Zt ranges over a finite effective alphabet Σ, then κθ (x) ≤ log2 |Σ|.
33
When LLMs Develop Languages
Theorem E.4 (Information-theoretic lower bound on expected generated tokens). Fix a query x and let |Yx | ≥ 2. Let π be any interactive CLSR policy with final answer Yb = g(T , x). If π achieves accuracy at least α ∈ (0, 1),
Pπ (Yb = Y | X = x) ≥ α, (21)
then with δ = 1 − α its expected token cost satisfies max{Ireq (x, δ), 0} Eπ [L | X = x] ≥ , (22) κθ (x) where Ireq (x, δ) = H(Y | X = x) − h2 (δ) − δ log2 (|Yx | − 1). (23)
Proof of Theorem E.4. Let δ = P(Yb ̸= Y | X = x) ≤ 1 − α. By Fano’s inequality applied to estimating Y from Yb over the evaluator-induced answer set Yx ,
H(Y | Yb , X = x) ≤ h2 (δ) + δ log2 (|Yx | − 1). (24)
Therefore, I(Y ; Yb | X = x) = H(Y | X = x) − H(Y | Yb , X = x) ≥ Ireq (x, δ). (25)
Since Yb is a deterministic function of (T , x), data processing gives
I(Y ; T | X = x) ≥ I(Y ; Yb | X = x) ≥ Ireq (x, δ). (26)
It remains to upper-bound the information in the transcript by its expected active length. Using a variable-length chain rule and conditioning on the event that the process is active at step t, TX max
I(Y ; T | X = x) ≤ Pπ (L ≥ t | X = x) κθ (x). (27)
t=1
The inequality follows from the definition of κθ (x) over all reachable active histories; inactive steps emit no new token and contribute no active-token information. Finally, TX max
Pπ (L ≥ t | X = x) = Eπ [L | X = x], (28)
t=1
which yields Ireq (x, δ) ≤ κθ (x)Eπ [L | X = x]. (29) Taking the nonnegative part of Ireq gives the stated bound. □ Remark E.5 (On the “difficulty” term and evaluator-induced answer sets). When Y is not naturally finite (e.g., free-form strings), one may let Yx be the set of equivalence classes induced by the benchmark evaluator (all outputs judged identical), or consider a finite packing argument (Fano method) on a finite subset of hypotheses. The same proof yields a lower bound in terms of the entropy/packing size of the induced hypothesis class.
E.5. CLSR subsumes program-execution pipelines under interpreter realizability Program-execution pipelines. A program-execution (PE) inference pipeline is any procedure that: (i) makes a finite number of calls to the frozen backbone LLM (under some prompting format) to produce an LLM-generated program transcript S ∈ Σ∗ , and (ii) applies a deterministic computable executor Exec : Σ∗ → Yx that does not access external knowledge (only performs logic/symbolic manipulation). The overall output is YbPE = Exec(S) (possibly followed by trivial formatting). Let the total number of LLM-generated tokens in the PE procedure (across all its calls) be |TPE |, with expected cost E[|TPE | | X = x] = B and accuracy P(YbPE = Y | X = x) ≥ α. (30)
34
When LLMs Develop Languages
Interpreter realizability (assumption). We formalize the key premise used in the main text. Assumption E.6 (Interpreter realizability). For every deterministic computable executor Exec : Σ∗ → Yx in the allowed executor class, there exists a single LSF SExec such that, for any input string s ∈ Σ∗ , one LLM call conditioned on SExec and given s as input produces Exec(s) with failure probability at most εint < 12 , and with output length at most |Exec(s)| + c0 for a universal constant c0 .
Amplification lemma. Lemma E.7 (Majority-vote amplification). Let U1 , . . . , Um be i.i.d. Bernoulli variables with P(Ui = 1) ≥ 1 − ε and ε < 12 . Let U b = Maj(U1 , . . . , Um ). Then b = 0) ≤ exp − 2m(1/2 − ε)2 . P(U (31) In particular, to make the error at most δ, it suffices to take m = O(log(1/δ)). Pm Proof of Lemma E.7. This is a direct application of Hoeffding’s inequality to the sum i=1 Ui . □ Theorem E.8 (CLSR subsumes PE under interpreter realizability). Fix a query x and assume Assumption E.6. For any PE pipeline with expected LLM-generated token cost B and accuracy at least α, let ℓ̄Exec (x) = E[|Exec(S)| + c0 | X = x] , (32) where S is the PE-generated transcript and c0 is the interpreter-output overhead in Assumption E.6. Then for any target δ ∈ (0, 1), there exists an LSF-only CLSR protocol π such that
- (Accuracy) Pπ (Yb = Y | X = x) ≥ α − δ.
- (Token cost) Eπ [|T | | X = x] ≤ B + O ℓ̄Exec (x) log(1/δ) .
If ℓ̄Exec (x) = O(1), as in short-answer benchmark settings, the overhead reduces to O(log(1/δ)).
Proof of Theorem E.8. Construct a CLSR protocol π that emulates the PE pipeline in two stages. Stage 1: reproduce the PE transcript. Because the PE pipeline makes a finite number of frozen-LLM calls under a fixed prompting format, the CLSR pool can include an LSF or fixed LSF-composition plan that reproduces the same prompting format and decoding choices. Let S ′ denote the transcript produced by this CLSR stage. By construction, S ′ has the same distribution as the PE-generated transcript S up to constant formatting overhead, so E[|S ′ | | X = x] ≤ B + O(1). (33)
Stage 2: internally execute. Given S ′ , invoke the interpreter LSF SExec from Assumption E.6. A single invocation returns Exec(S ′ ) with failure probability at most εint < 1/2. Repeat the interpreter call m times independently by resampling and take a majority vote over the produced outputs. By Lemma E.7, choosing m = O(log(1/δ)) makes the internal-execution error probability at most δ. Accuracy. Let EPE be the event that the PE pipeline is correct, i.e., Exec(S) = Y . Let Eint be the event that the amplified internal interpreter returns Exec(S ′ ). Because S ′ matches the PE transcript distribution, P(EPE | X = x) ≥ α. By amplification, P(Eint | X = x) ≥ 1 − δ. A union bound gives Pπ (Yb = Y | X = x) ≥ P(EPE ∩ Eint | X = x) ≥ α − δ. (34)
Token cost. The CLSR transcript consists of the PE-mimicking transcript S ′ plus m interpreter outputs. Each interpreter output has length at most |Exec(S ′ )| + c0 . Therefore, Eπ [|T | | X = x] ≤ B + O(1) + O ℓ̄Exec (x) log(1/δ) , (35) which proves the stated bound after absorbing constants. □ Remark E.9 (When interpreter realizability is plausible or fails). Assumption E.6 is a modeling premise: it is plausible when Exec lies within the backbone’s algorithmic competence and the required computation fits within effective context and reliability limits; it may fail for executors requiring exact long-horizon computation or strict formal guarantees. Theorem E.8 should therefore be read as a conditional subsumption: if the backbone can act as an interpreter for the executor class, then CLSR can match PE without external execution.
35
When LLMs Develop Languages
F. Appendix: Related work Instead of emitting verbose natural language traces (CoT) or executable programs (tool-based pipelines), CLSR learns a compact Language Symbolism Framework (LSF) and refines it through a small number of recursive rounds. This insight is also motivated by the view of neuronal communication of interacting neural blocks (Pei & Wang, 2023; Pei et al., 2024b; 2025), which are analogous to the communication protocol between agents. In the following, we position CLSR relative to the most relevant lines of work and clarify the key distinctions. Natural-language reasoning prompts and decomposition. Chain-of-Thought prompting and its variants elicit step-by-step natural-language reasoning and typically improve accuracy but often increase generation length and latency (Wei et al., 2022; Kojima et al., 2022). A large body of work improves performance by (i) more robust decoding/aggregation such as self-consistency (Wang et al., 2025a), (ii) better decomposition protocols such as least-to-most prompting (Zhou et al., 2022) and plan-and-solve prompting (Wang et al., 2023), or (iii) automatic prompt optimization/evolution (Yang et al., 2023; Fernando et al., 2024). These methods operate within natural language and generally scale the computation by generating more text or more samples. In contrast, CLSR changes the intermediate language itself: it compresses reasoning into LSF and uses recursion to refine the representation while remaining token-efficient. Thus, CLSR can be viewed as a different point in the design space where the goal is not to elaborate the chain but to densify it. Search-style inference over thoughts. Tree/graph/search-style inference methods generalize CoT by exploring multiple candidate intermediate steps with self-evaluation, backtracking, or explicit control structures (Yao et al., 2023; Besta et al., 2024; Sel et al., 2024). Their strength is improved robustness via exploration, but they incur a higher inference-time cost. CLSR targets a different regime: it performs single-trajectory recursive refinement under a strict token budget. Rather than exploring many branches, CLSR aims to learn a stable symbolic scaffold LSF that reduces the need for breadth. Program-execution pipelines and tool enhancement. Program-based methods such as Program-aided Language (Gao et al., 2023) and Program-of-Thoughts (Chen et al., 2023) instruct the LLM to output executable code and delegate correctness-critical computation to an external runtime (e.g., a Python interpreter). In addition, a broader line of tool- augmented reasoning/acting uses external APIs to reduce hallucination and improve factuality or task completion (Yao et al., 2022; Schick et al., 2023). CLSR is explicitly LLM-only: it does not rely on external compilers or interpreters. The intermediate representation is symbolic, but not a program meant for execution by an external tool; instead, it is designed to be self-interpretable by the same LLM under recursion. This distinction is essential to our scope: CLSR aims to recover much of the benefit of symbolic structure while keeping deployment minimal and avoiding tool dependency. Token-efficient and compressed reasoning traces. Recent work explicitly targets shorter reasoning traces, including generating concise intermediate drafts or sketches instead of verbose CoT (Xu et al., 2025b; Aytes et al., 2025a). Other approaches compress reasoning into dense or latent representations, such as compressed contemplation tokens or explicit CoT compression (Nayab et al., 2024a; Wang et al., 2025b). CLSR differs in two ways. First, CLSR introduces an explicit discrete symbolic representation (LSF) with a stable, reusable schema across tasks, rather than only asking the model to “write less” in natural language. Second, CLSR couples the representation with a recursive refinement operator and an adaptive stopping policy, which is directly connected to our theoretical account of accuracy–token trade-offs. Empirically, this yields an instance-wise compute allocation: many examples stop at T = 1, while hard cases receive extra refinement. Test-time self-improvement and iterative refinement. A complementary line studies improving model output through iterative self-feedback or reflection at test time (Madaan et al., 2023; Shinn et al., 2023; Zhang et al., 2025a). These approaches typically refine natural-language outputs (or agent trajectories) using critique/feedback loops. CLSR shares the high-level principle of iterative refinement, but focuses specifically on refining a compact symbolic intermediate (LSF) to improve accuracy per token without requiring external feedback channels or environment interaction. Faithfulness and interpretability of the generated traces. Recent studies state that natural-language reasoning traces should not be automatically interpreted as faithful explanations of the causality of internal models (Turpin et al., 2023; Barez et al., 2025). CLSR adopts a pragmatic stance: intermediate traces are algorithmic artifacts used to improve accuracy–token frontier, not guaranteed causal explanations. Meanwhile, LSF is intentionally compact and constraint-oriented, making it easier to inspect missing constraints, inconsistent variable bindings, or arithmetic errors than long-form narrative CoT. Summary of novelty. Across these related areas, CLSR contributes a distinct combination: (i) a framework that automatically generates discrete language symbolism framework positioned between natural language and executable code, (ii) a recursive refinement operator that improves accuracy without external tools, and (iii) an adaptive compute allocation view and empirical evidence that directly targets the optimal accuracy–token frontier.
36