4.5 Planning and Control 4.5.1 Agent Control and Planning

The Agent Chooses the Next Operation

agent controllerAgent state enters a controller, which selects answer, deliberation or external action. Environment observations return to evidence and memory; stop conditions are explicit. State + evidenceGoal, history, budgetControllerAnswer / think / actExecute + observeModel, tool, environmentAssess / stopCheck outcome + costnew observations → update state / memory

More thinking changes a candidate. An action may reveal new evidence or change the environment.

Updating context, beliefs or memory is not necessarily a parameter update.
Stop on a checked result, a justified handoff, a budget limit, or lack of a useful next action.

Ana: Think More, or Obtain New Evidence?

Current gap Useful next operation What it can resolve
Policy translated incorrectly Re-read policy; repair formalization Semantic fidelity
Proof search unfinished Allocate solver time Formal consequence
Filing record unreliable Query authoritative filing system Evidence about F
Source conflict unresolved Inspect provenance / request review Dependence and factual dispute

P(R ∣ O) = 11/21 concerns review priority. It does not authorize approval.
Choose an operation that addresses the actual uncertainty.

Case Study: Plan and Execute


原 PPT 独立图片资源
Case Study: Plan and Execute
plan-and-execute diagram
https://langchain-ai.github.io/langgraph/tutorials/plan-and-execute/plan-and-execute/

4.5 Planning and Control 4.5.2 System I / II in Agents



原 PPT 独立图片资源

4.5 Planning and Control 4.5.2 System I / II in Agents

原 PPT 独立图片资源

System I / II: An Operational Analogy

Dimension Direct response Additional deliberation
Computation Generate the answer directly Reason, branch, revise or check
Useful when Current policy and evidence suffice Extra work can resolve uncertainty
Failure Confident shortcut Repeated error; unnecessary work
Measure Success, actual compute, latency, evidence gained

Fast is not automatically shallow. Slow is not automatically correct.
These labels do not establish human-like mental systems inside an LLM.

No fixed “95% / 5%” allocation is assumed. A direct answer still requires neural computation.

Training, Decoding and Agent Control

Layer What changes? Example
Training Model parameters SFT, RLVR, distillation
Decoding / inference Candidate trajectory and budget Sampling, search, reasoning tokens
Agent control Next operation and environment state Call a tool, observe, retry, stop

A single model can support several modes. A search can repeatedly call a direct policy.

Nucleus sampling is a decoding choice, not a definition of System I.
Public o1 evidence does not establish a specific MCTS + CriticGPT architecture.

4.5 Planning and Control 4.5.3 Adaptive Reasoning

Recent Work: Different Control Problems

Question Representative work What is controlled?
Expose two modes Qwen3 (May 2025) Prompt / template + budget
Learn when to use each Thinkless; AdaptThink (2025) Mode-selection policy
Predict required budget Sonata (ICLR 2026) Compute allocation before reasoning
Make future direct answers better System 2 distillation (2024) Parameters, using selected outputs
Check simple-task behavior S1-Bench (IJCAI 2026) Evaluation, not a controller

Research selection checked on 2026-11-15. Versions and primary sources are recorded in the reading page.

Qwen3: Make Both Modes Available

qwen modeMode-fusion data Thinking + direct answers → Post-training Learn both behaviors → Template + budget Choose / constrain mode → Response With or without long CoT Mode-fusion dataThinking + direct answersPost-trainingLearn both behaviorsTemplate + budgetChoose / constrain modeResponseWith or without long CoT

The May 2025 report provides a controllable interface.
Mode availability alone does not demonstrate an optimal learned router.

§4.3: thinking-mode fusion. §4.7: budget analysis. These claims refer to the original unified-mode version.

Thinkless: Learn the Mode Token

thinkless modeWarm-up distillation Short + long examples → Select control token <short> or <think> → Generate + score Correctness; short bias → DeGRPO update Control / response losses Warm-up distillationShort + long examplesSelect control token<short> or <think>Generate + scoreCorrectness; short biasDeGRPO updateControl / response losses
Table 1 · 1.5B Baseline → Thinkless accuracy Mean response tokens
GSM8K 83.47% → 84.18% 1919 → 624
MATH-500 86.08% → 81.84% 5675 → 2555

Token savings can accompany an accuracy loss. Thinking frequency, token count and wall time are different metrics.

AdaptThink: Prefer Direct Answers Under a Constraint

adaptthink modeReference policy Estimate task reward → Sample both modes Balanced exploration → Constrained objective Favor NoThinking, Retain task quality → Policy update Learn a mode per input Reference policyEstimate task rewardSample both modesBalanced explorationConstrained objectiveFavor NoThinkingRetain task qualityPolicy updateLearn a mode per input

Cold start: a long-thinking model may never sample direct answers without explicit exploration.

Importance sampling: correct for the sampling distribution while learning from both modes.

The performance constraint is an optimization target—not a proof that every future answer is correct.

AdaptThink: Quality and Cost Together

DeepSeek-R1-Distill-Qwen-1.5B · original Thinking → AdaptThink · Table 1

Dataset Accuracy (%) Response tokens NoThinking share
GSM8K 79.0 → 83.1 978 → 480 86.9%
MATH500 80.6 → 82.0 4887 → 1782 76.8%
AIME 2024 29.4 → 31.0 12073 → 6679 40.4%

Three-dataset macro average: +2.4 pp accuracy; −53.0% response length.

16K evaluation context; temperature 0.6. AIME: 16 samples per problem, averaged—not best-of-16. Token reduction is not measured wall-clock speedup.

Sonata: Predict a Budget Before Thinking

sonata budgetOffline calibration Multi-path consistency → Train small adapter Predict agreement, from hidden states → Online query prefill Read representation → Allocate budget Then generate reasoning Offline calibrationMulti-path consistencyTrain small adapterPredict agreementfrom hidden statesOnline query prefillRead representationAllocate budgetThen generate reasoning

Self-consistency is a proxy for thinking need, not a truth certificate.

“Latent space” here supplies features to a budget predictor. It does not mean all reasoning becomes a Coconut-style latent chain.

Sonata: What the 2026 Evidence Supports

Reported in ICLR 2026 proceedings

20–60% fewer thinking tokens at the same accuracy across reported model / task settings.

Models include Qwen3-8B / 32B, GPT-OSS-120B, Qwen3-235B-A22B.

Read the boundary

Tasks include AIME25, GSM8K, MATH500, GPQA and LiveCodeBench.

Not a universal per-query guarantee.
Not a tool-use success benchmark.
Not the same as measured latency.

Evaluate calibration cost, routing overhead, difficult-tail failures and actual task quality.

Version frozen to the conference proceedings; later author summaries report different experiment ranges.