


3.1 Reasoning Processes and Search 3.1.1 Explicit Reasoning and Decomposition

A cafeteria starts with 23 apples, uses 20 for lunch, then buys 6 more. How many remain?
Standard-prompting output in the paper:
“The answer is 27.”
Which operation went wrong?
Example from Wei et al., Figure 1; not a new model run.
GSM8K · standard few-shot prompting
| Model | Accuracy |
|---|---|
| PaLM 540B | 17.9% |
The prompt supplies question–answer examples, but no intermediate reasoning.
Wei et al., Table 2. Historical benchmark setting.
What if the examples showed how to reach the answer?
Example question: Roger has 5 tennis balls and buys 2 cans with 3 balls each. How many balls does he have?
Answer: 11.
Question → final answer
Two cans add 2\times3=6 balls.
Roger now has 5+6=11 balls.
Answer: 11.
Question → reasoning steps → answer
Apply the pattern to the apples: 23-20=3;\quad3+6=9, replacing the paper’s direct answer 27.
Few-shot CoT: change the demonstrations, keep the model weights fixed. Adapted from Wei et al., Figure 1.











A missing dependency or wrong sub-answer can propagate. No interpreter is required.
Language model: map the story to variables, operations and executable Python.
Interpreter: run the generated program and return its computed answer.
baked = 200
sold = 93 + 39
returned = 6
answer = baked - sold + returned # 74
“Returned” means +6 here. An interpreter also executes the wrong translation (-6) faithfully.
Write Python. Store the result in ans.
# Example: 5 balls + 2 cans, 3 per can.
balls = 5
new_balls = 2 * 3
ans = balls + new_balls
# New question: 23 apples; use 20
# for lunch, then buy 6. How many remain?
# Python:
Few-shot: question–program examples, then a new question with its program left blank.
Zero-shot: instruction + new question, without the worked examples.
apples = 23
remaining = apples - 20
ans = remaining + 6
Python executes the code: ans = 9.
Illustrative prompt and continuation; one demonstration shown.

3.1 Reasoning Processes and Search 3.1.2 Test-Time Scaling I: Search and Feedback


How many candidate paths or branches should we explore?
How long should we continue a path, and how many times should we revise it?
Which candidates should we keep, and when should we stop?
























ALFWorld: use the observed state to complete a household task.
Act fails: it has the knife, but tries to clean it before reaching the sink; commands repeat.
ReAct succeeds: track the completed subgoal; go to the sink, clean the knife, then place it.
Act 45% → ReAct 71%
Success rate · PaLM-540B · 134 tasks.
Best of 6 prompt sets for each method; Table 3.
First trial fails: the lamp was on desk 1, but the agent moved to desk 2 and tried to use it.
Next trial succeeds: remember the location; return to desk 1, take the mug, and use the lamp.
130 / 134 tasks solved
ReAct + Reflexion · GPT-3 · Figure 3.
Cumulative across up to 12 trials, not one-shot.
ReAct updates the current plan; Reflexion reuses a lesson on the next attempt. Neither updates model weights.
Paper trajectories, paraphrased. The two result panels use different evaluation protocols.



Apple case: 23-20+6. A first answer says 27; the correct answer is 9.
| Control | What changes in this case | What it buys |
|---|---|---|
| Candidate breadth | Sample answers: 27, 9, 9 | Expose alternatives |
| Sequential work | Recheck 23-20=3; then +6 | Repair the arithmetic |
| Selection & stopping | Vote for 9; or execute the expression | Return a checked answer |
Learning: checked experience → update \theta → freeze \theta^{\prime} → use the same controls again.
Constructed paths, not a benchmark. A majority can still be wrong. Count all generation and checking costs.