Chain-of-thought (CoT)
Chain-of-thought prompting asks the LLM to reason step by step before producing the final answer — 'let's think through this carefully' or 'show your work'. The technique dramatically improves performance on multi-step reasoning tasks (math word problems, logic puzzles, complex queries) by giving the model space to decompose the problem.
CoT was introduced by Wei et al. (2022) and quickly became table-stakes for reasoning tasks. The mechanism is that the intermediate tokens generated during reasoning act as scratch space — the model uses its own output as additional context for subsequent tokens, enabling reasoning chains too long for direct inference. Modern reasoning models (o1, o3, Claude with extended thinking) automate CoT internally; for general-purpose LLMs, explicit CoT prompting remains effective. The trade-off is latency (reasoning takes tokens, tokens take time) and cost (more tokens billed); the trade-off is almost always worth it on tasks where accuracy matters.
Related terms
- Tree-of-thought (ToT)
Tree-of-thought prompting extends chain-of-thought by exploring multiple reasoning paths in parallel, evaluating each, and pursuing the most promising — analogous to tree search in game-playing AI.
- Few-shot prompting
Few-shot prompting provides the model with example input-output pairs in the prompt to demonstrate the desired pattern before asking it to handle the actual input.
- System prompt
A system prompt is the initial instruction given to an LLM at the start of a session that sets behaviour, persona, output format, and constraints — distinct from user messages that follow.