What Large Language Models Are, How They Function, and How to Use Them Effectively
A large language model (LLM) is a mathematical function trained to predict the next token given all preceding tokens. A token is a fragment of text—typically a word, sub-word, or punctuation mark.
During training, the model was shown enormous quantities of text and repeatedly asked to predict the next token. Over billions of such predictions it learned statistical regularities: grammar, common facts, reasoning patterns, and stylistic conventions.
It does not store facts in a database, retrieve documents, or possess understanding in the human sense. Apparent knowledge and reasoning are emergent properties of very accurate next-token prediction.
Click a bar to select the next token. Higher bars represent higher probability continuations.
Context: "The capital of France is"
Inference proceeds token by token. Advance through the steps to see the process.
Start by advancing through the generation steps.
Temperature controls sampling randomness. Lower values favor the highest-probability tokens.
Balanced sampling.
Because the model only continues the sequence it is given, the tokens in the prompt strongly shape the probability distribution of what follows. Clear structure raises the probability of the desired continuation.
Toggle structure elements to see how they constrain possible continuations.
Hallucination occurs when a fluent but incorrect continuation has high probability. The model has no internal flag for truth; it optimizes for likely text.
Both paths can have high probability. Structure and verification reduce the chance of taking the incorrect one.
High probability under well-structured context. Matches training patterns accurately.
Also high probability. Fluent but factually wrong. No internal truth signal.
It has no persistent memory across separate conversations unless context or external tools supply it. It cannot access information outside its training data or the current context window unless tools are invoked. Small changes in phrasing can shift the distribution.
For most tasks, use this minimal structure. Fill the fields to assemble a strong prompt.
Treat the first output as a sample from the distribution. If incomplete or incorrect, revise the prompt and generate again.
Complex tasks rarely succeed in a single generation. Recursive prompting feeds the model's output (or a verification result) back as the next input. These workflows are modeled as graphs of roles and dependencies. Every loop must be counted, every exit must carry evidence, and escalation is a deliberate one-way handoff when the budget is exhausted.
The simulator below shows a multi-agent graph (Orchestrator → Planner → parallel Workers → Integrator → Quality gate) with four realistic traces. Select a scenario, then step or play through the full execution. Loops are counted against a global budget; escalations include the evidence bundle.
MISSION:QUALITY BAR: tests pass, coverage ≥ 90%, eval score ≥ 0.9, no p0 bugs. GLOBAL BUDGET: ≤ 5 iterations, ≤ 3 retries per subtask, wall-clock limit. ROLES - orchestrator: owns state, budget, routing. Never does the work itself. - planner: decomposes mission into a subtask DAG; revises plan on failure feedback. - workers (×3): execute subtasks in parallel; report result + self-check + confidence. - integrator: merges outputs, runs build + test suite. - quality gate: scores result against QUALITY BAR. Output: PASS/FAIL + reasons + scores. - architect: escalation tier 1 — deep review, may replan once. - human: escalation tier 2 — makes the call, never debugs from scratch. GRAPH 1. orchestrator → planner: mission + constraints. 2. planner → workers: subtask DAG, fan-out in parallel. Each subtask retries ≤ 3 times with its failure reason attached. 3. workers → integrator: fan-in, merge + build + tests. 4. integrator → quality gate: test report + eval scores. 5. gate PASS → ship → done. 6. gate FAIL → if iterations < 5: loop to planner with failure reasons + scores (one iteration). Else → architect. 7. architect fixes or replans once → back to step 2. Architect can't resolve → human. 8. human decides → ship → done; human input returns to orchestrator as new constraint. ESCALATION BUNDLE (required) {iteration n/5, per-subtask attempts, last gate reasons + scores, diff summary, 2 options} STOP CONDITIONS ship, or human decision. Final status: DONE (iters: n, path: direct | via-architect) | ESCALATED (human) + budget remaining.
Design principle: every loop has a counted exit, every exit carries evidence. "Keep trying" becomes "try within budget, then escalate with proof." The four scenarios demonstrate first-try success, a single counted retry, budget exhaustion resolved by the architect, and a case that requires human decision.