In-Context Learning
Definition
In-context learning (ICL) is the meta-capability of LLMs to understand and adapt to tasks from information provided entirely within the context window—the prompt. Unlike traditional machine learning where adapting to a new task requires gradient descent on training data, in-context learning happens at inference time: the model reads examples, instructions, or demonstrations in the prompt and immediately generalizes. ICL encompasses zero-shot (instruction only), one-shot (one example), and few-shot (multiple examples) settings. The mechanism behind ICL is an active research area: leading theories suggest the model learns to implement learning algorithms during pre-training, enabling it to 'run' these algorithms on whatever task is presented in context.
Why It Matters
In-context learning is what makes LLMs practical as general-purpose AI components without task-specific training pipelines. The ability to teach an LLM a new task or adapt its behavior through prompt modification enables rapid prototyping and deployment of AI features. For 99helpers customers building applications, ICL means they can customize chatbot behavior—tone, output format, domain focus—through prompt engineering rather than model training. Understanding ICL helps explain both its strengths (fast adaptation, no training required) and its limitations (reliability decreases with task complexity, consistency requires careful example curation).
How It Works
ICL manifests in three primary forms: (1) task demonstration—showing (input, output) pairs that define a new classification or transformation; (2) task description—describing the task in natural language so the model understands what to do; (3) chain-of-thought—providing step-by-step reasoning examples that teach the model to reason through problems explicitly. Research shows that ICL is sensitive to: example selection (representative examples outperform random selection), example order (recent examples have more influence), example format (matching the exact format of the expected output), and task naturalness (tasks similar to pre-training text patterns work better).
In-Context Learning — Adapt Without Weight Updates
Prompt (context window)
Few-shot examples
"The movie was fantastic!"
Sentiment: Positive
"Total waste of time."
Sentiment: Negative
"Best experience I've had in years!"
Sentiment: ???
LLM
Weights unchanged — no gradient update
Predicted Output
Sentiment: Positive
Generalized from 2 in-prompt examples
No fine-tuning required. Examples live only in the prompt — they are not stored between sessions.
Real-World Example
A 99helpers developer wants their LLM to extract structured data from free-form customer onboarding forms in a format specific to their CRM. Rather than fine-tuning, they use ICL: two examples of (raw form text, structured JSON output) are placed in the prompt. The LLM generalizes from these two examples to correctly extract data from novel form submissions in the exact required JSON format. When the CRM format changes, they update only the examples in the prompt—no retraining required. This ICL-based approach ships in one day versus the weeks a fine-tuning approach would require.
Common Mistakes
- ✕Conflating in-context learning with fine-tuning—ICL modifies behavior within a single inference call; fine-tuning permanently updates model weights.
- ✕Assuming ICL examples are retained across sessions—in-context learning is stateless; examples in one conversation do not carry over to the next.
- ✕Using ICL for tasks requiring precise, consistent behavior at scale—ICL reliability varies with input distribution; fine-tuning provides more robust consistency for production.
Related Terms
Few-Shot Learning
Few-shot learning provides an LLM with a small number of input-output examples within the prompt, demonstrating the desired task format and behavior without updating model weights.
Zero-Shot Learning
Zero-shot learning is the ability of LLMs to perform tasks from natural language instructions alone, without any task-specific examples, by generalizing from pre-training knowledge to new task types.
Fine-Tuning
Fine-tuning adapts a pre-trained LLM to a specific task or domain by continuing training on a smaller, curated dataset, improving performance on targeted use cases while preserving general language capabilities.
Large Language Model (LLM)
A large language model is a neural network trained on vast amounts of text that learns to predict and generate human-like text, enabling tasks like answering questions, writing, translation, and code generation.
Instruction Tuning
Instruction tuning fine-tunes a pre-trained language model on diverse (instruction, response) pairs, transforming a text-completion model into an assistant that reliably follows human directives.
Ready to build your AI chatbot?
Put these concepts into practice with 99helpers — no code required.
Start free trial →