Logit Bias
Definition
Logit bias is a parameter (supported by OpenAI API as logit_bias) that modifies the raw logit scores for specific tokens before sampling. Each token in the vocabulary can be assigned a bias value ranging from -100 (effectively banned) to +100 (always generated when in context). A logit_bias of -100 for a token makes it essentially impossible for the model to generate that token. A value of 100 makes the model heavily favor that token. Common use cases: banning specific words or phrases from responses, forcing the model to use specific vocabulary, implementing content filters at the token level, or creating constrained generation tasks (forcing outputs from a restricted set of options).
Why It Matters
Logit bias provides token-level control that sits between full guardrails (which evaluate complete responses) and simple post-processing (which filters output after generation). For 99helpers customers, logit bias is useful for: forcing the model to use only approved terminology (e.g., always say 'Support Team' not 'Customer Service'), preventing generation of competitor names at the token level, or implementing strict format constraints (e.g., preventing the model from generating backticks if you never want code blocks). While useful, logit bias should be applied sparingly—aggressive biasing can produce incoherent outputs if the suppressed tokens are needed for natural language generation.
How It Works
Logit bias usage in OpenAI API: response = openai.chat.completions.create(model='gpt-4o', messages=[...], logit_bias={2602: -100, 4830: -100}). Token IDs are looked up using tiktoken: import tiktoken; enc = tiktoken.encoding_for_model('gpt-4o'); enc.encode('CompetitorName') → [2602] (example ID). Setting to -100 effectively removes that token from the vocabulary for this request. Bias values of -1 to -5 moderately reduce probability (useful for reducing specific phrasing without eliminating it entirely); +1 to +5 moderately increase probability. Note: logit_bias applies to individual tokens, not sequences; multi-token words require biasing each component token.
Logit Bias — Adjusting Token Probabilities
Token
Base logit
Bias
Final logit
Prob
"yes"
2.1
+5 (+5 boost)
7.1
"no"
1.8
-100 (banned)
-98.2
"maybe"
1.5
0 (neutral)
1.5
"sure"
0.9
0 (neutral)
0.9
"never"
0.4
-5 (-5 suppress)
-4.6
Ban tokens
logit_bias: {token_id: -100}
Block specific words
Force tokens
logit_bias: {token_id: +5}
Increase likelihood
Constrain output
Only 'yes' or 'no'
Binary classification
Real-World Example
A 99helpers chatbot for an insurance company must never use competitor names in responses. Using logit_bias: the team encodes the five main competitor brand name tokens and sets their bias to -100. Testing: when users mention competitors by name, the bot discusses the query without generating the competitor name in its response. Note: this doesn't prevent the bot from understanding the query—it only affects generation. For compound competitor names ('Competitor One' = two tokens), both tokens are biased. Edge case: sometimes the model generates a misspelling or abbreviation to work around the bias—monitoring competitor mentions in production caught 3 such cases in the first week.
Common Mistakes
- ✕Biasing individual tokens without considering multi-token words—most proper nouns and brand names require biasing multiple tokens, not just one.
- ✕Using extreme bias values (-100, +100) for normal vocabulary restrictions—moderate values (-10 to -30) reduce frequency while maintaining coherent generation; extreme values can break generation flow.
- ✕Relying solely on logit_bias for content policy enforcement—determined adversarial prompts can use synonyms or circumlocutions to express the same content without using the biased tokens.
Related Terms
Log Probabilities (Logprobs)
Logprobs are the log-probabilities the LLM assigns to each token in its output, enabling applications to measure generation confidence, detect low-certainty completions, and implement custom sampling strategies.
Guardrails
Guardrails are input and output validation mechanisms layered around LLM calls to detect and block unsafe, off-topic, or non-compliant content, providing application-level safety beyond the model's built-in alignment.
Temperature
Temperature is an LLM parameter (0-2) that controls output randomness: low values produce focused, deterministic responses while high values produce more varied, creative outputs.
LLM API
An LLM API is a cloud service interface that provides programmatic access to large language models, allowing developers to send prompts and receive completions without managing model infrastructure.
Structured Output
Structured output constrains LLM responses to follow a specific format—typically JSON with defined fields—enabling reliable parsing and integration with downstream systems rather than free-form text generation.
Ready to build your AI chatbot?
Put these concepts into practice with 99helpers — no code required.
Start free trial →