Named Entity Recognition (NER)
Definition
Named Entity Recognition (NER) is a sequence labeling task that detects mentions of specific real-world objects in text and assigns them to predefined categories. Standard NER categories include: PER (person names), ORG (organizations), LOC (locations), DATE (temporal expressions), MONEY (monetary values), PRODUCT (product names), and EVENT (named events). NER is typically implemented as a sequence labeling model that assigns a BIO (Beginning-Inside-Outside) tag to each token: B-PER for the first token of a person name, I-PER for continuation, O for non-entity tokens. Modern NER systems use transformer-based models (BERT, RoBERTa fine-tuned for NER) that achieve F1 scores above 90% on standard benchmarks, significantly outperforming earlier rule-based and CRF-based approaches.
Why It Matters
NER is a foundational NLP capability that enables structured data extraction from unstructured text. For chatbot applications, NER extracts the specific entities needed to take action: an order number, a product name, a date range, or a customer identifier mentioned in the user's message. Without NER, chatbots must either ask follow-up questions to collect structured information or rely on users formatting information in expected ways. With NER, 'I need to check my order from last Thursday—it's order 98234' automatically extracts {order_id: '98234', date: 'last Thursday'}. For 99helpers customers building integrations between chatbots and backend systems, NER is the bridge between natural language user input and structured API calls.
How It Works
NER using spaCy (Python): import spacy; nlp = spacy.load('en_core_web_trf'); doc = nlp('Microsoft acquired GitHub for $7.5 billion in 2018'); for ent in doc.ents: print(ent.text, ent.label_) → Microsoft (ORG), GitHub (ORG), $7.5 billion (MONEY), 2018 (DATE). For domain-specific entities not in standard categories (product SKUs, internal system names, custom identifiers), fine-tuned NER models are needed. Using LLMs for NER: 'Extract all entities from the following text. Return JSON with entity text and type. Text: [text].' LLMs achieve excellent NER performance zero-shot for standard categories and few-shot for custom categories, without the need for labeled training data.
Named Entity Recognition — Entity Span Tagging
Input sentence with entity spans
BIO Tagging Scheme
Real-World Example
A 99helpers customer's chatbot for an e-commerce platform uses NER to process support queries. Input: 'My Airpods Pro 2nd gen case stopped charging. I bought it on March 15th, order #AP-7823.' NER extracts: {product: 'Airpods Pro 2nd gen', issue: 'case stopped charging', purchase_date: 'March 15th', order_id: 'AP-7823'}. This structured data automatically: looks up the order in the system to verify warranty status, routes to the Apple product support knowledge base, and pre-fills the warranty claim form with extracted information. What previously required an agent asking 3-4 follow-up questions is automated by NER.
Common Mistakes
- ✕Using only standard NER categories without training for domain-specific entities—out-of-the-box NER misses internal product names, SKUs, and custom identifiers that are critical for your application.
- ✕Treating NER as 100% reliable—NER models make mistakes on ambiguous text, rare entities, and non-standard formatting; build fallback handling for extraction failures.
- ✕Not handling multi-token entities as complete spans—'New York City' is a single LOC entity; processing token by token misses multi-word entity spans.
Related Terms
Natural Language Processing (NLP)
Natural Language Processing (NLP) is the field of AI focused on enabling computers to understand, interpret, and generate human language—powering applications from chatbots and search engines to translation and sentiment analysis.
Entity Extraction
Entity extraction is the process of identifying and pulling specific pieces of information from a user's message — such as names, dates, order numbers, or locations. These extracted values (entities) fill in the details the chatbot needs to complete a task, working alongside intent recognition to fully understand the user's request.
Information Extraction
Information extraction automatically identifies and structures specific facts from unstructured text—who did what, when, and where—transforming free-form documents into queryable databases.
Intent Detection
Intent detection classifies user messages into predefined categories representing the user's goal—such as 'check order status' or 'report a bug'—enabling chatbots to route queries to the appropriate responses or workflows.
Slot Filling
Slot filling is the dialogue management process of collecting all the required pieces of information (slots) needed to complete a task. The chatbot systematically asks for any missing slots — like date, time, or account number — until it has everything needed to fulfill the user's request.
Ready to build your AI chatbot?
Put these concepts into practice with 99helpers — no code required.
Start free trial →