Feature Spotlight: Enhanced Entity Extraction in Babel Street Insights
By Deric Lambdin
Open-source intelligence teams are drowning in irrelevant information within thousands of documents, reports, messages, and posts, often across languages and platforms. They need tools that will help them quickly surface and prioritize relevant documents for their review.
Relevancy is frequently determined by the people, organizations, locations, and products mentioned. That’s where multilingual entity extraction comes in — and why it is a core, foundational capability in Babel Street Insights.
In this feature spotlight, we’ll break down what entity extraction is, why it’s critical for modern OSINT workflows, and how high-quality entity extraction in Insights helps analysts move faster from discovery to decision.
What is entity extraction?
Entity extraction (also called named entity recognition or NER) is the process of adding structure to unstructured text by automatically identifying key entities: people, organizations, locations, and products. These entities are added to each document as structured metadata.
We’ve explored the fundamentals of entity extraction in earlier posts, including What Is Entity Extraction? and What’s the Difference Between Entity Extraction and Entity Resolution?, but the short version is this: entity extraction prepares raw text to more readily become usable intelligence.
Benefits of entity extraction for the human OSINT investigator
OSINT investigations depend on connecting signals across large, diverse, and often multilingual datasets. Entity extraction accelerates that work for both humans and AI.
For the human analyst
Entity extraction isn’t just about filtering; it’s about helping analysts get their bearings in a new domain and more rapidly gather relevant data.
Entity extraction helps human analysts:
- Discover unknown actors and more quickly understand what matters in an unfamiliar investigation space
- Learn the investigative landscape faster by filtering and refining search results to look at the context of each entity in this topic area
- Refine search parameters with confidence to better target the area of interest or investigative path
For Insights Investigator
Entity extraction also boosts the power of Insights Investigator, Babel Street’s agentic AI capability, whose answers are only as good as the data it is consuming. Entity-enriched search constrains and grounds AI behavior, clearly identifying when a potentially ambiguous entity such as “Paris” refers to a location or a person. Investigator’s human-in-control design enables users to edit Investigator-built search queries to reduce false positives and reference more relevant data.
As a result, entity extraction enables Investigator to:
- Achieve more focused source collection
- Produce better-grounded answers
- Support transparent, editable, traceable AI workflows
What’s better about entity extraction in Insights?
Many multilingual entity extraction models first translate documents to English before extracting entities, which compounds the errors from machine translation — particularly in complex languages such as Arabic, Chinese, Japanese, Korean, Persian, and Russian.
Entity extraction in Insights removes that risk by directly extracting entities from native-language text. Furthermore, entity extraction isn’t a bolt-on feature or a post-search assist. It’s applied automatically at ingestion, making entity metadata available to every document and every functionality within Insights.
How entity extraction shows up in Insights
Insights users will see the benefits of entity extraction in multiple areas:
- Highlighted entities appear in every document in a search result for easy skimming
- Entities found in the search results populate the search refinement pane for use as filters
- Entity-enriched exports can include extracted entity metadata for each document within exported PDFs to share findings, support reporting, or conduct follow-on analysis in external tools
Laying the groundwork for advanced investigation
Enhanced entity extraction is also the prerequisite for downstream capabilities such as entity resolution and Insights Investigator and helps teams cut through the noise to more quickly find what matters as the foundation of scalable, trustworthy OSINT.