Executive Summary 

In a search benchmark for German, Bitext Linguistic Analysis SDK returned more relevant results and missed fewer useful passages than every tested Elasticsearch configuration. In short, Bitext Linguistic Analysis improves German search quality by up to 36%.

The benchmark uses the test data from the MIRACL dataset with 700+ queries and 7,000+ manually-annotated relevance judgments and focuses on the top 10 results, the most relevant for RAG:

• Compared with Elasticsearch’s default analyzer, Bitext SDK improved precision by 35.0%, recall by 39.6%, and their combined F1 score by 36.1%

• Compared with Elasticsearch’s built-in German analyzer, Bitext increased precision by 8.7%, and recall by 13.6%, and 9.6% in F1

• Compared to the recommended Elastic German decompounding setup precision by 4.3%, recall by 10.2%, and F1 by 5.5%.

For users, the impact is straightforward: more relevant results, less noise and hallucinations, stronger candidate sets for reranking and RAG, and better evidence for search applications and AI agents before any expensive downstream model is applied. Because these gains were confirmed on an industry-standard test set, they provide strong evidence that better linguistic analysis can improve retrieval quality across enterprise search, hybrid retrieval, RAG grounding, and agentic workflows.

The Benchmark Data

Text analysis is one of the most important, and frequently underestimated, parts of search and AI pipelines. Before lexical search, vector search, reranking, RAG, or an AI agent can use a document, the system must decide which words and linguistic relationships to represent. The default option, no text analysis, can miss valid morphological relationships that convey meaning relations. Classical stemming can discover some of these morphological relationships and improve recall, but it also hurts precision because analysis is based on word endings rather than actual word morphology, as explained in our analysis of why German search needs lemmatization. Splitting compounds exposes some useful subwords, but mostly introduces duplicates and noisy terms.

Bitext Linguistic Analysis SDK addresses this problem through a combination of:

• original-form preservation and stopword handling (provided by Elastic)

• German-specific normalization (provided by Elastic)

• full-token lemmatization (provided by Bitext SDK)

• compound analysis and lemmatization (provided by Bitext SDK)

We evaluated this pipeline in a controlled Elasticsearch benchmark using German MIRACL datasets. The MIRACL project (Multilingual Information Retrieval Across a Continuum of Languages) is an open research initiative, widely adopted by academic groups across the world. It was created to provide high-quality datasets and benchmarks for evaluating retrieval systems across many languages, including those with relatively limited digital resources.

The Benchmark Setup

The final evaluation used:

• as target document database: 15,866,222 German Wikipedia passages;

• as queries: 712 German MIRACL test topics;

• as evaluation judgements: 7,317 MIRACL test relevance judgments;

• the same Elasticsearch version, corpus, settings, and evaluation code for every configuration;

• evaluation is focused on top 10 results provided, since they are the only ones evaluated by human experts in the MIRACL dataset; and also the ones that users and AI-based pipelines like RAG or agents typically use;

• frozen indices and query configurations, with no tuning after test results were observed.

The Four Competing Approaches to German Text Analysis

The comparison included four different approaches to text analysis, three approaches provided by Elastic, and a fourth one based on Bitext Linguistic Analysis. The goal is to evaluate if Bitext approach can improve all three alternatives provided by Elastic, since they are the most commonly used ones by major search engines.

• Approach 1. Elasticsearch default standard analysis: Standard tokenization and lowercasing, without any linguistic analysis –stemming or lemmatization.

• Approach 2. Elasticsearch built-in German analyzer: Adds to Approach 1 normalization of German phenomena, stop-word removal, and light stemming.

• Approach 3. Elastic German decompounder: Adds to Approach 2 analysis of full compound stems and splitting of component stems.

• Approach 4. Bitext linguistic analysis: Builds on Approach 2 and generates all linguistic data from scratch providing Bitext lemmatization (instead of Elastic stemming) and Bitext decompounding (instead of Elastic compound stemming) for German.

The Results

Meaning of the metrics:

• nDCG measures ranking quality in the first ten results, rewarding systems that place known relevant passages higher.

• Precision measures the proportion of the first ten results that are relevant.

• Recall measures how much of the known relevant material appears on the first result page.

• F1 balances first-page precision and recall.

• MRR measures how early the first relevant result appears.

The full evaluation covered all 712 topics (queries) and Approach 4, based on Bitext Linguistic Analysis, produced improvement across all dimensions over Approach 3. The Bitext configuration had the highest nDCG, Recall, Precision, F1, and MRR among the reported configurations.

Examples of Actual Search Cases

At the end of this document we add some examples of different linguistic phenomena in German and how they are handled by different approaches. These examples help understand the reason behind the improvements provided by Approach 4.

Why the Results Matter

The benchmark shows that linguistic analysis of input text, prior to executing downstream tasks like search, improves results, mostly through lemmatization and compounds.

Elastic’s built-in German analysis (Approach 2) produced a major improvement over standard analysis (Approach 1). Adding index-time decompounding (Approach 3) improved the Elastic result further. This confirms that German compound structure provides useful retrieval evidence.

Bitext Linguistic Analysis (Approach 4) improved again over all 3 approaches by replacing approximate full-token stemming with dictionary-based lemmatization and by linguistically splitting and representing compound components as lemmas. This provides a more linguistically controlled document representation that combines:

original token

+ full-token lemma

+ component lemmas

The system therefore gained component-level matching in the document index without indiscriminately expanding every query.

When text preprocessing misses valid word forms or compound relationships:

• relevant documents never enter the candidate set;

• rerankers have nothing to rescue;

• RAG systems ground answers in weaker evidence;

• AI agents may fail to find the source required to complete a task.

When token analysis (stemming) introduces noisy terms:

• search term statistics are distorted;

• irrelevant documents may be promoted;

• more expensive downstream models must compensate.

In short, linguistic analysis strengthens the retrieval layer before those expensive downstream costs are incurred.

What this means for search and AI systems

Better linguistic analysis improves the retrieval evidence available to every downstream stage:

• For lexical search, it improves lexical matching and top-page ranking.

• For hybrid retrieval, it strengthens the sparse lexical channel before it is combined with embeddings.

• For RAG, it improves the candidate passages supplied to rerankers and language models.

• For AI agents, it reduces the risk that a useful document is missed because the query and document use different inflected or compound forms.

Most importantly, the benchmark does not claim that linguistic analysis replaces vector retrieval, reranking, or LLM reasoning. It shows that these AI-based pipelines work better, when they have a stronger starting point based on linguistic analysis, since lexical evidence is normalized accurately.

Conclusion

The German MIRACL evaluation supports three conclusions.

• German-specific stemming (Approach 2) improves over Elasticsearch’s default standard analyzer (Approach 1).

• Compound-aware indexing (Approach 3) improves over stemming (Approach 2).

• Bitext’s linguistic analysis (Approach 4) produced the strongest top-ranking result combines full-token lemmatization and compound splitting plus lemmatization using linguistic analysis.

The practical takeaway is not that linguistic analysis replaces modern ranking or generation systems; it is that those systems perform better when the lexical evidence entering the pipeline is more accurate.

Search quality can be improved before adding another model, reranker, or generation layer. Full-token lemmatization and controlled compound analysis provide stronger lexical evidence at the beginning of the retrieval pipeline.

Annex – Examples of how Approach 4 contributes to improving result

The aggregate benchmark shows that Approach 4, the Bitext configuration, delivered the strongest overall German retrieval results. This section includes some examples of queries and results that make the practical differences between approaches easier to see. In every example, the ranks refer to the same passage that human evaluators marked as relevant.

These examples show two complementary benefits. Lemmatization connects different grammatical forms, such as wiegt, wogen, and wog—weighs and weighed. Compound analysis uncovers useful concepts inside words such as Lehrerausbildung—teacher training—and Zahlzeichen—numeral symbols.

For users, the result is straightforward: relevant information appears closer to the top of the results, where it is more likely to be seen, selected, and passed to downstream systems such as rerankers, RAG applications, and AI agents.