Synthetic data

Synthetic Text: The Moment for Enterprise Applications Is Now

Leveraging technology that generates text is coming to the main theaters and Forbes is the most recent one: “The Biggest Opportunity In Generative AI Is Language, Not Images”

Different names are in use: generative AI, as in the article; synthetic text, following the popular term “synthetic data”; NLG (Natural Language Generation) is the most traditional term maybe not so trendy just for that reason.

Synthetic Text, as we will call it, started to follow the path of synthetic image recently. Synthetic image and video have been a huge success in sectors like self-driven cars.

For text, the initial successes have come from tabular data. In structured or tabular text, what’s generated is names (James O’Reilly, Bethesda Pharmaceuticals Inc.) or phrases (Junior Accountant, out of order) properly combined in tables or relational structures.

The next step in synthetic text seems to be unstructured data, where actual full sentences are produced, rather than phrases or names in tables.

Report generation, based on numeric tables, is an intermediate step between generating tabular data and actually generating full sentences from scratch. It’s very popular for sectors like e-commerce, finance or pharma.

At Bitext, we are focused on generating unstructured text for customer service applications and solving problems like:

  • How do I generate hundreds/thousands of variations of a customer request (like “cancel my account”) so I can train a virtual assistant?
  • Can I use text generation to produce comprehensive evaluation datasets?
  • How do you express a given request (“can I cancel my account now?”) in colloquial register (“can u pls cancel account”) because my target is young adults?

 

You can take a look at a sample data in our GitHub Repository

admin

Recent Posts

Bitext Linguistic Analyzer Improves Elasticsearch’s English Search Quality

Our earlier German MIRACL benchmark showed that linguistic analysis can substantially improve lexical search in…

18 hours ago

Hybrid Search Is the New Baseline: Better Lexical Search Is the Next Advantage

Vector search, also known as semantic search, has transformed enterprise search in the past few…

2 months ago

Bitext Linguistic Analyzer beats every Elasticsearch Configuration on German Search Quality

In a search benchmark for German, Bitext Linguistic Analysis SDK returned more relevant results and…

2 months ago

Stemming kills AI Accuracy: Why German Search Needs Lemmatization

Search systems have relied on stemming for decades. The reason is simple: stemming is fast,…

3 months ago

Some of your RAG-related issues have an easy & quick solution: decompounding

Most teams working with Elasticsearch, OpenSearch or RAG pipelines focus on ranking, embeddings or model…

5 months ago

Some of your RAG-related issues have an easy & quick solution: lemmatization

Some RAG issues have a simpler fix than people think: better text normalization. One common…

5 months ago