Category: Speech and text comprehension

Speech and text comprehension

Roberta SPEAKER – Easily design your own dialog assistant

Advantages

  • Creativity & Innovation
  • Knowledge & Complexity reduction
  • Time efficiency & Agility

Functional areas

  • Marketing / Sales
  • Support / Customer service

To the demonstrator

→ Go to the »Roberta SPEAKER« clip
To overview

Roberta SPEAKER – Easily design your own dialog assistant

Develop your own dialog assistant in an intuitive way – with the visual, no-code programming interface »Open Roberta®«!

In which areas does the AI application offer the greatest benefit?

Speech technologies play a key role in the development of new digital services and technologies and already serve as a reliable assistant at home. Whether Alexa or Siri, voice communication with computers has long since arrived in everyday life: A quick question about the likelihood of rain or the risk of traffic jams on the way home, as well as controlling music or lights within your own four walls – communication with voice assistants is becoming increasingly common. But how can voice assistants be used in companies or even in schools?

Data-secure, customized – Roberta SPEAKER for companies and in education

The KI.NRW demonstrator »Roberta SPEAKER« allows even programming novices to develop dialog functions easily and intuitively on their own. Users from different industries can benefit from this: The technology enables companies to develop their own dialog assistants, for instance to control their machines via voice commands. In addition to the autonomous design of the dialogs, the advantages also include data security, because unlike many commercial voice assistants, Roberta SPEAKER does not require an internet connection – all data is processed locally. Teachers and educational stakeholders will also be able to use Roberta SPEAKER in the future to better teach young people about the use of AI technologies in everyday life and build their digital skills.

The programming language for dialog control is NEPO®, which is assembled on the open-source platform Open Roberta from Fraunhofer IAIS using »drag and drop«, thus avoiding obstacles such as typing or syntax errors. The speech recognition model can be adapted specifically to the user’s own requirements and thus run on a microcomputer. As a result, there is no need for expensive and complex hardware. For the communication between users and voice assistants, additional elements such as a microphone and speakers are also included.

The demonstrator was developed as part of the SPEAKER project funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK). The SPEAKER project aims to establish a leading voice assistance platform »made in Germany« for business-to-business (B2B) applications. The platform is intended to be open, modular, and scalable and provide technologies, services, and data via service interfaces. The consortium leaders are the Fraunhofer Institutes IAIS and IIS.

What are Roberta SPEAKER’s quality characteristics?

  • Low effort: The KI.NRW demonstrator shows that AI language models can work even on a small (portable) processor. This opens up innovative functional possibilities. One of the benefits is that the AI models work locally, so there is no need for an internet connection. This way, the Roberta SPEAKER box demonstrates that voice assistants can be implemented almost anywhere to interact with people and carry out simple tasks with little effort.
  • Intuitive and individually customizable: The intuitive programming interface allows everyone to easily develop functioning program sequences, so that students can generate their own voice commands using the demonstrator. The programming interface is the Open Roberta Lab, a freely available, data-secure, and open programming platform developed by the educational initiative »Roberta® – Learning with Robots« by Fraunhofer IAIS.
  • Easy communication: By using artificial intelligence, users can communicate with the voice assistants via spoken language. The dialog assistant understands questions and commands, can derive actions from the user’s intentions and can formulate answers and communicate these via the loudspeaker or derive actions.
  • Powerful and resource-saving: The voice-controlled box is only ready for use when the artificial intelligence technologies, the AI models, can function on a small processor. This is why the developers are focusing on resource-saving AI technology.      
»Dialog systems are everywhere. With ›Roberta Speaker‹, we make it possible for anyone to create their own dialogs to control IoT devices using drag-and-drop with almost no prior knowledge.«
Thorsten Leimbach
Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS

Which AI technology is used in the KI.NRW demonstrator?

Automated Speech Recognition (ASR)

Technical systems capable of understanding spoken commands allow for natural communication between humans and machines. Speech recognition reliably converts spoken information into digital text in real time – even under difficult conditions, such as when there is background noise, as can occur in an industrial environment, for example, or when regional dialects are spoken.

Intent Recognition

Only domain-specific knowledge makes the language system useful in certain areas of application. Intent recognition, the recognition of intentions from the spoken text, plays a key role here. An intent classifier recognizes the subject of the text and searches for the factual answer. With the help of verbalization techniques, the system then ensures that the response is fully formulated.

Text-to-Speech (TTS)

In human-machine interaction, it is often an advantage when text information does not have to be read, from a display for instance, but is transmitted using natural speech. Based on deep learning technology, state-of-the-art algorithms generate highly natural-sounding speech output with excellent intelligibility and fluent intonation.

What does the AI demonstrator show?

The KI.NRW demonstrator »Roberta SPEAKER« allows both companies and students from different types of schools, such as secondary schools or vocational schools, to develop their own voice assistant using a simple programming interface. In this way, the expression »do it yourself« is given a new meaning and people can start learning about AI technologies without any prior knowledge.

Request a non-binding consultation with our experts now!

Where can I find more information?

Study »Modern language technologies«

Find out where we encounter modern language technologies in everyday life and at work and what economic opportunities they offer.

Go to the study

Roberta SPEAKER for companies

Are you interested in using Roberta SPEAKER to conduct AI trainings in your company or would you like to know how you can integrate the Fraunhofer language technology into your processes?

Enquire now!

Roberta SPEAKER in education

Do you want to use Roberta SPEAKER in an educational context?

Enquire now!

Contact the team of developers

Thorsten Leimbach

Head of Smart Coding and Learning

Fraunhofer IAIS
Schloss Birlinghoven
53757 Sankt Augustin

Phone +49 2241 142404

send email

Beate Jost

Technical Manager at Roberta

Fraunhofer IAIS
Schloss Birlinghoven
53757 Sankt Augustin

Phone +49 2241 142441

send email

Dr.-Ing. Oliver Walter

Team leader
Real Time Speech Recognition

Fraunhofer IAIS
Schloss Birlinghoven
53757 Sankt Augustin

Phone +49 2241 2541

send email

Kevin Reich

Research associate

Fraunhofer IAIS
Schloss Birlinghoven
53757 Sankt Augustin

Phone +49 2241 2552

send email

Speech and text comprehension

Chatbot with knowledge graph
“Covid Q&A”

Currently, no more data is supplied for our demonstrator. The prototype is temporarily not available.

Advantages

  • Real-time results 24/7

Functional areas

  • Human Resources
  • IT / Infrastructure
  • Marketing / Sales
  • Production / Assembly
  • Support / Customer service
To overview

Chatbot with knowledge graph
“Covid Q&A”

The chatbot demonstrates the functionality of an online dialog system that uses the strengths of a knowledge graph.

Where does the AI application offer the greatest benefit?

A chatbot supported by AI offers a wide range of possible applications and assists companies in any area of communication with employees or customers, regardless of the industry. A classic example are customer-oriented service offerings that are made available twenty-four hours a day. Companies use them on websites, in online stores, on support pages, in apps, or via instant messaging systems to simplify navigation on websites, to answer specific customer inquiries, or to structure access to service and customer support.

However, chatbots can be used successfully not only in external but also in internal corporate communications. Application examples can be found in the onboarding of new employees, in HR or administrative processes such as questions about vacation requests or payroll tax settlements, as well as in the support of complex assembly instructions in production.

Chatbots can also be supplemented by the component of acoustic speech recognition as well as acoustic speech synthesis. In this extended form, we speak of voice assistants (voicebot), similar to Siri or Alexa.

What are the quality indicators of such AI applications?

Understanding texts with Natural Language Understanding (NLU)

NLU methods are based on semantic representations of texts. They can understand and map relationships between words. These semantic representations exceed the possibilities of the classical rule-based methods of text mining.

Access information and prepare dialogs with Dialog Management (DM) and Knowledge Graph (KG)

Knowledge Graphs structure data and knowledge, enable semantic linking, and in many cases are the basis for making artificial intelligence applications explainable and providing results that are comprehensible to humans.

Generate texts with Natural Language Generation (NLG)

Text synthesis is the counterpart to text comprehension. Here, text is generated automatically, which can afterwards be transformed into acoustic speech signals.

“Knowledge graphs that integrate different data sources,
form the basis for many AI applications and assistants.”
Prof. Dr. rer. nat Jens Lehmann
Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS

What does the demonstrator of KI.NRW show?

An AI-based dialog system (“question answering system”) shapes the search for information more efficiently and conveniently for the user. The KI.NRW demonstrator shows such an AI-supported chatbot for querying Corona case numbers worldwide. The animations explain step by step how it operates.

Step 1: Understanding natural language

By using AI, speech recognition is oriented to real conversations of humans and the so-called natural language (“natural language understanding NLU”). The models extract information from the text, which they represent internally in a way that allows further processing. This enables the system to understand less common terms, dialects or everyday language. Associations and connections between words are also taken into account appropriately, for example, that the word invoice is related in content to the word payment.

Step 2: Structure data and knowledge

Knowledge graphs structure data and knowledge, enable semantic linking and, in many cases, are the basis for shaping AI applications explainable. A knowledge graph is able to combine a wide variety of information sources into a dynamic knowledge base. In the case of the KI.NRW demonstrator, Corona case numbers from Johns Hopkins University and the Robert-Koch-Institut are included. They are made accessible with the status of the previous day via a knowledge graph.

Step 3: Generate answer

Finally, an answer is generated that matches the asked question. Thus, this process is the matching counterpart to the first step, the understanding of language. The structured data is now converted into text and output. The output as an acoustic signal can also follow here (for example with the so-called voicebots).

The AI chatbot in action

Try out the AI chatbot for yourself: As an example, the chatbot was set up on data that depicts the worldwide case numbers surrounding the Corona pandemic. Anyone who asks a corresponding question in English via the chat window will receive an immediate answer.

Test questions for the knowledge query can be the following:

  • “Are there new cases in Mexico?”
  • “How many cases were there in total in Germany until 25th October 2020?”
  • “How many new cases were found in Argentina on 10th November 2020?”
  • “Which country had the highest number of cases on 8th November 2020?”

Where is more information to be found?

AI provider from NRW

Our AI map illustrates who offers AI methods related to “knowledge and inference” in their portfolio.

To the AI map

AI methods around knowledge and inference

Many “good practice examples” as inspiration for the possible applications of this AI.

To the examples

AI products “made in NRW”

Filter our AI map by “knowledge and inference”.

To the AI map

Contact us

Roman Teucher

Research Engineer

Fraunhofer IAIS
Zwickauer Str. 46
01069 Dresden

Phone +49 351 85477961

E-Mail senden

To the top