Interactive AI avatars have a fundamental flaw.  AI avatars are rather shallow characters.  They talk in generalities and don’t engage much more than you would expect from the role of a receptionist.  Not ditzy, but lacking the depth of knowledge on a subject. 

The platforms that generate and stream AI avatars, such as HeyGen, don’t provide direct control of parameters that affect hallucinations and other behavior like response length.  Further, the attachment of a knowledge base has limitations on size and how it is processed into a vector database.  The existing platforms can act as a user interface for an external custom LLM and RAG, which allows for complete control of all parameters and the knowledge base.  Of course, there are restrictions on what the avatar can actually physically do, such as body movements, which are  dependent on the functions available in the API for these platforms. 

What’s exciting is that by separating the avatar’s interface from the AI answer generation, we can create avatars that handle more complex tasks, have more interesting conversations, and genuinely feel smarter and more lifelike.

What’s RAG and why pair it with LLMs?

RAG combines AI’s creative capabilities with the ability to quickly fetch relevant info from a large number of documents, databases, video, audio, etc.  With the depth of a RAG knowledge base, interactions feel both smart and researched. By managing RAG and LLMs externally (instead of being limited to the built-in knowledge bases and instructions), you can create more of a personality for the avatar, in addition to, having it reference large amounts of internal data.  That internal data could be anything from a catalog of customer service scripts and instructions, to medical research for patient Q and A, to all the personal letters, articles, documents, books, and speeches of a historical figure.

Why external RAG and LLMs for Interactive AI Avatars

• Bigger Knowledge Bases: You can include a massive number of documents and resources, giving your avatar access to much richer information.

• Personality Plus: Greater control over LLM instructions helps craft avatars that feel genuinely relatable, personable, and context-aware.

• Better Performance: External setups can handle more processing power, making avatar interactions smoother, faster, and more responsive.

AI Avatar + RAG Examples

• Customer Support: Avatars can tackle tricky customer questions with ease, providing helpful, detailed answers.

• Education: Avatars can become awesome personal tutors, adapting lessons exactly to a student’s learning style.

• Healthcare: Imagine a knowledgeable avatar providing detailed, personalized support and medical information right when you need it.

Brand Experience and RAG Avatars as Art

As much as I hesitate to use words like “brand experience,” the combination of the visual experience of talking to a video person and the emotional experience of having a conversation with that human-lRAG and LLM technology keeps improving, businesses using interactive avatars will find new ways to impress customers, improve efficiency, and make digital experiences more enjoyable.

Video AI Avatar Generation and Streaming

The AI video avatar generation and streaming platforms that we have utilized are HeyGen, Tavus, and Simli.  Ravatar is the 3D AI avatar platform of note.  There are a few others.  Some, we have not had the chance to use yet.  Others did not have adequate API control for our use in interactive AI avatar development.

Summary

Using external LLMs and RAG tech with interactive avatars makes for a smarter, more engaging digital experience. It’s an exciting way for companies to stay ahead and provide interactions that users will genuinely love.