Key Concepts

Zep is a memory layer for AI assistants and agents. With Zep, you can build AI assistants and agents that continually learn from user interactions, allowing them to offer personalized and accurate user experiences.

As your users interacts with your LLM application, Zep learns facts about them. With very low latency, you can retrieve relevant user facts to include in your prompts. Doing so reduces hallucinations, improves recall, and reduces the cost of LLM calls.

Zep builds on Zep’s memory layer with tools to help you build more determinstic and acccurate applications:

• Dialog Classification is a flexible low-latency API for understanding intent, segmenting users, determining the state of a conversation and more, allowing you to select appropriate prompts and models, and manage application flow.
• Structured Data Extraction extracts data from conversations with high-fidelity and low-latency, enabling you to confidently populate your data store, call third-party applications, and build custom workflows.

Long-term Memory

Incorporating an entire chat history into a prompt can lead to hallucinations, poor recall, and slow, costly LLM calls. Our goal is to enable you to ground the LLM in appropriate context by incorporating relevant historical data from chat histories, and other user and business data, into your prompts, resulting in significantly improved LLM comprehension and knowledge.

The RAG (Retrieval-Augmented Generation) method for accessing past conversations relevant to current messages presents significant challenges. Merely embedding and subzsequently retrieving messages related to the current dialogue is ineffective. Individual messages often lack sufficient context, and the retrieval is based on relevance rather than chronological order, leading to LLM confusion.

Facts

Zep addresses the challenges presented by RAG by extracting key observations or facts from chat histories and continuously updating a fact table as the conversation evolves. This process ensures that conflicting facts are resolved and deduplicated, and the relationship between conflicting information is preserved. A stylized example below:

Learn more about Facts

Chat History and Sessions

Sessions represent a conversation, and can be associated with Users in a one-to-many relationship.

Chat messages are added to Sessions, with each session having many messages. Facts are extracted from a Session’s chat history, and are stored in the Session’s fact table.

Building a Prompt with Zep

Zep’s artifacts, such as Facts and Chat History, can be included in your prompts to augment the LLM’s understanding of the conversation. Typically, you’d use a Chat API and structure your prompts as a series of messages: System, User, Assistant` etc.

More on this in the Building a Prompt section.