Key Concepts

Zep is a memory layer for AI assistants and agents that continuously learns from user interactions and changing business data. Zep ensures that your Agent has a complete and holistic view of the user, enabling you to build more personalized and accurate user experiences.

Using user chat histories and business data, Zep automatically constructs a knowledge graph for each of your users. The knowledge graph contains entities, relationships, and facts related to your user. As facts change or are superseded, Zep updates the graph to reflect their new state. Using Zep, you can build prompts that provide your agent with the information it needs to personalize responses and solve problems. Ensuring your prompts have the right information reduces hallucinations, improves recall, and reduces the cost of LLM calls.

This guide covers key concepts for using Zep effectively:

• How Zep fits into your application
• The Zep Knowledge Graph
• User vs Group graphs
• Managing changes in facts over time
• Business data vs Chat Message data
• Users and Chat Sessions
• Adding Memory
• Retrieving memory
• Improving Fact Quality
• Using Zep as an agentic tool
• Other Zep Features

How Zep fits into your application

Your application sends Zep messages and other interactions your agent has with a human. Zep can also ingest data from your business sources in JSON, text, or chat message format. These sources may include CRM applications, emails, billing data, or conversations on other communication platforms like Slack.

Zep fuses this data together on a knowledge graph, building a holistic view of the user’s world and the relationships between entities. Zep offers a number of APIs for adding and retrieving memory. In addition to populating a prompt with Zep’s memory, Zep’s search APIs can be used to build agentic tools.

The example below shows Zep’s memory.context field resulting from a call to memory.get(). This is an opinionated, easy to use context string that can be added to your prompt and contains facts and graph entities relevant to the current conversation with a user. For more about the temporal context of facts, see Managing changes in facts over time.

Zep also returns a number of other artifacts in the memory.get() response, including raw facts objects. Zep’s search methods can also be used to retrieve nodes, edges, and facts.

Memory Context

Memory context is a string containing relevant facts and entities for the session. It is always present in the result of memory.get() call and can be optionally received with the response of memory.add() call.

1
# pass in the session ID of the conversation thread
2
memory = zep_client.memory.get(session_id="session_id") 
3
print(memory.context)

FACTS and ENTITIES represent relevant context to the current conversation.
# These are the most relevant facts and their valid date ranges
# format: FACT (Date range: from - to)
<FACTS>
  - Emily is experiencing issues with logging in. (2024-11-14 02:13:19+00:00 -
    present) 
  - User account Emily0e62 has a suspended status due to payment failure. 
    (2024-11-14 02:03:58+00:00 - present) 
  - user has the id of Emily0e62 (2024-11-14 02:03:54 - present)
  - The failed transaction used a card with last four digits 1234. (2024-09-15
    00:00:00+00:00 - present)
  - The reason for the transaction failure was 'Card expired'. (2024-09-15
    00:00:00+00:00 - present)
  - user has the name of Emily Painter (2024-11-14 02:03:54 - present) 
  - Account Emily0e62 made a failed transaction of 99.99. (2024-07-30 
    00:00:00+00:00 - 2024-08-30 00:00:00+00:00)
</FACTS>
# These are the most relevant entities
# ENTITY_NAME: entity summary
<ENTITIES>
  - Emily0e62: Emily0e62 is a user account associated with a transaction,
    currently suspended due to payment failure, and is also experiencing issues
    with logging in. 
  - Card expired: The node represents the reason for the transaction failure, 
    which is indicated as 'Card expired'. 
  - Magic Pen Tool: The tool being used by the user that is malfunctioning. 
  - User: user 
  - Support Agent: Support agent responding to the user's bug report. 
  - SupportBot: SupportBot is the virtual assistant providing support to the user, 
    Emily, identified as SupportBot. 
  - Emily Painter: Emily is a user reporting a bug with the magic pen tool, 
    similar to Emily Painter, who is expressing frustration with the AI art
    generation tool and seeking assistance regarding issues with the PaintWiz app.
</ENTITIES>

You can then include this context in your system prompt:

The Knowledge Graph

Zep automatically constructs a knowledge graph for each of your users. The knowledge graph contains entities, relationships, and facts related to your user, while automatically handling changing relationships and facts.

Here’s an example of how Zep might extract graph data from a chat message, and then update the graph once new information is available:

Each node and edge contains certain attributes - notably, a fact is always stored as an edge attribute. There are also datetime attributes for when the fact becomes valid and when it becomes invalid.

User vs Group graphs

Zep automatically creates a knowledge graph for each User of your application. You as the developer can also create a “group graph” for memory to be used by a group of Users.

For example, you could create a group graph for your company’s product information or even messages related to a group chat. This avoids having to add the same data to each user graph. To do so, you’d use the graph.add() and graph.search() methods (see Retrieving memory).

Group knowledge is not retrieved via the memory.get() method and is not included in the memory.context string. To use user and group graphs simultaneously, you need to add group-specific context to your prompt alongside the memory.context string.

Managing changes in facts over time

When incorporating new data, Zep looks for existing nodes and edges in graph and decides whether to add new nodes/edges or to update existing ones. An update could mean updating an edge (for example, indicating the previous fact is no longer valid).

For example, in the animation above, Kendra initially loves Adidas shoes. She later is angry that the shoes broke and states a preference for Puma shoes. As a result, Zep invalidates the fact that Kendra loves Adidas shoes and creates two new facts: “Kendra’s Adidas shoes broke” and “Kendra likes Puma shoes”.

Zep also looks for dates in all ingested data, such as the timestamp on a chat message or an article’s publication date, informing how Zep sets the following edge attributes. This assists your agent in reasoning with time.

The valid_at and invalid_at attributes for each fact are then included in the memory.context string which is given to your agent:

# format: FACT (Date range: from - to)
User account Emily0e62 has a suspended status due to payment failure. (2024-11-14 02:03:58+00:00 - present)

Business data vs Chat Message data

Zep can ingest either unstructured text (e.g. documents, articles, chat messages) or JSON data (e.g. business data, or any other form of structured data). Conversational data is ingested through memory.add() in structured chat message format, and all other data is ingested through the graph.add() method.

Users and Chat Sessions

A Session is a series of chat messages (e.g., between a user and your agent). Users may have multiple Sessions.

Entities, relationships, and facts are extracted from the messages in a Session and added to the user’s knowledge graph. All of a user’s Sessions contribute to a single, shared knowledge graph for that user.

1
client = AsyncZep(
2
    api_key=API_KEY,
3
)
4
5
await client.user.add(
6
    user_id=user_id,
7
    email="[email protected]",
8
    first_name="Jane",
9
    last_name="Smith",
10
)
11
12
# Generate a new session identifier. You can map this to a business object in your app.
13
session_id = uuid.uuid4().hex
14
15
await client.memory.add_session(
16
    session_id=session_id,
17
    user_id=user_id,
18
)

The Session Guide contains more information on working with Sessions, including how to delete a Session.

Adding Memory

There are two ways to add data to Zep: memory.add() and graph.add().

Using memory.add()

Add your chat history to Zep using the memory.add() method. memory.add is session-specific and expects data in chat message format, including a role name (e.g., user’s real name), role_type (AI, human, tool), and message content. Zep stores the chat history and builds a user-level knowledge graph from the messages.

The example below adds messages to Zep’s memory for the user in the given session:

1
from zep_cloud.client import AsyncZep
2
from zep_cloud.types import Message
3
4
zep_client = AsyncZep(
5
    api_key=API_KEY,
6
)
7
8
messages = [
9
    Message(
10
        role="Jane",
11
        role_type="user",
12
        content="Who was Octavia Butler?",
13
    )
14
]
15
16
await zep_client.memory.add(session_id, messages=messages)

Additionally, for latency-sensitive applications, you can request the memory context directly in the response to the memory.add() call. Read more here.

Using graph.add()

The graph.add() method enables you to add business data as a JSON object or unstructured text. It also supports creating Group graphs by passing in a group_id as opposed to a user_id.

The example below adds JSON business data to Zep’s memory for the given user:

1
message = await zepClient.graph.add(
2
    data=json_data,
3
    user_id=user_id,
4
    type="json",
5
)

Retrieving memory

There are four ways to retrieve memory from Zep: memory.get(), memory.search_sessions(), graph.search(), and methods for retrieving specific nodes, edges, or episodes using UUIDs.

Using memory.get()

The memory.get() method is a user-friendly, high-level API for retrieving relevant context from Zep. It uses the latest messages of the given session to generate a context string for your prompt. It also returns recent chat messages and raw facts that may provide additional context for your agent. memory.get is user and session-specific and cannot retrieve data from group graphs.

The example below gets the memory.context string for the given session:

1
memory = zep_client.memory.get(session_id="session_id")
2
# the context field described above
3
context = memory.context

Using memory.search_sessions()

The memory.search_sessions() is a convenience method for searching user and session-specific facts and chat messages. The method returns a list of either facts or messages, depending on the search scope.

The search_sessions() method returns relevant facts from the graph, regardless of whether the source is chat history or business data.

The example below searches a user’s facts using the query text:

1
results = zep_client.memory.search_sessions(
2
    user_id="user_id", 
3
    text=query_text, 
4
    search_type="similarity", # or "mmr"
5
    search_scope="facts", 
6
    limit=10
7
)

Using graph.search()

The graph.search() method lets you search the graph directly, returning raw edges and/or nodes, as opposed to facts. You can customize search parameters, such as the reranker used. For more on how search works, visit the Graph Search guide. This method works for both User and Group graphs.

The example below returns the most relevant edges based on the query text. Note that the search scope defaults to edges.

1
search_results = await zep_client.graph.search(
2
    user_id=user_id,
3
    query=query,
4
    reranker="mmr",
5
    mmr_lambda=0.5, # tune diversity vs relevance
6
)

Retrieving specific nodes, edges, and episodes

Zep offers several utility methods for retrieving specific nodes, edges, or episodes by UUID, or all elements for a user or group. See the Graph API reference for more.

Example:

1
nodes = await zep_client.graph.node.get_by_user_id(user_id)

Improving Fact Quality

By using Zep’s fact rating feature, you can make Zep automatically assign a rating to every fact using your own custom rating instruction. Then, when retrieving memory, you can set a minimum rating threshold so that the memory only contains the highest quality facts for your use case. Read more here.

Using Zep as an agentic tool

Zep’s memory retrieval methods can be used as agentic tools, enabling your agent to query Zep for relevant information.

In the example below, a LangChain LangGraph tool is created to search for facts in a user’s graph.

1
from zep_cloud.client import AsyncZep
2
3
from langchain_core.tools import tool
4
from langchain_openai import ChatOpenAI
5
from langgraph.graph import StateGraph
6
from langgraph.prebuilt import ToolNode
7
8
zep = AsyncZep(api_key=os.environ.get('ZEP_API_KEY'))
9
10
@tool
11
async def search_facts(state: State, query: str, limit: int = 5):
12
    """Search for facts in all conversations had with a user.
13
    
14
    Args:
15
        state (State): The Agent's state.
16
        query (str): The search query.
17
        limit (int): The number of results to return. Defaults to 5.
18
    Returns:
19
        list: A list of facts that match the search query.
20
    """
21
    return await zep.memory.search_sessions(
22
      user_id=state['user_name'], 
23
      text=query, 
24
      limit=limit, 
25
      search_scope="facts"
26
    )
27
28
tools = [search_facts]
29
tool_node = ToolNode(tools)
30
llm = ChatOpenAI(model='gpt-4o-mini', temperature=0).bind_tools(tools)

Other Zep Features

Additionally, Zep builds on Zep’s memory layer with tools to help you build more deterministic and accurate applications:

• Dialog Classification is a flexible low-latency API for understanding intent, segmenting users, determining the state of a conversation and more, allowing you to select appropriate prompts and models, and manage application flow.
• Structured Data Extraction extracts data from conversations with high-fidelity and low-latency, enabling you to confidently populate your data store, call third-party applications, and build custom workflows.