Long-term Memory for LangChain Apps

Building an Agent using LangChain and Zep Open Source

You are viewing the Zep Open Source v0.x documentation. This version is no longer supported, and documentation is provided as a reference only.

The current documentation for Zep Community Edition is available here.

Langchain Python and LangchainJS ship with a ZepMemory class.

Managing Chat History Memory

Zep’s ZepMemory class can be used to provide long-term memory for your Langchain chat apps or agents. Zep will store the entire historical message stream, automatically summarize messages, enrich them with token counts, timestamps, metadata and more.

You can also provide your bot or agent with access to relevant messages in long-term storage by using Zep’s built-in vector search.

Note: Installing Zep

A Zep server install is required. You will also need to have the Zep Python SDK or Zep typescript SDK installed in order to use the Langchain ZepMemory class.

See the Zep Quick Start Guide for details.

Using Zep as a LangChain Memory Store

Using Zep with the LangChain’s Python LangChain Expression Language (LCEL)

The ZepChatMessageHistory class is used to provide long-term memory to LangChain apps built using the LangChain Expression Language (LCEL).

First we’ll create a ChatPromptTemplate that includes a placeholder for the message history. Then we’ll create a ChatOpenAI model and chain it to the prompt.

1from langchain.chat_models import ChatOpenAI
2from langchain.memory.chat_message_histories import ZepChatMessageHistory
3from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
4from langchain_core.runnables.history import RunnableWithMessageHistory
5
6prompt = ChatPromptTemplate.from_messages(
7    [
8        ("system", "You're an assistant who's good at {ability}"),
9        MessagesPlaceholder(variable_name="history"),
10        ("human", "{question}"),
11    ]
12)
13
14chain = prompt | ChatOpenAI(model="gpt-3.5-turbo-1106", api_key=openai_api_key)
15
16session_id = "your-users-unique-session-id"

Now we’ll instantiate a ZepChatMessageHistory, provide it with a unique session ID, and the Zep server URL and API key (if required). We’ll also create a RunnableWithMessageHistory that will provide the message history to the chain. We’ll specify the input and history message keys. The input messages key is the key that the chain will use to access the input messages. The history messages key is the key that the chain will use to access the message history. Ensure that these match your prompt template.

1zep_chat_history = ZepChatMessageHistory(
2    session_id=session_id,
3    url="http://localhost:8000",
4    # api_key=<your_api_key>,
5)
6
7chain_with_history = RunnableWithMessageHistory(
8    chain,
9    lambda session_id: zep_chat_history,
10    input_messages_key="question",
11    history_messages_key="history",
12)
13
14chain_with_history.invoke(
15    {"ability": "math", "question": "What does cosine mean?"},
16    config={"configurable": {"session_id": session_id}},
17)
AIMessage(content='Cosine is a trigonometric function that relates the angle of a right-angled triangle to the ratio of the length of the adjacent side to the length of the hypotenuse. In simpler terms, cosine is a way to calculate the ratio of the length of one side of a right-angled triangle to the length of the hypotenuse, given an angle. This function is widely used in mathematics, physics, engineering, and many other fields.')

Let’s invoke the chain again, this time with a different question. We’ll also print the message history to see what’s been stored.

1chain_with_history.invoke(
2    {"ability": "math", "question": "What's its inverse"},
3    config={"configurable": {"session_id": session_id}},
4)

The prior message context was used to answer the question. Zep automatically generates summaries of the message history. These will be added to the message history sent to LAngChain and can be used to provide long-term context to the chain.

AIMessage(content='The inverse of the cosine function is called arccosine or inverse cosine. It is denoted as "arccos" or "cos^(-1)". The arccosine function takes a value between -1 and 1 as input and returns the angle in radians whose cosine is the input value. In other words, if y = arccos(x), then x = cos(y). The arccosine function is the inverse of the cosine function, allowing us to find the angle given the cosine value.')

Using LangChain’s ZepMemory class

Note: ZepMemory is used with the legacy LangChain Python API

The following examples illustrate using Zep as a Langchain memory store. In Python, the ZepMemory class is used when building LangChain app’s with the legacy API. See above for details on building apps with the LangChain Expression Language (LCEL).

Python

Using Zep as your Langchain app’s long-term memory simple: initialize the ZepMemory with your Zep instance URL, API key, and your user’s session identifier.

ZepMemory instantiation
1from langchain.memory import ZepMemory
2
3# Set this to your Zep server URL
4ZEP_API_URL = "http://localhost:8000"
5ZEP_API_KEY = "<your JWT token>" # optional
6
7session_id = str(uuid4())  # This is a unique identifier for the user
8
9# Set up ZepMemory instance
10memory = ZepMemory(
11    session_id=session_id,
12    url=ZEP_API_URL,
13    api_key=zep_api_key,
14    memory_key="chat_history",
15)

TypeScript

Using Zep as your Langchain app’s long-term memory simple: initialize the ZepMemory with your Zep instance URL and your user’s session identifier (see Zep Concepts) and then utilize it as the chat memory for a Langchain Conversation.

ZepMemory instantiation
1import { ChatOpenAI } from "langchain/chat_models/openai";
2import { ConversationChain } from "langchain/chains";
3import { ZepMemory } from "langchain/memory/zep";
4import { v4 as uuidv4 } from "uuid";
5
6// Set this to your Zep server URL
7const zepApiURL = "http://localhost:8000";
8const zepApiKey = "<your JWT token>"; // optional
9const sessionId = uuid4(); // This is a unique identifier for the user
10
11// Set up Zep Memory
12const memory = new ZepMemory({
13  sessionId,
14  baseURL: zepApiURL,
15  apiKey: zepApiKey,
16});

Once you’ve created the memory, use it in your chain or with your agent.

Python

Running a chain with ZepMemory
1llm = ChatOpenAI(model_name="gpt-3.5-turbo")
2chain = ConversationChain(llm=llm, verbose=True, memory=memory)
3chain.run(
4    input="What is the book's relevance to the challenges facing contemporary society?"
5)
Output:
> Entering new AgentExecutor chain...
AI: Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.
> Finished chain.
'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.'

Inspecting the zep_chat_history, we see that a summary was automatically created, and that the history has been enriched with token counts, UUIDs, and timestamps.

Inspecting the Zep memory
1def print_messages(messages):
2    for m in messages:
3        print(m.type, ":\n", m.dict())
4
5
6print(memory.chat_memory.zep_summary)
7print("\n")
8print_messages(memory.chat_memory.messages)
9
10The human inquires about Octavia Butler. The AI identifies her as an American science fiction author. The human then asks which books of hers were made into movies. The AI responds by mentioning the FX series Kindred, based on her novel of the same name. The human then asks about her contemporaries, and the AI lists Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.
11
12
13system :
14 {'content': 'The human inquires about Octavia Butler. The AI identifies her as an American science fiction author. The human then asks which books of hers were made into movies. The AI responds by mentioning the FX series Kindred, based on her novel of the same name. The human then asks about her contemporaries, and the AI lists Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.', 'additional_kwargs': {}}
15human :
16 {'content': 'What awards did she win?', 'additional_kwargs': {'uuid': '6b733f0b-6778-49ae-b3ec-4e077c039f31', 'created_at': '2023-07-09T19:23:16.611232Z', 'token_count': 8, 'metadata': {'system': {'entities': [], 'intent': 'The subject is inquiring about the awards that someone, whose identity is not specified, has won.'}}}, 'example': False}
17ai :
18 {'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'additional_kwargs': {'uuid': '2f6d80c6-3c08-4fd4-8d4e-7bbee341ac90', 'created_at': '2023-07-09T19:23:16.618947Z', 'token_count': 21, 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 14, 'Start': 0, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}, {'Label': 'WORK_OF_ART', 'Matches': [{'End': 33, 'Start': 19, 'Text': 'the Hugo Award'}], 'Name': 'the Hugo Award'}, {'Label': 'EVENT', 'Matches': [{'End': 81, 'Start': 57, 'Text': 'the MacArthur Fellowship'}], 'Name': 'the MacArthur Fellowship'}], 'intent': 'The subject is stating that Octavia Butler received the Hugo Award, the Nebula Award, and the MacArthur Fellowship.'}}}, 'example': False}
19human :
20 {'content': 'Which other women sci-fi writers might I want to read?', 'additional_kwargs': {'uuid': 'ccdcc901-ea39-4981-862f-6fe22ab9289b', 'created_at': '2023-07-09T19:23:16.62678Z', 'token_count': 14, 'metadata': {'system': {'entities': [], 'intent': 'The subject is seeking recommendations for additional women science fiction writers to explore.'}}}, 'example': False}
21<snip />

TypeScript

Running a chain with ZepMemory
1const chain = new ConversationChain({ llm: model, memory });
2const response = await chain.run({
3  input = "What is the book's relevance to the challenges facing contemporary society?",
4});
Output:
> Entering new AgentExecutor chain...
AI: Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.
> Finished chain.
'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.'

Inspecting the ZepMemory, we see that a summary was automatically created, and that the history has been enriched with token counts, UUIDs, and timestamps.

Inspecting the Zep Memory
1const memoryVariables = await zepMemory.loadMemoryVariables({});
2console.log(memoryVariables);
Output:
The AI provides a synopsis of Parable of the Sower, a science fiction novel by Octavia Butler, about a young woman navigating a dystopian future. The human asks for recommendations for other women sci-fi writers and the AI suggests Ursula K. Le Guin and Joanna Russ. The AI also notes that Butler was a highly acclaimed writer, having won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.
{'uuid': 'becfe5f4-d24c-4487-b572-2e836d7cedc8', 'created_at': '2023-05-10T23:28:10.380343Z', 'role': 'human', 'content': "WWhat is the book's relevance to the challenges facing contemporary society?", 'token_count': 16}
{'uuid': '8ea4875c-bf42-4092-b3b8-308394963cb7', 'created_at': '2023-05-10T23:28:10.396994Z', 'role': 'ai', 'content': 'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.', 'token_count': 0}
...
{'uuid': '2d95ff94-b52d-49bd-ade4-5e1e553e8cac', 'created_at': '2023-05-10T23:28:02.704311Z', 'role': 'ai', 'content': 'Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', 'token_count': 31}

Search Zep’s message history from your Langchain app

Zep supports both vector search using a Langchain Retriever as well as via an instance of ZepMemory.

Using a ZepMemory instance

If you don’t need to provide a Retriever to your chain or agent, you can search the long-term message history for a session directly from an instance of ZepMemory.

Python

Search for relevant historical messages
1search_results = memory.chat_memory.search("who are some famous women sci-fi authors?")
2for r in search_results:
3    if r.dist > 0.8:  # Only print results with similarity of 0.8 or higher
4        print(r.message, r.dist)
Output:
{'uuid': 'ccdcc901-ea39-4981-862f-6fe22ab9289b', 'created_at': '2023-07-09T19:23:16.62678Z', 'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'metadata': {'system': {'entities': [], 'intent': 'The subject is seeking recommendations for additional women science fiction writers to explore.'}}, 'token_count': 14} 0.9119619869747062
{'uuid': '7977099a-0c62-4c98-bfff-465bbab6c9c3', 'created_at': '2023-07-09T19:23:16.631721Z', 'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 40, 'Start': 23, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 55, 'Start': 44, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}], 'intent': 'The subject is suggesting that the person should consider reading the works of Ursula K. Le Guin or Joanna Russ.'}}, 'token_count': 18} 0.8534346954749745
{'uuid': 'b05e2eb5-c103-4973-9458-928726f08655', 'created_at': '2023-07-09T19:23:16.603098Z', 'role': 'ai', 'content': "Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.", 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 16, 'Start': 0, 'Text': "Octavia Butler's"}], 'Name': "Octavia Butler's"}, {'Label': 'ORG', 'Matches': [{'End': 58, 'Start': 41, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 76, 'Start':