Working with Chat History

Learn how to persist, retrieve, and search chat history using Zep’s Memory API

You are viewing the Zep Open Source v0.x documentation. This version is no longer supported, and documentation is provided as a reference only.

The current documentation for Zep Community Edition is available here.

Zep’s Memory API persists your app’s chat history and metadata to a Session, enriches the memory, and enables vector similarity search over historical chat messages.

There are two approaches to populating your prompt with chat history:

  1. Retrieve the most recent N messages (and potentially a summary) from a Session and use them to construct your prompt.
  2. Search over the Session’s chat history for messages that are relevant and use them to construct your prompt.

Both of these approaches may be useful, with the first providing the LLM with context as to the most recent interactions with a human. The second approach enables you to look back further in the chat history and retrieve messages that are relevant to the current conversation in a token-efficient manner.

Initializing the Zep Client

Please see the SDK documentation for more information on initializing the Zep client.

Note: zep-python supports async use. All methods are available as both sync and async, with the async methods prefixed with a. For example, zep-python has both zep_client.memory.add_memory and zep_client.memory.aadd_memory methods.

Next Steps

Zep’s Memory Enrichment

You’ve likely noticed that alongside the role and content you provided to Zep when persisting a memory, Zep also returns a unique identifier, a summary, UUID, a timestamp, token count, extracted entities, and more. The token count is a useful tool to use when constructing prompts, while the extracted entities and other metadata may be useful for building more sophisticated applications.

Zep performs auto-summarization when a session exceeds the message window. This is returned in the summary field of the memory when you call get_memory and may be used when constructing prompts in order to provide your agent or chain with a longer-term memory of the conversation. Read more about the Summarizer Extractor.

Zep also automatically extracts Entities from, and runs an Intent Analysis on, messages in the conversation. The extracted entities and intents are stored in system metadata and available for hybrid searches (see Hybrid Search above).

1{
2  "summary": {
3    "uuid": "afe3957b-032f-47e0-8317-ed2953a2fb49",
4    "created_at": "2023-05-16T22:59:22.979937Z",
5    "content": "The AI provides a summary of Octavia Butler's Parable of the Sower, detailing the story of Lauren Olamina in a dystopian future. When the human asks for recommendations for other women sci-fi writers, the AI suggests Ursula K. Le Guin and Joanna Russ. The human follows up by asking about Butler's awards, and the AI lists the Hugo Award, Nebula Award, and MacArthur Fellowship. They also discuss Butler's contemporaries, the FX adaptation of Kindred, and Butler's background as an American science fiction author.",
6    "recent_message_uuid": "8834423d-9388-4a6a-bee2-091754407241",
7    "token_count": 644
8  },
9  "messages": [
10    {
11      "uuid": "96c2597e-7c13-48ee-bb5e-1f4f4663b2d2",
12      "created_at": "2023-05-16T22:59:33.612956Z",
13      "role": "ai",
14      "content": "Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.",
15      "system": {
16        "intent": "The subject is providing information about a science fiction novel called \"Parable of the Sower\" by Octavia Butler, including a brief summary of its plot and setting.",
17        "entities": [
18          {
19            "Label": "PERSON",
20            "Matches": [
21              {
22                "End": 46,
23                "Start": 32,
24                "Text": "Octavia Butler"
25              }
26            ],
27            "Name": "Octavia Butler"
28          },
29          {
30            "Label": "PERSON",
31            "Matches": [
32              {
33                "End": 46,
34                "Start": 32,
35                "Text": "Lauren Olamina"
36              }
37            ],
38            "Name": "Lauren Olamina"
39          },
40          {
41            "Label": "WORK_OF_ART",
42            "Matches": [
43              {
44                "End": 0,
45                "Start": 20,
46                "Text": "Parable of the Sower"
47              }
48            ],
49            "Name": "Parable of the Sower"
50          }
51        ]
52      },
53      "token_count": 253
54    },
55    {
56      "uuid": "b235e682-75b3-44b2-b083-4572cdbc86b1",
57      "created_at": "2023-05-16T22:59:33.612956Z",
58      "role": "human",
59      "content": "Write a short synopsis of Butler's book, Parable of the Sower. What is it about?",
60      "metadata": {
61        "system": {
62          "intent": "The subject is requesting a brief explanation or summary of Octavia Butler's book, \"Parable of the Sower.\"",
63          "entities": [
64            {
65              "Label": "WORKS_OF_ART",
66              "Matches": [
67                {
68                  "End": 46,
69                  "Start": 32,
70                  "Text": "Parable of the Sower"
71                }
72              ],
73              "Name": "Parable of the Sower"
74            }
75          ]
76        }
77      },
78      "token_count": 21
79    }
80  ]
81}